Control method and control device of automatic cleaning equipment and related equipment

Abstract

The embodiment of the invention discloses a control method and device of automatic cleaning equipment and related equipment, the automatic cleaning equipment comprises an equipment body and a mechanical arm arranged on the equipment body, the equipment body is provided with a first sensor, and the mechanical arm is provided with a second sensor. The automatic cleaning equipment is controlled to move in the working environment, and visual information is collected through the first sensor and / or the second sensor in the moving process; in response to visual information collected by the first sensor and / or the second sensor, a low area is recognized, the low area cannot accommodate the equipment body, and the mechanical arm is controlled to stretch into the low area to collect the visual information; and the visual information collected by the first sensor and the second sensor in the mapping mode is fused to establish a map of the working environment, so that the navigation capability and the obstacle avoidance capability of the automatic cleaning equipment are improved.

Description

Technical Field

[0001] This application relates to the field of automatic cleaning equipment technology, and in particular to a control method, control device, and related equipment for automatic cleaning equipment. Background Technology

[0002] In existing technologies, automated cleaning devices (such as robotic vacuum cleaners) typically rely on lidar, visual sensors, or RGB-D cameras fixedly mounted on the top of the device for environmental perception and map building. Because the sensor's viewing angle is fixed and cannot be flexibly adjusted, blind spots exist in its detection of low-lying spaces under furniture such as beds and sofas. This prevents the effective acquisition of visual information in these areas, resulting in an incomplete environmental map and limiting the navigation and obstacle avoidance capabilities of the automated cleaning device. Summary of the Invention

[0003] This application provides a control method, control device, and related equipment for automatic cleaning equipment, used to acquire visual information of low-ceilinged spaces to construct a map of the working environment, thereby improving the navigation and obstacle avoidance capabilities of the automatic cleaning equipment.

[0004] In a first aspect, this application provides a control method for an automatic cleaning device, the automatic cleaning device including a device body and a robotic arm disposed on the device body, the device body being provided with a first sensor, and the robotic arm being provided with a second sensor, the method comprising: In mapping mode, the automatic cleaning equipment is controlled to move in the working environment, and visual information is collected through the first sensor and / or the second sensor during the movement. In response to the identification of a low-lying area by visual information acquired through the first sensor and / or the second sensor, where the low-lying area cannot accommodate the device body, the robotic arm is controlled to extend into the low-lying area to acquire visual information. The visual information collected by the first sensor and the second sensor in the mapping mode is fused to create a map of the working environment.

[0005] In some embodiments, the method further includes: When initially entering mapping mode and / or in an open area, the robotic arm is controlled to extend above the device body and within a first height range, the field of view of the second sensor is oriented horizontally or downward at a preset angle to the horizontal.

[0006] In some embodiments, the robotic arm is located within the projected outline of the device body.

[0007] In some embodiments, the method further includes: Upon initial entry into mapping mode, the automatic cleaning device is controlled to rotate, and during this rotation, visual information collected by the first and second sensors is acquired; and / or, In an open area, the automatic cleaning equipment is controlled to rotate, and visual information collected by the first and second sensors is acquired during the rotation.

[0008] In some embodiments, the acquisition of visual information via the first sensor and / or the second sensor during movement includes: The automatic cleaning equipment is controlled to move within the working environment according to a mapping path, the mapping path including at least one scanning node located in an open area; In response to the automatic cleaning device arriving at the scanning node, the automatic cleaning device is controlled to rotate at the scanning node, and during the rotation, visual information collected by the first sensor and / or the second sensor is acquired.

[0009] In some embodiments, identifying low-lying areas using visual information acquired by the first sensor and / or the second sensor includes: Obstacle identification is performed using visual information collected by the first sensor and / or the second sensor. If an obstacle is identified that is off the ground and the distance between the obstacle and the ground is less than a preset height, then a low-lying area is identified.

[0010] In some embodiments, controlling the robotic arm to extend into the low-lying area to acquire visual information includes: Control the automatic cleaning equipment to move to the entrance position of the low-ceiling area; The robotic arm is controlled to enter a low-position scanning posture to acquire visual information of the low-lying area, and during the acquisition process, the robotic arm is controlled to swing horizontally and / or the device body moves along the boundary of the low-lying area.

[0011] In some embodiments, the robotic arm includes a support rod and a movable joint, the second sensor is disposed on the robotic arm, and controlling the robotic arm to enter a low-position scanning posture includes: The support rod is controlled to extend above the device body and within a second height range, and the movable joint is at a preset angle downward and horizontally so that the robotic arm extends under the obstacle, and the second sensor faces the low-lying area.

[0012] In some embodiments, the method further includes: Control the automatic cleaning equipment to move to a position below the high-altitude area; The robotic arm is controlled to enter a high-level scanning posture to collect visual information in the high-altitude area, and during the collection process, the robotic arm is controlled to swing and / or the device body moves along the boundary of the high-altitude area.

[0013] In some embodiments, the robotic arm includes a support rod and a movable joint, the second sensor is disposed on the robotic arm, and controlling the robotic arm to enter a high-position scanning posture includes: The support rod is controlled to extend above the device body and within a third height range, and the movable joint is at a preset angle to the horizontal direction so that the robotic arm swings back and forth within the preset angle range, and the second sensor faces the high-altitude area.

[0014] In some embodiments, fusing the visual information acquired by the first sensor and the second sensor in the mapping mode to establish a map of the working environment includes: First point cloud data is obtained based on visual information collected by the first sensor in mapping mode, and second point cloud data is obtained based on visual information collected by the second sensor in mapping mode. Align the overlapping areas of the first point cloud data and the second point cloud data in the same coordinate system; The aligned first and second point cloud data are merged to obtain merged point cloud data; A map of the working environment is created based on the fused point cloud data.

[0015] Secondly, this application provides a control device for an automatic cleaning device, the automatic cleaning device including a device body and a robotic arm disposed on the device body, the device body being provided with a first sensor, and the robotic arm being provided with a second sensor, the device comprising: The acquisition module is configured to acquire visual information via the first sensor and / or the second sensor as the automatic cleaning equipment moves in the working environment; The control module is configured to, in mapping mode, control the robotic arm to extend out of the device body and control the automatic cleaning device to move in the working environment, and in response to visual information collected by the first sensor and / or the second sensor to identify a low area that cannot accommodate the device body, control the robotic arm to extend into the low area to collect visual information. The mapping module is configured to fuse visual information collected by the first sensor and the second sensor in the mapping mode to create a map of the working environment.

[0016] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0017] Fourthly, this application provides an automatic cleaning device, which includes a device body and a robotic arm disposed on the device body. The device body is provided with a first sensor, and the robotic arm is provided with a second sensor. The automatic cleaning device further includes: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first aspects.

[0018] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0019] According to the specific embodiments provided in this application, the following technical effects are disclosed: The solution provided in this application achieves omnidirectional, blind-spot-free perception of the environment through the collaboration of dual vision sensors in the device body and the robotic arm, along with the active control of the robotic arm. Specifically, the first sensor of the device body is responsible for scanning and mapping the surrounding environment, while the second sensor at the end of the robotic arm can actively extend into low-lying areas (such as under furniture) that traditional devices cannot reach, acquiring visual information about these areas. Furthermore, by fusing visual information from different perspectives and positions of the first and second sensors, a complete, accurate environmental map containing details of low-lying areas can be generated. This effectively solves the problem of missing map information due to fixed sensor perspectives, providing a reliable environmental data foundation for subsequent cleaning path planning, navigation, and obstacle avoidance by the automatic cleaning device, thereby improving its navigation and obstacle avoidance capabilities.

[0020] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1A schematic diagram of an automatic cleaning device provided in an embodiment of this application.

[0023] Figure 2 A flowchart illustrating the steps of a control method for an automatic cleaning device provided in an embodiment of this application.

[0024] Figure 3 This is a schematic diagram illustrating the scanning of an open area in the control method of the automatic cleaning equipment provided in this application embodiment.

[0025] Figure 4 This is a schematic diagram illustrating the low-profile area scanning process in the control method of the automatic cleaning equipment provided in this application embodiment.

[0026] Figure 5 This is a schematic diagram illustrating the high-altitude area scanning process in the control method of the automatic cleaning equipment provided in this application embodiment.

[0027] Figure 6 A flowchart illustrating an example of a control method for an automatic cleaning device provided in this application.

[0028] Figure 7 A schematic block diagram of the control device for an automatic cleaning equipment provided in an embodiment of this application.

[0029] Figure 8 A schematic block diagram of an automatic cleaning device provided in an embodiment of this application.

[0030] Figure label: 100. Equipment body; 110. Robotic arm; 120. First sensor; 130. Second sensor; 140. Gripper; 150. Ground; 160. Ground obstacle; 170. High-altitude obstacle. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0032] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0033] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0034] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0035] In existing technologies, automated cleaning devices (such as robotic vacuum cleaners) typically rely on lidar, visual sensors, or RGB-D cameras fixedly mounted on the top of the device for environmental perception and map building. Because the sensor's viewing angle is fixed and cannot be flexibly adjusted, blind spots exist in its detection of low-lying spaces under furniture such as beds and sofas. This prevents the effective acquisition of visual information in these areas, resulting in an incomplete environmental map and limiting the navigation and obstacle avoidance capabilities of the automated cleaning device.

[0036] To address the aforementioned issues, this application provides a control method, control device, and related equipment for automatic cleaning equipment, used to acquire visual information from low-ceilinged spaces to construct a map of the working environment, thereby improving the navigation and obstacle avoidance capabilities of the automatic cleaning equipment.

[0037] The present application will be further described below with reference to the accompanying drawings.

[0038] refer to Figure 1 , Figure 1 This is a schematic diagram of an automatic cleaning device provided in an embodiment of this application. In some embodiments, the automatic cleaning device may include a circular device body 100, on which a robotic arm 110 may be provided, and at the end of the robotic arm 110 may be a gripper 140 for gripping and cleaning objects. A first sensor 120 is provided on the device body 100, and a second sensor 130 is provided on the robotic arm 110.

[0039] In some embodiments, the device body 100 is a mobile robot platform, and its bottom may be equipped with drive wheels to achieve autonomous movement. The robotic arm 110 may be a multi-joint robotic arm 110, with its base fixed to the top of the device body 100. The gripper 140 is installed at the end effector interface of the robotic arm 110 for performing cleaning operations such as grasping and wiping. The first sensor 120 and the second sensor 130 may be one of a lidar, a vision sensor, or an ultrasonic sensor, respectively. The first sensor 120 may be disposed on the device body 100 for global navigation and mapping, and the second sensor 130 may be disposed at the end of the robotic arm 110 or near the gripper 140 for local operation and obstacle detection.

[0040] refer to Figure 2 , Figure 2 This application provides a flowchart of the steps of a control method for an automatic cleaning device according to an embodiment of the present application; in some embodiments, the present application provides a control method for an automatic cleaning device, applied to, for example... Figure 1 The automatic cleaning device shown includes a device body 100 and a robotic arm 110 disposed on the device body 100. The device body 100 is provided with a first sensor 120, and the robotic arm 110 is provided with a second sensor 130. The method may include at least the following steps: Step 201: In mapping mode, control the automatic cleaning equipment to move in the working environment, and collect visual information through the first sensor 120 and / or the second sensor 130 during the movement. Step 202: In response to the visual information collected by the first sensor 120 and / or the second sensor 130, a low area is identified, and the low area cannot accommodate the device body 100. The robotic arm 110 is controlled to extend into the low area to collect visual information. Step 203: The visual information collected by the first sensor 120 and the second sensor 130 in the mapping mode is fused to create a map of the working environment.

[0041] Based on steps 201 to 203 above, the solution provided in this application embodiment achieves omnidirectional, blind-spot-free perception of the environmental space through the collaboration of dual vision sensors of the device body 100 and the robotic arm 110, as well as the active control of the robotic arm 110. Specifically, the first sensor 120 of the device body 100 is responsible for scanning and mapping the main environment, while the robotic arm 110 and its end-effector, the second sensor 130, can actively extend into low-lying areas (such as the bottom of furniture) that traditional devices cannot reach, acquiring visual information of these low-lying areas. Furthermore, by fusing visual information from different perspectives and positions of the first sensor 120 and the second sensor 130, a complete, accurate environmental map containing details of low-lying areas can be generated, effectively solving the problem of missing map information caused by fixed sensor perspectives. This provides a reliable environmental data foundation for the subsequent cleaning path planning, navigation, and obstacle avoidance of the automatic cleaning equipment, thereby improving the navigation and obstacle avoidance capabilities of the automatic cleaning equipment.

[0042] The robotic arm 110 may include a retractable mechanical structure with multi-degree-of-freedom motion capabilities, which can be implemented using a combination of electric actuators and rotary joints, and its spatial pose adjustment can be achieved by driving a servo motor or stepper motor. The first sensor 120 may include a main environmental sensing device mounted on the top of the device body 100, which can be implemented using LiDAR or a depth camera, to acquire three-dimensional information about the environment surrounding the device body 100. The second sensor 130 may include an auxiliary detection device mounted on the end effector of the robotic arm 110, which can be implemented using a wide-angle camera or an infrared sensor, and its installation position changes with the movement of the robotic arm 110. The mapping mode may include the device performing environmental scanning and map building operations, which can be triggered by a preset program or user commands. In this mode, the device performs systematic spatial exploration. Visual information fusion may include coordinate alignment and data stitching of spatial data acquired by different sensors, which can be achieved using SLAM algorithms combined with feature point matching technology to ensure accurate integration of data collected from different perspectives in a unified coordinate system.

[0043] Understandably, during the initial mapping phase, the device deploys its robotic arm 110, positioning the second sensor 130 in a horizontal or downward-tilted detection posture. When the device detects an obstacle off the ground via the first sensor 120 during movement, it calculates the distance between the bottom of the obstacle and the ground. If this distance is less than the height of the device body 100, it is identified as a low-lying area. At this point, the robotic arm 110 is adjusted to a low-level scanning posture, allowing the second sensor 130 to extend into the area below the obstacle for multi-angle scanning. Through the combined effect of the horizontal swing of the robotic arm 110 and the movement of the device body 100 along the boundary, the second sensor 130 can completely acquire the three-dimensional structural data of the low-lying area. Subsequently, the main environmental data collected by the first sensor 120 and the detailed data acquired by the second sensor 130 are fused using a feature matching algorithm to generate an environmental map containing a complete spatial topology.

[0044] It is worth noting that traditional solutions are limited by the blind spots of fixed sensors and cannot acquire data from low-lying areas. This solution uses a robotic arm 110 to dynamically adjust the spatial position of the sensors, enabling the second sensor 130 to penetrate areas that traditional solutions cannot detect. Combined with dual-sensor data fusion technology, this effectively expands the environmental perception range and accurately acquires the dimensions of the space under obstacles and the distribution of obstacles inside.

[0045] Through the above technical solution, this application effectively solves the technical problem of map loss caused by blind spots in low-ceilinged areas, enabling automatic cleaning equipment to construct a complete environmental map that includes details of the space under furniture. This technical solution improves the navigation accuracy of the equipment in complex home environments, provides reliable spatial data support for subsequent path planning and obstacle avoidance operations, and avoids cleaning omissions caused by map loss.

[0046] refer to Figure 3 , Figure 3 The schematic diagram of scanning an open area in the control method of the automatic cleaning equipment provided in the embodiments of this application shows that the equipment body 100 is placed on the ground 150. In some embodiments, this application further proposes a control method for an automatic cleaning equipment, which further includes, upon initially entering the mapping mode and / or in an open area, controlling the robotic arm 110 to extend above the equipment body 100 and within a first height range, the field of view of the second sensor 130 is oriented towards the horizontal direction or downward at a preset angle to the horizontal direction; the robotic arm 110 is located within the projected outline range of the equipment body 100.

[0047] The first height range can include the height range of the robotic arm 110 when it is extended above the device body 100. The vertical displacement of the robotic arm 110 can be adjusted using a telescopic rod or hydraulic device to prevent collisions with surrounding obstacles. The preset angle can include the tilt of the field of view direction of the second sensor 130 relative to the horizontal plane. The pitch angle of the robotic arm 110, on which the second sensor 130 is mounted, can be adjusted by rotating a joint to expand the scanning range of the ground 150 or low-lying areas. The projection contour range can include the area projected onto the horizontal plane when the robotic arm 110 is extended, which does not exceed the outer contour boundary of the device body 100. The position of the robotic arm 110 can be monitored in real time using limit switches or position sensors to ensure stability during device movement.

[0048] Understandably, when the automated cleaning equipment enters mapping mode or performs tasks in an open area, the robotic arm 110 is controlled to extend upwards to a first height range, while the field of view of the second sensor 130 is adjusted to be horizontal or tilted downwards at a preset angle. For example, in an open area, the robotic arm 110 can extend to a height of 50 centimeters above the ground, and the robotic arm 110 equipped with the second sensor 130 tilts downwards at 30 degrees to cover the ground area in front. During this process, the extension range of the robotic arm 110 always remains within the projected outline of the equipment body 100, avoiding center of gravity shift or interference with obstacles due to exceeding the boundary of the equipment body 100.

[0049] It is worth noting that in existing technologies, sensors are fixedly installed on the top of the equipment, making it impossible to flexibly adjust their height and angle according to the environment, resulting in blind spots in low-lying areas. This solution, through the adjustable height of the robotic arm 110 and the sensor pitch angle, can actively adjust the scanning range in the early stages of mapping or in open areas. Combined with the constraints of the projected contour range, it not only expands the environmental perception capability but also ensures the stability of equipment movement.

[0050] Through the above technical solution, this application can dynamically adjust the height of the robotic arm 110 and the sensor viewing angle during the mapping process, effectively covering environmental information in low-lying and horizontal areas, and improving the integrity of map construction. At the same time, the extension range of the robotic arm 110 is limited within the projected outline of the equipment, avoiding equipment imbalance or collision with obstacles due to excessive extension of the robotic arm 110, and enhancing the adaptability of the automatic cleaning equipment in complex environments.

[0051] In some embodiments, this application further proposes controlling the automatic cleaning device to rotate when initially entering the mapping mode, and acquiring visual information collected by the first sensor 120 and / or the second sensor 130 during the rotation; and / or controlling the automatic cleaning device to rotate in an open area, and acquiring visual information collected by the first sensor 120 and / or the second sensor 130 during the rotation.

[0052] The mapping mode can include the automatic cleaning equipment's environmental map construction process, which can be initiated through a preset program to allow the equipment to enter an autonomous movement and data acquisition process. Rotation can include the device body 100 adjusting its angle around a vertical axis, which can be achieved using differential speed control of the drive wheels, changing the speed difference between the left and right wheels to generate rotational motion. Open areas can include unobstructed, flat open spaces in the working environment, determined by real-time detection of obstacle distribution using sensors; when the obstacle density is below a preset threshold, it is considered an open area. Acquiring visual information during rotation can include the device simultaneously triggering data acquisition from the first sensor 120 and the second sensor 130 during rotation. A time synchronization module can correlate the sensor acquisition cycle with changes in the rotation angle to ensure complete recording of visual information from different perspectives.

[0053] For example, such as Figure 3 As shown, the robotic arm 110 can be raised to a mid-position (e.g., 15cm to 35cm above the ground, which is the first height range, adjustable based on obstacles detected by the second sensor 130). Simultaneously, the robotic arm 110 must not extend beyond the robot's body to avoid interfering with the operation of the main sensor. In open areas, it first performs a 360° horizontal scan in coordination with the main sensor and the robotic arm 110's camera, maintaining this posture while scanning to acquire mapping information and identify obstacles in the lower half of the room area.

[0054] Understandably, when the automatic cleaning device first activates its mapping mode, it performs a stationary rotation at its initial position, such as a 360-degree clockwise or counterclockwise rotation. During this process, the first sensor 120 on the top of the device body 100 and the second sensor 130 at the end of the robotic arm 110 continuously collect visual information about the surrounding environment, such as scanning the outlines of horizontal obstacles using LiDAR or capturing the spatial structure in the vertical direction using a depth camera. When the automatic cleaning device rotates in place at the same location, it can acquire environmental visual information within a nearby area at once, avoiding repeated scanning and data redundancy caused by frequent movement of the device, thus effectively improving mapping efficiency. When the device moves to an open area, such as the center of a living room or a corridor, it performs the rotation again. At this time, the first sensor 120 acquires a large-scale environmental data at a fixed height, while the second sensor 130 supplements the visual information of low-lying areas through the posture adjustment of the robotic arm 110. The rotation angle range can be dynamically adjusted according to the complexity of the environment; for example, a 180-degree rotation can be performed in a confined space to reduce the risk of collision, while multiple 360-degree rotations can be performed in an open area to increase data coverage density.

[0055] It's worth noting that traditional automated cleaning equipment relies solely on sensors with a fixed viewing angle for single scans during mapping, resulting in a lack of visual information in low-lying areas and complex structures. For example, when the equipment is stationary, the sensors cannot capture details of obstacles to the side and rear, while the equipment may miss some areas due to viewing angle limitations when moving in a straight line. This solution actively controls the equipment's rotation, allowing the sensors to acquire multi-angle and multi-directional visual information as they dynamically adjust their viewing angle. For instance, during rotation, the bottom outline of obstacles behind the equipment can be covered, or multiple rotations can eliminate scanning blind spots caused by the sensor's limited field of view. Furthermore, performing rotation in open areas fully utilizes the operational freedom in an unobstructed environment, and multiple rotations improve the data acquisition accuracy of key areas.

[0056] Through the above technical solution, this application can effectively improve the ability of automated cleaning equipment to capture complex environmental features during the mapping process. By coordinating the control of rotational motion and sensor data acquisition, the equipment can acquire more comprehensive environmental information. For example, it can quickly establish a large-scale spatial outline during the initial rotation phase and supplement details such as ground flatness and obstacle distribution during the rotation phase in open areas. This dynamic data acquisition method can reduce the problem of insufficient detection in low-lying areas caused by fixed sensor perspectives, thereby providing a more complete environmental model foundation for subsequent map fusion and path planning.

[0057] In some embodiments, this application further proposes a control method for an automatic cleaning device, wherein in mapping mode the automatic cleaning device is controlled to move along a mapping path within the working environment, the mapping path including at least one scanning node located in an open area; in response to the automatic cleaning device reaching the scanning node, the automatic cleaning device is controlled to rotate at the scanning node, and during the rotation, visual information collected by the first sensor 120 and / or the second sensor 130 is acquired.

[0058] The mapping path can include a pre-planned or dynamically generated movement trajectory covering the working environment. This can be implemented using a grid-based path planning algorithm or a random coverage algorithm to ensure the automated cleaning equipment can traverse the area to be mapped. Scanning nodes can include pre-defined locations along the path for multi-angle visual acquisition. These can be center points of open areas determined through environmental feature recognition or coordinate positioning, providing the sensor with an unobstructed field of view. The rotation process can include the device body 100 rotating around its own axis. This can be achieved using a stepper motor or servo motor to drive the wheel set at differential speed, expanding the sensor's horizontal scanning range through rotation.

[0059] Understandably, in mapping mode, the automated cleaning device moves to each scanning node according to a preset mapping path. When the device reaches a scanning node, it triggers a rotation action, such as a 360-degree rotation or a segmented rotation, while simultaneously acquiring visual information from the perspectives of the device body 100 and the robotic arm 110 through the first sensor 120 and the second sensor 130, respectively. For example, the scanning node can be set in the center of a room or at the intersection of a corridor, covering environmental features such as surrounding walls and furniture edges through rotation. During this process, the robotic arm 110 can remain extended, allowing the second sensor 130 to supplement data collection from different heights and angles, such as adjusting the pitch angle of the robotic arm 110 during rotation to capture details of the ceiling or floor.

[0060] Through the above technical solutions, this application can improve the integrity and detail accuracy of environmental maps, especially in acquiring panoramic data in open areas by rotating scanning nodes, avoiding feature omissions caused by a single movement path. Simultaneously, the coordinated movement of the robotic arm 110 and the device body 100 expands the sensor coverage from a single plane to three-dimensional space, providing more comprehensive basic data for subsequent map fusion.

[0061] refer to Figure 4 , Figure 4 This is a schematic diagram of scanning low-lying areas in the control method of the automatic cleaning equipment provided in the embodiments of this application; in some embodiments, this application further proposes to identify obstacles by collecting visual information from the first sensor 120 and / or the second sensor 130; if an obstacle 160 above the ground is identified, and the distance between the obstacle 160 above the ground and the ground 150 is less than a preset height, then it is determined that a low-lying area has been identified.

[0062] The visual information may include environmental images or 3D point cloud data acquired through optical sensors, such as RGB cameras, depth cameras, or LiDAR, used to extract obstacle contours and spatial location information. The ground-level obstacle 160 may include objects suspended above the ground with accessible spaces at their base, identified through image segmentation algorithms combining point cloud height analysis; examples include furniture such as beds, sofas, or cabinets. The preset height may include a critical space height threshold that the device body 100 cannot access, set according to the physical dimensions of the device body 100, for example, 10 centimeters, used to determine whether low-lying areas require detection by the robotic arm 110.

[0063] It is worth noting that this application can enhance semantic understanding potential by combining RGB information acquired from RGB cameras and other related devices, providing a better foundation for subsequent object recognition and scene understanding (such as recognizing tables, chairs, and windows). This multimodal data fusion enables automated cleaning devices not only to perceive space but also to understand environmental semantics, supporting more intelligent decision-making.

[0064] For example, after completing the initial scanning of all room areas, the automatic cleaning device in this application can move to areas inaccessible to the main body, adjusting the posture of the robotic arm 110 to such a position. Figure 4 As shown, the robotic arm 110 is lowered to a low position (for example, as low as 1 cm above the ground, which corresponds to the second height range; the robotic arm 110 camera can extend into low-lying areas with a ground height of at least 4.5 cm). The robotic arm 110 camera is extended into the low-lying area to scan, while the robotic arm 110 base rotates and the body moves, supplementing the mapping information of the low-lying blind spots and marking objects.

[0065] Understandably, during the movement of the automated cleaning equipment, the first sensor 120 and the second sensor 130 continuously collect environmental visual information. Image processing algorithms are used to detect obstacles from this visual information, identifying the boundaries and bottom spaces of suspended objects. When the vertical distance between the bottom of an object and the ground is less than a preset height, the area is determined to be a low-lying area. For example, when the distance between the bed bottom and the ground is detected to be 8 centimeters and the height of the equipment body 100 is 12 centimeters, the robotic arm 110 is triggered to enter the area for supplementary detection.

[0066] It is worth noting that existing technologies rely on fixed-viewpoint sensors, which cannot identify the height information of the space 160° below the obstacle on the ground. This leads to low-lying areas being incorrectly marked as impassable or completely ignored. This solution, by dynamically analyzing the obstacle's height above the ground and combining it with preset thresholds, can accurately identify the existence and detectability of low-lying areas.

[0067] Through the above technical solution, this application solves the problem of blind spots in low-altitude space detection caused by the fixed sensor perspective in the prior art. By using height threshold determination and collaborative detection with the robotic arm 110, it ensures that visual information in low-altitude areas is completely collected, thereby improving the integrity of the environmental map and the accuracy of navigation path planning.

[0068] In some embodiments, this application further proposes a control method for an automatic cleaning device, which controls a robotic arm 110 to extend into a low-lying area to collect visual information. This includes controlling the automatic cleaning device to move to the entrance position of the low-lying area, controlling the robotic arm 110 to enter a low-level scanning posture to collect visual information from the low-lying area, and controlling the robotic arm 110 to swing horizontally and / or the device body 100 to move along the boundary of the low-lying area during the collection process.

[0069] The entrance location to the low-lying area can include a starting point below an obstacle 160 that the robotic arm 110 can enter. This can be determined by identifying the coordinates of the bottom edge of the obstacle using a first sensor 120 or a second sensor 130. This location must meet the spatial condition that the device body 100 cannot enter, but the end effector of the robotic arm 110 can reach. The low-level scanning posture can include the robotic arm 110's end effector entering the low-lying area at a height lower than the device body 100. This can be achieved by adjusting the pitch angle of the multi-joint robotic arm 110, for example, by tilting the robotic arm 110, which is equipped with the second sensor 130, at a 10°-30° angle to the ground to adapt to spaces of different heights. Horizontal swing can include the end effector of the robotic arm 110 reciprocating in a horizontal plane. A servo motor can be used to drive the joints of the robotic arm 110 to achieve a swing amplitude of 5°-15°, covering the lateral space of the low-lying area through the motion trajectory. Moving along the boundary can include the device body 100 maintaining a constant distance from the edge of the low-lying area, for example, by measuring the distance to the obstacle in real time with a lidar and controlling the speed of the drive wheels to achieve interval movements of 10 to 20 centimeters.

[0070] Understandably, once the sensor detects an obstacle 160 off the ground, the path planning module calculates the geometric center point of the obstacle's bottom edge as the entry point. After the drive system controls the device body 100 to move to this coordinate point, the actuator of the robotic arm 110 drives the support rod to extend from the top of the device, while the movable joint adjusts the pitch angle so that the robotic arm 110, equipped with the second sensor 130, extends under the obstacle. For example, during the data acquisition phase, the end of the robotic arm 110 swings horizontally at a frequency of 2-3 times per second to expand the scanning coverage, while the device body 100 moves along the obstacle boundary at a speed of 0.1-0.3 meters per second, forming a spiral scanning path through this combined motion. During this process, the second sensor 130 acquires 3D point cloud data of the low-lying area at a collection frequency of 30-60 frames per second.

[0071] It is worth noting that traditional fixed sensors, due to limitations in installation location, cannot acquire spatial information below obstacles 160 on the ground. However, this method, through the attitude adjustment and motion coordination of the robotic arm 110, enables the robotic arm 110, equipped with the second sensor 130, to penetrate deep into low-lying areas. In existing technologies, devices detour around such areas, resulting in map gaps. This solution, however, uses the dynamic scanning of the robotic arm 110 to completely acquire structural data of hidden spaces such as under beds and sofas.

[0072] Through the above technical solution, this application achieves effective detection of low-lying and concealed areas, eliminating the environmental perception blind spots of traditional cleaning equipment in areas such as under beds and cabinets. The coordinated action of the horizontal swing of the robotic arm 110 and the movement of the equipment body 100 ensures scanning coverage of low-lying areas, providing a reliable data foundation for constructing a complete environmental map and improving the navigation accuracy and cleaning coverage of automatic cleaning equipment in complex home environments.

[0073] In some embodiments, it is understood that by controlling the robotic arm 110 to extend into a low-lying area to collect visual information, including controlling the automatic cleaning device to move to the entrance position of the low-lying area; controlling the robotic arm 110 to enter a low-level scanning posture, and controlling the robotic arm 110 to swing horizontally and / or the device body 100 to move along the boundary of the low-lying area during the acquisition process, this application can eliminate the blind spots in the field of view of the automatic cleaning device when performing environmental perception and map building in the prior art. The active movement of the robotic arm 110 can effectively reduce the occlusion problem inherent in fixed-viewpoint sensors, and can detect areas that traditional sweeping robots cannot enter or cannot be identified by sensors, such as under furniture, behind furniture, and narrow gaps, so as to ensure that there are no blind spots in environmental perception and thus improve the reliability of the constructed environmental map.

[0074] In some embodiments, this application further proposes a control method for an automatic cleaning device. The robotic arm 110 includes a support rod and a movable joint. A second sensor 130 is disposed on the robotic arm 110. Controlling the robotic arm 110 to enter a low-position scanning posture includes: controlling the support rod to extend above the device body 100 and within a second height range, and controlling the movable joint to be at a preset angle downward and horizontally, so that the end of the robotic arm 110 with the second sensor 130 installed extends below the ground obstacle 160, and the second sensor 130 faces the low-lying area.

[0075] The support rod may include a rigid rod-like structure connecting the device body 100 and the movable joint, which can be implemented as an electrically telescopic rod, used to adjust the vertical height of the robotic arm 110 relative to the device body 100. The movable joint may include a connecting component with angle adjustment function, which can be implemented as a servo-driven rotary joint, used to change the pitch angle of the sensor end, which is the end of the robotic arm 110 on which the second sensor 130 is installed. The sensor end may include a movable component on which the second sensor 130 is installed, which can be implemented as a camera assembly with a gimbal structure, used to adjust the detection direction in low-profile areas. The second sensor 130 may include a sensing device for acquiring visual information, which can be implemented as a depth camera or a lidar module, used to acquire three-dimensional spatial information in low-profile areas.

[0076] In some embodiments, the robotic arm 110 may adopt a multi-segment telescopic and multi-joint linkage structure, which may include a first support rod, a first movable joint, a second support rod, a second movable joint, and a third support rod connected in sequence. One end of the first support rod is fixedly connected to the top base of the device body 100, and the other end is hinged to one end of the second support rod through the first movable joint. The other end of the second support rod is hinged to one end of the third support rod through the second movable joint. The other end of the third support rod serves as the sensor end, used to mount the sensor. Adjacent support rod segments are rotatably connected through corresponding movable joints. That is, the first movable joint is used to drive the second support rod to pitch or rotate horizontally relative to the first support rod, and the second movable joint is used to drive the third support rod to finely adjust its angle relative to the second support rod. Through the coordinated action of the two movable joints, the robotic arm 110 can flexibly extend and precisely adjust its angle in three-dimensional space, thereby driving the sensor end to smoothly reach the detection position in a low area.

[0077] In some embodiments, the installation position of the second sensor 130 can be selected according to the detection scenario. In addition to being installed at the sensor end at the other end of the third support rod, it can also be fixed in the middle region of the robotic arm 110, such as the middle section of the second support rod, the connection between the first movable joint and the second support rod, or the front section of the third support rod. When the second sensor 130 is installed in the middle of the robotic arm 110, the extension and retraction of the multiple support rods and the angle adjustment of the movable joint can be used to ensure that the sensor detection direction accurately covers the target area: for low-lying area detection, the first movable joint drives the second support rod to tilt downwards, and the second movable joint fine-tunes the posture of the third support rod. Even if the sensor is located in the middle, it can be extended into the area below the ground obstacle 160 through the overall extension and bending of the robotic arm 110; for high-altitude area detection, the support rod extends to the third height range, the movable joint tilts upwards, and the sensor installed in the middle can cover the bottom and sides of the high-altitude obstacle through the swing of the robotic arm 110.

[0078] Understandably, when the automated cleaning equipment detects a low-lying area, the support rod is controlled to extend above the equipment body 100, with its extension length adjustable to a second height range, for example, between 10 and 20 centimeters. Subsequently, the movable joint tilts downward to form a preset angle with the horizontal direction, for example, between 30 and 45 degrees, allowing the sensor tip to extend below the obstacle 160. During this process, the second sensor 130 continuously collects visual information within the low-lying area, while the robotic arm 110 can swing horizontally or the equipment body 100 can move along the boundary of the low-lying area to expand the detection coverage. Through the height adjustment of the support rod and the angle adjustment of the movable joint, the sensor tip can flexibly adapt to low spaces of different heights, avoiding collisions with obstacles.

[0079] It is worth noting that the sensors of existing automated cleaning equipment are fixed to the top of the equipment body 100, and their height and pitch angle cannot be adjusted, resulting in blind spots in low-lying areas. This solution, however, utilizes the multi-degree-of-freedom adjustment capability of the robotic arm 110, allowing the sensor tip to penetrate deep into low-lying areas and acquire visual information about those areas, thereby effectively improving the completeness of map construction.

[0080] Through the above technical solution, this application solves the problem of missing visual information in low-lying areas in the prior art, enabling automatic cleaning equipment to accurately identify the structural features of narrow spaces such as under beds and sofas, avoiding navigation path planning errors caused by incomplete maps, and reducing the risk of physical interference between the robotic arm 110 and obstacles during the detection process.

[0081] refer to Figure 5 , Figure 5 The schematic diagram of the control method for the automatic cleaning equipment provided in the embodiments of this application is shown, in which a high-altitude area is scanned, and a high-altitude obstacle 170 is set in the high-altitude area; in some embodiments, this application further proposes to control the automatic cleaning equipment to move to a position below the high-altitude area; control the robotic arm 110 to enter a high-level scanning posture to collect visual information of the high-altitude area, and control the robotic arm 110 to swing and / or the equipment body 100 to move along the boundary of the high-altitude area during the collection process.

[0082] The high-altitude area can include a three-dimensional space located above the device body 100 and beyond the detection range of the fixed sensors on the device body 100, such as the top of a cabinet or below a hanging cabinet, which can be determined by visual recognition or a preset height threshold. The high-altitude scanning posture can include the robotic arm 110 adjusting to a specific height and angle to cover the high-altitude area, which can be achieved by the extension and retraction of the support rod and the rotation of the movable joint. The swinging can include the robotic arm 110 periodically reciprocating in a horizontal or vertical plane, for example, by driving the movable joint to rotate within a preset angle range through a servo motor, thereby expanding the scanning coverage of the second sensor 130.

[0083] For example, in this application, after the automated cleaning device completes the enhanced scanning of the low-lying area, the posture of the robotic arm 110 can be adjusted to such a position. Figure 5 As shown, the robotic arm 110 is raised to a high-angle viewing position (e.g., the height above the ground is adjustable from 30cm to 50cm, which is the third height range, and can be adjusted according to the obstacles above identified by the robotic arm 110's camera). At the same time, as the main body moves, the wrist joint of the robotic arm 110 swings back and forth in real time (e.g., from 0° to -45°) to scan suspended obstacles such as ceilings, upper walls, chandeliers, electric fans, table and chair beams, etc., to supplement vertical (height) mapping information and mark the suspended obstacles that the robotic arm 110 needs to avoid during operation.

[0084] Understandably, when the automated cleaning equipment detects a high-altitude area, the equipment body 100 moves directly below that area, and the support rod of the robotic arm 110 extends upward to a third height range. The movable joint tilts upward at a preset angle to the horizontal, aligning the second sensor 130, carried at the sensor's end, with the sensor aligned with the high-altitude area. During data acquisition, the robotic arm 110 changes the detection direction of the second sensor 130 through swinging motions, or the equipment body 100 moves along the boundary of the high-altitude area, thereby acquiring visual information from multiple angles and positions. For example, when the high-altitude area is a suspended shelf, the robotic arm 110 can capture information about the distribution of obstacles at the bottom and sides of the shelf during its swinging motion.

[0085] It is worth noting that traditional automated cleaning equipment, limited by the height and viewing angle of its fixedly installed sensors, cannot effectively detect high-altitude areas above the equipment body 100, resulting in blind spots in map construction. This solution, through the posture adjustment and coordinated movement of the robotic arm 110, enables the second sensor 130 to flexibly cover high-altitude areas, filling the gaps in the sensor's detection range.

[0086] Through the above technical solution, this application realizes the acquisition of three-dimensional environmental information in high-altitude areas, solving the problem of missing detection in high-altitude areas due to the fixed sensor perspective in the prior art. Through the high-position scanning posture of the robotic arm 110 and the coordinated movement of the equipment body 100, the obstacle distribution, boundary contours and other features of the high-altitude area are completely recorded, providing data support for subsequent map construction and path planning, and improving the navigation accuracy of the automatic cleaning equipment in complex three-dimensional environments.

[0087] In some embodiments, this application controls the automatic cleaning device to move to a position below a high-altitude area. The robotic arm 110 can then be controlled to enter a high-level scanning posture to collect visual information about the high-altitude area. During the collection process, the robotic arm 110 can be controlled to swing and / or the device body 100 can be controlled to move along the boundary of the high-altitude area. For example, the robotic arm 110 can be raised to a high-level upward-looking position to scan suspended obstacles such as ceilings and upper walls. Subsequently, this application can also identify low-lying areas using visual information collected by the first sensor 120 and / or the second sensor 130, and control the robotic arm 110 to extend into these low-lying areas to collect visual information. For example, the robotic arm 110 can be lowered to a low-level scanning area under a bed. It is worth noting that through the coordinated high-level and low-level scanning in the above process, this application can achieve three-dimensional coverage of the automatic cleaning device's working environment, solving the problem of traditional robotic vacuum cleaner maps lacking height information and low-lying area information. It can effectively construct a complete three-dimensional model including ceilings, chandeliers, bookshelves, sofa surfaces, and under beds at various heights, ensuring the three-dimensional integrity of the environmental map and providing a precise spatial basis for subsequent navigation.

[0088] In some embodiments, when the robotic arm 110 is in a high-position scanning posture, the motion trajectory of its sensor end is located within a three-dimensional space with the center of the device body 100 as the projection reference. This space covers the spatial area from the top of the device body 100 to the bottom of the high-altitude obstacle 170 in the vertical direction and forms a limited scanning coverage area in the horizontal direction.

[0089] In some embodiments, during the high-altitude area scanning process, the space below the high-altitude area is divided into multiple area nodes with different scanning characteristics; the robotic arm 110 is controlled to adopt corresponding scanning strategies at different area nodes: a horizontal scanning method is used at nodes near the edge of the high-altitude area, a composite stereo scanning mode is used at nodes below the center of the high-altitude area, and an adaptive supplementary scanning method is used at nodes in transition areas; as the device body 100 moves along the boundary of the high-altitude area, the coordinated movement of the robotic arm 110 is used to systematically collect visual information from each area node, and the movement mode of the robotic arm 110 is coordinated with the movement trajectory of the device body 100.

[0090] In some embodiments, this application further proposes a robotic arm 110 including a support rod and a movable joint, a second sensor 130 disposed on the robotic arm 110, and controlling the robotic arm 110 to enter a high-level scanning posture, including controlling the support rod to extend above the device body 100 and within a third height range, the movable joint to be at a preset angle upward and horizontal, so that the sensor end can swing back and forth within a preset angle range, and the second sensor 130 facing the high-altitude area.

[0091] The support rod may include a rigid structure for connecting the device body 100 and the movable joint, and can be implemented using an electric telescopic rod or a hydraulic rod. The height of the sensor tip can be changed by adjusting the telescopic length of the support rod. The movable joint may include a connecting component with rotational freedom, and can be implemented using a hinge structure driven by a servo motor. The orientation of the sensor tip can be changed by adjusting the angle of the movable joint. The third height range may include the height range of the support rod after it extends out of the device body 100, and can be controlled by preset stroke parameters or sensor feedback to ensure that the sensor tip can cover high-altitude areas. The preset angle range may include the allowable swing amplitude when the movable joint tilts upward, and can be achieved by setting the motor rotation angle range, so that the sensor tip can reciprocate and scan within a specified area.

[0092] Understandably, when it is necessary to collect visual information from high-altitude areas, this application can first control the support rod to extend from above the device body 100 to a third height range, for example, by raising the support rod to a predetermined height using an electric push rod. Then, the angle of the movable joint is adjusted so that it tilts upwards to form a preset angle with the horizontal direction. At this time, the sensor end is positioned near the high-altitude area, and the field of view of the second sensor 130 faces this area. During the acquisition process, the movable joint is further controlled to swing back and forth within a preset angle range, for example, swinging periodically within a range of ±15 degrees at a speed of 5 degrees per second, thereby expanding the scanning coverage of the second sensor 130.

[0093] Through the above technical solution, this application can achieve comprehensive collection of visual information in high-altitude areas, fill the blind spots in high-altitude detection caused by the fixed installation of sensors in the prior art, thereby improving the integrity of environmental maps and the navigation accuracy of automatic cleaning equipment.

[0094] In some embodiments, this application further proposes a control method for an automatic cleaning device. In mapping mode, visual information collected by a first sensor 120 and a second sensor 130 is fused to establish a map of the working environment. The method includes: acquiring first point cloud data based on visual information collected by the first sensor 120 in mapping mode, and acquiring second point cloud data based on visual information collected by the second sensor 130 in mapping mode; aligning the overlapping areas of the first point cloud data and the second point cloud data in the same coordinate system; fusing the aligned first point cloud data and the second point cloud data to obtain fused point cloud data; and establishing a map of the working environment based on the fused point cloud data.

[0095] The first point cloud data can include a set of three-dimensional spatial coordinates converted from visual information collected by sensors on the device body 100. This can be generated after scanning the working environment using LiDAR or a depth camera, and is used to describe the obstacle distribution in the movable area of ​​the device body 100. The second point cloud data can include a set of three-dimensional spatial coordinates converted from visual information collected by sensors at the end of the robotic arm 110. This can be generated after scanning low-lying or high-altitude areas using an adjustable-angle depth camera, and is used to supplement blind spots not covered by the sensors of the device body 100. Alignment within the same coordinate system can include mapping data collected by different sensors to a unified spatial reference system. This can be achieved using algorithms based on feature point matching or synchronous localization and mapping (SMR) techniques based on motion trajectories, ensuring spatial consistency of point cloud data from different sources. The fused point cloud data can include a synthetic dataset obtained by removing redundant information and superimposing complementary information from two aligned point cloud sets. This can be processed using voxel filtering or probabilistic fusion algorithms to form a three-dimensional model covering the entire working environment.

[0096] Understandably, during the movement of the automated cleaning equipment, sensors on the equipment body 100 continuously collect visual information in the horizontal direction and the traversable area of ​​the equipment body 100, generating the first point cloud data. When the robotic arm 110 extends into a low-lying area or adjusts to a high-lying posture, the sensors at its end collect visual information of the corresponding area, generating the second point cloud data. A coordinate transformation algorithm is used to convert the two sets of point cloud data to the same coordinate system, for example, establishing a coordinate transformation matrix based on the relative positional relationship between the equipment body 100 and the robotic arm 110, eliminating spatial deviations caused by differences in sensor installation positions. Subsequently, by comparing the geometric features of overlapping areas, such as planar contours or edge curvature, the two sets of point clouds are precisely aligned. The aligned point clouds are then fused using a fusion algorithm to remove duplicate parts, for example, averaging the two sets of points at the same location or retaining high-precision data, while preserving the unique point cloud information of both low-lying and high-lying areas. Finally, an environmental map containing complete obstacle distribution and spatial structure is generated based on the fused point cloud data.

[0097] It is worth noting that fixed sensors in existing technologies cannot acquire visual information about low-lying or high-altitude areas, resulting in blind spots in the map. This solution, however, utilizes a robotic arm 110 equipped with an adjustable-angle sensor. Combined with point cloud data alignment and fusion technology, the detection ranges of the sensors on the main unit 100 and the robotic arm 110 are complementary. This allows data from low-lying, high-altitude, and conventional areas to be integrated into a single map, eliminating blind spots caused by fixed sensor perspectives and improving the completeness of the environmental map through multi-source data fusion. For example, in low-lying area detection, point cloud data collected by the end effector of the robotic arm 110 fills in areas not covered by the main unit 100 sensors; in high-altitude area detection, point cloud data collected by the robotic arm 110 after adjusting its posture supplements the three-dimensional spatial information missed by conventional horizontal scanning. The resulting map accurately reflects the obstacle distribution at different heights in the working environment, providing more comprehensive environmental model support for the path planning and obstacle avoidance of automated cleaning equipment.

[0098] refer to Figure 6 , Figure 6This is an example flowchart of a control method for an automatic cleaning device provided in an embodiment of this application. In some embodiments, the method may specifically include: firstly, starting a three-dimensional mapping mode, controlling the robotic arm 110 to extend out of the device body 100 and controlling the device to move in the working environment, while simultaneously acquiring visual information through a first visual sensor and a second visual sensor; then, executing the host sensor scanning process and the robotic arm 110 segmented scanning process in parallel, wherein the robotic arm 110 segmented scanning includes: a low-level scanning stage, controlling the robotic arm 110 to lower and extend into a low area to supplement blind spot visual information; a mid-level scanning stage, controlling the robotic arm 110 to rise to a mid-level height to acquire obstacle information in the lower half of the area; a high-level scanning stage, controlling the robotic arm 110 to rise and swing upwards to acquire vertical height information; furthermore, using multi-sensor data fusion technology, aligning and optimizing the acquired visual information and point cloud data in the same coordinate system to construct an environmental map containing three-dimensional semantic information.

[0099] It is understood that this application acquires first point cloud data based on visual information collected by the first sensor 120 and second point cloud data based on visual information collected by the second sensor 130; the point cloud data are aligned and fused in the same coordinate system to establish a map of the working environment. The above-mentioned process of observing the same area from multiple perspectives to make the data from the multi-angle scanning of the robotic arm 110 complementary can improve the map details and accuracy, thereby obtaining denser point clouds and more accurate surface geometry, making the modeling of complex objects (such as chair legs and plants) more refined, and making the map more consistent with the actual environment.

[0100] In some embodiments, the method of this application may include sequentially executing the following stages: first, performing a mid-range scan, during which the robotic arm 110 is controlled to extend above the device body 100 and within a first height range, and the field of view of the second sensor 130 is oriented horizontally or downward at a preset angle to the horizontal direction, so as to quickly scan an open area and establish a basic framework of the working environment; then, performing a low-profile area scan, in response to the visual information collected by the first sensor 120 and / or the second sensor 130 identifying a low-profile area that cannot accommodate the device body 100, the robotic arm 110 is controlled to extend into the low-profile area to collect visual information to supplement the low-profile area. Details of blind spots in low-lying areas; then, a high-level scan is performed, controlling the automatic cleaning equipment to move to a position below the high-altitude area, controlling the support rod to extend above the equipment body 100 and within the third height range, the movable joint to be at a preset angle upward and horizontal, so that the sensor end swings back and forth within the preset angle range, the second sensor 130 faces the high-altitude area, and visual information of the high-altitude area is obtained; furthermore, this application can use multi-sensor data fusion technology to align and optimize the visual information and point cloud data collected in the mid-level, low-lying and high-level scanning stages in the same coordinate system, and construct an environmental map containing three-dimensional semantic information.

[0101] Understandably, by performing the above-mentioned sequential scanning scheme—intermediate scanning, low scanning, and high scanning—intermediate scanning can prioritize covering a large area, establishing the overall framework of the environment first. Then, low and high scanning can be used to supplement the details of the corresponding areas, thereby avoiding repetition or omissions during the scanning process, improving mapping efficiency, and ensuring the comprehensiveness and accuracy of the map. This provides more efficient and reliable environmental model support for the path planning and obstacle avoidance of automatic cleaning equipment.

[0102] refer to Figure 7 , Figure 7 This is a schematic block diagram of a control device for an automatic cleaning equipment provided in an embodiment of this application. In some embodiments, this application provides a control device 700 for an automatic cleaning equipment. The automatic cleaning equipment includes a device body and a robotic arm disposed on the device body. The device body is provided with a first sensor, and the robotic arm is provided with a second sensor. The device includes: The acquisition module 701 is configured to acquire visual information through a first sensor and a second sensor during the process of the robotic arm extending out of the device body and the automatic cleaning device moving in the working environment. The control module 702 is configured to, in mapping mode, control the robotic arm to extend out of the equipment body and control the automatic cleaning equipment to move in the working environment, and in response to visual information collected by the first sensor and / or the second sensor to identify a low area where the equipment body cannot be accommodated, control the robotic arm to extend into the low area to collect visual information. The mapping module 703 is configured to fuse visual information acquired by the first and second sensors in mapping mode to create a map of the working environment.

[0103] This application also provides an automatic cleaning device, including: One or more processors; and A memory associated with one or more processors, the memory being used to store program instructions that, when read and executed by one or more processors, perform the steps of the method of any of the first aspects.

[0104] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the methods in the first aspect.

[0105] in, Figure 8 The architecture of the automatic cleaning device provided in the embodiments of this application is illustrated by way of example, wherein, Figure 8 The automatic cleaning equipment 800 can be an automatic cleaning device, and the automatic cleaning equipment 800 may include a control device 700 installed on the automatic cleaning equipment.

[0106] Furthermore, such as Figure 8 As shown, automatic cleaning devices may include a processor 810, a video display adapter 811, a disk drive 812, an input / output interface 813, a network interface 814, and a memory 820. The processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820 can communicate with each other via a communication bus 830.

[0107] The processor 810 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0108] The memory 820 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 820 can store the operating system 821 for controlling the terminal's operation, and the basic input / output system (BIOS) 822 for controlling the terminal's low-level operations. Additionally, it can store a web browser 823, a data storage management system 824, and a control device 700 installed on the automatic cleaning equipment, etc. The aforementioned control device can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 820 and executed by the processor 810.

[0109] The input / output interface 813 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0110] Network interface 814 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0111] Bus 830 includes a pathway for transmitting information between various components of the device, such as processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820.

[0112] It should be noted that although the above-described device only shows the processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, memory 820, bus 830, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0113] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0114] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A control method for an automatic cleaning device, characterized in that, The automatic cleaning device includes a device body and a robotic arm disposed on the device body. The device body is equipped with a first sensor, and the robotic arm is equipped with a second sensor. The method includes: In mapping mode, the automatic cleaning equipment is controlled to move in the working environment, and visual information is collected through the first sensor and / or the second sensor during the movement. In response to the identification of a low-lying area by visual information acquired through the first sensor and / or the second sensor, where the low-lying area cannot accommodate the device body, the robotic arm is controlled to extend into the low-lying area to acquire visual information. The visual information collected by the first sensor and the second sensor in the mapping mode is fused to create a map of the working environment.

2. The method according to claim 1, characterized in that, The method further includes: When initially entering mapping mode and / or in an open area, the robotic arm is controlled to extend above the device body and within a first height range, the field of view of the second sensor is oriented horizontally or downward at a preset angle to the horizontal.

3. The method according to claim 2, characterized in that, The robotic arm is located within the projected outline of the device body.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Upon initial entry into mapping mode, the automatic cleaning device is controlled to rotate, and during this rotation, visual information collected by the first and second sensors is acquired; and / or, In an open area, the automatic cleaning equipment is controlled to rotate, and visual information collected by the first and second sensors is acquired during the rotation.

5. The method according to any one of claims 1 to 3, characterized in that, The acquisition of visual information through the first sensor and / or the second sensor during movement includes: The automatic cleaning equipment is controlled to move within the working environment according to a mapping path, the mapping path including at least one scanning node located in an open area; In response to the automatic cleaning device arriving at the scanning node, the automatic cleaning device is controlled to rotate at the scanning node, and during the rotation, visual information collected by the first sensor and / or the second sensor is acquired.

6. The method according to claim 1, characterized in that, The identification of low-lying areas using visual information acquired by the first sensor and / or the second sensor includes: Obstacle identification is performed using visual information collected by the first sensor and / or the second sensor. If an obstacle is identified that is off the ground and the distance between the obstacle and the ground is less than a preset height, then a low-lying area is identified.

7. The method according to claim 1 or 6, characterized in that, The control of the robotic arm to extend into the low-lying area to collect visual information includes: Control the automatic cleaning equipment to move to the entrance position of the low-ceiling area; The robotic arm is controlled to enter a low-position scanning posture to acquire visual information of the low-lying area, and during the acquisition process, the robotic arm is controlled to swing horizontally and / or the device body moves along the boundary of the low-lying area.

8. The method according to claim 7, characterized in that, The robotic arm includes a support rod and a movable joint. The second sensor is disposed on the robotic arm. Controlling the robotic arm to enter a low-position scanning posture includes: The support rod is controlled to extend above the device body and within a second height range, and the movable joint is at a preset angle downward and horizontally so that the robotic arm extends under the obstacle, and the second sensor faces the low-lying area.

9. The method according to claim 1, characterized in that, The method further includes: Control the automatic cleaning equipment to move to a position below the high-altitude area; The robotic arm is controlled to enter a high-level scanning posture to collect visual information in the high-altitude area, and during the collection process, the robotic arm is controlled to swing and / or the device body moves along the boundary of the high-altitude area.

10. The method according to claim 9, characterized in that, The robotic arm includes a support rod and a movable joint. The second sensor is disposed on the robotic arm. Controlling the robotic arm to enter a high-position scanning posture includes: The support rod is controlled to extend above the device body and within a third height range, and the movable joint is at a preset angle to the horizontal direction so that the robotic arm swings back and forth within the preset angle range, and the second sensor is directed toward the high-altitude area.

11. The method according to claim 1, characterized in that, The process of fusing visual information collected by the first sensor and the second sensor in the mapping mode to create a map of the working environment includes: First point cloud data is obtained based on visual information collected by the first sensor in mapping mode, and second point cloud data is obtained based on visual information collected by the second sensor in mapping mode. Align the overlapping areas of the first point cloud data and the second point cloud data in the same coordinate system; The aligned first and second point cloud data are merged to obtain merged point cloud data; A map of the working environment is created based on the fused point cloud data.

12. A control device for an automatic cleaning equipment, characterized in that, The automatic cleaning device includes a device body and a robotic arm disposed on the device body. The device body is equipped with a first sensor, and the robotic arm is equipped with a second sensor. The device includes: The acquisition module is configured to acquire visual information via the first sensor and / or the second sensor as the automatic cleaning equipment moves in the working environment; The control module is configured to, in mapping mode, control the robotic arm to extend out of the device body and control the automatic cleaning device to move in the working environment, and in response to visual information collected by the first sensor and / or the second sensor to identify a low area that cannot accommodate the device body, control the robotic arm to extend into the low area to collect visual information. The mapping module is configured to fuse visual information collected by the first sensor and the second sensor in the mapping mode to create a map of the working environment.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 11.

14. An automatic cleaning device, characterized in that, The automatic cleaning device includes a device body and a robotic arm disposed on the device body. The device body is equipped with a first sensor, and the robotic arm is equipped with a second sensor. The automatic cleaning device also includes: One or more processors; and A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1 to 11.

Images