Household cleaning and article homing method based on humanoid robot, storage medium

Abstract

The application discloses a kind of based on humanoid robot's household cleaning and article homing method, storage medium, belong to robot field.Method includes: S1, based on household space point cloud data constructs household space three-dimensional model;S2, according to shape factor and bounding box volume distinguishes fixed furniture and movable article;S3, the point cloud of fixed furniture is carried out cluster analysis, obtains function area division;S4, based on preset article standard storage location database, identifies scattered article in movable article;S5, the image of function area is collected, based on the image of function area calculates gray value deviation, local height difference and edge gradient, based on gray value deviation, local height difference and edge gradient identifies pollution type;S6, according to pollution type matching cleaning mode and cleaning parameter, article type identification is carried out to scattered article, and calls preset homing database and preset homing rule.Improves cleaning effect, improves pollution identification precision, improves article homing accuracy.

Description

Technical Field

[0001] This invention relates to the field of robotics, particularly to the field of robot application design, and especially to a method and storage medium for household cleaning and item placement based on a humanoid robot. Background Technology

[0002] Existing household cleaning technologies, represented by robotic vacuum cleaners and handheld vacuum cleaners, scan the ground using infrared or simple visual sensors and use fixed vacuuming or sweeping modes to handle dust and debris. They can only achieve cleaning in a single dimension of the ground, with fixed cleaning tools and parameters, and no ability to handle stains.

[0003] Existing item recognition and mobility technologies include machine vision-based item classification systems (such as item recognition modules in supermarket self-checkout), which identify item types by extracting image features, but these only reach the recognition level and cannot be combined with automatic relocation functions in home scenarios.

[0004] CN120458438A discloses a humanoid robot capable of sweeping floors and its control system. The robot includes a bipedal, multi-degree-of-freedom jointed humanoid body, a cleaning module integrating a vacuuming device, roller brush, mop, and automatic switching mechanism, a perception system including LiDAR, a navigation and path planning system based on SLAM algorithm, and a human-machine interface supporting voice interaction. Its control system includes an embedded processor, a task scheduling unit, a status monitoring unit, a fault handling module, and an energy management unit, which can coordinate the operation of each module. The advantages of this invention are: it solves the problems of limited cleaning capabilities in existing sweeping robots and the lack of cleaning functions in humanoid robots, achieving an organic combination of flexible movement and efficient cleaning, meeting the intelligent cleaning needs in complex indoor environments. However, this solution lacks adaptability to home environments and does not design specific grasping and placement logic for fragile or irregularly shaped items.

[0005] CN120770724A discloses a high-precision service robot control system for housekeeping cleaning based on multimodal perception. It includes a depth vision perception module, which uses a binocular camera to acquire 3D information of the environment and identify the position and shape of objects such as furniture and obstacles; a contact pressure perception module, distributed in the robot's cleaning components, used to sense the contact pressure and force state with the surface of the object being cleaned; and a core control unit, a dynamic adjustment module, and a remote communication module. This invention employs a depth vision perception module and a contact pressure perception module to more comprehensively perceive environmental and object information. While the depth vision perception module uses a binocular camera to acquire 3D information of the environment and identify the position and shape of objects such as furniture and obstacles, and the contact pressure perception module is distributed in the robot's cleaning components to sense the contact pressure and force state with the surface of the object being cleaned, this solution does not cover cleaning in multiple scenarios such as desktops and countertops, nor does it have an item placement function.

[0006] The existing technology has the following drawbacks: 1. Functional fragmentation: Cleaning equipment can only handle limited pollution on the ground (dust, garbage), and cannot cover stains on desktops, countertops, and other areas. Putting items back in place relies entirely on manual labor, and an integrated "cleaning + putting away" solution has not been formed, so the burden of manual housework has not been fundamentally reduced.

[0007] 2. Low recognition accuracy: Existing cleaning equipment uses traditional sensors and algorithms, which cannot quantify and identify the type of pollution (such as dust thickness and stain concentration). The cleaning methods and parameters are fixed, resulting in high dust residue rate and incomplete stain removal.

[0008] 3. Lack of logical placement: There is a lack of specific placement rules for home scenarios. The material of the items (fragile / common) and storage habits (categorization / proximity) are not taken into account. Furthermore, it cannot handle unfamiliar items, resulting in low accuracy in placement and easy damage or misplacement of items.

[0009] 4. Lack of quality closed-loop: There is no systematic effect detection mechanism after cleaning and putting things back in place, and it is impossible to automatically correct substandard operations, making it difficult to maintain cleanliness.

[0010] 5. Insufficient personalization: It cannot customize cleaning plans and placement rules according to the user's home layout and living habits, resulting in poor adaptability. Summary of the Invention

[0011] The technical problem to be solved by the present invention is: in order to solve at least one technical problem existing in the prior art, the present invention provides a method and storage medium for home cleaning and item placement based on a humanoid robot.

[0012] The technical solution adopted by this invention to solve its technical problem is: This invention provides a method for home cleaning and item return based on a humanoid robot, comprising: S1. The robot collects point cloud data of the home space and constructs a three-dimensional model of the home space based on the point cloud data. S2. Calculate the shape factor and bounding box volume of objects based on point cloud data of the home space, and distinguish between fixed furniture and movable items based on the shape factor and bounding box volume; S3. Perform cluster analysis on the point cloud of fixed furniture to obtain functional area division; S4. Based on a pre-defined database of standard storage locations for items, identify scattered items among movable items; S5. Acquire images of the functional area, calculate gray value deviation, local height difference and edge gradient based on the images of the functional area, and identify the type of contamination based on gray value deviation, local height difference and edge gradient; S6. Match cleaning methods and cleaning parameters according to the type of pollution, start the cleaning process, identify the type of scattered items, and call the preset placement database and preset placement rules to start the placement process.

[0013] By using point cloud modeling to achieve structured cognition of home space, the system first distinguishes between fixed furniture and movable items, then divides functional areas, and can accurately locate scattered items and contaminated areas; at the same time, it executes cleaning and return processes in parallel, improving the operating efficiency of home service robots and enhancing the level of automated cleanliness of the home environment.

[0014] Furthermore, in step S5, the contamination type is identified based on grayscale deviation, local height difference, and edge gradient, including: When 5% ≤ grayscale deviation < 10%, the pollution type is dust; When the grayscale value deviation is ≥10%, the local height difference is <preset height difference threshold, and the edge gradient is <preset edge gradient threshold, the contamination type is stain. When the grayscale value deviation is ≥10%, the local height difference is <preset height difference threshold, and the edge gradient is ≥preset edge gradient threshold, the pollution type is garbage; When the grayscale value deviation is ≥10% and the local height difference is ≥the preset height difference threshold, the pollution type is garbage.

[0015] By combining grayscale deviation, local height difference, and edge gradient, it is possible to accurately distinguish between dust, stains, low-height food residue, and three-dimensional debris, avoiding misidentification in complex scenarios and improving the robustness and accuracy of pollution identification.

[0016] Furthermore, in step S6, matching the cleaning method and cleaning parameters according to the type of contamination includes: When the type of contamination is dust, the cleaning intensity corresponding to the dust is calculated based on the dust thickness, particle hardness, and robot working wear. When the type of contamination is stain, the number of cleaning cycles corresponding to the stain is calculated based on the surface material tolerance, cleaning success rate, and stain stubbornness. The cleaning speed corresponding to the stain is calculated based on the number of cleaning cycles, humidity changes, friction changes, and color difference changes between the stained area and the background area.

[0017] We have established detailed parameter calculation models for dust and stains to match the cleaning intensity, number of cleaning sessions, and cleaning speed with the actual pollution level and floor material. This ensures effective cleaning while avoiding over-cleaning that could damage the furniture surface and extend the lifespan of your home.

[0018] Furthermore, the formula for calculating the cleaning power corresponding to the dust is as follows: ; ; in, The cleaning intensity corresponding to dust; This represents the upper limit of the surface material's strength. This is the force coefficient; Dust thickness; This is the particulate matter hardness weighting coefficient; Mohs hardness of particulate matter; This represents the robot loss coefficient. Accumulate the robot's working hours.

[0019] By incorporating dust thickness, particle hardness, and robot wear and tear into the cleaning intensity calculation and setting an upper limit protection for material intensity, adaptive, quantifiable, safe, and controllable dust cleaning is achieved, significantly improving cleaning effectiveness and equipment safety.

[0020] Furthermore, the formula for calculating the number of cleaning cycles corresponding to the stain is as follows: ; in, The number of cleaning cycles corresponding to the stain; Stain stubbornness; For surface material weighting coefficients; The surface material's tolerance coefficient; To improve the success rate of cleaning.

[0021] The cleaning frequency is adaptively determined based on the stubbornness of stains, material tolerance, and historical cleaning success rate, avoiding insufficient cleaning of stubborn stains or excessive wiping of sensitive materials, and improving adaptability to different home scenarios.

[0022] Furthermore, the formula for calculating the cleaning speed corresponding to the stain is as follows: ; ; in, The corresponding cleaning speed for the stain; As the reference speed; The number of cleaning cycles corresponding to the stain; To integrate changes in color difference, humidity, and friction; This refers to the color difference between the stained area and the background area. For changes in humidity in the stained area; This represents the change in friction in the stained area.

[0023] By integrating multi-dimensional information such as color difference, humidity, and friction, the cleaning speed is dynamically adjusted, and the wiping speed is optimized in real time according to the degree of stain removal, thereby improving the stain removal efficiency.

[0024] Furthermore, the preset placement rules include the rule of handling with care, the rule of classifying and placing, and the principle of placing in the nearest location; The rules for handling items gently include: calculating the gripping force of the item based on the robot's material stiffness, the item's center of gravity, and the item's fragility level; The principle of nearest relocation includes: calculating the path cost from the current location to the relocation location, and selecting the path corresponding to the minimum path cost.

[0025] By combining three rules—handling with care, categorizing and placing items, and returning them to their nearest location—the system ensures the safety of fragile items, shortens the return path, and improves operational efficiency, achieving efficient, safe, and orderly automatic placement of household items.

[0026] Furthermore, the formula for calculating the clamping force of the item is as follows: ; in, The clamping force of the item; For safety factor; For the quality of the goods; It is the acceleration due to gravity; This is the decay coefficient for the item's fragility level; The item is classified as fragile. The reference contact area; This represents the actual clamping contact area. The reference stiffness for robot materials; Real-time stiffness of robot materials; The distance is the baseline center of gravity. This is the actual distance between the object's center of gravity and the clamping point.

[0027] By coupling multiple factors such as item quality, fragility level, gripping area, robot stiffness, and center of gravity position into the gripping force model and setting an upper limit for the force, it can achieve flexible, stable, and non-destructive gripping of various items, and is particularly suitable for the safe gripping and return of fragile items in home settings.

[0028] Furthermore, in step S2, fixed furniture and movable items are distinguished based on shape factor and bounding box volume, including: If the bounding box volume of an item is greater than or equal to a preset volume threshold and the shape factor is greater than or equal to a preset shape factor threshold, it is determined to be fixed furniture; otherwise, it is determined to be movable item.

[0029] In another aspect, the present invention provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of a method for household cleaning and item placement.

[0030] The above-mentioned methods for household cleaning and item return, implemented through computer programs, can be directly deployed in humanoid robots, service robots, or smart home control systems. They are highly versatile, easy to port, and conducive to widespread promotion and industrial application.

[0031] The beneficial effects of this invention are: 1. Integrated processing improves efficiency: Through the full automation of "scanning and modeling - recognition and analysis - cleaning and putting away - effect detection", the time for cleaning and putting away at home is shortened, the burden of manual housework is reduced, and the cleanliness of the home environment is improved. This effect is achieved by the all-round perception capability of 3D vision sensor, the accurate modeling capability of SLAM algorithm, and the parallel execution logic of cleaning and putting away.

[0032] 2. Quantitative Achievement of Cleaning Effect: Based on a quantitative identification model of pollution types and personalized cleaning parameters, the system reduces dust residue, improves stain removal and garbage removal rates, and enhances cleaning effectiveness. Specifically, it employs a multi-dimensional pollution identification method combining grayscale deviation, local height difference, and edge gradient to accurately distinguish between dust, stains, food residue, and various debris. This effectively solves the identification challenges of easily misjudged scenarios such as colored garbage and low-height residue, significantly improving the accuracy and robustness of pollution type judgment.

[0033] 3. By using an item recognition model, a clamping force quantification formula (zero damage to fragile items), and a placement rule system, the accuracy of item placement is improved, eliminating misplacement and damage. Attached Figure Description

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] Figure 1 This is a flowchart of the home cleaning and item return method based on a humanoid robot according to the present invention. Detailed Implementation

[0036] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0037] Example 1: like Figure 1 As shown, this embodiment provides a method for home cleaning and item return based on a humanoid robot, specifically including: S1. The 3D vision sensor of the humanoid robot performs a 360° all-round scan of the home space and collects point cloud data of the home space (sampling frequency 10Hz, point cloud density 1000 points / cm²); the SLAM algorithm is used to construct a three-dimensional model of the home space based on the point cloud data of the home space.

[0038] S2. Calculate the shape factor and bounding box volume of objects based on point cloud data of the home space, and distinguish between fixed furniture and movable items based on the shape factor and bounding box volume.

[0039] When the bounding box volume is greater than or equal to a preset volume threshold and the shape factor is greater than or equal to a preset shape factor threshold, it is determined to be fixed furniture; otherwise, it is determined to be movable item. The preset volume threshold is preferably 1m × 0.5m × 0.5m, and the preset shape factor threshold is preferably 100. The shape factor of fixed furniture such as sofas, beds, and wardrobes is generally between 100 and 500, while the shape factor of movable items such as books, glasses, and tableware is generally between 1 and 50.

[0040] The shape factor is used to characterize the spatial form of three-dimensional objects in a home, reflecting the ratio of the object's surface area to its volume. The greater the expansiveness (the larger the ratio of surface area to volume, such as sofas and wardrobes), the larger the shape factor S value; the more compact the form (the smaller the ratio of surface area to volume, such as cups and books), the smaller the shape factor S value.

[0041] The formula for calculating the shape factor is as follows: ,in, For shape factor; The correction coefficient for the home scene was calibrated based on a large number of home object samples. , to fit the size units of household objects; The three-dimensional surface area of ​​the object is calculated using the triangulation method of the point cloud data; The three-dimensional volume of the object is calculated using the voxel filling method from the point cloud data.

[0042] Using only volume thresholds can easily misclassify large storage boxes and bulky toys as furniture, while using only shape factors can easily misclassify small, hollow, or extended items (small clothes hangers, small ornaments) as furniture. By combining shape factors with bounding box volume to distinguish between fixed furniture and movable items, this method can accurately, stably, and robustly classify object types in the home environment, avoiding misclassification problems caused by relying solely on size or shape. This improves the overall reliability of functional area division, contamination detection, and item placement. Furthermore, this method has low computational cost, high real-time performance, and does not require deep learning models or extensive sample training, making it more suitable for deployment in home robot terminals with limited computing power. It improves recognition accuracy while reducing system implementation costs and complexity.

[0043] S3. Perform cluster analysis on the point cloud of fixed furniture to obtain functional area divisions, including: S31. Spatial Clustering Based on DBSCAN: The density clustering algorithm (DBSCAN) is used to cluster the point cloud of fixed furniture (this algorithm is suitable for clustering irregular objects in three-dimensional space, does not require pre-setting the number of clusters, and adapts to the diverse shapes of home furniture). Spatial Euclidean distance is used as the clustering similarity index. The neighborhood radius (adapted to the size of home furniture, such as 0.5m) and the minimum number of point clouds are set (to filter noise points, such as small debris on the ground). Spatially adjacent fixed furniture point clouds are clustered into a furniture cluster.

[0044] S32. Initial division of functional areas: Based on prior knowledge of the home scene, the clustered furniture clusters are classified (e.g., sofa + coffee table is the living room furniture cluster, bed + wardrobe is the master bedroom furniture cluster, dining table + dining chair is the dining room furniture cluster). Based on the three-dimensional coordinates of each furniture cluster, the preliminary boundaries of each functional area are determined.

[0045] S33. Plane Equation Optimization and Boundary Calibration: Substitute the initially divided functional area boundaries into the plane equation ax+by+cz+d=0, where (x,y,z) are the three-dimensional coordinates of any point in space. Optimize the boundary parameters using the least squares method to calculate the precise three-dimensional boundary coordinates of the functional areas, ensuring that the boundary positioning error is ≤2cm. Finally, the division of functional areas such as the living room, bedroom, and dining room is completed.

[0046] S34. Area Verification: Verify the spatial continuity of the divided functional areas to eliminate errors in area division caused by sparse furniture and ensure the integrity of the functional areas.

[0047] S4. Based on a preset database of standard item storage locations (containing the standard location coordinates of each item), identify scattered items among movable items. Specifically, calculate the current position of the object ( , , ) and standard position ( , , The distance D is considered as scattered items if D > 5cm.

[0048] In addition, objects can also be considered scattered items when they are ≤3cm from the edge of the table. The specific method is as follows: First, the 3D contour of the desktop is extracted: In the 3D model of the divided functional areas, the desktop (dining table, coffee table, desk, etc.) is identified as a sub-component of fixed furniture. The edge contour point cloud of the desktop is extracted by the point cloud edge detection algorithm (such as the 3D extension of Canny edge detection) to obtain the 3D coordinate set of all points on the edge of the desktop.

[0049] Secondly, the key coordinates of the item are determined: the three-dimensional coordinates of the bottom contact point of the item to be judged are extracted (the lowest point of contact between the item and the table represents the actual placement of the item on the table, avoiding errors caused by calculation based on the center of gravity of the item).

[0050] Then, the shortest distance is calculated: Under a unified three-dimensional coordinate system, the Euclidean distance from the bottom contact point of the object to all points in the point cloud of the outline of the table edge is calculated, and the minimum value among them is taken as the distance of the object from the edge of the table.

[0051] Finally, distance determination: compare the calculated minimum value with the example threshold of 3cm. If it is less than or equal to 3cm, it is directly determined to be a scattered item.

[0052] S5. Acquire images of the functional area, calculate grayscale deviation, local height difference, and edge gradient based on the images of the functional area, and identify the type of contamination based on the grayscale deviation, local height difference, and edge gradient. ;in, The average gray value of the clean area; The values ​​represent standard, normal grayscale values. The calculations of local height differences and edge gradients are based on existing techniques and will not be elaborated upon here.

[0053] In some feasible implementations, contamination types are identified based on grayscale deviation, local height difference, and edge gradient, including: When the grayscale value deviation is less than 5%, there is no contamination; When 5% ≤ grayscale deviation < 10%, the pollution type is dust; When the grayscale value deviation is ≥10%, the local height difference is <preset height difference threshold, and the edge gradient is <preset edge gradient threshold, the contamination type is stain (blurred edge, diffusion, gradient, corresponding to coffee stains, oil stains, etc.). When the grayscale value deviation is ≥10%, the local height difference is <preset height difference threshold, and the edge gradient is ≥preset edge gradient threshold, the pollution type is garbage (corresponding to low-height garbage such as rice grains, bread crumbs, and vegetable leaves). When the grayscale value deviation is ≥10% and the local height difference is ≥the preset height difference threshold, the pollution type is garbage (corresponding to tall garbage such as food bags and packaging boxes).

[0054] Specifically, the preset height difference threshold is preferably 2mm, and the preset edge gradient threshold is preferably 40.

[0055] The above scheme adopts a coarse-to-fine, progressive feature combination approach: first, grayscale deviation is used for coarse classification (no pollution / dust / other); then, local height difference is used for further subdivision (low height / high height); finally, in the low height category, edge gradient is used to further distinguish stains and low-height waste. This approach can accurately distinguish dust, stains, low-height food residue, and three-dimensional debris, avoiding misidentification in complex scenarios and improving the robustness and accuracy of pollution identification.

[0056] In addition, a dedicated sample library can be built for common household waste (food scraps, packaging bags, fruit peels, paper scraps, etc.) to assist in waste identification and eliminate interference from other movable items. Simultaneously, combined with a self-learning mechanism for unfamiliar items, if a new type of household waste is identified, the sample library can be updated based on user feedback, achieving continuous iteration of waste identification capabilities. Specifically, the basic ResNet neural network is fine-tuned through transfer learning. Based on a general image recognition model, over 100,000 household waste samples (covering food scraps, packaging bags, etc., from different angles, under different lighting conditions, and in different arrangements) are added for training. The parameters of the network's convolutional and fully connected layers are optimized to improve the feature extraction capability of household waste. A scheme combining grayscale deviation, local height difference, edge gradient, and sample training ensures a waste identification confidence level of ≥90%.

[0057] S6. Match cleaning methods and cleaning parameters according to the type of pollution, start the cleaning process, identify the type of scattered items, and call the preset placement database and preset placement rules to start the placement process.

[0058] In some feasible implementations, cleaning methods and parameters are matched according to the type of contamination, including: When the type of contamination is dust, the cleaning method is vacuuming. The cleaning intensity is calculated based on the dust thickness, particle hardness, and robot working wear and tear, and different speeds are used according to the dust thickness.

[0059] Specifically, the formula for calculating the cleaning power corresponding to dust is as follows: ; ; ; in, The cleaning intensity corresponding to dust; This represents the upper limit of the surface material's strength. This is the force coefficient; Dust thickness; The particulate matter hardness weighting coefficient is preferably 0.3. The Mohs hardness of particulate matter is preferably 1 to 6, with 1 for dust, 4 for sand, and 6 for hard particles. The robot loss coefficient (preferably 0.0005); Accumulated working hours for the robot; The coefficient is a media correction factor. Different floor materials have different characteristics of dust adhesion and accumulation (preferably 0.7~1.3, 0.7 for wood flooring, 1.0 for ceramic tile, and 1.3 for carpet). The surface density of dust (which can be obtained by a laser dust sensor, preferably 0~0.5, which is the range of common household dust); The humidity attenuation coefficient is preferably 0.005. The relative humidity of the environment is the most important factor. The higher the humidity, the easier it is for dust particles to stick together or become damp, which may reduce the actual thickness of the cleanable loose layer (preferably 0~100%). It is the surface roughness coefficient (preferably 0.8~1.2, 0.8 for smooth surfaces and 1.2 for rough surfaces); The comprehensive conversion factor is (preferably 0.00005).

[0060] It should be noted that the higher the hardness of particulate matter, the more difficult it is to remove; therefore, it is more difficult to remove it through... Linearly increase the force, while simultaneously... The upper limit of cleaning intensity is a customized design for various floor materials (solid wood, tile, carpet) to ensure cleaning effectiveness while avoiding over-cleaning. The robot uses a fan to vacuum, and as the fan ages, it automatically compensates for the reduced cleaning intensity to ensure stable cleaning results. This is an optimized solution designed based on the robot's working characteristics.

[0061] Specifically, the cleaning speeds for dust are as follows:

[0062] When the type of contamination is a stain, the cleaning method is to use a microfiber cloth with detergent, and the cleaning force is fixed at 1N to avoid damaging the surface. The number of cleaning times for the stain is calculated based on the surface material tolerance, cleaning success rate and stain stubbornness. The cleaning speed for the stain is calculated based on the number of cleaning times, humidity changes, friction changes and color difference changes between the stained area and the background area.

[0063] Specifically, the formula for calculating the number of cleaning cycles required for a stain is as follows: ; in, The number of cleaning cycles (≥2) corresponding to the stain; Stain stubbornness is categorized as follows: (Light 5-15: water stains, dust stains; Medium 16-35: fingerprints, sauce stains, undried oil stains; Severe 36-60: dried oil stains, baking splatter stains). The surface material weighting coefficient is preferably 0.4. The surface material tolerance coefficient is preferably 0.5~1.2, 0.5 for solid wood and 1.2 for glass. The cleaning success rate is calculated based on the cleaning success rate of similar stains over the past 30 days (e.g., if the cleaning success rate of grease stains in a family's kitchen is only 80%). =0.2 (automatically increasing the number of uses by 20%), the robot will remember the characteristics of stains in the user's home, becoming more and more accurate with use.

[0064] It should be noted that, The square root function is used to represent the slowdown in the number of cleaning cycles as stubbornness increases, thus avoiding over-cleaning. A surface material tolerance coefficient is also introduced. This solves the problem of easily damaging soft household materials during cleaning. It also improves the cleaning success rate. By leveraging the robot's long-term memory and data optimization capabilities, adaptive parameter adjustments can be made based on the robot's past cleaning experience.

[0065] Specifically, the formula for calculating the cleaning speed for each stain is as follows: ; ; in, The appropriate cleaning speed for the stain (limited to 0.2~0.6m / s, avoid being too fast or too slow); The reference speed is (preferably 0.6 m / s). The number of cleaning cycles corresponding to the stain; To integrate changes in color difference, humidity, and friction; The color difference between the stained area and the background area (≤45, the upper limit of color difference for common household stains). The humidity in the stained area changes; stain residue can cause abnormal humidity. This refers to the change in friction in the stained area; stubborn stains can cause increased friction.

[0066] It should be noted that, and Inversely proportional, the wiping speed decreases with more wiping sessions to ensure sufficient contact time and effectiveness for each wipe. Through the combined action of three-dimensional features, an adaptive closed-loop adjustment is achieved to correct for color difference, humidity difference, and force difference. When =15, the correction factor is 1, with no additional adjustment. When the value is greater than 15, the correction factor is less than 1, reducing the speed. When the value is less than 15, the correction factor should be greater than 1, and the speed should be increased appropriately.

[0067] When the pollution type is garbage, the cleaning method is to use a broom and dustpan. When the humanoid robot performs garbage cleaning, its dexterous hand grips the upper part of the broom handle (30-40cm from the brush head). The bionic knuckles of the dexterous hand are equipped with anti-slip rubber pads, and the gripping contact area with the broom handle is ≥4cm². After gripping, the broom handle forms an angle of 60°-75° with the horizontal plane. The robot's right arm grips the broom, and its left arm grips the dustpan, achieving a coordinated sweeping and garbage collection operation. The gripping force of the broom is quantitatively controlled according to the formula F=μ×m×g, where the coefficient of friction μ between the anti-slip rubber pad and the broom handle is 0.6, the broom mass m is 0.5-1kg, and the gripping force is controlled between 3-6N. During the sweeping process, the dexterous hand makes real-time fine adjustments to the gripping force based on the force feedback, with a force fluctuation range ≤±0.5N. The robot moves linearly with a step distance of 10-15cm, cleaning at a speed of 0.6±0.2m / s. Its dexterous arm drives the broom in a horizontal reciprocating motion of ±30°, with a sweeping frequency of 1-2 times / second. During cleaning, the dustpan opening is kept directly facing the sweeping direction of the broom, sweeping debris into the dustpan for collection. After collection, the robot moves to the preset position of the trash can, and its left arm moves the dustpan 5cm above the trash can, tilting the dustpan to empty the debris into the trash can. If debris sticks to the broom head, the robot controls the broom head to gently scrape the edge of the trash can for self-cleaning. After the brush head finishes self-cleaning, the robot moves the broom and dustpan to the preset designated storage location. Then, the robot performs a garbage overflow detection operation at the garbage can. There are two detection methods: one is to collect image information of the garbage can through a visual sensor, and the AI ​​processor analyzes the image to determine whether the garbage is higher than the edge of the garbage can. If it is determined to be overflowing, the garbage bag is recycled. The other is to detect the weight m of the garbage bag through the weight sensor built into the robot's end effector. When m ≥ 2kg (to prevent the garbage bag from breaking due to excessive weight), the garbage bag is automatically sealed and the sealed garbage bag is moved to the preset garbage recycling area.

[0068] Leveraging the robot's SLAM navigation and bipedal walking capabilities, it can detect overflow at all garbage collection points throughout the house without requiring additional sensors on the garbage cans. It is compatible with ordinary garbage cans, making it more versatile. The overflow detection sensors and the disposal actuators are all universal modules for robots, requiring no additional hardware. Furthermore, the detection and cleaning actions are seamlessly integrated, improving overall operational efficiency.

[0069] In some feasible implementations, a trained ResNet neural network model (covering 500+ common household items) is used to identify item types with an accuracy rate of ≥98%. A preset placement database is invoked, storing the mapping relationship between item type and placement location (e.g., tableware → first shelf of kitchen sterilizer, books → middle shelf of living room bookshelf), which users can customize via an app. Preset placement rules include gentle handling, categorized placement, and proximity-based placement principles.

[0070] The first rule, the "handle with care" rule, includes: calculating the gripping force of the item based on the robot's material stiffness, the item's center of gravity, and the item's fragility level. The formula for calculating the gripping force is as follows: ; in, The clamping force of the item; For safety factor (preferably 1.2); For the quality of the goods; It is the acceleration due to gravity; The fragile grade attenuation coefficient of the item (preferably 0.15); The item's fragility level (preferably 1 to 5, with 1 being the least fragile and 5 being the most fragile, as indicated by the user's app). The reference contact area (preferably 2 cm²) The actual clamping contact area is the largest possible area. The larger the actual clamping contact area, the greater the clamping force and the less likely it is to damage the item. The reference stiffness for the robot material is preferably 100 N / m, which can provide sufficient support while avoiding rigid impact. The real-time stiffness of the robot material (collected by strain gauges) represents the robot actuator's ability to resist deformation, i.e., its hardness or softness. The reference center distance represents the distance from the center of gravity to the clamping point in the ideal clamping position. The distance between the center of gravity of the item and the clamping point is 2 to 5 cm, preferably 3 cm. It is a typical distance from the center of gravity to the edge of common household items (such as cups, bowls, and mobile phones).

[0071] Actual clamping contact area The calculation method is as follows: The robot's dexterous hand is equipped with a high-resolution flexible pressure sensor array (resolution ≥ 0.1 cm²). The sensor array consists of multiple independent pressure detection units, which can detect the pressure value of each unit (≥ 0.01 N is considered effective contact). After the dexterous hand completes the grasping of the object, the pressure sensor array detects the pressure value of each unit and filters out the effective detection units with a pressure value ≥ 0.01 N. The area formed by these units is the actual gripping contact area. Based on the calibration parameters of the pressure sensor array (the area of ​​each detection unit), the sum of the areas of all effective detection units is calculated, which is the actual gripping contact area. (cm²); If the object's posture changes slightly during clamping, the sensor array will update the pressure data in real time and correct accordingly. value.

[0072] By incorporating real-time material stiffness into the robot, the system automatically increases the maximum gripping force when the robot's end effector deforms due to aging or load (material softening), ensuring gripping stability. This integrates the mechanical characteristics of the end effector into the control algorithm, achieving integrated hardware and software safety control. Furthermore, by calculating the object's center of gravity distance, the system appropriately increases the gripping force to ensure stability when the gripping point is far from the center of gravity, and decreases the gripping force to reduce compression when it is close to the center of gravity. This upgrade from a "single force" to a "coordinated force and center of gravity position" approach solves the problem of balancing gripping stability and safety for objects of different shapes.

[0073] The system employs a four-tiered control system—segmented speed control, pressure feedback, attitude correction, and low-speed release—to prevent collisions and compression between the item and the placement surface. Specifically: 1. Based on the 3D model, obtain the precise 3D coordinates of the area where the item is placed (such as the left shelf of a restaurant cabinet), and determine the height coordinates of the surface where the item is placed.

[0074] 2. High-speed and low-speed segmented control is used during the transfer of items to the designated area: High-altitude phase (distance from the landing surface > 5cm): descent speed is 0.2m / s, rapid approach; Low-altitude phase (≤5cm from the landing surface): The descent speed drops sharply to below 0.05m / s, approaching the landing surface at low speed, significantly reducing the collision kinetic energy.

[0075] 3. The robot's dexterous hand is equipped with a high-resolution flexible pressure sensor array on the inside. When the sensor detects a change in pressure, it immediately stops the item from descending and reduces the clamping force to less than 30% of the original clamping force (it must be greater than the lower limit of the clamping force, which is fixed at 0.05N) to avoid squeezing the item.

[0076] 4. The placement posture of the object is captured in real time by a 3D vision sensor to ensure that the object is in parallel contact with the placement surface (tilt angle ≤ 1°). If there is a tilt, the robotic arm will finely adjust the joint angle to correct the posture before releasing.

[0077] 5. After posture correction, the dexterous hand slowly opens (opening speed 0.1cm / s) to release the item; after release, the vision sensor detects the placement deviation of the item. If the deviation is ≤2cm, the item is gently placed; if the deviation is >2cm, the robotic arm makes slight adjustments to ensure accurate placement.

[0078] The second rule, the categorization and placement rule, includes: folding clothes by season / type, specifically: 1. Accurate Clothing Recognition and Classification: Clothing is dually recognized using ResNet optimization algorithm and 3D point cloud. It not only identifies clothing type (tops, pants, skirts, socks, etc.) and seasonal attributes (summer: short sleeves / shorts / dresses; winter: sweaters / thick pants / down jackets, etc.), but also extracts the three-dimensional size parameters (length, width, thickness) and fabric characteristics (cotton, silk, knit, etc.). The recognition accuracy is ≥98%. The recognition results are synchronized to the dexterous hand control module in real time, providing a basis for the subsequent clamping and folding action parameter adaptation.

[0079] 2. Standardized Folding Scheme Matching: A pre-set clothing folding database stores standardized folding steps for different types and seasons of clothing (e.g., summer T-shirts: three-fold method; trousers: half-fold + rolled hem; winter sweaters: half-fold method). It also adapts finger movement parameters (e.g., pinching force, bending angle, and knuckle coordination rhythm) to suit the multi-degree-of-freedom characteristics of humanoid dexterous hands. Simultaneously, based on the clothing's three-dimensional dimensions and fabric characteristics, it adaptively adjusts the dexterous hand's range of motion (e.g., reducing pinching force to 0.1~0.3N for silk clothing, and increasing knuckle bending torque for thick sweaters) to ensure that the size of the folded clothing matches the storage space.

[0080] 3. Dexterous Hand Coordinating with Robotic Arm for Folding Actions: The humanoid robot's multi-degree-of-freedom robotic arm is equipped with a humanoid dexterous hand at its end. The dexterous hand has a built-in pressure sensor module that can sense the contact pressure with the clothing in real time. Before folding, the dexterous hand holds the edge of the clothing by "fingertip contact + gentle pinching" to prevent the fabric from being stretched and deformed. During the folding process, the dexterous hand completes the coordinated bending, translation, and stretching actions of the fingers according to preset steps (such as when folding a T-shirt, the thumb and forefinger hold the collar, and the other fingers help to smooth out the wrinkles). The robotic arm adjusts its spatial posture in sync, and the overall motion control accuracy is ≤0.1cm. The visual sensor provides real-time feedback on the shape of the clothing. If wrinkles or folding deviations are detected, the dexterous hand dynamically corrects them through fine-tuning actions of the knuckles (such as gently pushing and smoothing with a single finger) to avoid irregular folding.

[0081] 4. Folding size detection and storage matching: After folding, the folding size of the garment is detected by a 3D vision sensor and compared with the size of the preset storage space (wardrobe compartment, storage box) to ensure that the matching degree between the folded size and the storage space is ≥90%. If the matching degree does not meet the standard, the dexterous hand can accurately grasp the edge of the garment and readjust the folding angle / amplitude until it meets the requirements.

[0082] 5. Folding and Returning to Place: The dexterous hand uses a "squeeze and lift" method to pick up properly folded clothes and moves them to the designated storage location according to the placement rules (e.g., summer clothes → upper layer of the master bedroom wardrobe, winter thick sweaters → middle layer of the wardrobe). When returning to place, the dexterous hand adjusts the spread of its fingers according to the size of the storage space, slowly releases the clothes, and ensures that the clothes are placed flat, completing the integrated operation of "identification-folding-returning".

[0083] The third rule, the principle of nearest return location, includes: calculating the path cost from the current location to the return location, and selecting the path corresponding to the minimum path cost.

[0084] Specifically, path cost , From current position to node The actual distance For nodes Manhattan distance to the original location.

[0085] In some feasible implementations, the following steps are also included: After the cleaning and return processes are completed, the effectiveness is tested by comparing the dust residue rate, stain removal rate, and visual recognition. If the effect is not up to standard, the cleaning parameters are adjusted and the item type identification and return are performed again.

[0086] Cleaning effect test: 1. Dust Residue Rate , The area of ​​residual dust. The total area of ​​dusty areas before cleaning is considered acceptable if R ≤ 5%.

[0087] 2. Stain removal rate , represents the area of ​​the remaining stain. The total area of ​​the stained area before cleaning is considered acceptable if R ≥ 90%.

[0088] 3. Waste detection: Visual recognition indicates that the cleaned area is free of waste (confidence level ≥ 95%) to meet the standard.

[0089] Repositioning effect test: 1. Completeness: All scattered items are moved to their original positions without any omissions to meet the standard.

[0090] 2. Accuracy: The items are placed in the correct locations that match the database, and there are no misplaced items, which is considered satisfactory.

[0091] 3. Neatness: The deviation between the edge of the item and the boundary of the placement area is ≤2cm to meet the standard.

[0092] Corrective actions: If dust cleaning is not up to standard, increase cleaning intensity by 20% and decrease speed by 30%; if stain cleaning is not up to standard, increase the number of wipes by 1 and add more cleaning agent; if placement is not up to standard, re-identify the item and its placement location, and perform the placement operation again until the standard is met.

[0093] Specifically, , , It is based on pixel-physical size calibration of 3D vision sensor and is calculated in a unified pixel coordinate system. The core is to extract the pixel range of the corresponding area through vision sensor and then convert it into the actual physical area. All area extractions have been denoised to eliminate interference from lighting, reflection and other factors.

[0094] In addition, users can set cleaning times (e.g., 6:00 PM daily, Saturday mornings), prioritize cleaning areas (increasing cleaning frequency by 50%), and modify return locations via the terminal app. The system updates the database in real time. Infrared sensors detect human signals within a 2-meter range, with a sampling frequency set to 10Hz. When a human signal is detected, the system pauses operation; if no human signal is detected for 30 consecutive seconds (300 frames), the system resumes operation. When an unrecorded item is identified, it is moved to a temporary storage area, and multi-view images of the item are captured and pushed to the user's terminal. After receiving feedback on the return location from the user, the system updates the return database and the item recognition model training sample library. Subsequently, similar items will automatically return to their original locations. After the operation is completed, a report is generated that includes the cleaning area, operation time, relevant parameters, quantity and type of returned items, test results, etc., and is pushed to the user terminal.

[0095] Example 2: This embodiment also proposes a storage medium storing a home cleaning and item placement method based on a humanoid robot. When the humanoid robot-based home cleaning and item placement program is executed, it implements the steps of the humanoid robot-based home cleaning and item placement method described above. Since this storage medium adopts all the technical solutions of all the above embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated upon here.

[0096] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for home cleaning and item return based on a humanoid robot, characterized in that, include: S1. The robot collects point cloud data of the home space and constructs a three-dimensional model of the home space based on the point cloud data. S2. Calculate the shape factor and bounding box volume of objects based on point cloud data of the home space, and distinguish between fixed furniture and movable items based on the shape factor and bounding box volume; S3. Perform cluster analysis on the point cloud of fixed furniture to obtain functional area division; S4. Based on a pre-defined database of standard storage locations for items, identify scattered items among movable items; S5. Acquire images of the functional area, calculate gray value deviation, local height difference and edge gradient based on the images of the functional area, and identify the type of contamination based on gray value deviation, local height difference and edge gradient; S6. Match cleaning methods and cleaning parameters according to the type of pollution, start the cleaning process, identify the type of scattered items, and call the preset placement database and preset placement rules to start the placement process.

2. The method for home cleaning and item return based on a humanoid robot according to claim 1, characterized in that, In step S5, the type of contamination is identified based on grayscale deviation, local height difference, and edge gradient, including: When 5% ≤ grayscale deviation < 10%, the pollution type is dust; When the grayscale value deviation is ≥10%, the local height difference is <preset height difference threshold, and the edge gradient is <preset edge gradient threshold, the contamination type is stain. When the grayscale value deviation is ≥10%, the local height difference is <preset height difference threshold, and the edge gradient is ≥preset edge gradient threshold, the pollution type is garbage; When the grayscale value deviation is ≥10% and the local height difference is ≥the preset height difference threshold, the pollution type is garbage.

3. The method for home cleaning and item return based on a humanoid robot according to claim 1, characterized in that, In step S6, matching the cleaning method and cleaning parameters according to the type of contamination includes: When the type of contamination is dust, the cleaning intensity corresponding to the dust is calculated based on the dust thickness, particle hardness, and robot working wear. When the type of contamination is stain, the number of cleaning cycles corresponding to the stain is calculated based on the surface material tolerance, cleaning success rate, and stain stubbornness. The cleaning speed corresponding to the stain is calculated based on the number of cleaning cycles, humidity changes, friction changes, and color difference changes between the stained area and the background area.

4. The method for home cleaning and item return based on a humanoid robot according to claim 3, characterized in that, The formula for calculating the cleaning power corresponding to the dust is as follows: ； ； in, The cleaning intensity corresponding to dust; This represents the upper limit of the surface material's strength. This is the force coefficient; Dust thickness; This is the particulate matter hardness weighting coefficient; Mohs hardness of particulate matter; This represents the robot loss coefficient. Accumulate the robot's working hours.

5. The method for home cleaning and item return based on a humanoid robot according to claim 3, characterized in that, The formula for calculating the number of cleaning cycles required to address the stain is as follows: ； in, The number of cleaning cycles corresponding to the stain; Stain stubbornness; For surface material weighting coefficients; The surface material's tolerance coefficient; To improve the success rate of cleaning.

6. The method for home cleaning and item return based on a humanoid robot according to claim 5, characterized in that, The formula for calculating the cleaning speed corresponding to the stain is as follows: ； ； in, The corresponding cleaning speed for the stain; As the reference speed; The number of cleaning cycles corresponding to the stain; To integrate changes in color difference, humidity, and friction; This refers to the color difference between the stained area and the background area. For changes in humidity in the stained area; This represents the change in friction in the stained area.

7. The method for home cleaning and item return based on a humanoid robot according to claim 1, characterized in that, The preset placement rules include the rule of handling with care, the rule of classifying and placing, and the principle of placing in the nearest place; The rules for handling items gently include: calculating the gripping force of the item based on the robot's material stiffness, the item's center of gravity, and the item's fragility level; The principle of nearest relocation includes: calculating the path cost from the current location to the relocation location, and selecting the path corresponding to the minimum path cost.

8. The method for home cleaning and item return based on a humanoid robot according to claim 7, characterized in that, The formula for calculating the clamping force of the item is as follows: ； in, The clamping force of the item; For safety factor; For the quality of the goods; It is the acceleration due to gravity; This is the decay coefficient for the item's fragility level; The item is classified as fragile. The reference contact area; This represents the actual clamping contact area. The reference stiffness for robot materials; Real-time stiffness of robot materials; The distance is the baseline center of gravity. This is the actual distance between the object's center of gravity and the clamping point.

9. The method for home cleaning and item return based on a humanoid robot according to claim 1, characterized in that, In step S2, the distinction between fixed furniture and movable items is made based on shape factor and bounding box volume, including: When the bounding box volume is greater than or equal to a preset volume threshold and the shape factor is greater than or equal to a preset shape factor threshold, it is determined to be fixed furniture; otherwise, it is determined to be movable item.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the household cleaning and item placement method as described in any one of claims 1 to 9.

Images