A global cleaning system and a control method thereof

Abstract

The application discloses a global cleaning system and a control method thereof. The control method of the global cleaning system comprises the following steps: constructing a memory base, the memory base is used for storing five entities of a timestamp, a spatial coordinate, an image, text and an object; obtaining entity information of an intelligent agent and recording the entity information to the memory base; and performing a task by the global cleaning system according to the memory base, wherein an end-side device in the global cleaning system performs a cleaning task according to the memory base. The application has the advantages that the sensing and iteration of a cleaning area environment can be more lightweight and intelligent.

Description

Technical Field

[0001] This application relates to the field of intelligent cleaning technology, and in particular to a full-area cleaning system and its control method. Background Technology

[0002] The demand for large-scale implementation of smart property management and unmanned sanitation services continues to rise. Existing commercial unmanned cleaning operation and maintenance management systems and supporting robot cluster operation technologies have formed two mainstream implementation paradigms, which are widely used in lightweight sanitation duty scenarios in small and medium-sized parks. These are the basic known technologies that this invention is aligned with and adapted to. The overall architecture is mature and the implementation link is conventional, which can support the closed loop of conventional low-complexity cleaning operations.

[0003] Category 1: Isolated, stand-alone unmanned cleaning technology paradigm. This paradigm is the most commonly used and well-known solution in current small-scale property management scenarios. The core logic is that a single cleaning robot is equipped with LiDAR, wheel speed odometer, and local lightweight embedded computing unit. It pre-builds a global fixed grid map offline and locally fixes the single robot path planning algorithm and fixed-point reciprocating cleaning logic. It only supports independent start and stop of a single machine, fixed-point patrol, and local obstacle avoidance cleaning operations. The cloud only provides basic online equipment status ledger viewing and simple monitoring of power alarms. There is no cross-device data communication, no dynamic task linkage, and no real-time iteration capability of environmental data. It is suitable for low-frequency basic cleaning scenarios in single, regular small parks.

[0004] The second category: Weakly linked cloud-based centralized scheduling cleaning cluster technology paradigm. This paradigm is optimized and iterated for scenarios involving the batch deployment of multiple robots in large and medium-sized property parks, aligning with existing mainstream and patented technology solutions, and fully implementing standardized implementation links. The overall architecture adopts a simple cloud-based management platform + multiple homogeneous cleaning robots. A basic task dispatch ledger module is built in the cloud, and fixed cleaning areas and fixed time slots are manually defined in batches, with unified start and stop operation instructions issued. The robot relies on homogeneous laser SLAM to reuse the offline park map, and each robot completes local obstacle avoidance and closed-loop cleaning along the preset area path locally. The backend only summarizes offline ledger data on operation mileage, coverage, and equipment failure. There is no real-time environmental data interaction between robots, no dynamic congestion avoidance coordination, no garbage hotspot linkage and replenishment strategy, and no nearby real-time computing power transfer at the edge, nor local emergency linkage capability for sudden working conditions. The entire link relies on remote polling management in the cloud.

[0005] The closest existing mature technical solution to this invention is: a cloud-based centralized fixed scheduling system + laser static mapping + a multi-sweeping robot independent operation and linkage cleaning system, coupled with an integrated operation and maintenance solution for a commercial property sanitation robot cluster management platform. Mainstream sanitation robot manufacturers on the market all have standardized implementation products, and the core implementation steps are complete and traceable. Firstly, preliminary static data preparation: Property maintenance personnel shut down dynamic pedestrian and vehicle traffic throughout the park, relying on high-configuration laser radar inspection vehicles to traverse the entire area, collecting fixed terrain data of the park in one go, generating a globally unified static grid navigation map offline, manually marking fixed cleaning zones, fire lanes, restricted areas, and charging pile locations, and importing all data into the cloud management platform for later storage; Secondly, cloud-based manual fixed scheduling configuration: The operation and maintenance backend manually divides multiple sweeping robots into dedicated operation areas according to fixed cleaning times in the morning, noon, and evening, binding fixed-point patrol routes, reciprocating cleaning frequency, and charging return thresholds, and presets basic operation parameters adapted to normal weather conditions, without dynamic adaptive allocation logic; Thirdly, batch remote issuance of operation instructions: The cloud sends unified commands for power-on, area cleaning, resume cleaning, and return-to-home monitoring at preset times each day. The robots receive the commands synchronously and execute them rigidly according to the preset trajectory. Fourth, single-machine local closed-loop monitoring operation: Each cleaning robot navigates independently based on a local fixed static map, equipped only with basic infrared obstacle avoidance sensors to avoid temporary static obstacles. It does not recognize dynamic pedestrian flow or temporary piles of garbage and debris. During single-machine operation, there is no communication, avoidance, or replacement between robots. Fifth, offline ledger retrospective maintenance: After the daily operation is completed, the robot uploads fixed ledger data offline, including operation mileage, remaining battery power, and fault codes. The cloud only performs data summary and pop-up alarms. It does not retrospectively optimize the cleaning schedule for the next day or iteratively update the park environment map.

[0006] A few existing solutions are equipped with lightweight environmental perception modules, which only collect local temporary garbage locations on a single device and do not perform multi-device full-area fusion statistics. Mainstream cleaning navigation systems generally rely on explicit geometric representations such as point clouds and 3D Gaussian maps constructed by LiDAR as the core storage form of maps. This type of representation data is massive, has high update costs, and poor compatibility with AI semantic models and natural language commands. At the same time, the industry lacks a dedicated multimodal spatiotemporal memory sharing base. All environmental data and operation data are scattered and stored locally on a single device and in fragmented ledgers in the cloud, making it impossible to synchronize and reuse or iterate and evolve across devices in real time.

[0007] Based on the above existing technical solutions, and benchmarking against the complex real-world conditions of mixed pedestrian and vehicle traffic, variable weather, dynamic terrain transformation, and sudden garbage accumulation, the existing technologies have four core structural technical defects. These are all rigid technical pain points that this invention addresses in a closed-loop manner. There are no issues related to subjective commercial adaptability; all are shortcomings in the underlying architecture, data interaction, perception deployment, and scheduling linkage.

[0008] 1. The environmental data across the entire area is severely fragmented and isolated, with extremely poor map reuse and iteration capabilities, and weak adaptability to dynamic park conditions. All existing benchmark solutions adopt a single-machine local scattered storage and cloud-based static ledger storage mode, lacking a unified spatiotemporal environmental memory carrier; 3D spatial topography, real-time garbage heat distribution, park passage semantic attributes, traffic topology routes, and day and night weather time series multi-dimensional data are fixed in different robots and different backend modules, making real-time interoperability and sharing impossible; under the scenario of temporary addition of fences, green plant trimming, road repair, and routine dynamic topography changes such as outdoor clutter in shops, static maps quickly become invalid, with frequent single-machine checkpoints, positioning drift, and planned path breaks, requiring manual secondary full-area re-surveying and map redrawing, doubling the operation and maintenance manpower costs, interrupting and stalling full-area cleaning operations, and failing to adapt to the routine dynamic operation and maintenance rhythm of property management.

[0009] 2. The lack of coordinated control among all types of heterogeneous cleaning equipment, the disconnect between human and machine cleaning scheduling, and extremely low overall resource utilization. Existing technology only supports simple, fixed-area operations for cleaning robots of the same model and origin, and cannot be compatible with cross-scenario heterogeneous linkage between large-scale sanitation flagship equipment, small flexible inspection sweepers, quadrupedal off-road cleaning robots, and low-profile indoor sweepers; it does not incorporate on-duty human cleaners and park inspection personnel into a closed-loop coordination mechanism, resulting in congestion and overlapping blind spots between intelligent equipment and human operation routes, leading to frequent missed cleanings; there is no dynamic intelligent computing power scheduling and adaptation mechanism, and in scenarios such as peak garbage surges or sudden park events requiring additional cleaning, it is impossible to draw on nearby idle equipment or allocate nearby cleaning personnel for emergency supplementary cleaning, resulting in equipment idling and power consumption and idle personnel during off-peak hours, serious waste of sanitation resources across the entire area, and delayed and excessive emergency cleaning response.

[0010] 3. Reliance on expensive LiDAR hardware results in high deployment and upgrade costs, and perception and positioning failures occur in complex lighting scenarios. Existing benchmark solutions rely entirely on industrial-grade multi-line LiDAR to complete the entire process of mapping, positioning, and navigation. The cost of purchasing hardware per unit and subsequent calibration and maintenance remains high, placing a significant financial burden on small and medium-sized property parks for large-scale deployment. At the same time, LiDAR has limited resistance to backlight, rain, snow, fog, and dust interference. In complex scenarios such as morning commutes in backlight, low-light conditions at night, dusty weather during the rainy season, and dim lighting in underground parking garages, positioning jumps, mapping distortions, and obstacle avoidance failures are frequently triggered. This leads to prominent safety and maintenance risks such as robots scraping against people and vehicles, getting stuck at checkpoints, and missing areas. The threshold for full-domain adaptation and implementation is extremely high.

[0011] 4. Traditional 3D explicit geometric representations are cumbersome and inefficient, lacking AI adaptation and semantic interaction capabilities. Current cleaning system maps generally use explicit geometric representations such as point clouds and 3D Gaussians for storage, which have four major drawbacks: First, the data volume is enormous, consuming high computing power for device storage and transmission, putting significant pressure on small and medium-sized devices; second, environmental updates are difficult, requiring a complete reconstruction of the geometric model for local scene changes, making it unsuitable for dynamic property scenarios; third, AI compatibility is poor, as pure geometric structures are difficult for large visual and language models to understand and adapt to; fourth, human-machine interaction coupling is low, unable to connect with natural language commands to achieve semantic retrieval and intelligent cleaning, only supporting mechanical path navigation, with extremely low intelligent limits. Summary of the Invention

[0012] This application provides a full-area cleaning system and its control method, which has the advantage of being able to perceive and iterate the cleaning area environment in a more lightweight and intelligent manner.

[0013] The above-mentioned objective of this application is achieved through the following technical solution: On the one hand, this application provides a method for controlling a full-area cleaning system, including the following steps: S1: Construct a memory base, which is used to store five types of entities: timestamps, spatial coordinates, images, text, and objects; S2: Acquire the entity information of the intelligent agent and record it to the memory base; S3: The whole-area cleaning system performs operations based on the memory base. The end-side devices in the whole-area cleaning system perform cleaning operations based on the memory base.

[0014] Furthermore, the object entities in step S1 include a single object, a cluster of multiple objects, a region, a route, or a closed space, and each object has a correspondence between a timestamp and spatial coordinates.

[0015] Furthermore, in step S2, obtaining the intelligent agent entity information includes the following steps: Obtain the timestamp of the intelligent agent and record it to the memory base; Acquire photos taken by the intelligent agent at a certain timestamp, infer the pose of the photos and the camera's intrinsic parameters based on the photos, and record the photos as image entities in the memory base. The object information is extracted from the photo using a visual language model and recorded in the memory base.

[0016] Furthermore, for the photos taken by the intelligent agent, the 6-dimensional pose of the photo is calculated through the VGGT neural network, and the viewpoint and direction of view in the 6-dimensional pose are recorded as attributes of the image entity in the memory base.

[0017] Furthermore, in the step of identifying and extracting object information from photos, the spatial position of the object is calculated using camera extrinsic and intrinsic parameters and recorded in the memory base; Object identification and extraction are achieved through detection and segmentation models. The two-dimensional bounding box or mask of the object obtained by detection and segmentation in the image is then combined with the depth information of the image and the intrinsic and extrinsic parameters of the camera. The depth information is obtained through a feedforward visual localization network, and the position and size of the target object in three-dimensional space are obtained through inverse projection.

[0018] Furthermore, a 2D semantic mask to 3D point cloud surface mapping technique is adopted, relying on the pixel-level index correspondence between the 2D semantic map and the 3D reconstructed point map to achieve direct reverse projection of 2D semantic labels to 3D space. First, open vocabulary 2D semantic segmentation is performed on the input image, and a pixel-by-pixel instance mask is generated based on the text prompts. After filtering by a confidence threshold, the union is taken to output a binary semantic label map. At the same time, the 3D reconstruction backbone network predicts the 3D spatial coordinates of the same image pixel by pixel, outputting a 3D point map with a resolution proportional to the image, along with a point-by-point reconstruction confidence score. The two naturally share the index relationship of the pixel grid. The core of the mapping is the functional composition of the pixel dimension: the 3D coordinates of each pixel position directly inherit the semantic label of the corresponding pixel. In engineering implementation, the semantic mask needs to be resampled to the point map resolution using nearest neighbor interpolation, strictly preserving discrete label attributes and avoiding the blurring of category boundaries caused by bilinear interpolation, which would destroy the discreteness assumption of semantic labels. Then, the aligned mask and the point map are flattened synchronously, and the spatial coordinates and semantic labels are matched pixel by pixel to complete the reverse mapping and output the initial 3D semantic point cloud.

[0019] The initial point cloud is located in the camera coordinate system and needs to be adapted to multi-view fusion by coordinate system normalization: first, the basis vectors from the camera to the body are rearranged, and then the attitude is corrected by pitch, yaw and rotation matrices; a three-level backoff strategy is adopted to restore the true scale, prioritizing the anchor scale with the motion baseline, secondly selecting the ground installation height prior, and taking the default value of the unit scale when there is no effective prior; finally, low-quality points are filtered according to the confidence percentile, and the corresponding semantic labels are removed simultaneously to eliminate mapping misalignment noise.

[0020] Furthermore, the end-side equipment in the whole-area cleaning system includes any one or more of the following: large outdoor sanitation cleaning vehicles, small flexible transfer cleaning robots for parks, quadruped cleaning robots, and sweeping machines.

[0021] Furthermore, step S2 also includes step S20: updating the memory base, including the following steps: S201: Obtain the new image P and the shooting position captured by the intelligent agent; search the memory base for the original image Q at the shooting position; calculate the relative pose based on the new image P and the original image Q, and then calculate the absolute coordinates of the new image P based on the absolute coordinates of the original image Q. S202: When the similarity between the new image P and the original image Q in the six-dimensional pose reaches a set threshold, replace the original image Q with the new image P; S203: When a new semantic object derived from a new image P overlaps with the original semantic object S in space by more than a set value, and the types are the same or conflict with each other, the original semantic object S is updated using the new semantic object derived from the new image P.

[0022] Furthermore, step S2 also includes the step of: collecting entity information through the end-side device. When the edge device first enters the memoryless space, it freely explores the graphless region using the A* algorithm and builds up memory information. The reward and penalty scores of the A* algorithm are calculated as follows: the road width d is estimated, the route traveled is used as a skeleton, and a penalty region is formed in the vertical direction of the route using the width d. The closer to the penalty region, the greater the penalty, and the farther away from the penalty region, the smaller the penalty. The centroid of the penalty region is used as the center point. The closer to the centroid, the greater the penalty, and the farther away from the centroid, the smaller the penalty. The two penalty scores are weighted according to a set ratio.

[0023] On another front, this application provides a full-area cleaning system, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the steps in the method described above.

[0024] In another aspect, this application provides a computer-readable medium storing a computer program that, when executed by a computer, implements the steps in the method described above.

[0025] In summary, the beneficial effects of this application are as follows: 1. This invention abandons the industry-standard point cloud and 3D Gaussian explicit geometric representation, and pioneers the use of pose-anchored images as the core storage form for the memory base. Combined with a five-in-one multimodal fusion architecture, the data storage volume is greatly reduced and the incremental update efficiency is doubled, completely solving the pain points of traditional maps being bulky, difficult to update, and costly to deploy. At the same time, the images are natively adapted to AI visual models and large language models, realizing the integration of environmental cognition, semantic understanding, and language interaction, breaking the intelligent semantic barrier of traditional cleaning systems.

[0026] 2. This invention relies on a five-in-one spatiotemporal memory unit, integrating multi-dimensional data of time sequence, space, vision, objects, and language to achieve garbage hotspot prediction, time-priority cleaning, and semantic task breakdown; coupled with dynamic and elastic scheduling of cloud, edge, and terminal, it is compatible with large-scale, quadrupedal, and indoor multi-form heterogeneous robots and human-human integrated collaboration, enabling nearby emergency cleaning and off-peak avoidance to prevent congestion; compared with the existing technology, the same model of equipment has the defects of fixed segmentation, human-machine separation, blind spot overlap and missed cleaning, and peak response lag.

[0027] 3. This invention uses VGGT pure vision SLAM to completely replace expensive LiDAR, significantly reducing the cost of hardware procurement and subsequent calibration and maintenance for a single device; at the same time, it enhances the anti-interference capabilities in backlight, low light, rain, snow and dust, and ensures accurate positioning in all scenarios such as underground parking garages, nighttime parks and early morning commutes without drifting or getting stuck; compared with the shortcomings of existing laser solutions, such as high cost, difficulty in deployment and frequent failures and downtime in severe weather, the threshold for mass deployment is greatly reduced. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating the principle of the present invention. Detailed Implementation

[0029] The specific embodiments of this application are described in detail below with reference to the accompanying drawings.

[0030] Example 1: A control method for a full-area cleaning system, comprising the following steps: S1: Construct a memory base, which stores five types of entities: timestamps, spatial coordinates, images, text, and objects. Images, text, and objects are organized according to time and space. Object entities include a single object, a cluster of multiple objects, a region, a route, and a closed space. Each object has a correspondence between a timestamp and spatial coordinates. The time of an object is generally organized according to a point in time or a time period, and the space of an object is organized according to polygons, circles, polyhedra, center points, sections, curves, etc.

[0031] Therefore, this memory base can be viewed as a cluster of objects organized spatiotemporally. This memory module does not explicitly store conventional 3D models such as meshes, point clouds, voxels, or 3D Gaussians. For unmanned cleaning systems, the main objects of concern include garbage, dirty areas, drivable areas (large vehicles), drivable areas (small vehicles), walkable areas (robots), stairs, steps, walls, toilets, and uncleanable areas.

[0032] This spatiotemporal memory base follows the design concept of Map as Memory, using Pose-Anchored Images (PAI) as the sole core representation of spatiotemporal memory across the entire domain, abandoning the explicit geometric representations such as point clouds and 3D Gaussian that traditional map sweeping relies on.

[0033] This base does not store any point cloud, 3D Gaussian, or mesh geometric data. It innovatively uses a keyframe image array with precise pose annotations as its core memory unit, supporting the entire process of SLAM mapping, dynamic planning, and intelligent decision-making across all scenarios. Compared to traditional explicit 3D geometric representations, it boasts five core advantages: First, it is extremely lightweight, with image storage volume far smaller than dense point clouds and 3D Gaussian models, significantly reducing local storage and cloud transmission computing power overhead, making it compatible with various lightweight cleaning terminal deployments; second, it is natively AI-friendly, with image features directly interfaced with VGGT vision. First, it features perceptual SLAM and multimodal large models, eliminating the need for secondary geometric data transformation and significantly improving the efficiency of feature extraction and scene understanding. Second, it is highly coupled with natural language, allowing image memory to be quickly parsed into semantic text that the large model can recognize, supporting language command retrieval, semantic cleanup, and intelligent report generation. Third, it is incrementally easy to update, requiring only the replacement of a single frame anchor image corresponding to the changing scene, without the need for global map reconstruction, thus adapting to dynamic environment iterations in the property. Fourth, it is easily reusable across multiple scenarios, with pose-based image sequences that can be directly adapted to multiple tasks such as mapping, localization, obstacle avoidance, and scheduling, making it far more versatile than a single geometric map.

[0034] S2: Acquire the entity information of the intelligent agent and record it to the memory base. Acquiring the entity information of the intelligent agent includes the following steps: Obtain the timestamp of the intelligent agent and record it to the memory base; Acquire photos taken by the intelligent agent at a certain timestamp, infer the pose of the photos and the camera's intrinsic parameters based on the photos, and record the photos as image entities in the memory base. The object information is identified and extracted from the photo and recorded in the memory base.

[0035] For photos taken by the intelligent agent, the 6-dimensional pose of the photo is calculated through the VGGT neural network, and the viewpoint and direction of view in the 6-dimensional pose are recorded as attributes of the image entity in the memory base.

[0036] In the step of identifying and extracting object information from photos, the spatial position of the object is calculated using camera extrinsic and intrinsic parameters and recorded in a memory base.

[0037] This step primarily involves constructing the memory based on the VGGT neural network and VLM. First, this module records the timestamp of the intelligent agent and the photos taken at a specific timestamp. Based on multiple photos and the VGGT neural network, the 6-dimensional pose (viewpoint and direction of view) and camera intrinsic parameters of each photo are calculated. Each photo is stored as an object [image] (time point, viewpoint, direction of view, text, and other attributes). Simultaneously, the VLM is used to identify and extract object information such as objects, routes, and regions from the photos. This object information, along with the timestamp and other attributes (recognized by the VLM), is organized into an object. The spatial location of the object can be calculated using the camera extrinsic and intrinsic parameters calculated by the VGGT and mapped to absolute world coordinates. The [object] (time point, spatial coordinates, spatial region, type, text, and other attributes) is then stored.

[0038] VGGT and its variants are feedforward visual localization networks based on deep neural networks. They take multiple images from different perspectives as input (which can be photos taken by different cameras at different times, with varying resolutions, and no calibration is required), and directly output the relative 6DOF pose (including 3 positional degrees of freedom and 3 directional degrees of freedom) of each photo relative to the first photo through a single inference. Feedforward visual localization networks are generally pre-trained with a large amount of data, thus exhibiting strong robustness (resistant to noise, unconstrained lighting, partial occlusion, dynamic interference, etc.) and versatility.

[0039] Extracting object information from photos is achieved through detection and segmentation models. Detection and segmentation in an image yields a two-dimensional bounding box or mask of the object. This two-dimensional bounding box or mask, combined with the image's depth information (usually obtained synchronously through a feedforward visual localization network) and camera intrinsic and extrinsic parameters, allows for the inverse projection of the target object's position and size in three-dimensional space.

[0040] After all the space within a region has been traversed, the spatial coordinates of all objects must be converted to absolute world coordinates (latitude and longitude, with north as the y-axis and the top as the z-axis). This paper employs a 2D semantic mask to 3D point cloud surface mapping technique, relying on the pixel-level index correspondence between the 2D semantic map and the 3D reconstructed point map to achieve direct reverse projection of 2D semantic labels to 3D space. First, open-vocabulary 2D semantic segmentation is performed on the input image. A pixel-by-pixel instance mask is generated based on text prompts, filtered by a confidence threshold, and then the union is taken to output a binary semantic label map. Simultaneously, a 3D reconstruction backbone network predicts the 3D spatial coordinates of the same image pixel by pixel, outputting a 3D point map with a resolution proportional to the image, along with point-by-point reconstruction confidence. The two naturally share the pixel grid index relationship. The core of the mapping is the functional composition of pixel dimensions: the 3D coordinates of each pixel directly inherit the semantic label of the corresponding pixel. In engineering implementation, the semantic mask needs to be resampled to the point map resolution using nearest-neighbor interpolation, strictly preserving discrete label attributes and avoiding the blurring of category boundaries caused by bilinear interpolation, which would violate the discreteness assumption of semantic labels. Then, the aligned mask and point map are simultaneously flattened, and the spatial coordinates and semantic labels are matched pixel by pixel to complete the reverse mapping and output the initial 3D semantic point cloud.

[0041] The initial point cloud is located in the camera coordinate system and needs to be adapted to multi-view fusion by coordinate system normalization: first, the basis vectors from the camera to the body are rearranged, and then the attitude is corrected by pitch, yaw and rotation matrices; a three-level backoff strategy is adopted to restore the true scale, prioritizing the anchor scale with the motion baseline, secondly selecting the ground installation height prior, and taking the default value of the unit scale when there is no effective prior; finally, low-quality points are filtered according to the confidence percentile, and the corresponding semantic labels are removed simultaneously to eliminate mapping misalignment noise.

[0042] Step S2 also includes step S20: updating the memory base, including the following steps: S201: Obtain the new image P and the shooting position captured by the intelligent agent; search the memory base for the original image Q at the shooting position; calculate the relative pose based on the new image P and the original image Q, and then calculate the absolute coordinates of the new image P based on the absolute coordinates of the original image Q. S202: When the similarity between the new image P and the original image Q in the six-dimensional pose reaches a set threshold, replace the original image Q with the new image P; S203: When a new semantic object derived from a new image P overlaps with the original semantic object S in space by more than a set value, and the types are the same or conflict with each other, the original semantic object S is updated using the new semantic object derived from the new image P.

[0043] Specifically, in this embodiment, when updating the memory base: when the intelligent subject accesses the area again, the area may have changed, therefore the memory needs to be updated. The update rules are as follows.

[0044] When the intelligent agent returns to a new location, if it is unaware of its own position, it can first use GPS to lock the range and search for the original image in the memory module within the locked range. The relative pose of the newly captured image P and the original image Q in memory is calculated (via VGGT), and then the absolute coordinates of p are calculated based on the absolute coordinates of the original image.

[0045] When the similarity between image P and the existing image R in six-dimensional pose reaches a certain threshold (viewpoint distance less than a certain threshold and viewing angle less than a certain value), image P can replace image R. This means adding image P as a new object to the memory module and deleting / archiving image R from the memory, including other non-image objects derived from image R. New semantic objects derived from image P can also be added to the memory system.

[0046] When a new semantic object derived from P has a large spatial overlap with the original semantic object S (the overlap ratio IOU is greater than a certain threshold, or the area of ​​the overlapping region is a certain percentage of the area of ​​a certain object region), and the types are related (the same or conflicting), the original object can be updated (updating the region or updating the semantic attributes). If the original object is completely covered by the new object, the original object can be deleted.

[0047] The maximum update frequency can be used for objects such as garbage, dirty areas, and images; a medium update frequency can be used for objects such as drivable (walkable) areas; and a lower update frequency can be used for objects such as walls, steps, stairs, and toilets.

[0048] S3: The all-area cleaning system performs operations based on the memory base. The end-side devices within the all-area cleaning system perform cleaning operations based on the memory base. The end-side devices in the all-area cleaning system include any one or more of the following: large outdoor sanitation sweeping vehicles, flexible small-sized cleaning robots for industrial parks, quadruped cleaning robots, and sweeping machines.

[0049] The intelligent agent mentioned in step 2 is a terminal device with camera acquisition function. For a full-area cleaning system, it can be a terminal device or a separate terminal. When collecting entity information through the terminal device: When the edge device first enters the memoryless space, it freely explores the graphless region using the A* algorithm and builds up memory information. The reward and penalty scores of the A* algorithm are calculated as follows: the road width d is estimated, the route traveled is used as a skeleton, and a penalty region is formed in the vertical direction of the route using the width d. The closer to the penalty region, the greater the penalty, and the farther away from the penalty region, the smaller the penalty. The centroid of the penalty region is used as the center point. The closer to the centroid, the greater the penalty, and the farther away from the centroid, the smaller the penalty. The two penalty scores are weighted according to a set ratio.

[0050] The constructed memory base does not record explicit 3D representations (such as point clouds, voxels, 3D Gaussians, meshes, etc.), but only images, text, semantic objects, poses, timestamps, etc. It records dense image pixel information, rather than sparse image features, possessing dense visual information, strong anti-interference capabilities, and better handling scenarios with missing visual anchors and dynamic interference, resulting in stronger localization robustness. Due to the use of a more robust feedforward localization model and visual language model, and by recording only image and semantic information, the spatial semantic map used in this invention has richer high-level semantic features and stronger map self-evolution capabilities, preventing error accumulation during map self-evolution.

[0051] Example 2: A full-area cleaning system includes a memory and a processor. The memory stores a computer program. When the computer program is called and executed by the processor, it implements the steps in the method described in Example 1.

[0052] Example 3: A computer-readable medium storing a computer program, which, when executed by a computer, implements the steps of the method described in Example 1.

[0053] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of this application, and these all fall within the protection scope of this application.

Claims

1. A control method for a full-area cleaning system, characterized in that, Includes the following steps: S1: Construct a memory base, which is used to store five types of entities: timestamps, spatial coordinates, images, text, and objects; S2: Acquire the entity information of the intelligent agent and record it to the memory base; S3: The whole-area cleaning system performs operations based on the memory base. The end-side devices in the whole-area cleaning system perform cleaning operations based on the memory base.

2. The control method for the whole-area cleaning system according to claim 1, characterized in that, The object entities in step S1 include a single object, a cluster of multiple objects, a region, a route, or a closed space. Each object has a correspondence between a timestamp and spatial coordinates.

3. The control method for the whole-area cleaning system according to claim 1, characterized in that, In step S2, obtaining the intelligent agent entity information includes the following steps: Obtain the timestamp of the intelligent agent and record it to the memory base; Acquire photos taken by the intelligent agent at a certain timestamp, infer the pose of the photos and the camera's intrinsic parameters based on the photos, and record the photos as image entities in the memory base. The object information is extracted from the photo using a visual language model and recorded in the memory base.

4. The control method for the whole-area cleaning system according to claim 1, characterized in that, For photos taken by the intelligent agent, the 6-dimensional pose of the photo is calculated through the VGGT neural network, and the viewpoint and direction of view in the 6-dimensional pose are recorded as attributes of the image entity in the memory base.

5. The control method for a full-area cleaning system according to claim 1, characterized in that, In the step of identifying and extracting object information from photos, the spatial position of the object is calculated using camera extrinsic and intrinsic parameters and recorded in the memory base; Object identification and extraction are achieved through detection and segmentation models. The two-dimensional bounding box or mask of the object obtained by detection and segmentation in the image is then combined with the depth information of the image and the intrinsic and extrinsic parameters of the camera. The depth information is obtained through a feedforward visual localization network, and the position and size of the target object in three-dimensional space are obtained through inverse projection.

6. The control method for the whole-area cleaning system according to claim 1, characterized in that, The end-side equipment in the comprehensive cleaning system includes any one or more of the following: large outdoor sanitation sweeping vehicles, small flexible transfer sweeping robots for parks, quadruped sweeping robots, and floor sweepers.

7. The control method for the whole-area cleaning system according to claim 1, characterized in that, Step S2 also includes step S20: updating the memory base, including the following steps: S201: Obtain the new image P and the shooting position captured by the intelligent agent; search the memory base for the original image Q at the shooting position; calculate the relative pose based on the new image P and the original image Q, and then calculate the absolute coordinates of the new image P based on the absolute coordinates of the original image Q. S202: When the similarity between the new image P and the original image Q in the six-dimensional pose reaches a set threshold, replace the original image Q with the new image P; S203: When a new semantic object derived from a new image P overlaps with the original semantic object S in space by more than a set value, and the types are the same or conflict with each other, the original semantic object S is updated using the new semantic object derived from the new image P.

8. The control method for a full-area cleaning system according to claim 1, characterized in that, Step S2 also includes the step of: collecting entity information through the end-side device. When the edge device first enters the memoryless space, it freely explores the graphless region using the A* algorithm and builds up memory information. The reward and penalty scores of the A* algorithm are calculated as follows: the road width d is estimated, the route traveled is used as a skeleton, and a penalty region is formed in the vertical direction of the route using the width d. The closer to the penalty region, the greater the penalty, and the farther away from the penalty region, the smaller the penalty. The centroid of the penalty region is used as the center point. The closer to the centroid, the greater the penalty, and the farther away from the centroid, the smaller the penalty. The two penalty scores are weighted according to a set ratio.

9. A comprehensive cleaning system, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is invoked and executed by the processor, it implements the steps of the method as described in any one of claims 1-7.

10. A computer-readable medium, characterized in that, The computer-readable medium stores a computer program that, when executed by a computer, implements the steps of the method as described in any one of claims 1-7.

Images