Full-automatic garbage clearance and control method based on collaborative scheduling

Abstract

According to the full-automatic garbage clearance and control method based on collaborative scheduling, the operation data bin fused with closed-loop constraints is constructed, it is guaranteed that data depended on in the follow-up collaborative control process is accurate, real and effective, and then the dynamic operation characteristic matrix is subjected to cyclic correlation in the dynamic operation characteristic matrix; it is ensured that each robot establishes direct association with other robots, it is avoided that the multi-robot cooperation effect is discounted due to weak association of end nodes, meanwhile, the association relationship is automatically updated along with changes of robot positions corrected according to closed-loop constraints, it is ensured that the optimal cooperation scheduling performance can be kept all the time when cooperation scheduling is conducted on the multiple robots, and the cooperative scheduling efficiency is improved. The executed task allocation strategy is always optimal through the multi-target reward function, so that full-automatic garbage collection of multi-robot collaborative scheduling, dynamic path planning and accurate state monitoring is realized, and the garbage collection efficiency and intelligent level of communities or communities are improved.

Description

Technical Field

[0001] This application relates to the field of robot automatic control technology, specifically to a fully automated garbage collection and control method based on collaborative scheduling. Background Technology

[0002] With the advancement of smart community construction, robot-based door-to-door automated garbage collection has become an important direction for improving community service levels. In community or residential area application scenarios, garbage collection robots need to go to designated locations in each residential building to complete garbage collection tasks, but existing technologies have significant shortcomings.

[0003] Currently, most community waste recycling robots operate independently, lacking an effective collaborative scheduling mechanism. Although some systems can detect the amount of waste and generate recycling tasks through sensors, task allocation is often based on a simple distance-first principle, without comprehensively considering multi-dimensional factors such as robot status, environmental risks, and insufficient indoor and outdoor navigation accuracy. For example, the sanitation task management method proposed in patent CN20069456A can only generate tasks based on the fullness of the garbage bins, and cannot cope with complex scenarios of multi-robot collaborative operations.

[0004] In terms of path planning, existing technologies mostly employ fixed routes or simple shortest path algorithms, which are difficult to adapt to dynamic environments such as narrow passages and temporary obstacles within communities. When multiple robots operate simultaneously, path conflicts or task point congestion can easily occur, affecting overall efficiency. Although patent CN120562840A constructs a global operation map, it lacks optimization for community door-to-door recycling scenarios, particularly in handling unexpected tasks and dynamic obstacle avoidance.

[0005] Furthermore, existing systems monitor robot status in a relatively simple way, typically focusing only on basic parameters such as battery level, while neglecting key indicators affecting task execution, such as positioning accuracy and sensor health. In terms of environmental risk assessment, there is also a lack of quantitative assessment mechanisms for community-sensitive issues such as waste leakage and odor spread.

[0006] Therefore, there is an urgent need for a fully automated waste collection and control method that can achieve multi-robot collaborative scheduling, dynamic path planning, and precise status monitoring, in order to improve the efficiency and intelligence level of waste recycling in communities or neighborhoods. Summary of the Invention

[0007] This application aims to achieve fully automated waste collection and transportation with multi-robot collaborative scheduling, dynamic path planning, and precise status monitoring, thereby improving the efficiency and intelligence level of waste recycling in communities or neighborhoods. It provides a fully automated waste collection and transportation and control method based on collaborative scheduling.

[0008] To achieve this objective, the following technical solution is adopted in this application: A fully automated waste collection and control method based on collaborative scheduling is provided, comprising the following steps: M1, builds and updates a dynamic scheduling view for the fusion of multi-source heterogeneous data; M2, establish a multi-factor coupled urgency assessment and task allocation mechanism; M3 implements coordinated control based on error state Kalman filtering and closed-loop correction.

[0009] Preferably, the method for constructing the dynamic scheduling view in step M1 includes the following steps: M11 integrates one or more multi-source heterogeneous data, including closed-loop constraints generated by the closed-loop detection module, real-time position, speed, and task status data of all online robots, waste status sensor data, and environmental topology information, to establish a time-stamped operation data warehouse. M12, extract multi-dimensional features from the job data warehouse and perform spatiotemporal fusion to generate a dynamic job feature matrix; M13, based on the dynamic task feature matrix, identifies the associated paths of each task area, constructs a task topology network, and integrates the task topology network, the closed-loop constraints, and the real-time pose, speed, and task status data of each robot into the dynamic scheduling view.

[0010] Preferably, the method for establishing a multi-factor coupled urgency assessment and task allocation mechanism in step M2 includes the following steps: M21, calculating the waste decomposition index based on waste surface image data; M22, combining leachate concentration data and unit interlayer gap length, analyzes the degree of leachate diffusion and establishes a leachate hazard assessment model; M23, calculate the pollution source escape risk value, and use the leachate hazard assessment model to calculate the landfill leachate hazard value. Then, combine the pollution source risk value and the landfill leachate hazard value to determine the urgency of treatment. M24 considers the processing urgency, robot health status, and path efficiency, and uses a designed multi-objective reward function. Generate the optimal task allocation strategy.

[0011] Preferably, in step M24, the method for generating the optimal task allocation strategy includes the following steps: M241, evaluating and generating candidate strategies; M242, utilizing The optimal strategy is evaluated using the following method: Candidate strategies Substitute into the multi-objective reward function In the middle, predicting the robot's execution strategy Total rewards that can be obtained Points, and will have the highest The strategy of dividing As the optimal task allocation strategy; M243, based on the strategy Based on the actual execution results, the actual total reward points will be recalculated. and will , The feedback is sent to the candidate policy generation module to retrain the candidate policy generation model.

[0012] Preferably, the robot health used in the multi-objective reward function includes a positioning reliability score for the robot, which is characterized by the position confidence carried in the closed-loop constraints calculated by the closed-loop detection module in the robot. The method for generating the closed-loop constraint by the closed-loop detection module includes the following steps: A1, the closed-loop detection module continuously receives image frames from the visual sensor and point cloud frames from the lidar, and calculates the global appearance descriptor of the current frame; A2, match the global appearance descriptor of the current frame with the historical key frame descriptors stored in the maintained local sliding window map, filter out several similar closed-loop candidate frames, and obtain the appearance similarity score between the current frame and each of the closed-loop candidate frames. A3, perform geometric consistency verification on each closed-loop candidate frame, check the reprojection error of matching feature points and the point cloud registration error, and calculate the geometric verification score between the current frame and the closed-loop candidate frame. A4. Determine whether the appearance similarity score and geometric verification score between the current frame and the closed-loop candidate frame are both greater than the corresponding adaptive score threshold. If so, proceed to step A5; If not, then the closed-loop candidate frame is removed; A5 sends the closed-loop constraints to the backend optimizer for global pose graph optimization and correction of cumulative errors. The position confidence is the reciprocal of the covariance of the pose correction.

[0013] Preferably, in step M3, the process of implementing the coordinated control based on error state Kalman filtering and closed-loop correction includes the following steps: M31, in the optimal strategy During execution, based on the closed-loop detection results and path tracking status, the robot's actual walking path is tracked and corrected online in real time; The M32 closed-loop detection module performs global optimization of the robot's pose based on the observation output fed back by the error state Kalman filter, and synchronizes the globally optimized pose to the central scheduling platform to update the dynamic scheduling view, forming a closed-loop feedback from single-machine localization to collaborative scheduling.

[0014] Preferably, in step M31, the method for real-time tracking and online correction of the robot's actual walking path includes the following steps: B1 uses a model predictive controller to calculate the robot's expected linear velocity and angular velocity based on the path points in the planned initial walking path; B2, calculates the lateral and orientation deviations between the robot's actual pose and the initial walking path in real time; B3, dynamically adjusts the prediction time domain and control time domain parameters of the path tracking controller based on the path curvature, robot speed and deviation magnitude; B4. When the lateral or orientation deviation exceeds a preset threshold, local replanning is triggered to generate a new path from the current position to the next sub-target point. B5, the pose correction amount confirmed by closed-loop detection is introduced as feedback into the path tracking controller to smoothly compensate for pose jumps in subsequent control cycles.

[0015] Preferably, in step B1, the method for planning the initial walking path includes the following steps: S31, The path planner receives semantic task instructions from the scheduling system and parses them into specific coordinate sequences in a multi-level semantic map; S32, based on the parsed coordinate sequence, plans the initial global path on the coarse-resolution raster map; S33, guided by the initial global path, local path planning is performed, while real-time laser data and dynamic semantic layer information are integrated to avoid static and dynamic obstacles.

[0016] This application has the following beneficial effects: 1. The multi-source data integrated in the constructed operation data warehouse includes closed-loop constraints continuously generated by the closed-loop detection module. These closed-loop constraints include global pose correction and position confidence. Therefore, the data in the constructed operation data warehouse is more effective, which is conducive to improving the efficiency of subsequent fully automatic control of waste collection and transportation and the accuracy and effectiveness of collaborative scheduling.

[0017] 2. In the dynamic task feature matrix, the circular association ensures that each robot establishes a direct association with other robots, avoiding the problem of weak association of end nodes. At the same time, the matrix is ​​dynamically rearranged as the robots move. When the robot position changes, the association relationship is automatically updated. Thus, when multiple robots are coordinated and scheduled in steps L1-L5, the optimal coordinated scheduling performance can always be maintained.

[0018] 3. Utilize the designed multi-objective reward function To evaluate whether the task allocation strategy is optimal, the reliability score of the robot positioning by the closed-loop detection module was considered. Based on the analysis of leachate hazards, gas emission hazards, and the dynamic calculation of processing priorities based on the duration of waste retention, the environmental risks were assessed by combining environmental wind speed and wind direction factors. The task allocation strategy was generated based on the robot's health status and remaining loading capacity, so that the final task allocation strategy was optimized, which improved the effectiveness and automation level of multi-robot control.

[0019] 4. The closed-loop constraints in the scheduling credibility map have undergone the screening process of closed-loop candidate frames in steps A1-A5, which ensures the credibility of the generated location confidence and thus improves the spatial perception reliability of subsequent multi-machine collaborative scheduling based on accurate indoor and outdoor fusion positioning information.

[0020] 5. Through steps M31-M32, an error state Kalman filter is used to jointly estimate the robot's pose and cleaning status. At the same time, a closed-loop detection module is embedded. The scheduling strategy is updated through pose correction constraints and task completion feedback. The two-way collaboration greatly improves the accuracy of pose correction and task strategy allocation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly described below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a diagram illustrating the implementation steps of the fully automated waste collection and control method based on collaborative scheduling provided in this application embodiment. Detailed Implementation

[0023] The technical solution of this application will be further described below with reference to the accompanying drawings and specific embodiments.

[0024] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual images. They should not be construed as limiting the scope of this application. To better illustrate the embodiments of this application, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0025] In the accompanying drawings of the embodiments of this application, the same or similar reference numerals correspond to the same or similar components. In the description of this application, it should be understood that if terms such as "upper," "lower," "left," "right," "inner," and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting this application. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0026] In the description of this application, unless otherwise expressly specified and limited, the term "connection" or similar designation indicating a connection between components should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0027] Accurate indoor and outdoor positioning and navigation of each waste collection robot performing tasks, providing reliable location information for multi-robot scheduling, is a prerequisite for achieving efficient and accurate collaborative scheduling. Effective collaborative scheduling is also fundamental to achieving fully automated waste collection and improving the efficiency and intelligence of community or residential waste recycling, as described in this application. Therefore, before introducing the fully automated waste collection and control method based on collaborative scheduling provided in this embodiment, the indoor and outdoor fusion navigation method for waste collection robots provided in this embodiment will be explained first, followed by the robot collaborative scheduling method based on fusion navigation, and finally the fully automated waste collection and control method based on collaborative scheduling provided in this embodiment.

[0028] The core technology of the indoor and outdoor fusion navigation method for garbage collection robots provided in this application lies in the synergistic effect of six core steps: multi-layer semantic map construction, tightly coupled multi-sensor fusion positioning and closed-loop correction, semantic and dynamic path planning, real-time tracking and online correction of walking paths, seamless cross-regional switching, and task-oriented active perception. This enables the garbage collection robot to achieve fully autonomous, accurate, and robust navigation from the community garbage station to the doorstep of the resident.

[0029] Specifically, the indoor and outdoor integrated navigation method for the cleaning robot provided in this embodiment includes the following steps: S1, construct and update a multi-level semantic map that integrates semantic surface information from visual sensors and geometric structure information from LiDAR, and includes dynamic semantic layers; S2 fuses lidar point cloud features, visual odometry, and inertial measurement unit data in a tightly coupled manner, and introduces a closed-loop detection module to correct the cumulative errors generated during long-term operation. S3, based on semantic and dynamic environment understanding, plans the path. The path planner comprehensively considers static obstacles, dynamic obstacle prediction results, semantic rules and task priorities. Then, by comparing the deviation between the planned path and the actual walking path and integrating the closed-loop detection results, it generates control commands in real time to track and correct the actual walking path online in real time. S4 defines the navigation switching strategy for indoor and outdoor transition areas, and dynamically adjusts the contribution weight of different sensors in fusion positioning based on the credibility of environmental features. S5, establish an error state Kalman filter, and use the path integral error and closed-loop correction as the observation input; S6, determine whether the sensor signal meets the conditions for enabling the degradation navigation strategy. If so, then a degraded navigation strategy based on historical path and multimodal fusion will be enabled; If not, proceed to step S7; S7, determine whether the navigation target point has been reached. If so, then by identifying specific semantic targets, localization verification and pose fine-tuning are performed to complete the action interaction with the target task; If not, the observed output of the error state Kalman filter is fed back to the closed-loop detection module.

[0030] Step S1, the method for constructing and updating a multi-level semantic map specifically includes the following steps: S11 collects environmental data through the RGB-D camera and LiDAR on the cleaning robot, and uses a real-time semantic segmentation network to identify static semantic landmarks, including doors, windows, elevator buttons, trash cans, house numbers, etc. In this embodiment, the cleaning robot simultaneously collects environmental data using its onboard RGB-D camera and LiDAR. The RGB-D camera provides color images and their corresponding pixel depth information, while the LiDAR provides a precise 3D point cloud of the surrounding environment. The real-time semantic segmentation network employs an improved DeepLabV3 model, deployed on the robot's local computing unit or an edge server. Its specific recognition process is as follows: The improved DeepLabV3 model receives two types of data simultaneously: an RGB image, which provides rich color and texture information; and a density depth map (derived from an RGB-D camera) aligned with the image through coordinate transformation or a sparse depth map (derived from LiDAR) generated through point cloud projection. These two types of data together constitute the model's input.

[0031] The encoder portion of the model contains a backbone network (preferably MobileNetV3 or EfficientNet-Lite in this embodiment) for extracting high-level feature maps from RGB images. Simultaneously, depth information is processed either as a separate channel or through a parallel branch. At specific layers of the encoder, visual and depth features are fused, enabling the network to simultaneously understand "what it is" (appearance) and "where it is" (geometry).

[0032] The fused features are fed into the decoder. The decoder gradually recovers the spatial resolution of the feature map through upsampling and skip connections, ultimately outputting a segmentation map of the same size as the input RGB image. Each pixel in this segmentation map is classified into a predefined semantic category, such as door, window, elevator button, trash can, house number, wall, ground, etc.

[0033] The post-processing module performs connected component analysis on the segmentation results, clustering pixels belonging to the same object to separate different instances. For example, it distinguishes between two adjacent doors. For each identified instance, the system records its pixel-level mask, category label, and confidence score. Combining depth information or point cloud data, the 3D geometric bounding box of the instance in the robot coordinate system is also calculated.

[0034] For example, when a robot enters a building, its camera captures an image containing a brown door and a metal nameplate on it. The segmentation network accurately classifies the pixels in the door region as "door" and the pixels in the nameplate region as "number". The post-processing module identifies the door and the number as two independent instances. Subsequently, the system uses the center of the 3D bounding box of the "door" instance as its geometric anchor point and the text such as "501" recognized by OCR technology from the "number" instance as its semantic attribute, awaiting association with the map.

[0035] S12, associates and binds the identified semantic landmarks with their geometric locations in the environment map to form an independent semantic layer, and stores the category, confidence level, timestamp and physical size information of each semantic landmark; After identifying semantic instances in step S11, step S12 needs to transform these abstract identification results into structured information that can be queried and used in the environment map. The system maintains a semantic layer independent of the base occupancy raster map, which is essentially a spatial database or graph structure. The process of associating and binding the identified semantic landmarks with their geometric locations in the environment map in step S12 is as follows: For each identified semantic instance (e.g., a "door"), the system utilizes the robot's current precise pose (calculated via its self-localization module) and the depth information corresponding to the instance's pixel mask (from an RGB-D camera) or its corresponding portion in the point cloud. Through coordinate transformation, it calculates the 3D coordinates of the instance's geometric anchor point (e.g., the center point of the door) in the global map coordinate system. Subsequently, the system creates a semantic landmark object. Its core attributes include: a unique identifier generated by the system for the semantic instance, the instance's category, the calculated global coordinates for the instance, and the vertex coordinates of the instance's 3D bounding box. Finally, the semantic instance (the complete landmark object) including these attributes is stored as a semantic layer. Thus, the semantic layer and the geometric raster map form a two-layer structure through a shared coordinate system: the bottom layer contains geometric occupancy information, and the top layer contains semantic object information.

[0036] The confidence score of a semantic landmark is the average or median of the classification confidence scores of all pixels for that instance by the real-time semantic segmentation network. The timestamp is the current system time that recorded the landmark's creation or last verification. The physical dimensions are calculated from the length, width, and height of the 3D bounding box of the instance in three-dimensional space.

[0037] For example, when a robot identifies a "door" and a "door number," the system calculates the center coordinates of the "door" as (10.5, 3.2, 0.0) and creates a landmark D_001 (a unique identifier). Simultaneously, it calculates the coordinates of the "door number" as (10.5, 3.2, 2.1) (located above the door) and creates a landmark P_001. Both are stored in separate semantic layers. Subsequently, the path planner can query for "doors near coordinates (10.5, 3.2, 0.0)," while the task verification module can query for "the text content of door number P_001 associated with door D_001," thus achieving precise task interaction.

[0038] S13, establish a dynamic semantic layer to record the predicted trajectory areas of temporary obstacles and moving objects, as well as seasonally changing scene elements, and assign different weights and lifecycles to semantic information at different levels. The dynamic semantic layer is a data structure independent of the static semantic layer, specifically designed to manage temporary and dynamically changing environmental information. Its creation and management process is as follows: When the robot identifies a temporary object not recorded in the map, such as a "cardboard box" or a "construction cone," through real-time perception (such as semantic segmentation or obstacle detection), the system creates a temporary landmark for it in the dynamic layer. This landmark includes its location, bounding box, type, and an initial lifecycle value (such as 300 seconds).

[0039] For detected dynamic objects (such as pedestrians and vehicles), the system tracks them, recording not only their current position but also predicting their trajectory over the next few seconds based on a motion model (such as a constant velocity model). This predicted trajectory is recorded in a dynamic layer as a series of timestamped "risk areas," which can be accessed and avoided by the local path planner in real time.

[0040] Each element in the dynamic layer is associated with a countdown lifecycle. If no subsequent observations confirm the existence of the element (e.g., the "cardboard box" is no longer there when the robot passes by again), its lifecycle gradually decays to zero, and the element is then automatically removed from the dynamic layer, ensuring that the map does not become bloated and outdated due to transient information.

[0041] The system assigns different weights and lifecycles to semantic information at different levels to achieve resource optimization and intelligent emphasis on navigation strategies. In this embodiment, weights are used to measure the importance of landmarks in positioning and planning. For example, core structural landmarks, such as elevators, stairwells, and main unit doors, are given a high weight of 0.7-1, as they are the foundation for global positioning and critical path nodes; task-related landmarks, such as apartment doors and house numbers, are given a medium weight of 0.3-0.7, as they are crucial for task execution but can be updated; all elements in the dynamic layer are given a low weight of 0-0.3, as they are only used for real-time obstacle avoidance and do not affect global positioning. Core structural landmarks are permanently retained unless explicitly detected as changed, with a lifecycle set to several months or permanently. Semi-static landmarks, such as apartment doors, are allowed to change due to renovations, with a lifecycle set to several weeks or months. All elements in the dynamic layer are set according to their corresponding dynamic characteristics, and are filtered out when expired, with a lifecycle set to several minutes or seconds.

[0042] S14. Based on the robot's real-time positioning information, when the robot revisits a mapped area, it compares the current observation with the semantic information stored in the map, and incrementally updates, adds, or deletes the semantic landmarks that have changed.

[0043] For example, when the robot returns to the door of the mapped "Room 1001," its sensors detect that the original "brown wooden door" has been replaced by a "white metal door." The system updates the currently identified "door" instance with the D_1001 landmark recorded in the semantic layer, overwriting the old features with new visual features and increasing its confidence (because a more modern object has been observed). Simultaneously, the timestamp is updated. If a previously recorded dynamic obstacle, such as a bicycle, has disappeared, the system automatically removes it from the dynamic layer after it is no longer observed, based on its short lifespan. This ensures continuous synchronization between the semantic map and the real environment and preserves the historical relevance of landmarks (e.g., D_1001 always corresponds to Room 1001).

[0044] Step S2 fuses lidar point cloud features, visual odometry data, and inertial measurement unit data using a tightly coupled approach, specifically including the following steps: S21 minimizes the reprojection error of visual feature points, the point-to-line and / or line-to-surface distance error of laser point cloud features, and the relative pose error generated by IMU pre-integration in the same objective function. The purpose of the tightly coupled scheme provided in this embodiment is to deeply fuse the raw observation data from multiple sensors at the state estimation level.

[0045] For a map point observed in multiple frames (World coordinate system), which is in the first... The pixel observation value on the frame image is Reprojection error ,in, It is the transformation matrix from the robot body to the world coordinate system (state to be determined). yes The inverse matrix, It refers to the fixed external parameters from the camera to the body. It is a camera projection model. This application directly uses visual feature points as optimization variables and integrates pixel brightness information as constraints into the overall optimization. Compared with conventional loose coupling, which calculates a pose separately by vision, it avoids the linearization error of visual odometry itself and can effectively track fast motion by fusing IMU data, thus improving accuracy and robustness.

[0046] For the laser point in the current frame Find its corresponding edge line (from point) in the local map. , Defined, the direction vector is The point-to-line distance error is Find a plane in the local map (by the normal vector). On the surface (Definition), point-to-surface distance error is This application performs joint optimization of geometric distance errors (including point-to-line distance errors and point-to-surface distance errors) together with the error terms of vision and IMU in the state estimation problem at the same time. The robot's pose is corrected by the geometric constraints of laser, the photometric constraints of vision, and the dynamic constraints of IMU, rather than being calculated independently and then fused. This approach can more fundamentally utilize the complementary characteristics of different sensors and provide stronger constraints when geometric features are missing (such as in long corridors) or visual features are missing (such as in low-light environments), thereby achieving higher accuracy and robustness at the system level.

[0047] In this application, the relative pose error generated by IMU pre-integration Defined as the difference between the pre-integrated measurement and the theoretical value predicted by the state variables. The purpose of the IMU pre-integration in this application is to convert frequent IMU observations (preferably 100-500Hz) into a compact form that is easy to fuse with low-frequency visual or laser observations (preferably 10-30Hz).

[0048] This application uses state variables To describe the robot in The complete state at any given moment. Among them, This represents a 3×3 rotation matrix, indicating the orientation from the body coordinate system to the world coordinate system; This represents a 3×1 vector, indicating the velocity of the machine in the world coordinate system. It is a 3×1 vector representing the position of the origin of the body coordinate system in the world coordinate system; The vector is 3×1, representing the zero bias of the accelerometer; It is a 3×1 vector representing the zero bias of the gyroscope.

[0049] In this application, the pre-integral measurement value is within a time interval. Within this range, the relative motion increment obtained by integrating the raw IMU data is considered a "black box" measurement, which is only related to the IMU's readings and zero bias during this period, and not to the global state. Irrelevant They represent Time and time.

[0050] Pre-integral measurements include . Indicates that the robot is from arrive Relative rotation; Indicates that the robot is from arrive The relative velocity change; Indicates that the robot is from arrive The relative position change; This represents the difference between the current zero-bias estimate of acceleration and the zero-bias reference value of acceleration used during pre-integration; This represents the difference between the current zero bias estimate of the gyroscope and the zero bias reference value of the gyroscope used during pre-integration.

[0051] Pre-integration error The calculation method is expressed as follows:

[0052] In the above formula, This means mapping the rotation matrix to a rotation vector; , , , , State variables Zero bias Zero bias exist The value at time; , , , State variables Zero bias Zero bias exist The value at time; for Time's up The time interval between moments; for The reverse, State variables exist The value at time; for The transpose of .

[0053] Conventional loose coupling treats the IMU as an independent filter, failing to correct for its own error sources (such as zero bias). This application, however, directly incorporates the IMU's raw observations (through pre-integration) as constraints into the optimization, achieving deeper fusion by integrating incremental motion with... State decoupling at each time step allows IMU constraints to be rapidly calculated and iterated during optimization. This is achieved by pre-integrating the error. With reprojection error The point-to-line distance error is Point-to-surface distance error is By optimizing within the same objective function, the high-frequency dynamic information of the IMU compensates for the blurring of vision and laser during rapid movement, while the absolute observations of vision and laser effectively estimate and correct the zero bias of the IMU, suppressing its drift, and thus significantly improving navigation accuracy.

[0054] In this embodiment, the objective function used in step S21 is expressed as follows:

[0055] The state vector to be optimized includes the robot pose, velocity, IMU zero bias, and inverse depth or 3D coordinates of visual and laser feature points within the sliding window.

[0056] 、 、 、 Let represent the covariance matrices of pre-integration error, reprojection error, point-to-line distance error, and point-to-plane distance error, respectively. This represents the summation of the squares of the Mahalanobis norm over all point-to-surface distance errors; This represents the summation of the squares of the Mahalanobis norm over all point-to-line distance errors; This represents the summation of the squares of the Mahalanobis norm over all reprojection errors; This represents the summation of the squares of the Mahalanobis norm over all pre-integration errors; In this embodiment, the objective function is preferably solved using the Gauss-Newton method. Specifically, the Hessian matrix of its Jacobian matrix is ​​solved, and the state variables are continuously updated. This continues until the objective function converges to a local minimum.

[0057] S22, using the high-frequency data of the minimized IMU to compensate for point cloud distortion and image rolling shutter distortion caused by robot movement during lidar scanning and camera exposure; A mechanical LiDAR scan takes approximately 100ms. During this time, the robot's own movement distorts the point cloud, affecting the navigation efficiency and accuracy of indoor / outdoor fusion. To address this issue, this embodiment utilizes IMU high-frequency data for point cloud motion distortion compensation. The method is as follows: Record scan start time and end time Then, for each point in the minimized point cloud, based on the timestamp of when it was hit by the laser... (exist and (between), obtained from IMU data through interpolation. Time Robot Compared to Relative pose transformation at time t. Finally, use The coordinates of this point from The coordinate system at time is transformed to Under the same coordinates, the influence of robot motion during scanning is eliminated, and the point cloud after distortion is obtained.

[0058] Obtain this through interpolation. Time Robot Compared to Relative pose transformation at time t. The method is briefly described below: The IMU outputs angular velocity and acceleration at high frequencies (e.g., 100Hz). Through integration, a series of discrete, high-frequency robot poses can be obtained, forming a pose sequence. When it is necessary to acquire... When determining the pose at a given time, find the nearest neighbors in the pose sequence. The front and rear poses , For the translation part, directly in and Linear interpolation is performed between the translation vectors; for the rotation part, and The rotation matrix is ​​converted into a quaternion, and then spherical linear interpolation is performed in the quaternion space to obtain... Smooth rotation of time.

[0059] use The coordinates of this point from The coordinate system at time is transformed to The method under the same coordinates is briefly described below: get After calculating the translation vector and smooth rotation at each moment, we can then calculate... pose relative to the world coordinate system at any moment Then according to and This allows us to calculate the coordinate transformation matrix between the two. Finally, for... Multiplying by this coordinate transformation matrix yields the relative pose transformation. .

[0060] The method for image rolling shutter distortion compensation using IMU high-frequency data in this embodiment is as follows: For CMOS cameras using a rolling shutter, the image is exposed line by line, rather than a global instantaneous exposure. The compensation method is as follows: For each row in the image, based on its exposure timestamp, the relative pose transformation of that row's exposure time relative to the exposure time of the first (or middle) row of the image is also obtained by interpolation from the IMU data. Then use By combining camera-body external parameters, the 3D position of feature points in the camera coordinate system can be corrected, or the image can be corrected at the image level through affine transformation, thereby compensating for image distortion caused by the camera's own motion.

[0061] Interpolation The method and principle are the same as those for interpolation. This will not be elaborated further. The principle of correcting the 3D position of feature points in the camera coordinate system or correcting them at the image level through affine transformation is the same as that of using the coordinate transformation matrix mentioned above. Perform coordinate transformation to obtain relative pose transformation. The process will not be elaborated further.

[0062] S23, extract edge points and planar points from the laser point cloud as geometric features, and extract ORB feature points from the visual image and calculate their descriptors for subsequent matching and tracking; The method for extracting edge points and planar points from laser point clouds in this embodiment is as follows: In a point cloud frame, for each point Take several neighboring points in the vicinity, fit a local plane, and judge based on the structural features of the region. Is it an edge point or a planar point? For example, a robot scans a corridor and acquires a frame of laser point cloud. For each point in the point cloud... Calculate the curvature of the local region formed by the point and its five adjacent points (front and back), and set a curvature threshold. If the curvature is higher than the curvature threshold, then it is determined that... The point is an edge point; otherwise, it is considered a planar point.

[0063] In this embodiment, ORB feature points consist of FAST corner points and descriptors. For example, when a robot captures an image containing a window and a tree, the algorithm searches for points in the image whose brightness differs sufficiently from the surrounding pixels, such as the four corners of the window, gaps between leaves, and the outline of the tree trunk. These points are called FAST corner points. The descriptor is a mathematical vector that serves as the identifier for the feature point, used for matching between different images. For example, a pixel block is selected centered on the detected window corner point. Then, within this pixel block, 512 pairs of pixels are randomly selected according to a predefined, rotation-affected pattern. For each pair of pixels, their brightness is compared. If pixel A in the pair is brighter than pixel B, it is recorded as 1; otherwise, it is recorded as 0. Finally, the 512 0s or 1s combined into a 512-bit binary string is the descriptor of the window corner point.

[0064] In this embodiment, edge points and planar points describing the geometric contours of the environment are obtained from the laser point cloud in step S23. At the same time, ORB feature points and descriptors describing the texture of the environment with specific rotation invariance are obtained from the visual image, providing a rich and complementary data foundation for subsequent tightly coupled fusion localization.

[0065] S24, maintain a local sliding window map, and match the features in the local sliding window map of the multimodal features of the current frame; A local sliding window map is a dynamically updated environment model that retains only the most recent (e.g., 10) keyframes and their associated features. In this embodiment, the method for maintaining a local sliding window map is as follows: When the robot's pose changes beyond a certain distance or angle, a new keyframe is created; Store the following information from the newly created keyframe in the window: Laser keyframe: contains the 3D coordinates of edge points and planar points extracted from the point cloud of that frame in the world coordinate system; Visual keyframe: Contains the world coordinates and binary descriptors of the ORB feature points extracted from the image of that frame.

[0066] When the window is full, the oldest keyframe and its features are removed according to the "first-in, first-out" principle. For example, when a robot moves through a corridor, a keyframe is created every 1 meter it moves forward or rotates 15 degrees. The window always keeps the most up-to-date features of the environment within the last 15 meters of its trajectory, and earlier features are automatically discarded, which ensures the continuity of the environment representation while controlling computational complexity.

[0067] In this embodiment, the purpose of matching the multimodal features of the current frame with features in the local sliding window map (local map) is to find corresponding relationships between the features of the current frame in the local map, so as to establish optimization constraints. The matching includes laser feature matching and visual feature matching. The laser feature matching method is as follows: Based on the initial pose estimation, the edge points and planar points of the current frame are transformed to the world coordinate system. Then, for each edge point in the current frame, the five nearest neighbors are found among all edge points in the local map using methods such as KD-Tree, and a straight line is fitted using these five points to establish a point-line correspondence. For each planar point in the current frame, the five nearest neighbors are similarly found, and a plane is fitted to establish a point-plane correspondence.

[0068] The visual feature matching method is as follows: For each ORB feature point in the current frame, its binary descriptor is compared with the descriptors of all visual keyframes in the local map using Hamming distance. Then, the nearest neighbor and second nearest neighbor ratio test is used to find the two map feature points with the closest and second closest Hamming distances. If both the nearest and second nearest distances are less than a preset threshold (e.g., 0.8), the closest match is accepted, and a matching correspondence between ORB feature points and map feature points is established.

[0069] Through the above matching, each feature of the current frame has a corresponding matching feature in the local map. These matching pairs are the observation basis for constructing various error terms in step S21, driving tight coupling optimization to continuously correct the robot pose.

[0070] In step S2 of the indoor-outdoor fusion navigation method for the cleaning robot provided in this embodiment, the method for the closed-loop detection module to correct the cumulative error generated by long-term operation includes the following steps: A1, the closed-loop detection module continuously receives image frames from the visual sensor and point cloud frames from the LiDAR, and calculates the global appearance descriptor of the current frame. The calculation method is as follows: The purpose of computing a global appearance descriptor is to compress a frame of sensor data (image or laser point cloud) into a compact, rapidly comparable vector to characterize the macroscopic appearance of a scene. Global appearance descriptors include visual global descriptors and laser global descriptors. There are many existing methods for computing visual global descriptors. For example, one approach is to first input the current image into a pre-trained convolutional neural network to extract dense feature maps. Then, by learning a series of prototype anchors in the network layers, the soft-assigned weights and residuals of each local feature relative to all anchors are calculated. Finally, all residuals are aggregated to generate a fixed-length vector representing the visual global descriptor.

[0071] The method for calculating the global laser descriptor is as follows: The current frame's laser point cloud is divided into multiple sector-shaped cylinders centered on the sensor, along both the horizontal azimuth and radial distance axes. Then, within each cylinder, the Z-coordinate value (or the statistical height value of the point) of the highest point is recorded. Finally, these values ​​are arranged into a two-dimensional matrix, which constitutes the core descriptor.

[0072] A2, match the global appearance descriptor of the current frame with the historical key frame descriptors stored in the maintained local sliding window map, filter out several similar closed-loop candidate frames, and obtain the appearance similarity score between the current frame and each candidate frame. It is important to emphasize that the loop closure candidate frames are selected from the local sliding window map fused and maintained through the tightly coupled method in steps S21-S24. The loop closure candidate frames themselves already possess the characteristics of minimizing pre-integration error, reprojection error, point-to-line distance error, and point-to-surface distance error. Furthermore, they compensate for point cloud distortion and image rolling shutter distortion caused by the robot's motion during LiDAR scanning and camera exposure. Therefore, in step A2, the global appearance associated with the global appearance descriptor corresponding to the loop closure candidate frames selected through descriptor matching better reflects the effectiveness of the scene features currently in which the robot is located, thereby improving the effectiveness of loop closure detection based on the current frame.

[0073] In this embodiment, the purpose of step A2 is to quickly find a small number of candidate frames from a massive amount of historical keyframes that are similar in appearance to the current frame and can be effectively used as loop closure detection. The specific method is as follows: Before the system runs, a "visual dictionary" is trained using a large, navigation-independent image dataset (such as cityscapes). This dictionary consists of hundreds or thousands of "visual words," each representing a common local image pattern, such as wheel textures, window edges, or leaf patterns. For each historical keyframe in the map, the system extracts its ORB features and then represents them as a sparse "bag-of-words vector" (equivalent to a descriptor) based on the frequency of these features in the visual dictionary. For example, a keyframe taken in a parking lot might have a bag-of-words vector with high values ​​for visual words like "wheels" or "concrete surface." When a new current frame arrives, the system also calculates a bag-of-words vector for it. The system compares the similarity between the bag-of-words vector of the current frame and the bag-of-words vectors of all historical keyframes. The top K historical keyframes with the highest similarity are initially selected as candidate frames for loop closure.

[0074] The method for calculating the appearance similarity score between the current frame and the closed-loop candidate frame is as follows: The current frame and each historical keyframe are represented as a sparse vector (bag-of-words vector), which describes the frequency of different "visual words" in the salience. The similarity score between the current frame and the closed-loop candidate frame is preferably the reciprocal of the distance (Manhattan distance or Euclidean distance) between the bag-of-words vector of the current frame and the bag-of-words vector of the candidate frame; the closer the distance, the higher the score.

[0075] A3 performs geometric consistency verification on each closed-loop candidate frame, checks the reprojection error of matching feature points and the point cloud registration error, and calculates the geometric verification score between the current frame and the closed-loop candidate frame. The purpose of performing geometric consistency verification on candidate frames is to eliminate false matches caused by environmental repetition (such as two similar gates). The verification method is as follows: For visual candidate frames, the fundamental matrix is ​​first solved iteratively using ORB feature points between the current frame and the visual candidate frames via the RANSAC algorithm. Then, the number of inliers conforming to this fundamental matrix is ​​calculated, and the inlier rate (number of inliers / total number of matches) is calculated. A higher inlier rate indicates that the viewpoints of the two frames conform to geometric constraints, and the better the geometric consistency. For example, although two doors may look similar, if the matching feature points cannot be fitted with a reasonable fundamental matrix, the inlier rate will be extremely low, thus excluding the visual candidate frame. The method of solving the fundamental matrix using the RANSAC algorithm is an existing method and will not be described in detail.

[0076] For laser candidate frames, iterative nearest-neighbor coarse matching is first performed using the laser point clouds of the current frame and the candidate frame. Then, the root mean square error after registration is calculated. The smaller the error, the higher the structural overlap and geometric consistency of the two point clouds in 3D space. For example, two intersections with different structures may be misselected by the appearance descriptor, but iterative nearest-neighbor coarse matching will produce a huge error due to the inability to properly align the point clouds, thus excluding the candidate frame. In this embodiment, the method of using iterative nearest-neighbor coarse matching is as follows: first, find the nearest neighbor point in the target point cloud (candidate frame) for each point in the current frame point cloud; then calculate a rigid body transformation that minimizes the sum of the distances of all matching point pairs to align the two points in 3D space, thereby estimating their relative pose transformation.

[0077] The geometric verification score between the current frame and the closed-loop candidate frame includes the geometric verification scores between the current frame and the visual candidate frame, and between the current frame and the laser candidate frame. The geometric verification score between the current frame and the visual candidate frame is calculated as follows: After solving the geometric transformation (such as the fundamental matrix) relationship between two frames using the Random Sampling Algorithm (RANSAC), the inlier rate is statistically analyzed as the geometric verification score.

[0078] The geometric verification score between the current frame and the laser candidate frame is calculated as follows: Coarse registration is performed based on the iterative nearest point algorithm, and then the reciprocal of the root mean square error after registration is calculated as the geometric verification score. The smaller the root mean square error, the higher the score.

[0079] A4. Determine whether the appearance similarity score and geometric verification score of the current frame and the closed-loop candidate frame are both greater than the corresponding adaptive score threshold. If so, proceed to step A5; If not, then remove the candidate frame for closing the loop. A5 sends the closed-loop constraints to the backend optimizer for global pose graph optimization and correction of accumulated errors.

[0080] In this embodiment, the closed-loop constraint refers to the relative pose transformation relationship between the current frame and a certain historical keyframe when the closed-loop detection confirms that the current frame and a certain historical keyframe come from the same physical location. This relationship is determined by the transformation matrix. This indicates the correct relative positional relationship from the current frame to the loop closure candidate frame. For example, if the current frame and the loop closure frame should coincide, then... It is close to the identity matrix.

[0081] In the pose graph, nodes represent the robot's historical poses, and edges represent constraints between poses (including odometry edges and loop closure edges). When adding loop closure constraints, the optimizer adds a new edge between the current frame node and the loop closure frame node, with the constraint being... Then, the optimizer minimizes the error function of all edges (as in a least squares problem) and adjusts the poses of all nodes in the graph to satisfy the overall constraints.

[0082] In the technical application scenario of this application, the accumulated error originates from odometry drift, causing the robot's estimated trajectory to deviate from the true path. Closed-loop constraints provide "anchor points," forcing the optimizer to align the current pose with historical accurate poses. Through optimization, all poses from the closed-loop frame to the current frame are adjusted, and the trajectory is pulled back to the correct shape, thereby correcting the accumulated error.

[0083] In step S3, the method for understanding the planning path based on semantics and dynamic environment includes the following steps: S31, The path planner receives a semantic task instruction from the scheduling system and parses it into a specific coordinate sequence in the multi-level semantic map; Semantic task instructions are as follows: Semantic task instructions are usually not direct coordinates, but natural language or structured data instructions issued by the scheduling system that conform to business logic, such as cleaning up kitchen waste at the door of Unit 2, Building 8, Room 1001, 10th Floor, Xingfu Community.

[0084] The method for parsing a multi-level semantic map into a specific coordinate sequence is as follows: After receiving the above instructions, the path planner performs an "addressing" query in its maintained multi-level semantic map, resolving the abstract semantic address into a series of specific spatial coordinates. First, the planner uses "Happy Community" as the entry point and finds the corresponding map node in the top-level area of ​​the semantic map. Then, it searches for the next level of semantic labels in the map sequentially. For example, under the "Happy Community" node, it finds the child node "Building 8"; under the "Building 8" node, it finds the child node "Unit 2"; under the "Unit 2" node, it finds the child nodes "Elevator" or "Unit Door". Knowing it needs to go to the 10th floor, the planner directs the robot to the elevator and binds "call the elevator and go to the 10th floor" as an action command to the path point. Upon reaching the 10th floor, the planner searches the floor map for a door object with the semantic label "Room 1001". The system obtains the geometric anchor point, i.e., the center coordinates of the door, from the attributes of this door object. Finally, the planner outputs a coordinate sequence from the starting point (the robot's current position) to the ending point (the door of room 1001), expressed as: robot's current position coordinates - coordinates of the door of unit 2, building 8 - coordinates inside the elevator - coordinates of the elevator entrance on the 10th floor - coordinates of the door of room 1001.

[0085] S32, based on the parsed coordinate sequence, plans the initial global path on the coarse-resolution raster map; In this embodiment, the environment is first modeled as a grid, and each grid is assigned a "free", "occupied", or "unknown" state. By evaluating the cost from the starting point to the ending point in the coordinate sequence (e.g., minimizing the cost of the path already traveled plus the estimated cost to the ending point), a collision-free path with the minimum cost (e.g., the shortest) is searched as the initial global path. The planning process is constrained by a semantic rule base. For example, the rule "prohibited areas include green belts" will make the path cost across green belts infinite, thus forcing the initial global path search algorithm to detour.

[0086] S33, guided by the initial global path, performs local path planning, while integrating real-time laser data and dynamic semantic layer information to avoid static and dynamic obstacles.

[0087] In this embodiment, the method for local path planning guided by the initial global path is as follows: First, sample possible velocity combinations (including linear and angular velocities) near the robot's current velocity for the next moment, and simulate multiple short-duration motion trajectories. For example, a pre-trained neural network model can be used to predict and simulate multiple short-duration (e.g., 3 seconds) motion trajectories.

[0088] Then, each motion trajectory is scored using an evaluation function that incorporates the following information: Orientation: Measures the degree of alignment between the trajectory endpoint and the local target (the next waypoint on the initial global path); Idle state: Drives the robot to move towards an open area; Speed: Encourage faster speeds to improve efficiency; Real-time laser data: As input to the obstacle avoidance algorithm, it calculates the distance to the nearest obstacle on the trajectory and generates obstacle cost. The closer the distance, the higher the cost and the lower the trajectory score. Dynamic semantic layer information includes temporary static obstacles (such as piled-up debris) and dynamic obstacle prediction areas (such as the future walking position of a pedestrian). For temporary static obstacles, a high obstacle cost is incurred, similar to permanent obstacles detected by laser. For dynamic obstacle prediction areas, it predicts whether the movement trajectory will enter these "future danger zones." If so, an extremely high cost is imposed to achieve proactive preventative avoidance.

[0089] Finally, the evaluation function integrates the above index values, outputs the final score, and uses the trajectory with the highest score as the local path planned for the next moment. Through the dynamic cyclic calculations in steps S31-S33, the robot can safely and smoothly avoid all real-time perceived static and dynamic obstacles in the initial global path.

[0090] In step S3, the method of fusing closed-loop detection results and real-time tracking and online correction of the actual walking path includes the following steps: B1 uses a model predictive controller to calculate the robot's expected linear velocity and angular velocity based on the planned path points; In this embodiment, the model predictive controller, based on the robot's current state (velocity, orientation, and position) and a candidate control sequence (such as an initial global path or a local path), uses a built-in robot kinematics model to predict multiple possible trajectories the robot will take within a future period (prediction time domain). The prediction method can employ the evaluation function described above, and will not be elaborated further.

[0091] The goal of the model predictive controller is to find an optimal control sequence that minimizes the overall cost of the corresponding predicted trajectory. In this embodiment, the cost function includes: Path tracking deviation: Calculates the sum of the squares of the lateral and orientation deviations between a point on the predicted trajectory and the planned path. Lateral deviation is the vertical distance between the robot's current position and the nearest point on the planned path; orientation deviation is the angle between the robot's current orientation and the tangent direction at the nearest point on the planned path. Changes in control input: Punishing drastic changes in control commands (such as sharp turns or rapid acceleration). Constraint violation: Satisfying the robot's dynamic constraints (such as maximum speed, maximum acceleration) and obstacle avoidance constraints.

[0092] Solving the cost function yields the optimal future control sequence. However, the model predictive controller only executes the first control command in the candidate control sequence (the robot's current walking speed and angular velocity). In the next control cycle, it collects the new actual state and repeats the entire process, thus continuously iterating and optimizing forward.

[0093] B2, calculates the lateral and orientation deviations between the robot's actual pose and the planned path in real time; B3, dynamically adjusts the prediction time domain and control time domain parameters of the path tracking controller based on the path curvature, robot speed and deviation magnitude; Path curvature reflects the degree of curvature of the path; the greater the curvature, the sharper the turn. The model predictive controller calculates the instantaneous curvature using geometric methods by querying three consecutive closely adjacent path points on the planned path ahead. Preferably, the curvature is the reciprocal of the radius of the circumcircle of the triangle formed by these three path points.

[0094] The pose correction provided by closed-loop detection is a key quality assurance measure reflecting the overall positioning uncertainty of the system. When the correction is small, it indicates accurate positioning and high system reliability. In this case, the model predictive controller can use a longer prediction time domain, allowing it to "see" further and plan a smoother, more energy-efficient trajectory. When the correction is large, it indicates that the positioning may experience significant jumps or errors, and the environment is complex. In this case, the model predictive controller is instructed to shorten both the prediction and control time domains, forcing it to become more "short-sighted" and "cautious," focusing on the most recent and reliable path tracking, prioritizing control stability and safety.

[0095] B4. When the lateral deviation or orientation deviation calculated in step B2 exceeds the preset threshold, local replanning is triggered and the process jumps to step S33 to generate a new path from the current position to the next sub-target point. The local path planning process has been explained above and will not be repeated here.

[0096] B5 introduces the pose correction value confirmed by closed-loop detection as feedback into the path tracking controller to smoothly compensate for pose jumps in subsequent control cycles. The calculation method of the pose correction value in closed-loop detection, and the method of how the path tracking controller adjusts the prediction time domain and control time domain parameters, have been explained above and will not be repeated here.

[0097] In step S3, the actual walking path is tracked and corrected online in real time based on the closed-loop constraints of step A5 (when dynamically adjusting the prediction time domain and control time domain parameters of the path tracking controller in step B3, the pose correction amount confirmed by the closed-loop detection in steps A1-A5 is introduced), thereby further improving the accuracy of the correction of the actual walking path.

[0098] In step S4 of the indoor-outdoor fusion navigation method for the cleaning robot provided in this embodiment, a navigation switching strategy for the indoor-outdoor transition area is defined, specifically including: Indoor / outdoor switching zones are pre-marked in a multi-level semantic map, and specific positioning and navigation strategies are defined for these zones. When the robot approaches the switching area, the positioning system increases the search weight of landmarks unique to the switching area in advance to improve the positioning success rate and efficiency at the switching moment. When passing through unit doors and elevators, the robot communicates with IoT devices to automatically open doors, call elevators, and select floors. In outdoor areas where GPS signals are available, GPS positioning results are introduced as weak constraints into the fusion positioning system to suppress long-term drift in open areas. In response to the unique enclosed environment of an elevator, the robot switches to a dead reckoning mode based on IMU and wheeled odometers after entering the elevator, and achieves precise floor positioning by recognizing floor buttons inside the elevator or receiving floor information from the elevator control system.

[0099] In step S4, the method for dynamically adjusting the contribution weights of different sensors in the fusion localization based on the credibility of environmental features specifically includes: Set an initial confidence weight for LiDAR, vision, and IMU respectively; Real-time evaluation of the quality of each sensor. For LiDAR, evaluation metrics include point cloud density and scene geometry richness; for vision, evaluation metrics include image brightness, texture richness, and number of tracked feature points. When the robot enters environments with sparse visual features or repetitive geometric features, such as long corridors or white walls, it automatically reduces the weight of visual sensors and increases the weight of LiDAR and odometry. When drastic changes in ambient lighting cause a severe deterioration in image quality, the weight of the visual sensor is significantly reduced, or it may even be temporarily excluded from the fusion system.

[0100] Finally, the adjusted weights are dynamically applied to the multi-sensor state estimation process using filters (such as Kalman filters).

[0101] In step S5, the path integral error is the cumulative error generated when integrating the robot's displacement and rotation angle to calculate the pose, including the cumulative error generated when using positioning methods such as wheel odometry, visual odometry, or inertial navigation systems to calculate the pose.

[0102] In step S4, the dynamic adjustment of the contribution weights of each sensor in the fusion positioning is applied to the original observation data layer or the tightly coupled optimization layer of step S2. By selecting the most reliable sensor, the rate of error injection is reduced at the source. The aim is to generate an optimal positioning result with minimal drift before navigation deviations or severe deviations occur. Although step S4 can significantly reduce the growth rate of path integral error, the error will still accumulate over time. This application addresses this issue through step S5.

[0103] The error state Kalman filter established in step S5 operates at the pose estimation layer. It takes the output of the front-end fusion system (including path integral error) as an observation and combines it with another absolute prediction (closed-loop correction) to estimate the extent of the cumulative error and make corrections.

[0104] In this embodiment, the error state Kalman filter does not directly estimate the robot's absolute pose, but rather estimates the error between the true pose and the predicted pose, which is defined as the "nominal state".

[0105] The working process of the error-state Kalman filter is as follows: Computational robots in the current nominal state at time Compared to the actual state Error status nominal state The nominal state is calculated based on the tightly coupled feature data of step S2, and carries the cumulative error involved in step S2 or the correction result after correcting the cumulative error. The tightly coupled process of step S2 applies the contribution weight dynamic adjustment result of step S4. The nominal state includes pose, velocity, angular velocity and position obtained by integration. The specific calculation of the nominal state using existing methods is not within the scope of the claims of this application and will not be specifically explained. Error status This represents the difference between the actual state and the nominal state, including attitude error, position error, and velocity error.

[0106] First, predict the current situation. The next moment nominal state at time Simultaneously predict error state The distribution (such as directly taking the value 0 or...) Error state at time Alternatively, an error state fitting function can be used to fit the error state at each time step, and then the solution can be obtained. Error state at any given moment.

[0107] The filter starts working when there is an observation input. In this embodiment, the observation input of the filter includes: Path integration error. Path integration error is not a direct sensor reading, but rather obtained indirectly through zero-velocity correction or motion constraints. For example, when a robot stops briefly, its true velocity should be 0. At this time, the non-zero velocity value in the nominal state is the observed velocity error. This observation tells the filter that the integration process has caused velocity drift.

[0108] Closed-loop pose correction. Once the closed-loop detection is confirmed, high-precision absolute pose observation is obtained. . The calculation principle is the same as the relative pose transformation matrix mentioned above. They are the same, the difference lies in the choice of reference point, therefore... The calculation process will not be elaborated further.

[0109] The filter will change the current nominal state. Transformed to the observation space, and By comparison, the observed residuals are obtained. .

[0110]

[0111] This represents the observation space transformation function.

[0112] Observation residuals The cumulative error of the nominal state is revealed.

[0113] The filter, based on the uncertainty (covariance) of the path integral error and the closed-loop pose correction, uses the Kalman gain formula to compare the predicted error state with the observed error (residual). ) are fused together to obtain Time-optimal error state estimation .

[0114] Then, the filter will Injected into nominal state In China, Revised to ( (as corrected) The optimal robot pose at any given time.

[0115] In step S6, the conditions for enabling the degradation navigation strategy include: The point cloud density of the lidar continues to be below the threshold for an extended period of time. The number of stable feature points extracted by the visual sensor remains below the threshold for an extended period of time. The data quality assessment scores of the main external sensors (such as LiDAR and vision cameras) are all below the preset corresponding thresholds.

[0116] If any one or more of the above three conditions are met, a degradation navigation strategy based on historical path and multimodal fusion will be activated, specifically including the following steps: S61, when any one or more of the above three conditions are met, the system immediately triggers a degraded navigation mode to control the robot to decelerate; S62, switch to dead reckoning mode based on IMU and wheel odometry, such as navigation using data collected by inertial measurement unit alone, or navigation using data collected by wheel odometry alone; S63, based on a multi-level semantic map and the most recent historical walking path, performs probabilistic path tracking, that is, attempts to move slowly along the historical path or a known safe passage. S64, attempts to periodically activate potentially recoverable sensors for repositioning; S65: If the robot cannot recover or leave the degraded area within the predetermined time, the robot will stop and send a distress signal to the superior dispatch system.

[0117] In step S7, the method for the cleaning robot to perform localization verification and pose fine-tuning by identifying specific semantic targets specifically includes the following steps: S71: When the robot arrives near the navigation target point, it actively adjusts the posture and focus of the vision sensor to focus on recognizing the house number or a specific trash can. S72 compares the identified house number information with the target address in the task instruction to confirm whether the task execution location is correct; S73, after confirming the correct position, uses the current accurate visual positioning results to make a fine adjustment to the robot's pose to ensure that the robot's garbage disposal port is precisely aligned with the garbage interaction point at the resident's door. The alignment method can use existing image matching algorithms, without going into specifics. S74. If the expected target (such as a house number) is not identified or the identification result does not match, the robot is controlled to move within a small range to re-attempt identification from multiple perspectives or to relocalize by combining laser point cloud features. S75 stores the location of a successfully executed task as a new keyframe and semantic delivery in the environment map or multi-level semantic map for reference in subsequent navigation tasks.

[0118] In step S7, after determining that the robot has not yet reached the navigation target point at the current moment, the error state Kalman filter feeds back the observation output to the closed-loop detection module. The error state Kalman filter utilizes... Will Revised to The feedback is sent to the closed-loop detection module. After real-time pose correction by the filter in steps S51-S54, a robot pose sequence with higher global consistency is obtained, which serves as the input to the closed-loop detection module.

[0119] The loop closure detection module needs to compare the current frame with historical keyframes. If the robot pose used to generate these historical keyframes itself has a large drift (i.e., accumulated error), then loop closure detection will be very difficult, and may even produce incorrect loops. The error state Kalman filter, by fusing the IMU, wheel velocimeter, and possibly other other sensors (tightly coupled), outputs a pose sequence with highly accurate local relative motion and significantly suppressed short-term drift. This provides a locally smooth and accurate trajectory for loop closure detection, greatly improving the success rate and accuracy of loop closure detection.

[0120] When the loop closure detection module successfully finds an effective loop closure based on the high-quality pose provided by the filter (e.g., confirming the current pose), With historical position After identifying the same point, perform the following operations to globally optimize the robot's pose: Construct a pose graph. Nodes in the pose graph represent key poses in the robot's history. Nodes in the pose graph include odometry edges and closed-loop edges. Odometry edges represent relative transformation constraints between adjacent poses provided by filters; closed-loop edges connect... and The constraints are ideally transformed into the identity matrix (at this time) and (For the same point).

[0121] Then, global pose optimization is performed using closed-loop constraints. Due to accumulated errors, the edges connected by odometry are... arrive The path is inconsistent with the path implied by the closed-loop edge (which should coincide), which means in the pose graph, this path cannot be "closed". In this case, the pose graph optimizer performs global pose optimization by adjusting the poses of all nodes in the graph, aiming to minimize the overall error of all constraints (odometry edges and closed-loop edges). Specifically, the optimizer will... arrive All pose nodes between them are "stretched" or "compressed", and the entire trajectory is "pulled back" to the correct position (the odometer edge coincides with the closed loop edge), thereby correcting all the errors accumulated during this period at once.

[0122] This application also provides an indoor-outdoor integrated navigation system for a waste collection robot, including: The multi-level semantic map construction and update module is used to construct and update a multi-level semantic map that includes semantic landmarks from visual sensors, geometric structure information from LiDAR, and dynamic semantic images. The tightly coupled module is used to fuse lidar point cloud features, visual odometry data, and inertial measurement unit data in a tightly coupled manner. A closed-loop detection module, connected to the tightly coupled module, is used to correct the accumulated error generated during long-term operation using the observation output fed back by the tightly coupled feature data and / or the error state Kalman filter. The planned path planning module connects the closed-loop detection module and the multi-level semantic map construction and update module. It is used to plan the planned path based on semantic and dynamic environment understanding, and to track and correct the actual walking path in real time by fusing the closed-loop detection results. The contribution weight dynamic adjustment module is connected to the tightly coupled module, defines the navigation switching strategy for indoor and outdoor transition areas, and dynamically adjusts the contribution weight of different sensors in fusion positioning based on the credibility of environmental features. The filter construction module connects the closed-loop detection module and the tightly coupled module, and is used to construct an error-state Kalman filter, and uses the path integral error and the closed-loop correction as the observation input of the filter; The first judgment module is used to determine whether the sensor signal meets the conditions for enabling the degradation navigation strategy. If so, then a degraded navigation strategy based on historical path and multimodal fusion will be enabled; If not, output the second judgment instruction; The second judgment module, connected to the first judgment module, is used to determine whether the navigation target point has been reached. If so, the instruction pose optimizer performs localization verification and pose fine-tuning by identifying specific semantic targets, thereby completing the action interaction with the target task; If not, the observed output of the error state Kalman filter is fed back to the closed-loop detection module. In summary, the indoor-outdoor fusion navigation method for waste collection robots provided in this application establishes a feedback mechanism to bidirectionally correct accumulated errors from long-term operation by introducing closed-loop detection and error state Kalman filtering, ensuring global navigation accuracy. By constructing a multi-level semantic map and fusing laser, vision, and inertial data in a tightly coupled manner, high-precision and robust localization and navigation of the waste collection robot in complex community environments are achieved. Real-time tracking and online correction of the actual walking path, based on the closed-loop constraints in step A5, enables the system to automatically adjust the prediction and control time domains of the model predictive controller, thereby planning a smoother, more energy-efficient, more stable, and safer walking path. After real-time pose correction by the filter, a robot pose sequence with higher global consistency is obtained, which serves as the input to the closed-loop detection module, making subsequent global pose optimization more effective and accurate.

[0123] The following describes the robot cooperative scheduling method based on fusion navigation provided in this application. The cooperative scheduling method includes the following steps: L1, construct a dynamic scheduling view based on the real-time pose and task status of multiple robots. The dynamic scheduling view integrates the closed-loop constraint information from the closed-loop detection module of each robot. L2, based on the observation output of the dynamic scheduling view and the error state Kalman filter, performs robot health status assessment and task execution reliability prediction. L3 dynamically assigns cleaning tasks to each robot based on the health status assessment results and generates an initial walking path; L4, during task execution, tracks and corrects the robot's actual walking path in real time and online based on the closed-loop detection results and path tracking status; L5, the closed-loop detection module performs global optimization of the robot's pose based on the observation output fed back by the error state Kalman filter, and synchronizes the optimized pose to the central scheduling platform to update the dynamic scheduling view, forming a closed-loop feedback from single-machine localization to collaborative scheduling.

[0124] In step L1, the method for constructing a dynamic scheduling view based on the real-time pose and task status of multiple robots includes the following steps: L11, the central dispatch platform continuously receives real-time pose, speed and task status data of all online robots; Suppose that in the "Happy Community," a central dispatch platform is managing three waste collection robots: Robot-01, Robot-02, and Robot-03. Robot-01 is performing the task of "collecting kitchen waste from apartment 1001, unit 2, building 8." Robot-01 sends a data packet to the central dispatch platform several times per second via its onboard computer. The data packet content includes: A unique identifier for the robot, such as robot_id: "01"; Timestamp, such as 2025-10-01-09:00:04; Coordinate position: Coordinates in the world coordinate system (e.g., (x: 105.3, y: 87.1, z: 0.0)), provided by the indoor and outdoor fusion navigation system of the cleaning robot mentioned above, which is positioning data containing uncorrected drift or positioning data after drift correction by the closed-loop detection module; velocity vector; Battery level; Task number, such as "Walk to Building 8, Room 1001"; Task completion rate, such as 65%.

[0125] After receiving the data sent by Robot-01, the central dispatch platform displays Robot-01 as an icon moving at coordinates (105.3, 87.1, 0.0) on the digital map.

[0126] L12 receives and parses the newly generated closed-loop constraints from the closed-loop detection modules in each robot. The closed-loop constraints characterize the correction amount and position confidence of the robot's positioning error. For example, at the current moment, when Robot-01 drives to the entrance of Building 8, its closed-loop detection module confirms that the current scene matches the historical keyframe of "Building 8 Unit 2 Door" stored in the map (environment map or local sliding window map) through the visual bag-of-words model and geometric verification, and successfully triggers the closed loop.

[0127] The closed-loop detection module calculates a closed-loop constraint: it finds a discrepancy between the current position calculated based on the odometer (defined as point B) and the actual position of "Building 8, Unit 2 Door" recorded on the map (point A). , This represents the pose angle deviation. Robot-01 encapsulates this closed-loop constraint into a message and sends it to the central scheduling platform. An example of the message content is as follows: A unique identifier for the robot, such as robot_id: "01"; Pose correction matrix: This is used to characterize the correction amount in closed-loop constraints. The results are obtained through the closed-loop candidate frame screening process described in steps A1-A5 above, and will not be repeated here.

[0128] Corrected precise pose: (104.5, 87.4, 0.0); Pose covariance: The smaller the value, the higher the confidence level of the correction.

[0129] The platform analyzed the message and determined that Robot-01's exact location was (104.5, 87.4, 0.0), and that the message was highly reliable.

[0130] L13 integrates the analytical closed-loop constraints with the robot pose, marking high-confidence robot precise location regions and potential positioning drift regions on the map of the dynamic scheduling view. For example, positions with confidence levels exceeding the confidence threshold are identified as robot precise positioning positions, while positions below the confidence threshold are identified as potential positioning drift regions.

[0131] In this application, the closed-loop constraints used for single-machine positioning are transformed into a system-level scheduling reliability map, so that the scheduling decision is based on the global accuracy of the robot's position rather than simply GPS or odometry coordinates, thereby improving the spatial perception reliability of multi-machine collaboration from the source.

[0132] In step L2, the method for assessing the robot's health status and predicting the reliability of task execution includes the following steps: L21 extracts the observed output of the error state Kalman filter, including the pose error covariance and the sensor zero bias estimate; It should be noted here that the pose error covariance is calculated by the filter based on the path integral error and the closed-loop pose correction. The covariance is calculated based on the uncertainty.

[0133] Define the pose error covariance matrix as follows: First, the current motion model (e.g., IMU or odometry model) and inherent noise (e.g., accelerometer noise, gyroscope noise) are predicted. The next moment Covariance of error state at time step When there is observation (such as When the input is received, the filter calculates the uncertainty of the observation, that is, it calculates the observation noise covariance matrix. Subsequently, the filter calculates the Kalman gain. Finally utilize To update or reduce the covariance of the error state: , express The covariance of the error state at time 1 to time 2.

[0134] The sensor zero bias estimate includes the components of the optimal error state estimate. The error includes one or more of the following: position error, attitude error, velocity error, accelerometer bias error, and gyroscope bias error. In this embodiment, it is preferred to use... This represents the sensor zero-bias estimate, simplifying the subsequent collaborative scheduling calculation process.

[0135] L22, based on the magnitude of the pose error covariance, calculates the uncertainty score of the robot's current localization. The calculation method is as follows: First, from the complete pose error covariance matrix In the process, the position error is extracted. The corresponding 3×3 submatrix For example, the robot's position error covariance submatrix After one closed-loop correction, it is:

[0136] Then, a measure of location uncertainty is calculated. In this embodiment, the location uncertainty is extracted. The largest eigenvalue in (e.g., 0.01 in the example above) as a measure of location uncertainty.

[0137] Finally, the metrics are mapped to human-defined rating values ​​using a pre-built mapping function. For example, when If the score exceeds the preset threshold, the score is 60.

[0138] Step L22 transforms the abstract, multidimensional mathematical statistics (covariance matrix) of the underlying filter into a top-level, easy-to-understand, and easy-to-use business decision indicator. This enables schedulers or scheduling systems without state estimation expertise to intuitively and quantitatively assess the working state of each robot, which is a key step in achieving intelligent and predictive scheduling.

[0139] L23, based on the changing trend of the sensor zero bias estimate, predicts the positioning reliability decay curve of the robot over a period of time in the future; In this embodiment, the method for plotting the positioning reliability decay curve is as follows: First, collect historical zero-bias data for the same robot. The system continuously records the optimal error state estimate. The sequence of changes in the zero-bias components of the sensors carried in the device (including accelerometer zero-bias and gyroscope zero-bias) over time. To simplify the calculation, this embodiment directly records... A sequence of changes over time.

[0140] Then, for the most recent period of time (e.g., distance from the current) Perform linear regression or sliding window averaging on the change sequence (within 60 seconds prior to the time point) to calculate its rate of change (slope). Then, the attenuation coefficient is calculated based on the rate of change. , For empirical coefficients, and according to Predicted reliability score , The change is the predicted positioning reliability decay curve.

[0141] For example, suppose the current state of robot Robot-01 is: Gyroscope zero bias In the past 60 seconds, It increases at a rate of 0.0001 rad / s² (slope). The current reliability score R0 = 85 points (uncertainty score calculated in step L22).

[0142] Then, calculate the attenuation coefficient. , For example, the slope representing the zero bias of the accelerometer can be 0.005. These are empirical coefficients. Assume... ,but .

[0143] Finally, based on the above reliability score The calculation formula is used to obtain the result when When the value is 60 seconds, divide, when When the value is 120 seconds, Based on this, it can be estimated that the robot's positioning reliability will drop below the threshold of 60 points in approximately 70 seconds. Therefore, this embodiment analyzes the trend of zero bias change and establishes a mathematical model of the slow drift and reliability decay within the sensor, achieving a forward-looking prediction of the robot's walking status from "currently healthy" to "when it will become unhealthy in the future." Based on this forward-looking prediction, the mobilization platform promptly transfers the robot's task to a more reliable robot before the reliability drops below the threshold, avoiding the use of a robot whose reliability is about to fail in critical tasks, and can also plan the robot's path to the maintenance point in advance.

[0144] The health status assessment results used in step L3 include the current reliability score. Reliability attenuation coefficient The final health status is determined by any one or more of the following: sensor zero-bias stability, historical task success rate, etc. In this embodiment, the final health status is categorized as follows: and The person is in excellent health. and The person is in good health. 40 or Health status requires attention; 40 or The health status is at risk.

[0145] The strategy for dynamically allocating cleaning tasks to each robot in this embodiment includes: 1. Categorize tasks according to priority High-priority tasks include: emergency waste overflow handling and VIP user services; Precision and sensitive tasks include: working in confined spaces and locations requiring precise docking; Routine task: Garbage collection from ordinary households; Low priority tasks: area patrol, map update.

[0146] 2. According to the allocation rules of health status For example, if the robot's health status is excellent, it can be assigned high-priority and precision-sensitive tasks, and is also allowed to perform long-distance tasks (e.g., greater than 2km); when the robot's health status is good, it can be assigned regular and precision-sensitive tasks, and is allowed to perform medium-distance tasks (e.g., 1-2km); when the robot's health status is attentive, it can be assigned regular and low-priority tasks, with the task distance limited to less than 1km; when the robot's health status is at risk, the robot's cleaning tasks are suspended, and it is dispatched to a maintenance station.

[0147] 3. Consider load balancing Even if a robot is in excellent health, the scheduling platform does not allow any single machine to be overloaded. By considering the current task queue length of each robot, path conflicts caused by task allocation are avoided.

[0148] In step L3 of this embodiment, the method for generating an initial walking path for the robot is as follows: using the robot's current position, the task endpoint position, and the health status assessment result as inputs to the initial path planner, the optimal initial path is output.

[0149] Specifically, based on the multi-level semantic map built and updated for the robot in step S1, the scheduling platform searches for several global paths from the robot's current position to the task endpoint. Then, the path cost function is adjusted based on the robot's health status. The adjustment method is as follows: For robots in excellent health, the goal is to minimize the total distance. For robots in good health, a balance is struck between distance and path complexity. For robots in a healthy state, priority should be given to choosing wide roads with rich features; For robots whose health status is at risk, generate a safe path directly to the maintenance station.

[0150] Finally, the initial walking path (i.e. the planned path in step S3) is planned based on semantic and dynamic environment understanding. The planning method of the planned path has been explained in detail when introducing the indoor and outdoor fusion navigation method of the cleaning robot. Therefore, the process of generating the initial walking path will not be repeated here.

[0151] In this embodiment, task allocation is deeply integrated with the robot's intrinsic navigation health status, achieving optimal resource allocation and proactive management of task execution risks.

[0152] In step L4, the method for real-time tracking and online correction of the robot's actual walking path has been explained in detail in steps B1-B5 above, and will not be repeated here.

[0153] In step L5, the method by which the closed-loop detection module performs global optimization of the robot's pose based on the observation output fed back by the error state Kalman filter has been explained in detail in steps C1-C2 above, and will not be repeated here.

[0154] The following example illustrates the method used in step L5 where the central scheduling platform updates the dynamic scheduling view using the globally optimized pose: For example, the Xingfu Community mentioned above includes key areas such as Building 8, the central garden, and the garbage transfer station. Robot-01 is currently performing the task of "garbage collection in Room 1001 of Building 8"; Robot-02 is currently performing the task of "garbage bin emptying in the central garden"; Robot-03 is idle and on standby. Due to its movement in the long corridor, Robot-01 experiences a cumulative drift of approximately 0.8 meters in its positioning due to the sparse visual features.

[0155] When Robot-01 walks to the entrance of Building 8, its loop closure detection module identifies that the scene matches the keyframe "Building 8 Unit Door" stored in the map. After the loop closure detection is confirmed, the backend optimizer performs position map optimization (steps C1-C2), correcting Robot-01's pose from the drifting coordinates (105.3, 87.1) to the precise coordinates (104.5, 87.4). Then, Robot-01 encapsulates the optimized high-precision pose into a message and sends it to the central scheduling platform.

[0156] After receiving the message, the central dispatch platform first verifies the integrity and rationality of the data, and checks whether the timestamps are continuous and whether the covariance matrix represents high confidence. Then, on the digital map of the dynamic dispatch view, the Robot-01 icon is instantly moved from its original position (105.3, 87.1) to a new position (104.5, 87.4), while the robot's historical trajectory is updated, and the most recent path segment is smoothly corrected.

[0157] Since the pose of instantaneously moving to a new location is based on a closed-loop constraint with high confidence, the platform performs a visualization update of the confidence of each landmark around the new location (such as within a preset radius). For example, it removes the yellow or orange warning signs that may have previously existed, which represent low to medium confidence, and updates them to high-confidence landmarks or areas represented by dark green.

[0158] For example, the view state of Robot-01 before the update was: The map shows that Robot-01's location coordinates are (105.3, 87.1), within the yellow medium confidence circle; the current task is: to go to room 1001 (task completion progress 65%), estimated arrival time: 2 minutes later.

[0159] The updated view status of Robot-01 is as follows: The map shows that Robot-01's location coordinates are (104.5, 87.4), within the dark green high confidence circle; the current task is: to go to room 1001 (task completion progress 92%) (progress updated due to location correction), estimated arrival time: 30 seconds later (time updated due to distance recalculation).

[0160] Subsequently, based on the updated dynamic scheduling view with confidence level visualization, the robot's path planning, conflict and collaborative tasks are re-evaluated, and the evaluation results are output.

[0161] In this embodiment, the process of recalculating path planning based on the dynamically updated confidence-based scheduling view is as follows: For example, the platform discovered that Robot-01's original planned path was a straight line from (105.3, 87.1) to room 1001, but the current precise location shows that it is actually very close to the target. At this point, the platform immediately cancels Robot-01's original complex path planning and generates a simplified final approach path, such as moving 5 meters directly from the current position to room 1001. This replanning process is also achieved through the planned path generation process described in steps S31-S33 above, and will not be repeated here.

[0162] In this embodiment, the method for re-evaluating potential conflicts of the robot based on the updated dynamic scheduling view is briefly described as follows: For example, based on the drift position, the system predicted that Robot-01 would meet Robot-02, which was returning to the garbage station, in a narrow passage in one minute. However, the updated confidence visualization showed that Robot-01 had actually already passed through the narrow passage, and there was no conflict. Therefore, the "slow down and avoid" command sent to both robots was immediately canceled, and their normal driving speed was restored.

[0163] The method for identifying collaborative task opportunities based on the updated dynamic scheduling view is briefly described below: For example, based on the updated location display after confidence level visualization, Robot-01 is located in the center of Building 8 after completing its current task. At this time, the platform detects that there are still 3 pending garbage collection tasks in Building 8 (such as rooms 1002, 1003, and 1005). Therefore, it cancels the original plan to schedule Robot-02 to come from a distance to handle these tasks, and immediately generates an optimized task sequence for Robot-01, such as: after completing the task in room 1001, handle the garbage collection tasks in rooms 1002, 1003, and 1005 in sequence, while notifying Robot-02 to handle the next task.

[0164] Therefore, based on the new precise location of Robot-01, the platform recalculated the task allocation for the entire community. For example, Robot-01 was responsible for all the remaining tasks in Building 8; Robot-02 was reassigned to Building 3 to handle the emergency overflow of trash cans; and Robot-03 returned to the trash station as originally planned, but the route was optimized to avoid detours.

[0165] In summary, the robot cooperative scheduling method based on fusion navigation provided in this embodiment, based on the real-time tracking and precise correction of the robot's actual walking path by the closed-loop detection module, updates the dynamic scheduling view by utilizing the observation output of the error state Kalman filter, and establishes a real-time closed loop from single-machine localization optimization to global state synchronization and then to system decision optimization. This enables the entire multi-robot system to have "instantaneous adaptive capability" for cooperative scheduling, that is, after any robot obtains a more accurate positioning position, it can immediately transform into a collective behavior of more intelligent cooperative scheduling of the entire system, thereby making cooperative scheduling more effective, accurate and reliable.

[0166] The following describes the fully automated waste collection and control method based on collaborative scheduling provided in this embodiment. Figure 1 As shown, the method includes the following steps: M1, constructs a dynamic scheduling view for the fusion of multi-source heterogeneous data; It should be noted that the dynamic scheduling view constructed and maintained in step M1 differs from the dynamic scheduling view constructed in step L1. In constructing the dynamic scheduling view in step M1, the multi-source heterogeneous data fused includes closed-loop constraints generated by the closed-loop detection module, real-time position, speed, and task status data of all online robots, as well as waste status sensor data and environmental topology information that may not be fused in step L1.

[0167] Step M1, which constructs the dynamic scheduling view, specifically includes the following steps: M11 integrates heterogeneous data from multiple sources to establish a timestamped job data warehouse. The specific method is as follows: First, the received multi-source heterogeneous data is spatiotemporally aligned. The system establishes a unified time reference and spatial coordinate system, and each data source pushes data packets in real time, with each data packet carrying the original timestamp and coordinate information.

[0168] The data packet sending format of the closed-loop detection module is, for example: {Timestamp T1, Robot ID-01, Correction coordinates (x1, y1), Confidence level 0.95, Correction amount ( x=0.2, y=0.1)}; Robot pose system sends: {Timestamp T1, Robot ID-01, Real-time coordinates (x1', y1'), Speed ​​0.5m / s, Task status: Heading to Building A}; Waste sensor sends: {Timestamp T1, Sensor ID-S05, Waste volume 85%, Leakage detection positive, Temperature 28℃} The environmental topology system provides: {Building A coordinates, passage width, elevator location, restricted area}, where the restricted area is provided by the dynamically updated multi-level semantic map mentioned above.

[0169] Then, the received raw data undergoes quality assessment and repair. For example, if an anomaly is detected in the speed data of robot Robot-01 (instantaneously jumping from 0.3 m / s to 3 m / s), a sliding window smoothing repair is performed by combining data from preceding and following frames. Another example is the discovery of missing data from sensor S08; in this case, spatiotemporal correlation interpolation is used to estimate the speed based on real-time data from neighboring sensors S07 and S09. Low-confidence data (e.g., confidence < 0.6) identified within the closed-loop constraints are marked as data to be verified.

[0170] Subsequently, the cleaned data is standardized and dimension-unified. For example, coordinate data is uniformly converted to the community global coordinate system; various status data (such as speed, amount of garbage, temperature, etc.) are normalized to the range of [0,1]; discrete task status (such as going to Building A to perform recycling) is encoded into feature vectors; and environmental topology information is encoded into a graph structure, such as nodes representing key locations and edges representing connectivity.

[0171] Subsequently, feature association and fusion are performed on the unified processed multi-source data to establish semantic relationships between the data and generate fused features. For example, the real-time pose of robot Robot-01 is associated with newly generated or recent closed-loop correction constraints to generate pose reliability features; the garbage status data is associated with the topology of the building where it is located to establish environmental risk distribution features; and the robot's task status is associated with its current position and speed to generate task execution progress features.

[0172] Finally, the various fusion features are timestamped and stored in the database, and the data is organized according to time series to build a job data warehouse. For example, with a time granularity of 100ms, all data streams are aligned, and a comprehensive data record is generated for each time slice as shown in the example below: Timestamp: T1+100ms; Robot status: {Robot-01: coordinates, speed, task, credibility; Bobot-02: coordinates, speed, task, credibility; ...}; Waste status: {Sensor No. S05: Waste quantity, leakage risk; Sensor No. S06: Waste quantity, leakage risk; ...}; Environmental topology: {Passage status, obstacle information}; Data quality: {Completeness score, Timeliness score}.

[0173] This embodiment constructs a task data warehouse containing timestamps. When the scheduling system needs to make decisions, it can quickly obtain the precise status of robots, waste distribution, and environmental constraints within the community at any given time, thereby generating an optimal scheduling strategy. For example, task allocation can be adjusted based on robot pose reliability, and collection priorities can be dynamically adjusted based on leachate risk to improve overall system performance.

[0174] It is important to emphasize that the multi-source data integrated in the constructed operation data warehouse includes closed-loop constraints continuously generated by the closed-loop detection module. These closed-loop constraints include global pose correction and position confidence. Therefore, the data in the constructed operation data warehouse is more effective, which in turn helps to improve the efficiency of subsequent fully automated control of waste collection and transportation and the accuracy and effectiveness of collaborative scheduling.

[0175] M12, generating the dynamic job feature matrix, the method is as follows: First, multi-dimensional feature extraction is performed from the job data warehouse, such as in the current... The system continuously extracts real-time pose features of the robot (such as robot position, orientation angle, velocity vector, and then calculates acceleration and motion curvature), waste status features (waste weight, volume, surface image HSV features, temperature change rate, etc.), environmental topology features (building distribution, path width, slope, obstacle density, etc.), and time-related features (calculates the temporal change rate and periodic pattern of each feature based on timestamps).

[0176] Then, the extracted multi-dimensional features are spatiotemporally fused. For example, the feature vector of robot A is: [position (10.2, 5.3), speed 0.8 m / s, garbage weight 15 kg, path width 2.1 m, rate of change 0.2]; the feature vector of robot B is: [position (25.1, 8.7), speed 1.2 m / s, garbage weight 8 kg, path width 3.5 m, rate of change 0.1]; the feature vector of robot C is: [position (15.6, 12.4), speed 0.5 m / s, garbage weight 22 kg, path width 1.8 m, rate of change 0.3].

[0177] The method for spatiotemporal fusion of the above-mentioned multi-dimensional features of robots A, B, and C to form a dynamic operation feature matrix is ​​expressed as follows: Dynamic job feature matrix =

[0178] The first row of the dynamic task feature matrix above represents the association features between robot A and robot B; the second row represents the association features between robot B and robot C; and the third row represents the association features between robot C and robot A. The circular association ensures that each robot establishes a direct association with other robots, avoiding the problem of weak association between end nodes. At the same time, the matrix is ​​dynamically rearranged as the robots move. When the robot position changes, the association relationship is automatically updated. Thus, when performing collaborative scheduling of multiple robots through the above steps L1-L5, the optimal collaborative scheduling performance can always be maintained.

[0179] M13, based on a dynamic job feature matrix, identifies the associated paths of each job area and constructs a job topology network; Based on the dynamic job feature matrix, the method for identifying the associated paths of each job area is as follows: The dynamic operation feature matrix contains multi-dimensional temporal features, including: robot historical trajectory data, task execution time distribution features, inter-regional traffic frequency statistics, environmental obstacle distribution information, real-time traffic flow patterns, etc.

[0180] First, based on the multi-dimensional temporal features contained in the dynamic operation feature matrix, the movement patterns of the robot swarm within the community or neighborhood are analyzed, and the correlation strength between any two operation areas is calculated, specifically including: Traffic frequency: The number of times a robot travels from one area to another per unit of time; Traffic efficiency: Calculates the average travel time and energy consumption between two areas based on historical data; Task relevance: Analyze the logical relationship between adjacent task points, such as tasks on different floors of the same building having a natural relevance.

[0181] The weighted sum of traffic frequency, traffic efficiency, and task relevance between two regions represents the strength of the relationship between the two regions.

[0182] For example, in the Xingfu Community, the system discovered through analysis of historical data that: The path from Building 8 to the central waste station is frequently used by 80% of the robots; there is a shortcut between Building 3 and Building 5, which saves 40% of the travel time compared to the main road; during morning and evening rush hours, the roads around Building 2 are severely congested, requiring the identification of alternative routes.

[0183] Based on the above analysis, the system identified the following paths: the path from Building 8 to the central waste station as the core associated path; the path from Building 3 to Building 5 as the secondary associated path; and the passageway on the north side of Building 2 as a temporary avoidance path.

[0184] In step M13, the method for constructing the job topology network is as follows: Each work area (such as building unit entrance, garbage collection point, charging station) is first defined as a node in the topology network. Node attributes include: geographical coordinates, task type distribution, service time window, resource capacity limit, etc.

[0185] Then, the relationships between work areas are established. First, edge weights are calculated. Edge weights include travel cost edges, task logic edges, and resource dependency edges. The travel cost edges have their weights dynamically adjusted based on real-time traffic conditions, and the weight is a weighted sum of travel time, energy consumption cost, and travel risk. Task logic edges are logically connected based on task sequence relationships; resource dependency edges are dependently connected based on facility sharing relationships.

[0186] In step M2, the method for establishing a multi-factor coupled urgency assessment and task allocation mechanism includes the following steps: M21, the waste decomposition index is calculated based on waste surface image data, using the following method: First, images of the garbage surface are captured using an RGB camera mounted on the robot, and then converted from the RGB color space to the HSV color space. In the HSV space, the hue, saturation, and lightness values ​​of each pixel are extracted.

[0187] Next, the current image is divided into multiple monitoring layers, and compared layer by layer with the garbage surface image from the previous acquisition cycle. The comprehensive chromaticity deviation of all pixels between the corresponding monitoring layers is calculated, and the calculation method is as follows: The preferred method for dividing an image into multiple monitoring layers is spatial grid partitioning, such as dividing a garbage surface image into M×N uniform grid units, with each grid serving as an independent monitoring layer.

[0188] Then, the hue difference value is calculated for each pixel in the current image and the garbage surface image collected in the previous cycle. Saturation attenuation factor Brightness variation factor ; The weighting coefficient represents the hue difference and is used to adjust for hue variations. In the final comprehensive color deviation The proportion of contribution in; , These represent the hue values ​​of the current image and the compared garbage surface image, respectively. The weighting coefficients representing the saturation decay factor; , These represent the saturation values ​​of the current image and the compared garbage surface image, respectively; This represents the saturation decay penalty coefficient; , These represent the brightness values ​​of the current image and the compared garbage surface image, respectively; This represents the weighting coefficient of the brightness variation factor.

[0189] Then, the overall chromaticity deviation value is calculated for each pixel. , The spatial importance mask is preferably calculated using a center-weighted two-dimensional Gaussian function in this embodiment. The calculation method is briefly described below: The image coordinates are normalized so that the center coordinates are (0,0) and the coordinates of the four corners are between [-1,1]. Then the calculation is performed. , Represents the normalized coordinates of the pixels; , Indicates the control of the Gaussian function at... and The parameter for the width of the direction. The smaller the value, the more the weight is concentrated in the center; the larger the value, the more evenly the weight distribution.

[0190] Finally, the fractional statistic method was used to calculate the comprehensive color deviation of the monitoring layer. ,in, The upper quartile represents the chromaticity deviation of 75% of the pixels, dividing the dataset into two parts: 75% of the pixels have a chromaticity deviation value less than or equal to... 25% of the pixels have a chromaticity deviation value greater than ; This represents the 90th percentile, indicating that 90% of the pixels have a chromaticity deviation value less than or equal to... It focuses on the starting threshold of the 10% of pixels with the largest deviations in the data; Representing the lower quartile, indicating that 10% of pixels have a chromaticity deviation value less than or equal to... This reflects the level of the 10% of pixels with the smallest deviation in the data.

[0191] After calculating the comprehensive color deviation value between the monitoring layers, the final step is to calculate the waste decomposition index: Based on the comprehensive color deviation values ​​between monitoring layers, standardization is performed using a prior knowledge base of waste types. For perishable waste, a higher decay sensitivity coefficient is assigned; for non-perishable waste, the weight is appropriately reduced. Then, the decay level score of the current monitoring layer is obtained by weighted calculation of the comprehensive color deviation values ​​of each type of waste on that monitoring layer. Finally, the waste decay index of the waste surface image is obtained by combining the decay level scores of all monitoring layers (weighted summation).

[0192] M22, combining leachate concentration data and unit interlayer gap length, analyzes the degree of leachate diffusion and establishes a leachate hazard assessment model. The specific method is as follows: First, multi-source exudate data are collected and preprocessed, including the following methods: Real-time leachate concentration data is acquired through a distributed leachate sensor network to eliminate transient noise interference from the sensors. Subsequently, images of the interlayer structure are acquired using a high-resolution image sensor to identify and quantify the interlayer gap features. It should be noted that deep learning segmentation networks can be applied to identify and quantify these interlayer gap features; however, since the methods for identifying and quantifying these features are not within the scope of this application, they will not be specifically described. Finally, a spatiotemporal correlation database of leachate concentration and gap length is established to record baseline parameters under different waste types and environmental conditions.

[0193] Then, a variable permeation-diffusion model is established. , The variable permeability coefficient represents the correlation between permeability height and permeability. It represents the seepage flow rate (the volume of leachate passing through the voids between the layers of waste per unit time). It indicates the head height (the driving force for the flow of seepage in the voids). This indicates the seepage path length (the actual path length that leachate travels as it flows between layers of waste).

[0194] In addition, the effective diffusion path is calculated based on the topological structure of the interlayer gaps. , It represents the theoretical maximum diffusion coefficient in an ideal, homogeneous medium, describing the inherent ability of a specific permeate to diffuse or seep due to a concentration gradient when it is not constrained by complex geometries. The interstitial structure influence function quantifies the specific impact of the complex and non-ideal physical structure between waste layers on leachate diffusion capacity, and is related to interstitial density, interstitial directionality, and interstitial connectivity.

[0195] Subsequently, a multi-level hazard assessment index system was designed, such as the first-level hazard index: the leakage retention hazard index. , The toxicity coefficient of the pollutant. The concentration of the exudate. This is the volume of exudate retention. Retention time; Secondary hazard index: Leakage diffusion hazard index , Indicates the environmental sensitivity coefficient. Indicates the area potentially affected by pollution; Level 3 Hazard Index: Comprehensive Hazard Value , This indicates the chemical factors that take into account the composition of waste. , , For adaptive weights.

[0196] Once the multi-level hazard assessment index system is designed, the leakage hazard assessment model is completed.

[0197] M23, calculate the pollution source escape risk value, and use the established leachate hazard assessment model to calculate the landfill leachate hazard value. Then, combine the pollution source risk value and the landfill leachate hazard value to determine the urgency of treatment. In this embodiment, the pollution source escape risk value is the weighted sum of the pollution intensity of the pollution source and the escape intensity per unit residence time. Wherein, the pollution intensity of the pollution source... The calculation formula is: , Indicates the first The weighting coefficients for each pollutant (determined based on factors such as toxicity and diffusion). Indicates the first The measured concentrations of the pollutants, Indicates the first Environmental safety thresholds for various pollutants.

[0198] Emission intensity per unit residence time , This represents the equivalent volume of the gas, calculated based on the type and concentration of pollutants. It is the product of wind speed, diffusion time, and diffusion attenuation coefficient; The wind direction factor is adjusted based on the density of residential areas. The effect of temperature on gas diffusion rate is considered as a temperature influence factor. This indicates the time the garbage has been stored.

[0199] Then the pollution intensity of the pollution source and the intensity of escaping per unit residence time We perform a weighted summation to obtain the pollution source escape risk value.

[0200] The hazard value of landfill leachate is represented by the three-level hazard index values ​​mentioned above. The urgency of treating the landfill is obtained by weighting and summing the pollution source escape risk value and the landfill leachate hazard value.

[0201] M24 takes into account urgency, robot health status, and path efficiency, and generates the optimal allocation strategy through a designed multi-objective reward function.

[0202] In this embodiment, a multi-objective reward function is designed. The expression is as follows:

[0203] In the calculation formula, , , Indicates adaptive weights; This represents the processing urgency value calculated in step M23; This indicates the robot's current health status, calculated comprehensively based on positioning reliability score, battery health, and mechanical wear (e.g., a weighted sum of the three). It should be noted that in this embodiment, positioning reliability is characterized by the position confidence level carried in the closed-loop constraints calculated by the robot's closed-loop detection module. After optimizing and correcting the accumulated error of the robot's global pose graph through steps A1-A5 above, the reciprocal of the covariance of the pose correction is used as the position confidence level.

[0204] The path efficiency reward component is a weighted sum of the distance efficiency, time efficiency, and energy efficiency of the movement. Additionally, the multi-objective reward function can include penalty terms, such as the number of path conflicts or the degree of robot overload.

[0205] Methods for generating optimal task allocation strategies include: First, candidate strategies are evaluated and generated. Define "which robot performs which task, and the order of execution." The method for generating candidate strategies is based on the same principle as the method for generating initial walking paths, the difference being that one plans the initial walking path based on the robot's current position, task endpoint, and health status assessment results, while the other allocates tasks based on the current states of all robots and all task information. More preferably, a pre-trained policy neural network can be used to generate... One candidate task allocation strategy There are many existing policy neural networks used to generate task assignment strategies, so we will not go into detail about how to train policy neural networks.

[0206] Then, using The optimal strategy is evaluated using the following method: Candidate strategies (Calculate the corresponding strategy based on the robot state, task state, and other data carried in the candidate strategy) , , Substitute into the multi-objective reward function In the middle, predicting the robot's execution strategy Total rewards that can be obtained Points, and will have the highest The strategy of dividing This is the optimal task allocation strategy.

[0207] Ultimately, based on the strategy Based on the actual execution results, the actual total reward points will be recalculated. and will , The feedback is sent to the candidate policy generation module to retrain the candidate policy generation model.

[0208] In step M3, the coordinated control process based on error state Kalman filtering and closed-loop correction has been described in detail in L1-L5 above, and will not be repeated here.

[0209] In summary, the fully automated waste collection and control method based on collaborative scheduling provided in this application first ensures the accuracy, authenticity, and effectiveness of the data upon which the subsequent collaborative control process depends by constructing an operational data warehouse with closed-loop constraints. Then, through circular association in the dynamic operational feature matrix, it ensures that each machine establishes a direct connection with other robots, avoiding the reduction in multi-machine collaborative effects due to weak correlation of end nodes. At the same time, this association relationship is automatically updated as the robot position changes according to the closed-loop constraint correction, ensuring that optimal collaborative scheduling performance is always maintained when multiple robots are collaboratively scheduled. Through a multi-objective reward function, the task allocation strategy is always optimized, thereby realizing fully automated waste collection with multi-robot collaborative scheduling, dynamic path planning, and accurate status monitoring, improving the efficiency and intelligence level of waste recycling in communities or neighborhoods.

[0210] It should be stated that the above-described specific embodiments are merely preferred embodiments and technical principles applied in this application. Those skilled in the art should understand that various modifications, equivalent substitutions, and variations can be made to this application. However, such variations, as long as they do not depart from the spirit of this application, should be within the scope of protection of this application. Furthermore, some terminology used in this application's specification and claims is not limiting but merely for ease of description.

Claims

1. A fully automated waste collection and control method based on collaborative scheduling, characterized by the following steps: include: M1, builds and updates a dynamic scheduling view for the fusion of multi-source heterogeneous data; M2, establish a multi-factor coupled urgency assessment and task allocation mechanism; M3 implements coordinated control based on error state Kalman filtering and closed-loop correction.

2. The fully automated waste collection and control method based on collaborative scheduling according to claim 1, characterized in that, Step M1, the method for constructing the dynamic scheduling view, includes the following steps: M11 integrates one or more multi-source heterogeneous data, including closed-loop constraints generated by the closed-loop detection module, real-time position, speed, and task status data of all online robots, waste status sensor data, and environmental topology information, to establish a time-stamped operation data warehouse. M12, extract multi-dimensional features from the job data warehouse and perform spatiotemporal fusion to generate a dynamic job feature matrix; M13, based on the dynamic task feature matrix, identifies the associated paths of each task area, constructs a task topology network, and integrates the task topology network, the closed-loop constraints, and the real-time pose, speed, and task status data of each robot into the dynamic scheduling view.

3. The fully automated waste collection and control method based on collaborative scheduling according to claim 1, characterized in that, Step M2, the method for establishing a multi-factor coupled urgency assessment and task allocation mechanism, includes the following steps: M21, calculating the waste decomposition index based on waste surface image data; M22, combining leachate concentration data and unit interlayer gap length, analyzes the degree of leachate diffusion and establishes a leachate hazard assessment model; M23, calculate the pollution source escape risk value, and use the leachate hazard assessment model to calculate the landfill leachate hazard value. Then, combine the pollution source risk value and the landfill leachate hazard value to determine the urgency of treatment. M24 considers the processing urgency, robot health status, and path efficiency, and uses a designed multi-objective reward function. Generate the optimal task allocation strategy.

4. The fully automated waste collection and control method based on collaborative scheduling according to claim 3, characterized in that, In step M24, the method for generating the optimal task allocation strategy includes the following steps: M241, evaluating and generating candidate strategies; M242, utilizing The optimal strategy is evaluated using the following method: Candidate strategies Substitute into the multi-objective reward function In the middle, predicting the robot's execution strategy Total rewards that can be obtained Points, and will have the highest The strategy of dividing As the optimal task allocation strategy; M243, based on the strategy Based on the actual execution results, the actual total reward points will be recalculated. and will , The feedback is sent to the candidate policy generation module to retrain the candidate policy generation model.

5. The fully automated waste collection and control method based on collaborative scheduling according to claim 3 or 4, characterized in that, The robot health used in the multi-objective reward function includes a positioning reliability score for the robot, which is characterized by the position confidence carried in the closed-loop constraints calculated by the closed-loop detection module in the robot. The method for generating the closed-loop constraint by the closed-loop detection module includes the following steps: A1, the closed-loop detection module continuously receives image frames from the visual sensor and point cloud frames from the lidar, and calculates the global appearance descriptor of the current frame; A2, match the global appearance descriptor of the current frame with the historical key frame descriptors stored in the maintained local sliding window map, filter out several similar closed-loop candidate frames, and obtain the appearance similarity score between the current frame and each of the closed-loop candidate frames. A3, perform geometric consistency verification on each closed-loop candidate frame, check the reprojection error of matching feature points and the point cloud registration error, and calculate the geometric verification score between the current frame and the closed-loop candidate frame. A4. Determine whether the appearance similarity score and geometric verification score between the current frame and the closed-loop candidate frame are both greater than the corresponding adaptive score threshold. If so, proceed to step A5; If not, then the closed-loop candidate frame is removed; A5 sends the closed-loop constraints to the backend optimizer for global pose graph optimization and correction of cumulative errors. The position confidence is the reciprocal of the covariance of the pose correction.

6. The fully automated waste collection and control method based on collaborative scheduling according to claim 1, characterized in that, In step M3, the coordinated control process based on error state Kalman filtering and closed-loop correction includes the following steps: M31, in the optimal strategy During execution, based on the closed-loop detection results and path tracking status, the robot's actual walking path is tracked and corrected online in real time; The M32 closed-loop detection module performs global optimization of the robot's pose based on the observation output fed back by the error state Kalman filter, and synchronizes the globally optimized pose to the central scheduling platform to update the dynamic scheduling view, forming a closed-loop feedback from single-machine localization to collaborative scheduling.

7. The fully automated waste collection and control method based on collaborative scheduling according to claim 6, characterized in that, In step M31, the method for real-time tracking and online correction of the robot's actual walking path includes the following steps: B1 uses a model predictive controller to calculate the robot's expected linear velocity and angular velocity based on the path points in the planned initial walking path; B2, calculates the lateral and orientation deviations between the robot's actual pose and the initial walking path in real time; B3, dynamically adjusts the prediction time domain and control time domain parameters of the path tracking controller based on the path curvature, robot speed and deviation magnitude; B4. When the lateral or orientation deviation exceeds a preset threshold, local replanning is triggered to generate a new path from the current position to the next sub-target point. B5, the pose correction amount confirmed by closed-loop detection is introduced as feedback into the path tracking controller to smoothly compensate for pose jumps in subsequent control cycles.

8. The fully automated waste collection and control method based on system scheduling according to claim 1, characterized in that, In step B1, the method for planning the initial walking path includes the following steps: S31, The path planner receives semantic task instructions from the scheduling system and parses them into specific coordinate sequences in a multi-level semantic map; S32, based on the parsed coordinate sequence, plans the initial global path on the coarse-resolution raster map; S33, guided by the initial global path, local path planning is performed, while real-time laser data and dynamic semantic layer information are integrated to avoid static and dynamic obstacles.

Images