A visual navigation method, system, medium, and program product for smart community robots.

By extracting static semantic regions and dividing them into repeating and non-repeating texture regions in a smart community environment, constructing a 3D reference plane, and correcting pose data, the problem of low positioning accuracy in dynamic environments of traditional visual navigation algorithms is solved, thus improving navigation accuracy and stability.

CN123065460APending Publication Date: 2026-11-10SMCAIOT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611133001.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-29
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In complex outdoor or semi-outdoor environments like smart communities, which contain a large number of dynamic objects such as pedestrians and vehicles, traditional visual navigation algorithms are prone to mistaking feature points on dynamic objects for static road signs, leading to deviations in feature matching and reducing the algorithm's positioning accuracy and navigation robustness.

Method used

By extracting static semantic regions and calculating spatial similarity, the image is divided into repeating texture regions and non-repeating texture regions. The non-repeating texture regions are used for initial pose estimation, and a three-dimensional reference plane for the repeating texture regions is constructed. Illumination interference pixels are removed, and the initial pose data is corrected using normal residuals.

Benefits of technology

It improves the visual navigation accuracy of smart community robots in complex environments, reduces the probability of mismatch in areas with repetitive textures, and enhances navigation stability and positioning accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123065460A_ABST
    Figure CN123065460A_ABST
Patent Text Reader

Abstract

A visual navigation method, system, medium, and program product for smart community robots are disclosed. In this method, static semantic regions are first extracted from an image sequence and divided into repeating and non-repeating texture regions based on feature space similarity. The non-repeating texture regions are used for pose estimation to obtain an initial pose, while a 3D reference plane is constructed based on the depth data of the repeating texture regions. The repeating texture regions of the current frame are reprojected onto this plane using the initial pose, and illumination interference pixels are accurately removed by calculating photometric errors. Finally, the spatial distance residual between the interference-free repeating texture regions and the 3D reference plane is constructed, and its projection generates a normal residual to correct the initial pose, obtaining the target pose data, thereby controlling the smart community robot to achieve navigation. This application aims to improve the global positioning accuracy of smart community robots in smart communities.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence software, and in particular relates to a visual navigation method, system, medium and program product for smart community robots. Background Technology

[0002] With the deepening application of artificial intelligence software in smart community scenarios, robot autonomous navigation algorithms based on computer vision have become a research hotspot. However, traditional visual navigation algorithms mainly rely on the extraction of geometric features of low-level pixels for pose estimation. In complex outdoor or semi-outdoor environments such as smart communities containing a large number of dynamic objects such as pedestrians and vehicles, traditional algorithms are prone to mistaking feature points on dynamic objects for static road signs, resulting in deviations in feature matching and thus reducing the positioning accuracy and navigation robustness of the algorithm.

[0003] To address the shortcomings of traditional visual navigation algorithms in terms of robustness reduction in dynamic environments, this paper proposes an AI-based visual navigation algorithm that incorporates deep learning semantic segmentation networks. Specifically, this technique introduces a convolutional neural network model at the visual front end to perform pixel-level semantic category prediction on the input image sequence, thereby identifying potential dynamic targets such as pedestrians and moving vehicles. Subsequently, the technique uses a masking mechanism to remove visual data within the dynamic target regions, retaining only data from static background regions such as buildings, roads, and trees for subsequent camera pose calculation. This AI algorithm, which incorporates semantic prior information, effectively filters dynamic interference and improves the stability of pose estimation in dynamic scenes containing moving targets.

[0004] In smart communities, robots often need to navigate along homogeneous brick walls with highly repetitive textures and continuous, uniform fences. These areas are also frequently accompanied by dynamic light and shadow changes caused by natural winds affecting roadside trees. Semantic segmentation networks can correctly classify brick walls and fences as static backgrounds and directly use them as the basis for subsequent spatial location inference. However, due to the homogeneity of textures and local lighting perturbations in these static backgrounds, the spatial reference information they provide is highly ambiguous. When navigation algorithms perform continuous position calculations based on this ambiguous static background data, they are prone to position matching deviations in highly similar scene inputs. As the robot continues to move, this calculation error accumulates, ultimately leading to a decrease in the global positioning accuracy of the smart community robot within the smart community. Summary of the Invention

[0005] This application provides a visual navigation method, system, medium, and program product for smart community robots, which can improve the global positioning accuracy of smart community robots in smart communities.

[0006] In the first aspect, this application provides a visual navigation method for smart community robots, which uses a preset semantic recognition model to extract static semantic regions from a continuous visual image sequence collected by the smart community robot;

[0007] Extract image features from static semantic regions and calculate the spatial similarity of image features. Regions with spatial similarity greater than a preset similarity threshold are considered repetitive texture regions, while regions with spatial similarity less than or equal to the preset similarity threshold are considered non-repetitive texture regions.

[0008] Based on the visual features of non-repeating texture regions, pose estimation is performed between adjacent frame images to obtain the initial pose data of the current frame image;

[0009] Construct a 3D reference plane corresponding to the repeating texture region based on the depth data of the repeating texture region;

[0010] Based on the initial pose data, the repetitive texture region in the current frame image is reprojected onto the 3D reference plane of the adjacent previous frame image to obtain the reprojected pixels.

[0011] Calculate the photometric error between the reprojected pixel and the corresponding position in the previous frame image;

[0012] Pixels with a photometric error greater than a preset error threshold are identified as lighting interference pixels, and these pixels are removed from the duplicate texture area.

[0013] Construct the spatial distance residual from the repeated texture region to the 3D reference plane after removing pixels that interfere with lighting;

[0014] The spatial distance residual is projected onto the normal vector direction of the three-dimensional reference plane to generate the normal residual;

[0015] The initial pose data is corrected using the normal residual to obtain the target pose data;

[0016] The smart community robot is controlled to navigate based on the target pose data.

[0017] By extracting static semantic regions and calculating spatial similarity, the image is divided into repetitive and non-repetitive texture regions. Initial pose estimation is performed using the non-repetitive texture regions, reducing the probability of mismatches in repetitive texture regions that is common in traditional feature point methods, thus improving the reliability of the initial pose. Based on this, a 3D reference plane is constructed using the depth data of the repetitive texture regions, and photometric errors are calculated through reprojection to eliminate illumination-interference pixels, reducing the negative impact of ambient lighting changes on pose estimation. Subsequently, the spatial distance residual from the interference-free repetitive texture regions to the 3D reference plane is constructed and projected onto the normal vector direction to generate a normal residual. This normal residual is used to correct the initial pose data. By utilizing the structured features of repetitive texture regions such as walls and fences commonly found in community environments, effective geometric constraints are imposed on the robot's pose, thereby improving the visual navigation accuracy of the smart community robot in complex environments containing a large number of repetitive textures.

[0018] In conjunction with some implementations of the first aspect, in some implementations, image features of static semantic regions are extracted, and spatial similarity of image features is calculated, specifically including:

[0019] Extract the gradient magnitude and gradient direction of each pixel in the static semantic region as image features;

[0020] One-dimensional projection is performed on image features along multiple preset projection directions to generate gradient distribution sequences in each projection direction;

[0021] Autocorrelation analysis is performed on the gradient distribution sequence to extract the periodic peak characteristics of the gradient distribution sequence;

[0022] Spatial similarity is calculated based on the peak intensity and peak spacing of the periodic peak characteristics;

[0023] Regions with spatial similarity greater than a preset similarity threshold are designated as repeating texture regions, while regions with spatial similarity less than or equal to the preset similarity threshold are designated as non-repeating texture regions.

[0024] By extracting the gradient magnitude and direction of pixels in static semantic regions as image features, and generating gradient distribution sequences through one-dimensional projection along multiple preset directions, complex two-dimensional image texture information is transformed into easily processed one-dimensional sequences, reducing the dimensionality of data processing. Autocorrelation analysis of this gradient distribution sequence accurately extracts periodic peak features reflecting texture repetition patterns. Subsequently, spatial similarity is calculated based on peak intensity and peak spacing, reducing the interference of image noise or minor viewpoint changes on texture region attribute determination, improving the accuracy of distinguishing between repetitive and non-repetitive texture regions, and thus enhancing the reliability of region feature extraction in visual navigation systems.

[0025] In conjunction with some implementations of the first aspect, in some implementations, a three-dimensional reference plane corresponding to the repeating texture region is constructed based on the depth data of the repeating texture region, specifically including:

[0026] The depth data of the repeating texture region is converted into an initial 3D point set in the 3D point cloud space;

[0027] Calculate the local curvature of each 3D point in the initial 3D point set, and delete 3D points with local curvature greater than a preset curvature threshold to obtain a smooth 3D point set;

[0028] The smooth 3D point set is divided into multiple depth intervals along the direction of the preset initial normal vector, and the point cloud density in each depth interval is calculated.

[0029] Extract the target depth range with the highest point cloud density, and use the 3D points within the target depth range as the core point set;

[0030] Principal component analysis is performed on the core point set to fit and generate a three-dimensional reference plane corresponding to the repeating texture region.

[0031] By employing the aforementioned technical solution, the depth data of repetitive texture regions is converted into an initial 3D point set in a 3D point cloud space. By calculating local curvature and removing 3D points with curvature exceeding a threshold, noise, outliers, and small non-planar protrusions in the point cloud data are filtered out, resulting in a smoother 3D point set. The smooth 3D point set is divided into multiple depth intervals along the initial normal vector direction, and the point cloud density is statistically analyzed. The target depth interval with the highest density is extracted as the core point set, which can accurately remove foreground occlusions or background interference points, and pinpoint the most important planar structures in the repetitive texture region. Principal component analysis is performed on the core point set obtained through multiple screenings to fit a 3D reference plane. This reduces the interference of irrelevant point clouds on the plane fitting process, improves the fit between the constructed 3D reference plane and the real physical environment, and thus enhances the accuracy of spatial distance residual calculation.

[0032] In conjunction with some implementations of the first aspect, in some implementations, after controlling the smart community robot to navigate based on target pose data, the method further includes:

[0033] Based on the target pose data, the three-dimensional reference plane is transformed into the global coordinate system for spatial stitching to obtain a global static structure map;

[0034] If the percentage of pixels in the repeating texture region in the current frame image is lower than the preset occlusion threshold, the smart community robot is determined to be in a visual occlusion state.

[0035] Based on the dead reckoning data of the smart community robot, an extended virtual reference surface corresponding to the current position of the smart community robot is extracted from the global static structure map;

[0036] Visual features of the visible non-repeating texture regions in the current frame image are extracted for pose estimation to obtain the initial blind zone pose data under visual occlusion.

[0037] Construct the virtual distance residual from the spatial location coordinates of the smart community robot to the extended virtual reference surface;

[0038] The virtual distance residual is projected onto the normal vector direction of the extended virtual reference surface to generate a virtual normal residual;

[0039] The initial blind zone pose data is corrected by using virtual normal residuals to obtain target blind zone pose data, so as to maintain the normal translation degree of freedom constraint of the smart community robot under visual occlusion.

[0040] By transforming a 3D reference plane to a global coordinate system and stitching them together, a global static structural map is obtained, enabling effective memorization of historical structural information about the community environment. When the robot is determined to be in a visual occlusion state due to the low pixel proportion of repetitive texture regions in the current frame image, the corresponding extended virtual reference surface is extracted from the global static structural map using dead reckoning data. This is combined with visual features of visible non-repetitive texture regions to estimate the initial blind zone pose data, reducing the risk of pose estimation failure caused by a sharp reduction in effective visual features under occlusion. By constructing a virtual distance residual from the robot's spatial position coordinates to the extended virtual reference surface and projecting it to generate a virtual normal residual to correct the initial blind zone pose data, the robot's normal translational degree of freedom can be maintained using prior global map structural information even under visually restricted conditions. This suppresses the cumulative drift error generated by dead reckoning during occlusion and improves the robot's positioning accuracy and navigation stability under visual occlusion conditions.

[0041] In conjunction with some implementation methods of the first aspect, in some implementation methods, based on the dead reckoning data of the smart community robot, an extended virtual reference surface corresponding to the current position of the smart community robot is extracted from the global static structure map, specifically including:

[0042] Extract the pose covariance matrix from the dead reckoning data, and construct a spatial error ellipsoid centered on the current position based on the pose covariance matrix;

[0043] By performing spatial intersection calculations between the spatial error ellipsoid and the global static structure map, multiple candidate reference surfaces located within the spatial error ellipsoid are obtained.

[0044] Obtain the last effective reference surface of the smart community robot before it enters a visual occlusion state, and calculate the angle between the normal vectors and the boundary distance between each candidate reference surface and the last effective reference surface.

[0045] Candidate reference surfaces whose included angle between normal vectors is less than a preset angle threshold and whose boundary distance is less than a preset distance threshold are selected as target candidate surfaces.

[0046] The target candidate surface is used as an extended virtual reference surface corresponding to the current position of the smart community robot.

[0047] By extracting the pose covariance matrix from dead reckoning data to construct a spatial error ellipsoid centered on the current position, the uncertainty range in the dead reckoning process can be reasonably quantified. This allows for precise delineation of the search area in the global static structure map, narrowing the matching range of candidate reference surfaces and reducing the computational overhead and probability of mismatches caused by blind searching in a large map. Furthermore, by calculating the normal angle and boundary distance between each candidate reference surface and the last effective reference surface before the robot enters visual occlusion, and selecting target candidate surfaces that meet threshold conditions as extended virtual reference surfaces, it is ensured that the selected virtual reference surfaces have a high degree of continuity and geometric consistency with the structural features upon which the robot relied before occlusion in terms of spatial attitude and physical position. This improves the accuracy of extended virtual reference surface extraction, reduces the risk of mis-extraction of reference surfaces due to accumulated errors in dead reckoning, and thus enhances the reliability of blind zone pose correction under visual occlusion conditions.

[0048] In conjunction with some implementations of the first aspect, in some implementations, after correcting the initial blind zone pose data using virtual normal residuals to obtain the target blind zone pose data, the method further includes:

[0049] Obtain the depth data of the occluder corresponding to the visual occlusion state in the current frame image, and perform planar fitting on the depth data of the occluder to obtain the dynamic physical occlusion surface;

[0050] Calculate the normal projection distance between the dynamic physical occlusion surface and the extended virtual reference surface, and use it as the real-time dynamic corridor width;

[0051] When the real-time dynamic corridor width is determined to be less than the preset safe passage threshold, the constraint on the normal translation degree of freedom of the extended virtual reference surface to the target blind zone pose data is released.

[0052] Construct the physical distance residual from the spatial location coordinates of the smart community robot to the dynamic physical occlusion surface;

[0053] Projecting the physical distance residual onto the normal vector direction of the dynamic physical occlusion surface yields the obstacle avoidance normal residual.

[0054] The obstacle avoidance normal residual is used to correct the target blind zone pose data in order to control the lateral obstacle avoidance navigation of the smart community robot.

[0055] By acquiring the depth data of the occlusion object corresponding to the visual occlusion state and fitting it to obtain the dynamic physical occlusion surface, and then calculating the normal projection distance between the occlusion surface and the extended virtual reference surface, the real-time dynamic corridor width that the robot can currently pass through can be quantified in a real-time and intuitive manner, accurately assessing the risk of passage collisions in the environment. When it is determined that the corridor width is less than a preset safe passage threshold, the normal translation degree of freedom constraint of the extended virtual reference surface on the target blind zone pose data is promptly released. Subsequently, by constructing the physical distance residual from the robot's spatial position to the dynamic physical occlusion surface, and projecting it to generate the obstacle avoidance normal residual to correct the pose data, the robot can adaptively adjust its lateral pose according to the approach of the actual dynamic obstacle. This improves the robot's obstacle avoidance flexibility in narrow environments or environments with approaching dynamic occlusion objects, reduces the risk of collisions with obstacles due to forcibly maintaining normal constraints, and enhances navigation safety.

[0056] In conjunction with some implementations of the first aspect, in some implementations, the depth data of the occlusion object corresponding to the visual occlusion state in the current frame image is obtained, and the depth data of the occlusion object is fitted to a plane to obtain a dynamic physical occlusion surface, specifically including:

[0057] Calculate the relative motion vector of each depth point in the depth data of the occluder, and delete depth points whose relative motion vector is less than a preset vector threshold to obtain a clean dynamic point set;

[0058] The pure dynamic point set is projected onto the horizontal plane along the preset gravity direction to obtain a two-dimensional dynamic occupied grid.

[0059] Extract the edge grid point closest to the smart community robot in the two-dimensional dynamic occupied grid, and perform two-dimensional straight line fitting on the edge grid point to obtain the dynamic approach boundary line;

[0060] The dynamic approach boundary line is spatially stretched along the preset gravity direction to obtain an envelope plane perpendicular to the horizontal plane, and the envelope plane is used as a dynamic physical occlusion surface.

[0061] By calculating the relative motion vectors of each depth point in the occlusion depth data and removing depth points with motion vectors less than a threshold, interference from static background environment and depth sensor noise can be effectively filtered out. This accurately extracts the pure dynamic point set that truly constitutes a dynamic occlusion threat, reducing the interference of irrelevant data on subsequent processing. This pure dynamic point set is projected onto a horizontal plane along the gravity direction to generate a two-dimensional dynamic occupancy grid. The edge grid points closest to the robot are then extracted and fitted with two-dimensional straight lines. This simplifies the complex and irregular three-dimensional dynamic obstacle point cloud into a two-dimensional boundary line that most closely approximates the robot, reducing the dimensionality and computational complexity of data processing. Finally, this dynamic approximate boundary line is spatially stretched along the gravity direction to generate an envelope plane perpendicular to the horizontal plane, constructing a dynamic physical occlusion surface that conforms to physical reality. This allows subsequent obstacle avoidance distance residual calculations to closely match the actual danger boundary of the obstacle, improving the accuracy and real-time performance of the dynamic physical occlusion surface construction and enhancing the system's obstacle avoidance response efficiency in dynamic and complex environments.

[0062] In a second aspect, embodiments of this application provide a smart community robot visual navigation system, which includes: one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and one or more processors call the computer instructions to cause the system to perform the method described in the first aspect and any possible implementation thereof.

[0063] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a system, cause the system to perform the method described in the first aspect and any possible implementation thereof.

[0064] Fourthly, embodiments of this application provide a computer program product that, when run on a system, causes the system to execute the method described in any possible implementation of the first aspect.

[0065] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0066] 1. This application provides a visual navigation method for smart community robots. By extracting static semantic regions and calculating spatial similarity, the image is divided into repetitive texture regions and non-repetitive texture regions. Initial pose estimation is performed using non-repetitive texture regions, reducing the probability of mismatches in repetitive texture regions that is common in traditional feature point methods, thus improving the reliability of the initial pose. Based on this, a three-dimensional reference plane is constructed using the depth data of the repetitive texture regions, and illumination interference pixels are eliminated by calculating photometric errors through reprojection, reducing the negative impact of ambient lighting changes on pose estimation. Subsequently, the spatial distance residual from the interference-eliminating repetitive texture regions to the three-dimensional reference plane is constructed and projected onto the normal vector direction to generate a normal residual. This normal residual is used to correct the initial pose data. By utilizing the structured features of repetitive texture regions such as walls and fences commonly found in community environments, effective geometric constraints are imposed on the robot's pose, thereby improving the visual navigation accuracy of smart community robots in complex environments containing a large number of repetitive textures.

[0067] 2. This application provides a visual navigation method for smart community robots. By transforming a 3D reference plane to a global coordinate system and spatially stitching it together to obtain a global static structural map, it achieves effective memorization of historical structural information of the community environment. When the robot is determined to be in a visual occlusion state due to the low pixel proportion of repetitive texture areas in the current frame image, the corresponding extended virtual reference surface is extracted from the global static structural map using dead reckoning data. Combined with the visual features of visible non-repetitive texture areas, pose estimation is performed to obtain initial blind zone pose data, reducing the risk of pose estimation failure caused by a sharp reduction in effective visual features under occlusion. By constructing a virtual distance residual from the robot's spatial position coordinates to the extended virtual reference surface and projecting it to generate a virtual normal residual to correct the initial blind zone pose data, the method can maintain the constraint of the robot's normal translation degree of freedom using prior global map structural information under visual constraints. This suppresses the cumulative drift error generated by dead reckoning during occlusion and improves the robot's positioning accuracy and navigation stability under visual occlusion conditions.

[0068] 3. This application provides a visual navigation method for smart community robots. By acquiring the depth data of occlusion objects corresponding to visual occlusion states and fitting it to obtain a dynamic physical occlusion surface, the normal projection distance between the occlusion surface and the extended virtual reference surface is calculated. This allows for real-time and intuitive quantification of the current passable dynamic corridor width, accurately assessing the risk of collisions in the environment. When the corridor width is determined to be less than a preset safe passage threshold, the normal translation degree of freedom constraint of the extended virtual reference surface on the target blind zone pose data is promptly released. Subsequently, by constructing the physical distance residual from the robot's spatial position to the dynamic physical occlusion surface and projecting it to generate an obstacle avoidance normal residual to correct the pose data, the robot can adaptively adjust its lateral pose according to the approach of actual dynamic obstacles. This improves the robot's obstacle avoidance flexibility in narrow environments or environments with approaching dynamic occlusion objects, reduces the risk of collisions with obstacles due to forcibly maintaining normal constraints, and enhances navigation safety. Attached Figure Description

[0069] Figure 1 This is a flowchart illustrating a visual navigation method for a smart community robot in an embodiment of this application.

[0070] Figure 2 This is another flowchart illustrating a visual navigation method for a smart community robot in an embodiment of this application.

[0071] Figure 3 This is another flowchart illustrating a visual navigation method for a smart community robot in this application.

[0072] Figure 4 This is a schematic diagram of the physical device structure of a smart community robot visual navigation system provided in an embodiment of this application. Detailed Implementation

[0073] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0074] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0075] The following example is used in conjunction with Figure 1 The present application describes a visual navigation method for a smart community robot in its embodiments:

[0076] Please see Figure 1 This is a flowchart illustrating a visual navigation method for a smart community robot in an embodiment of this application.

[0077] S101. Extract static semantic regions from the continuous visual image sequence collected by the smart community robot using a preset semantic recognition model;

[0078] Pre-defined semantic recognition models refer to artificial intelligence network models that have been trained on a large amount of data beforehand and are capable of classifying pixels in an image and assigning semantic labels. These include, but are not limited to, image segmentation models based on convolutional neural networks, fully convolutional networks, or visual transformers. Continuous visual image sequences refer to a collection of multiple frames of images acquired sequentially by the visual sensors mounted on a smart community robot during its movement. These include, but are not limited to, color image sequences, grayscale image sequences, or depth image sequences. Static semantic regions refer to the set of pixels in an image that belong to objects with fixed physical locations, including but not limited to areas such as building exterior walls, the ground, fixed fences, or streetlights.

[0079] A pre-defined semantic recognition model based on convolutional neural networks can be used to extract static semantic regions from continuous visual image sequences collected by smart community robots. The system acquires continuous visual image sequences collected by the visual sensors mounted on the smart community robot as input data. The underlying structure of this pre-defined semantic recognition model adopts an encoder-decoder architecture. The encoder uses a deep residual network to extract high-level image features at multiple scales layer by layer, while the decoder gradually restores the spatial resolution of the image through multi-level upsampling operations and feature stitching techniques, ultimately outputting a pixel-level semantic category probability map with the same size as the original image. The model's training set contains a massive number of labeled images of smart community scenes, with detailed annotations covering multiple categories of both static and dynamic objects. The test set consists of independently collected images of the same scene, used to verify the model's segmentation accuracy under different lighting and viewing angles. The system inputs the continuous visual image sequence frame by frame into the pre-defined semantic recognition model, and the model outputs a category label corresponding to each pixel. The system iterates through the output results, extracts pixel sets labeled as static categories such as walls, ground, or buildings, and merges these pixel sets at their corresponding positions in the original image to generate complete static semantic regions.

[0080] A pre-defined semantic recognition model based on a visual attention mechanism can also be used to extract static semantic regions from continuous visual image sequences collected by smart community robots. The underlying structure of this pre-defined semantic recognition model adopts a visual transformer architecture, dividing the input continuous visual image sequence into multiple fixed-size image patches. A multi-head self-attention mechanism captures global semantic associations and long-distance dependencies between image patches, and finally, a multilayer perceptron decodes and outputs pixel-level semantic segmentation masks. The training set consists of panoramic images of smart communities under various weather and lighting conditions, along with corresponding manually calibrated pixel-level static semantic masks; the test set is used to evaluate the model's generalization ability and robustness in unknown community environments. The system inputs the continuous visual image sequence into the pre-defined semantic recognition model, which directly outputs static and dynamic object masks after forward propagation. The system extracts the static object mask and performs pixel-level masking processing on the original continuous visual image sequence based on this mask, filtering out areas outside the mask, thereby accurately extracting the static semantic regions from the continuous visual image sequence.

[0081] S102. Extract image features of static semantic regions and calculate the spatial similarity of image features. Regions with spatial similarity greater than a preset similarity threshold are considered as repetitive texture regions.

[0082] The system extracts image features from static semantic regions and calculates the spatial similarity of these features. Regions with spatial similarity greater than a preset similarity threshold are classified as repeating texture regions, while regions with spatial similarity less than or equal to the preset similarity threshold are classified as non-repeating texture regions. Specifically, the system extracts the gradient magnitude and gradient direction of each pixel in the static semantic region as image features. It then performs one-dimensional projection of the image features along multiple preset projection directions to generate gradient distribution sequences along each projection direction. Autocorrelation analysis is performed on the gradient distribution sequences to extract periodic peak features. Spatial similarity is calculated based on the peak intensity and peak spacing of the periodic peak features. Regions with spatial similarity greater than the preset similarity threshold are classified as repeating texture regions, while regions with spatial similarity less than or equal to the preset similarity threshold are classified as non-repeating texture regions.

[0083] Image features here specifically refer to information that reflects local texture and structural changes in an image, specifically the gradient magnitude and direction of pixels, including but not limited to edge intensity or local contrast features. Spatial similarity is a measure of the repeatability and consistency of image features in spatial distribution. A preset similarity threshold is a numerical boundary used to distinguish whether textures are repeatable. It is derived from statistical analysis of historical feature data of typical repeating textures (such as brick walls and blinds) in a community environment. When the spatial similarity is greater than this threshold, it is determined to be a repeating texture region; when it is less than this threshold, it is determined to be a non-repeating texture region. Gradient magnitude represents the drastic change in pixel grayscale value, and gradient direction represents the spatial direction of the fastest grayscale value change. Projection direction refers to the angular direction of one-dimensional dimensionality reduction and aggregation of two-dimensional features, including but not limited to horizontal, vertical, or specific diagonal directions. Gradient distribution sequence is a one-dimensional data set formed after feature projection. Periodic peak features refer to regularly occurring local maxima in the sequence. Peak intensity represents the numerical magnitude of the maxima, and peak spacing represents the distance between adjacent maxima.

[0084] Image features of static semantic regions can be extracted and repetitive texture regions can be segmented by calculating the statistical distribution of image features. The system uses an edge detection operator to traverse the pixels of the static semantic region, calculates the partial derivatives in the horizontal and vertical directions, and synthesizes the gradient magnitude and gradient direction of each pixel as image features. The system sets multiple preset projection directions, such as horizontal, vertical, and diagonal, and projects the gradient magnitudes onto the corresponding one-dimensional coordinate axes according to the gradient direction, forming a gradient distribution sequence in each projection direction. The system applies an autocorrelation function to the gradient distribution sequence to calculate the correlation score between the sequence and itself after shifting by different hysteresis. The system identifies local maxima in the correlation score sequence as periodic peak features. The system extracts the peak intensity of these maxima and the peak spacing between adjacent maxima, and calculates the spatial similarity by weighted summing of the mean of the peak intensity and the variance of the peak spacing. The system compares the calculated spatial similarity with a preset similarity threshold. Regions with spatial similarity greater than the preset similarity threshold are marked and output as repetitive texture regions, while regions with spatial similarity less than the preset similarity threshold are marked and output as non-repetitive texture regions.

[0085] The system can also extract image features from static semantic regions and classify repetitive texture regions through frequency domain transformation analysis. The system uses a Gaussian derivative filter to perform convolution operations on the static semantic region, directly obtaining the smoothed and denoised gradient magnitude and direction as image features. The system generates multiple preset projection directions according to preset angular intervals, and uses Lardon transform to integrate the two-dimensional gradient information along these directions, generating a one-dimensional gradient distribution sequence. The system transforms the gradient distribution sequence to the frequency domain, calculates its power spectral density, and then obtains an autocorrelation sequence through inverse Fourier transform, extracting significant periodic peak features. The system constructs a feature vector using the peak intensity and peak spacing of the periodic peak features, inputs it into a pre-defined similarity evaluation function, and outputs a normalized spatial similarity value. The system judges this spatial similarity value; if the value is greater than a preset similarity threshold, the corresponding region is classified as a repetitive texture region; if the value is not greater than the preset similarity threshold, it is classified as a non-repetitive texture region, thus completing the accurate texture attribute classification of the static semantic region.

[0086] S103. Based on the visual features of non-repeating texture regions, perform pose estimation between adjacent frame images to obtain the initial pose data of the current frame image;

[0087] Visual features refer to unique and identifiable local image patterns, including but not limited to corners, edges, patches, or contours of specific shapes. Adjacent frame images refer to two images with immediately adjacent timestamps in a continuous visual image sequence, typically representing the previous frame and the current frame. Pose estimation is the process of calculating the position and orientation changes of a camera (i.e., a robot) in three-dimensional space. Initial pose data refers to the preliminary calculated six-degree-of-freedom transformation data containing three-dimensional translation vectors and three-dimensional rotation matrices.

[0088] The system utilizes a feature point matching algorithm to estimate the pose between adjacent frames based on visual features of non-repeating texture regions, obtaining the initial pose data for the current frame. The system extracts scale-invariant feature points as visual features within non-repeating texture regions of adjacent frames and generates a corresponding high-dimensional feature descriptor for each feature point. Employing a nearest neighbor search strategy, the system calculates the Euclidean distance between the feature descriptors of the current and previous frames to find the best-matching feature point pair. To eliminate potential mismatches, the system applies a random sampling consensus algorithm to rigorously filter the matched feature point pairs, retaining only the set of interior points that meet epipolar geometry constraints. The system then uses this filtered set of interior points to construct an essential matrix or fundamental matrix. The system performs singular value decomposition on the constructed matrix to solve for the rotation matrix and translation vector between adjacent frames, combining these two to output the initial pose data for the current frame.

[0089] The system can also utilize direct photometric alignment based on visual features of non-repeating texture regions to estimate the pose between adjacent frames, obtaining the initial pose data for the current frame. The system extracts pixels with significant gradient changes in the non-repeating texture regions of the previous frame as visual features. Assuming grayscale invariance between adjacent frames, the system constructs a photometric error function between the current and previous frames at the corresponding pixel locations of these visual features. The system parameterizes the pose transformation into Lie algebra form and uses the Gauss-Newton method or the Levenberg-Marquardt algorithm for nonlinear optimization iteration of the photometric error function. In each iteration, the system calculates the Jacobian matrix of the photometric error with respect to the pose parameters, continuously updating the pose increment until the photometric error is minimized or the preset number of iterations is reached. The system converts the finally converged pose parameters into a rotation matrix and a translation vector, outputting the initial pose data for the current frame.

[0090] S104. Construct a three-dimensional reference plane corresponding to the repeating texture region based on the depth data of the repeating texture region;

[0091] The system constructs a 3D reference plane corresponding to the repeating texture region based on the depth data of the repeating texture region, specifically including:

[0092] The depth data of the repetitive texture region is converted into an initial 3D point set in the 3D point cloud space. The local curvature of each 3D point in the initial 3D point set is calculated, and 3D points with local curvature greater than a preset curvature threshold are deleted to obtain a smooth 3D point set. The smooth 3D point set is divided into multiple depth intervals along the preset initial normal vector direction, and the point cloud density in each depth interval is calculated. The target depth interval with the highest point cloud density is extracted, and the 3D points in the target depth interval are used as the core point set. Principal component analysis is performed on the core point set to fit and generate a 3D reference plane corresponding to the repetitive texture region.

[0093] Depth data refers to numerical information reflecting the physical distance from the surface of an object in a scene to the camera sensor, including but not limited to depth maps, binocular parallax maps, or LiDAR point clouds. A 3D reference plane is a mathematical plane model fitted in 3D space representing the physical surface of a region with repeating textures. The initial 3D point set is a spatial point array converted from depth data. Local curvature is a geometric quantity that measures the degree of curvature of a 3D surface at a given point. A preset curvature threshold is a criterion used to distinguish between flat and curved surfaces, derived from statistical analysis of the curvature of typical planar structures such as community walls and fences. A smoothed 3D point set is a flat point array retained after removing high-curvature points. The preset initial normal vector direction is a pre-estimated plane orientation, including but not limited to direction vectors obtained based on the camera's optical axis or coarse plane fitting. A depth interval is multiple distance segments divided along the normal vector. Point cloud density refers to the number of 3D points contained within a specific depth interval. The target depth interval is the distance segment with the highest point cloud density. The core point set is the optimal set of 3D points used for final plane fitting.

[0094] A 3D reference plane corresponding to the repetitive texture region can be constructed based on the depth data of the repetitive texture region through voxelized mesh statistics. The system, combined with the camera's intrinsic parameter matrix, transforms the 2D coordinates and depth data of each pixel within the repetitive texture region, mapping them to a 3D point cloud space to generate an initial 3D point set. The system uses the K-nearest neighbor algorithm to search for the neighboring points of each 3D point in the initial 3D point set, calculates the eigenvalues ​​of the covariance matrix of the neighboring points, and uses the ratio of the minimum eigenvalue to the sum of all eigenvalues ​​as the local curvature. The system compares the local curvature with a preset curvature threshold, deleting 3D points with local curvature greater than the preset threshold, retaining the smooth 3D point set. The system obtains a preset initial normal vector direction, projects the smooth 3D point set onto this direction, and divides the projection space into multiple depth intervals according to a fixed distance step. The system counts the number of 3D points contained in each depth interval as the point cloud density, selects the target depth interval with the highest point cloud density, and extracts the 3D points in this interval as the core point set. The system calculates the centroid of the core point set, performs principal component analysis on the centroid-removed core point set, extracts the eigenvector corresponding to the minimum eigenvalue as the plane normal vector, and combines the centroid coordinates to generate a three-dimensional reference plane corresponding to the repeated texture region.

[0095] A 3D reference plane corresponding to the repetitive texture region can also be constructed based on the depth data of the repetitive texture region using kernel density estimation. The system utilizes the inverse projection transformation formula from depth map to point cloud to convert the depth data of the repetitive texture region pixel-by-pixel into an initial 3D point set. The system fits a local quadratic surface to each point in the initial 3D point set, using the Gaussian curvature of the local quadratic surface as the local curvature of that point. The system filters out 3D points with local curvature greater than a preset curvature threshold, generating a smooth 3D point set. After determining the preset initial normal vector direction, the system calculates the projection distance of all points in the smooth 3D point set in that direction. The system uses a Gaussian kernel function to perform kernel density estimation on the projection distance, generating a continuous point cloud density distribution curve, which is then divided into multiple continuous depth intervals. The system locates the interval containing the highest peak of the density distribution curve as the target depth interval, and 3D points falling into this target depth interval are assigned to the core point set. The system constructs a three-dimensional coordinate matrix of the core point set, performs singular value decomposition on it, uses the left singular vector corresponding to the minimum singular value to determine the normal vector parameters of the plane, and combines the mean coordinates of the core point set to fit and generate a three-dimensional reference plane corresponding to the repeating texture region.

[0096] S105. Based on the initial pose data, reproject the repeating texture region in the current frame image to the three-dimensional reference plane of the adjacent previous frame image to obtain the reprojected pixels.

[0097] Reprojection refers to the geometric transformation process of mapping a two-dimensional pixel from one camera's perspective to the image plane of another camera's perspective, using the camera's intrinsic and extrinsic parameters and the three-dimensional spatial structure. A reprojected pixel is the specific coordinate position of a pixel in the current frame's image within the two-dimensional pixel coordinate system of the previous frame's image after the three-dimensional spatial transformation.

[0098] The homography matrix transformation can be used to reproject the repetitive texture regions in the current frame image onto the 3D reference plane of the adjacent previous frame image based on the initial pose data, thus obtaining the reprojected pixels. The system calculates the homography matrix between the current and previous frames based on the normal vector of the 3D reference plane of the previous frame image and its perpendicular distance to the camera's optical center, combined with the rotation and translation vectors from the initial pose data. The system extracts the 2D homogeneous coordinates of all pixels in the repetitive texture regions of the current frame image. The system performs matrix multiplication on these 2D homogeneous coordinates with the calculated homography matrix to calculate the new homogeneous coordinates corresponding to the previous frame image coordinate system. The system performs perspective division normalization on the new homogeneous coordinates, converting them into 2D pixel coordinates on the plane of the previous frame image, thereby obtaining the accurate reprojected pixels.

[0099] Alternatively, the 3D spatial ray intersection method can be used to reproject the repetitive texture regions in the current frame image onto the 3D reference plane of the adjacent previous frame image based on the initial pose data, obtaining the reprojected pixels. The system backprojects the pixels of the repetitive texture regions in the current frame image into a set of 3D spatial rays in the current camera coordinate system based on the camera's intrinsic parameter matrix. The system then uses the pose transformation matrix from the initial pose data to uniformly transform these 3D spatial rays to the camera coordinate system of the previous frame image. The system establishes mathematical analytical equations for the ray equations and the 3D reference plane, solving them simultaneously to calculate the coordinates of the 3D intersection point of each ray with the 3D reference plane. The system again uses the camera's intrinsic parameter matrix to perform perspective projection on these 3D intersection point coordinates, mapping them onto the 2D pixel plane of the previous frame image to obtain the accurate reprojected pixel positions.

[0100] S106. Calculate the photometric error between the reprojected pixel and the corresponding position in the previous frame image.

[0101] Photometric error refers to the difference in pixel intensity observed at the same physical point in different image frames. It is usually expressed as a difference in grayscale values ​​or color channel values, including but not limited to absolute grayscale difference, squared grayscale difference, or normalized cross-correlation measure. Corresponding position refers to the precise geometric coordinates of the reprojected pixel in the coordinate system of the previous frame image.

[0102] The photometric error between the reprojected pixel and the corresponding position in the previous frame image can be calculated using the absolute gray-level difference method. The system obtains the precise floating-point coordinates of the reprojected pixel in the coordinate system of the previous frame image. Since floating-point coordinates typically do not fall on integer pixel grids, the system employs a bilinear interpolation algorithm, using the gray-level values ​​of the four neighboring integer pixels around the floating-point coordinates to calculate the interpolated gray-level value at that corresponding position. The system extracts the gray-level values ​​of the pixels corresponding to the original repeating texture region in the current frame image. The system subtracts the interpolated gray-level value at the corresponding position in the previous frame image from the gray-level value of the current frame image and takes the absolute value. The system outputs and stores this absolute difference as the photometric error between the reprojected pixel and the corresponding position in the previous frame image.

[0103] The system can also use the image patch normalized cross-correlation method to calculate the photometric error between the reprojected pixel and the corresponding position in the previous frame. The system extracts a local image patch of a preset size centered on the pixel in the repeating texture region of the current frame. The system then extracts a local image patch of the same size centered on the corresponding position of the reprojected pixel in the previous frame. The system calculates the mean and standard deviation of the grayscale values ​​of all pixels within these two local image patches. The system uses the mean and standard deviation to perform zero-mean normalization on the two image patches and calculates the normalized cross-correlation coefficient. The system converts the cross-correlation coefficient into an error metric, for example, by subtracting the calculated cross-correlation coefficient from a constant, and uses the resulting value as the photometric error between the reprojected pixel and the corresponding position in the previous frame.

[0104] S107. Pixels with light intensity errors greater than a preset error threshold are identified as light interference pixels, and light interference pixels are removed from the repeated texture area.

[0105] The preset error threshold is a numerical boundary used to determine whether a pixel is severely affected by illumination interference. This threshold is derived from historical statistical analysis of camera sensor noise models and typical illumination fluctuations in a community environment. It represents the maximum normal observation noise that the system can tolerate. Illumination-interference pixels refer to pixels whose appearance characteristics are drastically altered due to external environmental factors (such as the movement of leaf shadows, strong reflection of sunlight, or changes in local light sources), including but not limited to highlight reflection points or deep shadow points.

[0106] A fixed threshold mask filtering method can be used to identify pixels with photometric errors exceeding a preset error threshold as lighting interference pixels and remove them from the repetitive texture region. The system iterates through the calculated photometric error values ​​of all reprojected pixels. Each photometric error value is compared to a pre-set global fixed error threshold. Pixels with photometric errors strictly exceeding this threshold are marked as lighting interference pixels. A binary mask image with the same size as the repetitive texture region is created. The mask value corresponding to the lighting interference pixels is set to zero, while the mask value for other positions not exceeding the threshold is set to one. The system uses this binary mask image to perform a bitwise AND operation on the pixel set of the repetitive texture region, thereby directly filtering and completely removing lighting interference pixels.

[0107] An adaptive threshold dynamic removal method can also be used to identify pixels with photometric errors exceeding a preset error threshold as lighting interference pixels and remove them from repeated texture regions. The system collects the photometric errors of all pixels within the repeated texture region of the current frame and calculates the overall mean and standard deviation of the photometric error distribution. The system dynamically generates a preset error threshold for the current frame based on the mean and standard deviation; for example, the mean is increased by a factor of the standard deviation. The system checks the photometric error of each pixel to determine if it exceeds the dynamically generated preset error threshold; if it does, it is identified as a lighting interference pixel. The system maintains a list of valid pixel coordinates in memory. During traversal, only pixel coordinates that do not exceed the threshold are added to this list. By updating the list of valid pixel coordinates, the system dynamically removes lighting interference pixels from repeated texture regions.

[0108] S108. Construct the spatial distance residual from the repeated texture region to the 3D reference plane after removing pixels that cause lighting interference;

[0109] Spatial distance residual refers to the geometric distance deviation between a 3D point in the current frame image and the 3D reference plane constructed from the previous frame image. It reflects the magnitude of the error in the alignment of the current initial pose data in 3D space.

[0110] The spatial distance residual from the repeated texture region to the 3D reference plane can be constructed using the point-to-plane orthogonal distance calculation method after removing pixels affected by illumination. The system uses initial pose data and camera depth information to back-project the effective 2D pixels within the repeated texture region (after removing pixels affected by illumination) to the 3D space in the current camera coordinate system, generating a 3D point cloud set for the current frame. The system obtains the mathematical expression for the 3D reference plane constructed from the previous frame image, including the plane's unit normal vector parameter and the perpendicular distance parameter from the plane to the origin. The system substitutes the 3D point cloud coordinates of the current frame into the expression for this 3D reference plane, calculating the orthogonal projection distance of each 3D point along the plane's normal vector direction to the 3D reference plane. The system aggregates all calculated orthogonal projection distances to construct the spatial distance residual reflecting pose deviation.

[0111] The spatial distance residual from the repeated texture region to the 3D reference plane can also be constructed using the 3D coordinate transformation difference method. The system combines the pixel depth map of the repeated texture region after removing illumination interference pixels to convert it into a 3D point set for the current frame. Using the transformation matrix in the initial pose data, the system uniformly transforms the 3D point set of the current frame to the 3D coordinate system of the previous frame. On the 3D reference plane of the previous frame, the system finds the nearest in-plane projection point for each transformed 3D point. The system calculates the 3D Euclidean distance vector between the transformed 3D point coordinates and the corresponding in-plane projection point coordinates. The system extracts the magnitude of this distance vector as a scalar error, sums up the scalar errors of all 3D points, and uses this to construct the spatial distance residual reflecting the pose deviation.

[0112] S109. Project the spatial distance residual onto the normal vector direction of the three-dimensional reference plane to generate the normal residual;

[0113] The normal vector direction refers to the spatial direction perpendicular to the three-dimensional reference plane, representing the dimension where the planar structure changes most drastically. The normal residual refers to the projection component of the spatial distance residual onto the normal vector direction, i.e., considering only geometric errors perpendicular to the plane.

[0114] The spatial distance residual can be projected onto the normal vector direction of a 3D reference plane using a vector dot product projection method, generating a normal residual. The system extracts the unit normal vector parameters of the 3D reference plane constructed from the previous frame image. The system obtains the constructed spatial distance residual and represents it as a 3D error vector from the projection point on the 3D reference plane to the 3D point in the current frame. The system performs a vector dot product operation on each 3D error vector with the unit normal vector of the 3D reference plane. The result of the dot product directly extracts the projected scalar value of the error vector in the normal vector direction, filtering out other components perpendicular to the normal vector. The system collects and combines these projected scalar values ​​to generate a sequence of normal residuals for subsequent optimization.

[0115] The spatial distance residual can also be projected onto the normal vector direction of the 3D reference plane using a coordinate system rotation transformation method to generate a normal residual. Based on the normal vector of the 3D reference plane, the system constructs a local 3D coordinate system, ensuring that the Z-axis of this local coordinate system is perfectly parallel to the normal vector of the 3D reference plane. The system calculates the rotation matrix from the current camera coordinate system to this local coordinate system. The system then transforms the 3D coordinate points containing the spatial distance residual to the local coordinate system using this rotation matrix. In the local coordinate system, the system directly extracts the Z-axis component of the transformed coordinates; this Z-axis component represents the distance deviation strictly perpendicular to the 3D reference plane. The system discards the X-axis and Y-axis components, outputting the Z-axis component of all points as the normal residual.

[0116] S110. Correct the initial pose data using the normal residual to obtain the target pose data;

[0117] Correction refers to the mathematical process of adjusting and optimizing the initially estimated parameters using the calculated error data. Target pose data refers to the highly accurate six-degree-of-freedom position and attitude information of the robot, which is finally output after multiple rounds of iterative optimization.

[0118] The Gauss-Newton optimization algorithm can be used to correct the initial pose data using the normal residuals to obtain the target pose data. The system uses the initial pose data as the initial iteration value for optimization and constructs an objective function that minimizes the sum of squares of the normal residuals. The system derives the Jacobian matrix of the normal residuals with respect to the small perturbations of the pose Lie algebra, which reflects the specific influence of pose changes on the normal residuals. In each iteration, the system uses the Jacobian matrix and the current normal residuals to construct a normal equation, and calculates the pose increment by solving the normal equation. The system updates the current pose data with the calculated pose increment through a Lie group exponent mapping, and repeats the iterative calculation until the pose increment is less than the minimum value or the maximum number of iterations is reached. The final converged pose is output as the target pose data.

[0119] The Kalman filter framework can also be used to correct the initial pose data using the normal residual to obtain the target pose data. The system uses the initial pose data as the state prediction value of the filter and the normal residual as the observation residual of the filter. The system establishes an observation model, mapping the pose state to the expected normal residual space and calculating the observation matrix. The system combines the state prediction covariance and the observation noise covariance to calculate the Kalman gain matrix. The system uses the Kalman gain matrix to weight the observation residuals and calculates the state correction. The system superimposes the state correction onto the initial pose data for state updating, and the optimal state estimate after filtering and updating is output as the target pose data.

[0120] S111: Control the smart community robot to navigate based on the target pose data.

[0121] Navigation refers to the comprehensive process by which a smart community robot autonomously plans a safe and efficient driving path based on its current location and set destination, and controls the chassis to execute the path. This includes, but is not limited to, global path planning, local obstacle avoidance, and motion control.

[0122] A global path tracking control method can be used to control a smart community robot for navigation based on target pose data. The system inputs the target pose data into the robot's global coordinate system and updates the robot's current precise position and heading angle on a high-precision map of the smart community in real time. The system retrieves a pre-planned global reference path and calculates the lateral and heading deviations between the current target pose and the global reference path. The system uses a pure tracking algorithm or a model predictive control algorithm to calculate the linear and angular velocity commands required for the robot chassis based on these deviations. The system sends the velocity commands to the robot's motor drive module, driving the wheels to rotate according to the commands, enabling the robot to smoothly travel along the predetermined path and achieve navigation.

[0123] Local dynamic obstacle avoidance planning can also be used to control the smart community robot for navigation based on target pose data. The system uses target pose data to convert local environmental obstacle data collected by the robot's onboard LiDAR or depth camera into the robot's body coordinate system, constructing a real-time local cost map. Combining the current starting point determined by the target pose data and the set local target point, the system applies a dynamic window method or artificial potential field method to search for a local path within the local cost map. During the search process, the system evaluates the safety, distance from the target, and kinematic feasibility of multiple candidate trajectories, selecting the optimal trajectory with the lowest cost. Based on the optimal trajectory, the system generates control commands to guide the smart community robot to avoid dynamic obstacles and navigate towards the target point.

[0124] In the above embodiments, by extracting static semantic regions and calculating spatial similarity, the image is divided into repeating texture regions and non-repeating texture regions. Initial pose estimation is performed using non-repeating texture regions, reducing the probability of mismatches that easily occur in repeating texture regions using traditional feature point methods and improving the reliability of the initial pose. Based on this, a three-dimensional reference plane is constructed using the depth data of the repeating texture regions, and illumination interference pixels are eliminated by calculating photometric errors through reprojection, reducing the negative impact of ambient lighting changes on pose estimation. Subsequently, the spatial distance residual from the interference-eliminating repeating texture regions to the three-dimensional reference plane is constructed and projected onto the normal vector direction to generate a normal residual. This normal residual is used to correct the initial pose data. By utilizing the structured features of repeating texture regions such as walls and fences commonly found in community environments, effective geometric constraints are imposed on the robot's pose, thereby improving the visual navigation accuracy of the smart community robot in complex environments containing a large number of repeating textures.

[0125] In the above embodiments, by co-processing repeating and non-repeating texture regions and constructing a 3D reference plane for normal residual correction, high-precision pose estimation and navigation of the robot in conventional complex texture environments are achieved. However, in actual smart community environments, robots often encounter unexpected situations such as pedestrian occlusion and large-area loss of local environmental features, leading to a significant reduction in effective texture features within the current field of view, thereby increasing the risk of visual positioning failure. To maintain high-precision positioning and navigation even under visual occlusion conditions, this application provides another visual navigation method for smart community robots based on the above embodiments. The following is a combination of... Figure 2 Another visual navigation method for smart community robots in this application embodiment is described below:

[0126] Please see Figure 2 This is another flowchart illustrating a visual navigation method for a smart community robot in this application.

[0127] S201. Based on the target pose data, the three-dimensional reference plane is transformed into the global coordinate system for spatial stitching to obtain a global static structure map;

[0128] A global coordinate system refers to a unified world coordinate reference system established or pre-defined when a smart community robot starts up. It is used to uniformly describe spatial data at different times and locations, including but not limited to a fixed coordinate system with the robot's starting position as the origin or a pre-constructed high-precision absolute coordinate system for the community. Spatial stitching refers to the process of uniformly transforming and fusing local spatial data acquired from different perspectives or at different times into a common reference system, including but not limited to point cloud registration and fusion or clustering and merging of planar parameters. A global static structural map is a complete environmental geometric model formed by the continuous accumulation and fusion of multiple local 3D reference planes under a global coordinate system, including but not limited to a topological map containing rich planar parameters or a globally occupied grid map.

[0129] The system utilizes coordinate transformation and planar parameter clustering to transform a 3D reference plane into a global coordinate system based on target pose data, enabling spatial stitching and generating a global static structure map. The system extracts rotation and translation vectors from the target pose data to construct a homogeneous transformation matrix from the current camera coordinate system to the global coordinate system. It then obtains the local normal vector and distance parameters to the origin of the current 3D reference plane, performs mathematical transformations on these parameters using the homogeneous transformation matrix, and calculates the global normal vector and global distance parameters of the 3D reference plane in the global coordinate system. The system inputs the transformed global plane parameters into the global static structure map database. It iterates through the historical plane parameters stored in the database, calculating the angle between the normal vectors and the distance difference between newly added planes and historical planes. For planes with minimal angle and distance differences, the system uses a weighted average to merge and update their parameters; for planes with significant differences, they are saved independently as new structural features, thus completing spatial stitching and continuously updating the global static structure map.

[0130] The system can also utilize core point set registration and global refitting to transform the 3D reference plane to the global coordinate system based on the target pose data, and then perform spatial stitching to obtain a global static structure map. The system extracts the 3D coordinates of the core point set retained when constructing the current 3D reference plane. Using the target pose data, the system uniformly maps each 3D point in this core point set to the 3D space of the global coordinate system using rigid body transformation formulas. The system merges the mapped core point set with the global point cloud library maintained in the global static structure map. To eliminate redundant points and noise generated during the stitching process, the system periodically applies voxel downsampling filtering to the global point cloud library. On the filtered global point cloud library, the system uses a random sampling consensus algorithm to re-extract and fit multiple large-scale planar models globally. The system combines these refitted large-scale planar models and outputs and stores them as the updated global static structure map.

[0131] S202. If the pixel proportion of the repeating texture region in the current frame image is lower than the preset occlusion threshold, the smart community robot is determined to enter a visual occlusion state.

[0132] Pixel percentage refers to the ratio of the number of pixels in a specific area of ​​an image to the total number of pixels in the entire image. The preset occlusion threshold is a numerical limit used to determine whether the robot's field of vision is severely obstructed. This threshold is derived from historical statistical analysis of numerous visual failure cases such as pedestrian and vehicle occlusion in community environments, representing the minimum amount of texture information required to maintain basic visual localization. Visual occlusion state refers to an abnormal operating mode where the robot's visual sensors are largely obstructed by dynamic obstacles or sudden environmental changes, resulting in the inability to acquire sufficient effective environmental features. This includes, but is not limited to, the camera being completely blocked by nearby pedestrians or entering narrow blind spots with extremely scarce features.

[0133] A method using image mask area statistics can be used to determine if a smart community robot has entered a visual occlusion state when the pixel percentage of a repetitive texture region in the current frame image is lower than a preset occlusion threshold. The system acquires a binary mask image of the repetitive texture region identified in the previous steps, where pixels belonging to the repetitive texture region have a value of one, and pixels in other regions have a value of zero. The system iterates through all pixels in the binary mask image, accumulating the total number of pixels with a value of one to obtain the absolute pixel area of ​​the repetitive texture region. The system acquires the overall resolution information of the current frame image and calculates the total number of pixels in the entire image. The system divides the absolute pixel area of ​​the repetitive texture region by the total number of pixels to calculate the precise pixel percentage. The system compares this pixel percentage value with a preset occlusion threshold set in system memory. When the pixel percentage value is found to be significantly lower than the preset occlusion threshold, the system immediately generates an occlusion alarm signal and switches the robot's system status flag to visual occlusion state.

[0134] The image segmentation grid evaluation method can also be used to determine if the smart community robot has entered a visual occlusion state when the percentage of pixels representing repetitive texture regions in the current frame image is lower than a preset occlusion threshold. The system uniformly divides the current frame image into multiple fixed-size rectangular image block grids. The system analyzes each image block grid one by one, counting the number of pixels marked as repetitive texture regions within the grid. The system sets a grid validity criterion: when the number of repetitive texture pixels in a grid exceeds half of the total pixels in the grid, the grid is marked as a valid texture grid. The system counts the number of valid texture grids in the entire image and divides it by the total number of grids to obtain an approximate pixel percentage. The system compares this approximate pixel percentage with the preset occlusion threshold. If the approximate pixel percentage is lower than the preset occlusion threshold, the system determines that the current field of view has lost most of its structured features, thus confirming that the smart community robot has officially entered a visual occlusion state and activating the corresponding blind spot navigation algorithm module.

[0135] S203. Based on the dead reckoning data of the smart community robot, extract the extended virtual reference surface corresponding to the current position of the smart community robot from the global static structure map.

[0136] Based on the dead reckoning data of the smart community robot, the system extracts an extended virtual reference surface corresponding to the current position of the smart community robot from the global static structure map, specifically including:

[0137] Extract the pose covariance matrix from the dead reckoning data, and construct a spatial error ellipsoid centered on the current position based on the pose covariance matrix; perform spatial intersection calculation between the spatial error ellipsoid and the global static structure map to obtain multiple candidate reference surfaces located within the spatial error ellipsoid; obtain the last effective reference surface of the smart community robot before entering the visual occlusion state, and calculate the normal vector angle and boundary distance between each candidate reference surface and the last effective reference surface; select candidate reference surfaces whose normal vector angle is less than a preset angle threshold and whose boundary distance is less than a preset distance threshold as target candidate surfaces; use the target candidate surfaces as extended virtual reference surfaces corresponding to the current position of the smart community robot.

[0138] Dead reckoning data refers to the robot's position and attitude information continuously estimated through integration calculations using internal sensors (such as inertial measurement units or wheeled odometry) without relying on external environmental features. The pose covariance matrix is ​​a mathematical matrix describing the degree of uncertainty of each state variable in the dead reckoning data and their interrelationships. The spatial error ellipsoid is a geometric envelope constructed in three-dimensional space based on the pose covariance matrix, representing the robot's possible position range at a certain confidence level. A candidate reference surface is a global map plane located inside the spatial error ellipsoid that may match the current environmental structure. The last valid reference surface is a high-precision three-dimensional reference plane successfully constructed and confirmed by the robot just before triggering visual occlusion. The angle between normal vectors is the spatial angular difference between the normal vectors of two planes. The boundary distance is the shortest geometric distance between two planes on their spatial physical boundary. The preset angle threshold is the tolerance limit for determining whether two planes are parallel, derived from the flatness statistics of community wall structures. The preset distance threshold is the spatial tolerance for determining whether two planes belong to the same physical surface, derived from the maximum allowable physical gap between adjacent wall segments. The extended virtual reference surface is a priori geometrically constrained plane extracted from the global map and used to replace real visual observations during occlusion.

[0139] A combination of algebraic analysis and vector operations can be used to extract the pose covariance matrix from dead reckoning data, and a spatial error ellipsoid centered on the current position can be constructed based on the pose covariance matrix. The spatial error ellipsoid is then spatially intersected with a global static structure map to obtain multiple candidate reference surfaces within the ellipsoid. The last effective reference surface before the smart community robot enters a visual occlusion state is obtained, and the normal vector angle and boundary distance between each candidate reference surface and the last effective reference surface are calculated. Candidate reference surfaces with normal vector angles less than a preset angle threshold and boundary distances less than a preset distance threshold are selected as target candidate surfaces. These target candidate surfaces are then used as extended virtual reference surfaces corresponding to the current position of the smart community robot. The system extracts the pose covariance matrix at the current moment from a filter fused from the chassis odometer and inertial navigation module. The system performs eigenvalue decomposition on the positional portion of this covariance matrix, uses eigenvectors to determine the principal axis direction of the ellipsoid, and uses the square root of the eigenvalue multiplied by the confidence coefficient to determine the principal axis length. An algebraic equation for the spatial error ellipsoid is then generated centered on the current reckoning position. The system substitutes the boundary vertex coordinates of each plane in the global static structure map into the ellipsoid equation, filtering out planes with at least one vertex satisfying the ellipsoidal inequality as candidate reference surfaces. The system retrieves the normal vector and bounding box data of the last valid reference surface from memory. The system uses the inverse cosine function of the vector dot product to calculate the angle between the normal vectors of the candidate reference surface and the last valid reference surface, and calculates the shortest Euclidean distance between their bounding boxes as the boundary distance. The system compares the calculation results with preset angle thresholds and preset distance thresholds, filtering out target candidate surfaces that simultaneously meet both threshold conditions, and confirming them as extended virtual reference surfaces.

[0140] The system can also utilize 3D voxelization search and topology matching to extract the pose covariance matrix from dead reckoning data, and construct a spatial error ellipsoid centered on the current position based on the pose covariance matrix. The spatial error ellipsoid is then spatially intersected with the global static structure map to obtain multiple candidate reference surfaces within the ellipsoid. The last effective reference surface before the smart community robot enters a visual occlusion state is obtained, and the normal angle and boundary distance between each candidate reference surface and the last effective reference surface are calculated. Candidate reference surfaces with normal angles less than a preset angle threshold and boundary distances less than a preset distance threshold are selected as target candidate surfaces. These target candidate surfaces are then used as extended virtual reference surfaces corresponding to the current position of the smart community robot. The system obtains the pose covariance matrix output from the dead reckoning data, and generates a discretized 3D voxel bounding box around the current position to approximate the spatial error ellipsoid based on the variance range determined by the diagonal elements of the matrix. The system queries the octree index of the global static structure map for leaf nodes that spatially overlap with this 3D voxel bounding box, and extracts the planes contained within these nodes as candidate reference surfaces. The system reads the topological properties and geometric parameters of the last valid reference surface. The system converts the normal vectors of the candidate reference surfaces into quaternion representations, calculates the spherical distance between the candidate surface and the quaternion of the last valid reference surface as the angle between the normal vectors, and simultaneously calculates the projection length of the line connecting the centroids of the two planes in the direction perpendicular to the normal vector as the boundary distance. The system discards surfaces whose normal vector angle is greater than a preset angle threshold or whose boundary distance is greater than a preset distance threshold, retaining the target candidate surfaces that meet the conditions and designating them as extended virtual reference surfaces for blind-spot navigation.

[0141] S204. Extract the visual features of the visible non-repeating texture regions in the current frame image to perform pose estimation and obtain the initial blind zone pose data under visual occlusion.

[0142] Visible non-repeating texture regions refer to a collection of scattered pixels in an image that are not obscured by dynamic obstacles and possess unique local texture features under visual occlusion conditions. These include, but are not limited to, distant building corners, unobscured ground patches, or road sign edges. Initial blind zone pose data refers to the preliminary estimates of the robot's position and orientation calculated solely based on these remaining visible non-repeating texture features when visual features are extremely scarce.

[0143] The system utilizes sparse optical flow tracking and epipolar geometric constraints to extract visual features of visible non-repeating texture regions in the current frame image for pose estimation, obtaining initial blind zone pose data under visual occlusion. The system uses an image mask to block out occluded areas, applying a corner detection algorithm only within visible non-repeating texture regions to extract a small number of significant corner points as visual features. The system employs a pyramid optical flow algorithm to perform pixel-level tracking of these sparse corner points between the previous and current frames, establishing feature correspondences between adjacent frames. Due to the sparse number of feature points and their potential for noise, the system incorporates the camera's intrinsic parameter matrix and prior motion trends provided by dead reckoning data to construct an epipolar geometric constraint model. The system uses a random sample consensus algorithm combined with epipolar geometric constraints to eliminate mismatched points in optical flow tracking and solves the essential matrix using the retained correct match pairs. The system decomposes the essential matrix, calculates the rotation and translation changes of the current frame relative to the previous frame, and accumulates this with the absolute pose of the previous frame to obtain the initial blind zone pose data under visual occlusion.

[0144] The system can also utilize a deep learning-based relative pose regression model to extract visual features of visible non-repeating texture regions in the current frame image for pose estimation, obtaining initial blind spot pose data under visual occlusion conditions. The system calls a pre-trained relative pose regression model, whose underlying structure employs a deep convolutional neural network with an attention mechanism, automatically focusing on unoccluded effective texture regions in the image. The model's training set contains numerous consecutive image pairs simulating various severe occlusion scenarios (such as pedestrians touching faces or vehicles crossing) in a smart community, along with corresponding high-precision relative pose labels; the test set consists of independent image pairs collected in real occlusion scenarios. The system inputs the visible non-repeating texture regions of the previous and current frames into the model. Through forward propagation, the model uses an attention mechanism to extract deep semantic associations of residual visual features between the two frames, directly regressing and outputting a vector containing six degrees of freedom relative pose changes as output data. The system then performs coordinate transformation and synthesis on this relative pose change and the robot's historical poses to calculate the initial blind spot pose data under visual occlusion conditions.

[0145] S205. Construct the virtual distance residual from the spatial location coordinates of the smart community robot to the extended virtual reference surface;

[0146] Spatial position coordinates refer to the three-dimensional translation vectors in the initial blind zone pose data that represent the specific position of the robot's optical center or chassis center in the global coordinate system. Virtual distance residuals refer to the geometric distance deviation between the robot's spatial position coordinates and the extended virtual reference surface; they quantify the degree of spatial mismatch between the current estimated position and the prior global map structure.

[0147] The virtual distance residual between the spatial position coordinates of a smart community robot and an extended virtual reference surface can be constructed using the substitution method of algebraic equations from a point to a plane. The system extracts the robot's three-dimensional spatial position coordinates from the initial blind zone pose data, representing them as coordinate vectors containing X, Y, and Z components. The system obtains the parameters of the standard algebraic equation of the extended virtual reference surface in the global coordinate system, including the plane's normal vector components and constant terms. The system substitutes each component of the robot's three-dimensional spatial position coordinates into the standard algebraic equation of the plane, calculates the sum of the products of the coordinate components and the normal vector components, and adds the constant term. The system uses the absolute value of the calculated algebraic sum as the orthogonal distance from the robot's spatial position coordinates to the extended virtual reference surface, and directly defines and outputs this orthogonal distance as the virtual distance residual.

[0148] The system can also utilize geometric projection and Euclidean distance calculation to construct a virtual distance residual from the spatial position coordinates of the smart community robot to the extended virtual reference surface. The system extracts the spatial position coordinates from the initial blind zone pose data. Based on the geometric properties of the extended virtual reference surface, the system calculates the coordinates of the three-dimensional orthogonal projection point of this spatial position coordinate point on the extended virtual reference surface. The system constructs a three-dimensional spatial vector pointing from the original spatial position coordinate point to this orthogonal projection point. The system calculates the L2 norm of this three-dimensional spatial vector, i.e., calculates the straight-line Euclidean distance between the two points. The system stores the calculated Euclidean distance value as a virtual distance residual reflecting the degree of pose drift for subsequent use by the optimization module.

[0149] S206. Project the virtual distance residual onto the normal vector direction of the extended virtual reference surface to generate a virtual normal residual;

[0150] The normal vector direction of the extended virtual reference surface refers to the spatial direction vector that is strictly perpendicular to the surface of the virtual plane in the global coordinate system. The virtual normal residual refers to the scalar error component that is retained only in the normal vector direction after the virtual distance residual in three-dimensional space has been constrained and stripped.

[0151] The virtual distance residual can be projected onto the normal vector direction of the extended virtual reference surface using the vector dot product projection method, generating a virtual normal residual. The system obtains the unit normal vector of the extended virtual reference surface, whose magnitude has been normalized to one. The system represents the virtual distance residual calculated in the previous steps as a three-dimensional error vector pointing from the extended virtual reference surface to the robot's spatial position coordinates. The system performs a vector dot product operation between the three-dimensional error vector and the unit normal vector. Since the magnitude of the unit normal vector is one, the result of the dot product operation directly outputs the scalar projection length of the error vector in the normal vector direction. The system extracts this projection length scalar and outputs it as the pure virtual normal residual.

[0152] The virtual distance residual can also be projected onto the normal vector direction of the extended virtual reference surface using the local coordinate system decomposition method, generating a virtual normal residual. The system establishes a local reference coordinate system with the projection point on the extended virtual reference surface as the origin, the normal vector of that plane as the Z-axis, and any two orthogonal directions within the plane as the X-axis and Y-axis. The system calculates the rotation transformation matrix from the global coordinate system to this local reference coordinate system. Using this rotation transformation matrix, the system transforms the 3D error vector containing the virtual distance residual into the local reference coordinate system. In the local reference coordinate system, the system directly extracts the Z-axis coordinate component of the transformed vector. Since the Z-axis is strictly aligned with the normal vector direction, the system directly uses this Z-axis coordinate component as the virtual normal residual, discarding the X-axis and Y-axis components.

[0153] S207. The initial blind zone pose data is corrected using virtual normal residuals to obtain target blind zone pose data, so as to maintain the normal translation degree of freedom constraint on the smart community robot under visual occlusion.

[0154] Target blind zone pose data refers to the high-precision robot position and orientation information output after depth optimization through virtual geometric constraints under visual occlusion. Normal translational degree of freedom constraints restrict the robot's irregular sliding in the direction perpendicular to the reference plane, forcing its trajectory to conform to the physical boundary constraints of the global map.

[0155] A graph optimization algorithm can be used to correct the initial blind zone pose data using virtual normal residuals to obtain the target blind zone pose data, thus maintaining the normal translational degree of freedom constraint for the smart community robot under visual occlusion. The system constructs a pose graph optimization framework, using the initial blind zone pose data as pose nodes in the graph and dead reckoning data as relative motion edges between nodes. The system adds an extended virtual reference surface as a fixed landmark node to the graph and constructs the calculated virtual normal residuals as unary constraint edges connecting pose nodes and landmark nodes. The system defines a global nonlinear least squares objective function including a normal residual penalty term. The system uses the Levenburg-Marquardt algorithm to iteratively solve this objective function, calculating the pose node update amount that minimizes the virtual normal residual. The system superimposes the update amount onto the initial pose node and outputs the converged optimization result as the target blind zone pose data, thereby achieving normal constraint.

[0156] The Extended Kalman Filter (EKF) state update method can also be used to correct the initial blind zone pose data using virtual normal residuals to obtain the target blind zone pose data, thus maintaining the normal translational degree of freedom constraint for the smart community robot under visual occlusion. The system sets the initial blind zone pose data as the prior state prediction value of the EKF filter and extracts the corresponding state prediction covariance matrix. The system uses the virtual normal residual as the actual observation input of the filter and constructs a linearized observation matrix that maps the pose state to the normal distance space. The system calculates the Kalman gain matrix at the current time by combining the preset virtual observation noise covariance. The system uses the Kalman gain matrix to transform the virtual normal residual into a state correction vector and adds it to the prior state prediction value to complete the posterior state update. The system outputs the updated posterior state estimate as the target blind zone pose data, thereby maintaining the normal translational degree of freedom constraint within the filtering framework.

[0157] In the above embodiments, a global static structural map is obtained by spatially stitching the three-dimensional reference plane to the global coordinate system, thus achieving effective memorization of historical structural information of the community environment. When the proportion of pixels in the repeating texture region in the current frame image is too low, indicating that the robot has entered a visual occlusion state, the corresponding extended virtual reference surface is extracted from the global static structural map using dead reckoning data. This is combined with the visual features of the visible non-repeating texture region to perform pose estimation and obtain the initial blind zone pose data, reducing the risk of pose estimation failure caused by a sharp reduction in effective visual features under occlusion. By constructing a virtual distance residual from the robot's spatial position coordinates to the extended virtual reference surface and projecting it to generate a virtual normal residual to correct the initial blind zone pose data, the robot's normal translation degree of freedom can be maintained using prior global map structural information even under visually restricted conditions. This suppresses the cumulative drift error generated by dead reckoning during occlusion and improves the robot's positioning accuracy and navigation stability under visual occlusion conditions.

[0158] Through the above embodiments, when encountering visual occlusion, the robot can maintain the normal translation degree of freedom constraint by using the extended virtual reference surface extracted from the global map, effectively suppressing dead reckoning drift in blind spots and ensuring basic navigation stability. However, when the object causing visual occlusion is a dynamically approaching physical obstacle (such as an oncoming pedestrian or vehicle), and the physical space accessible to the robot is severely compressed, continuing to forcibly maintain the original virtual normal constraint will result in the robot lacking avoidance space, easily leading to collision accidents. Therefore, in order to further improve the robot's obstacle avoidance flexibility and safety in narrow, dynamically approaching environments, this application provides another visual navigation method for smart community robots based on the above embodiments. The following is a combination of... Figure 3 This application describes yet another visual navigation method for smart community robots:

[0159] Please see Figure 3 This is another flowchart illustrating a visual navigation method for a smart community robot in this application.

[0160] S301. Obtain the depth data of the occlusion object corresponding to the visual occlusion state in the current frame image, and perform plane fitting on the depth data of the occlusion object to obtain the dynamic physical occlusion surface.

[0161] The system acquires the depth data of the occlusion objects corresponding to the visual occlusion state in the current frame image, and performs planar fitting on the depth data to obtain the dynamic physical occlusion surface, specifically including:

[0162] Calculate the relative motion vector of each depth point in the depth data of the occlusion object, and delete depth points whose relative motion vector is less than a preset vector threshold to obtain a clean dynamic point set; project the clean dynamic point set onto the horizontal plane along a preset gravity direction to obtain a two-dimensional dynamic occupancy grid; extract the edge grid point closest to the smart community robot in the two-dimensional dynamic occupancy grid, and perform two-dimensional straight line fitting on the edge grid point to obtain a dynamic approach boundary line; stretch the dynamic approach boundary line in space along the preset gravity direction to obtain an envelope plane perpendicular to the horizontal plane, and use the envelope plane as the dynamic physical occupancy surface.

[0163] Occlusion depth data refers to the set of 3D spatial distance point clouds of dynamic obstacle surfaces covering the front of the lens, collected by a depth camera or LiDAR under visual occlusion conditions. Dynamic physical occlusion surface refers to the 3D spatial geometric plane representing the approach front of the dynamic obstacle, generated by mathematical fitting of the above point cloud. Relative motion vector refers to the displacement direction and velocity magnitude of each 3D point in the depth data relative to the global static environment between consecutive time frames. Preset vector threshold is a numerical boundary used to distinguish between static noise in the environment and real dynamic objects, derived from historical statistical analysis of inherent sensor measurement noise and the amplitude of minute vibrations of typical static objects in the environment. Pure dynamic point set is the set of 3D points that strictly belong to the moving obstacle after removing static background and noise points. Preset gravity direction refers to the spatial vector direction perpendicular to the horizontal ground downwards, usually obtained by calibration of the robot's internal inertial measurement unit. 2D dynamic occupancy grid is a grid map on a horizontal plane, compressed along the gravity direction, used to represent the space occupied by dynamic objects. Dynamic approach boundary line is a straight line fitted on a 2D horizontal plane representing the foremost contour of the obstacle. The envelope plane is a three-dimensional vertical plane generated by re-stretching a two-dimensional straight line along the direction of gravity.

[0164] The system can utilize a geometric projection method based on scene optical flow and random sampling consistency to obtain the depth data of occluded objects corresponding to the visual occlusion state in the current frame image, and perform planar fitting on the occlusion depth data to obtain a dynamic physical occlusion surface. The system acquires the occlusion depth data output by the depth sensor, calculates the 3D scene optical flow by combining it with the image grayscale information of adjacent frames, and extracts the relative motion vectors of each depth point. The system compares the magnitude of the relative motion vector of each depth point with a preset vector threshold one by one, deleting depth points with a magnitude smaller than the preset vector threshold, and retaining the remaining points to form a pure dynamic point set. The system reads the preset gravity direction vector output by the inertial measurement unit, orthogonally projects all 3D points in the pure dynamic point set onto the robot's local horizontal plane along the preset gravity direction, discretizes the horizontal plane, counts the number of projected points in each grid, and generates a 2D dynamic occupancy grid. The system traverses the 2D dynamic occupancy grid radially outward from the robot's coordinate origin, extracting the edge grid points closest to the robot's origin that are marked as occupied. The system employs a random sampling consensus algorithm to iteratively fit these edge grid points, eliminating outliers to obtain an optimal two-dimensional straight line as the dynamic approach boundary line. Using this dynamic approach boundary line as a reference, the system performs a fixed-height three-dimensional spatial stretching operation along the opposite direction of the preset gravity direction, generating an envelope plane perpendicular to the horizontal plane. The system outputs this envelope plane as a dynamic physical occlusion surface.

[0165] The system can also utilize a boundary stretching method based on iterative nearest-point registration and total least squares to obtain the depth data of occluded objects corresponding to the visual occlusion state in the current frame image, and perform planar fitting on the occluded object depth data to obtain a dynamic physical occlusion surface. The system uses the iterative nearest-point algorithm to calculate the global self-motion compensation matrix between the depth data of occluded objects in two adjacent frames, transforming the 3D point cloud of the current frame to the coordinate system of the previous frame, and obtaining the relative motion vector of each depth point by calculating the coordinate difference between corresponding points. The system filters out depth points whose relative motion vector values ​​are less than a preset vector threshold to obtain a clean dynamic point set. Based on a pre-calibrated preset gravity direction, the system rotates the coordinate system of the clean dynamic point set until the Z-axis is parallel to the gravity direction, then discards the Z-axis coordinates, flattens the point cloud onto the XY plane, and constructs a two-dimensional dynamic occupancy grid. The system applies a convex hull detection algorithm to extract the convex hull vertices of the two-dimensional dynamic occupancy grid facing the robot, and uses these vertices as the edge grid points closest to the smart community robot. The system uses total least squares to fit two-dimensional straight lines to the extracted edge grid points, minimizing the sum of squared orthogonal distances from all points to the line, and calculates the dynamic approach boundary line. The system then combines the analytical equation of this dynamic approach boundary line with vector parameters of the preset gravity direction, constructing an infinitely extending vertical envelope plane through algebraic analytical geometry, and extracting the portion corresponding to the robot's height as the dynamic physical occlusion surface.

[0166] S302. Calculate the normal projection distance between the dynamic physical occlusion surface and the extended virtual reference surface, and use it as the real-time dynamic corridor width.

[0167] Normal projection distance refers to the geometric length of a straight line between a specific plane and another plane or point in three-dimensional space, measured along the direction of the normal vector of that plane. Real-time dynamic corridor width refers to the actual width of the remaining passable physical space currently occupied by the robot under extreme conditions of visual occlusion and the presence of dynamically approaching obstacles, formed by the combined compression of static virtual walls and dynamic physical walls.

[0168] The normal projection distance between the dynamic physical occlusion surface and the extended virtual reference surface can be calculated using the analytical intersection method of plane equations, and used as the real-time dynamic corridor width. The system obtains the standard algebraic analytical equations of the dynamic physical occlusion surface and the extended virtual reference surface generated in the previous steps in the global coordinate system. The system extracts the unit normal vector parameter from the analytical equation of the extended virtual reference surface. The system selects a feature point representing the geometric center of the obstacle on the dynamic physical occlusion surface and extracts the three-dimensional spatial coordinates of this feature point. The system substitutes the three-dimensional spatial coordinates of this feature point into the point-to-plane distance calculation formula of the extended virtual reference surface, uses the dot product operation of the feature point coordinates and the unit normal vector of the extended virtual reference surface, and combines it with the constant term of the plane to calculate the absolute orthogonal distance from the feature point to the extended virtual reference surface along the normal vector direction. The system directly defines and outputs this calculated absolute orthogonal distance value as the real-time dynamic corridor width.

[0169] The system can also utilize the discrete point cloud uniform sampling projection method to calculate the normal projection distance between the dynamic physical occlusion surface and the extended virtual reference surface, which serves as the real-time dynamic corridor width. Within the effective physical boundary of the dynamic physical occlusion surface, the system performs uniform mesh sampling according to a preset fixed spatial step size, generating a set of discrete 3D sampled point clouds covering the plane. The system acquires the geometric parameters of the extended virtual reference surface and constructs a spatial projection transformation matrix. Using this spatial projection transformation matrix, the system projects each discrete 3D sampled point on the dynamic physical occlusion surface onto the extended virtual reference surface along the normal vector direction of the extended virtual reference surface, and calculates the 3D Euclidean distance between each sampled point and its corresponding projection point. The system iterates through all calculated Euclidean distance values, extracts the minimum distance value, and uses this minimum distance value as the most conservative and safest normal projection distance, outputting it as the real-time dynamic corridor width to the navigation control module.

[0170] S303. When it is determined that the real-time dynamic corridor width is less than the preset safe passage threshold, release the constraint of the normal translation degree of freedom of the extended virtual reference surface on the target blind zone pose data.

[0171] The preset safe passage threshold is the limit width used to determine whether a robot can pass through a narrow space without collision. This threshold is derived from the maximum width of the smart community robot's physical chassis plus a pre-set safety buffer margin. Constraint removal refers to removing or disabling specific geometric constraints in the mathematical model of pose optimization or state estimation, allowing the target variable to regain the freedom to adjust in a specific direction.

[0172] The graph optimization objective function weight dynamic adjustment method can be used to remove the normal translation degree of freedom constraint of the extended virtual reference surface in the target blind zone pose data when the real-time dynamic corridor width is determined to be less than the preset safe passage threshold. The system acquires the calculated real-time dynamic corridor width value in real time and compares it with the preset safe passage threshold set in memory. When the judgment logic determines that the real-time dynamic corridor width is strictly less than the preset safe passage threshold, the system immediately sends a constraint removal command to the underlying pose graph optimizer. After receiving the command, the pose graph optimizer locates the error term in the objective function representing the normal constraint of the extended virtual reference surface. The system forcibly modifies all elements in the information matrix (i.e., weight matrix) corresponding to this error term to zero. By resetting the weights to zero, this normal constraint will no longer have any impact on the gradient descent of the target blind zone pose data during the nonlinear least squares iterative solution process, thus completely removing the normal translation degree of freedom constraint at the mathematical level.

[0173] Furthermore, the Kalman filter observation model reset method can be used to remove the normal translation degree of freedom constraint of the extended virtual reference surface on the target blind zone pose data when the real-time dynamic corridor width is determined to be less than the preset safe passage threshold. The system continuously monitors the changes in the real-time dynamic corridor width. Once the width value is detected to drop below the preset safe passage threshold, the system triggers the filter's observation model reconstruction mechanism. The system enters the update step of the extended Kalman filter and identifies the virtual normal residual input channel from the extended virtual reference surface in the observation vector. By modifying the dimension of the observation matrix, the system directly removes the row vector corresponding to the virtual normal residual from the observation matrix and deletes the corresponding noise term in the observation noise covariance matrix. After the observation model reset is completed, the filter will completely ignore the measurement information of the extended virtual reference surface in subsequent state update calculations, so that the target blind zone pose data is no longer stretched and constrained by the reference surface in the normal translation direction.

[0174] S304. Construct the physical distance residual from the spatial location coordinates of the smart community robot to the dynamic physical occlusion surface;

[0175] Spatial position coordinates refer to the three-dimensional translation vector representing the absolute position of the robot's body center in the three-dimensional global coordinate system within the target blind zone pose data. Physical distance residual refers to the geometric distance deviation between the robot's spatial position coordinates and the dynamic physical occlusion surface; it quantifies the spatial urgency between the robot's current position and the approaching front of the dynamic obstacle.

[0176] The algebraic distance calculation method from a point to a plane can be used to construct the physical distance residual from the spatial position coordinates of a smart community robot to a dynamic physical occlusion surface. The system extracts the 3D spatial position coordinates representing the robot's center position from the current target blind zone pose data. The system obtains the standard algebraic equation of the dynamically occluded surface generated by the previous fitting in the global coordinate system, extracting the unit normal vector parameter and constant term parameter of the plane. The system directly substitutes the robot's 3D spatial position coordinates into the standard algebraic equation of the dynamically occluded surface, calculates the dot product of the coordinate vector and the unit normal vector, and adds the dot product result to the constant term. The system takes the absolute value of the algebraic sum, which represents the shortest orthogonal straight-line distance from the robot's center to the dynamically occluded surface. The system records and outputs this orthogonal straight-line distance value as the physical distance residual.

[0177] The system can also utilize 3D vector projection calculation to construct the physical distance residual from the spatial position coordinates of the smart community robot to the dynamic physical occlusion surface. The system extracts the spatial position coordinates from the target blind zone pose data. The system arbitrarily selects a known reference point on the dynamic physical occlusion surface and extracts its 3D coordinates. The system uses the robot's spatial position coordinates minus the 3D coordinates of the reference point to construct a 3D spatial vector pointing from the dynamic physical occlusion surface to the robot. The system obtains the unit normal vector of the dynamic physical occlusion surface. The system calculates the dot product between the constructed 3D spatial vector and the unit normal vector. The absolute value of the dot product is the projected length of the 3D spatial vector along the normal vector direction, which is precisely equivalent to the perpendicular distance from the robot to the plane. The system outputs the calculated projected length scalar as the physical distance residual.

[0178] S305. Project the physical distance residual onto the normal vector direction of the dynamic physical occlusion surface to obtain the obstacle avoidance normal residual;

[0179] The normal vector direction of a dynamic physical occlusion surface refers to the spatial vector direction perpendicular to the fitted plane of the dynamic obstacle and typically pointing towards the robot. Obstacle avoidance normal residual refers to the scalar error component retained only in the normal vector direction of the dynamic physical occlusion surface after constraining and projecting the physical distance residual.

[0180] The physical distance residual can be projected onto the normal vector direction of the dynamic physical occlusion surface using the vector dot product projection method to obtain the obstacle avoidance normal residual. The system extracts the unit normal vector parameter of the dynamic physical occlusion surface, ensuring that the magnitude of this vector has been precisely normalized to one. The system represents the physical distance residual calculated in the previous steps as a three-dimensional error vector pointing from the orthogonal projection point on the dynamic physical occlusion surface to the robot's spatial position coordinates. The system performs a vector dot product operation between this three-dimensional error vector and the unit normal vector of the dynamic physical occlusion surface. Since the magnitude of the unit normal vector is one, the result of the dot product operation directly extracts the projected scalar value of the three-dimensional error vector in the normal vector direction, while automatically eliminating all other components perpendicular to the normal vector. The system outputs the calculated projected scalar value as the pure obstacle avoidance normal residual.

[0181] The physical distance residual can also be projected onto the normal vector direction of the dynamic physical occlusion surface using a local coordinate system rotation transformation method to obtain the obstacle avoidance normal residual. The system uses any point on the dynamic physical occlusion surface as the origin, the unit normal vector of that surface as the Z-axis, and selects two mutually orthogonal vectors as the X-axis and Y-axis within the plane to construct a local reference coordinate system attached to the dynamic physical occlusion surface. The system calculates the 3D rotation matrix for transforming from the global coordinate system to this local reference coordinate system. Using this 3D rotation matrix, the system transforms the entire 3D error vector containing the physical distance residual to the local reference coordinate system. In the local reference coordinate system, the system directly reads the Z-axis coordinate component of the transformed vector. Since the Z-axis is strictly aligned with the normal vector direction of the dynamic physical occlusion surface, the system directly extracts this Z-axis coordinate component as the obstacle avoidance normal residual, while discarding the X-axis and Y-axis coordinate data.

[0182] S306. The target blind zone pose data is corrected by using the obstacle avoidance normal residual to control the lateral obstacle avoidance navigation of the smart community robot.

[0183] Correction refers to the mathematical iterative process of adjusting and optimizing existing pose estimation results using newly introduced error terms. Lateral avoidance navigation refers to the comprehensive control process that controls the robot to generate lateral displacement perpendicular to the original trajectory while maintaining the overall forward trend in order to avoid obstacles approaching from directly in front.

[0184] A nonlinear graph optimization framework can be used to correct the target blind zone pose data using obstacle avoidance normal residuals to control the lateral avoidance navigation of a smart community robot. The system uses the current target blind zone pose data as the node to be optimized in the graph optimization model. The calculated obstacle avoidance normal residual is added to the graph optimization model as a new unary repulsion edge, and this repulsion edge is assigned a very high information matrix weight, constructing a global objective function that includes an obstacle avoidance penalty term. The system uses the Gauss-Newton method to iteratively solve this objective function, calculating the pose correction increment that minimizes the obstacle avoidance normal residual, and then superimposing this increment onto the original pose to complete the correction. The system sends the corrected pose data to a pure tracking controller. The controller compares the corrected pose with the preset path, calculates the significant lateral deviation, and then outputs chassis drive commands with lateral angular velocity based on the lateral deviation to control the robot to perform lateral avoidance navigation actions.

[0185] Furthermore, an artificial potential field combined with a filtering update method can be used to correct the target blind zone pose data using the obstacle avoidance normal residual, thereby controlling the lateral avoidance navigation of the smart community robot. The system transforms the obstacle avoidance normal residual into a virtual repulsive potential energy term in the artificial potential field. During the state update phase of the extended Kalman filter, the gradient of this virtual repulsive potential energy term is used as an additional pseudo-observation input to construct a corrected observation matrix. The system calculates the Kalman gain and uses the pseudo-observation input to perform a post-hoc state update on the target blind zone pose data, causing the updated pose to deviate away from obstacles in the mathematical space. The system inputs the deviated pose data into a dynamic window method local path planner. Based on the deviated pose, the planner resamples the chassis velocity space, evaluates a locally optimal trajectory that can offset the pose deviation, and generates corresponding linear and angular velocity commands to drive the robot chassis to achieve lateral avoidance navigation.

[0186] In the above embodiments, by acquiring the depth data of the occlusion object corresponding to the visual occlusion state and fitting it to obtain the dynamic physical occlusion surface, and then calculating the normal projection distance between the occlusion surface and the extended virtual reference surface, the real-time dynamic corridor width that the robot can currently pass through can be quantified in real time and intuitively, and the collision risk in the environment can be accurately assessed. When it is determined that the corridor width is less than the preset safe passage threshold, the normal translation degree of freedom constraint of the extended virtual reference surface on the target blind zone pose data is released in a timely manner. Subsequently, by constructing the physical distance residual from the robot's spatial position to the dynamic physical occlusion surface, and projecting it to generate the obstacle avoidance normal residual to correct the pose data, the robot can adaptively adjust its lateral pose according to the approach of the actual dynamic obstacle, which improves the obstacle avoidance flexibility of the robot in narrow environments or environments with approaching dynamic occlusion objects, reduces the risk of collision with obstacles due to forcibly maintaining normal constraints, and improves the safety of navigation.

[0187] The system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 4 This is a schematic diagram of the physical device structure of a smart community robot visual navigation system provided in an embodiment of this application.

[0188] It should be noted that, Figure 4 The structure of the system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0189] like Figure 4 As shown, the system includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 402 or a program loaded from storage portion 408 into Random Access Memory (RAM) 403, such as executing the methods described in the above embodiments. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0190] The following components are connected to I / O interface 405: input section 406 including a camera, infrared sensor, etc.; output section 407 including a liquid crystal display (LCD) and speakers, etc.; storage section 408 including a hard disk, etc.; and communication section 409 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. Communication section 409 performs communication processing via a network such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0191] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the various functions defined in the present invention.

[0192] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. The transmitted data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0193] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0194] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the system described in the above embodiments; or it may exist independently and not assembled into the system. The storage medium carries one or more computer programs that, when executed by a processor of a system, cause the system to implement the methods provided in the above embodiments.

[0195] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0196] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0197] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0198] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A visual navigation method for smart community robots, characterized in that, include: Static semantic regions are extracted from continuous visual image sequences collected by smart community robots using a pre-defined semantic recognition model; Extract image features from the static semantic region and calculate the spatial similarity of the image features. Regions with spatial similarity greater than a preset similarity threshold are considered repetitive texture regions, and regions with spatial similarity less than or equal to the preset similarity threshold are considered non-repetitive texture regions. Based on the visual features of the non-repeating texture region, pose estimation is performed between adjacent frame images to obtain the initial pose data of the current frame image. Construct a three-dimensional reference plane corresponding to the repeating texture region based on the depth data of the repeating texture region; Based on the initial pose data, the repeated texture region in the current frame image is reprojected onto the three-dimensional reference plane of the adjacent previous frame image to obtain reprojected pixels. Calculate the photometric error between the reprojected pixel and the corresponding position in the previous frame image; Pixels with a photometric error greater than a preset error threshold are identified as light interference pixels and are removed from the repeating texture area. Construct the spatial distance residual from the repeated texture region to the three-dimensional reference plane after removing the illumination interference pixels; The spatial distance residual is projected onto the normal vector direction of the three-dimensional reference plane to generate a normal residual; The initial pose data is corrected using the normal residual to obtain the target pose data; The smart community robot is controlled to navigate based on the target pose data.

2. The method according to claim 1, characterized in that, The step of extracting image features from the static semantic region and calculating the spatial similarity of the image features specifically includes: Extract the gradient magnitude and gradient direction of each pixel in the static semantic region as image features; The image features are projected in one dimension along multiple preset projection directions to generate gradient distribution sequences in each projection direction. Autocorrelation analysis is performed on the gradient distribution sequence to extract the periodic peak features of the gradient distribution sequence; Spatial similarity is calculated based on the peak intensity and peak spacing of the periodic peak characteristics; Regions with spatial similarity greater than a preset similarity threshold are designated as repeating texture regions, and regions with spatial similarity less than or equal to the preset similarity threshold are designated as non-repeating texture regions.

3. The method according to claim 1, characterized in that, The step of constructing a three-dimensional reference plane corresponding to the repeating texture region based on the depth data of the repeating texture region specifically includes: The depth data of the repeating texture region is converted into an initial 3D point set in the 3D point cloud space; Calculate the local curvature of each three-dimensional point in the initial three-dimensional point set, and delete the three-dimensional points whose local curvature is greater than a preset curvature threshold to obtain a smooth three-dimensional point set; The smooth 3D point set is divided into multiple depth intervals along the direction of a preset initial normal vector, and the point cloud density in each depth interval is calculated. Extract the target depth range with the highest point cloud density, and use the three-dimensional points within the target depth range as the core point set; Principal component analysis is performed on the core point set to fit and generate a three-dimensional reference plane corresponding to the repeating texture region.

4. The method according to claim 1, characterized in that, After controlling the smart community robot to navigate based on the target pose data, the method further includes: Based on the target pose data, the three-dimensional reference plane is transformed into the global coordinate system for spatial stitching to obtain a global static structure map; If the pixel percentage of the repeating texture region in the current frame image is lower than a preset occlusion threshold, the smart community robot is determined to have entered a visual occlusion state. Based on the dead reckoning data of the smart community robot, an extended virtual reference surface corresponding to the current position of the smart community robot is extracted from the global static structure map; Visual features of the visible non-repeating texture regions in the current frame image are extracted for pose estimation to obtain the initial blind zone pose data under the visual occlusion state; Construct the virtual distance residual from the spatial position coordinates of the smart community robot to the extended virtual reference surface; The virtual distance residual is projected onto the normal vector direction of the extended virtual reference surface to generate a virtual normal residual; The initial blind zone pose data is corrected using the virtual normal residual to obtain the target blind zone pose data, so as to maintain the normal translational degree of freedom constraint on the smart community robot under the visual occlusion state.

5. The method according to claim 4, characterized in that, The step of extracting an extended virtual reference surface corresponding to the current position of the smart community robot from the global static structure map based on the dead reckoning data of the smart community robot specifically includes: Extract the pose covariance matrix from the dead reckoning data, and construct a spatial error ellipsoid centered on the current position based on the pose covariance matrix; The spatial error ellipsoid is spatially intersected with the global static structure map to obtain multiple candidate reference surfaces located within the spatial error ellipsoid. Obtain the last effective reference surface of the smart community robot before entering the visual occlusion state, and calculate the normal vector angle and boundary distance between each candidate reference surface and the last effective reference surface; Candidate reference surfaces whose included angle between the normal vectors is less than a preset angle threshold and whose boundary distance is less than a preset distance threshold are selected as target candidate surfaces. The target candidate surface is used as an extended virtual reference surface corresponding to the current position of the smart community robot.

6. The method according to claim 4, characterized in that, After correcting the initial blind zone pose data using the virtual normal residual to obtain the target blind zone pose data, the method further includes: Obtain the depth data of the occlusion object corresponding to the visual occlusion state in the current frame image, and perform planar fitting on the depth data of the occlusion object to obtain the dynamic physical occlusion surface; Calculate the normal projection distance between the dynamic physical occlusion surface and the extended virtual reference surface, and use it as the real-time dynamic corridor width; If it is determined that the width of the real-time dynamic corridor is less than the preset safe passage threshold, the constraint of the normal translation degree of freedom of the extended virtual reference surface on the pose data of the target blind zone is released. Construct the physical distance residual from the spatial location coordinates of the smart community robot to the dynamic physical occlusion surface; The physical distance residual is projected onto the normal vector direction of the dynamic physical occlusion surface to obtain the obstacle avoidance normal residual; The obstacle avoidance normal residual is used to correct the target blind zone pose data in order to control the smart community robot's lateral obstacle avoidance navigation.

7. The method according to claim 6, characterized in that, The step of obtaining the depth data of the occlusion object corresponding to the visual occlusion state in the current frame image, and performing planar fitting on the depth data of the occlusion object to obtain the dynamic physical occlusion surface, specifically includes: Calculate the relative motion vector of each depth point in the depth data of the occlusion, and delete depth points whose relative motion vector is less than a preset vector threshold to obtain a clean dynamic point set; The pure dynamic point set is projected onto the horizontal plane along a preset gravity direction to obtain a two-dimensional dynamic occupation grid. Extract the edge grid point closest to the smart community robot in the two-dimensional dynamic occupied grid, and perform two-dimensional straight line fitting on the edge grid point to obtain the dynamic approach boundary line; The dynamic approach boundary line is spatially stretched along the preset gravity direction to obtain an envelope plane perpendicular to the horizontal plane, and the envelope plane is used as a dynamic physical occlusion surface.

8. A smart community robot visual navigation system, characterized in that, The system includes: One or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the system, the system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the system, the system performs the method as described in any one of claims 1-7.