Geometric feature-based region segmentation and exploration estimation method and device, computer equipment and storage medium

Abstract

The invention discloses a region segmentation and exploration estimation method and device based on geometric features, computer equipment and a storage medium. The method comprises the following steps: constructing a wall map; performing region segmentation on the wall map to obtain a region segmentation result; robot position information and direction information are obtained, and a visible area is obtained through calculation according to the position information and the direction information; calculating according to the visible area to obtain an exploration area point set; performing calculation according to the region segmentation result and the exploration region point set to obtain a region exploration rate; setting an exploration priority according to the regional exploration rate; formulating a path planning and exploration strategy; and guiding the robot to act in combination with the exploration priority, the path planning and the exploration strategy. The method has the remarkable technical effects of improving the navigation efficiency, reducing redundant exploration, enhancing the practicability and reliability of the navigation system and the like. The technical scheme can be applied to the field of medical treatment and logistics robots.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation technology, and more specifically to a method, device, computer equipment and storage medium for region segmentation and exploration estimation based on geometric features. Background Art

[0002] In the field of object navigation, with the rapid development of technologies such as artificial intelligence and computer vision, existing research has made some progress in semantic map construction and target direction prediction. However, in practical applications, many problems still need to be solved, which seriously restrict the performance and efficiency of navigation systems.

[0003] From a navigation strategy perspective, traditional methods have significant limitations when searching specific areas. These strategies typically employ a mechanical traversal approach, making it difficult to accurately assess the current state of exploration. Specifically, during exploration, the robot often needs to completely search its current area before entering a new one. This strategy is extremely inefficient.

[0004] From the perspective of marginal utility theory, in the early stages of exploration, unit operations can yield a high level of valuable information, yielding significant returns. For example, when a robot first explores an unfamiliar environment, it may discover a wealth of new targets, paths, and other information. However, as exploration progresses, the value of the new information gained with each additional operation gradually decreases, exhibiting a phenomenon of diminishing marginal utility. This results in significant time and resources being wasted on low-value exploration, making the entire navigation process lengthy and inefficient.

[0005] When it comes to identifying target-free areas, existing technologies primarily rely on visual perception to build exploration maps. While visual perception technology can capture environmental information to a certain extent, due to precision errors and model limitations, it is difficult to accurately mark target-free areas on the map.

[0006] This problem is particularly acute in complex indoor environments. Because indoor environments typically feature complex layouts and numerous obstacles, the model's judgment of area boundaries is inaccurate. The robot may mistakenly identify previously searched areas as unexplored, leading to repeated exploration. This repeated exploration not only significantly wastes resources, such as energy consumption and equipment wear, but also significantly reduces the navigation system's adaptability and flexibility in diverse environments. When the environment changes, such as furniture repositioning or the introduction of new obstacles, the robot struggles to quickly adjust its navigation strategy, making it difficult to efficiently complete the navigation task.

[0007] In summary, existing object navigation technologies have many deficiencies in navigation strategies and object-free area recognition, which seriously affect the performance and efficiency of navigation systems. Therefore, it is necessary to improve existing technologies to address these issues and enhance the accuracy and efficiency of object navigation. Summary of the Invention

[0008] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device, computer equipment and storage medium for region segmentation and exploration estimation based on geometric features.

[0009] To achieve the above object, the present invention adopts the following technical solutions:

[0010] Region segmentation and exploration estimation methods based on geometric features, including:

[0011] Build a wall map;

[0012] Performing region segmentation on the wall map to obtain a region segmentation result;

[0013] Obtaining the robot's position information and direction information, and calculating the visible area based on the position information and the direction information;

[0014] Calculate the exploration area point set according to the visible area;

[0015] Calculating a regional exploration rate based on the regional segmentation result and the exploration region point set;

[0016] Setting exploration priorities based on the exploration rate of said area;

[0017] Develop path planning and exploration strategies;

[0018] The exploration priority, the path planning, and the exploration strategy are combined to guide the robot's actions.

[0019] The present invention also provides a region segmentation and exploration estimation device based on geometric features, comprising:

[0020] Construction unit, used to construct the wall map;

[0021] a segmentation unit, configured to perform regional segmentation on the wall map to obtain a regional segmentation result;

[0022] an acquisition calculation unit, configured to obtain position information and direction information of the robot, and calculate a visible area based on the position information and the direction information;

[0023] A first calculation unit, configured to calculate an exploration area point set according to the visible area;

[0024] A second calculation unit is used to calculate the region exploration rate according to the region segmentation result and the exploration region point set;

[0025] a setting unit, configured to set an exploration priority according to the area exploration rate;

[0026] formulation unit, used to formulate path planning and exploration strategies;

[0027] A guidance unit is combined to combine the exploration priority, the path planning and the exploration strategy to guide the robot action.

[0028] The present invention further provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0029] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0030] Compared with the prior art, the present invention has the following advantages: by transforming the search mode from point-to-point to area-level search and accurately segmenting the constructed wall map, a clear environmental segmentation result can be obtained, which enables the robot to understand the environmental structure from a macro perspective. Based on the regional segmentation results and the regional exploration rate calculated by the exploration area point set, the robot can clearly understand the exploration progress and potential value of each area, thereby formulating a more efficient exploration path plan, reducing the time spent in low-value areas, and greatly improving navigation efficiency. The exploration priority is set according to the regional exploration rate, and corresponding path planning and exploration strategies are formulated, so that the robot can respond to environmental changes more flexibly, avoiding the resource waste and inefficiency caused by fixed path planning in traditional methods. In addition, the regional exploration rate is calculated based on the regional segmentation results and the exploration area point set. This exploration estimation mechanism provides clear exploration guidance for the robot. The robot can judge the exploration degree of each area based on the regional exploration rate. When the exploration rate of a certain area reaches a certain threshold, it can be marked as a fully explored area and no longer searched repeatedly. At the same time, the exploration priority setting and path planning strategy are also based on the exploration estimation results, so that the robot can prioritize exploring areas with larger unknown spaces, avoiding unnecessary waste of time and resources. Whether in simple or complex environments, this navigation method based on exploration estimation can effectively reduce redundant exploration and improve the adaptability and flexibility of the navigation system.

[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 A schematic diagram of an application scenario of the geometric feature-based region segmentation and exploration estimation method provided by an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of a flow chart of a region segmentation and exploration estimation method based on geometric features provided in an embodiment of the present invention;

[0035] Figure 3 A schematic block diagram of a region segmentation and exploration estimation device based on geometric features provided by an embodiment of the present invention;

[0036] Figure 4 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0038] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0039] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0040] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0041] See also Figure 1 and Figure 2 , Figure 1 Schematic diagram of an application scenario of the region segmentation and exploration estimation method based on geometric features provided in an embodiment of the present invention. Figure 2 A schematic flow chart of a method for region segmentation and exploration estimation based on geometric features provided in an embodiment of the present invention. This method for region segmentation and exploration estimation based on geometric features is applied to a server, which interacts with a terminal for data exchange. By transforming the search mode from point-to-point to region-level search, with the help of precise region division and exploration rate calculation, it is possible to comprehensively evaluate environmental segments and more accurately estimate unknown spaces, so that the robot can more efficiently plan exploration paths, reduce the time spent in low-value areas, and greatly improve navigation efficiency. In addition, in terms of reducing redundant exploration, through precise region segmentation and exploration estimation, the robot can effectively avoid repeated searches of explored areas. Whether in simple or complex environments, it can reduce resource waste, improve the adaptability and flexibility of the navigation system, enable the robot to better cope with various unknown environments, and more reliably complete object navigation tasks, thereby enhancing the practicality and reliability of the navigation system in practical applications.

[0042] Figure 2 FIG is a flow chart of a method for region segmentation and exploration estimation based on geometric features provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S180.

[0043] S110, constructing a wall map;

[0044] Specifically, the constructed wall map provides the robot with precise environmental geometry information. Based on the position and shape of the walls in the wall map, the robot can determine its position and posture in real time. Compared to traditional localization methods based on feature point matching, the wall information in the wall map is more stable and unique, effectively reducing positioning errors and improving the robot's navigation accuracy. For example, in complex environments such as large warehouses, the robot can accurately follow walls, avoid collisions with obstacles, and improve navigation reliability.

[0045] In one embodiment, constructing the wall map includes:

[0046] Obtain image sequences and robot poses;

[0047] Specifically, a high-resolution visual sensor (such as an RGB-D camera) is installed on the robot to continuously capture images of the surrounding environment at a certain frame rate (for example, every 30 frames), forming an image sequence. The visual sensor operates continuously during robot movement, ensuring rich visual information of the environment. The captured images are preprocessed, including denoising and color correction. Denoising can remove random noise in the image using methods such as median filtering and Gaussian filtering. Color correction adjusts the image color based on sensor characteristics to make it more realistic. The robot's pose information is acquired using sensors such as the inertial measurement unit (IMU) and wheel odometry. The IMU measures the robot's acceleration and angular velocity, integrating them to determine the robot's displacement and attitude change. The wheel odometry calculates the robot's distance and direction based on the number of wheel revolutions and wheel diameter. To improve the accuracy of pose estimation, multi-sensor fusion can be used to combine data from sensors such as the IMU and wheel odometry. For example, a Kalman filter algorithm can be used to fuse the data from various sensors to obtain more accurate robot pose information.

[0048] generating a point cloud model according to the image sequence and the robot pose;

[0049] Specifically, feature points are extracted from image sequences. Common feature extraction algorithms include SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features). The extracted feature points are matched between different images, and the correspondence between the images is determined by calculating the similarity between the feature points. Combined with the robot's pose information, the matched feature points are reconstructed in three dimensions to generate preliminary 3D point cloud data. For example, for a pair of matched feature points, the coordinates of the feature points in 3D space can be calculated based on the changes in the robot's pose at the time the two frames were captured. As the robot moves, new image sequences and pose information are continuously collected, and the newly generated 3D point cloud data is fused with the previous point cloud data. During the fusion process, registration methods such as the Iterative Closest Point (ICP) algorithm are used to find the optimal transformation matrix between two adjacent point clouds, minimizing the error between the two frames. Through continuous iteration and fusion, a complete point cloud model is gradually constructed, which contains 3D spatial information of the surrounding environment.

[0050] generating a 3D point cloud according to the point cloud model;

[0051] Specifically, the generated point cloud model is streamlined to remove redundant point cloud data and improve the processing efficiency of the point cloud. Common simplification methods include voxel grid filtering and random sampling. Voxel grid filtering divides the point cloud space into a regular voxel grid and replaces all points in each voxel grid with the centroid of all points in the voxel grid; random sampling randomly selects some points in the point cloud according to a certain probability. The streamlined point cloud is optimized by smoothing and normal vector estimation. Smoothing can remove noise and sharp edges in the point cloud, making the point cloud smoother; normal vector estimation is used to describe the surface direction of the point cloud data at each point, providing a basis for subsequent processing. After simplification and optimization, high-quality 3D point cloud data is obtained. This 3D point cloud can more accurately represent the three-dimensional structure of the surrounding environment and provide a reliable data basis for subsequent visual obstacle recognition.

[0052] Acquiring visual obstacle identification information based on the 3D point cloud;

[0053] Specifically, obstacle detection is performed on 3D point clouds using methods based on geometric features or machine learning algorithms. Geometric methods identify possible obstacles by analyzing features such as the shape and density of the point cloud. For example, areas of localized high density or irregular shapes in the point cloud may indicate the presence of an obstacle. Machine learning algorithms can classify point cloud data by training classifiers to identify different types of obstacles. Common machine learning algorithms include support vector machines (SVMs) and random forests. During training, a large amount of labeled data is required to improve the accuracy of the classifier. Detected obstacles are segmented to separate each obstacle from the point cloud. Segmentation methods can include region growing algorithms and clustering algorithms. Region growing algorithms start with a seed point and, based on certain similarity criteria, group adjacent points into the same obstacle region. Clustering algorithms divide point cloud data into clusters, each representing a single obstacle.

[0054] Performing coordinate transformation according to the visual obstacle identification information to obtain coordinate values;

[0055] Specifically, the robot coordinate system and the world coordinate system are defined. The robot coordinate system has the robot's center as its origin and the robot's forward direction as the positive X-axis. The world coordinate system is a fixed global coordinate system used to represent the robot's position in the entire environment. Based on the robot's position information, the visual obstacle identification information is converted from the robot coordinate system to the world coordinate system. During this conversion process, the robot's translational and rotational motion must be considered, and coordinate transformation is achieved using rotation matrices and translation vectors. After the coordinate transformation, the coordinate values ​​of the visual obstacle in the world coordinate system are obtained. These coordinate values ​​accurately represent the obstacle's position in the environment and provide key information for subsequent wall map construction.

[0056] Projection is performed according to the coordinate values ​​to obtain a wall map.

[0057] Specifically, an appropriate projection method is selected to project the coordinate values ​​of obstacles onto a two-dimensional plane to generate a wall map. Common projection methods include orthographic projection and perspective projection. Orthographic projection projects points in three-dimensional space directly onto a two-dimensional plane, preserving the actual size and shape of the object. Perspective projection simulates the visual effect of the human eye, where the farther the object is from the viewpoint, the smaller its size on the projection plane. Based on the projected coordinate values, the boundaries of obstacles such as walls are drawn on the two-dimensional plane to generate a wall map. During the drawing process, graphic elements such as line segments and polygons can be used to represent walls. Walls can also be labeled and categorized to help the robot better understand the wall map information.

[0058] In other words, by combining image sequences and robot pose information to generate a point cloud model, the advantages of both visual sensors and the robot's own sensors are fully utilized, improving the accuracy and completeness of the point cloud data. Compared to single-sensor data, the fused data more accurately reflects the 3D structure of the environment, thereby constructing a more precise wall map. Furthermore, visual obstacle recognition based on the 3D point cloud enables more accurate obstacle detection and segmentation, obtaining precise obstacle location information. This precise obstacle information, after coordinate transformation and projection, can be more accurately mapped on the wall map, reducing errors and uncertainties in the map. Furthermore, as the robot moves, the image sequence and pose information are continuously updated, generating new point cloud data in real time and updating the wall map. This approach adapts to environmental changes, such as the movement, addition, or disappearance of obstacles, ensuring that the wall map remains consistent with the actual environment. Advanced feature extraction, matching, and obstacle recognition algorithms can handle complex environmental scenarios, such as varying lighting and missing textures. Even in complex environments, an accurate wall map can be constructed, providing reliable support for robot navigation.

[0059] More specifically, the map is segmented based on tall obstacles (mainly walls) where the robot’s vision is blocked. t and robot poses p0,…,p t Data is collected, and visual obstructions are identified through 3D point cloud modeling and a height threshold h (e.g., 1.8 meters), and a wall map W is constructed. The specific main process is as follows:

[0060] RGB-D data obtains RGB images and depth images:

[0061] RGB image is: I RGB ={(u i ,v i ,r i ,g i ,b i )|i=1,2,...,m};

[0062] The depth image is: I D ={(u i ,v i ,d i )|i=1,2,...,m};

[0063] Among them, (u i ,v i ) is the pixel coordinate, (r i ,g i ,b i ) is the RGB color value, d i is the depth value.

[0064] Point cloud generation model:p i =(x i ,y i ,z i )=f cam (u i ,v i ,d i ,K);

[0065] Among them, f cam is the camera model function, and K is the camera intrinsic parameter matrix:

[0066]

[0067] The specific conversion formula is: i =(u i -c x )×d i / f x ,y i =(v i -c y )×di / f y , z i =d i ;

[0068] Generate 3D point cloud: P = {p i =(x i ,y i ,z i )|i=1,2,...,n};

[0069] Visual obstacle recognition: O = {p i ∈P|z i >h threshold};

[0070] Coordinate transformation:

[0071] Wall map projection: W = {(x i ,y i )|(x i ,y i ,z i )∈O map}.

[0072] S120, performing region segmentation on the wall map to obtain a region segmentation result;

[0073] Specifically, through region segmentation, the wall map is divided into distinct regions. The robot can then quickly search for the optimal path based on the information in these regions. For example, when planning a path, the robot can prioritize open spaces and avoid areas with dense obstacles, thereby reducing path search time and computational complexity. Region segmentation is capable of handling complex wall map environments, such as those containing multiple rooms, corridors, and obstacles. The robot can dynamically adjust its path planning strategy based on the characteristics of each region to adapt to environmental changes. Furthermore, the segmented regions have distinct types and characteristics, allowing the robot to better understand the different areas in the environment. For example, it can distinguish between traversable and impassable areas and identify specific functional areas such as rooms and corridors, providing more accurate environmental information for the robot's task execution. The region segmentation results can serve as the basis for constructing semantic maps, providing the robot with higher-level semantic information. By associating regions with objects and functions in the real environment, the robot can understand the meaning of different areas, such as "kitchen" and "bedroom," enabling more intelligent navigation and interaction.

[0074] In one embodiment, performing region segmentation on the wall map to obtain a region segmentation result includes:

[0075] Preprocessing the wall map to obtain a binary image;

[0076] Specifically, if the wall map is a color image, it is first converted into a grayscale image. Common grayscale conversion formulas, such as the weighted average method (Gray = 0.299*R + 0.587*G + 0.114*B), can be used to convert the RGB three channel values ​​of the color image into a single grayscale value. This step can reduce the data dimension and reduce the computational complexity of subsequent processing. Use a suitable threshold segmentation method to convert the grayscale image into a binary image. For example, a global threshold method (such as the Otsu algorithm) is used to automatically determine a threshold, and pixels with grayscale values ​​greater than the threshold are set to white (representing passable areas), and pixels with grayscale values ​​less than the threshold are set to black (representing obstacle areas). Local thresholding methods can also be used according to actual conditions to adapt to wall maps under different lighting conditions.

[0077] Performing distance transformation on the binary image to obtain a distance map;

[0078] Specifically, the binary image is processed using the Euclidean distance transform algorithm. For each white pixel in the binary image, the Euclidean distance to the nearest black pixel is calculated. By constructing a distance matrix, dynamic programming or a fast marching algorithm can be used to efficiently calculate distance values. Each pixel value in the distance map represents the distance from that point to the nearest obstacle.

[0079] determining a region center according to the distance map;

[0080] Specifically, local maximum points are found in the distance map as the center of the region. A non-maximum suppression algorithm can be used to compare the distance values ​​of each pixel in its neighborhood. If the distance value of the point is greater than that of all points in its neighborhood, it is marked as a local maximum point. These local maximum points usually correspond to the center of the passable area. The detected local maximum points are screened to remove some false center points caused by noise or algorithm errors. Screening can be performed based on features such as the size of the distance value and the distribution of pixels in the neighborhood to retain regional centers with significant features.

[0081] Performing region segmentation according to the region center 3D to obtain an initial segmentation result;

[0082] Specifically, a well-determined region center is used as a seed point, and a region growing algorithm is used for segmentation. Starting from the seed point, adjacent pixels are merged into the same region based on a certain similarity criterion (such as a distance threshold). During the growing process, the region boundaries are continuously updated until no pixels that meet the similarity criterion can be added. If there are multiple region centers, region growing is performed on each region center separately to obtain multiple independent regions. During the growing process, it is necessary to avoid overlap and intersection between different regions. This can be done by marking the visited pixels to ensure that each pixel belongs to only one region.

[0083] The initial segmentation result is optimized to obtain the region segmentation result.

[0084] Specifically, the multiple small areas obtained by the initial segmentation are merged. The merging can be performed based on the area, shape, adjacent relationship and other characteristics of the area. For example, the area with a smaller area and a closer distance to the adjacent area is merged into the adjacent large area to reduce the fragmentation of the area. Check whether the boundaries of the segmented area are accurate and correct the unreasonable boundaries. The edge detection algorithm (such as Canny edge detection) can be used to re-detect the edge information of the wall map, and the segmentation boundary can be adjusted according to the edge information to make it more consistent with the actual area division. In addition, the boundary of the segmented area is smoothed to remove jagged edges. Morphological operations (such as corrosion, expansion, opening operation, closing operation) can be used to smooth the boundary to make the area boundary smoother and more natural.

[0085] Specifically, by preprocessing the wall map into a binary image and applying a distance transform to generate a distance map, regional features within the wall map can be more accurately extracted. The distance values ​​in the distance map reflect the distance from a pixel to an obstacle, providing a more reliable basis for determining region centers and performing segmentation. Determining region centers based on the distance map avoids the subjectivity and errors inherent in manually setting center points, improving the accuracy of region center determination. Accurate region centers guide the region growing algorithm to produce more reasonable initial segmentation results. Furthermore, optimization of the initial segmentation results, including region merging, boundary correction, and smoothing, removes noise and errors during the segmentation process, enhancing the robustness of the segmentation. Even if the initial segmentation results exhibit minor flaws, optimization can yield more accurate segmentation results. This technical solution is highly adaptable and can handle wall maps of varying lighting conditions and complexity. The preprocessing and distance transform steps automatically adapt to varying image features, ensuring the stability of region segmentation.

[0086] More specifically, the wall map W is subjected to a distance transformation D w , mark the area within δ = 1.5 units from the wall as the wall area.

[0087] The formula is: Wall area = {(x,y)|D w (x,y)≤δ}.

[0088] Perform Euclidean distance transformation D on the preprocessed binary image (binary of wall and non-wall areas) e , create a distance map, which provides the basis for subsequent region center detection. The region center detection threshold τ = 3 can be set (adjusted according to map resolution and environment complexity).

[0089] By performing local maximum detection on the distance map, the potential area center c (including the distance from c1 to c n For point c i (coordinates are (x, y)), if its value D on the distance map e (x, y) exceeds the threshold τ and is the maximum value in its neighborhood N(x, y), that is, D e (x,y)=max (x',y')∈N(x,y) D e (x',y'), then this point is considered the center of the region.

[0090] The center of the detected area c i As the seed point s i , the watershed algorithm is applied to segment the region. By simulating the "flood" process, according to the point (x, y) to the seed point s i The terrain distance P(x,y,s i ), divide the region, and the calculation formula of the region label R(x,y) is: R(x,y)=argmin i {P(x,y,s i )}.

[0091] In order to make the segmentation result more reasonable, the initial segmentation is optimized. Small areas with an area smaller than the threshold α (set α = 10) are merged with adjacent larger areas. The merging rule is:

[0092]

[0093] Among them, |R i | represents the size of region i, N R (i) represents the set of adjacent regions of region i.

[0094] S130, obtaining the robot's position information and direction information, and calculating a visible area based on the position information and the direction information;

[0095] Specifically, accurate robot position and orientation information provides a reliable foundation for navigation, enabling the robot to more precisely determine its position and orientation within the environment. Combined with the calculated visible area, the robot can pre-plan an optimal path to avoid obstacles, reducing collisions and detours during navigation and improving navigation efficiency. In a dynamic environment, the robot's position and orientation information constantly changes. By acquiring and updating this information in real time and recalculating the visible area, the robot can promptly adapt to environmental changes, adjust its navigation strategy, and ensure accurate and safe navigation. Furthermore, the calculated visible area provides the robot with an intuitive understanding of its surroundings. The robot can understand the traversable area and obstacle distribution around it, thereby better understanding the structure and characteristics of the environment. This helps the robot perform more advanced environmental perception tasks such as object recognition, scene understanding, and semantic segmentation.

[0096] In one embodiment, obtaining the robot's position information and direction information, and calculating the visible area based on the position information and the direction information, includes:

[0097] Get the robot's position and direction information;

[0098] Specifically, sensors such as the Global Positioning System (GPS), Inertial Measurement Unit (IMU), and wheel encoders are integrated. GPS provides the robot's approximate position coordinates in outdoor or open environments. The IMU, which contains an accelerometer and gyroscope, measures the robot's acceleration and angular velocity, obtaining displacement and rotation information through integration. Wheel encoders, mounted on the robot's wheels, record the number of wheel rotations and calculate the distance traveled. The data from these three sensors is fused, and a Kalman filter algorithm is used to assign different weights to each sensor based on its accuracy and reliability, resulting in more accurate and stable robot position information. A camera captures images of the surrounding environment. Using image processing and feature matching algorithms, the current image is compared with a pre-built map or a known image database to determine the robot's position within the environment. For example, the SIFT (Scale-Invariant Feature Transform) algorithm is used to extract feature points from the image, which are then matched with feature points in the map to calculate the robot's position.

[0099] The gyroscope in the IMU directly measures the robot's angular velocity, which is integrated to determine directional changes, but this can introduce drift errors. The magnetometer measures the direction of the Earth's magnetic field and can be used as a directional reference. A complementary filtering algorithm is used to fuse the gyroscope and magnetometer data, adjusting the fusion weight based on their characteristics to reduce gyroscope drift errors and improve directional accuracy. Using image information captured by the camera, the robot's orientation is estimated by analyzing the relative position and direction of movement of objects in the image. For example, the robot's orientation can be inferred based on the changing trends of the ground texture or the arrangement of surrounding objects in the image.

[0100] Set the robot's maximum visible distance;

[0101] Specifically, the maximum visible distance is set according to the specific tasks and application scenarios of the robot. For example, in a warehouse inspection task, in order to cover the entire warehouse area, the maximum visible distance can be set to 10-20 meters according to the size and layout of the warehouse; in an autonomous driving scenario, in order to promptly detect obstacles and traffic conditions ahead, the maximum visible distance can be set to 50-100 meters. The maximum visible distance is reasonably set based on the effective detection range and accuracy of the sensors used by the robot (such as lidar, cameras, etc.). For example, if the effective detection distance of the lidar is 30 meters, the maximum visible distance can be set within 30 meters to ensure that the information obtained is accurate and reliable.

[0102] The visible area is calculated according to the position information, the direction information and the maximum visible distance.

[0103] Specifically, in a two-dimensional plane, with the robot's position as the center and the maximum visible distance as the radius, the center angle of a sector is determined based on the robot's orientation information. For example, if the robot is facing straight ahead, the center angle can be set to 120 degrees, forming a sector-shaped area as the visible area. Using environmental information acquired by sensors, each point within the sector is determined to be obstructed by an obstacle. For lidar data, the location of obstacles is determined based on the reflection of the laser beam; for camera images, obstacles are identified using an image segmentation algorithm. Points obstructed by obstacles are removed from the sector to obtain the final visible area.

[0104] In three-dimensional space, with the robot's position as the vertex and the maximum visible distance as the base radius, the cone's axis direction is determined based on the robot's orientation information. For example, a quaternion is used to represent the robot's orientation, which is converted into a rotation matrix to determine the cone's axis. A 3D point cloud of the environment is obtained. For each point within the cone, a ray casting algorithm is used to check whether it is occluded by an obstacle point cloud. Occluded points are removed from the cone to obtain the 3D visible area.

[0105] More specifically, after obtaining the region segmentation result, according to the robot position loc0,…,loc t , direction information d0,…,d t and the maximum visible distance d max (Assuming 5 meters) Calculate the visible area V t , the calculation formula is:

[0106] V t ={p|||p-loc t ||≤d max &LoS(loc t ,p)=True};

[0107] Among them, LoS (loc t ,p) is the sight function, which is implemented by the ray tracing Bresenham algorithm. When loc t Returns True if there is an unobstructed path between and p.

[0108] S140, obtaining an exploration area point set according to the visible area calculation;

[0109] Specifically, an accurate exploration area point set provides a crucial basis for the robot's mission planning and decision-making. Based on the distribution and characteristics of the point set, the robot can formulate a rational exploration path and task execution strategy. For example, in a search task, the robot can prioritize the points and explore each sampling point sequentially, improving the search success rate.

[0110] In one embodiment, calculating the exploration area point set based on the visible area includes:

[0111] Combine simultaneous positioning and map construction to obtain a traversable area;

[0112] Specifically, the robot is equipped with a variety of sensors, such as lidar, cameras, and inertial measurement units (IMUs). Lidar is used to obtain distance information about the surrounding environment, cameras capture image data, and IMUs measure the robot's acceleration and angular velocity. These sensors collect data in real time as the robot moves. For example, the lidar emits laser beams at a constant frequency and receives reflected signals, generating point cloud data of the surrounding environment; the camera continuously captures images to obtain visual information about the environment.

[0113] Preprocess and extract features from sensor data. For lidar data, operations such as point cloud filtering and feature point extraction can be performed to extract features such as edges and corners in the environment. For camera image data, feature detection algorithms (such as SIFT and SURF) can be used to extract key points and feature descriptors from the image. Based on the feature information extracted by the front-end, SLAM algorithms are used for localization and map construction. Common SLAM algorithms include filtering-based algorithms (such as extended Kalman filter SLAM) and graph optimization-based algorithms (such as g2o and Ceres). By continuously updating the robot's pose estimate and map information, the robot's real-time position in the environment and a map representation of its surroundings are obtained. Based on the constructed map, combined with the robot's kinematic model and physical characteristics, the robot can determine the areas it can safely traverse. For example, based on the robot's size and terrain height information, areas in the map that are higher than the robot's chassis height or contain obstacles are marked as non-traversable, while the remaining areas are marked as traversable.

[0114] An intersection and a merge are performed on the visible area and the traversable area to obtain the exploration area point set.

[0115] Specifically, the spatial relationship between the visible area and the traversable area is judged, and their intersection is determined. Spatial index structures (such as quadtrees, R-trees, etc.) can be used to accelerate the judgment of spatial relationships. For the intersection part, an exploration area point set is generated according to demand. Uniform sampling, density sampling and other methods can be used to select a certain number of points in the intersection area as the exploration area point set. For example, the sampling points are evenly distributed at certain intervals in the intersection area, or the number of sampling points is increased in the information-rich area based on the information density in the intersection area. The generated exploration area point set is merged with the existing exploration area point set, duplicate points are removed, and the point set is optimized. A clustering algorithm can be used to cluster the point set, and then the center point of each cluster is selected as the representative point to reduce the number of point sets and improve the quality of the point set.

[0116] In other words, by combining SLAM with traversable areas, the robot can effectively avoid inaccessible or dangerous areas, such as steep slopes, deep pits, and areas with dense obstacles. This significantly reduces the risk of safety incidents such as collisions and entrapment during exploration, thereby improving exploration reliability. SLAM algorithms can dynamically adjust the robot's pose estimation and map construction based on different environmental characteristics, enabling the robot to adapt to a variety of complex environments, such as indoors, outdoors, and underground. Regardless of environmental changes, the robot can accurately determine the traversable area, ensuring safe exploration. Furthermore, by intersecting and merging the visible and traversable areas, the resulting exploration area point set focuses more on areas that the robot can actually reach and observe. The robot can prioritize exploration in these areas, avoiding wasting time and resources in ineffective areas and improving exploration efficiency. The exploration area point set contains sample points from areas that the robot can reach and see. Further exploration and analysis of these sample points can provide more accurate environmental information. For example, in robot navigation, an accurate exploration area point set can help the robot more accurately locate its position and plan its path.

[0117] More specifically, the existing mapping methods such as simultaneous localization and map construction are combined to obtain the traversable area M, and then the estimated exploration area point set E is obtained by taking the intersection. The formula is:

[0118]

[0119] The formula indicates that within the entire time period T, the intersection of the visible area and the traversable area at each moment is merged to obtain the total exploration area, that is, the exploration area point set E.

[0120] S150, calculating a region exploration rate based on the region segmentation result and the exploration region point set;

[0121] Specifically, for each point in the exploration area point set, determine whether it is located in a specific area in the region segmentation result. If the region segmentation result is represented by a binary image, it can be judged by the pixel value. For example, if the region segmentation result is a binary image, where the value 1 represents the target area and the value 0 represents the background area, then determine whether the pixel value corresponding to the coordinate of the point is 1. If the region segmentation result is represented by a label matrix, it can be judged by the label value. For example, if the region segmentation result is a label matrix, where the value of each element represents the region category to which the position belongs, then determine whether the label value corresponding to the coordinate of the point is the label value of a specific region.

[0122] In addition, all points in the exploration area point set are traversed and the number of points falling within each specific area is counted. The spatial index structure can be used to speed up the query process and improve statistical efficiency. For each specific area, the regional exploration rate is calculated according to the following formula:

[0123] Regional exploration rate = number of points falling within the region / total number of points in the region (can be determined by estimating the area of ​​the region or presetting the number of points);

[0124] The total number of points for a region can be determined in a number of ways:

[0125] Area estimation: Calculate the area of ​​each region based on the region segmentation results, and then estimate the total number of points in the region based on the preset point density (such as the number of points per square meter).

[0126] Preset points: During the area segmentation process, a total number of points is set for each area in advance. The number of points can be set based on experience or actual needs.

[0127] Output the calculated exploration rate for each region in a table or list, including information such as region name and exploration rate. Use visualization tools (such as Matplotlib or OpenCV) to visualize the region segmentation results and the set of points in the explored regions, and annotate the exploration rate for each region in the plot. For example, use different colors to represent different regions and display the exploration rate value next to the region.

[0128] In other words, the regional exploration rate provides a quantitative metric for assessing the extent of exploration of a specific area by a robot or detection device. By comparing the exploration rates of different areas, we can intuitively understand which areas have been fully explored and which areas require further exploration. Based on the regional exploration rate, exploration strategies can be adjusted promptly. For example, for areas with low exploration rates, the detection device's dwell time can be increased or the detection direction can be adjusted to improve exploration effectiveness. Furthermore, when resources are limited, regional exploration rate can be used to rationally allocate resources. For example, in a robotic exploration mission, more sensor resources, computing resources, and energy resources can be allocated to areas with low exploration rates to improve overall exploration efficiency. By accurately calculating the regional exploration rate, we can avoid wasting resources in areas that have already been fully explored, thereby improving resource utilization efficiency.

[0129] More specifically, according to the region segmentation result R i The exploration rate r of each region is calculated using the exploration area point set E to evaluate the degree of exploration of the region. The calculation formula of the regional exploration rate r is:

[0130]

[0131] Among them, R irepresents the point set of the i-th region, and the regional exploration rate r represents the proportion of explored points in the total number of points in the region.

[0132] S160, setting an exploration priority according to the area exploration rate;

[0133] Specifically, different levels of exploration rate thresholds are set, and the exploration rate of a region is compared with the threshold to divide the exploration priority. For example, three thresholds are set: a low exploration rate threshold (0.2), a medium exploration rate threshold (0.4), and a high exploration rate threshold (0.6). Regions with exploration rates below the low exploration rate threshold are set to high priority; regions with exploration rates between the low and medium exploration rate thresholds are set to medium priority; and regions with exploration rates above the medium exploration rate threshold are set to low priority.

[0134] Sort all regions by exploration rate from high to low or low to high. If sorting from low to high, the top regions are set as high priority, the middle regions are set as medium priority, and the bottom regions are set as low priority. For example, if there are 5 regions, the top 2 regions by exploration rate are set as high priority, the middle 2 regions are set as medium priority, and the last region is set as low priority.

[0135] In addition to exploration rate, factors such as regional importance and relevance to the mission objective can also be considered. For example, in a search and rescue mission, if a certain area is likely to be where trapped people are located (even if its exploration rate is relatively high), its exploration priority can be appropriately increased. Weights are assigned to different factors, a comprehensive score is calculated, and exploration priorities are set based on this comprehensive score. For example, if the exploration rate weight is 0.6, the regional importance weight is 0.3, and the relevance to the mission objective weight is 0.1, the weighted sum is used to obtain the comprehensive score, and then the priority is determined based on the comprehensive score.

[0136] Map the exploration priorities (high, medium, and low) set according to the rules to specific values ​​or identifiers for subsequent processing. For example, high priority is mapped to 3, medium priority is mapped to 2, and low priority is mapped to 1. Store information such as area name, exploration rate, and exploration priority in a database or file system for subsequent exploration task scheduling and decision-making. For example, store it in a relational database and create a table containing the above fields. In addition, set a fixed time interval (such as every hour or every day) to regularly recalculate the area exploration rate and reset the exploration priority based on the new exploration rate data. When specific events occur, such as the discovery of new important clues or major changes in the regional environment, immediately re-evaluate the exploration rate of the relevant area and adjust its exploration priority. For example, during the robot exploration process, if the sensor detects an abnormal signal in an area that originally had a high exploration rate, the exploration priority of that area will be increased.

[0137] In other words, by setting exploration priorities, robots or detection equipment can prioritize areas with low exploration rates or high importance, avoiding wasting time and resources on already-explored or non-critical areas. This allows them to cover more critical areas within a limited timeframe and improve overall exploration efficiency. For example, in a warehouse inventory task, prioritizing areas with low exploration rates and high cargo value allows for faster inventory completion. When faced with urgent tasks or specific targets, exploration priorities can be used to quickly adjust exploration directions to meet mission requirements. For example, in search and rescue scenarios, prioritizing areas where trapped people may be present increases the success rate of rescue. Furthermore, exploration priorities can be used to rationally allocate human, material, and time resources. In high-priority areas, more resources are allocated for in-depth exploration; in low-priority areas, resources are appropriately reduced. For example, in geological exploration tasks, more exploration personnel and equipment are deployed to areas with high exploration value and low exploration rates. Overinvesting resources in low-priority areas is avoided, reducing unnecessary costs. For example, in environmental monitoring tasks, excessive monitoring equipment should be avoided in areas that are already well-monitored and experience minimal environmental fluctuations. Furthermore, the exploration priority setting method can be flexibly adjusted to suit different mission scenarios and objectives, offering strong adaptability. Whether in indoor navigation, outdoor exploration, or industrial inspection scenarios, properly setting exploration priorities can improve mission execution. In complex and ever-changing environments, dynamically adjusting exploration priorities allows for timely adaptation to environmental changes and ensures the smooth progress of exploration missions. For example, at a natural disaster rescue site, the search area priority is continuously adjusted as rescue efforts progress and the on-site environment changes.

[0138] In one embodiment, setting the exploration priority according to the area exploration rate includes:

[0139] Set high and low thresholds;

[0140] When the exploration rate of the area is set to be lower than the low threshold, it is set to be a high priority;

[0141] When the area exploration rate is set to be between the high threshold and the low threshold, it is set to be a medium priority;

[0142] When the area exploration rate is set to be greater than the high threshold, it is set to a low priority.

[0143] Specifically, determine the low threshold based on data distribution and application requirements. A common approach is to set the low threshold to the mean minus a certain multiple of the standard deviation, such as low threshold = mean - 1 × standard deviation. Low threshold = 0.4 - 1 × 0.15 = 0.25. Alternatively, based on experience, the low threshold can be set to a relatively low value, such as 0.2, to indicate areas with significantly lower exploration rates than the average. Similarly, the high threshold can be set to the mean plus a certain multiple of the standard deviation, such as high threshold = mean + 1 × standard deviation. High threshold = 0.4 + 1 × 0.15 = 0.55. Alternatively, based on experience, the high threshold can be set to a relatively high value, such as 0.6, to indicate areas with significantly higher exploration rates than the average.

[0144] For each region, determine its exploration rate versus the low and high thresholds:

[0145] If the exploration rate of a region is less than the low threshold, the exploration priority of that region is set to high. For example, the exploration rate of clothing store A is 0.2, which is less than the low threshold of 0.25, so it is set to high priority.

[0146] If the exploration rate of a region is between the high and low thresholds, the exploration priority of that region is set to medium. For example, the exploration rate of the food court is 0.45, which is between the low threshold of 0.25 and the high threshold of 0.55, so it is set to medium priority.

[0147] If the exploration rate of a region is greater than the high threshold, the exploration priority of that region is set to low. For example, the exploration rate of the cosmetics counter is 0.6, which is greater than the high threshold of 0.55, so it is set to low priority.

[0148] In other words, by setting high and low thresholds, areas with low exploration rates can be identified as high-priority, allowing robots or detection equipment to explore these areas first, avoiding wasting time and resources on already-explored areas. For example, in a warehouse inventory task, prioritizing exploration of areas with low exploration rates can more quickly identify inventory discrepancies or anomalies. The high and low thresholds can be flexibly adjusted according to different application scenarios and task objectives, enabling targeted exploration of different types of areas. For example, in archaeological exploration scenarios, a lower low threshold can be set for areas that may contain important cultural relics, giving these areas a higher exploration priority. Furthermore, human, material, and time resources can be rationally allocated based on exploration priorities. In high-priority areas, more resources can be invested in in-depth exploration; in low-priority areas, resource investment can be appropriately reduced. For example, in geological exploration tasks, more exploration personnel and equipment can be deployed to areas with high exploration value and low exploration rates to improve resource utilization efficiency. Overinvesting resources in low-priority areas can be avoided, reducing unnecessary costs. For example, in environmental monitoring tasks, excessive monitoring equipment can be avoided in areas that are already well-monitored and have minimal environmental changes, reducing monitoring costs.

[0149] More specifically, the regional exploration rate r is compared with the preset high and low thresholds (such as r high =0.8, r low =0.3) comparison. <r low Set to high priority and explore first; low ≤r≤r high Medium priority, explore at the right time; r>r high Low priority, reduce exploration.

[0150] S170, formulate path planning and exploration strategies;

[0151] Specifically, through rational path planning, a robot can find the optimal or near-optimal path from its current location to its target location, reducing travel time and distance, thereby quickly completing its mission. For example, in logistics delivery tasks, optimized path planning can significantly shorten delivery time. Scientific exploration strategies can guide robots or devices to explore the environment in a targeted manner, avoiding blind exploration and improving exploration efficiency. For example, at a disaster rescue site, a priority-based regional exploration strategy can enable rescuers to more quickly locate trapped people and important clues.

[0152] In one embodiment, formulating a path planning and exploration strategy includes:

[0153] Using the regional semantic map information, the path to the target area is planned, that is, the path planning is obtained;

[0154] Specifically, various sensors, such as lidar, cameras, and infrared sensors, are used to collect environmental information. Lidar can obtain precise distance data, cameras can capture visual features such as object color and shape, and infrared sensors can detect special information such as temperature. For example, in indoor robot navigation, lidar constructs the room's outline, and cameras identify objects such as furniture, doors, and windows. Based on this collected data, deep learning algorithms (such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are used to semantically annotate areas in the environment. This annotation includes the area's function (e.g., bedroom, kitchen, conference room), and possible objects (e.g., sofa, refrigerator, projector, etc.). For example, a trained model can identify an area in an image as a "bedroom" and annotate possible objects such as a "bed" and "wardrobe." As the robot or device moves, new environmental information is continuously collected, and the semantic map of the area is updated in a timely manner. For example, when a robot enters a previously unexplored room, the room's semantic information is updated in real time. The information collected by different sensors is integrated to improve the accuracy and reliability of the semantic map. Simultaneously, the semantic map is corrected based on actual conditions to eliminate errors. For example, when there are differences in the detection results of the same object by the lidar and camera, the information of the two can be combined through the data fusion algorithm to obtain a more accurate object location and attributes.

[0155] Based on the specific task requirements, determine the location and functional requirements of the target area. For example, in a logistics delivery task, the target area might be a specific shelf location in the warehouse; in a home service robot task, the target area might be next to the sofa in the living room. Leverage the regional semantic map to find an area that matches the target function. For example, if the task requires placing an item in the "kitchen," semantic matching is used to find the kitchen area in the map. Evaluate the suitability of different path search algorithms (such as the A* algorithm, Dijkstra's algorithm, and the RRT (Rapid Random Tree) algorithm) based on the environment characteristics and task requirements. For example, in static, known environments, the A* algorithm can quickly find the optimal path; in dynamic, complex environments, the RRT algorithm is more suitable for real-time path planning. Select an appropriate algorithm and combine the obstacle information (such as walls and furniture) in the regional semantic map with the location of the target area to plan a path from the current location to the target area. For example, when using the A* algorithm, the feasible area and obstacle area in the map are gridded. The cost function is calculated for each grid cell to find the optimal path from the starting point to the destination.

[0156] The planned path is analyzed to remove redundant turns and detours, making it simpler. For example, if a path contains multiple unnecessary turns, an optimization algorithm can remove them to reduce the path length. The path is also smoothed to ensure smoother robot travel. For example, spline curve fitting can be used to connect discrete points on the path into a smooth curve, improving the robot's driving comfort and safety.

[0157] Determine the frontier point set at the boundary of the explored and unexplored areas, select the appropriate frontier point as the action target, and adjust the route when encountering obstacles, thus obtaining the exploration strategy.

[0158] Specifically, based on the division of explored and unexplored areas, region boundaries are identified. For example, when a robot explores an unknown environment, sensors detect the edge of an unexplored area and determine the location of the boundary. A series of leading points are selected along the region boundary to form a leading point set. The selection of leading points can be optimized based on factors such as distance, angle, and information content. For example, leading points that are close to the current position and likely contain important information are selected.

[0159] A priority assessment is performed on each frontier point in the frontier point set, taking into account factors such as its distance from the target area, the complexity of the surrounding environment, and the potential value of the information it contains. For example, frontier points that are close to the target area, have simple environments, and have high information value are given higher priority. Based on the priority assessment results, an appropriate frontier point is selected as the action target. For example, the frontier point with the highest priority is selected as the robot's next exploration target.

[0160] As the robot moves toward its target, it uses sensors to detect obstacles in its surroundings in real time. For example, a LiDAR sensor can detect an obstacle ahead. When an obstacle is detected, a local path planning algorithm (such as the Dynamic Windowing Algorithm (DWA) or artificial potential field method) is used to adjust the robot's route based on the obstacle's location and size, avoiding the obstacle and continuing toward the target. For example, the DWA algorithm calculates a feasible direction and speed based on the robot's current speed and acceleration and the location of the obstacle, achieving real-time obstacle avoidance.

[0161] In other words, utilizing regional semantic map information can more accurately consider obstacles in the environment and the functional requirements of the target area, allowing for more efficient path planning. For example, in a warehouse environment, using shelf semantic information, a path can be planned that avoids the shelves and quickly reaches the target shelf. By selecting an appropriate path search algorithm and combining it with preprocessing of the regional semantic map, a path from the current location to the target area can be quickly found, reducing the time required for path planning. For example, in complex indoor environments, using an optimized A* algorithm can significantly improve path planning speed. Furthermore, exploration strategies based on frontier point sets can selectively select exploration targets, avoiding blind exploration and improving exploration efficiency. For example, in an unknown outdoor environment, a robot can prioritize exploration of areas likely to contain important resources based on the priority of frontier points. By enabling timely route adjustments when encountering obstacles, the exploration strategy can adapt to environmental changes and uncertainties. For example, at a disaster relief site, when encountering new obstacles or terrain changes, the robot can adjust its exploration route in real time to continue its rescue mission. Furthermore, combining regional semantic map information with path planning enables robots or equipment to more accurately reach their target areas and complete their mission objectives. For example, in medical robotic surgery, precise path planning ensures that surgical instruments accurately reach the lesion site, improving the success rate of the operation. By rationally selecting frontier points and determining action targets, exploration strategies can cover more areas and discover more information. For example, in archaeological exploration missions, cultural relics and important information can be more comprehensively discovered, improving the quality of archaeological work.

[0162] S180: Combine the exploration priority, the path planning, and the exploration strategy to guide robot actions.

[0163] Specifically, information on exploration priorities, path planning, and exploration strategies is integrated into a comprehensive decision-making system. The optimal action plan is selected based on the current environmental state and task requirements. During the robot's operation, the robot's status and environmental changes are monitored in real time, and feedback is promptly transmitted to the decision-making system. Based on this feedback, the decision-making system makes real-time adjustments to the action plan to ensure the robot successfully completes the task.

[0164] In other words, combined with exploration priorities, robots can prioritize exploration of important areas, avoiding wasting time and resources in irrelevant areas. For example, in archaeological exploration missions, robots can prioritize exploration of areas likely to contain cultural relics based on exploration priorities, improving the targetedness and efficiency of exploration. Through appropriate path planning and exploration strategies, robots can select optimal paths, reduce travel distance and the number of turns, increase travel speed, and thus reduce exploration time. Furthermore, the integrated application of exploration priorities, path planning, and exploration strategies enables robots to better adapt to complex environmental changes, such as the appearance of dynamic obstacles and terrain changes, by dynamically adjusting their paths and exploration targets to ensure smooth mission execution. For example, in rescue missions, robots can flexibly adjust their exploration paths based on real-time on-site conditions to quickly reach the target area. Accurate path planning and effective exploration strategies enable robots to more accurately reach target areas and complete various tasks, such as material delivery and information collection, thereby improving mission success rates.

[0165] In one embodiment, the robot continuously updates its exploration data during its movement. If it discovers a new high-priority area, it determines whether to adjust its path based on distance and cost. When the robot reaches the target area, it detects targets based on its exploration rate. If a suspected target is detected, it uses VLM technology to confirm it. If confirmed, the mission is completed; otherwise, exploration continues.

[0166] Specifically, key features are extracted from the updated exploration data, such as the information richness of the area, task relevance, and degree of unknown. Machine learning algorithms, such as support vector machines (SVM) and random forests, are used to analyze and classify these features to determine whether there are new high-priority areas. For example, in a search and rescue mission, based on the distribution of heat sources and sound characteristics in the image data, it is determined whether there are areas where survivors may be present, and these areas are marked as high-priority areas. A priority assessment is performed on the identified potential high-priority areas, taking into account factors such as the distance of the area from the current location, the cost of reaching the area, and the value of the information the area may contain. By establishing a priority assessment model, a priority score is assigned to each area and the areas are ranked. For example, a weighted scoring method is used to assign corresponding weights according to the importance of different factors to calculate the comprehensive score for each area.

[0167] When a new high-priority area is discovered, the distance from the robot's current position to the area, as well as the time required to reach the area, energy consumption, and other costs are calculated. The distance can be calculated using the path planning algorithm in the map (such as the A* algorithm and the Dijkstra algorithm); the cost can be estimated based on the robot's motion model and energy consumption model. For example, in an outdoor navigation scenario, the driving time and fuel consumption required for the robot to reach the target area are calculated, taking into account factors such as terrain undulations and road conditions. Based on the calculated results of distance and cost, a path adjustment decision rule is formulated. For example, a threshold is set. When the sum of the distance difference and cost difference between the new area and the current target area exceeds the threshold, the path is adjusted to the new area; otherwise, the original path is continued to the current target area. At the same time, the decision rules are dynamically adjusted considering the urgency and importance of the task.

[0168] When the robot approaches the target area, it uses positioning technology (such as the Global Positioning System (GPS), Inertial Navigation System (INS), and visual positioning) to determine its current position and match it with the target area's location information. A certain position error threshold is set to determine whether the robot has reached the target area. For example, in indoor positioning, visual positioning technology combined with pre-placed visual markers can achieve high-precision position matching. When the robot reaches the target area, it sends an arrival confirmation signal to the control system and records the arrival time. Simultaneously, the target area exploration mission is initiated. Within the target area, the robot's onboard sensors monitor the extent of the explored area in real time. Using map segmentation and area coverage algorithms, the ratio of the explored area to the entire target area is calculated to obtain the exploration rate. For example, the target area can be divided into multiple small grids, and the number of grids covered by the robot's sensors is counted to calculate the coverage ratio. A target exploration rate threshold is set. When the actual exploration rate reaches this threshold, the robot is considered to have fully explored the target area and can enter the target detection phase. Otherwise, exploration continues within the area.

[0169] Within the target area, the robot's visual sensor collects image data. Using image processing and computer vision algorithms, it extracts target object features, such as color, shape, and texture. These features are then matched against a predefined target feature library to determine whether a suspected target exists. For example, in industrial inspection tasks, the robot extracts product appearance features and compares them with those of standard products to detect defective products. Based on the matching results, suspected targets with a high degree of match are selected and marked in the image or map. The robot also records the suspected target's location information to facilitate subsequent verification.

[0170] Load a vision-language model (VLM) into the robot system and perform initialization. The VLM combines visual and language information to understand the content of images and answer related questions. For example, a VLM model based on the Transformer architecture, such as the CLIP model, is used. This model is pre-trained on a large-scale image-text dataset and has strong visual and language understanding capabilities. Image data of a suspected target is input into the VLM model, along with relevant questions such as "Is this the target object?" and "What is the status of the object?" The VLM model generates corresponding responses based on the image and question, and based on the response, determines whether the suspected target is the actual target. If it is confirmed to be the target, the robot completes the mission; otherwise, it continues exploring within the target area or moves to another area based on the path adjustment decision rule.

[0171] In other words, by continuously updating exploration data, the robot can perceive dynamic changes in the environment in real time, promptly discover new high-priority areas, and flexibly adjust its path based on distance and cost, avoiding wasting time and resources in less important areas, thereby improving exploration efficiency. For example, in search and rescue scenarios, the robot can quickly respond to environmental changes and promptly proceed to areas where survivors may be, thereby increasing the success rate of rescue operations. Using exploration rate detection and VLM technology to identify suspected targets can improve target detection accuracy. Exploration rate detection ensures that the robot has fully explored the target area, reducing the possibility of missed detections; VLM technology combines visual and language information to more accurately identify target objects and avoid misidentification. Furthermore, this technical feature enables the robot to adapt to diverse task types and complexities. Whether searching for objects indoors, geological exploration outdoors, or search and rescue missions in emergency situations, the robot can flexibly adjust its exploration strategy and path planning based on the mission objectives and environmental characteristics, improving its adaptability and flexibility. During operations, the robot can autonomously determine whether to adjust its path or complete the mission based on real-time data, reducing the need for human intervention and enhancing its autonomous decision-making capabilities. For example, robots can independently complete long, wide-area exploration missions, reducing labor costs and operational difficulty. Furthermore, through appropriate path adjustment and exploration strategies, robots can reduce unnecessary travel distance and operations, thereby lowering energy consumption. For example, by avoiding repeated exploration of previously explored areas and selecting the optimal path to reach the target area, robots improve energy efficiency and extend their operating time. Accurate target detection and efficient task execution can increase mission success rates and reduce losses caused by mission failures. Furthermore, by optimizing resource utilization, task execution costs are reduced, improving overall efficiency. For example, in industrial production, improving the accuracy and efficiency of product inspection can reduce defective rates and increase production efficiency. Furthermore, continuously updating exploration data and multi-sensor data fusion technology can provide robots with comprehensive and accurate environmental perception information, giving them a deeper understanding of their surroundings. This helps robots better plan their paths and identify target objects, improving the quality and reliability of task execution. The application of VLM technology provides a reliable means for confirming suspected targets. By combining visual and language information, the VLM model enables a deeper analysis and understanding of the target object, avoiding the potential misjudgment caused by a single sensor or algorithm, and improving the reliability of target confirmation.

[0172] For example, applying geometric feature-based region segmentation and exploration estimation methods to hospital ward disinfection and inspection robots enables efficient exploration of ward environments, precise planning of disinfection tasks, and timely detection of anomalies. By constructing a wall map of the ward, performing region segmentation, calculating region exploration rates, and setting priorities, the robot can autonomously plan paths and exploration strategies to complete tasks such as disinfection and inspection, thereby improving the intelligent level of hospital environmental management.

[0173] The specific scenario is as follows: A hospital ward, containing three beds, two cabinets, and an IV stand, requires regular disinfection and inspection. Due to the ward's large size, manual disinfection and inspection are inefficient and pose a risk of infection. Therefore, a disinfection and inspection robot is introduced that utilizes geometric feature-based region segmentation and exploration estimation methods.

[0174] After entering the ward, the robot captures image sequences through its camera and combines its own position information to generate a point cloud model and 3D point cloud. A deep learning algorithm is used to identify obstacles such as beds and cabinets, transforming their coordinates and projecting them into a wall map. The wall map undergoes preprocessing and distance transformation to determine the centers of various areas within the ward, followed by segmentation and optimization. The ward is divided into distinct zones, including the disinfection zone (surrounding the beds) and the inspection zone (the corridors and public areas within the ward). During movement, the robot acquires its position and orientation in real time to calculate its visible area. Simultaneous localization and map building techniques are used to determine traversable areas and generate a set of explored area points. As the robot explores, the number of cells explored within each zone is counted to calculate the exploration rate. A high threshold of 80% and a low threshold of 30% are set, designating the disinfection zone and parts of the inspection zone as high-priority areas. Based on the ward's regional semantic map, a path to the high-priority area is planned. A set of frontier points at the boundary of explored and unexplored areas is determined, and appropriate frontier points are selected as action targets. The robot moves within the ward according to a planned path and exploration strategy, prioritizing high-priority disinfection areas. During the disinfection process, it monitors environmental parameters (such as UV intensity and disinfectant concentration) in real time to ensure effectiveness. It also conducts inspections of equipment status and patient conditions within the ward, promptly alerting any anomalies.

[0175] The above-mentioned geometric feature-based region segmentation and exploration estimation method transforms the search mode from point-to-point to region-level search and uses precise region segmentation on the constructed wall map to obtain clear environmental segmentation results, allowing the robot to understand the environmental structure from a macro perspective. Based on the region segmentation results and the region exploration rate calculated from the exploration region point set, the robot can clearly understand the exploration progress and potential value of each region, thereby formulating more efficient exploration path planning, reducing the time spent in low-value areas, and greatly improving navigation efficiency. Setting exploration priorities based on the region exploration rate and formulating corresponding path planning and exploration strategies enable the robot to respond more flexibly to environmental changes and avoid the resource waste and inefficiency caused by fixed path planning in traditional methods. In addition, the region exploration rate is calculated based on the region segmentation results and the exploration region point set. This exploration estimation mechanism provides clear exploration guidance for the robot. The robot can judge the exploration degree of each region based on the region exploration rate. When the exploration rate of a region reaches a certain threshold, it can be marked as fully explored and no longer searched. At the same time, the exploration priority setting and path planning strategy are also based on the exploration estimation results, allowing the robot to prioritize exploring larger areas of unknown space, avoiding unnecessary waste of time and resources. Whether in simple or complex environments, this exploration estimation-based navigation method effectively reduces redundant exploration, improving the adaptability and flexibility of the navigation system. Furthermore, through region segmentation and exploration estimation, this technology enables robots to better understand the structure of their environment and accurately estimate unknown spaces. When faced with an unknown environment, the robot can quickly build an environmental model based on the region segmentation results and determine exploration directions and priorities through exploration estimation. By improving navigation efficiency and reducing redundant exploration, the practicality and reliability of the navigation system in real-world applications are enhanced. In real-world applications, robots must complete navigation tasks within limited time and resources. Traditional navigation methods may fail to meet deadlines due to inefficiency and excessive redundant exploration. This technology enables robots to more efficiently plan exploration paths, reduce unnecessary resource waste, and ensure that robots can more reliably complete object navigation tasks. For example, in applications such as logistics and warehouse management, where robots must quickly and accurately reach their target locations, this technology can significantly improve robot navigation performance, enhancing work efficiency and service quality.

[0176] Figure 3 FIG is a schematic block diagram of a region segmentation and exploration estimation device 300 based on geometric features provided by an embodiment of the present invention. Figure 3As shown, corresponding to the above-mentioned region segmentation and exploration estimation method based on geometric features, the present invention further provides a region segmentation and exploration estimation device 300 based on geometric features. The region segmentation and exploration estimation device 300 based on geometric features includes a unit for executing the above-mentioned region segmentation and exploration estimation method based on geometric features, and the device can be configured in a server. Specifically, please refer to Figure 3 The region segmentation and exploration estimation device 300 based on geometric features includes:

[0177] A construction unit 301 is used to construct a wall map;

[0178] A segmentation unit 302 is configured to perform region segmentation on the wall map to obtain a region segmentation result;

[0179] An acquisition calculation unit 303 is used to obtain the position information and direction information of the robot, and calculate the visible area according to the position information and the direction information;

[0180] A first calculation unit 304 is configured to calculate an exploration area point set according to the visible area;

[0181] A second calculation unit 305 is configured to calculate a region exploration rate based on the region segmentation result and the exploration region point set;

[0182] a setting unit 306, configured to set an exploration priority according to the area exploration rate;

[0183] a formulation unit 307 for formulating path planning and exploration strategies;

[0184] The guidance unit 308 is configured to combine the exploration priority, the path planning, and the exploration strategy to guide the robot's actions.

[0185] It should be noted that technical personnel in the relevant field can clearly understand that the specific implementation process of the above-mentioned region segmentation and exploration estimation device 300 based on geometric features and each unit can refer to the corresponding description in the aforementioned method embodiment. For the convenience and conciseness of the description, it will not be repeated here.

[0186] The above-mentioned region segmentation and exploration estimation device 300 based on geometric features can be implemented in the form of a computer program. The computer program can be used in Figure 4 Runs on the computer device shown.

[0187] See also Figure 4 , Figure 4 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0188] See Figure 4 The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0189] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can enable the processor 502 to perform a region segmentation and exploration estimation method based on geometric features.

[0190] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0191] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a region segmentation and exploration estimation method based on geometric features.

[0192] The network interface 505 is used to communicate with other devices through the network. Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0193] The processor 502 is configured to execute a computer program 5032 stored in the memory to implement the following steps:

[0194] Construct a wall map; perform regional segmentation on the wall map to obtain a regional segmentation result; obtain robot position information and orientation information, and calculate a visible area based on the position information and the orientation information; calculate an exploration area point set based on the visible area; calculate a regional exploration rate based on the regional segmentation result and the exploration area point set; set an exploration priority based on the regional exploration rate; formulate a path planning and exploration strategy; and combine the exploration priority, the path planning, and the exploration strategy to guide the robot's actions.

[0195] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0196] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0197] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0198] Construct a wall map; perform regional segmentation on the wall map to obtain a regional segmentation result; obtain robot position information and orientation information, and calculate a visible area based on the position information and the orientation information; calculate an exploration area point set based on the visible area; calculate a regional exploration rate based on the regional segmentation result and the exploration area point set; set an exploration priority based on the regional exploration rate; formulate a path planning and exploration strategy; and combine the exploration priority, the path planning, and the exploration strategy to guide the robot's actions.

[0199] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0200] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0201] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0202] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0203] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0204] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. The region segmentation and exploration estimation method based on geometric features is characterized by: include: Build a wall map; Performing region segmentation on the wall map to obtain a region segmentation result; Obtaining the robot's position information and direction information, and calculating the visible area based on the position information and the direction information; Calculate the exploration area point set according to the visible area; Calculating a regional exploration rate based on the regional segmentation result and the exploration region point set; Setting exploration priorities based on the exploration rate of said area; Develop path planning and exploration strategies; The exploration priority, the path planning, and the exploration strategy are combined to guide the robot's actions.

2. The method for region segmentation and exploration estimation based on geometric features according to claim 1, characterized in that: The wall map construction includes: Obtain image sequences and robot poses; generating a point cloud model according to the image sequence and the robot pose; generating a 3D point cloud according to the point cloud model; Acquiring visual obstacle identification information based on the 3D point cloud; Performing coordinate transformation according to the visual obstacle identification information to obtain coordinate values; Projection is performed according to the coordinate values ​​to obtain a wall map.

3. The method for region segmentation and exploration estimation based on geometric features according to claim 1, characterized in that: The performing region segmentation on the wall map to obtain a region segmentation result includes: Preprocessing the wall map to obtain a binary image; Performing distance transformation on the binary image to obtain a distance map; determining a region center according to the distance map; Performing region segmentation according to the region center 3D to obtain an initial segmentation result; The initial segmentation result is optimized to obtain the region segmentation result.

4. The method for region segmentation and exploration estimation based on geometric features according to claim 1, characterized in that: The obtaining of the robot's position information and direction information, and calculating the visible area based on the position information and the direction information, includes: Get the robot's position and direction information; Set the robot's maximum visible distance; The visible area is calculated according to the position information, the direction information and the maximum visible distance.

5. The method for region segmentation and exploration estimation based on geometric features according to claim 1, characterized in that: The step of calculating the exploration area point set according to the visible area includes: Combine simultaneous positioning and map construction to obtain a traversable area; An intersection and a merge are performed on the visible area and the traversable area to obtain the exploration area point set.

6. The method for region segmentation and exploration estimation based on geometric features according to claim 1, characterized in that: The setting of the exploration priority according to the area exploration rate includes: Set high and low thresholds; When the exploration rate of the area is set to be lower than the low threshold, it is set to be a high priority; When the area exploration rate is set to be between the high threshold and the low threshold, it is set to be a medium priority; When the area exploration rate is set to be greater than the high threshold, it is set to a low priority.

7. The method for region segmentation and exploration estimation based on geometric features according to claim 1, characterized in that: The path planning and exploration strategy is formulated, including: Using the regional semantic map information, the path to the target area is planned, that is, the path planning is obtained; Determine the frontier point set at the boundary of the explored and unexplored areas, select the appropriate frontier point as the action target, and adjust the route when encountering obstacles, thus obtaining the exploration strategy.

8. A region segmentation and exploration estimation device based on geometric features, characterized in that: include: Construction unit, used to construct the wall map; a segmentation unit, configured to perform regional segmentation on the wall map to obtain a regional segmentation result; an acquisition calculation unit, configured to obtain position information and direction information of the robot, and calculate a visible area based on the position information and the direction information; A first calculation unit, configured to calculate an exploration area point set according to the visible area; A second calculation unit is used to calculate the region exploration rate according to the region segmentation result and the exploration region point set; a setting unit, configured to set an exploration priority according to the area exploration rate; formulation unit, used to formulate path planning and exploration strategies; A guidance unit is combined to combine the exploration priority, the path planning and the exploration strategy to guide the robot action.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images