An environment-adaptive navigation strategy adjustment system based on visual semantic segmentation

Abstract

This invention relates to the field of image enhancement technology, and more particularly to an environment-adaptive navigation strategy adjustment system based on visual semantic segmentation. The environment-adaptive navigation strategy adjustment system based on visual semantic segmentation includes a semantic annotation module, an environment understanding module, a navigation decision module, and a motion control module. The semantic annotation module acquires environmental images, performs noise suppression on the environmental images to generate standard environmental images, performs pixel-level semantic annotation on the standard environmental images, and outputs semantic segmentation results. The environment understanding module extracts semantic constraint information based on the semantic segmentation results, constructs an environmental semantic feature map based on the semantic constraint information, and identifies the environment type based on the environmental semantic feature map to generate an environment type label. This invention uses image enhancement technology to segment and extract environment type labels, dynamically adjusts navigation strategies and speed curves, and improves scene perception intelligence and path decision rationality.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to an environment-adaptive navigation strategy adjustment system based on visual semantic segmentation. Background Technology

[0002] Existing robot autonomous navigation and environmental perception technologies typically rely on a single type of navigation strategy or manually set path planning methods for motion control. For example, they may use fixed path planning rules, simple obstacle avoidance strategies, or manually adjusted motion parameters to achieve robot movement. While this approach can largely accomplish navigation tasks in environments with simple structures, sparse obstacles, or well-defined target paths, it becomes increasingly difficult to accurately perceive environmental semantics and dynamically generate optimal navigation strategies in complex environments or scenarios involving parallel analysis of multi-source visual information. This is due to the complexity of the environment, the dense distribution of obstacles, the large variations in channel width, and the presence of dynamic targets. While existing research can achieve obstacle avoidance through manually set navigation paths or simple obstacle detection based on LiDAR / cameras, it generally suffers from the following problems: First, it can only achieve static or approximate path planning, lacking real-time analysis of environmental changes and robot dynamic states; second, it cannot perform fine-grained classification of the environment and channel connectivity analysis based on multi-source visual and semantic information, resulting in a lack of adaptability in navigation strategies and insufficient efficiency and safety for robots in complex environments; third, current navigation decisions and motion control rely heavily on manual operation, empirical parameters, or low-level perception information, lacking the ability for automatic optimization, intelligent decision-making, and dynamic path adjustment based on environmental semantic understanding. Summary of the Invention

[0003] Therefore, it is necessary for the present invention to provide an environment adaptive navigation strategy adjustment system based on visual semantic segmentation to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, an environment-adaptive navigation strategy adjustment system based on visual semantic segmentation includes the following modules:

[0005] The semantic annotation module is used to acquire environmental images, suppress noise in the environmental images, and generate standard environmental images; it then performs pixel-level semantic annotation on the standard environmental images and outputs semantic segmentation results; specifically, the semantic annotation module performs pixel-level semantic annotation on the standard environmental images and outputs semantic segmentation results including:

[0006] A standard environment image is input into a pre-trained visual semantic segmentation network, which includes an encoder layer, a feature pyramid fusion layer, and a decoder layer.

[0007] The encoder layer is used to perform multi-scale convolution extraction on standard environmental images to generate navigation-aware feature maps;

[0008] The feature pyramid fusion layer is used to upsample and laterally connect the navigation-aware feature maps to generate a fused feature map.

[0009] The decoder layer is used to generate pixel-level class probability maps based on the fused feature maps, and to perform threshold segmentation on the pixel-level class probability maps to output semantic segmentation results;

[0010] The environment understanding module is used to extract semantic constraint information based on the semantic segmentation results; construct an environment semantic feature map based on the semantic constraint information; identify the environment type based on the environment semantic feature map, and generate an environment type label for the environment type.

[0011] The navigation decision module is used to acquire navigation target data; generate an adaptive navigation strategy based on the navigation target data and environment type labels; determine the forward path based on the adaptive navigation strategy, and record the path point sequence of the forward path.

[0012] The motion control module is used to convert the path point sequence into motion control instructions and execute the motion control instructions to complete the robot's forward movement task.

[0013] The beneficial effects of this invention are as follows:

[0014] (1) By performing noise suppression and pixel-level semantic annotation on environmental images through the semantic annotation module, obstacles, passable areas and other environmental semantic information can be accurately identified, achieving high precision and high resolution of environmental perception, and solving the problems of rough perception and incomplete obstacle recognition in traditional navigation systems.

[0015] (2) The environment understanding module constructs an environment semantic feature map based on semantic constraint information and analyzes the channel connectivity and environment structure to realize the automatic classification of open scenes and narrow channels. The accuracy of environment type recognition is ≥95%, which provides a reliable basis for navigation strategy and improves the intelligence level of navigation decision.

[0016] (3) The navigation decision module combines navigation target data and environment type labels to generate an adaptive navigation strategy. It can dynamically adjust the steering angle, travel speed and movement step length according to the environment type, realize the unified control of fast navigation in open scenes and precise obstacle avoidance in narrow passages, and improve navigation efficiency by about 40% compared with the traditional fixed strategy.

[0017] (4) By generating and smoothing the path point sequence, the planned path is made continuous and conforms to the dynamic constraints. The motion control module calculates motion parameters based on the path points and generates executable control commands to enable the robot to move smoothly and accurately along the planned path, while ensuring dynamic obstacle avoidance and safe driving.

[0018] (5) It supports real-time acquisition and processing of multi-source environmental information, and can dynamically update navigation strategies and motion control parameters according to environmental changes and feedback information, so as to realize intelligent obstacle avoidance, path optimization and efficient autonomous navigation in complex environments. Attached Figure Description

[0019] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0020] Figure 1 This is a schematic diagram of the execution flow of an environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to the present invention;

[0021] Figure 2 This is a flowchart illustrating the steps of the navigation decision module in this invention;

[0022] Figure 3 This is a schematic diagram of a module of an environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to the present invention;

[0023] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0026] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0027] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides an environment-adaptive navigation strategy adjustment system based on visual semantic segmentation. The system includes a semantic annotation module 101, an environment understanding module 102, a navigation decision module 103, and a motion control module 104. The system performs the following steps:

[0028] S1: Acquire environmental images, suppress noise in the environmental images, and generate standard environmental images; perform pixel-level semantic annotation based on the standard environmental images, and output semantic segmentation results;

[0029] In one embodiment, the system uses a 1080p industrial-grade RGB camera (30Hz) to acquire environmental images. After obtaining intrinsic and extrinsic parameters through camera calibration, the image is subjected to 3×3 median filtering for noise suppression to generate a standard environmental image. Subsequently, the image is input into a pre-trained visual semantic segmentation network, which includes an encoder, a feature pyramid fusion layer, and a decoder. The network outputs a pixel-level category probability map, and semantic segmentation results of obstacles, passable areas, and special landmarks are obtained through threshold segmentation, thereby achieving high-precision semantic recognition of complex environments.

[0030] In another embodiment, assuming 1000 environmental images are collected, each with a resolution of 1920×1080 pixels and a median filtering neighborhood of 5×5; the semantic segmentation network outputs 8 categories (C=8), and the threshold is set to 0.6. Then, approximately 18% of the pixels are identified as obstacles, 72% as passable areas, and the remainder as undetermined areas. The average recognition accuracy is approximately 94%, which can provide reliable input for subsequent environmental understanding and navigation decisions.

[0031] S2: Extract semantic constraint information based on semantic segmentation results; construct an environmental semantic feature map based on semantic constraint information; identify the environment type based on the environmental semantic feature map to generate an environment type label for the environment type;

[0032] In one embodiment, the system statistically analyzes the pixel distribution of obstacles, passable areas, and landmarks in the semantic segmentation results, and constructs an environmental semantic feature map by combining it with channel connectivity analysis. By calculating obstacle density, channel width, and channel connectivity, the system identifies the environment type as an open scene or a narrow channel, and generates environment type labels for use by the navigation decision module, thereby achieving high-precision quantification of environment classification and enhancing the intelligence and adaptability of the navigation strategy.

[0033] In another embodiment, assuming that obstacles account for 20% of the pixels, the passable area accounts for 65%, and the maximum passable width is 2.3m, it is identified as an open scene; if obstacles account for 35%, the passable area accounts for 50%, and the maximum passable width is 0.6m, it is identified as a narrow passage; the accuracy of environment type recognition is about 95%, which can provide a reliable basis for path planning and speed adjustment.

[0034] Of particular importance, S2 includes:

[0035] Semantic constraint information is extracted from the semantic segmentation results; a semantic constraint information matrix is ​​calculated based on the semantic constraint information; semantic classification is performed based on the semantic constraint information matrix to obtain semantic labels; feature encoding is performed based on the semantic labels to obtain feature vectors; and an environmental semantic feature map is constructed based on the feature vectors.

[0036] In one embodiment, RGB images and depth maps of the target environment are acquired, and semantic segmentation (e.g., using DeepLabv3+ or SegNet) is performed on the images to obtain the semantic category probability distribution for each pixel or voxel. Semantic constraint information is extracted based on the semantic segmentation results, including passable areas, obstacle areas, and regions of interest (e.g., doorways, stairs). Subsequently, a semantic constraint information matrix is ​​calculated based on the semantic constraint information, and each cell in the matrix is ​​encoded with its corresponding semantic category to obtain a multi-dimensional feature vector (e.g., each cell is a 12-dimensional vector containing semantic category probability, neighborhood mean, and variance). All feature vectors are mapped onto an environmental spatial grid to form an environmental semantic feature map, which is used for subsequent environmental analysis and navigation decisions.

[0037] In another embodiment, it is assumed that the semantic segmentation yields a 512×512 pixel image, with each semantic category comprising approximately 60% of the passable area, 30% of the obstacle area, and 10% of the interest area. After extracting semantic constraint information, a constraint matrix of size 64×64 is constructed, with each matrix unit containing a 12-dimensional encoded vector. By mapping the feature vectors to an environment grid, an environmental semantic feature map is generated, where the continuous passable area is assumed to have 8 main regions, each occupying an average of 50 grid units, with a maximum span of approximately 5m×5m and a minimum span of approximately 1m×1m.

[0038] Calculate channel connectivity based on semantic feature maps; identify regional topological relationships based on channel connectivity; determine environment types based on regional topological relationships; and generate environment type labels based on environment types.

[0039] In one embodiment, connectivity analysis is first performed on the passable units in the environmental semantic feature map, identifying all connected regions using either 8-connectivity (2D) or 26-connectivity (3D) rules. Then, the topological features of each connected region are calculated, including area, aspect ratio, number of branches, and main channel direction. Based on the geometric and topological features of the connected regions, topological relationships between regions (such as connected channels, branching, or isolation) are identified. Finally, based on the topological relationships and geometric features, the environment type is classified into open scenes, narrow passages, or complex intersections, and an environment type label is generated for each region for navigation strategy selection and path planning.

[0040] In another embodiment, it is assumed that 12 main connected regions are identified in the environmental semantic feature map, with the largest connected region having an area of ​​approximately 30 m² and the smallest approximately 3 m²; the aspect ratio ranges from 1.1 to 5.2; and the number of branches ranges from 0 to 3. Based on connectivity analysis, 5 regions are determined to be open scenes (aspect ratio ≤ 2 and branches ≤ 1), 4 regions are narrow passages (aspect ratio > 3 and area < 10 m²), and the remaining 3 regions are complex intersections (branches ≥ 2). Finally, environment type labels are generated for all regions, with open scenes labeled "E01", narrow passages labeled "E02", and complex intersections labeled "E03", forming a complete environment type label map for subsequent navigation and path optimization.

[0041] S3: Acquire navigation target data; generate an adaptive navigation strategy based on the navigation target data and environment type label; determine the forward path based on the adaptive navigation strategy, and record the path point sequence of the forward path;

[0042] In one embodiment, the system obtains the coordinates of the navigation target and generates an adaptive navigation strategy by combining the environment type label: in open scenes, the maximum turning angle is set to ±45° / second and the travel speed is increased by 50%; in narrow passages, the step size is reduced to 0.3m and the travel speed is reduced by 30%; then, ray detection of unobstructed areas is performed within the direction search angle range to generate a continuous forward path, and the path point sequence is recorded for use by the motion control module, so as to achieve the continuity and high safety of path planning.

[0043] In another embodiment, assuming the current navigation target is 12m from the starting point and the environment type is a narrow passage; the direction search angle range is ±30°, the step size is 0.3m, a total of 40 path points are generated, with an average spacing of 0.28m; the maximum turning angle is limited to ±15° / second, the planned path passes through the gap between two obstacles, ensuring that the robot moves forward smoothly and avoids obstacles successfully, and the total path planning time is about 22 seconds.

[0044] S4: Convert the path point sequence into motion control instructions and execute the motion control instructions to complete the robot's forward movement task.

[0045] In one embodiment, the system smooths the path point sequence, calculates the motion parameters (linear velocity and angular velocity) corresponding to each path point, and converts them into executable control quantities such as wheel speed or rudder angle; generates continuous motion control commands, monitors sensor feedback in real time to correct path deviations, and ensures that the robot moves smoothly along the planned path, thereby achieving accurate execution and high safety of autonomous navigation tasks.

[0046] In another embodiment, assuming the path point sequence contains 40 points with an average spacing of 0.28m, a linear velocity of 0.6m / s, and an upper limit of angular velocity of 0.15rad / s; control commands are sent to the actuator at 50ms intervals; the robot's actual total travel time is approximately 22 seconds, with an average path deviation of 0.04m, successfully avoiding two obstacles, and the entire navigation task is completed smoothly, providing quantifiable performance indicators for environmental adaptive navigation.

[0047] Of particular importance, S4 includes:

[0048] The path point sequence is smoothed to obtain a corrected path point sequence; motion parameters, including linear velocity and angular velocity, are calculated based on the corrected path point sequence.

[0049] In one embodiment, the acquired original pathpoint sequence is first subjected to cubic spline smoothing to obtain a continuously smoothed corrected pathpoint sequence. Then, based on the corrected pathpoint sequence, motion parameters, including the linear velocity at each pathpoint, are calculated according to a preset sampling interval. and angular velocity The calculation results are then filtered to reduce noise. For example, a weighted average filter is applied to each point using three points before and after it.

[0050] In another embodiment, assuming the original pathpoint sequence contains 50 points, a corrected pathpoint sequence is obtained after spline smoothing. The calculated linear velocity sequence is... m / s, angular velocity sequence is rad / s. This sequence is used for subsequent conversion of executable control variables.

[0051] The motion parameters are converted into executable control quantities; motion control commands are generated based on the executable control quantities; the motion control commands are time-scheduled to obtain the command sending time; the motion control commands are executed according to the command sending time to complete the robot's forward movement task.

[0052] In one embodiment, linear velocity and angular velocity are converted into executable control quantities for each drive unit based on the robot's kinematic model according to the motion parameters. For example, the differential wheel speed of the chassis is mapped to the left and right wheel speed commands, and the angular velocity of the robotic arm joints is mapped to the target motor speed. Then, standardized control commands (such as PWM signals or CAN messages) are generated based on the executable control quantities, and timing scheduling is performed to determine the command transmission schedule. Finally, the motion control commands are executed according to the transmission time sequence to achieve smooth robot movement.

[0053] In another embodiment, assuming the differential wheel robot chassis corresponds to 50 path points, and the left and right wheel speeds at each point are respectively... rad / s and rad / s, the angular velocity of the three joints of the robotic arm is rad / s. Control commands are generated via PWM mapping and sent at 50Hz. The command transmission time sequence is as follows: The robot performs actions along a smooth path without any obvious jumps or abrupt changes.

[0054] Preferably, the semantic annotation module acquires environmental images, performs noise suppression on the environmental images, and generates standard environmental images, including:

[0055] Acquire environmental images and convert them into grayscale images; for any original pixel in the grayscale image, perform a weighted average of the pixel grayscale values ​​within a preset neighborhood centered on the original pixel to obtain the weighted average value.

[0056] In one embodiment, the system uses a 1080p industrial RGB camera (30Hz frame rate) to acquire environmental images and performs grayscale processing on each frame. The RGB three channels of the color image are linearly combined using 0.299R + 0.587G + 0.114B to convert them into grayscale values. Then, for each pixel, a 3×3 or 5×5 pixel preset neighborhood is constructed centered on that pixel. The grayscale values ​​within this preset neighborhood are then weighted and averaged using a weight matrix, with the weights being largest at the center and decreasing towards the edges to reduce noise interference and preserve local detail. Finally, a weighted average grayscale value for each pixel is obtained, providing the initial grayscale basis for subsequent median filtering to generate a standard environmental image.

[0057] In another embodiment, assuming each frame has a resolution of 1920×1080 pixels, a 5×5 neighborhood matrix is ​​used, with a center weight of 0.4 and decreasing weights of 0.15, 0.1, etc., for the surrounding neighborhoods. The processed weighted average value ranges from 0 to 255, with an average of approximately 128 and a standard deviation of approximately 35. The weighted calculation takes approximately 5ms per frame, ensuring real-time performance. This method can effectively suppress noise in low-light or locally reflective conditions, providing stable grayscale input for generating standard environmental images.

[0058] Use the weighted average as the new grayscale value; sort all the new grayscale values ​​from smallest to largest to form a grayscale set; select the median value of the grayscale set and use the median value to update the grayscale value of the original pixel to generate a standard environment image.

[0059] In one embodiment, the system assigns the obtained weighted average value to the corresponding pixel as the initial grayscale value, and sorts the pixel values ​​of the entire image in ascending order to form a grayscale set. Then, it extracts the median value of the set as the global median for pixel value updates; that is, if the pixel grayscale value is higher than the median, it is adjusted downwards, and if it is lower than the median, it is finely adjusted upwards, achieving smoothing of local noise and extreme points. The final output is a standard environmental image with reduced image noise and good preservation of edge and texture information, which can be directly used as input to a visual semantic segmentation network, improving segmentation accuracy and robustness.

[0060] In another embodiment, assuming the image being processed contains 2,073,600 pixels (1920×1080), and the grayscale range after weighted average sorting is 0~255, with a median of 128; after updating the grayscale of each pixel according to the median, the overall grayscale mean of the image is approximately 130, and the standard deviation is approximately 30; local noise (such as single-pixel bright spots or dark spots) is effectively suppressed by approximately 95%, and the loss of edge information is less than 5%. This standard environment image generation process takes approximately 8ms / frame, which can meet the real-time processing requirements of 30Hz and provide stable input for subsequent semantic segmentation.

[0061] Preferably, the encoder layer is used to perform multi-scale convolution extraction on the standard environment image to generate navigation-aware feature maps, including:

[0062] Read the standard environment image and adjust it to the preset input size to generate the adjusted environment image; input the adjusted environment image into the first convolution stage and the second convolution stage in sequence;

[0063] In one embodiment, the system first reads a standardized environmental image (original resolution 1920×1080) and adjusts it to a preset input size (e.g., 640×360 pixels) using bilinear interpolation to ensure consistent input size for the convolutional network, thus adapting to subsequent convolution calculations. The adjusted environmental image is then sequentially fed into a first convolutional stage and a second convolutional stage for processing. The first convolutional stage primarily extracts local texture information, such as wall textures, road surface features, and obstacle edges; the second convolutional stage further extracts global semantic information, including channels, open areas, and target point locations.

[0064] In another embodiment, assuming the original input image is 1280×720 pixels, it is scaled to 512×288 pixels using bilinear interpolation. The first convolutional stage includes three convolutional operations: Conv1 (3×3, 32 channels) → ReLU → MaxPool2×2; Conv2 (3×3, 64 channels) → ReLU → MaxPool2×2; Conv3 (3×3, 128 channels) → ReLU → GlobalAvgPool. The second convolutional stage includes two convolutional layers: Conv4 (3×3, 128 channels) → ReLU; Conv5 (1×1, 64 channels) → ReLU, generating texture visual features and semantic visual features. This ensures a processing time of approximately 10ms per frame under GPU acceleration, supporting real-time navigation scenarios.

[0065] In the first convolution stage, a first convolution operation is performed on the adjusted environment image to output texture visual features; in the second convolution stage, a second convolution operation is performed on the texture visual features to output semantic visual features; the texture visual features and semantic visual features are fused to generate a navigation-aware feature map.

[0066] In one embodiment, the first convolutional stage performs multi-scale convolution operations on the adjusted environmental image to extract local texture information, including ground texture, wall structure, and obstacle boundaries. The generated texture visual feature map is then input into the second convolutional stage, where global semantic information, such as channel width, open areas, and potential target regions, is extracted through convolution and non-linear activation operations. Subsequently, the texture visual features from the first convolutional stage and the semantic visual features from the second convolutional stage are fused along the channel dimension to generate a navigation-aware feature map. The fusion method includes pixel-wise weighted summation or channel concatenation, ensuring that the feature map contains both local details and global semantics, providing reliable input for subsequent navigation strategy generation.

[0067] In another embodiment, assume the output feature map size of the first convolutional stage is 128×72×128, and the output feature map size of the second convolutional stage is 128×72×64. The fusion operation concatenates the channels of the two to obtain a final navigation-aware feature map size of 128×72×192. The fusion uses a channel weighting coefficient of 0.6:0.4 to ensure a balance between texture and semantic information. The feature map can accurately capture approximately 50 obstacle points, 3 channel boundaries, and the center position of open areas in the scene. The computation time per frame is approximately 12ms, which can be used for real-time navigation path planning and environmental adaptability analysis.

[0068] Preferably, the feature pyramid fusion layer is used to upsample and laterally connect the navigation-aware feature maps to generate a fused feature map, including:

[0069] The navigation perception feature map is upsampled to obtain an upsampled feature map; the drivable channel feature map is extracted from the navigation perception feature map; the pixel feature values ​​of the upsampled feature map are read to form a first channel value sequence; the pixel feature values ​​of the drivable channel feature map are read to form a second channel value sequence.

[0070] In one embodiment, the system first upsamples the input navigation-aware feature map (size 128×72×192) to 256×144×192 using bilinear upsampling to improve spatial resolution and facilitate refined channel analysis. Simultaneously, a drivable channel feature map is extracted from the original feature map, typically by obtaining the channel pixel positions through a semantic category mask. Subsequently, the channel feature values ​​of each pixel in the upsampled feature map are read sequentially to generate a first channel value sequence, used to preserve texture and semantic fusion information; and the channel feature values ​​of the corresponding pixels in the drivable channel feature map are read to form a second channel value sequence, used to emphasize channel feasibility features.

[0071] In another embodiment, assuming the upsampled feature map size is 256×144×192, the drivable channel feature map covers approximately 21,000 pixels. Reading the channel values ​​from the upsampled feature map yields a first channel value sequence of length 5,654,688 (256×144×192), and reading the channel values ​​from the drivable channel feature map forms a second channel value sequence of length 4,032,000. These two sequences contain channel texture, spatial semantics, and channel feasibility information, providing raw data for subsequent sequential fusion and channel importance analysis.

[0072] The first channel value sequence and the second channel value sequence are concatenated sequentially to generate an extended channel vector; the channel order is recorded according to the extended channel vector; and a fused feature map is generated based on the channel order.

[0073] In one embodiment, the first channel value sequence and the second channel value sequence are directly concatenated in temporal or spatial order to form an extended channel vector, preserving the integrity of the original texture and channel feasibility information. Subsequently, the original index of each element in the extended channel vector is recorded to form a channel order mapping table. Finally, the extended channel vector is remapped to the original feature map spatial dimensions to generate a fused feature map. This map is spatially aligned with the input environment image and simultaneously incorporates both texture features and drivable channel features in terms of channels, providing complete input for navigation path planning or environmental adaptive decision-making.

[0074] In another embodiment, assuming the length of the first channel value sequence is 5,654,688 and the length of the second channel value sequence is 4,032,000, after sequential concatenation, the resulting extended channel vector has a length of 9,686,688. The channel sequence is recorded using an index table, with the first 192 channels of the original sequence labeled as upsampled features and the last 192 channels labeled as drivable channel features. The extended channel vector is remapped to a spatial size of 256×144 to generate a fused feature map, with a final size of 256×144×384. Each pixel contains both the original texture and channel feasibility features, used as input for subsequent object detection or navigation decision networks.

[0075] Preferably, the decoder layer for generating pixel-level class probability maps based on the fused feature map includes:

[0076] A decoding convolutional layer is applied based on the fused feature map to obtain an initial score map for pixel categories; the normalized probability of pixel categories is calculated based on the initial score map; and the maximum value among the normalized probabilities of pixel categories is selected as the predicted label.

[0077] In one embodiment, the fused feature map (size 256×144×384) generated in the previous stage is input into the decoder convolutional layer for pixel-by-pixel class prediction. The decoder consists of three convolutional layers: Deconv1 (3×3 convolution, 128 channels) → ReLU → upsampling; Deconv2 (3×3 convolution, 64 channels) → ReLU → upsampling; and Deconv3 (1×1 convolution, C-class channels, where C is the number of predefined environment categories, such as drivable / non-drivable / obstacles, etc.). The convolution outputs an initial score map of pixel categories. Softmax normalization is performed on the C-channel values ​​of each pixel to generate a category probability distribution for each pixel. Subsequently, the category corresponding to the channel with the highest probability is selected as the predicted label for each pixel, used for preliminary environment segmentation and channel drivability determination.

[0078] In another embodiment, assume the fused feature map size is 256×144×384, and the number of environment categories C=3 (drivable, obstacle, boundary). After the decoder outputs the initial score map, each pixel receives a score for 3 categories. For example, the pixel in row 100, column 50 has a score of [2.1, 0.8, 0.4], which, after Softmax normalization, yields probabilities of [0.65, 0.25, 0.10]. The maximum value of 0.65, corresponding to category 1 (drivable), is selected as the predicted label. After the entire image pixel prediction is completed, a 256×144 pixel predicted label map is obtained, where drivable category pixels account for approximately 60%, obstacle category approximately 30%, and boundary category approximately 10%.

[0079] Pixel-level category probability maps are generated based on predicted labels.

[0080] In one embodiment, the category probability value corresponding to the predicted label of each pixel is remapped back to the original image space to form a pixel-level category probability map. Each pixel in this probability map contains C-class probability information, which can be used for subsequent path planning, environmental assessment, or navigation decisions. This map not only preserves the predicted label but also reflects the model's uncertainty regarding pixel categories, helping to improve navigation safety and robustness.

[0081] In another embodiment, assuming the predicted label map is 256×144, the number of categories C=3, and the probability value corresponding to each pixel is [0.65, 0.25, 0.10]. The overall probability map statistics show that the average probability for drivable categories is 0.68±0.12, the average probability for obstacle categories is 0.22±0.10, and the average probability for boundary categories is 0.10±0.05. When visualizing the probability map, red represents obstacles, green represents drivable categories, and blue represents boundaries. The color intensity corresponds to the category probability, used to assist in automatic path planning and obstacle avoidance strategy optimization.

[0082] Preferably, the navigation decision module is specifically used for:

[0083] S31: Obtain navigation target data; match the navigation target data and environment type label to determine the navigation target environment type; when the navigation target environment type is an open scene, set the preset maximum turning angle to ±45° and configure the travel speed to 1.5 times the preset initial travel speed, and record it as an open area navigation strategy.

[0084] In one embodiment, the system receives navigation target data, including target location coordinates, target category, and target priority, and simultaneously acquires environment type labels (such as open scene, narrow passage, obstacle-dense area, etc.). By matching the navigation target data with the environment type labels, the current navigation environment type is determined to be an open scene. Subsequently, based on the characteristics of the open scene, the system adjusts the navigation control parameters as follows: setting the preset maximum steering angle to ±45°, and increasing the travel speed by 1.5 times based on the preset initial travel speed. This parameter setting can improve navigation efficiency while ensuring stability, and this setting is recorded as an open area navigation strategy for subsequent path planning and action execution.

[0085] In another embodiment, assume the system receives navigation target data including 5 target points, with environment type labels of "open, open, narrow, open, narrow". After matching, it is found that target points 1, 2, and 4 are located in open environments. The system sets the maximum steering angle to ±44~45°, increases the travel speed from the default 2m / s to 3m / s, and generates an open area navigation strategy. This strategy can generate specific parameter lists for different target points, for example: target point 1: steering ±45° / second, speed 3m / s; target point 2: steering ±44° / second, speed 3.1m / s.

[0086] S32: When the navigation target environment type is a narrow passage, configure the maximum forward step length to be 0.3m and reduce the travel speed to 0.7 times the preset initial travel speed, and record it as a narrow area navigation strategy; merge the open area navigation strategy and the narrow area navigation strategy into an adaptive navigation strategy; determine the forward path according to the adaptive navigation strategy, and record the path point sequence of the forward path.

[0087] In one embodiment, when the system detects that the target point is located in a narrow passage, it automatically adjusts the maximum forward stride to 0.3m and reduces the travel speed to 0.7 times the preset initial travel speed to enhance navigation accuracy and safety. Subsequently, the system merges the open-area navigation strategy with the narrow-area navigation strategy to form an adaptive navigation strategy. During the fusion process, the strategy can be switched according to the real-time environment type, and the forward path is calculated based on the adaptive strategy. A path point sequence is generated, recording the coordinates, speed, and direction of each path node, providing precise trajectory information for the navigation controller.

[0088] In another embodiment, it is assumed that the environment where target points 3 and 5 are located is a narrow passage. The system sets the maximum forward step length to 0.3m and reduces the travel speed from the default 2m / s to 1.4m / s, generating a narrow area navigation strategy. Subsequently, the open area navigation strategy is merged with the narrow area navigation strategy to generate an adaptive strategy. For example, the forward path contains 10 path points: points 1-3 are located in the open area, with a speed of approximately 3m / s and a turning angle of ±45° / s; points 4-7 are located in the narrow passage, with a speed of approximately 1.4m / s and a step length of 0.3m; points 8-10 return to the open area, with the speed restored to 3m / s. The path point sequence records the coordinates, speed, turning angle, and strategy type of each point, providing a complete reference for navigation execution.

[0089] Preferably, the navigation target environment type is determined by matching the navigation target data and the environment type label, including:

[0090] Extract the target location coordinates from the navigation target data; draw an environment type label map based on the environment type label; project the target location coordinates onto the environment type label map to obtain the target environment label;

[0091] In one embodiment, the system first receives navigation target data, including the target's three-dimensional or two-dimensional location coordinates (such as GPS coordinates or indoor positioning coordinates), and obtains environment type labels (such as open scene, narrow passage, obstacle-dense area, etc.). Based on the environment type labels, an environment type label map is generated, with each pixel corresponding to an environment type label. Subsequently, the coordinates of the navigation target are projected onto the environment type label map through spatial mapping or pixel mapping to obtain the target environment label, thereby identifying the environment type in which the target is located and providing basic information for subsequent navigation strategy formulation.

[0092] In another embodiment, it is assumed that the navigation target data received by the system contains 10 target points with two-dimensional coordinates of (12.5, 8.3), (15.2, 7.8), (20.1, 9.5), etc. The system draws an environment type label map with a resolution of 0.1 m / pixel, and each pixel is labeled as open (1) or narrow (0). After projecting the 10 target coordinates onto the label map, the projected label sequence is [1,1,0,1,0,0,1,1,0,1], which is used for subsequent environment type determination.

[0093] Calculate the percentage of pixels in the target environment label; compare the percentage of pixels with a preset scene classification threshold; when the percentage of pixels is greater than the preset scene classification threshold, mark it as an open scene; when the percentage of pixels is less than or equal to the preset scene classification threshold, mark it as a narrow passage; determine the navigation target environment type based on the open scene and the narrow passage.

[0094] In one embodiment, the system counts the number of pixels projected onto the target area on the environment type label map and calculates the percentage of pixels in an open scene. This percentage is compared with a preset scene classification threshold (e.g., 0.6): when the percentage is greater than the threshold, the target environment type is marked as an open scene; when the percentage is less than or equal to the threshold, it is marked as a narrow passage. Finally, the environment type of each target point is comprehensively determined to obtain the complete navigation target environment type, providing a basis for the generation of adaptive navigation strategies.

[0095] In another embodiment, assuming the percentage of open pixels projected from target points 1-10 is [0.72, 0.65, 0.48, 0.81, 0.52, 0.40, 0.77, 0.68, 0.50, 0.74], and the preset scene segmentation threshold is 0.6, the target points in the open scene are marked as 1, 2, 4, 7, 8, and 10 (6 points), and the target points in the narrow passage are marked as 3, 5, 6, and 9 (4 points). Finally, the system generates a navigation target environment type sequence [open, open, narrow, open, narrow, narrow, open, open, narrow, open] based on these markings, which is used for subsequent path planning and navigation strategy adjustment.

[0096] Preferably, the forward path is determined according to the adaptive navigation strategy, and the sequence of waypoints along the forward path is recorded, including:

[0097] The maximum forward step length and the direction search angle range are extracted based on the adaptive navigation strategy; within the direction search angle range, candidate forward directions are generated at preset angle intervals.

[0098] In one embodiment, the system first obtains the current navigation target environment type (such as an open scene or a narrow passage) according to an adaptive navigation strategy, and extracts the maximum forward step length (e.g., 0.5m / step in an open scene and 0.3m / step in a narrow passage) and the direction search angle range (e.g., ±45° or ±20°) accordingly. Then, within this direction search angle range, a sequence of candidate forward directions is generated at preset angle intervals (e.g., 5°) to provide alternative directions for path planning.

[0099] In another embodiment, assuming the current navigation target environment is an open scene, the system extracts a maximum forward step length of 0.6m, a direction search angle range of ±40°, and a preset angle interval of 10°. Then, the generated candidate forward direction sequence is [-40°, -30°, -20°, -10°, 0°, 10°, 20°, 30°, 40°], a total of 9 candidate directions, used for subsequent path feasibility assessment.

[0100] Obtain the current coordinate position; perform ray detection from the current coordinate position with the maximum forward step length to identify the unobstructed area; determine the forward direction based on the unobstructed area; mark the forward path along the forward direction; record the path point sequence of the forward path.

[0101] In one embodiment, the system acquires the current coordinates of the robot or mobile platform and performs ray detection along the candidate forward direction with the maximum step length to identify whether there are obstacles and drivable areas ahead. Based on the detection results, the system selects the direction corresponding to the longest obstacle-free ray ahead as the actual forward direction and marks the forward path on the environmental map along this direction. At the same time, it records the path point sequence (including coordinates, direction, and step length information) to provide basic data for real-time navigation and subsequent path optimization.

[0102] In another embodiment, assuming the current coordinates are (12.5m, 8.3m), the maximum forward step length is 0.6m, and the candidate forward direction sequence is [-40°, -30°, -20°, -10°, 0°, 10°, 20°, 30°, 40°]. Ray detection identifies unobstructed ray lengths (in meters) of [0.4, 0.5, 0.6, 0.6, 0.6, 0.5, 0.6, 0.4, 0.3]. The system selects the longest direction, 0°, as the forward direction and records the path point sequence along this direction [(12.5, 8.3), (13.1, 8.3), (13.7, 8.3)], completing one forward decision. Subsequent path points are iteratively updated in the same manner to form a complete forward path.

[0103] Preferably, performing raycasting from the current coordinate position with the maximum step length to identify unobstructed areas includes:

[0104] Construct a ray segment from the current coordinate position along the maximum forward step length; detect the label value of the ray segment, and if the label value is a passable label, mark it as an unobstructed direction;

[0105] In one embodiment, the system obtains the current coordinate position of the mobile platform or robot and constructs a ray segment along a pre-set maximum step length in the candidate direction. For each discrete sampling point on the ray segment, the corresponding label value of the environmental map is read. When the label value belongs to the passable category (such as ground, road, or barrier-free area), the ray segment is marked as a barrier-free direction, providing feasible direction candidates for subsequent path planning.

[0106] In another embodiment, assuming the current coordinates are (12.3m, 8.7m) and the maximum forward step length is 0.6m, the system constructs ray segments along the candidate directions [-30°, -15°, 0°, 15°, 30°], sampling 6 points at 0.1m intervals for each ray segment. The detected label values ​​for each ray segment are as follows: 30° direction: [accessible, accessible, accessible, obstacle, obstacle, obstacle]; -15° direction: [accessible, accessible, accessible, accessible, accessible, accessible, accessible]; 0° direction: [accessible, accessible, accessible, accessible, accessible, accessible]; 15° direction: [accessible, obstacle, obstacle, obstacle, obstacle, obstacle]; 30° direction: [accessible, accessible, obstacle, obstacle, obstacle, obstacle].

[0107] Identify accessible areas based on the direction of accessibility.

[0108] In one embodiment, the system uses the marked barrier-free direction as a basis to divide the range of continuous passable ray segments into barrier-free areas. These barrier-free areas include not only directional information but also ray segment lengths, starting coordinates, and coverage areas, providing the path planning module with feasible region candidates to select the optimal direction of travel and mark the path.

[0109] In another embodiment, assuming the identified unobstructed directions are -15° and 0°, the corresponding continuous passable lengths of the ray segments are 0.6m and 0.6m, respectively. The system constructs rectangular or fan-shaped unobstructed areas along these two ray segments, forming unobstructed areas with areas of 0.12m² and 0.15m², respectively, and records the coordinate sequence of its center point [(12.6,8.7),(12.9,8.7)] for subsequent navigation path selection and forward strategy decision-making.

[0110] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0111] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. An environment-adaptive navigation strategy adjustment system based on visual semantic segmentation, characterized in that, Includes the following modules: The semantic annotation module is used to acquire environmental images, suppress noise in the environmental images, and generate standard environmental images; it then performs pixel-level semantic annotation on the standard environmental images and outputs semantic segmentation results; specifically, the semantic annotation module performs pixel-level semantic annotation on the standard environmental images and outputs semantic segmentation results including: A standard environment image is input into a pre-trained visual semantic segmentation network, which includes an encoder layer, a feature pyramid fusion layer, and a decoder layer. The encoder layer is used to perform multi-scale convolution extraction on standard environmental images to generate navigation-aware feature maps; The feature pyramid fusion layer is used to upsample and laterally connect the navigation-aware feature maps to generate a fused feature map. The decoder layer is used to generate pixel-level class probability maps based on the fused feature maps, and to perform threshold segmentation on the pixel-level class probability maps to output semantic segmentation results; The environment understanding module is used to extract semantic constraint information based on the semantic segmentation results; construct an environment semantic feature map based on the semantic constraint information; identify the environment type based on the environment semantic feature map to generate an environment type label, where the environment type label includes open scene, narrow passage, and obstacle-dense area; The navigation decision module is used to acquire navigation target data, which includes target location coordinates, target category, and target priority; generate an adaptive navigation strategy based on the navigation target data and environment type labels; determine the forward path based on the adaptive navigation strategy, and record the path point sequence of the forward path; The motion control module is used to convert the path point sequence into motion control instructions and execute the motion control instructions to complete the robot's forward movement task.

2. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 1, characterized in that, The semantic annotation module acquires environmental images, performs noise suppression on these images, and generates standard environmental images, including: Acquire environmental images and convert them into grayscale images; for any original pixel in the grayscale image, perform a weighted average of the pixel grayscale values ​​within a preset neighborhood centered on the original pixel to obtain the weighted average value. Use the weighted average as the new grayscale value; sort all the new grayscale values ​​from smallest to largest to form a grayscale set; select the median value of the grayscale set and use the median value to update the grayscale value of the original pixel to generate a standard environment image.

3. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 1, characterized in that, The encoder layer is used to perform multi-scale convolutional extraction on standard environment images to generate navigation-aware feature maps, including: Read the standard environment image and adjust it to the preset input size to generate the adjusted environment image; input the adjusted environment image into the first convolution stage and the second convolution stage in sequence; In the first convolution stage, a first convolution operation is performed on the adjusted environment image to output texture visual features; in the second convolution stage, a second convolution operation is performed on the texture visual features to output semantic visual features; the texture visual features and semantic visual features are fused to generate a navigation-aware feature map.

4. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 1, characterized in that, The feature pyramid fusion layer is used to upsample and laterally connect the navigation-aware feature maps to generate a fused feature map, including: The navigation perception feature map is upsampled to obtain an upsampled feature map; the drivable channel feature map is extracted from the navigation perception feature map; the pixel feature values ​​of the upsampled feature map are read to form a first channel value sequence; the pixel feature values ​​of the drivable channel feature map are read to form a second channel value sequence. The first channel value sequence and the second channel value sequence are concatenated sequentially to generate an extended channel vector; the channel order is recorded according to the extended channel vector; and a fused feature map is generated based on the channel order.

5. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 1, characterized in that, The decoder layer, used to generate pixel-level class probability maps based on the fused feature map, includes: A decoding convolutional layer is applied based on the fused feature map to obtain an initial score map for pixel categories; the normalized probability of pixel categories is calculated based on the initial score map; and the maximum value among the normalized probabilities of pixel categories is selected as the predicted label. Pixel-level category probability maps are generated based on predicted labels.

6. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 1, characterized in that, The navigation decision module is specifically used for: Obtain navigation target data; match the navigation target data and environment type labels to determine the navigation target environment type; when the navigation target environment type is an open scene, set the preset maximum turning angle to ±45° and configure the travel speed to 1.5 times the preset initial travel speed, and record it as an open area navigation strategy. When the navigation target environment type is a narrow passage, the maximum forward step length is configured to be 0.3m and the travel speed is reduced to 0.7 times the preset initial travel speed, which is also recorded as the narrow area navigation strategy; The open area navigation strategy and the narrow area navigation strategy are combined into an adaptive navigation strategy; The forward path is determined based on the adaptive navigation strategy, and the sequence of waypoints along the forward path is recorded.

7. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 6, characterized in that, Based on the navigation target data and environment type labels, the navigation target environment type is determined by matching, including: Extract the target location coordinates from the navigation target data; draw an environment type label map based on the environment type label; project the target location coordinates onto the environment type label map to obtain the target environment label; Calculate the percentage of pixels in the target environment label; compare the percentage of pixels with a preset scene classification threshold; when the percentage of pixels is greater than the preset scene classification threshold, mark it as an open scene; when the percentage of pixels is less than or equal to the preset scene classification threshold, mark it as a narrow passage; determine the navigation target environment type based on the open scene and the narrow passage.

8. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 6, characterized in that, The forward path is determined based on the adaptive navigation strategy, and the sequence of waypoints along the forward path is recorded, including: The maximum forward step length and the direction search angle range are extracted based on the adaptive navigation strategy; within the direction search angle range, candidate forward directions are generated at preset angle intervals. Obtain the current coordinate position; perform ray detection from the current coordinate position with the maximum forward step length to identify the unobstructed area; determine the forward direction based on the unobstructed area; mark the forward path along the forward direction; record the path point sequence of the forward path.

9. The environment adaptive navigation strategy adjustment system based on visual semantic segmentation according to claim 8, characterized in that, Perform raycasting from the current coordinate position with the maximum step length to identify unobstructed areas, including: Construct a ray segment from the current coordinate position along the maximum forward step length; detect the label value of the ray segment, and if the label value is a passable label, mark it as an accessible direction; identify accessible areas based on the accessible direction.

Images