Intelligent identification method for terrain probability distribution based on lightweight convolutional neural network

Abstract

The present application provides a kind of topography probability distribution intelligent identification method based on lightweight convolutional neural network, constructs special terrain identification network by fusing the advantages of MobileNetV3 and ShuffleNetV2, improves model accuracy by combining channel attention mechanism and knowledge distillation technology, realizes the real-time inference of embedded platform using structured pruning and INT8 quantization.At the same time, this method not only outputs the probability distribution of five types of terrain and the confidence evaluation based on information entropy, but also realizes the unsupervised ROI automatic positioning through the improved Grad-CAM++ algorithm, without additional labeling of boundary box data.The present application can realize high-precision, high-robustness terrain identification with very low hardware cost and computing overhead, and provide reliable environment perception basis for exoskeleton proactive adaptive control.

Description

Technical Field

[0001] This invention relates to the field of exoskeleton application technology, and in particular to a method for intelligent recognition of terrain probability distribution based on a lightweight convolutional neural network, as well as a system, electronic device, and computer-readable storage medium for intelligent recognition of terrain probability distribution based on a lightweight convolutional neural network. Background Technology

[0002] Lower limb exoskeleton robots, as wearable intelligent devices, have demonstrated significant application value in fields such as rehabilitation medicine, industrial assistance, and military logistics. Their core capability lies in sensing the user's movement intentions and providing precise assistance, while accurate terrain recognition is a crucial prerequisite for achieving adaptive control in complex terrains. However, existing exoskeleton terrain recognition technologies still face numerous unresolved technical bottlenecks, severely hindering their widespread application in real-world scenarios.

[0003] First, the inherent lag in reactive control is the primary problem faced by existing systems. Traditional exoskeletons rely entirely on proprioceptors (such as IMUs and joint encoders) for gait recognition and control, representing a typical "reactive" control architecture. The system can only adjust its control strategy based on detected changes in motion (such as sudden joint angle changes or acceleration impacts) when the user's feet actually come into contact with new terrain (such as stepping onto a staircase). This lag leads to incorrect timing of the application of assist torque, resulting in a noticeable "dragging" or "insufficient assistance" sensation for the user. In complex terrains such as stairs and slopes, this can even cause tripping and falls, leading to safety accidents.

[0004] Secondly, monocular vision suffers from insufficient accuracy and robustness. Low-cost monocular cameras, due to their small size, low power consumption, and low cost, have become the preferred sensor for exoskeleton environmental perception. However, monocular vision lacks depth information and is susceptible to factors such as lighting changes, texture loss, occlusion, and motion blur. Traditional terrain classification methods based on handcrafted features (such as HOG, SIFT, and LBP) rely on manually designed feature extractors, resulting in poor generalization ability and a classification accuracy of only 65%-75% in complex scenes, failing to meet the reliability requirements of clinical and industrial applications.

[0005] Third, the embedded deployment of deep learning models is challenging. In recent years, deep learning-based computer vision technology has made groundbreaking progress, achieving accuracy far exceeding traditional methods in image classification tasks. However, existing high-performance convolutional neural networks (such as VGG16, ResNet50, and InceptionV3) have a massive number of parameters (millions to hundreds of millions) and high computational complexity, requiring powerful GPUs for real-time inference. Exoskeletons, as portable wearable devices, typically use embedded microcontrollers (MCUs) or edge AI chips with limited computing power and power consumption, making them unable to support the operation of large-scale deep learning models. Some studies have attempted to use lightweight networks such as MobileNet and SqueezeNet, but these have not been specifically optimized for exoskeleton terrain recognition tasks, resulting in insufficient feature extraction capabilities and a significant drop in accuracy under complex lighting and changing viewing angles.

[0006] Fourth, the output information is limited and lacks the ability to quantify uncertainty. Most existing terrain recognition methods only output a single terrain classification result (such as "going up stairs"), failing to provide classification confidence and probability distribution information for each terrain type. This prevents subsequent multi-sensor fusion modules from performing uncertainty assessment and decision weight allocation. When the visual recognition result is ambiguous (such as misclassifying a slope as stairs), the system cannot trigger an effective verification mechanism, easily leading to incorrect control strategies and severely impacting user experience and security.

[0007] Fifth, the localization of terrain regions of interest (ROIs) relies on additional annotations. To achieve more accurate estimation of terrain parameters (such as step height and slope angle), it is necessary to first locate the terrain regions in the image. Existing methods mostly use object detection networks (such as YOLO and SSD) for ROI localization, which requires a large dataset of labeled bounding boxes, resulting in high annotation costs. Furthermore, the large number of parameters in object detection models further increases the difficulty of embedded deployment. Summary of the Invention

[0008] To address the technical problems existing in the prior art, the present invention provides the following technical solution: On the one hand, a method for intelligent recognition of terrain probability distribution based on a lightweight convolutional neural network is provided, including the following steps: S1: Acquire raw RGB images of the environment in front of you using a monocular camera mounted on the exoskeleton torso; S2: Perform adaptive illumination compensation, scaling and cropping, standardization and online data augmentation preprocessing on the original image to obtain a standardized image; S3: Input the standardized image into the improved Mobile Shuffle Net-Terrain lightweight convolutional neural network model, perform forward propagation, and output the raw terrain prediction score and the feature map of the last convolutional layer; S4: Calculate the probability distribution and multi-dimensional comprehensive confidence of various terrain types based on the original prediction scores; S5: Generate Grad-CAM++ heatmaps based on feature maps and classification results, and extract regions of interest (ROI) and key feature points of the terrain; S6: Package and output the terrain probability distribution, confidence level, ROI information and key feature points according to the standard protocol.

[0009] Preferably, the adaptive illumination compensation in step S2 employs an algorithm based on Retinex theory, decomposing the image into illumination and reflection components. The image contrast is enhanced by adjusting the illumination component, as shown in the formula: in, For the original image, For light component, This is the reflection component.

[0010] Preferably, the improved Mobile Shuffle Net-Terrain model in step S3 is based on MobileNetV3 architecture, replacing the inverted residual block with ShuffleNetV2 basic units, and adding an efficient channel attention (ECA) module after each unit. The ECA module achieves local cross-channel interaction through adaptive convolution kernel size, and the channel attention weights... : in: The feature vector after global average pooling. The kernel size is One-dimensional convolution, For the sigmoid function, kernel size Based on the number of channels Adaptive determination.

[0011] Preferably, the model described in step S3 is trained using knowledge distillation, with ResNet50 as the teacher network and Mobile Shuffle Net-Terrain as the student network; the training process employs a composite loss function of label smoothing cross-entropy loss and KL divergence loss. :

[0012] in: For cross-entropy loss, For the predicted probability distribution of the student network, This is a real label; For KL divergence loss, The predicted probability distribution for the teacher network; The temperature coefficient is used to soften the probability distribution, enabling the student network to learn more information about inter-category similarity. This is the balancing coefficient, used to adjust the weights of the two parts of the loss.

[0013] Preferably, the model described in step S3 undergoes structured pruning and INT8 quantization. The channel pruning method based on L1 regularization is used to prune channels with a scaling factor of less than 0.001 in the BatchNorm layer, with a pruning ratio of 30%. INT8 quantization is performed using ONNXRuntime to convert the 32-bit floating-point weights and activation values ​​into 8-bit integers.

[0014] Preferably, the probability distribution calculation in step S4 uses a modified Softmax function, which prevents numerical overflow by subtracting the maximum original score. The formula is as follows: in: For the first The original prediction score of the class. The maximum value among all original predicted scores.

[0015] Preferably, the multi-dimensional comprehensive confidence level in step S4 is calculated based on information entropy, maximum probability value, and probability distribution dispersion, using the following formula: in: For information entropy, The maximum probability value, For dispersion, .

[0016] Preferably, the ROI extraction in step S5 uses the Grad-CAM++ algorithm, which includes the following steps: Calculate the gradient of the target category score relative to the last layer of feature map; The weight of each channel is calculated based on the higher-order gradient information. ; Generate a Grad-CAM++ heatmap using the following formula: in: For channel weights, For the first Feature maps of each channel; The heatmap is upsampled to the original image size, and after thresholding and morphological processing, the largest connected region is extracted as the terrain ROI by contour detection.

[0017] Preferably, the output in step S6 adopts the CANFD bus communication protocol, and the data frame includes terrain probability distribution, classification confidence, ROI bounding box coordinates, key feature point coordinates, timestamp and CRC check code.

[0018] On the other hand, a terrain probability distribution intelligent recognition system based on a lightweight convolutional neural network is provided, including: The image acquisition and adaptive preprocessing module is used to acquire images of the foreground environment and perform adaptive illumination compensation, scaling and cropping, and standardization preprocessing to output a standardized image. An improved lightweight CNN terrain classification module is adopted, using the Mobile Shuffle Net-Terrain model to extract features and classify terrain from standardized images, outputting the original prediction score and the feature map of the last convolutional layer; The multi-dimensional probability distribution and confidence assessment module is used to calculate the probability distribution and multi-dimensional comprehensive confidence of various terrain types. The Grad-CAM++ ROI localization and feature extraction module is used to generate Grad-CAM++ heatmaps based on feature maps and classification results, and to extract terrain ROIs and key feature points. The standardized multi-protocol data output interface module is used to package and output terrain probability distribution, confidence level, ROI information and key feature points to the exoskeleton main control system according to the CANFD protocol.

[0019] On the other hand, an electronic device is provided, comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the method described above.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement the above method.

[0021] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: 1. Extremely lightweight design for real-time inference on embedded platforms: The proposed Mobile ShuffleNet-Terrain network combines the advantages of MobileNetV3 and ShuffleNetV2, along with structured pruning and INT8 quantization techniques. The model has only 0.84M parameters and a file size of only 1.1MB, achieving an inference speed of 18.2fps on the K210 edge AI chip, meeting the real-time requirements of exoskeletons. Compared to existing lightweight models, the inference speed is improved by more than 3 times, and memory usage is reduced by more than 70%, solving the problem of difficult deployment of deep learning models on embedded platforms.

[0022] 2. High-precision terrain recognition with significantly enhanced robustness: Through improved network structure, knowledge distillation technology, and adaptive illumination compensation preprocessing, the classification accuracy of the method in this invention reaches 93.8%, which is more than 23% higher than traditional manual feature methods and more than 5% higher than the original lightweight model. Even in complex environments such as varying illumination and missing textures, the accuracy decreases by only 6.3%, demonstrating significantly enhanced system robustness.

[0023] 3. Multi-dimensional probability distribution and confidence assessment provide rich decision-making basis: This invention not only outputs a single terrain classification result, but also provides probability distributions for five terrain types and multi-dimensional confidence assessment based on information entropy, maximum probability value, and dispersion. This provides uncertainty quantification basis for subsequent multi-sensor fusion modules, enabling different verification mechanisms to be triggered according to the confidence level, thus improving the overall accuracy after multi-sensor fusion.

[0024] 4. Unsupervised ROI auto-localization without additional annotation costs: An improved Grad-CAM++ algorithm is used to achieve unsupervised auto-localization of terrain ROIs, eliminating the need for labeled bounding box data and significantly reducing dataset construction costs. ROI localization accuracy and key feature point extraction accuracy both exceed 90%, providing precise location information for subsequent terrain parameter estimation and control strategy adjustment.

[0025] 5. Standardized multi-protocol interface for easy integration and expansion: The system supports multiple industrial standard communication protocols such as CANFD, UART, and SPI, with standardized data frame formats, enabling seamless integration with exoskeleton control systems, multi-sensor fusion modules, and other peripherals. The system adopts a modular design, facilitating subsequent functional expansion and maintenance upgrades, and possesses strong practicality and widespread application value. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the method mechanism of the present invention; Figure 2 This is a schematic diagram of the system architecture of the present invention; Figure 3 This is a diagram of the network structure of the improved lightweight CNN terrain classification model of the present invention; Figure 4 This is a schematic diagram illustrating the working mechanism of the multi-dimensional probability distribution and confidence assessment module of the present invention; Figure 5 This is a schematic diagram of the Grad-CAM++ ROI localization and feature extraction process of the present invention; Figure 6 This is a schematic diagram of the method steps of the present invention; Figure 7 This is a schematic diagram of a hardware platform configuration system provided in Embodiment 1 of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0029] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0030] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0031] In this embodiment of the invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0032] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0033] I. Purpose of the Invention Combined with appendix Figure 1 The flowchart shown illustrates an intelligent terrain probability distribution recognition method based on an improved lightweight convolutional neural network, comprising the following steps: S1: Acquiring raw RGB images of the foreground environment using a monocular camera mounted on the exoskeleton torso; S2: Preprocessing the raw images with adaptive illumination compensation, scaling and cropping, standardization, and online data augmentation to obtain a standardized image; S3: Inputting the standardized image into an improved Mobile Shuffle Net-Terrain lightweight convolutional neural network model for forward propagation, outputting the raw terrain prediction score and the feature map of the last convolutional layer; S4: Calculating the probability distribution and multi-dimensional comprehensive confidence score of various terrain types based on the raw prediction score; S5: Generating a Grad-CAM++ heatmap based on the feature map and classification results, extracting the region of interest (ROI) and key feature points; S6: Packaging and outputting the terrain probability distribution, confidence score, ROI information, and key feature points according to a standard protocol.

[0034] This method is implemented based on the following system architecture: like Figure 2 As shown, the system consists of five core units: an image acquisition and adaptive preprocessing module, an improved lightweight CNN terrain classification module, a multi-dimensional probability distribution and confidence evaluation module, a Grad-CAM++ ROI localization and feature extraction module, and a standardized multi-protocol data output interface module. The image acquisition and adaptive preprocessing module is used to acquire images of the foreground environment and perform adaptive illumination compensation, scaling and cropping, and standardization preprocessing to output a standardized image. An improved lightweight CNN terrain classification module is adopted, using the Mobile Shuffle Net-Terrain model to extract features and classify terrain from standardized images, outputting the original prediction score and the feature map of the last convolutional layer; The multi-dimensional probability distribution and confidence assessment module is used to calculate the probability distribution and multi-dimensional comprehensive confidence of various terrain types. The Grad-CAM++ ROI localization and feature extraction module is used to generate Grad-CAM++ heatmaps based on feature maps and classification results, and to extract terrain ROIs and key feature points. The standardized multi-protocol data output interface module is used to package and output terrain probability distribution, confidence level, ROI information and key feature points to the exoskeleton main control system according to the CANFD protocol.

[0035] Each module interacts with other exoskeleton systems via an internal bus for high-speed data exchange, and connects externally through standardized communication interfaces to collaboratively complete the fully automated processing from raw image input to terrain probability distribution output. The overall system architecture follows the design principles of high cohesion and low coupling, facilitating subsequent functional expansion and maintenance upgrades.

[0036] The functions of each module are as follows: (I) Image Acquisition and Adaptive Preprocessing Module Image acquisition uses an OV5640 monocular CMOS camera module (multi-lens can also be used), with a resolution of 640×480, a frame rate of 30fps, and support for automatic exposure and automatic white balance. The camera is mounted in the center of the front of the exoskeleton's waist, tilted downwards at a 15° angle to the horizontal plane, covering a range of 0.5-5 meters in front of the direction of travel, ensuring complete capture of terrain information in front of the user.

[0037] The module preprocesses the raw BGR format image (640×480×3) captured by the camera and outputs a preprocessed normalized image tensor (1×3×160×160). The specific process includes the following steps: a. Color space conversion: Convert the BGR format image output by the camera to RGB format to match the input requirements of the deep learning model.

[0038] b. Adaptive Illumination Compensation: To address the issue of uneven illumination in images acquired by the exoskeleton under different lighting conditions (such as strong indoor light, outdoor shadows, and nighttime lighting), an adaptive illumination compensation algorithm based on Retinex theory is employed. This algorithm decomposes the image into illumination and reflection components, and enhances image contrast and detail by non-linearly adjusting the illumination component. The core formula is: in, For the original image, For light component, This refers to the reflection component. The illumination component is estimated. It is then compressed to obtain the enhanced reflection component. This refers to the image after illumination compensation.

[0039] c. Image Scaling and Cropping: The image is scaled from 640×480 to 256×256 using bilinear interpolation, and then center-cropped to obtain a 160×160 image to match the input size requirements of the CNN model. The formula for bilinear interpolation is: in, For the target pixel coordinates, This represents the relative position of the target pixel among its four adjacent pixels in the original image. These are the gray values ​​of four adjacent pixels in the original image.

[0040] d. Data Standardization: Linearly map image pixel values ​​from [0,255] to the [0,1] interval, and then standardize using the mean and standard deviation of the ImageNet dataset. in: For the first The passage is The original pixel value of the location. and The first The mean and standard deviation of the channels. For example, the mean values ​​of the three RGB channels are 0.485, 0.456, and 0.406, and the standard deviations are 0.229, 0.224, and 0.225, respectively.

[0041] e. Online Data Augmentation: During model training, data augmentation techniques such as random horizontal flipping (probability 0.5), random rotation (±15°), random brightness adjustment (±20%), random contrast adjustment (±15%), and random cropping are used to expand the diversity of the training dataset and improve the model's generalization ability and robustness. No data augmentation is performed during inference to ensure inference speed and consistency of results.

[0042] (ii) Improve the lightweight CNN terrain classification module This module is used to identify and output the raw prediction scores (logits) of 5 types of terrain and the feature map (5×5×192) of the last convolutional layer through a classification model.

[0043] like Figure 3 As shown, this module proposes a lightweight terrain recognition network (Mobile Shuffle Net-Terrain) that combines the advantages of MobileNetV3 and ShuffleNetV2. It is deeply optimized for exoskeleton terrain recognition tasks, and significantly improves feature extraction capabilities and classification accuracy while ensuring the model's lightweight design.

[0044] 1. Basic Network Structure Design The Mobile Shuffle Net-Terrain network adopts the overall architecture design concept of MobileNetV3, replacing the inverted residual block with the improved ShuffleNetV2 basic unit, combining the advantages of both: ShuffleNetV2's channel shuffle operation can effectively promote information exchange between different channels and improve feature utilization; while MobileNetV3's depthwise separable convolution and SE channel attention mechanism can enhance the model's attention to important features while reducing computational cost.

[0045] The specific structure of the network is shown in the table below: initial convolutional layer 160×160×3 80×80×16 3×3 2 16 H-Swish Shuffle bottleneck block 1 80×80×16 40×40×24 3×3 2 24 ReLU Shuffle bottleneck block 2 40×40×24 40×40×24 3×3 1 24 ReLU Shuffle bottleneck block 3 40×40×24 20×20×48 3×3 2 48 H-Swish Shuffle bottleneck block 4 20×20×48 20×20×48 3×3 1 48 H-Swish Shuffle bottleneck block 5 20×20×48 20×20×48 3×3 1 48 H-Swish Shuffle bottleneck block 6 20×20×48 10×10×96 3×3 2 96 H-Swish Shuffle bottleneck block 7 10×10×96 10×10×96 3×3 1 96 H-Swish Shuffle bottleneck block 8 10×10×96 10×10×96 3×3 1 96 H-Swish Shuffle bottleneck block 9 10×10×96 10×10×192 3×3 1 192 H-Swish Global average pooling layer 10×10×192 1×1×192 - - 192 - Fully connected layer 1 1×1×192 1×1×96 - - 96 H-Swish Dropout layer 1×1×96 1×1×96 - - 96 - Fully connected layer 2 (classification layer) 1×1×96 1×1×5 - - 5 - A lightweight, efficient Channel Attention (ECA) module is added after each Shuffle bottleneck block. The ECA module adaptively selects the kernel size to achieve local cross-channel interaction, significantly improving the model's feature extraction capability with extremely low computational overhead. The core formula of the ECA module is: in, The feature vector after global average pooling. The kernel size is One-dimensional convolution, For the sigmoid function, Channel attention weights. Kernel size. Based on the number of channels The adaptive determination is based on the following formula: in, , , This indicates taking the nearest odd number.

[0046] 2. Knowledge distillation technique improves model accuracy To further improve the accuracy of the lightweight model, knowledge distillation is employed. The large, high-precision ResNet50 network is used as the teacher network, and the Mobile Shuffle Net-Terrain proposed in this invention is used as the student network for training. Knowledge distillation uses the soft labels output by the teacher network as supervisory information to guide the student network in learning the teacher network's "hidden knowledge," thereby significantly improving the performance of the student network without increasing the number of model parameters or computational cost.

[0047] The loss function for knowledge distillation consists of two parts: the cross-entropy loss between the student network and the real labels, and the KL divergence loss between the student network and the teacher network's soft labels. The total loss function formula is: in: For cross-entropy loss, For the predicted probability distribution of the student network, This is a real label; For KL divergence loss, The predicted probability distribution for the teacher network; The temperature coefficient is used to soften the probability distribution, enabling the student network to learn more inter-category similarity information. In this invention, it is taken as... ; The balancing coefficient is used to adjust the weights of the two parts of the loss. In this invention, it is taken as... .

[0048] 3. Model pruning and quantization To enable the model to perform real-time inference on embedded platforms with limited computing power and power consumption, the trained model undergoes structured pruning and INT8 quantization: Structured pruning: A channel pruning method based on L1 regularization is employed. During training, L1 regularization is applied to the scaling factor (γ parameter of the Batch Norm layer) of each channel, causing the scaling factor of unimportant channels to approach 0. After training, channels with scaling factors less than a preset threshold (0.001 in this invention) are pruned, and then the pruned model is fine-tuned to restore model accuracy. Through structured pruning, 30% of redundant channels are ultimately removed, reducing the number of model parameters from 1.2M to 0.84M, and reducing computational cost (FLOPs) by 35%.

[0049] INT8 Quantization: The pruned model is quantized using the ONNX Runtime. The quantization process converts the 32-bit floating-point (FP32) weights and activation values ​​to 8-bit integers (INT8). Quantization parameters (scaling factor and zeros) are determined using a calibration dataset to minimize quantization error. After INT8 quantization, the model size is further reduced by 75%, inference speed is improved by 3-4 times, and memory usage is reduced by approximately 70%.

[0050] 4. Loss Function In addition to the knowledge distillation loss mentioned above, label smoothing cross-entropy loss is employed during the basic training phase of the model to improve its generalization ability and robustness. Label smoothing reduces the model's overconfidence in its predictions by converting the hard labels of the true labels into soft labels, thereby reducing the risk of overfitting. The formula for label smoothing cross-entropy loss is: in: One-hot encoding of the real label. The model predicts the first Class probability, Number of categories (in this invention) ), The label smoothing coefficient (taken in this invention) ).

[0051] (III) Multidimensional Probability Distribution and Confidence Assessment Module like Figure 4 As shown, this module performs probability distribution and confidence analysis on the original prediction scores (logits) of the improved lightweight CNN output, and finally outputs the probability distribution, classification confidence, and confidence level of various terrain types. The specific analysis process is as follows: a. Probability Distribution Calculation: The original predicted scores are converted into a probability distribution using an improved Softmax function. To prevent numerical overflow, the maximum value is subtracted from all original predicted scores before calculating Softmax. The improved Softmax function formula is as follows: in, For the first The original prediction score of the class. The maximum value among all original predicted scores. For the first The predicted probability of the class, and satisfying .

[0052] Example calculation: Assume the original predicted score output by the model is ,but Calculate the index values ​​for each category:

[0053] , , , , Summing gives the denominator: Therefore, the probability distribution of each class is as follows: That is, flat ground: 15.95%, going up stairs: 2.91%, going down stairs: 0.39%, uphill: 78.98%, downhill: 1.77%.

[0054] b. Multi-dimensional confidence assessment: A multi-dimensional confidence assessment method based on information entropy, maximum probability value and probability distribution dispersion is proposed, which comprehensively considers the uncertainty and discriminative power of the probability distribution to improve the accuracy of confidence assessment.

[0055] Information entropy: measures the uncertainty of a probability distribution. The higher the information entropy, the more uniform the probability distribution and the more uncertain the classification result.

[0056] The formula for calculating information entropy is: Information entropy reaches its maximum value when all categories have equal probabilities. When the probability of one class is 1 and the probabilities of other classes are 0, the information entropy is 0.

[0057] Maximum probability value: The higher the maximum probability value, the more certain the classification result.

[0058] Probability distribution dispersion: measures the degree of difference between the probabilities of each category in a probability distribution. The greater the dispersion, the clearer the classification result.

[0059] The formula for calculating the dispersion is: ,in, This represents the average probability.

[0060] Overall confidence level The calculation formula is: in, Let be the weight coefficient, and satisfy... In this invention, the method is based on experience. , , ; To achieve maximum dispersion, when the probability of one class is 1 and the probabilities of other classes are 0, .

[0061] Confidence The value of is in the range of [0,1], with a larger value indicating a more reliable classification result. The classification results are divided into three levels based on the confidence score: High confidence level: The classification results are reliable and can be output directly. Medium confidence level: The classification results have some uncertainty. The output results are marked as "to be verified" to trigger the laser ranging module for auxiliary verification. Low confidence level: If the classification result is unreliable, it is marked as "uncertain," and the system switches to pure laser ranging or body sensor control mode.

[0062] Example calculation: using the above probability distribution For example: Information entropy: ,

[0063] , Maximum probability value , Average probability , Dispersion: , , Overall confidence level The result is of medium confidence level and requires triggering the laser ranging module for auxiliary verification.

[0064] (iv) The Grad-CAM++ ROI localization and feature extraction module employs an improved Grad-weighted Class Activation Mapping++ (Grad-CAM++) algorithm to achieve unsupervised automatic terrain ROI localization. Compared with traditional CAM and Grad-CAM, Grad-CAM++ can better handle situations where multiple similar targets exist in an image, generating clearer and more accurate heatmaps, and more precisely locating the boundaries of terrain regions.

[0065] like Figure 5 As shown, Grad-CAM++ ROI localization and feature extraction yields the bounding box coordinates (x1, y1, x2, y2) and key feature points (such as stair edges, slope start points, and step corners) of the terrain ROI. The specific implementation steps of Grad-CAM++ are as follows: a. Forward Propagation: The input image is fed into the CNN model for forward propagation, resulting in the feature map of the last convolutional layer. (in and The width and height of the feature map. (Number of channels) and target category Score .

[0066] b. Gradient calculation: Calculate the target category score Relative to feature map gradient .

[0067] c. Weight Calculation: Perform global average pooling on the gradient of each channel to obtain the weight of that channel. Grad-CAM++ improves the weight calculation by considering higher-order information of the gradient, enabling a more accurate assessment of the contribution of each channel to the classification result. The weight calculation formula is: ,

[0068] The feature of the target category in the j-th channel of the i-th layer.

[0069] d. Heatmap generation: Multiply the feature map of each channel by its corresponding weight, sum them, and activate them using the ReLU function to obtain the Grad-CAM++ heatmap. : The ReLU function is used to filter negative gradients, retaining only features that contribute positively to the classification results.

[0070] e. Heatmap Upsampling and Post-processing: The generated heatmap is upsampled to the size of the original input image (160×160) using bilinear interpolation. Then, the heatmap is thresholded, with 0.4 of the maximum value of the heatmap used as the threshold, converting the heatmap into a binary image.

[0071] f. ROI Extraction and Feature Point Detection: Morphological operations (dilation and erosion) are performed on the binary image to remove noise and small connected regions. Then, a contour detection algorithm is used to extract the largest connected region, which is the terrain ROI region, and its bounding box coordinates (x1, y1, x2, y2) are calculated. Finally, Canny edge detection and Hough transform are performed on the ROI region to extract key feature points such as stair edges, slope starting points, and step corners, providing a foundation for subsequent terrain parameter estimation.

[0072] (v) Standardized multi-protocol data output interface module This module transmits standardized environmental perception data to the exoskeleton control system, multi-sensor fusion module, and laser ranging module via various communication protocols. To improve system compatibility and scalability, this module supports multiple industry-standard communication protocols, including CANFD, UART, and SPI. Among these, the CANFD bus, due to its high reliability, high transmission rate, and strong anti-interference capabilities, serves as the primary communication interface for data exchange with the exoskeleton control system and multi-sensor fusion module; UART and SPI serve as auxiliary interfaces for communication with the laser ranging module and other peripherals.

[0073] Taking the CANFD bus as an example, the data frame adopts the extended frame format, with an ID of 0x123 and a data length of 32 bytes. The specific field definitions are as follows:

[0074] The data transmission frequency is 10Hz, consistent with the model inference frequency. Before each data transmission, a CRC checksum is calculated and appended to the end of the data frame. Upon receiving the data, the receiver first verifies the CRC checksum; only data that passes the verification is processed, ensuring the reliability of data transmission.

[0075] II. System Operation The system is configured with five operating states, and a state machine is used to achieve fully automated control of the entire process, ensuring stable and reliable system operation. 1. Initialization State: After power-on, the system first performs hardware initialization, including camera initialization, AI coprocessor initialization, communication interface initialization, and memory initialization. Then, it loads the quantized CNN model and configuration parameters, performs sensor calibration, and warms up the model. After initialization is complete, the system enters standby mode, waiting for the user's startup command.

[0076] 2. Standby mode: The system is in low-power mode, with the camera and AI coprocessor in sleep mode, only the communication interface and main control unit are running. When the system receives a user's activation command (such as powering on the exoskeleton or activating the walking mode), the system wakes up the camera and AI coprocessor and enters normal operation.

[0077] 3. Normal Operation: The system acquires and processes images at a frequency of 10Hz, outputting terrain probability distribution, confidence level, and ROI information. Different operations are performed based on the confidence level: High confidence (C≥0.75): The environmental perception data is directly sent to the exoskeleton main control system, which adjusts the control strategy according to the terrain type and parameters.

[0078] Medium confidence level (0.5≤C<0.75): Send environmental perception data to the exoskeleton main control system, and at the same time send a verification request and ROI information to the laser ranging module, triggering the laser ranging module to scan the ROI area, obtain the geometric parameters of the terrain, and verify and correct the visual recognition results.

[0079] Low confidence level (C<0.5): No visual recognition results are sent; only an "uncertain" flag is sent to the multi-sensor fusion module. The system switches to pure laser ranging or body sensor control mode to ensure user safety.

[0080] 4. Fault Status: When the system detects a hardware fault (such as camera acquisition failure, AI coprocessor malfunction), software fault (such as model inference timeout, data transmission error), or sensor malfunction (such as image too dark or overexposed), it immediately triggers a fault alarm, alerts the user through sound and light signals, and automatically switches to a safety mode (such as stopping assist and locking joints) to prevent safety accidents.

[0081] 5. Power off state: When the system receives a power off command from the user, it saves the operation log and configuration parameters, turns off the camera and AI coprocessor, and enters the power off state.

[0082] The following section will detail the implementation process of the "Intelligent Recognition Method for Terrain Probability Distribution Based on Lightweight Convolutional Neural Network" in this application, based on the aforementioned system architecture and methodology.

[0083] III. Method Description like Figure 6 As shown, the method steps are as follows: Step 1: Dataset Construction and Model Training This step aims to significantly reduce the number of model parameters and computational cost while maintaining model accuracy through knowledge distillation, model pruning, and quantization techniques. The specific implementation process is as follows: 1. Dataset Construction: To train and evaluate the terrain recognition model proposed in this invention, a large-scale and diverse exoskeleton terrain image dataset was constructed. The dataset was collected from various real-world scenarios, including indoor hospital corridors, staircases, and offices, as well as outdoor sidewalks, parks, and hillsides. The acquisition device was the same OV5640 monocular camera used in this system, mounted on the front of the exoskeleton to simulate the field of view during actual use. The dataset contains 20,000 images, covering five terrain types: flat ground (4,000 images), ascending stairs (4,000 images), descending stairs (4,000 images), uphill (4,000 images), and downhill (4,000 images). The images cover terrain scenes with different lighting conditions (sunny, cloudy, night, and shadow), different angles (0°-30°), different textures (cement, wooden floor, tile, grass, and dirt road), and different distances (0.5-5 meters). The dataset was labeled, with each image labeled with its corresponding terrain category. The dataset was randomly divided into a training set (16,000 images), a validation set (2,000 images), and a test set (2,000 images) in a ratio of 8:1:1.

[0084] 2. Teacher Network Training: ResNet50 was used as the teacher network and trained on the constructed terrain dataset. The training parameters were set as follows: AdamW optimizer, weight decay of 0.01, initial learning rate of 0.001, cosine annealing learning rate decay strategy, batch size of 32, training epochs of 50, and label smoothing coefficient of 0.1. After training, the weights of the teacher network were saved for subsequent knowledge distillation.

[0085] 3. Student Network Construction and Knowledge Distillation Training: The Mobile Shuffle Net-Terrain student network proposed in this invention was constructed and trained using knowledge distillation technology. The training parameters were set as follows: the optimizer was AdamW, weight decay was 0.01, initial learning rate was 0.002, cosine annealing learning rate decay strategy was adopted, batch size was 64, training epochs were 60, temperature coefficient T=4, balancing coefficient α=0.3, and label smoothing coefficient was 0.1. During training, both real labels and soft labels output by the teacher network were used for supervision.

[0086] 4. Model Pruning and Fine-tuning: The trained student network underwent structured pruning. First, during training, L1 regularization was applied to the γ parameter of the BatchNorm layer with a regularization coefficient of 0.0001 for 10 epochs, bringing the γ parameter of unimportant channels close to 0. Then, channels with γ parameters less than 0.001 were pruned, removing a total of 30% of redundant channels. Finally, the pruned model was fine-tuned with the following training parameters: AdamW optimizer, initial learning rate of 0.0005, batch size of 64, and 15 training epochs to restore model accuracy.

[0087] 5. Model Quantization: The pruned model is quantized using ONNX Runtime with INT8 quantization. First, a calibration dataset of 1000 images is prepared, randomly sampled from the training set. Then, the model is quantized and calibrated using the calibration dataset to determine the quantization parameters (scaling factor and zero point). Finally, the INT8 quantized model file (.onnx format) is generated.

[0088] 6. Model Deployment: Deploy the quantized model onto the K210 AI coprocessor. The K210 is a low-cost, low-power edge AI chip with a built-in hardware KPU accelerator, capable of efficiently running convolutional neural networks. Write C language interface functions to automate image input, model inference, and result output.

[0089] Step 2: Image Acquisition and Adaptive Preprocessing The adaptive preprocessing step aims to effectively address issues such as uneven illumination and image blurring, improving the quality of the input image and providing a reliable data foundation for subsequent terrain classification. In this case, the preprocessing time for a single frame is approximately 5ms, far less than the model inference time, and will not affect the system's real-time performance. The processing procedure is as follows: 1. Image Acquisition: After the system enters normal operation, the OV5640 camera continuously acquires BGR format images at a frame rate of 30fps and stores them in the frame buffer.

[0090] 2. Frame extraction: Extract the latest frame image (640×480×3) from the frame buffer at a frequency of 10Hz to ensure that the latest terrain information is processed each time.

[0091] 3. Color space conversion: Convert the extracted BGR format image to RGB format using the cvtColor function in the OpenCV library.

[0092] 4. Adaptive Illumination Compensation: Adaptive illumination compensation based on Retinex theory is performed on the RGB image. First, Gaussian blur is used to estimate the illumination components. Then, the original image is divided by the illumination components to obtain the reflection components. Finally, the contrast of the reflection components is adjusted to obtain the illumination-compensated image.

[0093] 5. Image scaling and cropping: Use bilinear interpolation to scale the image from 640×480 to 256×256, and then crop a 160×160 image area from the center.

[0094] 6. Data Standardization: Convert the image pixel values ​​from [0,255] to the range [0,1], and then use the mean and standard deviation of the ImageNet dataset for standardization to obtain a standardized image tensor.

[0095] 7. Data format conversion: Convert the normalized image tensors into a format that the K210KPU can process, in preparation for input into the CNN model for inference.

[0096] Step 3: Terrain Feature Extraction and Classification 1. Model Loading: During system initialization, the INT8 quantized Mobile Shuffle Net-Terrain model is loaded into the memory of the K210KPU.

[0097] 2. Input data preparation: Input the preprocessed, normalized image tensor into the input buffer of the KPU.

[0098] 3. Model Inference: The KPU is activated to perform forward propagation inference. The KPU hardware accelerates the execution of convolution, pooling, activation, and other operations, which pass through the initial convolutional layer, 9 shuffle bottleneck blocks, global average pooling layer, and fully connected layer in sequence.

[0099] 4. Output Results: After inference is complete, the raw prediction scores (logits) of the five terrain types and the feature map of the last convolutional layer are read from the output buffer of the KPU.

[0100] 5. Data transmission: The original prediction score is sent to the multi-dimensional probability distribution and confidence evaluation module, and the feature map and terrain classification results (the category corresponding to the maximum value of the original score) are sent to the Grad-CAM++ROI localization and feature extraction module.

[0101] The K210KPU hardware acceleration enables fast model inference, with an inference time of approximately 45ms per frame. Including preprocessing and postprocessing time, the total system processing time is approximately 55ms, enabling it to run at 18fps, meeting the real-time requirements of exoskeletons.

[0102] Step 4: Probability Distribution Calculation and Confidence Assessment 1. Raw score reception: Receive the raw predicted scores of the 5 terrain categories from the output of the improved lightweight CNN module.

[0103] 2. Probability distribution calculation: The original prediction scores are converted into a probability distribution using the improved Softmax function, and the probability values ​​for various terrain types are calculated.

[0104] 3. Information entropy calculation: Calculate the information entropy H based on the probability distribution to measure the uncertainty of the classification result.

[0105] 4. Calculation of maximum probability value and dispersion: Find the maximum probability value p_max in the probability distribution and calculate the dispersion D of the probability distribution.

[0106] 5. Calculation of overall confidence score: Based on information entropy, maximum probability value, and dispersion, the overall confidence score C is calculated using a multi-dimensional confidence score evaluation formula.

[0107] 6. Confidence level classification: Based on the overall confidence level, the classification results are divided into three levels: high confidence, medium confidence, or low confidence.

[0108] 7. Data transmission: The terrain probability distribution, classification confidence level, and confidence level are sent to the standardized multi-protocol data output interface module.

[0109] The multi-dimensional confidence assessment method can accurately reflect the reliability of classification results, providing an important basis for subsequent multi-sensor fusion and control decisions. The correlation between the confidence assessment and the actual classification accuracy reaches 0.97, which is significantly better than the single confidence assessment method based on the maximum probability value.

[0110] Step 5: Grad-CAM++ ROI Localization and Key Feature Extraction The Grad-CAM++ algorithm can accurately locate terrain regions in images, with accurate ROI localization and key feature point extraction, providing precise location information for subsequent terrain parameter estimation and multi-sensor fusion. The specific implementation process is as follows: 1. Feature Map and Classification Result Reception: Receives the feature map of the last convolutional layer and the terrain classification result from the improved lightweight CNN module.

[0111] 2. Gradient calculation: Calculate the gradient of the target class score with respect to the feature map.

[0112] 3. Channel weight calculation: Calculate the weight of each channel based on the gradient.

[0113] 4. Heatmap generation: Multiply the feature map of each channel by the corresponding weight, sum them, and then activate them through the ReLU function to generate the Grad-CAM++ heatmap.

[0114] 5. Heatmap Upsampling: Upsample the heatmap to the original input image size of 160×160 using bilinear interpolation.

[0115] 6. Thresholding and Morphological Processing: Thresholding is performed on the heatmap to convert it into a binary image. Then, dilation and erosion operations are performed to remove noise and small connected regions.

[0116] 7. ROI Extraction: Use a contour detection algorithm to extract the largest connected region and calculate its bounding box coordinates (x1, y1, x2, y2).

[0117] 8. Key Feature Point Detection: Perform Canny edge detection on the ROI region, and then use Hough transform to extract lines and corner points to obtain key feature points such as stair edges, ramp start points, and step corner points.

[0118] 9. Data transmission: Send the ROI bounding box coordinates and key feature point information to the standardized multi-protocol data output interface module.

[0119] Step 6: Data Output and Multi-Sensor Interaction For data interaction, a standardized multi-protocol data output interface is used to achieve seamless integration with other systems of the exoskeleton, ensuring reliable and real-time data transmission.

[0120] 1. Data reception: Receives terrain probability distribution, confidence level and confidence level output by the multi-dimensional probability distribution and confidence assessment module, as well as ROI bounding box and key feature point information output by the Grad-CAM++ ROI localization and feature extraction module.

[0121] 2. Data Packaging: Pack all data into standard data frames according to the CANFD protocol format, and add timestamps and CRC checksums.

[0122] 3. Data transmission: Data frames are sent to the exoskeleton main control system and multi-sensor fusion module via the CANFD bus.

[0123] 4. Verification Request Sending: If the confidence level is medium confidence, a verification request and ROI information are sent to the laser ranging module via the UART interface, instructing the laser ranging module to scan the ROI area.

[0124] 5. Fusion Result Reception: Receives the fusion result returned by the multi-sensor fusion module, which combines visual recognition information and laser ranging information.

[0125] 6. Control command forwarding: The fusion result is forwarded to the exoskeleton adaptive control module for adjusting the control strategy and assist torque.

[0126] After multi-sensor fusion, the overall accuracy of terrain recognition is improved, and the system robustness is significantly enhanced.

[0127] Example 1 (a) Implementation environment and hardware platform This embodiment is performed on a lower limb exoskeleton robot used for rehabilitation and mobility assistance. This exoskeleton is primarily used to assist patients with lower limb motor dysfunction (such as stroke patients and spinal cord injury patients) in walking rehabilitation training. Figure 7 As shown, the hardware platform configuration of the exoskeleton is as follows: Main control unit: STM32H750 microcontroller, based on ARM Cortex-M7 core, 400MHz clock speed, with 2MB Flash and 1MB RAM, running FreeRTOS real-time operating system.

[0128] AI coprocessor: Kendryte K210, with a built-in dual-core RISC-V processor and hardware KPU accelerator, a main frequency of 400MHz, and supports INT8 quantization model inference.

[0129] Image acquisition unit: OV5640 monocular CMOS camera, resolution 640×480, frame rate 30fps, supports automatic exposure and automatic white balance.

[0130] Laser ranging unit: 3 VL53L1X one-dimensional laser ranging sensors, respectively installed on the torso, thigh and calf, with a measurement range of 0-4 meters and an accuracy of ±1cm.

[0131] Body sensors: MPU9250 nine-axis IMU (accelerometer, gyroscope, magnetometer), installed on the thigh, calf and torso; 12-bit absolute encoder, installed on the hip and knee joints, used to measure joint angles.

[0132] Actuation unit: DC servo motor, mounted on the hip and knee joints, providing assist torque.

[0133] (II) Model Training and Deployment 1. Dataset Construction: Following the method in step 1, a terrain dataset containing 20,000 images was constructed. The dataset covers the indoor environment (corridors, stairs, ramps) and outdoor environment (sidewalks, park paths, hills) of a hospital rehabilitation center.

[0134] 2. Teacher Network Training: The ResNet50 teacher network was trained using the PyTorch framework. Training was performed on an NVIDIA RTX 3090 GPU and took approximately 8 hours. After training, the teacher network achieved a classification accuracy of 96.8% on the test set.

[0135] 3. Student Network Training and Knowledge Distillation: A Mobile Shuffle Net-Terrain student network was constructed and trained using knowledge distillation. Training was also performed on an NVIDIA RTX 3090 GPU, taking approximately 6 hours. After training, the student network achieved a classification accuracy of 94.5% on the test set, a 2.3% improvement over the student network without knowledge distillation.

[0136] 4. Model Pruning and Fine-tuning: The student network was structurally pruned, removing 30% of redundant channels. After pruning, the initial accuracy of the model was 91.2%. After 15 rounds of fine-tuning training, the accuracy recovered to 94.1%, a decrease of only 0.4%, which is almost negligible.

[0137] 5. Model Quantization: The pruned model was quantized using ONNX Runtime with INT8 quantization. The accuracy of the quantized model was 93.8%, a decrease of only 0.3%, while the model size was reduced from 4.2MB to 1.1MB, and the inference speed was improved by 3.2 times.

[0138] 6. Model Deployment: Convert the quantized model to the Kmodel format supported by K210 and download it to the K210's Flash memory. Write C language interface functions to automate image acquisition, preprocessing, model inference, and result output.

[0139] (III) System Operation To comprehensively evaluate the performance of the intelligent terrain probability distribution recognition method proposed in this invention, detailed tests were conducted. The tests consisted of two parts: laboratory environment testing and real-world scenario testing.

[0140] 1. Environmental Testing In a laboratory environment, a test platform simulating different terrains was built, including flat ground, climbing stairs (step height 15cm), descending stairs (step height 15cm), going uphill (slope 15°), and going downhill (slope 15°). Five healthy volunteers were invited to wear exoskeletons and walk on the different terrains to test the system's performance. Test metrics included classification accuracy, inference speed, ROI localization accuracy, and confidence assessment accuracy.

[0141] Testing process: The volunteer puts on the exoskeleton, the system starts up and completes initialization.

[0142] The volunteers walked 10 meters on flat ground, then went up and down stairs, up and down slopes, repeating each type of terrain 5 times.

[0143] The system acquires and processes images in real time, recording the classification results, probability distribution, confidence level, ROI bounding box, and key feature points for each classification.

[0144] The actual terrain category and ROI area are manually labeled each time, and compared with the system output results to calculate various performance indicators.

[0145] Test results: Classification accuracy: The system achieved an average classification accuracy of 95.2% in a laboratory environment. Specifically, the accuracy was 97.5% on flat ground, 94.8% for going up stairs, 93.2% for going down stairs, 95.6% for going uphill, and 94.7% for going downhill.

[0146] Inference speed: The system's average inference speed is 18.2fps, with a maximum of 21.5fps and a minimum of 15.8fps, which meets the real-time requirements.

[0147] ROI positioning accuracy: The ROI positioning accuracy is 94.2%, and the average intersection-union ratio (IoU) is 0.87.

[0148] Accuracy of confidence assessment: When confidence level ≥ 0.75, the classification accuracy is 99.1%; when 0.5 ≤ C < 0.75, the classification accuracy is 87.3%; when C < 0.5, the classification accuracy is 52.4%. The correlation between confidence assessment and actual classification accuracy is 0.97.

[0149] 2. Scenario Testing The system was tested in a real-world environment at a hospital rehabilitation center, including indoor corridors, stairs, ramps, and outdoor walkways. Three stroke patients and two healthy volunteers were invited to wear the exoskeleton for a walking test to assess the system's robustness and adaptability in complex real-world environments.

[0150] Testing process: Patients and volunteers wear exoskeletons and undergo walking training under the guidance of rehabilitation therapists.

[0151] The system runs in real time, recording terrain identification results and control strategy adjustments.

[0152] Rehabilitation therapists and users rate the system's effectiveness and user experience.

[0153] Test results: Classification accuracy: The system achieved an average classification accuracy of 92.7% in real-world scenarios. Specifically, the accuracy rate was 94.3% for indoor environments and 90.5% for outdoor environments. Changes in lighting and missing textures were the main causes of classification errors.

[0154] Multi-sensor fusion effect: When the confidence result triggers laser ranging verification, the classification accuracy is improved to 98.7%, effectively solving the ambiguity problem of visual recognition.

[0155] User experience: All test users reported that the exoskeleton transitioned very smoothly between different terrains, with no noticeable lag or insufficient assistance. Rehabilitation therapists stated that the system accurately identified terrain changes and adjusted assistance strategies in a timely manner, significantly improving the safety and effectiveness of rehabilitation training.

[0156] (iv) Comparative Analysis To highlight the technical advantages of this invention, a comparative test was conducted between the method of this invention and three typical methods in the prior art. The comparison results are shown in the table below: Comparative Example 1: Original MobileNetV3-small model (FP32); Comparative Example 2: Original ShuffleNetV2 model (FP32).

[0157] Classification accuracy 88.3% 87.6% 93.8% Inference speed (K210 platform) 5.2fps 5.8fps 18.2fps Model parameter count 1.5M 1.2M 0.84M Model size 6.1MB 4.8MB 1.1MB Memory usage 7.5MB 6.2MB 1.5MB RO positioning accuracy 85.7% 84.3% 94.2% Confidence assessment of relevance 0.89 0.87 0.97 Accuracy decrease under changes in lighting 14.2% 15.6% 6.3% All methods were tested on the same hardware platform (K210AI coprocessor) and test dataset. The comparison results are shown in the table below: The comparison results show that the method of the present invention is significantly superior to the prior art in all performance indicators: 1. Highest classification accuracy: The classification accuracy of the method in this invention reaches 93.8%, which is 23.3% higher than the traditional HOG+SVM method, and 5.5% and 6.2% higher than the original MobileNetV3-small and ShuffleNetV2, respectively. This is mainly due to the improved network structure, knowledge distillation technology and adaptive preprocessing method.

[0158] 2. Fastest inference speed: The inference speed of this invention reaches 18.2fps, which is 3.5 times that of the original MobileNetV3-small and 7.3 times that of the traditional HOG+SVM method. This is mainly due to model pruning and INT8 quantization techniques, as well as hardware acceleration from the K210KPU.

[0159] 3. Minimal and Lightest Model: The model size of the method of this invention is only 1.1MB, and the memory usage is only 1.5MB, making it very suitable for deployment on resource-constrained embedded platforms.

[0160] 4. Most accurate ROI positioning: The method of this invention adopts the Grad-CAM++ algorithm, and the ROI positioning accuracy reaches 94.2%, which is 8.5% higher than the original MobileNetV3-small.

[0161] 5. Strongest robustness: The accuracy of the method of this invention decreases by only 6.3% under changes in illumination, which is much lower than other methods, indicating that it has stronger environmental adaptability and robustness.

[0162] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for intelligent recognition of terrain probability distribution based on a lightweight convolutional neural network, characterized in that, Includes the following steps: S1: Acquire raw RGB images of the environment in front of you using a monocular camera mounted on the exoskeleton torso; S2: Perform adaptive illumination compensation, scaling and cropping, standardization and online data augmentation preprocessing on the original image to obtain a standardized image; S3: Input the standardized image into the improved Mobile Shuffle Net-Terrain lightweight convolutional neural network model, perform forward propagation, and output the raw terrain prediction score and the feature map of the last convolutional layer; S4: Calculate the probability distribution and multi-dimensional comprehensive confidence of various terrain types based on the original prediction scores; S5: Generate Grad-CAM++ heatmaps based on feature maps and classification results, and extract regions of interest (ROI) and key feature points of the terrain; S6: Package and output the terrain probability distribution, confidence level, ROI information and key feature points according to the standard protocol.

2. The method according to claim 1, characterized in that, The adaptive illumination compensation described in step S2 employs an algorithm based on Retinex theory, decomposing the image into illumination and reflection components. Image contrast is enhanced by adjusting the illumination component, as shown in the formula: in, For the original image, For light component, This is the reflection component.

3. The method according to claim 1, characterized in that, The improved Mobile ShuffleNet-Terrain model described in step S3 is based on the MobileNetV3 architecture. It replaces the inverted residual blocks with ShuffleNetV2 basic units and adds an efficient channel attention (ECA) module after each unit. The ECA module achieves local cross-channel interaction through adaptive convolutional kernel size, and the channel attention weights... : in: The feature vector after global average pooling. The kernel size is One-dimensional convolution, For the sigmoid function, kernel size Based on the number of channels Adaptive determination.

4. The method according to claim 1, characterized in that, The model described in step S3 is trained using knowledge distillation, with ResNet50 as the teacher network and Mobile Shuffle Net-Terrain as the student network; the training process employs a composite loss function of label smoothing cross-entropy loss and KL divergence loss. : in: For cross-entropy loss, For the predicted probability distribution of the student network, This is a real label; For KL divergence loss, The predicted probability distribution for the teacher network; The temperature coefficient is used to soften the probability distribution, enabling the student network to learn more information about inter-category similarity. This is the balancing coefficient, used to adjust the weights of the two parts of the loss.

5. The method according to claim 1, characterized in that, The model described in step S3 undergoes structured pruning and INT8 quantization. The channel pruning method based on L1 regularization is used to prune channels with a scaling factor of less than 0.001 in the BatchNorm layer, with a pruning ratio of 30%. INT8 quantization is performed using ONNX Runtime to convert the 32-bit floating-point weights and activation values ​​into 8-bit integers.

6. The method according to claim 1, characterized in that, The probability distribution calculation in step S4 uses a modified Softmax function, which prevents numerical overflow by subtracting the maximum original score. The formula is as follows: in: For the first The original prediction score of the class. The maximum value among all original predicted scores.

7. The method according to claim 1, characterized in that, The multi-dimensional comprehensive confidence score mentioned in step S4 is calculated based on information entropy, maximum probability value, and probability distribution dispersion, and the formula is: in: For information entropy, The maximum probability value, For dispersion, .

8. The method according to claim 1, characterized in that, The ROI extraction in step S5 uses the Grad-CAM++ algorithm and includes the following steps: Calculate the gradient of the target category score relative to the last layer of feature map; The weight of each channel is calculated based on the higher-order gradient information. ; Generate a Grad-CAM++ heatmap using the following formula: in: For channel weights, For the first Feature maps of each channel; The heatmap is upsampled to the original image size, and after thresholding and morphological processing, the largest connected region is extracted as the terrain ROI by contour detection.

9. The method according to claim 1, characterized in that, The output in step S6 adopts the CANFD bus communication protocol, and the data frame includes terrain probability distribution, classification confidence, ROI bounding box coordinates, key feature point coordinates, timestamp and CRC check code.

10. A terrain probability distribution intelligent recognition system based on a lightweight convolutional neural network, characterized in that, include: The image acquisition and adaptive preprocessing module is used to acquire images of the foreground environment and perform adaptive illumination compensation, scaling and cropping, and standardization preprocessing to output a standardized image. An improved lightweight CNN terrain classification module is adopted, using the Mobile Shuffle Net-Terrain model to extract features and classify terrain from standardized images, outputting the original prediction score and the feature map of the last convolutional layer; The multi-dimensional probability distribution and confidence assessment module is used to calculate the probability distribution and multi-dimensional comprehensive confidence of various terrain types. The Grad-CAM++ ROI localization and feature extraction module is used to generate Grad-CAM++ heatmaps based on feature maps and classification results, and to extract terrain ROIs and key feature points. The standardized multi-protocol data output interface module is used to package and output terrain probability distribution, confidence level, ROI information and key feature points to the exoskeleton main control system according to the CANFD protocol.

Images