Degradation environment detection method and system based on deep learning

Abstract

The invention provides a degraded environment detection method and system based on deep learning, and the method comprises the steps: obtaining sensor data collected by a robot in real time, taking the sensor data of each frame as a particle, and calculating a position estimation value of the robot through employing an SLAM algorithm; calling position estimation values of the robot in historical time, and inputting the plurality of position estimation values into a pre-trained degradation detection model; the degradation detection model comprises a data processing module, a forward propagation module and a calculation module; the data processing module obtains an input tensor based on a plurality of input position estimated values and inputs the input tensor into the forward propagation module, and the forward propagation module calculates a first confidence coefficient and a second confidence coefficient based on a plurality of internally arranged basic blocks; the reckoning module determines whether a degraded environment occurs based on the first confidence and the second confidence. A degradation detection model is adopted for processing, and whether the robot enters a degradation environment or not is judged.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time positioning and map construction, and in particular to a method and system for detecting a degraded environment based on deep learning. Background Art

[0002] In the map construction process, degraded scenarios refer to situations where the algorithm cannot accurately estimate the robot's pose or generate a reliable map due to insufficient environmental features or limitations of sensor data. For example, in a long straight corridor, the lidar can only obtain parallel line features in a single direction and lacks lateral geometric constraints, which leads to a sharp increase in the uncertainty of pose estimation in the direction of motion; while visual SLAM is prone to tracking loss due to insufficient feature point extraction in low-texture white walls or solid-color ground environments. In addition, moving pedestrians or vehicles in dynamic scenes will introduce noise features that interfere with the modeling of static maps, while mirrors or glass curtain walls will cause measurement errors in sensors (such as lidar and cameras) and generate false environmental information. The core impact of degraded scenarios is that the observation data that the algorithm relies on cannot provide sufficient constraints, resulting in rank deficiency in the state estimation equation, which ultimately leads to positioning drift, map distortion, and even system crash.

[0003] SLAM (Simultaneous Localization and Mapping) is an advanced technology that allows robots or self-driving cars to simultaneously localize themselves and build maps of the environment in unknown environments. By integrating data from multiple sensors, such as lidar, cameras, and IMUs, SLAM is able to generate a map of the environment in real time and accurately track the robot's position and movement trajectory in that environment. The importance of this technology lies in the fact that it provides robots with the ability to navigate autonomously without prior map information. SLAM has a wide range of applications, including but not limited to self-driving cars, indoor service robots, drones, disaster relief, and industrial automation. However, current SLAM technology still has difficulty solving the problem of degraded scenarios. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a degraded environment detection method based on deep learning to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a method for detecting a degraded environment based on deep learning, the method comprising the following steps: Acquire the sensor data collected by the robot in real time, treat each frame of sensor data as a particle, and use the SLAM algorithm to calculate the robot's position estimate; Calling the robot's position estimation values ​​at historical times, and inputting multiple position estimation values ​​into a pre-trained degradation detection model; The degradation detection model includes a data processing module, a forward propagation module and a calculation module; The data processing module obtains an input tensor based on multiple input position estimation values, and inputs the input tensor into the forward propagation module. The forward propagation module calculates a first confidence level and a second confidence level based on multiple basic blocks set internally. The inference module determines whether a degraded environment occurs based on the first confidence level and the second confidence level.

[0006] Using the above scheme, the present invention first uses the SLAM algorithm to calculate the positioning data, and then uses the degradation detection model for processing. The processing of the degradation detection model includes: The process of forward propagation: The data first passes through a convolution layer with a convolution kernel size of 7*7 and a step size of 2. This reduces the spatial size of the feature map and helps the model extract abstract features. Then it passes through a batch normalization layer to solve the problem of internal covariate offset in the network through regularization. The steps are to first calculate the mean and variance of the input features. The formula is as follows: Among them, μ is the mean, represents the variance, represents the i-th feature, and N represents the number of features; Then standardize the features. The formula is as follows, adding a small amount of To prevent the denominator from being 0, we introduce learnable parameters and , scaling and translation are performed according to the following formula.

[0007] in, represents the normalized features, and x represents the features before normalization. is a constant, Represents the features after scale transformation.

[0008] This network layer can speed up the convergence of the model and alleviate the problem of gradient explosion or disappearance. The next layer is the activation function layer. We use the ReLU function to enhance the nonlinear expression ability of the model. Then, a maximum pooling layer for downsampling is used to further reduce the amount of calculation.

[0009] Next, this solution is based on the ResNet architecture and designs the backbone of the network, which consists of 4 layers. Each layer gradually extracts the features of the image through different numbers of BasicBlocks. From the shallow layer to the deep layer, the abstract level of the particle distribution features gradually increases. BasicBlock is a residual block structure that contains a 3*3 convolution layer, a batch normalization layer, and a ReLU function. The convolution layer can reduce the spatial size of the feature map, making the feature map more compact and highlighting important features. BasicBlock also uses identity mapping to allow features to skip certain network layers for propagation and maintain gradient flow. The use of batch normalization and ReLU activation functions makes network training more stable, accelerates convergence, and improves generalization ability. Finally, a feature map rich in high-level semantic information is generated, and downsampling is used to ensure the consistency of feature dimensions during residual connection.

[0010] This solution uses the transformer as the neck part of the network. First, the feature map of the backbone output size of 256*28*28 is flattened, and then a linear layer is used to adjust the features to match the dimension of the transformer input. The adjusted features are input to the transformer encoder, and the self-attention mechanism and feedforward network are used to process the sequence to further enhance the features, so that the model can capture long-distance dependencies and improve the expressiveness of the features.

[0011] Finally, in the head part of the network, we use a fully connected layer as the classification head, which can convert the high-dimensional features extracted from the complex network structure into specific category predictions and output a vector to describe the degree of degradation.

[0012] Combined with the above model, this scheme can deeply analyze the particle state in the particle filter, so as to accurately determine whether the robot enters a degraded environment. This not only improves the reliability of the system, but also provides a strong guarantee for the robot's autonomous navigation in complex environments.

[0013] In some embodiments of the present invention, in the step of determining whether a degraded environment occurs based on the first confidence level and the second confidence level in the inference module, it is determined whether the first confidence level is greater than the second confidence level. If the first confidence level is greater than the second confidence level, it is determined that no degraded environment occurs; if the first confidence level is not greater than the second confidence level, it is determined that a degraded environment occurs.

[0014] In some embodiments of the present invention, in the step in which the data processing module obtains an input tensor based on multiple input position estimation values, the data processing module maps the multiple input position estimation values ​​through an internally set linear mapping layer to obtain the input tensor.

[0015] In some embodiments of the present invention, in the step of inputting the input tensor into the forward propagation module, and the forward propagation module calculating the first confidence and the second confidence based on multiple basic blocks set internally, the forward propagation module includes a grayscale processing layer, a convolution layer, a batch normalization layer, an activation function layer, multiple consecutive basic blocks, multiple consecutive sub-processing modules composed of basic blocks and downsampling layers, a flattening processing layer, a mapping layer, a transformer encoder and a fully connected layer that are sequentially arranged.

[0016] In some embodiments of the present invention, the basic block includes a convolution layer, a batch normalization layer, an activation function layer, a convolution layer, a batch normalization layer and an activation function layer arranged in sequence.

[0017] In some embodiments of the present invention, the transformer encoder includes a multi-head attention layer, an addition layer, a normalization layer, a forward propagation layer, an addition layer and a normalization layer arranged in sequence.

[0018] In some embodiments of the present invention, in the step of calling the position estimation value of the robot at the historical time, the position estimation value at the historical time is subjected to data cleaning and normalization processing.

[0019] In some embodiments of the present invention, in the step of performing data cleaning and normalization processing on the location estimate values ​​at historical time: Abnormal and missing particles are removed through data cleaning to ensure that the number of particles in each sample is complete and the data format is correct; The cleaned data were normalized by Z-SCORE to eliminate the influence of dimensional differences and outlier particles.

[0020] In some embodiments of the present invention, the method further comprises pre-training the degradation detection model, and the pre-training step comprises: The robot's historical position estimates are constructed into training, test, and validation sets, where: Training set, used to train the neural network model; The test set is used to evaluate the performance of the model; The validation set is used to adjust the model parameters to prevent overfitting.

[0021] The second aspect of the present invention also provides a degradation environment detection system based on deep learning, the system includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, when the computer instructions are executed by the processor, the system implements the steps implemented by the method described above.

[0022] The third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps implemented by the aforementioned deep learning-based degraded environment detection method.

[0023] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention can be specifically pointed out and obtained in the specification and the accompanying drawings.

[0024] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0026] Figure 1 A schematic diagram of an implementation of a method for detecting a degraded environment based on deep learning according to the present invention; Figure 2 It is a schematic diagram of the training process of the degradation detection model of the degradation environment detection method based on deep learning of the present invention; Figure 3 Schematic diagram of the processing architecture of the SLAM algorithm of the present invention; Figure 4 Schematic diagram of the architecture of the degradation detection model in the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0028] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0029] In the degraded environment of the existing technology, SLAM often cannot achieve the desired effect. Degraded environment refers to scenes with relatively sparse texture features, such as long straight corridors with smooth walls on both sides and empty outdoor scenes. In such environments, SLAM cannot obtain enough feature information, resulting in inaccurate positioning. SLAM is divided into optimization-based SLAM and filter-based SLAM according to the back-end optimization method. Particle filter-based SLAM uses particles to replace the estimated position of the robot. By analyzing the distribution of particles, it can be determined whether the robot has entered a degraded environment, so as to adopt appropriate anti-degradation strategies.

[0030] Particle filtering is a nonlinear filtering technology that is suitable for dealing with nonlinear and non-Gaussian distribution problems. In SLAM, each particle represents a possible position of the robot, and the robot's state is tracked through prediction, update, and resampling steps. The principle of particle filtering is to simulate many possible trajectories and continuously update the weights of these trajectories based on sensor data and motion models, and finally estimate the robot's current position and map through particles with high weights.

[0031] like Figure 1 As shown, the present invention proposes a degraded environment detection method based on deep learning, and the steps of the method include: Step S100, acquiring sensor data collected by the robot in real time, taking each frame of sensor data as a particle, and using the SLAM algorithm to calculate the estimated position of the robot; In the specific implementation process, the SLAM algorithm updates and maintains particles by using RBPF in Gmapping. The coordinates of each particle represent an estimated value for the robot's position, and the neural network detects the degraded environment based on the coordinates.

[0032] In the specific implementation process, the service communication module in ROS is used to transmit real-time data between SLAM and the degradation detection model. SLAM is set as the client node, and the degradation detection model is set as the server node. The client will publish the coordinates of all particles at each timestamp. After receiving the coordinates, the server will execute the callback function. Specifically, the particle image is input to the degradation detection model. After the model is calculated, the confidence level for the degradation environment is output and returned to the client node.

[0033] Step S200, calling the position estimation values ​​of the robot at historical time, and inputting the multiple position estimation values ​​into the pre-trained degradation detection model; In the specific implementation process, the data acquisition module records the coordinates of all particle position estimates in each frame. The particles exist in the particle filter, which is used to update and maintain the estimated value of the robot positioning in the SLAM algorithm. Its distribution can reflect the current positioning status.

[0034] Through data acquisition, all particle coordinates in each frame of the SLAM algorithm are recorded in detail. These particles are the core components of the particle filter, which is responsible for estimating and updating the robot's positioning during the SLAM process. By accurately capturing the position information of the particles, the data acquisition module can provide important raw data for the evaluation of positioning accuracy and subsequent data processing.

[0035] Specifically, the SLAM algorithm calculates the robot's position estimate based on the sensor data of each frame, and maintains the estimate through a set of particles as containers, each of which carries the robot's position and the estimated value of the map.

[0036] Step S300, the degradation detection model includes a data processing module, a forward propagation module and a calculation module; The data processing module obtains an input tensor based on multiple input position estimation values, and inputs the input tensor into the forward propagation module. The forward propagation module calculates a first confidence level and a second confidence level based on multiple basic blocks set internally. The inference module determines whether a degraded environment occurs based on the first confidence level and the second confidence level.

[0037] In the specific implementation process, the real-time exchange of data between SLAM and degradation detection model is realized through service communication in ROS, so as to achieve real-time degradation detection; Specifically, the ROS framework refers to the Robot Operating System framework, which is a flexible framework designed specifically for robot software development. ROS is a framework that integrates a large number of tools, libraries, and protocols, and provides functions similar to those provided by an operating system. These functions include hardware abstract description, underlying driver management, execution of common functions, message passing between programs, and program distribution package management. ROS can greatly simplify the creation of complex tasks and stable behavior control under complex and diverse robot platforms.

[0038] Real-time data transmission between SLAM and degradation detection model is carried out through the service communication module in ROS. SLAM is set as the client node, and the degradation detection model is set as the server node. The client will publish the coordinates of all particles at each timestamp. After receiving it, the server will execute the callback function. Specifically, the particle image is input to the degradation detection model. After the model is calculated, the confidence level for the degradation environment is output and returned to the client node.

[0039] In this technical solution, the node communication module uses the service communication mechanism within the ROS framework to achieve efficient data exchange between the SLAM algorithm and the degradation detection model. In this architecture, the SLAM algorithm acts as a client node, and the degradation detection model acts as a server node. The client node is responsible for publishing a data packet containing the coordinates of all particles at each time point. After receiving this data, the server node triggers the corresponding callback function.

[0040] Using the above scheme, the present invention first uses the SLAM algorithm to calculate the positioning data, and then uses the degradation detection model for processing. The processing of the degradation detection model includes: The process of forward propagation: The data first passes through a convolution layer with a convolution kernel size of 7*7 and a step size of 2. This reduces the spatial size of the feature map and helps the model extract abstract features. Then it passes through a batch normalization layer to solve the problem of internal covariate offset in the network through regularization. The steps are to first calculate the mean and variance of the input feature x. The formula is as follows: Then standardize the features. The formula is as follows, adding a small amount of To prevent the denominator from being 0, we introduce learnable parameters and , scaling and translation are performed according to the following formula.

[0041] This network layer can speed up the convergence of the model and alleviate the problem of gradient explosion or disappearance. The next layer is the activation function layer. We use the ReLU function to enhance the nonlinear expression ability of the model. Then, a maximum pooling layer for downsampling is used to further reduce the amount of calculation.

[0042] Next, this solution is based on the ResNet architecture and designs the backbone of the network, which consists of 4 layers. Each layer gradually extracts the features of the image through different numbers of BasicBlocks. From the shallow layer to the deep layer, the abstract level of the particle distribution features gradually increases. BasicBlock is a residual block structure that contains a 3*3 convolution layer, a batch normalization layer, and a ReLU function. The convolution layer can reduce the spatial size of the feature map, making the feature map more compact and highlighting important features. BasicBlock also uses identity mapping to allow features to skip certain network layers for propagation and maintain gradient flow. The use of batch normalization and ReLU activation functions makes network training more stable, accelerates convergence, and improves generalization ability. Finally, a feature map rich in high-level semantic information is generated, and downsampling is used to ensure the consistency of feature dimensions during residual connection.

[0043] This solution uses the transformer as the neck part of the network. First, the feature map of the backbone output size of 256*28*28 is flattened, and then a linear layer is used to adjust the features to match the dimension of the transformer input. The adjusted features are input to the transformer encoder, and the self-attention mechanism and feedforward network are used to process the sequence to further enhance the features, so that the model can capture long-distance dependencies and improve the expressiveness of the features.

[0044] Finally, in the head part of the network, we use a fully connected layer as the classification head, which can convert the high-dimensional features extracted from the complex network structure into specific category predictions and output a vector to describe the degree of degradation.

[0045] Combined with the above model, this scheme can deeply analyze the particle state in the particle filter, so as to accurately determine whether the robot enters a degraded environment. This not only improves the reliability of the system, but also provides a strong guarantee for the robot's autonomous navigation in complex environments.

[0046] In some embodiments of the present invention, in the step of determining whether a degraded environment occurs based on the first confidence level and the second confidence level in the inference module, it is determined whether the first confidence level is greater than the second confidence level. If the first confidence level is greater than the second confidence level, it is determined that no degraded environment occurs; if the first confidence level is not greater than the second confidence level, it is determined that a degraded environment occurs.

[0047] In some embodiments of the present invention, in the step in which the data processing module obtains an input tensor based on multiple input position estimation values, the data processing module maps the multiple input position estimation values ​​through an internally set linear mapping layer to obtain the input tensor.

[0048] like Figure 4 As shown, in some embodiments of the present invention, in the step of inputting the input tensor into the forward propagation module, the forward propagation module calculates the first confidence and the second confidence based on multiple basic blocks set internally, and the forward propagation module includes a grayscale processing layer, a convolution layer, a batch normalization layer, an activation function layer, a plurality of consecutive basic blocks, a plurality of consecutive sub-processing modules composed of basic blocks and downsampling layers, a flattening processing layer, a mapping layer, a transformer encoder and a fully connected layer that are sequentially arranged.

[0049] In some embodiments of the present invention, the basic block includes a convolution layer, a batch normalization layer, an activation function layer, a convolution layer, a batch normalization layer and an activation function layer arranged in sequence.

[0050] In some embodiments of the present invention, the transformer encoder includes a multi-head attention layer, an addition layer, a normalization layer, a forward propagation layer, an addition layer and a normalization layer arranged in sequence.

[0051] In some embodiments of the present invention, in the step of calling the position estimation value of the robot at the historical time, the position estimation value at the historical time is subjected to data cleaning and normalization processing.

[0052] In some embodiments of the present invention, in the step of performing data cleaning and normalization processing on the location estimate values ​​at historical time: Abnormal and missing particles are removed through data cleaning to ensure that the number of particles in each sample is complete and the data format is correct; The cleaned data were normalized by Z-SCORE to eliminate the influence of dimensional differences and outlier particles.

[0053] In this technical solution, the data processing module first performs a data cleaning step on the collected raw particle data to remove outliers and missing values ​​to ensure the accuracy and completeness of the data. Then, the cleaned data is standardized using the Z-SCORE standardization method to eliminate the impact of different dimensions and reduce the potential interference of outliers on model performance. This process not only improves data quality, but also lays a solid foundation for subsequent data analysis and model training.

[0054] In some embodiments of the present invention, data is processed by data cleaning and normalization methods, and scale-invariant two-dimensional particle images are generated for model training to eliminate the negative impact of outliers on model performance. A specially designed linear mapping method is used to obtain scale-invariant two-dimensional images to improve the quality of model training.

[0055] In the specific implementation process, deep learning is an artificial intelligence technology that processes complex data patterns by simulating the neural network structure of the human brain. It can learn the mapping relationship from input data to output data. It continuously adjusts the weights and biases in the network through forward propagation and back propagation mechanisms to optimize its prediction ability.

[0056] like Figure 3 As shown in Figure 1, applying deep learning to particle filter particle analysis in SLAM can significantly improve the detection capability of degraded environments. In SLAM, a particle filter is an algorithm used to estimate the robot position and environmental map, which relies on a large number of particles to simulate possible states. By training a degradation detection model to identify the distribution characteristics and behavior patterns of particles, it is possible to predict whether the robot enters a degraded environment, such as when the sensor fails or the environmental features are not obvious. The advantage of this method is that it can handle complex nonlinear relationships and can automatically learn features from data, reducing the need for manual feature engineering. In addition, the generalization ability of the degradation detection model enables it to maintain a high detection accuracy when facing new or unseen environments, which is of great significance for improving the robustness and reliability of the SLAM system.

[0057] like Figure 2 As shown, in some embodiments of the present invention, the method further comprises pre-training the degradation detection model, and the pre-training step comprises: The robot's historical position estimates are constructed into training, test, and validation sets, where: Training set, used to train the neural network model; The test set is used to evaluate the performance of the model; The validation set is used to adjust the model parameters to prevent overfitting.

[0058] Specifically, the designed neural network degradation detection model is trained by collecting data sets for degradation environment detection. The trained model can detect whether the robot enters a degradation environment according to the state of particles.

[0059] This technical solution collects particle data in degradation scenarios, processes the collected data and performs data cleaning and data normalization, and uses a degradation detection model to perform degradation detection. The degradation detection model can autonomously learn the distribution of particles in different environments, replacing the process of manual feature extraction, and has good generalization capabilities for different scenarios. Secondly, this solution uses ROS service communication to transmit data, realizes real-time degradation detection, and provides a good application opportunity for anti-degradation strategies.

[0060] The above scheme is adopted to train the degradation detection model using the training set, so that the degradation detection model has the ability to detect and identify the degraded environment; the trained model is evaluated using the test set to ensure the accuracy and robustness of the model.

[0061] In the specific implementation process, the pre-training step also includes data set update and model fine-tuning: In the process of updating the dataset, data is collected in a completely new environment, and the number of samples with different labels is balanced to avoid overfitting problems. The model is retrained to improve its degradation detection accuracy and generalization ability.

[0062] In the process of model fine-tuning, the model's network architecture, optimizer, and loss function are adjusted to enhance model performance.

[0063] This technical solution further improves the accuracy and generalization ability of the model by continuously collecting data in new environments and retraining the model. And by further adjusting the design details of the model, its degradation detection ability is improved.

[0064] In some embodiments of the present invention, in the step of constructing the robot's historical position estimation values ​​into training sets, test sets, and validation sets, an Ackerman wheeled robot is used to collect data in a degraded environment, equipped with a single-line laser radar, and a particle filter-based SLAM algorithm - Gmapping is deployed to record the particle state of each frame.

[0065] Specifically, Gmapping is a SLAM algorithm based on particle filtering. RBPF (Rao-Blackwellized Particle Filter) is an efficient particle filtering algorithm in gmapping, which is used for real-time map construction and robot positioning in unknown environments. It improves computing efficiency and accuracy by using a smaller number of particles.

[0066] In this technical solution, the SLAM module adopts a SLAM algorithm based on particle filtering technology, namely Gmapping. In this algorithm, RBPF, as the core component, uses its efficient particle processing capabilities to achieve instant map construction and precise positioning of the robot in an unknown environment. Compared with traditional methods, RBPF optimizes the use of computing resources by reducing the number of required particles while maintaining high accuracy in positioning and map construction. The application of this algorithm significantly improves the performance of the SLAM system under conditions of limited resources.

[0067] By adopting the above scheme, the designed degradation detection model is trained through the collected data set for degradation environment detection. The trained model can detect whether the robot enters a degradation environment according to the state of particles.

[0068] In summary, this technical solution involves collecting particle data in degradation scenarios, followed by data cleaning and normalization. A degradation detection model based on ResNet and Transformer architecture is used for degradation identification. The model can automatically grasp the characteristics of particle distribution in various environments, eliminating the steps of manual feature extraction, and has the ability to generalize across scenarios. In addition, the solution uses the service communication mechanism of the Robot Operating System (ROS) to achieve real-time data transmission, ensuring the real-time nature of degradation detection and providing an appropriate opportunity for the implementation of anti-degradation measures.

[0069] The beneficial effects of this program include: 1. Efficient data processing and model training: The present invention uses a carefully designed data processing module, data cleaning and Z-SCORE normalization method to effectively eliminate abnormal particles and missing data, thereby improving the quality and consistency of the data. This preprocessing not only reduces noise interference during model training, but also enhances the model's adaptability to different environments. In addition, a linear mapping method with scale-invariant properties is designed to obtain a two-dimensional image that can keep the distribution state of the particle swarm unchanged, thereby improving the training performance of the model. In addition, by dividing the data set into a training set, a test set, and a validation set, the present invention can systematically evaluate and optimize the model performance to ensure its accuracy and robustness in practical applications. This scientific data management and processing method significantly improves the generalization ability and prediction effect of the model.

[0070] 2. Real-time and cross-platform compatibility: By using the ROS service communication mechanism, the present invention realizes real-time data exchange between the SLAM algorithm and the degradation detection model, ensuring the real-time and response speed of the degradation environment detection. This communication method not only improves the response efficiency of the system, but also enables the system to be flexibly applied to different robot platforms and operating systems, enhancing the applicability and flexibility of the system. In addition, through GPU acceleration and image matrix acceleration methods, it is ensured that while maintaining high accuracy, the real-time performance of SLAM will not be affected, which is particularly important for application scenarios that require rapid response.

[0071] 3. Innovative neural network structure and high accuracy: The degradation detection model used in the present invention is specially designed to deeply analyze the particle state in the particle filter, so as to accurately determine whether the robot has entered a degraded environment. Through supervised learning, the model can learn and master the characteristics of the degraded environment from a large amount of labeled data, and finally achieved an accuracy of up to 90% on the test set. This high-accuracy detection capability not only improves the reliability of the system, but also provides a strong guarantee for the robot's autonomous navigation in complex environments. At the same time, the model's innovative structure and training methods provide new perspectives and solutions for the application of deep learning in the field of SLAM.

[0072] An embodiment of the present invention also provides a degradation environment detection system based on deep learning, the system includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, when the computer instructions are executed by the processor, the system implements the steps implemented by the method described above.

[0073] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps implemented by the aforementioned method for detecting a degraded environment based on deep learning are implemented. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0074] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0075] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0076] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting a degraded environment based on deep learning, characterized in that: The steps of the method include: Acquire the sensor data collected by the robot in real time, treat each frame of sensor data as a particle, and use the SLAM algorithm to calculate the robot's position estimate; Calling the robot's position estimation values ​​at historical times, and inputting multiple position estimation values ​​into a pre-trained degradation detection model; The degradation detection model includes a data processing module, a forward propagation module and a calculation module; The data processing module obtains an input tensor based on multiple input position estimation values, and inputs the input tensor into the forward propagation module. The forward propagation module calculates a first confidence level and a second confidence level based on multiple basic blocks set internally. The inference module determines whether a degraded environment occurs based on the first confidence level and the second confidence level.

2. The method for detecting a degraded environment based on deep learning according to claim 1, characterized in that: In the step of determining whether a degraded environment occurs based on the first confidence level and the second confidence level in the inference module, it is determined whether the first confidence level is greater than the second confidence level. If the first confidence level is greater than the second confidence level, it is determined that no degraded environment occurs; if the first confidence level is not greater than the second confidence level, it is determined that a degraded environment occurs.

3. The method for detecting a degraded environment based on deep learning according to claim 1, characterized in that: In the step in which the data processing module obtains an input tensor based on multiple input position estimation values, the data processing module maps the multiple input position estimation values ​​through an internally set linear mapping layer to obtain the input tensor.

4. The method for detecting a degraded environment based on deep learning according to claim 1, characterized in that: In the step of inputting the input tensor into the forward propagation module, the forward propagation module calculates the first confidence and the second confidence based on multiple basic blocks set internally, the forward propagation module includes a grayscale processing layer, a convolution layer, a batch normalization layer, an activation function layer, multiple continuous basic blocks, multiple continuous sub-processing modules composed of basic blocks and downsampling layers, a flattening processing layer, a mapping layer, a transformer encoder and a fully connected layer that are sequentially arranged.

5. The method for detecting a degraded environment based on deep learning according to claim 1 or 4, characterized in that: The basic block includes a convolution layer, a batch normalization layer, an activation function layer, a convolution layer, a batch normalization layer and an activation function layer arranged in sequence.

6. The method for detecting a degraded environment based on deep learning according to claim 4, characterized in that: The transformer encoder includes a multi-head attention layer, an addition layer, a normalization layer, a forward propagation layer, an addition layer and a normalization layer arranged in sequence.

7. The method for detecting a degraded environment based on deep learning according to claim 1, characterized in that: In the step of calling the position estimate of the robot at the historical time, the position estimate of the historical time is subjected to data cleaning and normalization.

8. The method for detecting a degraded environment based on deep learning according to claim 7, characterized in that: In the step of data cleaning and normalization of the historical time position estimates: Abnormal and missing particles are removed through data cleaning to ensure that the number of particles in each sample is complete and the data format is correct; The cleaned data were normalized by Z-SCORE to eliminate the influence of dimensional differences and outlier particles.

9. The method for detecting a degraded environment based on deep learning according to claim 1, characterized in that: The method further includes pre-training the degradation detection model, and the pre-training step includes: The robot's historical position estimates are constructed into training, test, and validation sets, where: Training set, used to train the neural network model; The test set is used to evaluate the performance of the model; The validation set is used to adjust the model parameters to prevent overfitting.

10. A degradation environment detection system based on deep learning, characterized in that: The system includes a computer device, which includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method as described in any one of claims 1 to 9.

Images