Intelligent navigation system based on image recognition

Abstract

The invention relates to the technical field of intelligent navigation, and discloses an intelligent navigation system based on image recognition, comprising: an image data acquisition module for acquiring an image of an environment, the image data acquisition module comprising a binocular camera for generating a depth map, and processing the acquired image data through a data preprocessing module; and the data preprocessing module is connected with the image data acquisition module, and the data preprocessing module comprises a denoising unit, a contrast enhancement unit and an HDR processing unit. A multi-modal data fusion technology is adopted, images, laser radar and inertial measurement unit data are integrated, high-precision fusion of the data is realized in combination with an extended Kalman filter, the accuracy and stability of environmental perception are enhanced, the limitation of a single sensor in a complex environment is overcome, and the accuracy and stability of environmental perception are improved. The system can accurately sense the surrounding environment under the conditions of severe weather, illumination variation and the like, and the overall reliability and adaptability of the system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent navigation, and in particular to an intelligent navigation system based on image recognition. Background Art

[0002] With the development of artificial intelligence and autonomous driving technologies, the demand for the application of intelligent navigation systems based on image recognition in the fields of unmanned driving, drone navigation, industrial robots, etc. has increased. Intelligent navigation systems provide autonomous mobility capabilities for automated mobile platforms through real-time perception of the environment, path planning, and dynamic obstacle avoidance. However, traditional navigation systems face many technical bottlenecks in complex dynamic environments, including reduced perception accuracy, limited path planning flexibility, lack of system robustness, and reduced dynamic decision-making capabilities.

[0003] Traditional intelligent navigation systems rely on a single sensor for environmental perception, but the stability of data from a single sensor is reduced in complex or harsh environments. For example, the recognition effect of a camera will be significantly reduced under lighting changes, strong light or low light conditions, and the accuracy of a lidar will be reduced under weather conditions such as rain and snow. This will reduce the stability of the navigation system in complex environments and easily lead to misjudgment or missed detection.

[0004] Existing path planning methods mostly use traditional static path planning algorithms such as the A algorithm and the Dijkstra algorithm, which can generate the optimal path in a known, fixed environment, but lack flexibility in complex dynamic environments and cannot respond to the dynamic changes of obstacles in real time. In addition, traditional path planning methods respond slowly to environmental changes and lack adaptability.

[0005] Intelligent navigation systems rely on multiple sensors to obtain environmental information, but existing systems lack effective fault-tolerance mechanisms when sensors fail or data is abnormal, which can lead to system failure or safety hazards. Especially in application scenarios such as unmanned driving and drones, the stability of sensors and the fault-tolerance of the system are particularly important. Existing navigation systems have relatively limited fault-tolerance and self-adaptation capabilities, and are prone to stagnation when sensors fail or emergencies occur, affecting the continuity of navigation tasks.

[0006] In a dynamic environment, intelligent navigation systems need to be able to make quick decisions to avoid obstacles in real time. However, the dynamic decision-making capabilities of existing systems are generally reduced. Most of them rely on preset rules or simple obstacle avoidance algorithms and lack the ability to adaptively adjust paths and decisions in a dynamic environment. If they encounter complex or rapidly changing obstacles, the system cannot respond in time, which can easily lead to collisions or other safety issues.

[0007] Therefore, those skilled in the art provide an intelligent navigation system based on image recognition to solve the above-mentioned problems. Summary of the invention

[0008] In view of the deficiencies in the prior art, the present invention provides an intelligent navigation system based on image recognition to solve the problems raised in the above background technology.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: an intelligent navigation system based on image recognition, comprising:

[0010] An image data acquisition module is used to acquire images of the environment. The image data acquisition module includes a binocular camera unit, which is used to generate a depth map and process the acquired image data through a data preprocessing module;

[0011] A data preprocessing module is connected to the image data acquisition module. The data preprocessing module includes denoising, contrast enhancement and HDR processing units, and is used to preprocess the image data and transmit the preprocessed image data to the multimodal data fusion module;

[0012] A multimodal data fusion module is connected to the data preprocessing module. The multimodal data fusion module includes an extended Kalman filter and a weighted data fusion unit, which is used to fuse the preprocessed image data with the laser radar and inertial measurement unit data to generate environmental data, and transmit the environmental data to the image recognition module;

[0013] An image recognition module is connected to the multimodal data fusion module. The image recognition module includes a target detection unit and a semantic segmentation unit, which are used to detect and classify target objects based on environmental data and to perform semantic segmentation on scenes. The generated recognition data is transmitted to the path planning module.

[0014] A path planning module is connected to the image recognition module. The path planning module includes a static path planning unit and a dynamic path planning unit, and is used to perform path planning based on the recognition data transmitted by the image recognition module, and transmit the planned path to the positioning and navigation module;

[0015] A positioning and navigation module is connected to the path planning module and the multimodal data fusion module. The positioning and navigation module includes a visual inertial fusion positioning unit for generating position information based on the fused data and the planned path.

[0016] The fault-tolerant and safety module connects the image data acquisition module, the multimodal data fusion module, the image recognition module, the path planning module and the positioning and navigation module. The fault-tolerant and safety module includes an anomaly detection unit and an adaptive fault-tolerant unit, which are used to monitor the data status of each module and trigger the fault-tolerant mechanism to adjust the system when an anomaly is detected.

[0017] Preferably, the image data acquisition module uses a binocular camera, which generates a depth map. The calculation formula of the depth D is:

[0018]

[0019] Among them, D is the depth value, which indicates the distance of the object from the camera; f is the focal length of the camera; B is the camera baseline distance, that is, the distance between the binocular cameras; d is the parallax, that is, the position difference of the same object in the left and right camera images.

[0020] Preferably, the data preprocessing module uses Gaussian filtering for denoising, and the filter weight calculation formula is:

[0021]

[0022] Among them, G(x, y) is the filter weight; x and y are pixel coordinates; σ is the standard deviation, which controls the filter smoothing effect; exp is the exponential function; 2π is the constant part of the normalization coefficient.

[0023] Preferably, the multimodal data fusion module uses an extended Kalman filter for data fusion, wherein the state update and measurement update formulas are:

[0024] Status prediction: x k|k-1 =f(x k-1|k-1 ),

[0025] Among them, x k|k-1 is the predicted value of the state at the kth moment, f is the state transfer function, x k-1|k-1 is the estimated value of the state at the k-1th moment

[0026] Covariance prediction:

[0027] Among them, P k|k-1 is the state prediction error covariance matrix at the kth moment, P k-1|k-1 is the state estimation error covariance matrix at the k-1th moment, F k-1 is the state transfer matrix, Q is the process noise covariance matrix, is the transpose of the state transfer matrix;

[0028] Kalman gain:

[0029] Among them, K k is the Kalman gain matrix at the kth moment, H k is the observation matrix, R is the observation noise covariance matrix, is the transpose of the observation matrix;

[0030] Status update: x k|k =x k|k-1 +K k (z k-h(x k|k-1 )),

[0031] Among them, x k|k is the estimated value after the state update at the kth moment, z k is the observed value at the kth moment, and h is the observation function.

[0032] Preferably, the image recognition module implements real-time target detection through the YOLOv5 algorithm, wherein the loss function includes category loss, positioning loss and confidence loss, as follows:

[0033] Loss = λ class ·Loss class +λ box ·Loss box +λ obj ·Loss obj ,

[0034] Among them, Loss class is the category loss; Loss box is the bounding box loss; Loss obj is the confidence loss; class , box and λ obj is the weight coefficient of each loss.

[0035] Preferably, the image recognition module uses the DeepLabV3+ model for semantic segmentation, and the segmentation loss function is the cross entropy loss:

[0036]

[0037] Where N is the total number of pixels; y i is the true category label of pixel i; is the predicted probability of pixel i, Loss seg is the loss function, is the predicted probability Take the logarithm.

[0038] Preferably, the path planning module uses the A algorithm to perform static environment path planning, and the cost function of each node is:

[0039] f(n)=g(n)+h(n),

[0040] Among them, f(n) is the total cost from the starting point to the target; g(n) is the actual cost from the current node to the starting point; h(n) is the heuristic estimated cost.

[0041] Preferably, the path planning module uses a deep reinforcement learning method to make path decisions in a dynamic environment, and the objective function of its strategy update is:

[0042]

[0043] Among them, R is the cumulative reward; γ is the discount factor; r t is the immediate reward at time step t, T represents the total number of time steps in the planning or decision-making process, and t represents the current time step.

[0044] Preferably, the positioning and navigation module adopts a visual inertial fusion positioning method and realizes state estimation through a state equation:

[0045] x k+1 =Fx k +Bu k +w k ,

[0046] Among them, x k+1 is the state at the next moment; F is the state transfer matrix; u k is the control input; B is the control input coefficient matrix; w k is the process noise, x k is the state at time k.

[0047] Preferably, the fault tolerance and safety module monitors sensor data anomalies through an anomaly detection algorithm, and triggers the fault tolerance mechanism if the fault score exceeds a threshold. The fault score calculation formula is:

[0048]

[0049] Where N is the number of sampling points; x i is the value of the i-th sampling point; is the mean of the sampling values, and Fault Score is the fault score, which is used to measure the deviation of sensor data to determine whether the sensor status is abnormal.

[0050] The present invention provides an intelligent navigation system based on image recognition, which has the following beneficial effects:

[0051] 1. The present invention adopts multimodal data fusion technology to integrate image, lidar and inertial measurement unit data, and combines the extended Kalman filter to achieve high-precision data fusion, enhance the accuracy and stability of environmental perception, overcome the limitations of a single sensor in a complex environment, and enable the system to accurately perceive the surrounding environment under conditions such as bad weather and lighting changes, significantly improving the overall reliability and adaptability of the system.

[0052] 2. The present invention combines the A algorithm with the deep reinforcement learning algorithm, so that the path planning module has excellent performance in both static and dynamic environments, realizes real-time obstacle avoidance and path update, and enables the system to flexibly respond to dynamic obstacles and complex environmental changes, which is impossible to achieve with traditional path planning methods, bringing significant innovation to the intelligent navigation system.

[0053] 3. The present invention integrates abnormality monitoring and adaptive fault-tolerant mechanisms. By real-time monitoring of the status of each sensor and module, it can automatically trigger the fault-tolerant mechanism when a fault or abnormality occurs, adjust the parameters of each module or switch to the backup mode, effectively improve the robustness and stability of the system, and ensure the continuity of the system in complex environments and sensor failure conditions. It has significant innovation in improving safety and is particularly suitable for autonomous driving and unmanned systems that require high stability.

[0054] 4. The present invention realizes adaptive decision-making ability in dynamic environment through deep reinforcement learning, and can automatically adjust the path and obstacle avoidance strategy according to environmental changes, so that the system has instant response ability in complex dynamic scenes, can autonomously determine the best path of travel, and reduce the impact of external interference on navigation, making the system more advantageous in application scenarios such as unmanned driving and drones, demonstrating the innovative breakthrough of intelligent navigation systems in real-time decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a system framework diagram of the present invention. DETAILED DESCRIPTION

[0056] In order to make the technical personnel in the technical field understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in combination with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a partial embodiment of the present invention, not a complete embodiment. Based on the embodiment of the present invention, other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0057] The present invention is described in detail below in conjunction with the accompanying drawings:

[0058] Example:

[0059] Please see attached Figure 1 , an embodiment of the present invention provides an intelligent navigation system based on image recognition, comprising:

[0060] An image data acquisition module is used to acquire images of the environment. The image data acquisition module includes a binocular camera unit, which is used to generate a depth map and process the acquired image data through a data preprocessing module;

[0061] A data preprocessing module is connected to the image data acquisition module. The data preprocessing module includes denoising, contrast enhancement and HDR processing units, and is used to preprocess the image data and transmit the preprocessed image data to the multimodal data fusion module;

[0062] A multimodal data fusion module is connected to the data preprocessing module. The multimodal data fusion module includes an extended Kalman filter and a weighted data fusion unit, which is used to fuse the preprocessed image data with the laser radar and inertial measurement unit data to generate environmental data, and transmit the environmental data to the image recognition module;

[0063] An image recognition module is connected to the multimodal data fusion module. The image recognition module includes a target detection unit and a semantic segmentation unit, which are used to detect and classify target objects based on environmental data and to perform semantic segmentation on scenes. The generated recognition data is transmitted to the path planning module.

[0064] A path planning module is connected to the image recognition module. The path planning module includes a static path planning unit and a dynamic path planning unit, and is used to perform path planning based on the recognition data transmitted by the image recognition module, and transmit the planned path to the positioning and navigation module;

[0065] A positioning and navigation module is connected to the path planning module and the multimodal data fusion module. The positioning and navigation module includes a visual inertial fusion positioning unit for generating position information based on the fused data and the planned path.

[0066] The fault-tolerant and safety module connects the image data acquisition module, the multimodal data fusion module, the image recognition module, the path planning module and the positioning and navigation module. The fault-tolerant and safety module includes an anomaly detection unit and an adaptive fault-tolerant unit, which are used to monitor the data status of each module and trigger the fault-tolerant mechanism to adjust the system when an anomaly is detected.

[0067] Advantages of the image data acquisition module: The image information of the environment is collected through the binocular camera, and the depth map is generated to provide the system with accurate three-dimensional spatial information. In addition, through the combination with the data preprocessing module, the image acquisition module can provide clear and denoised image data, provide high-quality input for subsequent recognition and processing, adapt to different lighting conditions, and increase the perception accuracy of the system;

[0068] Advantages of the data preprocessing module: De-noising, contrast enhancement and HDR processing of image data can maintain image clarity and details under different lighting conditions. At the same time, HDR processing can generate high-quality image information under strong or low light conditions, improve the adaptability of the system in a variety of environments, and enable the navigation system to work normally under extreme lighting conditions;

[0069] Advantages of the multimodal data fusion module: By extending the Kalman filter and weighted data fusion unit, the data from the image, lidar and inertial measurement unit are fused into consistent environmental data, so that the system can maintain stable perception performance under different environmental conditions and achieve flexible and accurate environmental modeling;

[0070] The advantages of the image recognition module are that it can classify and detect target objects in the environment and identify key areas, so that the system can fully understand the surrounding scene, provide rich information for path planning, and ensure that the path planning module can make intelligent decisions;

[0071] Advantages of the path planning module: Through static path planning and dynamic path planning units, the route can be effectively planned in different environments, so that the system can adapt to the complex changes in the dynamic environment, improve the safety and efficiency of path selection, and ensure that the system can respond in a timely manner in a complex and changing environment;

[0072] Advantages of the positioning and navigation module: The visual inertial fusion positioning unit combines image data with inertial measurement unit data to achieve high-precision positioning;

[0073] The advantages of the fault-tolerant and safety modules improve the stability and robustness of the system. In the event of sensor failure or sudden environmental changes, the adaptive fault-tolerant mechanism switches to the backup mode or adjusts the system parameters to ensure that the system continues to operate stably.

[0074] The image data acquisition module uses a binocular camera, which generates a depth map. The calculation formula for the depth D is:

[0075]

[0076] Among them, D is the depth value, which indicates the distance of the object from the camera; f is the focal length of the camera; B is the camera baseline distance, that is, the distance between the binocular cameras; d is the parallax, that is, the position difference of the same object in the left and right camera images.

[0077] Wide-angle cameras can provide a wide field of view and capture more environmental information. They are particularly suitable for application scenarios that require monitoring of a wide space. Compared with standard cameras, wide-angle cameras can cover a wide area of ​​the environment and reduce blind spots, so that the system has comprehensive environmental perception capabilities when performing navigation tasks, enhancing the safety and reliability of the system.

[0078] The combination of binocular camera and wide-angle camera can provide environmental information from different perspectives, which helps the system obtain a complete image of the surrounding environment, further improves the spatial perception ability of the system, and provides comprehensive input data for the multimodal data fusion module;

[0079] The combination of binocular camera and wide-angle camera makes the system more adaptable in complex and changing environments, enabling the system to effectively cope with complex outdoor and indoor environments and improve the robustness of navigation;

[0080] Through the collaborative work of binocular cameras and wide-angle cameras, the system can maintain data stability under conditions of ambient lighting changes, occlusions, etc., to improve the reliability of the system in different environments and provide stable basic data for subsequent image processing and recognition.

[0081] In summary, the use of binocular cameras and wide-angle cameras can enhance the system's spatial perception capabilities, cover the viewing angle range, provide perception accuracy and reliability in complex dynamic environments, and provide technical support for the environmental perception module of the intelligent navigation system.

[0082] The data preprocessing module uses Gaussian filtering to remove noise. The filter weight calculation formula is:

[0083]

[0084] Among them, G(x, y) is the filter weight; x and y are pixel coordinates; σ is the standard deviation, which controls the filter smoothing effect; exp is the exponential function; 2π is the constant part of the normalization coefficient.

[0085] Gaussian filtering uses a normal distribution function to control the weight distribution. It can retain the edge features of the image when smoothing the noise. Compared with other denoising methods, Gaussian filtering has a good effect on retaining the edges of the image, avoiding blurring of image details while maintaining important structural information in the image, so that the subsequent target detection and semantic segmentation modules can accurately identify the key features in the image.

[0086] The denoised image data becomes smoother, reducing the image blur caused by noise interference and enhancing the system's adaptability to different scenes. Gaussian filtering reduces data fluctuations and variability in image data, thereby improving the stability of image data and increasing the robustness of the system in different lighting conditions and environments.

[0087] When removing noise, Gaussian filtering can improve the contrast of local areas of the image, making dark or blurred areas clear, which helps the image recognition module better capture target features and scene information and improve recognition accuracy;

[0088] After the data preprocessing module removes noise through a Gaussian filter, it can significantly improve the computing efficiency of subsequent modules. The denoised image data is more concise and clear, which helps to reduce the computing burden of the deep learning model, accelerate the recognition and decision-making process, and provide technical support for the system to achieve real-time navigation.

[0089] The multimodal data fusion module uses an extended Kalman filter for data fusion, where the state update and measurement update formulas are:

[0090] Status prediction: x k|k-1 =f(x k-1|k-1 ),

[0091] Among them, x k|k-1 is the predicted value of the state at the kth moment, f is the state transfer function, x k-1|k-1 is the estimated value of the state at the k-1th moment

[0092] Covariance prediction:

[0093] Among them, P k|k-1 is the state prediction error covariance matrix at the kth moment, P k-1|k-1 is the state estimation error covariance matrix at the k-1th moment, F k-1 is the state transfer matrix, Q is the process noise covariance matrix, is the transpose of the state transfer matrix;

[0094] Kalman gain:

[0095] Among them, K k is the Kalman gain matrix at the kth moment, H k is the observation matrix, R is the observation noise covariance matrix, is the transpose of the observation matrix;

[0096] Status update: x k|k =x k|k-1 +K k (z k -h(x k|k-1 )),

[0097] Among them, x k|k is the estimated value after the state update at the kth moment, z k is the observed value at the kth moment, and h is the observation function.

[0098] The multimodal data fusion module uses an extended Kalman filter for data fusion, which can effectively improve the system's environmental perception accuracy, enhance robustness to noise, dynamically adjust data weights and update state estimates in real time, so that the intelligent navigation system can maintain high response speed and reliability in complex dynamic environments. At the same time, through multi-sensor data fusion, it ensures the stable operation of the system under various environmental conditions, which is an important part of achieving high precision and high stability of the intelligent navigation system.

[0099] The image recognition module implements real-time target detection through the YOLOv5 algorithm, where the loss function includes category loss, positioning loss, and confidence loss, as follows:

[0100] Loss = λ class ·Loss class +λ box ·Loss box +λ obj ·Loss obj ,

[0101] Among them, Loss class is the category loss; Loss box is the bounding box loss; Loss obj is the confidence loss; class , box and λ obj is the weight coefficient of each loss.

[0102] YOLOv5 can detect target objects in real time in dynamic scenes. YOLOv5 can detect accurately, which brings significant advantages to the intelligent navigation system, enabling the system to provide stable target detection results in the face of changing environments, ensuring the reliability of the system's navigation and obstacle avoidance decisions in changing environments;

[0103] YOLOv5 generates a confidence score when detecting a target, which is used to quantify the credibility of each detection result. It is optimized through confidence loss to help the system determine the accuracy of target detection. The system can set a threshold based on the confidence value and filter out detection results with low confidence, thereby improving the reliability of target detection and reducing the adverse effects of false detection.

[0104] The image recognition module achieves real-time target detection through the YOLOv5 algorithm, and has the advantages of multi-target detection, precise positioning, high confidence and lightweight, providing strong environmental perception capabilities for the intelligent navigation system. The efficiency and accuracy of YOLOv5 ensure that the system can achieve real-time response in complex dynamic environments, and provide high-quality data support during navigation and obstacle avoidance. It is an important part of the stable and safe operation of the intelligent navigation system.

[0105] The image recognition module uses the DeepLabV3+ model for semantic segmentation, and the segmentation loss function is the cross entropy loss:

[0106]

[0107] Where N is the total number of pixels; y i is the true category label of pixel i; is the predicted probability of pixel i, Loss seg is the loss function, is the predicted probability Take the logarithm.

[0108] The image recognition module uses DeepLabV3+ for semantic segmentation, which can provide the system with high-precision and fine-grained scene understanding, improve dynamic decision-making capabilities, resist complex background interference capabilities and adaptability to multi-category scenarios. The application of the cross-entropy loss function further ensures the segmentation accuracy, enabling the system to provide stable and reliable navigation support in a changing environment. At the same time, the segmentation results simplify the complexity of path planning, allowing the intelligent navigation system to respond flexibly in a variety of scenarios and improve the safety and efficiency of navigation.

[0109] The path planning module uses the A algorithm for static environment path planning, and the cost function of each node is:

[0110] f(n)=g(n)+h(n),

[0111] Among them, f(n) is the total cost from the starting point to the target; g(n) is the actual cost from the current node to the starting point; h(n) is the heuristic estimated cost.

[0112] The path planning module uses the A algorithm for static environment path planning, which can reduce the amount of calculation while ensuring the optimality of the path and adapt to a variety of environmental topological structures. The use of heuristic estimation improves search efficiency and navigation response speed, enabling the system to quickly determine the optimal path in a static environment. In addition, the A algorithm has good integration and can be combined with other dynamic path planning methods to improve path planning performance in complex environments, provide efficient and accurate paths for intelligent navigation systems, and improve the system's resource utilization and navigation capabilities.

[0113] The path planning module uses deep reinforcement learning methods to make path decisions in a dynamic environment, and the objective function of its strategy update is:

[0114]

[0115] Among them, R is the cumulative reward; γ is the discount factor; r t is the immediate reward at time step t, T represents the total number of time steps in the planning or decision-making process, and t represents the current time step.

[0116] The path planning module uses deep reinforcement learning methods to make path decisions in a dynamic environment, which enables the system to adapt to environmental changes, achieve long-term path optimization, and reduce dependence on prior knowledge of the environment. The reward mechanism and strategy update make path planning safer and more efficient, enhance the adaptability and robustness of the system in uncertain environments, and improve the path selection ability in a dynamic environment through continuous learning and experience accumulation, providing possibilities for long-term improvement of the system, so that the intelligent navigation system has greater flexibility and potential for continuous optimization.

[0117] The positioning and navigation module uses visual inertial fusion positioning method and realizes state estimation through state equation:

[0118] x k+1 =Fx k +Bu k +w k ,

[0119] Among them, x k+1 is the state at the next moment; F is the state transfer matrix; u k is the control input; B is the control input coefficient matrix; w k is the process noise, x k is the state at time k.

[0120] The positioning and navigation module adopts the visual-inertial fusion positioning method, which achieves high-precision, low-drift, and real-time positioning through the fusion of visual and inertial measurement unit data. The visual-inertial fusion positioning method can adapt to environments without GPS, reduce sensor drift, and has adaptability and anti-interference capabilities in dynamic environments. Compared with other positioning methods, the visual-inertial fusion positioning method has the advantages of low cost, good error control, and high long-term navigation stability, and is suitable for a wide range of intelligent navigation application scenarios.

[0121] The fault tolerance and safety module monitors sensor data anomalies through anomaly detection algorithms. If the fault score exceeds the threshold, the fault tolerance mechanism is triggered. The fault score calculation formula is:

[0122]

[0123] Where N is the number of sampling points; x i is the value of the i-th sampling point; is the mean of the sampling values, and Fault Score is the fault score, which is used to measure the deviation of sensor data to determine whether the sensor status is abnormal.

[0124] The fault-tolerance and safety module monitors the abnormality of sensor data through anomaly detection algorithms, quantifies the degree of sensor deviation through fault scoring, and triggers the fault-tolerance mechanism when the data is abnormal. It can achieve real-time monitoring, improve system stability, enhance the robustness and fault-tolerance of the system, and at the same time reduce maintenance costs and prevent the impact of single-point failures on the system. The system can provide high-quality data support in various complex environments to ensure the continuity and accuracy of navigation.

[0125] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent navigation system based on image recognition, characterized in that: include: An image data acquisition module is used to acquire images of the environment. The image data acquisition module includes a binocular camera unit, which is used to generate a depth map and process the acquired image data through a data preprocessing module; A data preprocessing module is connected to the image data acquisition module. The data preprocessing module includes denoising, contrast enhancement and HDR processing units, and is used to preprocess the image data and transmit the preprocessed image data to the multimodal data fusion module; A multimodal data fusion module is connected to the data preprocessing module. The multimodal data fusion module includes an extended Kalman filter and a weighted data fusion unit, which is used to fuse the preprocessed image data with the laser radar and inertial measurement unit data to generate environmental data, and transmit the environmental data to the image recognition module; An image recognition module is connected to the multimodal data fusion module. The image recognition module includes a target detection unit and a semantic segmentation unit, which are used to detect and classify target objects based on environmental data and to perform semantic segmentation on scenes. The generated recognition data is transmitted to the path planning module. A path planning module is connected to the image recognition module. The path planning module includes a static path planning unit and a dynamic path planning unit, and is used to perform path planning based on the recognition data transmitted by the image recognition module, and transmit the planned path to the positioning and navigation module; A positioning and navigation module is connected to the path planning module and the multimodal data fusion module. The positioning and navigation module includes a visual inertial fusion positioning unit for generating position information based on the fused data and the planned path. The fault-tolerant and safety module connects the image data acquisition module, the multimodal data fusion module, the image recognition module, the path planning module and the positioning and navigation module. The fault-tolerant and safety module includes an anomaly detection unit and an adaptive fault-tolerant unit, which are used to monitor the data status of each module and trigger the fault-tolerant mechanism to adjust the system when an anomaly is detected.

2. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The image data acquisition module uses a binocular camera, which generates a depth map. The calculation formula of the depth D is: Among them, D is the depth value, which indicates the distance of the object from the camera; f is the focal length of the camera; B is the camera baseline distance, that is, the distance between the binocular cameras; d is the parallax, that is, the position difference of the same object in the left and right camera images.

3. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The data preprocessing module uses Gaussian filtering for denoising, and the filter weight calculation formula is: Among them, G(x, y) is the filter weight; x and y are pixel coordinates; σ is the standard deviation, which controls the filter smoothing effect; exp is the exponential function; 2π is the constant part of the normalization coefficient.

4. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The multimodal data fusion module uses an extended Kalman filter for data fusion, where the state update and measurement update formulas are: Status prediction: x k|k-1 =f(x k-1|k-1 ), Among them, x k|k-1 is the predicted value of the state at the kth moment, f is the state transfer function, x k-1|k-1 is the estimated value of the state at the k-1th moment Covariance prediction: Among them, P k|k-1 is the state prediction error covariance matrix at the kth moment, P k-1|k-1 is the state estimation error covariance matrix at the k-1th moment, F k-1 is the state transfer matrix, Q is the process noise covariance matrix, is the transpose of the state transfer matrix; Kalman gain: Among them, K k is the Kalman gain matrix at the kth moment, H k is the observation matrix, R is the observation noise covariance matrix, is the transpose of the observation matrix; Status update: x k|k =x k|k-1 +K k (z k -h(x k|k-1 )), Among them, x k|k is the estimated value after the state update at the kth moment, z k is the observed value at the kth moment, and h is the observation function.

5. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The image recognition module implements real-time target detection through the YOLOv5 algorithm, where the loss function includes category loss, positioning loss and confidence loss, as follows: Loss=λ class ·Loss class +λ box ·Loss box +λ obj ·Loss obj , Among them, Loss class is the category loss; Loss box is the bounding box loss; Loss obj is the confidence loss; class , box and λ obj is the weight coefficient of each loss.

6. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The image recognition module uses the DeepLabV3+ model for semantic segmentation, and the segmentation loss function is the cross entropy loss: Where N is the total number of pixels; yi is the true category label of pixel i; is the predicted probability of pixel i, Loss seg is the loss function, is the predicted probability Take the logarithm.

7. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The path planning module uses the A algorithm to perform static environment path planning, and the cost function of each node is: f(n)=g(n)+h(n), Among them, f(n) is the total cost from the starting point to the target; g(n) is the actual cost from the current node to the starting point; h(n) is the heuristic estimated cost.

8. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The path planning module uses deep reinforcement learning method to make path decisions in a dynamic environment, and the objective function of its strategy update is: Among them, R is the cumulative reward; γ is the discount factor; r t is the immediate reward at time step t, T represents the total number of time steps in the planning or decision-making process, and t represents the current time step.

9. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The positioning and navigation module adopts the visual inertial fusion positioning method and realizes state estimation through the state equation: x k+1 =Fx k +This k +w k , Among them, x k+1 is the state at the next moment; F is the state transfer matrix; u k is the control input; B is the control input coefficient matrix; w k is the process noise, x k is the state at time k.

10. The intelligent navigation system based on image recognition according to claim 1, characterized in that: The fault tolerance and safety module monitors sensor data anomalies through an anomaly detection algorithm. If the fault score exceeds the threshold, the fault tolerance mechanism is triggered. The fault score calculation formula is: Where N is the number of sampling points; x i is the value of the i-th sampling point; is the mean of the sampling values, and Fault Score is the fault score, which is used to measure the deviation of sensor data to determine whether the sensor status is abnormal.

Images