Blind zone coverage omnibearing detection method and system based on AI intelligent video detection

Abstract

The invention discloses a blind area coverage omnibearing detection method and system based on AI intelligent video detection, and relates to the technical field of video detection. The method comprises the following steps: collecting and marking a large number of sample data training models, and optimizing the models into a miniaturized model suitable for edge equipment to realize accurate recognition of various targets including pedestrians and vehicles; using a visual field model to mark a physical blind area and an optical flow method to detect a dynamic shielding blind area, using an ORB algorithm to complete multi-view image registration and splicing, and using an LSTM network to predict the position of the blind area of the next frame; the two-dimensional detection result is converted into three-dimensional space information through three-dimensional mapping and a PCL library, and the environment model is updated in real time; the system provides real-time feedback and blind guiding services, and ensures that a user can obtain surrounding environment information in time and take safe actions in combination with voice prompt and a vibration feedback mechanism. In the whole process, efficient information processing and feedback are achieved through a multi-thread framework, and the fluency and safety of user experience are guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of video detection technology, and specifically relates to a blind spot coverage omnidirectional detection method and system based on AI intelligent video detection. Background Art

[0002] In the field of assisted mobility for the visually impaired, traditional guide devices primarily rely on single-modal sensors such as ultrasonic and infrared sensors for obstacle detection. These sensors suffer from limited detection range and poor environmental adaptability. This is particularly true in complex urban environments, where mixed dynamic obstacles (such as moving vehicles and pedestrians) and static obstacles (such as fire hydrants and construction fences), as well as blind spots caused by sudden changes in lighting (such as at tunnel entrances and exits) and obstructions (such as roadside trees), pose key bottlenecks to the practicality of existing technologies.

[0003] Among existing technical solutions, vision-based guidance systems mostly use a single camera combined with traditional image processing algorithms. Although they can identify some obstacles, they face three core challenges:

[0004] Insufficient blind spot coverage: A single camera has physical viewing angle limitations, and dynamic occlusion (such as pedestrians passing by) can cause temporary vision loss.

[0005] Real-time bottleneck: Traditional algorithms (such as SIFT feature matching) have high computational complexity, making it difficult to implement real-time processing of 720P video streams on embedded devices.

[0006] Lack of three-dimensional perception: Two-dimensional detection results cannot provide spatial information such as obstacle distance and height, resulting in insufficient reliability of obstacle avoidance decisions.

[0007] In recent years, the development of deep learning and multi-sensor fusion technologies has provided new solutions for overcoming these bottlenecks. Lightweight AI-based object detection models (such as YOLOv5-tiny) can significantly improve detection efficiency, but direct application to blind guidance scenarios still requires addressing the following issues:

[0008] Image enhancement in low-light environments: Traditional histogram equalization can easily lead to overexposure of highlights and requires the use of an adaptive local enhancement algorithm.

[0009] Multi-view spatiotemporal consistency modeling: Feature matching between different cameras must overcome interference caused by lighting changes and perspective differences;

[0010] Spatiotemporal correlation of blind spot prediction: Detection results based on a single frame cannot cope with scenarios where occluders suddenly appear, and a temporal prediction model needs to be introduced.

[0011] To address the above-mentioned issues, this technical solution proposes a comprehensive blind spot detection method based on AI intelligent video detection. Through key technologies such as multimodal image preprocessing, lightweight target detection, multi-perspective fusion and blind spot prediction, and three-dimensional space modeling, it achieves high-precision, real-time, and full-coverage blind spot guidance functions in complex environments. Summary of the Invention

[0012] In order to overcome the shortcomings and deficiencies of the above-mentioned prior art, the first purpose of the present invention is to provide a network security situation awareness system based on big data; the second purpose of the present invention is to provide a blind spot coverage all-round detection method based on AI intelligent video detection.

[0013] The first object of the present invention adopts the following technical solution:

[0014] The blind spot coverage all-round detection method based on AI intelligent video detection has the following process:

[0015] Video preprocessing: Perform Gaussian filtering and denoising, adaptive histogram equalization, brightness adjustment, and white balance color correction on the collected video to improve image quality;

[0016] AI target detection and recognition: Utilizes lightweight models to perform real-time target detection on pre-processed videos, outputting the location and category of targets such as obstacles and traffic signs;

[0017] Blind spot analysis and multi-view fusion: Identify physical blind spots and dynamic occlusion blind spots on a single camera, construct a panoramic field of view by stitching multi-view images using ORB feature matching and the RANSAC algorithm, and use the LSTM model to predict potential obstacles in blind spots based on historical frame data.

[0018] Comprehensive spatial modeling: Mapping 2D detection targets to 3D space through multi-camera calibration and triangulation, building a 3D environment model using the PCL library and marking dangerous areas;

[0019] Real-time feedback and guidance: The detection results are converted into non-visual signals including voice prompts and vibration feedback, combined with the real-time output of the device's built-in system to assist the blind in obstacle avoidance and navigation.

[0020] Preferably, the video preprocessing step includes:

[0021] Gaussian kernel is used to perform pixel-by-pixel denoising on high-resolution video frames;

[0022] The image is divided into 8×8 sub-regions using the CLAHE algorithm for local histogram equalization to enhance the edge details of obstacles in low-light environments.

[0023] The white balance algorithm based on neutral gray area detection corrects the RGB color channel gain coefficients to restore the true color of the image.

[0024] Preferably, the AI ​​target detection and recognition steps include:

[0025] Data preparation: We collected over 100,000 images at different times, weather conditions, and scenarios. We annotated over 20 types of objects, including tactile bricks and fire hydrants, and recorded metadata. We used two-person annotation and cross-verification to ensure that the annotation error rate was below the set threshold.

[0026] Model training: Based on the COCO pre-trained YOLOv5-tiny model, using a cosine annealing learning rate strategy and mixed precision training, trained for 300 epochs on an NVIDIA A100 GPU;

[0027] Deploy inference: The model is quantized into INT8 format, compressed to 1 / 4 of its original size, and dynamic memory optimization is performed for edge devices to achieve real-time detection of 720P video streams.

[0028] Preferably, the blind spot analysis and multi-view fusion steps include:

[0029] Blind spot identification: Mark physical blind spots based on the camera FOV, and use the optical flow method to detect occlusion blind spots where motion vectors are missing for more than three consecutive frames;

[0030] Image stitching: FAST corner detection and BRIEF descriptor are used to extract features, and the homography matrix is ​​estimated through Hamming distance matching and RANSAC algorithm. The stitching error is controlled within ±2 pixels.

[0031] Blind spot prediction: Build a two-layer LSTM network, input the category, coordinates, and velocity vector of the target in the past five frames, and output the predicted position box of the blind spot target.

[0032] Preferably, the omnidirectional space modeling step includes:

[0033] 3D mapping: Use OpenCV's cv2.calibrateCamera to obtain the camera's internal and external parameters, and calculate the target depth through triangulation. The reprojection error is less than 0.5 pixels.

[0034] Environmental modeling: Use the PCL library to perform voxel filtering and mesh reconstruction on the 3D point cloud, combine it with the DeepLabv3 + semantic segmentation model to mark dangerous areas, and dynamically update the model every 5 seconds.

[0035] Preferably, the real-time feedback and guidance steps include:

[0036] Signal conversion: TTS synthesis is achieved through Baidu Voice API, and the volume is automatically adjusted according to the ambient noise; vibration feedback uses "direction + frequency" encoding and the intensity is adjusted through PWM technology;

[0037] Real-time interaction: It adopts a multi-threaded architecture including detection, feedback, and communication threads, and achieves end-to-end latency of less than 300ms through the Handler mechanism. It supports Bluetooth peripherals including bone conduction headphones, vibration bracelet connections, and user-defined feedback modes.

[0038] The second purpose of the present invention adopts the following technical solution:

[0039] The blind spot coverage all-round detection system based on AI intelligent video detection is used to implement the blind spot coverage all-round detection method based on AI intelligent video detection. The system includes

[0040] Video preprocessing module: denoising, brightness adjustment and color correction;

[0041] AI target detection and recognition module: collects and labels sample data under different environmental conditions; uses professional tools to label targets and record metadata; performs transfer learning based on the YOLOv5-tiny model architecture to optimize model performance; converts the model into a format suitable for edge devices, and performs quantization and memory optimization;

[0042] Comprehensive spatial modeling module: This module uses multi-camera calibration technology and triangulation to convert 2D detection results into 3D spatial information. It uses the PCL library to process 3D point cloud data, generate a mesh model, and identify hazardous areas through a semantic segmentation model. It dynamically updates the environmental model to reflect the latest situation.

[0043] Real-time feedback and guidance module: This module uses voice prompts and vibration feedback mechanisms to provide personalized, immediate environmental information notifications, helping users avoid obstacles. A multi-threaded architecture ensures fast response, supports Bluetooth peripheral connections, allows users to customize feedback modes, and collects user operation logs to continuously improve system performance.

[0044] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0045] 1. This invention integrates Gaussian filtering denoising, CLAHE local brightness enhancement, and white balance color correction in the video preprocessing stage, and performs targeted optimization for complex environments such as low light, noise, and color distortion, ensuring that the input image is clear and the features are complete. Target detection uses the YOLOv5-tiny lightweight model, combined with data enhancement, model quantization, and edge device memory optimization, to achieve real-time detection of 720P video streams.

[0046] 2. This invention integrates physical blind spot marking with dynamic occlusion detection through blind spot analysis technology. It uses ORB feature matching and the RANSAC algorithm to achieve multi-camera panoramic stitching, covering blind spots beyond the 180° field of view of a single camera, creating a 270°+ global field of view. It also introduces an LSTM trajectory prediction model to predict the position of moving targets in occluded blind spots. Combining 3D spatial modeling with PCL point cloud processing technology, it constructs a dynamic environment model with semantic annotations, helping blind people develop a more three-dimensional spatial cognition.

[0047] 3. The present invention integrates voice prompts and vibration feedback through a signal conversion mechanism to meet the information transmission needs in different scenarios; the system architecture adopts multi-threaded real-time scheduling, and achieves a second-level response to detection results through a Handler mechanism. It is also compatible with Bluetooth peripherals and combines operation log collection optimization strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 The flowchart of the blind spot coverage omnidirectional detection method based on AI intelligent video detection of the present invention is shown. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more example embodiments. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. can be adopted. In other cases, well-known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0052] Example 1:

[0053] See Figure 1As shown, the blind spot coverage omnidirectional detection method based on AI intelligent video detection in this embodiment has the following process:

[0054] Step 1: Video preprocessing.

[0055] Denoising: A Gaussian filtering algorithm is used to perform a weighted average of image pixels by constructing a Gaussian kernel function, effectively suppressing random noise generated during the acquisition process due to sensor noise or environmental interference. For example, for a video frame with a resolution of 1920×1080, a 3×3 or 5×5 Gaussian kernel is used to scan each pixel to reduce the impact of noise on subsequent detection.

[0056] Brightness adjustment: CLAHE (Calibrating and Adaptive Histogram Equalization) divides the image into multiple sub-regions and performs histogram equalization on each sub-region, avoiding over-enhancement caused by global equalization. For example, in videos captured in low-light environments, CLAHE can enhance dark details and make the edges of obstacles clearer.

[0057] Color correction: Adjusts color deviation based on the white balance algorithm. By detecting neutral gray areas in the image (such as white walls and gray roads), the gain coefficient of the color channel is corrected to ensure that the image color is restored accurately and avoid target recognition errors caused by color distortion.

[0058] Step 2: AI target detection and recognition.

[0059] Data preparation: Use vehicle-mounted cameras and wearable device cameras to collect images at different time periods (early morning, noon, evening), weather conditions (sunny, rainy, foggy), and scenes (campuses, commercial streets, subway stations), and collect no less than 100,000 samples in total.

[0060] Fine-grained labeling: Using the professional labeling tool LabelMe, pedestrians, vehicles, steps, and other objects are annotated with polygons or rectangular boxes. Labeling categories are refined to over 20 (e.g., "brick paving," "fire hydrant," "speed bump"), and image metadata (such as shooting location and lighting intensity) are recorded. A labeling verification mechanism is established, using two-person labeling and cross-verification to reduce error rates.

[0061] Model selection and training: Use the YOLOv5-tiny model architecture and perform transfer learning based on the COCO pre-trained model. Set the initial learning rate to The learning rate is dynamically adjusted using a cosine annealing strategy. The loss function integrates classification loss (cross entropy), localization loss (CIOU), and confidence loss. Mixed precision training is used to accelerate convergence during training. 300 epochs are trained on an NVIDIA A100 GPU, achieving a balance between accuracy and speed.

[0062] Deployment and inference: The trained PyTorch model is converted to TensorFlow Lite format. Model quantization tools are used to quantize 32-bit floating-point parameters to 8-bit integers, reducing the model size to 1 / 4 its original size. Memory optimization is performed for edge devices (such as smart glasses powered by Qualcomm Snapdragon 665), using a dynamic memory allocation strategy to ensure model loading times are less than 500ms and memory usage remains stable under 200MB during inference. Inference is accelerated using the device's GPU, enabling real-time detection of 720P video streams.

[0063] Step 3: Blind spot analysis and multi-perspective fusion.

[0064] Blind spot identification: A field of view model is established based on camera parameters (such as the 180° viewing angle of a fisheye lens), marking areas outside the edges of video frames as physical blind spots. For dynamic occlusion, an occlusion detection algorithm based on optical flow is used to calculate pixel motion vectors in adjacent frames. An area is considered an occlusion blind spot if its motion vector is missing for more than three consecutive frames and its pixel values ​​remain unchanged.

[0065] Image registration and stitching: The ORB algorithm is used to extract image feature points. Key points are first quickly located using the FAST corner detection algorithm, and then binary feature descriptors are generated using the BRIEF descriptor. Feature matching is performed using cv2.BFMatcher, using the Hamming distance to measure similarity. A matching threshold is set to filter out mismatched points. The homography matrix H is estimated using the RANSAC algorithm. Adjacent images are projected onto the same plane and stitched into a panoramic image using image fusion techniques (such as fade-in and fade-out weighted averaging). The stitching error is kept within ±2 pixels.

[0066] Blind spot prediction: An LSTM network model is constructed. The input is the object detection results (including object category, location coordinates, and velocity vector) from the past five frames, and the output is the predicted bounding box for the next frame's blind spot location. The network structure is set as two LSTM layers (128 neurons per layer) + a fully connected layer, using a Reluctant Unit (ReLU) activation function. For example, if a vehicle is continuously moving to the right and is obscured by a pedestrian, the LSTM predicts its position in the blind spot based on its historical trajectory, with a predicted position error of ≤10cm.

[0067] Step 4: Comprehensive spatial modeling

[0068] 3D Mapping: Through multi-camera calibration (obtaining intrinsic and extrinsic parameters), triangulation is used to map 2D detection targets into 3D space. For example, OpenCV's cv2.calibrateCamera function can be used for calibration and to calculate object depth.

[0069] Environmental Modeling: 3D point cloud data is converted into a mesh model using the PCL library. Voxel filtering parameters (e.g., leaf size 0.05m) are set to reduce the number of point clouds. For hazardous areas such as interrupted blind paths and steep slopes, semantic segmentation models (e.g., DeepLabv3+) are used to identify area types in the 2D image. These areas are then mapped to a 3D model and labeled with risk levels (e.g., high-risk steep slopes are marked in red). A dynamic update mechanism is established to refresh the model every 5 seconds based on new inspection data to ensure real-time and accurate environmental information.

[0070] Step 5: Real-time feedback and guidance.

[0071] Signal Conversion: Voice prompts: Using the Baidu Voice API for text-to-text conversion, personalized voice prompts are generated based on target type and distance. For example, "There are stairs 2 meters ahead, please take a detour." The speech synthesis delay is less than 200ms, and the volume automatically adapts to the ambient noise (using the device microphone to monitor the ambient sound intensity in real time).

[0072] Vibration feedback: Designed with a "direction + frequency" coding scheme, for example, obstacles on the left side are detected by the left vibration motor at a high frequency of 5Hz, obstacles on the right side are detected by the right motor at a low frequency of 3Hz, and obstacles directly in front of the vehicle activate simultaneous vibrations from both motors. PWM technology is used to control the vibration module voltage and adjust the vibration intensity (adjustable from 0-100%).

[0073] Real-time interaction: Utilizes a multi-threaded architecture (detection thread, feedback thread, and communication thread). After the detection thread outputs the result for each frame, it immediately notifies the feedback thread through a handler mechanism to perform signal conversion, ensuring end-to-end latency of less than 300ms. Compatible with Bluetooth peripherals (such as bone conduction headphones and vibrating wristbands), it connects in real time via the BLE protocol and supports user-defined feedback modes (such as voice priority or vibration priority). Regularly collects user operation logs (such as the number of false prompts and the frequency of emergency obstacle avoidance triggers) to optimize feedback strategies.

[0074] The beneficial effects of this embodiment are as follows: This embodiment achieves efficient denoising, accurate target detection, intelligent blind spot prediction, three-dimensional environment modeling and multimodal real-time feedback through multi-dimensional technology optimization, significantly improving the environmental perception ability and user interaction experience of the guide system, and providing safer and more convenient travel assistance for the visually impaired.

[0075] Example 2:

[0076] The blind spot coverage omnidirectional detection system based on AI intelligent video detection in this embodiment includes

[0077] Video preprocessing module: denoising, brightness adjustment and color correction.

[0078] AI target detection and recognition module: Collects and labels sample data under different environmental conditions; uses professional tools to label targets and record metadata; performs transfer learning based on the YOLOv5-tiny model architecture to optimize model performance; converts the model into a format suitable for edge devices, and performs quantization and memory optimization.

[0079] Comprehensive spatial modeling module: Through multi-camera calibration technology, triangulation is used to convert 2D detection results into 3D spatial information. The PCL library is used to process 3D point cloud data to generate a mesh model, and a semantic segmentation model is used to identify dangerous areas. The environmental model is dynamically updated to reflect the latest situation.

[0080] Real-time feedback and guidance module: This module uses voice prompts and vibration feedback mechanisms to provide personalized, immediate environmental information notifications, helping users avoid obstacles. A multi-threaded architecture ensures fast response, supports Bluetooth peripheral connections, allows users to customize feedback modes, and collects user operation logs to continuously improve system performance.

[0081] The beneficial effects of this embodiment are as follows: the system improves image quality through video preprocessing, AI target detection accurately identifies multiple types of targets, all-round spatial modeling realizes three-dimensional environmental perception, real-time feedback and guidance modules provide multi-modal instant prompts, significantly enhancing the environmental adaptability and travel safety of visually impaired people, while supporting user customization and continuous system optimization.

[0082] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

[0083] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. The blind spot coverage omnidirectional detection method based on AI intelligent video detection is characterized by: The method flow is as follows: Video preprocessing: Perform Gaussian filtering and denoising, adaptive histogram equalization, brightness adjustment, and white balance color correction on the collected video to improve image quality; AI target detection and recognition: Utilizes lightweight models to perform real-time target detection on pre-processed videos, outputting the location and category of targets such as obstacles and traffic signs; Blind spot analysis and multi-view fusion: Identify physical blind spots and dynamic occlusion blind spots on a single camera, construct a panoramic field of view by stitching multi-view images using ORB feature matching and the RANSAC algorithm, and use the LSTM model to predict potential obstacles in blind spots based on historical frame data. Comprehensive spatial modeling: Mapping 2D detection targets to 3D space through multi-camera calibration and triangulation, building a 3D environment model using the PCL library and marking dangerous areas; Real-time feedback and guidance: The detection results are converted into non-visual signals including voice prompts and vibration feedback, combined with the real-time output of the device's built-in system to assist the blind in obstacle avoidance and navigation.

2. The blind spot coverage omnidirectional detection method based on AI intelligent video detection according to claim 1 is characterized in that: The video preprocessing step includes: Gaussian kernel is used to perform pixel-by-pixel denoising on high-resolution video frames; The image is divided into 8×8 sub-regions using the CLAHE algorithm for local histogram equalization to enhance the edge details of obstacles in low-light environments. The white balance algorithm based on neutral gray area detection corrects the RGB color channel gain coefficients to restore the true color of the image.

3. The blind spot coverage omnidirectional detection method based on AI intelligent video detection according to claim 1 is characterized in that: The AI ​​target detection and recognition steps include: Data preparation: We collected over 100,000 images at different times, weather conditions, and scenarios. We annotated over 20 types of objects, including tactile bricks and fire hydrants, and recorded metadata. We used two-person annotation and cross-verification to ensure that the annotation error rate was below the set threshold. Model training: Based on the COCO pre-trained YOLOv5-tiny model, using a cosine annealing learning rate strategy and mixed precision training, trained for 300 epochs on an NVIDIA A100 GPU; Deploy inference: The model is quantized into INT8 format, compressed to 1 / 4 of its original size, and dynamic memory optimization is performed for edge devices to achieve real-time detection of 720P video streams.

4. The blind spot coverage omnidirectional detection method based on AI intelligent video detection according to claim 1 is characterized in that: The blind spot analysis and multi-view fusion steps include: Blind spot identification: Mark physical blind spots based on the camera FOV, and use the optical flow method to detect occlusion blind spots where motion vectors are missing for more than three consecutive frames; Image stitching: FAST corner detection and BRIEF descriptor are used to extract features, and the homography matrix is ​​estimated through Hamming distance matching and RANSAC algorithm. The stitching error is controlled within ±2 pixels. Blind spot prediction: Build a two-layer LSTM network, input the category, coordinates, and velocity vector of the target in the past five frames, and output the predicted position box of the blind spot target.

5. The blind spot coverage omnidirectional detection method based on AI intelligent video detection according to claim 1 is characterized in that: The omnidirectional space modeling step includes: 3D mapping: Use OpenCV's cv2.calibrateCamera to obtain the camera's internal and external parameters, and calculate the target depth through triangulation. The reprojection error is less than 0.5 pixels. Environmental modeling: Use the PCL library to perform voxel filtering and mesh reconstruction on the 3D point cloud, combine it with the DeepLabv3 + semantic segmentation model to mark dangerous areas, and dynamically update the model every 5 seconds.

6. The blind spot coverage omnidirectional detection method based on AI intelligent video detection according to claim 1 is characterized in that: The real-time feedback and guidance steps include: Signal conversion: TTS synthesis is achieved through Baidu Voice API, and the volume is automatically adjusted according to the ambient noise; vibration feedback uses "direction + frequency" encoding, and the intensity is adjusted through PWM technology; Real-time interaction: It adopts a multi-threaded architecture including detection, feedback, and communication threads, and achieves end-to-end latency of less than 300ms through the Handler mechanism. It supports Bluetooth peripherals including bone conduction headphones, vibration bracelet connections, and user-defined feedback modes.

7. A blind spot coverage omnidirectional detection system based on AI intelligent video detection, used to implement the blind spot coverage omnidirectional detection method based on AI intelligent video detection as claimed in claim 1, characterized in that: The system includes Video preprocessing module: denoising, brightness adjustment and color correction; AI target detection and recognition module: collects and labels sample data under different environmental conditions; uses professional tools to label targets and record metadata; performs transfer learning based on the YOLOv5-tiny model architecture to optimize model performance; converts the model into a format suitable for edge devices, and performs quantization and memory optimization; Omnidirectional spatial modeling module: Using multi-camera calibration technology and triangulation to convert 2D detection results into 3D spatial information; Use the PCL library to process 3D point cloud data, generate a mesh model, identify dangerous areas through a semantic segmentation model, and dynamically update the environment model to reflect the latest situation; Real-time feedback and guidance module: Using voice prompts and vibration feedback mechanisms, it provides personalized and immediate environmental information notifications to help users avoid obstacles; The multi-threaded architecture is designed to ensure fast response, support Bluetooth peripheral connection, allow users to customize feedback mode, and collect user operation logs to continuously improve system performance.

Images