BSD vehicle-mounted blind area monitoring system based on video radar fusion and calibration method thereof

Abstract

The invention provides a BSD vehicle-mounted blind area monitoring system based on video radar fusion and a calibration method of the BSD vehicle-mounted blind area monitoring system, and belongs to the technical field of vehicle safety auxiliary driving. The system comprises a vehicle-mounted monitoring host end, a remote client and a cloud server end. An image acquisition module, a radar acquisition module, a BSD installation and calibration module, a BSD blind area video monitoring module, a BSD blind area radar monitoring module, a BSD alarm module and a sound-light alarm module are integrated at the host end. The system realizes accurate detection and classification of pedestrians, non-motor vehicles and vehicle targets in a blind area by fusing camera and radar data. The method is characterized in that an automatic calibration method based on camera parameters and a homography matrix is adopted, an alarm area accurately matched with the actual physical distance can be generated without laying calibration cloth on site, and local and remote calibration is supported. The system has the advantages of being accurate in detection, easy and convenient to install, wide in adaptive vehicle type, capable of being remotely maintained and the like, and the driving safety is effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of vehicle video surveillance and vehicle safety assistance driving technology, specifically to a BSD (Blind Spot Detection) vehicle blind spot monitoring system and its calibration method based on video and radar fusion and with automatic calibration function, belonging to the category of intelligent driving assistance systems (ADAS) and vehicle active safety monitoring technology. Background Technology

[0002] With the continuous growth of motor vehicle ownership and the increasing complexity of traffic environments, traditional rearview mirrors offer a limited field of vision and cannot completely eliminate blind spots around vehicles. Especially during critical operations such as starting, turning, changing lanes, and making U-turns, blind spots around the vehicle are easily obscured. When pedestrians, non-motorized vehicles, or other vehicles are present in these blind spots, traffic accidents are highly likely to occur.

[0003] Current BSD (Blind Spot Detection) systems on the market primarily rely on radar technology. However, radar technology struggles to effectively identify obstacle types and cannot distinguish between pedestrians, non-motorized vehicles, or other targets. Furthermore, existing systems generally suffer from drawbacks such as large size, complex installation, high cost, and inconvenience for aftermarket installation, hindering their widespread adoption across various vehicle models.

[0004] This invention enables real-time detection and classification of pedestrians, non-motorized vehicles, and other vehicle targets within a vehicle's blind spot. It combines automatic calibration technology to generate alarm zones that precisely match the actual physical distance. It is applicable to Advanced Driver Assistance Systems (ADAS) and vehicle active safety monitoring, and is particularly suitable for blind spot monitoring using wide-angle cameras. -New Summary of the Invention To address the shortcomings of existing technologies, the purpose of this invention is to provide a BSD vehicle blind spot monitoring system based on video radar fusion and its calibration method.

[0005] A BSD vehicle blind spot monitoring system based on video radar fusion according to the present invention includes: Vehicle-mounted monitoring host terminal; The vehicle-mounted monitoring host includes: The image acquisition module is used to acquire video images of the vehicle's external blind spots; The radar acquisition module is used to collect target information within a preset range around the vehicle; The BSD installation and calibration module is used to automatically generate blind spot alarm areas that match the physical distance based on camera parameters and the mapping relationship between the image coordinate system and the world coordinate system. The BSD blind spot video detection module, through preprocessing, feature extraction, target detection and tracking, enables real-time identification and trajectory maintenance of pedestrians and obstacles in blind spots captured by wide-angle cameras.

[0006] The BSD blind zone radar monitoring module enables real-time target identification, state estimation, and trajectory maintenance through radar target detection and tracking.

[0007] The BSD alarm module is used to fuse and process the output data of the BSD blind spot video detection module and the BSD blind spot radar monitoring module, and generate an alarm decision based on the target's location information within the blind spot alarm area; And an audible and visual alarm module, used to execute the alarm decision and output audible and visual alarm signals.

[0008] Preferably, the image acquisition module includes at least one wide-angle camera for acquiring images of the vehicle's external blind spots.

[0009] Preferably, the radar acquisition module uses a 24GHz or 77GHz shortwave radar, which is installed on the outside of the vehicle to detect the distance, speed and direction of movement of targets within a preset range around the vehicle in real time.

[0010] Preferably, the BSD alarm module divides the blind spot alarm area into at least a first-level alarm area, a second-level alarm area, and a third-level alarm area according to a preset distance from the vehicle, and generates alarm decisions of different levels accordingly.

[0011] Preferably, the first-level alarm zone corresponds to a range within 1 meter of the vehicle, the second-level alarm zone corresponds to a range of 1 to 2 meters of the vehicle, and the third-level alarm zone corresponds to a range of 2 to 3 meters of the vehicle.

[0012] Preferably, the BSD installation and calibration module generates a mapping lookup table from image coordinates to world coordinates through wide-angle image distortion correction, top-view transformation, and establishment of homography matrix, so as to realize the automatic generation of blind spot alarm area.

[0013] Preferably, the BSD installation and calibration module supports local calibration mode and remote calibration mode; in local calibration mode, parameters are configured and calibrated after a local connection is established between the mobile terminal application and the vehicle monitoring host; in remote calibration mode, calibration is completed by sending calibration commands and parameters to the vehicle monitoring host through a cloud server.

[0014] Preferably, the sound and light alarm module includes an LED light source and a voice speaker, which can trigger the LED of the corresponding color to flash and broadcast the corresponding warning voice according to the received alarm decision.

[0015] Preferably, the system further includes a communication module for enabling wireless data communication between the vehicle-mounted monitoring host and remote clients and cloud servers.

[0016] A calibration method for a BSD vehicle blind spot monitoring system according to the present invention includes the following steps: Obtain the camera's internal parameters and distortion parameters; Based on the alignment relationship of the vehicle body reference lines in the image, a homography matrix between the image coordinate system and the world coordinate system is established. Based on the homography matrix, a coordinate mapping relationship from the image plane to the actual physical plane is generated; Based on the coordinate mapping relationship and the preset distance classification threshold, a blind spot alarm area matching the physical distance is automatically generated in the camera image.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves deep fusion of video and radar information, combining the advantages of radar's accurate ranging and speed measurement, unaffected by lighting conditions, with the ability of a camera to identify target types. This fusion mechanism effectively overcomes the limitations of pure radar systems in distinguishing between pedestrians, non-motorized vehicles, and other targets, and also compensates for the performance degradation of pure vision systems in adverse weather or at night, thus achieving higher precision and more reliable detection and classification of targets within blind spots.

[0018] 2. The automatic calibration mechanism of this invention, based on preset camera parameters and the homography matrix principle, eliminates the need for laying dedicated calibration cloth or performing complex manual marking on-site. Installers only need to adjust the camera angle to align the reference lines on the screen, and the system automatically generates alarm zones that precisely match the actual physical distance. This greatly simplifies the installation process, reduces the skill requirements for installers, and shortens deployment time, making it particularly suitable for large-scale, rapid installations in the aftermarket.

[0019] 3. This invention divides blind spots into multi-level (e.g., three-level) alarm zones based on distance, and corresponds to different audible and visual alarm strategies (e.g., flashing LEDs of different colors, differentiated voice prompts). This tiered alarm method provides drivers and pedestrians outside the vehicle with clearer and more accurate risk level information, avoiding interference from continuous alarms while ensuring a strong warning when danger is approaching, effectively improving the level of active safety. Attached Figure Description

[0020] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1This is a schematic diagram of the BSD vehicle-mounted video blind spot monitoring system in an embodiment of the present invention; Figure 2 This is a schematic diagram of the vehicle-mounted monitoring host structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of a graded alarm system in an embodiment of the present invention; Figure 4 This is a calibration flowchart in an embodiment of the present invention. Detailed Implementation

[0021] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0022] This invention discloses a BSD vehicle-mounted video blind spot monitoring system, which mainly includes an in-vehicle monitoring host, a remote client, a cloud server, and a display connected to the in-vehicle monitoring host. (See reference...) Figure 1 As shown, the components interact and transmit commands through network (4G / WiFi) and wired connections, forming a complete closed loop for blind spot monitoring and remote management.

[0023] The vehicle-mounted monitoring host is the core processing unit of the system, responsible for data acquisition, fusion processing, alarm decision-making, and execution. For example... Figure 2 As shown, it mainly includes the following modules: The system includes an image acquisition module, a radar acquisition module, a BSD blind zone video detection module, a BSD blind zone radar monitoring module, a video radar fusion processing module, a BSD alarm module, a BSD installation and calibration module, an audible and visual alarm module, a monitor, an extension cable, a GPS positioning module, a video processing module, a storage module, a communication module, and a main control module.

[0024] The following provides further explanation of each module.

[0025] The image acquisition module is used to capture images of both the interior and exterior of the vehicle. Depending on the camera's installation location, the acquired data is divided into interior and exterior image monitoring data.

[0026] The external view of the vehicle is captured using a wide-angle camera, which captures images of the blind spots on the front, rear, left, and right sides of the vehicle.

[0027] The vehicle interior scene is captured using pinhole cameras. The installation location and number are determined based on different vehicle models. Cameras can be installed inside the passenger compartment or facing the driver. Taking a bus as an example, one camera is installed facing the driver, one on each of the front and rear doors, and one can be installed on the rear side of the passenger compartment facing forward. The number of cameras can be increased according to actual needs. In a preferred embodiment, the camera capturing the vehicle interior scene is an infrared camera, which can be installed on the A-pillar or steering wheel to collect driver view information.

[0028] The radar acquisition module is used to collect real-time information on non-motorized vehicles, pedestrians, and other vehicles within a preset range (preferably 3 meters) around the vehicle. The radar used is primarily a 24 GHz or 77 GHz shortwave radar, capable of detecting both short-range and medium-to-long-range targets. The radar is installed on the outer side of the vehicle, including the front, left, right, and rear sides, covering the entire blind spot around the vehicle. When the vehicle speed is greater than or equal to 0 km / h, the radar module automatically operates in real-time, emitting microwave signals and receiving reflected echoes to monitor targets in the blind spot. The system can acquire information such as the target's distance, speed, direction of movement, and acceleration, and simultaneously make a preliminary judgment on the target type based on the intensity and characteristics of the reflected signal. The radar acquisition module works in conjunction with the BSD alarm module to exclude non-interfering objects and distant objects, triggering an alarm only when a target enters the danger zone. This effectively prevents potential collisions and improves driving safety in situations such as severe weather, driver negligence, novice driving, or rapid lane changes.

[0029] The installation location and number of radar sensors can be flexibly configured according to different vehicle models and customer needs, typically installed on the front, left, right, and rear sides of the vehicle. For small vehicles or specific application scenarios, installation with fewer than four radars can also be selected to reduce costs and installation complexity. The radar module supports interface with the vehicle's CAN bus or OBD system, enabling linkage with vehicle status information such as vehicle speed and steering wheel angle, further improving the accuracy of blind spot detection and the timeliness of alarms.

[0030] The radar acquisition module can work in parallel with the image acquisition module to achieve video-radar fusion detection, thereby improving the accuracy and reliability of target identification in blind spots. Fusion methods include: 1) The radar first detects potential targets, and then the image acquisition module performs visual confirmation and type identification; 2) Confidence-weighted fusion of radar data and image detection results is performed to generate high-precision blind zone alarm information; 3) In some cost-sensitive scenarios, low-cost blind spot monitoring can be achieved by using only the BSD image acquisition module.

[0031] The system can flexibly select the combination of radar and camera according to vehicle type and installation environment, and at the same time, it can achieve all-weather, all-scenario blind spot safety monitoring through the BSD alarm module for unified management and alarm output.

[0032] The BSD blind spot video detection module receives image data acquired via wide-angle BSD cameras. It includes a preprocessing module, feature extraction and detection, target detection module, and target tracking module.

[0033] The preprocessing module standardizes the video frames captured by the vehicle-mounted camera to improve detection accuracy and reduce computational load. First, for nighttime or shadowy scenes, it automatically enhances brightness and contrast to improve the visibility of pedestrians, non-motorized vehicles, and other vehicles in the image, thereby improving detection performance. Second, since the images captured by the wide-angle BSD camera exhibit fisheye distortion, distortion correction is necessary. A fisheye image correction lookup table (LUT) is generated during the fisheye camera's factory calibration phase. Using camera intrinsic parameters and the fisheye calibration model, the original image is quickly mapped to a viewpoint-corrected image, reducing distortion of pedestrians and obstacles.

[0034] The feature extraction and detection module extracts multi-level semantic information from images to support accurate pedestrian target identification. This module first employs a lightweight convolutional neural network as the backbone to extract features from preprocessed image frames, outputting multi-scale features encompassing spatial details to high-level semantic information. Subsequently, the neck section introduces a lightweight FPN / PAN structure to fuse multi-layer features in a top-down and bottom-up manner, thereby enhancing the detection capability for small-scale and occluded pedestrians. The final generated multi-scale feature map serves as the input to the target detection head, providing a reliable feature foundation for subsequent blind-spot pedestrian detection.

[0035] The object detection module is responsible for classifying pedestrian targets and regressing bounding boxes on multi-scale feature maps. The system employs an anchor-free point regression mechanism, directly predicting a vector composed of class confidence and boundary offset at each pixel location. During training, the classification branch uses Generalized Focal Loss (GFL), and the regression branch uses IoU / GIoULoss, with positive sample selection based on the target center region to improve the detection quality of occlusion and small targets. During inference, the prediction results of feature maps at each scale are decoded, and redundant boxes are filtered through non-maximum suppression (NMS), finally outputting pedestrian detection results in the format [Xmin, Ymin, Xmax, Ymax, score].

[0036] The target tracking module is used to associate the identities and maintain the trajectories of detected pedestrians in consecutive video frames, thereby determining their dynamic changes within blind spots. This module employs a lightweight IOU Tracker or SORT algorithm suitable for embedded environments. By matching the detection results [bbox, score] of the current frame with the target state of the previous frame using IOU or appearance features, it achieves target ID updates, trajectory continuation, new target creation, and management of lost targets. The resulting continuous pedestrian trajectory information can be further used for advanced functions such as blind spot assessment, risk analysis, and audible and visual alarms.

[0037] The BSD blind spot radar monitoring module detects and analyzes targets in blind spots based on collected fundamental physical quantities such as range, radial velocity, and azimuth. It generates preliminary target information through noise suppression, false alarm filtering, and feature analysis. The module consists of radar target detection and radar target tracking, enabling the identification, state estimation, and trajectory maintenance of targets in blind spots. This ensures stable and efficient blind spot monitoring capabilities in low visibility, driver inattention, or lane-changing scenarios.

[0038] The radar target detection module utilizes processed echo intensity, range, and velocity characteristics to cluster or segment potential targets within the blind zone, forming a detection list to achieve preliminary identification of targets such as pedestrians, non-motorized vehicles, and vehicles. Subsequently, the radar target tracking module employs a Kalman Filter or EKF to continuously track and estimate the state of the target, outputting dynamic information such as target position (X,Y) and velocity (Vx,Vy). Simultaneously, it performs target ID management, trajectory maintenance, and short-term occlusion compensation to ensure the stability and reliability of target trajectories in the blind zone, providing crucial information for subsequent image fusion, blind zone assessment, and risk warning.

[0039] The video radar fusion processing module receives data from the image acquisition module and the BSD blind spot radar monitoring module, and achieves high-precision detection of pedestrians, non-motorized vehicles, and vehicles within the blind spot through a multi-layer fusion algorithm. The fusion processing includes the following steps: 1) The radar detects potential targets first and provides range, radial velocity and azimuth information; 2) The image acquisition module performs visual recognition of the area corresponding to the radar target and classifies the target type through deep learning models (such as YOLO series models) to achieve accurate identification of pedestrians, bicycles, electric vehicles and motor vehicles; 3) The radar data and visual recognition results are fused with confidence weighting to form high-precision target status information, including the probability of target existence, position and relative motion trend; 4) Calculate the blind spot risk level based on the target location, vehicle speed, and driving trajectory, and generate real-time alarm decisions.

[0040] The BSD alarm module is the core intelligent unit of the vehicle blind spot monitoring system. It is used to receive data from the BSD blind spot video detection module, the BSD blind spot radar monitoring module and the video radar fusion processing module, and to perform multi-source data fusion, blind spot risk assessment, alarm decision and output management to achieve high-precision detection and hierarchical alarm of blind spot targets.

[0041] The BSD alarm module issues alerts based on alarm decisions, including flashing external audible and visual alarms and voice prompts, in-vehicle voice prompts, and full-screen display of blind spot targets on the monitor, providing multi-channel alerts. Based on the distance and hazard level of the target within the blind spot, it refers to... Figure 3 As shown, the alarm module can provide tiered alerts: Level 1 alarm (red zone, within 1 meter of danger): Triggers flashing of bright red LEDs outside the vehicle and voice announcement of "Danger, please pay attention to pedestrians or vehicles immediately", while the in-vehicle voice simultaneously reminds the driver; Level 2 alarm (yellow area, distance 1-2 meters): Triggers flashing yellow LED and voice prompt "Caution pedestrians / non-motorized vehicles"; Level 3 alarm (green zone, 2-3 meters away): The green LED flashes to indicate "Please stay away from the vehicle", serving only as a low-intensity reminder.

[0042] The audible and visual alarm integrates a bright red LED and an embedded speaker, boasts an IP67 protection rating, and supports all-weather application. The alarm module automatically selects the alarm channel and alert intensity based on the target type and hazard level, and records the alarm trigger event and timestamp information for subsequent data analysis and vehicle safety management.

[0043] Furthermore, the BSD alarm module allows for configurable parameters, including blind spot range, alarm level thresholds, audible and visual alarm strategies, trigger delays, and durations, to adapt to different vehicle models, driving environments, and customer needs. The alarm module interacts in real time with the main control module and the vehicle communication bus, and can be linked with vehicle status information (such as vehicle speed, steering angle, and acceleration) to optimize the accuracy of blind spot detection and the timeliness of alarms.

[0044] The BSD (Blind Spot Detection) module is installed and calibrated. BSD cameras are typically installed on the left, right, rear, or front of the vehicle. Depending on the model, they can be installed below the rearview mirror or in the center of the side of the vehicle. The installation position is aligned with the vehicle's baseline to ensure coverage of blind spots. However, depending on the actual vehicle, the BSD cameras on both sides can also be installed in the position of the left and right exterior rearview mirrors or in the center of the sides. The black circles indicate the suggested installation positions for the cameras. Figure 1 As shown.

[0045] In a preferred embodiment, the BSD camera uses an AI neural network to detect non-motorized vehicles and pedestrians in blind spots, such as electric vehicles, pedestrians, bicycles, and motor vehicles, providing early warnings. It can also be used for lane change warnings for the vehicle itself, preventing accidents during lane changes.

[0046] Furthermore, the number of BSD cameras can be selected according to customer needs. Four or fewer BSD cameras can be installed. Simultaneously, four or fewer radar modules can be installed, balancing cost advantages. For example, if the vehicle already has a reversing camera, only three cameras need to be installed.

[0047] BSD warnings only detect blind spots in real time, avoiding visual interference for the driver. Blind spot areas are generated automatically using a calibration algorithm, eliminating the need for manual drawing. Calibration is automatically calculated based on camera intrinsic and extrinsic parameters, homography matrix, and a preset mapping table to generate alarm zones that match the actual distance. Traditionally, defining hazardous areas involves manually plotting points / lines to draw the alarm zone, and these drawn areas cannot establish a relationship with the actual distance. This invention employs an automatic calibration mechanism based on camera parameters, automatically generating blind spot areas through a homography matrix and mapping lookup table, which correspond to the actual physical distance.

[0048] Furthermore, no calibration cloth needs to be laid before calibration; a combination of factory pre-calibration and installation fine-tuning is used. The camera's parameters are calculated at the factory using a checkerboard template, including the generation of the intrinsic parameter matrix, distortion parameters, and fisheye lookup table. The formula is as follows: Let the world coordinate point be Pw(Xw, Yw, Zw), and the camera coordinate point be Pc(Xc, Yc, Zc). The following conditions must be met: P c = RP w + T Where R is the rotation matrix and T is the translation matrix.

[0049] This invention establishes a homography matrix by aligning the intersection of the ground and the vertical plane on one side of the vehicle body as a reference line with the corresponding reference line in the camera image. Before leaving the factory, mapping data at different installation heights are collected, and a checkerboard pattern is used to establish the transformation relationship between image coordinates and world coordinates. During installation, the camera angle needs to be adjusted so that the vehicle body reference line in the image is aligned with the vehicle body reference line (the vertical line between the side of the vehicle body and the ground). The formula is as follows: using the homography matrix, a coordinate mapping table is generated from the distortion correction image to the top view. The image perspective is transformed through this coordinate mapping table.

[0050] Calibration process: 1. Prepare a template plane 2. Take several template images from different angles. 3. Automatically extract feature point coordinates from template image 4. Calculate camera parameters using mathematical models. 5. Calculate and optimize distortion parameters. 6. Generate a fisheye image conversion lookup table The calibration method is characterized by assuming a point Pw(Xw, Yw, Zw) in the world coordinate system, which is represented as Pc(Xc, Yc, Zc) in the camera coordinate system, and the relationship between the two is as follows:

[0051] R is a 3x3 rotation matrix; T is a 3x1 translation matrix; This invention establishes a homography matrix by aligning the intersection of the ground and the vertical plane on one side of the vehicle body as a reference line with the corresponding reference line in the camera image. Before leaving the factory, mapping data at different installation heights are collected, and a checkerboard pattern is used to establish the transformation relationship between image coordinates and world coordinates. During installation, the camera angle needs to be adjusted so that the vehicle body reference line in the image is aligned with the vehicle body reference line (the vertical line between the side of the vehicle body and the ground). The formula is as follows: using the homography matrix, a coordinate mapping table is generated from the distortion correction image to the top view. The image perspective is transformed through this coordinate mapping table.

[0052]

[0053] Xd, Yd, Zd: Represent the three-dimensional vectors after a three-dimensional linear transformation. These coordinates are the positions of a point in three-dimensional space in the "transformed coordinate system," obtained by linearly mapping the original coordinates using the transformation matrix.

[0054] Xr, Yr, Zr: Represent the three-dimensional vectors before the transformation. These coordinates are the positions of a point in three-dimensional space in the "original coordinate system" and are the input objects for the transformation.

[0055] a ij : Represents the elements of a 3×3 linear transformation matrix. This 3×3 matrix is ​​the core carrier of three-dimensional linear transformations, realizing the mapping "Xr, Yr, Zr → Xd, Yd, Zd" through matrix multiplication. Its elements (Aij) determine the properties of the transformation (such as rotation, scaling, mirroring, etc.).

[0056]

[0057] Xr1, Yr1: Point coordinates in the image coordinate system.

[0058] Xd1, Yd1: Coordinates of the corresponding points in the target coordinate system.

[0059] a ij The elements of a 3×3 matrix contain geometric transformation parameters such as rotation, scaling, and translation.

[0060] The system uses the vertical line on one side of the vehicle as a reference to align the body lines in the image with the actual edge lines of the vehicle, achieving initial camera angle alignment. After factory fisheye distortion correction, the wide-angle BSD camera first establishes a mapping from the corrected image coordinates to world coordinates using a homography matrix; then, it generates a coordinate mapping table between the top view and the distortion-corrected image using the homography matrix. During the factory stage, the system collects data at different installation heights to establish lookup table relationships. During installation, only minor adjustments to the camera angle are needed to automatically generate the coordinates of the actual spatial area in the image, achieving rapid calibration and automatic adjustment.

[0061] Furthermore, the BSD calibration module, refer to Figure 4 As shown, calibration is divided into local calibration and remote calibration. The calibration module will require the installer to input information such as the camera type and installation location. After receiving the calibration command, the BSD calibration module will automatically generate blind zone alarm areas based on the 1-meter width of each alarm zone. At this time, the user or installer can also manually fine-tune the alarm areas based on the generated areas.

[0062] BSD local calibration can be performed using a mobile app. Download the O&M app and connect to the vehicle monitoring host via Wi-Fi.

[0063] The manual calibration steps on the local client are as follows: 1) Enter the camera type; 2) Enter the camera installation height; 3) Enter the first-level blind zone distance width; 4) Enter the second-level blind zone distance width; 5) Enter the third-level blind zone distance width; 6) Click Calibrate to automatically generate the blind zone range.

[0064] The manual calibration steps for the remote client are as follows: 1) Enter the camera type; 2) Enter the camera installation height; 3) Enter the first-level blind zone distance width; 4) Enter the second-level blind zone distance width; 5) Enter the third-level blind zone distance width; 6) Click Calibrate; 7) The platform server sends the calibration information to the device; 8) The device generates and saves the blind zone mapping.

[0065] Display and video processing module: The image acquisition module, radar acquisition module and BSD alarm module are connected to transmit the monitoring video data / radar data to the BSD monitoring alarm module.

[0066] In a preferred embodiment, radar perception results are integrated, and the recognition accuracy is improved through information matching and confidence weighting. The judgment result is then submitted to the alarm decision module. Alarm prompts can be provided through various methods, including flashing external audible and visual alarms and voice alerts, in-vehicle voice alerts, and full-screen display of the target image in the blind spot.

[0067] The alarm information from the BSD alarm module is output to the audible and visual alarm, the display screen, and the device audio module. The audible and visual alarm module, or audible and visual alarm device, is used to broadcast audio and visual alerts to pedestrians. The audible and visual alarm in the alarm output module integrates a high-brightness red LED light source and an embedded speaker, and has IP67 protection rating, supporting all-weather outdoor applications. When a target enters the blind spot and triggers the alarm conditions, the LED flashes and plays the voice message "Please stay away from vehicles," promptly reminding pedestrians outside the vehicle.

[0068] The GPS positioning module is used to acquire real-time vehicle location information and send the information to the display. Simultaneously, the data is transmitted via a 4G network module to a cloud server for storage and processing. It is used to monitor the vehicle's driving trajectory and location.

[0069] The storage module supports SD and HDD storage of surveillance recordings.

[0070] The communication module is divided into wired communication module and wireless communication module, depending on whether a connection cable is used. The wired communication module communicates with the host and the display via a connection cable, while the wireless communication module connects the host, remote client, cloud server, and mobile application terminal (Operation and Maintenance Tool). The wireless communication module communicates via 4G and Wi-Fi modules.

[0071] The main control module and host information processing module are used to manage the routine work coordination of the host and communicate with remote terminals.

[0072] Remote Client: The remote client includes Android mobile phone, iOS mobile phone, Pad tablet, web client, and PC client. It is used to communicate with the cloud server, send information service requests, and receive relevant service information. Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features of the present invention can be arbitrarily combined with each other.

Claims

1. A BSD vehicle blind spot monitoring system based on video radar fusion, characterized in that, include: Vehicle-mounted monitoring host terminal; The vehicle-mounted monitoring host includes: The image acquisition module is used to acquire video images of the vehicle's external blind spots; The radar acquisition module is used to collect target information within a preset range around the vehicle; The BSD installation and calibration module is used to automatically generate blind spot alarm areas that match the physical distance based on camera parameters and the mapping relationship between the image coordinate system and the world coordinate system. The BSD alarm module is used to fuse and process the detection data from the image acquisition module and the radar acquisition module, and generate an alarm decision based on the target's location information within the blind zone alarm area. And an audible and visual alarm module, used to execute the alarm decision and output audible and visual alarm signals.

2. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 1, characterized in that, The image acquisition module includes at least one wide-angle camera for capturing images of the vehicle's external blind spots.

3. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 1, characterized in that, The radar acquisition module uses a 24GHz or 77GHz shortwave radar, which is installed on the outside of the vehicle to detect the distance, speed and direction of movement of targets within a preset range around the vehicle in real time.

4. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 1, characterized in that, The BSD alarm module divides the blind spot alarm area into at least a first-level alarm area, a second-level alarm area, and a third-level alarm area according to a preset distance from the vehicle, and generates alarm decisions for different levels accordingly.

5. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 4, characterized in that, The first-level alarm zone corresponds to the area within 1 meter of the vehicle, the second-level alarm zone corresponds to the area within 1 to 2 meters of the vehicle, and the third-level alarm zone corresponds to the area within 2 to 3 meters of the vehicle.

6. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 1, characterized in that, The BSD installation and calibration module generates a mapping lookup table from image coordinates to world coordinates through wide-angle image distortion correction, top-view transformation, and establishment of homography matrix, so as to realize the automatic generation of blind spot alarm area.

7. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 1 or 6, characterized in that, The BSD installation and calibration module supports both local and remote calibration modes. In the local calibration mode, parameters are configured and calibrated after a local connection is established between the mobile terminal application and the vehicle monitoring host. In the remote calibration mode, calibration is completed by sending calibration commands and parameters to the vehicle monitoring host through a cloud server.

8. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 1, characterized in that, The sound and light alarm module includes an LED light source and a voice speaker, which can trigger the LED of the corresponding color to flash and broadcast the corresponding warning voice according to the received alarm decision.

9. The BSD vehicle blind spot monitoring system based on video radar fusion according to claim 1, characterized in that, The system also includes a communication module for enabling wireless data communication between the vehicle-mounted monitoring host and remote clients and cloud servers.

10. A calibration method for a BSD vehicle blind spot monitoring system, based on the BSD vehicle blind spot monitoring system based on video radar fusion as described in any one of claims 1 to 9, characterized in that, Includes the following steps: Obtain the camera's internal parameters and distortion parameters; Based on the alignment relationship of the vehicle body reference lines in the image, a homography matrix between the image coordinate system and the world coordinate system is established. Based on the homography matrix, a coordinate mapping relationship from the image plane to the actual physical plane is generated; Based on the coordinate mapping relationship and the preset distance classification threshold, a blind spot alarm area matching the physical distance is automatically generated in the camera image.

Images