Heavy-duty car blind area early warning method, system and equipment based on PV-RCNN network and medium

Abstract

The invention discloses a PV-RCNN network-based dead zone early warning method, system and equipment for a heavy truck and a medium, mainly relates to the technical field of dead zone early warning, and aims to solve the problem that function accuracy is difficult to meet by using a single sensor in the existing scheme. Comprising the steps that when a preset tracking target is detected, the preset tracking target detected at the current moment and the preset tracking target detected at the last moment are compared, and a tracking target list is updated; detecting whether a preset tracking target involved in the tracking target list exists in a preset sight blind area or not, and when the preset tracking target exists in the preset sight blind area, performing information prompt on a driver; and detecting whether a preset tracking target existing in the preset sight blind area has a collision risk, and when the collision risk exists, performing preset early warning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of blind area early warning, in particular to a blind area early warning method, system, device and medium for heavy vehicles based on a PV-RCNN network. BACKGROUND

[0002] With the continuous development of social economy, the application scenarios of vehicles are expanding, and at the same time, road traffic safety is attracting increasing attention. In order to meet the increasingly stringent safety regulations, vehicles need to be equipped with higher standard active and passive safety systems. Blind area monitoring and early warning function as a key technology to improve driving safety can effectively reduce traffic accidents caused by driver's blind area. However, frequent false positives not only affect the driving experience, but also may weaken the driver's trust in the system. Therefore, there is an urgent need for a blind area monitoring solution with high precision, high efficiency and fast recognition capability to realize reliable and intelligent safety warning and effectively improve driving safety and user trust.

[0003] In the prior art, the blind area monitoring and early warning system lacks comprehensive monitoring capability under different complex conditions, and it is difficult to meet the accuracy of the function. The blind area monitoring and early warning system currently usually uses a single sensor, including using a camera to take a road image, and using a feature extraction and classifier to identify the target; using a radar sensor to detect the distance and contour of the target. SUMMARY

[0004] The present application provides a blind area early warning method, system, device and medium for heavy vehicles based on a PV-RCNN network to solve the problem that the existing solution uses a single sensor and cannot meet the accuracy of the function.

[0005] In a first aspect, the present application provides a blind area early warning method for heavy vehicles based on a PV-RCNN network, the method comprising: capturing road images and radar point cloud data through a camera and a radar installed at a preset position of a vehicle; using an improved PV-RCNN perception model to detect whether a preset tracking target exists in the capture area corresponding to the camera and the radar and to detect the basic information of the preset tracking target; when the preset tracking target is detected, comparing the preset tracking target detected at the current time and the last time, and updating a tracking target list; based on the basic information of the preset tracking target, detecting whether the preset tracking target involved in the tracking target list exists in a preset line-of-sight blind area, when it exists in the preset line-of-sight blind area, prompting the driver with information, and detecting whether the preset tracking target existing in the preset line-of-sight blind area has a collision risk, when it has a collision risk, performing a preset warning.

[0006] In an implementation manner of the present application, a preset tracking target and position information of the preset tracking target in a collection area corresponding to a camera and a radar are detected using an improved PV-RCNN perception model, and specifically includes: Converting the three-dimensional radar point cloud into a two-dimensional point cloud projection image; Establishing a mapping relationship between the radar point cloud and the point cloud projection image, and establishing a mapping relationship between the radar point cloud and the camera image; Mapping the image information of the camera image to the point cloud projection image through the radar point cloud as an intermediary; Processing the point cloud projection image using a lightweight and sparse method to obtain a front view image with sparsity; Obtaining pixel point cloud features from the front view image through a bilinear difference method; Extracting voxel point cloud features from the three-dimensional radar point cloud using an IDW-based linear interpolation method; Fusing the pixel point cloud features and the voxel point cloud features to obtain multi-modal fusion features; Interpolating the multi-modal fusion features of the point representation back to the voxels of the three-dimensional radar point cloud using an IDW-based linear interpolation method to obtain multi-modal fusion features of the voxel representation; Based on the multi-modal fusion features of the voxel representation, detecting whether a preset tracking target exists in a collection area corresponding to a camera and a radar, and outputting target existence and basic information of the preset tracking target; wherein the basic information at least includes: target category, target three-dimensional information, and target confidence.

[0007] In an implementation manner of the present application, when a preset tracking target is detected, the preset tracking target detected at the current time and the last time is compared, and a tracking target list is updated, and specifically includes: Comparing the preset tracking target detected at the current time and the last time; When a new preset tracking target appears, a preset tracking target ID corresponding to the preset tracking target is created; When the preset tracking target detected at the last time does not appear in the preset tracking target at the current time, the corresponding preset tracking target ID is deleted from the tracking target list.

[0008] In an implementation manner of the present application, whether the preset tracking target involved in the tracking target list exists in a preset line-of-sight blind area is detected, and when it exists in the preset line-of-sight blind area, information is prompted to the driver, and specifically includes: When the vehicle is driving straight, it is determined that the preset line-of-sight blind area is in front of the vehicle and on both sides of the vehicle body; When the vehicle is turning left, it is determined that the preset line-of-sight blind area is on the left side of the vehicle; When the vehicle is turning right, it is determined that the preset line-of-sight blind area is on the right side of the vehicle; When the preset tracking target exists in the preset line-of-sight blind area, information is prompted to the driver.

[0009] In an implementation manner of the present application, whether the preset tracking target existing in the preset line-of-sight blind area has a collision risk is detected, and when the collision risk exists, preset warning is performed, specifically including: According to the target three-dimensional information of the preset tracking target, the distance between the preset tracking target and the vehicle is calculated; According to the coordinate difference between the time frames, the speed of the preset tracking target is calculated; When the vehicle is straight and the distance between the preset tracking target and the vehicle is less than the preset safety distance, it is determined that the collision risk exists, preset warning is performed, and the speed of the preset tracking target is displayed; When the vehicle turns left or right, it is determined that the collision risk exists when entering the preset shadow area, and preset warning is performed.

[0010] In a second aspect, the present application provides a blind area warning system for heavy vehicles based on a PV-RCNN network, the system comprising: A collection module for collecting road images and radar point cloud data through a camera and a radar installed at a preset position of the vehicle; A detection module for detecting whether a preset tracking target exists in a collection area corresponding to the camera and the radar and basic information of the preset tracking target using an improved PV-RCNN perception model; An updating module for comparing the preset tracking target detected at the current time and the last time when the preset tracking target is detected, and updating a tracking target list; A warning module for detecting whether the preset tracking target involved in the tracking target list exists in the preset line-of-sight blind area based on the basic information of the preset tracking target, and when the preset tracking target exists in the preset line-of-sight blind area, information is prompted to the driver; detecting whether the preset tracking target existing in the preset line-of-sight blind area has a collision risk, and when the collision risk exists, preset warning is performed.

[0011] In an implementation manner of the present application, the detection module comprises a detection unit, for converting three-dimensional radar point cloud into two-dimensional point cloud projection image; establishing a mapping relationship between the radar point cloud and the point cloud projection image, and establishing a mapping relationship between the radar point cloud and the camera image; mapping the image information of the camera image to the point cloud projection image through the radar point cloud as an intermediary; adopting a light sparse method to process the point cloud projection image to obtain a front view image with sparsity; obtaining pixel point cloud features from the front view image through a bilinear difference method; The voxel point cloud feature is extracted from the three-dimensional radar point cloud using an IDW-based linear interpolation method; The pixel point cloud feature and the voxel point cloud feature are fused to obtain a multi-modal fusion feature; The multi-modal fusion feature of the point representation is interpolated back to the three-dimensional radar point cloud voxel using an IDW-based linear interpolation method to obtain a multi-modal fusion feature of the voxel representation. Based on the multi-modal fusion feature of the voxel representation, it is detected whether the preset tracking target exists in the collection area corresponding to the camera and the radar, and the target existence and the basic information of the preset tracking target are output; wherein the basic information at least includes: target category, target three-dimensional information and target confidence.

[0012] In an implementation manner of the present application, the updating module includes an updating unit, for comparing the preset tracking targets detected at the current moment and the last moment; When a new preset tracking target appears, a preset tracking target ID corresponding to the preset tracking target is created; When the preset tracking target detected at the last moment does not appear in the preset tracking target at the current moment, the corresponding preset tracking target ID is deleted from the tracking target list.

[0013] In a third aspect, the present application provides a heavy truck blind area early warning device based on a PV-RCNN network, the device comprising: a processor; and a memory having executable code stored thereon, when the executable code is executed, the processor executes any one of the above methods for early warning of the blind area of the heavy truck based on the PV-RCNN network.

[0014] In a fourth aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, when the computer instructions are executed, a method for early warning of the blind area of the heavy truck based on the PV-RCNN network is implemented.

[0015] From the above technical solutions, the present application has the following advantages: The camera and radar are used to collect road images and radar point cloud data, and a multi-modal data fusion perception is realized by combining an improved PV-RCNN perception model, which breaks through the limitations of a single sensor in the prior art and improves the accuracy and comprehensiveness of blind area target detection under complex working conditions. Compared with the perception scheme relying only on the camera, the method acquires target three-dimensional spatial information (distance, contour, three-dimensional coordinates) by means of radar point cloud data, avoids the defect that two-dimensional images cannot accurately locate the spatial position of the target, and can accurately judge whether the target is in the blind area range. Compared with the perception scheme relying only on the radar, the method realizes accurate classification (pedestrians, bicycles, motorcycles, etc.) of the target through camera image data, solving the problem of insufficient classification accuracy of the radar which can only determine the target category by contour. At the same time, the improved PV-RCNN model uses light sparse processing at the input end to filter irrelevant image information, reduce data volume and improve running efficiency, and realizes accurate alignment and fusion calculation of voxel features and pixel features through multi-modal feature deep fusion technology, so that it can adapt to complex scenes such as target occlusion, light intensity change and large scale difference, effectively reducing the common misjudgment and missed detection phenomena in the prior art, ensuring stable and accurate detection of blind area targets under various driving conditions, and meeting the high-precision requirements of blind area monitoring on perception algorithms.

[0016] Through the target tracking mechanism and the dynamic hierarchical warning strategy, the reliability and practicality of blind area warning are further improved, and the problem of inaccurate and frequent false alarm in the prior art is effectively solved. In the target tracking link, the method assigns a dedicated tracking ID to the detected main target, updates the tracking target list combined with the frame-by-frame detection result, can continuously track the motion state of the target, integrates the three-dimensional information of the target, the distance information and the speed information of the vehicle, and ensures real-time grasp of the target dynamics, avoiding monitoring interruption or misjudgment caused by target motion uncertainty. In the warning link, the method dynamically sets the blind area monitoring range according to the vehicle driving scene (straight, left turn, right turn), and implements hierarchical warning according to the risk characteristics in different scenes - information prompt when the target enters the blind area, and secondary warning when there is a collision risk, which ensures the timeliness of risk warning and avoids the interference of excessive warning on driving experience. In addition, the improved PV-RCNN model has good scene adaptability, and when the application scene is changed, it does not need to re-adjust the hand-designed features and classifier parameters, but only needs to supplement the new scene data set for training to adapt to the needs of different vehicle models and different driving scenes, greatly improving the universality and practicality of the technical scheme. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the present application, the drawings required to be used in the description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative labor on the premise of the drawings are within the scope of the present application.

[0018] Figure 1 is a blind area early warning method flowchart of a heavy truck based on a PV-RCNN network provided by an embodiment of the present application.

[0019] Figure 2 is a multi-modal deep feature fusion flowchart of a heavy truck based on a PV-RCNN network provided by an embodiment of the present application.

[0020] Figure 3 is a blind area monitoring diagram of a heavy truck when the vehicle is straight driving based on a PV-RCNN network provided by an embodiment of the present application.

[0021] Figure 4 is a blind area monitoring range diagram of a heavy truck when the vehicle is turning left based on a PV-RCNN network provided by an embodiment of the present application.

[0022] Figure 5 is a blind area monitoring range diagram of a heavy truck when the vehicle is turning right based on a PV-RCNN network provided by an embodiment of the present application.

[0023] Figure 6 is an internal structure schematic diagram of a blind area early warning system of a heavy truck based on a PV-RCNN network provided by an embodiment of the present application.

[0024] Figure 7 is an internal structure schematic diagram of a blind area early warning device of a heavy truck based on a PV-RCNN network provided by an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of the present application.

[0026] Those skilled in the art should understand that the embodiments described below are only preferred embodiments of the present disclosure, and do not represent that the present disclosure can only be implemented by the preferred embodiments, and the preferred embodiments are only used to explain the technical principles of the present disclosure, and not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the protection scope of the present disclosure.

[0027] It should also be noted that the terms "comprising", "containing" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0028] The technical solutions of the embodiments of the present application will be described in detail below with reference to the drawings.

[0029] The embodiments provide a blind area warning method for heavy trucks based on a PV-RCNN network, as shown in Figure 1 The method provided by the embodiments of the present application mainly includes the following steps: Step 110, collecting road images and radar point cloud data through the cameras and radars installed at the preset positions of the vehicle.

[0030] In some embodiments, cameras and radars are respectively installed in front of and on the side of the vehicle, and calibration is completed to ensure that the installation positions of the radars and cameras meet the requirements. The vehicle is driven to travel on different road sections, and the road images and radar point cloud data during the vehicle travel are collected and sent to the PV-RCNN perception model.

[0031] Step 120, using the improved PV-RCNN perception model to detect whether there is a preset tracking target in the collection area corresponding to the camera and the radar and the basic information of the preset tracking target.

[0032] As an example, as shown in Figure 2 The present step can be specifically: Converting the three-dimensional radar point cloud into a two-dimensional point cloud projection image; Establishing a mapping relationship between the radar point cloud and the point cloud projection image, and establishing a mapping relationship between the radar point cloud and the camera image; Mapping the image information of the camera image to the point cloud projection image through the radar point cloud as an intermediary; A lightweight sparsification method is used to process the point cloud projection map to obtain a sparse front view image; Pixel point cloud features are obtained from the front view image using the bilinear interpolation method. Voxel point cloud features are extracted from the voxel features of 3D radar point clouds using an IDW-based linear interpolation method. Feature fusion is performed on pixel point cloud features and voxel point cloud features to obtain multimodal fused features; The multimodal fusion features of point representation are interpolated back onto the voxels of the 3D radar point cloud using an IDW-based linear interpolation method to obtain the multimodal fusion features of voxel representation. Based on the multimodal fusion features of voxel representation, the system detects whether a preset tracking target exists in the acquisition area corresponding to the camera and radar, and outputs the target existence and basic information of the preset tracking target; among which, the basic information includes at least: target category, target 3D information, and target confidence.

[0033] It should be noted that the improved PV-RCNN algorithm uses an IDW-based linear interpolation method, which employs an inverse distance weighting approach to weight the predicted keypoint values. and key point center prediction values Interpolate onto the original design point cloud to obtain the full-resolution original point cloud weights and the original point cloud center offset, using the original point cloud as the basis. The first in Points For example, the propagation process is as follows: , In the formula, Index for key points; For the first Weight prediction for each key point ; It is the sigmoid function; The first in the original point cloud Weight prediction for each point ; For the first Predicted features of key points and the first key point in the original point cloud Interpolation weights between points.

[0034] , In the formula, For the first Center offset prediction for key points ; The first in the original point cloud Center offset prediction for each point, .

[0035] The training process of the improved PV-RCNN perception model can be specifically as follows: Data preparation phase: The training process begins with the construction of a high-quality dataset containing RGB images captured by cameras and three-dimensional point cloud data obtained by radars, both of which need to be accurately synchronized. Data labeling covers the categories of preset tracking targets (e.g., such as pedestrians, bicycles, and motorcycles), three-dimensional positions, and confidence levels, ensuring authenticity and diversity. To enhance the robustness of the model, data preprocessing includes image normalization, point cloud denoising, and coordinate alignment to eliminate sensor differences. In addition, data augmentation techniques such as image rotation and point cloud perturbation are used to expand the sample coverage and prevent overfitting.

[0036] Model training phase: The training adopts an end-to-end strategy, and the loss function combines classification loss, regression loss, and feature consistency loss to optimize multi-modal feature fusion. In the initial stage, the model learns to process camera images and radar point clouds independently, extracting features through bilinear interpolation and IDW methods. Subsequently, in the feature fusion layer, the model learns to integrate pixel point cloud features and voxel point cloud features to generate a unified multi-modal representation. The training uses the Adam optimizer to dynamically adjust the learning rate, ensuring stable convergence. The key lies in the application of the IDW interpolation method, which predicts the key point weight and center offset through inverse distance weighting, improving feature resolution. For example, the original point cloud weight is smoothed by a sigmoid function to avoid numerical instability.

[0037] Optimization verification phase: After training, the model is evaluated on an independent validation set, with indicators including target detection accuracy, recall rate, and three-dimensional positioning error. For problem areas such as insufficient feature fusion, transfer learning is used to fine-tune the model, introducing an attention mechanism to strengthen key features. At the same time, cross-validation is used to optimize hyperparameters such as interpolation weights and training rounds, balancing computational efficiency and detection accuracy. Finally, the model is tested in real-world scenarios before deployment to verify its generalization ability in complex environments and ensure the reliability of the output target existence, category, and three-dimensional information. Step 130, when the preset tracking target is detected, the preset tracking target detected at the current time and the last time is compared, and the tracking target list is updated.

[0038] It should be noted that the tracking target list is composed of preset tracking target IDs.

[0039] As an example, this step can be specifically as follows: Comparing the preset tracking target detected at the current time and the last time; When a new preset tracking target appears, a preset tracking target ID corresponding to the preset tracking target is created; When the preset tracking target detected at the last time does not appear in the preset tracking target at the current time, the corresponding preset tracking target ID is deleted from the tracking target list.

[0040] It should be noted that after detecting the road main target, a specific preset tracking target ID is assigned as the tracking target of the current frame. The tracking target is managed by adding or deleting tracking ID according to the detection results of subsequent frames. The target information detected by the perception model is integrated, and finally the integrated tracking target information is output to the subsequent functional module.

[0041] Step 140, based on the basic information of the preset tracking target, detecting whether the preset tracking target involved in the tracking target list exists in the preset line-of-sight blind area, when it exists in the preset line-of-sight blind area, information prompt is given to the driver; detecting whether the preset tracking target existing in the preset line-of-sight blind area has a collision risk, when there is a collision risk, preset warning is carried out.

[0042] In some embodiments, detecting whether the preset tracking target involved in the tracking target list exists in the preset line-of-sight blind area, when it exists in the preset line-of-sight blind area, information prompt is given to the driver, specifically including: When the vehicle is straight ahead, it is determined that the preset line-of-sight blind area is in front of the vehicle and on both sides of the vehicle body (as shown in Figure 3 When the vehicle turns left, it is determined that the preset line-of-sight blind area is on the left side of the vehicle; When the vehicle turns right, it is determined that the preset line-of-sight blind area is on the right side of the vehicle; When the vehicle turns right, it is determined that the preset line-of-sight blind area is on the right side of the vehicle; When there is a preset tracking target in the preset line-of-sight blind area, information prompt is given to the driver.

[0043] In some embodiments, detecting whether the preset tracking target existing in the preset line-of-sight blind area has a collision risk, when there is a collision risk, preset warning is carried out, specifically including: According to the target three-dimensional information of the preset tracking target, the distance between the preset tracking target and the vehicle is calculated; According to the coordinate difference between the time frames, the speed of the preset tracking target is calculated; When the vehicle is straight ahead and the distance between the preset tracking target and the vehicle is less than the preset safety distance, it is determined that there is a collision risk, preset warning is carried out, and the speed of the preset tracking target is displayed; When the vehicle turns left or right, it is determined that entering the preset shadow area causes a collision risk (as shown in Figure 4 The left turning preset shadow area is shown in Figure 5 The right turning preset shadow area is shown in

[0044] In addition, the present application Figure 6A blind area early warning system for heavy trucks based on a PV-RCNN network is provided for an embodiment of the present application. As shown in Figure 6 The system provided by the embodiment of the present application mainly comprises: The acquisition module 210 is configured to acquire road images and radar point cloud data through a camera and a radar installed at a preset position of a vehicle.

[0045] The detection module 220 is configured to detect whether a preset tracking target exists in a collection area corresponding to the camera and the radar and to obtain basic information of the preset tracking target by using an improved PV-RCNN perception model.

[0046] The detection module 220 comprises a detection unit, configured to convert three-dimensional radar point clouds into two-dimensional point cloud projection images; establish a mapping relationship between the radar point clouds and the point cloud projection images, and establish a mapping relationship between the radar point clouds and the camera images; intermediate the radar point clouds, and map image information of the camera images to the point cloud projection images; adopt a light and sparse method to process the point cloud projection images to obtain a sparse front view image; obtain pixel point cloud features from the front view image by using a bilinear difference method; extract voxel point cloud features from the three-dimensional radar point clouds by using an IDW-based linear interpolation method; perform feature fusion on the pixel point cloud features and the voxel point cloud features to obtain multi-modal fusion features; interpolate the multi-modal fusion features of the point representation back to the voxels of the three-dimensional radar point clouds by using the IDW-based linear interpolation method to obtain multi-modal fusion features of the voxel representation; detect whether the preset tracking target exists in the collection area corresponding to the camera and the radar based on the multi-modal fusion features of the voxel representation, and output target existence and basic information of the preset tracking target; wherein the basic information at least includes target category, target three-dimensional information, and target confidence.

[0047] The update module 230 is configured to compare the preset tracking target detected at a current time and the preset tracking target detected at a previous time, and update a tracking target list when the preset tracking target is detected; wherein the tracking target list is composed of preset tracking target IDs.

[0048] The update module 230 comprises an update unit, configured to compare the preset tracking target detected at the current time and the preset tracking target detected at the previous time; create a preset tracking target ID corresponding to the preset tracking target when a new preset tracking target appears; When the preset tracking target detected at the last moment does not appear in the preset tracking target at the current moment, the corresponding preset tracking target ID is deleted from the tracking target list.

[0049] The early warning module 240 is configured to detect whether the preset tracking target involved in the tracking target list exists in the preset line-of-sight blind area, and when the preset tracking target exists in the preset line-of-sight blind area, information is prompted to the driver; and detect whether the preset tracking target existing in the preset line-of-sight blind area has a collision risk, and when the preset tracking target has a collision risk, preset early warning is performed.

[0050] The above is a method embodiment in the present application, based on the same inventive concept, the present application embodiment also provides a heavy truck blind area early warning device based on a PV-RCNN network. As shown in the figure, Figure 7 The device includes a processor and a memory having executable code stored thereon, which, when executed, causes the processor to perform a heavy truck blind area early warning method based on a PV-RCNN network as described in one of the above embodiments.

[0051] Specifically, the server end collects road images and radar point cloud data through the cameras and radars installed at the preset positions of the vehicles; uses an improved PV-RCNN perception model to detect whether there is a preset tracking target and the basic information of the preset tracking target in the collection area corresponding to the cameras and radars; when the preset tracking target is detected, the preset tracking target detected at the current moment and the last moment is compared, and the tracking target list is updated; based on the basic information of the preset tracking target, it is detected whether the preset tracking target involved in the tracking target list exists in the preset line-of-sight blind area, and when the preset tracking target exists in the preset line-of-sight blind area, information is prompted to the driver; and detect whether the preset tracking target existing in the preset line-of-sight blind area has a collision risk, and when the preset tracking target has a collision risk, preset early warning is performed.

[0052] In addition, the present application embodiment also provides a non-volatile computer storage medium having executable instructions stored thereon, which, when executed, implement a heavy truck blind area early warning method based on a PV-RCNN network as described above.

[0053] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A blind spot warning method for heavy-duty vehicles based on PV-RCNN network, characterized in that, The method includes: The system collects road images and radar point cloud data by using cameras and radar installed at preset locations on the vehicle. Using an improved PV-RCNN perception model, the system detects whether the preset tracking target and its basic information exist in the acquisition areas corresponding to the camera and radar. When a preset tracking target is detected, the preset tracking target detected at the current time is compared with the one detected at the previous time, and the tracking target list is updated. Based on the basic information of the preset tracking targets, the system detects whether the preset tracking targets in the target list are within the preset blind spots. If they are within the preset blind spots, the system provides a notification to the driver. The system also detects whether there is a collision risk with the preset tracking targets within the preset blind spots. If there is a collision risk, the system issues a preset warning.

2. The blind spot warning method for heavy-duty vehicles based on PV-RCNN network according to claim 1, characterized in that, Using an improved PV-RCNN perception model, the system detects whether a pre-defined tracking target exists and its location information within the acquisition areas corresponding to the camera and radar. Specifically, this includes: Convert the 3D radar point cloud into a 2D point cloud projection map; Establish the mapping relationship between radar point clouds and point cloud projection maps, as well as the mapping relationship between radar point clouds and camera images; Using radar point clouds as an intermediary, image information from camera images is mapped onto point cloud projection maps; A lightweight sparsification method is used to process the point cloud projection map to obtain a sparse front view image; Pixel point cloud features are obtained from the front view image using the bilinear interpolation method. Voxel point cloud features are extracted from 3D radar point clouds using an IDW-based linear interpolation method. Feature fusion is performed on pixel point cloud features and voxel point cloud features to obtain multimodal fused features; The multimodal fusion features of point representation are interpolated back onto the voxels of the 3D radar point cloud using an IDW-based linear interpolation method to obtain the multimodal fusion features of voxel representation. Based on the multimodal fusion features of voxel representation, the system detects whether a preset tracking target exists in the acquisition area corresponding to the camera and radar, and outputs the target existence and basic information of the preset tracking target; among which, the basic information includes at least: target category, target 3D information, and target confidence.

3. The blind spot warning method for heavy-duty vehicles based on PV-RCNN network according to claim 1, characterized in that, When a preset tracking target is detected, the preset tracking target list is updated by comparing the current time with the target detected in the previous time step, specifically including: Compare the preset tracking targets detected at the current time with those detected at the previous time; When a new preset tracking target is added, a preset tracking target ID corresponding to the preset tracking target is created; If the preset tracking target detected in the previous moment does not appear in the preset tracking targets in the current moment, the corresponding preset tracking target ID is deleted from the tracking target list.

4. The blind spot warning method for heavy-duty vehicles based on PV-RCNN network according to claim 1, characterized in that, The system detects whether any of the preset tracking targets in the target tracking list are within a preset blind spot. If a target is found within the preset blind spot, the driver is alerted, specifically by: When the vehicle is traveling straight, determine that the preset blind spots are in front of the vehicle and on both sides of the vehicle. When the vehicle turns left, ensure that the preset blind spot is on the left side of the vehicle; When the vehicle turns right, ensure that the preset blind spot is on the right side of the vehicle; When a preset target is located within a preset blind spot, the driver will be alerted.

5. The blind spot warning method for heavy-duty vehicles based on PV-RCNN network according to claim 1, characterized in that, Detect whether there is a collision risk with a preset tracking target located within a preset blind spot. If a collision risk is detected, a preset warning is issued, including: Calculate the distance between the preset tracking target and the vehicle based on the target's three-dimensional information. Calculate the velocity of the preset tracking target based on the coordinate difference between time frames; When the vehicle is traveling straight and the distance between the preset tracking target and the vehicle is less than the preset safe distance, a collision risk is determined, a preset warning is issued, and the speed of the preset tracking target is displayed; When a vehicle turns left or right, if a collision risk is detected when it enters a preset shaded area, a preset warning will be issued.

6. A blind spot warning system for heavy-duty vehicles based on a PV-RCNN network, characterized in that, The system includes: The data acquisition module is used to acquire road images and radar point cloud data through cameras and radar installed at preset locations on the vehicle. The detection module is used to detect whether the preset tracking target and the basic information of the preset tracking target exist in the acquisition area corresponding to the camera and radar using an improved PV-RCNN perception model. The update module is used to compare the preset tracking targets detected at the current time with those detected at the previous time when a preset tracking target is detected, and update the tracking target list accordingly. The warning module is used to detect whether the preset tracking targets in the target list are within the preset blind spot based on the basic information of the preset tracking targets. When they are within the preset blind spot, the driver is alerted. The module also detects whether there is a collision risk for the preset tracking targets within the preset blind spot. When there is a collision risk, a preset warning is issued.

7. The blind spot warning system for heavy-duty vehicles based on PV-RCNN network according to claim 6, characterized in that, The detection module includes a detection unit. Used to convert 3D radar point clouds into 2D point cloud projection maps; Establish the mapping relationship between radar point clouds and point cloud projection maps, as well as the mapping relationship between radar point clouds and camera images; Using radar point clouds as an intermediary, image information from camera images is mapped onto point cloud projection maps; A lightweight sparsification method is used to process the point cloud projection map to obtain a sparse front view image; Pixel point cloud features are obtained from the front view image using the bilinear interpolation method. Voxel point cloud features are extracted from 3D radar point clouds using an IDW-based linear interpolation method. Feature fusion is performed on pixel point cloud features and voxel point cloud features to obtain multimodal fused features; The multimodal fusion features of point representation are interpolated back onto the voxels of the 3D radar point cloud using an IDW-based linear interpolation method to obtain the multimodal fusion features of voxel representation. Based on the multimodal fusion features of voxel representation, the system detects whether a preset tracking target exists in the acquisition area corresponding to the camera and radar, and outputs the target existence and basic information of the preset tracking target; among which, the basic information includes at least: target category, target 3D information, and target confidence.

8. The blind spot warning system for heavy-duty vehicles based on PV-RCNN network according to claim 6, characterized in that, The update module includes update units. Used to compare the preset tracking targets detected at the current time with those detected at the previous time; When a new preset tracking target is added, a preset tracking target ID corresponding to the preset tracking target is created; If the preset tracking target detected in the previous moment does not appear in the preset tracking targets in the current moment, the corresponding preset tracking target ID is deleted from the tracking target list.

9. A blind spot warning device for heavy-duty vehicles based on a PV-RCNN network, characterized in that, The device includes: processor; And a memory having executable code stored thereon, which, when executed, causes the processor to perform a blind spot warning method for heavy vehicles based on a PV-RCNN network as described in any one of claims 1-5.

10. A non-volatile computer storage medium, characterized in that, It stores computer instructions, which, when executed, implement a blind spot warning method for heavy vehicles based on a PV-RCNN network as described in any one of claims 1-5.

Images