BSD blind area monitoring method and system based on visual image and storage medium

Abstract

The invention relates to the field of vehicle blind area monitoring, in particular to a BSD blind area monitoring method and system based on a visual image and a storage medium. Comprising the following steps: acquiring a driving position side rear view range corresponding to the driving position man-machine engineering parameters; obtaining a side rear view range of the current driving position; acquiring a side rear image, and acquiring a blind area range of a driving position; acquiring a moving parameter of the moving target; obtaining pre-collision information, wherein the pre-collision information comprises a collision triggering condition; acquiring a driving state of the vehicle; and when the driving state of the vehicle meets the collision triggering condition, outputting early warning information. According to the method and the system, the actual driving position blind area visual field range is accurately obtained according to the man-machine engineering parameters of the driving position, the blind area monitoring range is limited in the accurate and reasonable range, false alarm is avoided, output is judged according to the driving state of the vehicle when the early warning information is output, invalid prompt information is reduced, and the driving safety is improved. And the influence on the normal driving of the vehicle is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle blind area monitoring, in particular to a BSD blind area monitoring method based on visual image, a system and a storage medium. BACKGROUND

[0002] When driving a vehicle, the driver is located in the driving position of the vehicle and observes the external environment of the vehicle through the windows, rearview mirrors and the like. Due to the limitations of the structure around the driving position, the positions on the sides of the vehicle and behind the vehicle are in the visual blind area. The traffic conditions in the blind area cannot be reflected in the driver's line of sight through the outside rearview mirrors and the inside rearview mirror, and collisions are likely to occur when the vehicle is operated to change direction or open / close the door.

[0003] In order to solve the problem of the blind area, some vehicles are equipped with a blind area monitoring function to monitor the blind area and give some reminders. The existing blind area monitoring generally includes a radar scheme and a visual scheme. The radar scheme is to install a radar on the vehicle to monitor the distance of objects in the blind area, and when a vehicle enters the blind area, a light reminder is given near the corresponding side of the rearview mirror. The visual scheme is to collect the pictures on both sides of the vehicle through the cameras located on both sides of the vehicle, and after processing, sound, light or picture reminders are given. Compared with the radar scheme, the visual scheme is more intuitive. A blind area detection accuracy determination method for a BSD system and related equipment are disclosed in Chinese patent CN116027285A. The method combines pre-crash time to achieve early warning through blind area picture acquisition, thereby improving the accuracy of pre-judgment. However, the blind area collision warning in the existing scheme only calculates the pre-crash of the speed and trajectory of the target in the external environment, and does not actually consider the situation of the driver in the vehicle. Different body types and driving postures of the driver have different blind areas for the vehicle. In fact, effective blind area reminders need to set the blind area to the maximum boundary to adapt to all drivers. Therefore, for different individual drivers, some blind area reminders are unnecessary, which will affect the attention of the driver and make him often in a tense driving state, thereby affecting the safety of vehicle driving. SUMMARY

[0004] In a first aspect, the embodiments of the present application provide a BSD blind area monitoring method based on visual image to solve the problem of insufficient matching degree of the existing blind area monitoring system for the driver.

[0005] The method comprises:

[0006] According to the driver-seat ergonomics parameters, a driver-seat side rear field of view range corresponding to the driver-seat ergonomics parameters is obtained;

[0007] According to the current driver-seat ergonomics parameters, a current driver-seat side rear field of view range is obtained;

[0008] Obtaining a side rear image, in which a driving position side rear view range is reversely calibrated, and obtaining a driving position blind area range;

[0009] Capturing a moving target in the driving position blind area range, and obtaining a moving parameter of the moving target;

[0010] Performing pre-collision time calculation according to the moving parameter, and obtaining pre-collision information, wherein the pre-collision information includes a collision trigger condition;

[0011] Obtaining a driving state of the vehicle, wherein the vehicle driving state includes a vehicle speed and a vehicle operation;

[0012] When the driving state of the vehicle meets the collision trigger condition, outputting a pre-warning information.

[0013] Thanks to the above method, the blind area monitoring method of the embodiment can first accurately obtain an actual driving position blind area view range according to the man-machine engineering parameters of the driving position, avoid capturing and calculating all moving targets in the side rear image, limit the blind area monitoring range to an accurate and reasonable range, avoid false positives, and reduce the algorithm requirement. Further, when outputting the pre-warning information, the driving state of the vehicle is combined to determine whether to output, reducing invalid prompt information and avoiding affecting the normal driving of the vehicle.

[0014] In a possible implementation, the obtaining of the driving position side rear view range corresponding to the man-machine engineering parameters of the driving position according to the man-machine engineering parameters of the driving position includes:

[0015] Obtaining a man-machine engineering sample, wherein the man-machine engineering sample includes the man-machine engineering parameters and driver body shape data;

[0016] Establishing a man-machine engineering calculation model, inputting the man-machine engineering sample into the man-machine engineering calculation model, and training the man-machine engineering calculation model.

[0017] In a possible implementation, the man-machine engineering parameters include a seat front-rear distance, a seat vertical height, a seat backrest angle, a steering wheel angle, and a rearview mirror angle.

[0018] In a possible implementation, the reading of the current man-machine engineering parameters of the driving position and the obtaining of the current driving position side rear view range include:

[0019] Loading the man-machine engineering calculation model after starting the vehicle;

[0020] Reading the current man-machine engineering parameters of the vehicle, inputting the man-machine engineering calculation model, and obtaining the current driver body shape data;

[0021] Based on the current driver's body shape data, obtain the driver's side and rear field of vision range.

[0022] In one possible implementation, the step of reverse-calibrating the driver's side rearward field of view range in the side rearward image includes:

[0023] Displays images of the vehicle's side and rear within the field of view of the camera;

[0024] The vehicle's rear-side image is delineated using the driver's side rear-side field of view.

[0025] The vehicle's side and rear view outside the driver's side rear view range is defined as the driver's side blind spot range.

[0026] Highlight the driver's blind spot area.

[0027] In one possible implementation, capturing a moving target within the driver's blind spot and obtaining the moving target's movement parameters includes:

[0028] The image within the driver's blind spot area in the side and rear view of the vehicle is preprocessed to obtain a preprocessed image;

[0029] The preprocessed image is separated into consecutive frame images;

[0030] The moving target is obtained from the consecutive frame images, and the movement parameters of the moving target are obtained.

[0031] In one possible implementation, the BSD blind spot prediction algorithm includes temporal and spatial information on the target type and the target's continuous trajectory within the BSD detection area;

[0032] The method of predicting the future trajectory of a target using the BSD blind spot prediction algorithm includes:

[0033] The target spatial location and boundary information are determined based on the spatial trajectory information of the BSD.

[0034] The target velocity and acceleration information are determined based on the continuous time information of the BSD.

[0035] Predicting the target's future trajectory based on the aforementioned spatial and temporal information.

[0036] In one possible implementation, the method of obtaining the collision risk warning level of the BSD blind spot monitoring system based on BSD blind spot monitoring information and vehicle driving status data includes:

[0037] Based on the spatial information of the trajectory prediction of the multiple targets and the temporal information of the trajectory prediction of the multiple targets, the collision detection weight coefficient and the multi-target TTC weight coefficient are obtained.

[0038] The target attribute weight coefficient, the trajectory prediction weight coefficient, and the TTC weight coefficient are used to determine the collision risk level of the BSD blind area monitoring system, and the TTC weight coefficient is greater than the trajectory prediction weight coefficient, and the trajectory prediction weight coefficient is greater than the attribute weight coefficient.

[0039] In a second aspect, the embodiments of the present application also provide a BSD blind area monitoring system based on visual images, which comprises:

[0040] An ergonomics processing module is configured to obtain a driving position side rear field of view corresponding to a driving position ergonomics parameter according to the driving position ergonomics parameter.

[0041] An ergonomics acquisition module is configured to obtain a current driving position ergonomics parameter and send the current driving position ergonomics parameter to the ergonomics processing module.

[0042] A video acquisition module is configured to obtain a side rear image of the vehicle.

[0043] A blind area acquisition module is configured to obtain a current driving position side rear field of view from the ergonomics processing module, obtain a side rear image of the vehicle from the video acquisition module, and obtain a driving position blind area range by inversely calibrating the driving position side rear field of view in the side rear image.

[0044] A blind area analysis module is configured to capture a moving target in the driving position blind area range and obtain a moving parameter of the moving target.

[0045] A pre-collision module is configured to calculate a pre-collision time according to the moving parameter and obtain pre-collision information, wherein the pre-collision information comprises a collision trigger condition.

[0046] A driving state monitoring module is configured to monitor a vehicle driving state, wherein the vehicle driving state comprises a vehicle speed and a vehicle operation.

[0047] A pre-warning module is configured to obtain the pre-collision information from the pre-collision module, obtain the vehicle driving state from the driving state monitoring module, and output a pre-warning information when the driving state of the vehicle meets the collision trigger condition.

[0048] In a third aspect, the embodiments of the present application provide a storage medium for storing an executable program for executing the BSD blind area monitoring method based on visual images of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 The figure is a general flowchart of the first embodiment of the present application.

[0050] Figure 2 The figure is a specific flowchart of step S2 of the first embodiment of the present application.

[0051] Figure 3 The specific flow chart for step S3 of the first embodiment of the present application is shown in FIG. 3.

[0052] Figure 4 The specific flow chart for step S4 of the first embodiment of the present application is shown in FIG. 4.

[0053] Figure 5 The schematic diagram of the module structure of the second embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0054] It should be apparent that the described embodiments are only some embodiments and not all embodiments. Based on the embodiments described below, all other embodiments obtained by those of ordinary skill in the art without creative effort are also within the scope of protection of the present application.

[0055] It should be understood that the controllers and control circuits involved in the embodiments are conventional control technology or units for those skilled in the art, and the control circuits of the controllers can be implemented by those of ordinary skill in the art using existing technology.

[0056] The disclosure of the embodiments provides many different implementations or examples for implementing different aspects of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described in the embodiments. Of course, they are only examples and the purpose is not to limit the present application. In addition, reference numerals and / or reference letters can be repeated in different examples in the embodiments. Such repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various implementations and / or settings being discussed.

[0057] First, the relationship between the existing blind area visual recognition and warning scheme on the vehicle and the state of the vehicle during driving and the warning information involved in the following embodiments is described in order to further understand the difference between the embodiments of the present application and the existing blind area visual recognition and warning scheme.

[0058] The existing visual recognition scheme is based on cameras installed on both sides of the vehicle to obtain side rear images on both sides of the vehicle. The cameras are generally installed below the rearview mirrors on both sides or in the area near the rearview mirrors to reflect the image that the human eye can observe from the rearview mirror. When a starting object, such as a vehicle in another lane, enters the side rear position of the vehicle, an indicator light installed on the outside rearview mirror or the A-pillar of the vehicle indicates that the vehicle should not perform a lane change operation at this time to avoid a collision. Some systems also correspondingly provide voice or prompt sound prompts. However, in fact, if the vehicle has no intention to change lanes, such early warning information is invalid early warning information. For example, existing vehicle "door kill" events occur frequently. When the driver or passenger opens the door, they need to pay special attention to vehicles or fast-moving targets such as electric vehicles and bicycles in the rear blind area, otherwise they may cause harm to people or vehicles outside the vehicle when opening the door. However, door opening operations generally occur when the vehicle is parked on the side of the road. In the case of a traffic jam, parking on the road generally does not occur. Therefore, during normal driving and parking, there is no condition for a door opening collision to occur. Therefore, early warning information in this state is also invalid early warning. With the increasing number of road participants, the above-mentioned early warning information often produces sound and light prompts during vehicle driving, affecting the attention of the driver and the quiet environment in the vehicle. Therefore, further judgment of the early warning information is needed on the basis of the existing blind area early warning. The vehicle is reminded or intervened when the vehicle operation meets the early warning condition. The intervention includes avoiding the condition through the control of the vehicle device, such as temporarily closing the door, etc.

[0059] In addition, the existing visual scheme for blind area monitoring actually relies on target elements in the camera shooting picture for identification and tracking. A large part of the targets are not in the blind area of the driver. Many potential hazards can be observed by the driver in the driving position through the rearview mirror and can be responded to. Identification and reminders are needed. Such redundant reminders further exacerbate the situation of frequent invalid early warning information affecting the driver, and the requirement for the computing power of the vehicle is also higher.

[0060] To solve the above-mentioned various types of invalid early warning situations, the first embodiment provides a BSD blind area monitoring method based on visual images.

[0061] As shown in Figure 1 The blind area monitoring method of the present embodiment includes the following steps:

[0062] S1. According to the driver ergonomics parameters, the driver side rear field of view range corresponding to the driver ergonomics parameters is obtained;

[0063] S2. According to the current driver ergonomics parameters, the current driver side rear field of view range is obtained;

[0064] S3. Obtain a side rear image, reverse calibrate the driving position side rear view range in the side rear image, and obtain a driving position blind area range;

[0065] S4. Capture a moving target in the driving position blind area range, and obtain a moving parameter of the moving target;

[0066] S5. Perform pre-collision time calculation according to the moving parameter, and obtain pre-collision information, the pre-collision information including a collision trigger condition;

[0067] S6. Obtain a driving state of the vehicle, the vehicle driving state including a vehicle speed and a vehicle operation;

[0068] S7. When the driving state of the vehicle meets the collision trigger condition, output a pre-warning information.

[0069] In step S1, the problem that the obtained rearview mirror field of view is different when different drivers adopt different driving postures is mainly solved. The driving position side rear view range corresponding to the driving position ergonomics parameter is obtained according to the driving position ergonomics parameter, and includes:

[0070] Obtain an ergonomics sample, the ergonomics sample including the ergonomics parameter and driver body shape data;

[0071] Establish an ergonomics calculation model, input the ergonomics sample into the ergonomics calculation model, and train the ergonomics calculation model.

[0072] The ergonomics calculation model can be stored in the form of a mapping table, or can be trained by using a neural network. When the mapping table is used, a calculation algorithm is used to map the data combination in the ergonomics sample and the corresponding field of view. The field of view is obtained by first associating the body shape characteristics of the corresponding driver, especially the position and height of the eyes, according to the parameters in the ergonomics sample. According to the position and height of the eyes, the driving position side rear view range can be obtained by combining the angle and curvature of the rearview mirror. The driving position side rear view range is added to the mapping table for storage and ready for calling.

[0073] When the ergonomics sample is trained by using a neural network, the ergonomics sample parameters are input into the neural network, and the driving position side rear view range is used as an output result to confirm the neural network, so that the ergonomics calculation model is trained to be able to obtain an accurate driving position side rear view range when the ergonomics sample parameters are input.

[0074] The ergonomic parameters in this embodiment include the distance between the seat and the front, the vertical height of the seat, the angle of the seat back, the angle of the steering wheel, and the angle of the rearview mirror. The size and curvature of the vehicle rearview mirror are constant for the same vehicle model, and can be combined as constants into the ergonomic parameters when training the neural network. When the neural network needs to be trained to adapt to a larger training model for different vehicles, the shape, size, and mirror curvature of the rearview mirror can also be trained and input as ergonomic parameters. In this embodiment, the vehicle is considered to be the same vehicle for clarity of explanation.

[0075] After the ergonomic calculation model is trained, the ergonomic calculation model is loaded in the vehicle control system when the vehicle starts, and then step S2 is performed. The current driver's side rear view range is obtained according to the current driver's seat ergonomic parameters. As shown in Figure 2 The specific process includes:

[0076] S21. The ergonomic calculation model is loaded after the vehicle starts.

[0077] S22. The current ergonomic parameters of the vehicle are read and input into the ergonomic calculation model to obtain the current driver's body shape data.

[0078] S23. The driver's side rear view range is obtained according to the current driver's body shape data.

[0079] At this time, step S3 can be performed to obtain the side rear image, and the driver's side blind area range is obtained by reverse calibration of the driver's side rear view range in the side rear image.

[0080] After obtaining the side rear image, the side rear image and the current driver's side rear view range exist in the vehicle control system, and at this time, the driver's side blind area range can be obtained by reverse calibration of the driver's side rear view range in the side rear image.

[0081] In theory, the areas outside the driver's side rear view range in the side rear influence are all the current driver's blind areas, but in fact, the positions far away from the vehicle do not affect the driving safety of the vehicle, and most side accidents occur in adjacent motor lanes or non-motor lanes. Therefore, by reverse calibration of the driver's side rear view range in the side rear image, the range outside the adjacent lanes can be removed, and the driver's blind area range that needs to be monitored and tracked can be further obtained.

[0082] Since the blind area is an area that cannot be seen by the driver through the rearview mirror, in order to intuitively show the driver the situation in the blind area range, as shown in Figure 3 The reverse calibration of the driver's side rear view range in the side rear image includes:

[0083] S31. Display the vehicle side rear image within the shooting field of view;

[0084] S32. Define the vehicle side rear image by the driver side rear field of view;

[0085] S33. The vehicle side rear image outside the driver side rear field of view as the driver blind area range;

[0086] S34. Highlight the driver blind area range.

[0087] After determining the driver blind area range, the blind area monitoring needs to capture and analyze the target in the image, so step S4. Capture the moving target in the driver blind area range, and obtain the moving parameter of the moving target.

[0088] As shown in Figure 4 , step S4, capturing the moving target in the driver blind area range and obtaining the moving parameter of the moving target includes:

[0089] S41. Preprocess the image in the driver blind area range in the vehicle side rear image, and obtain the preprocessed image;

[0090] S42. Separate the preprocessed image into continuous frame images;

[0091] S43. Get the moving target from the continuous frame image, and obtain the moving parameter of the moving target.

[0092] For example, in this embodiment, the deployment on HiSilicon embedded platform NPU (Neural Network Processing Unit) is used as an example to illustrate the deployment and execution. This part is the architecture and function of HiSilicon embedded platform itself, and the execution process is generally to perform visual intelligent algorithm first stage detection algorithm on the obtained video image, and get the target category and position information in the blind area range, wherein the video information is obtained by manually installing on both sides of the target vehicle; According to the depth of field of the camera and the distance information of the boundary of each target relative to the target vehicle determined by the pre human calibration, the image information of each target in the blind area for a period of time is obtained by the continuous video frame obtained by the camera, and the target attribute information is encoded and input into the subsequent Tracker (tracker) for second stage tracking algorithm processing.

[0093] For example, in this embodiment, the structure compatible optimization work of the intelligent algorithm for the first stage detection algorithm includes:

[0094] Insert the configuration operator of image preprocessing, related image operations include but are not limited to: image cropping, color gamut conversion, channel conversion, mean value reduction normalization; for the intelligent algorithm deployed on GPU, the pre-processing work of image can be completed by relying on NPU (Neural Network Processing Unit) performance, on the resource-limited embedded platform, the present scheme adopts AIPP (AI Preprocessing) operator for pre-processing part, improves the running efficiency of the whole visual algorithm and the system running stability;

[0095] The pre-processing process adopts the AIISP intelligent denoising algorithm of HiVision embedded platform, which is suitable for different image sensors for targeted image optimization. Through the intelligent denoising algorithm, it can make the image sensor remove noise more cleanly and retain more details at lower illumination, thereby improving the photosensitive ability of the image sensor at very low illumination, and enabling the visual intelligent algorithm to maintain good detection performance even in night low-illumination environment.

[0096] In some examples, wide-angle fisheye lens is adopted to solve the limitations of limited viewfinder range and narrow field of view of conventional lens, and to solve the problem of serious obstruction of road information acquisition in blind area, but it brings image optical distortion, which has certain influence on visual perception and intelligent algorithm processing; the distortion correction GDC (Geometry Distortion Correction) module of HiVision embedded platform is adopted. Through the calibration mapping correction method, a wider field of view is retained while eliminating the image distortion caused by the strong perspective effect of fisheye lens. The direct visual experience of the driver on the road condition is simultaneously improved.

[0097] In the target detection process, in order to adapt to the efficiency maximization of NPU, the supported YOLO series decoding mode is adopted for network decoding mode, aiming to improve the inference performance of visual algorithm;

[0098] In the target recognition and tracking process, the basic intelligent algorithm can be further optimized in network structure, algorithm function and thrust performance, wherein the network structure optimization includes: optimizing each type of operator according to NPU operator compatibility and recommended configuration; inter-layer deep fusion, multiple layers are fused together, small block loop calculation, data is transmitted in internal RAM, no DDR reading and writing; model weight rigid pruning; algorithm function optimization includes: adding a new detection head branch for small detection targets; adding attention mechanism to improve detection performance; inference performance optimization includes: Post-Training Quantization, quantizing the weight in the trained model from float32 to int8 or int4, and calibrating the data (activation) during model inference through a small amount of calibration dataset; enable inter-layer data sharing, when there is an event, the next layer must wait for all calculations to be completed before starting the calculation of the next layer, and when there is no event, the calculation is performed synchronously without depending on the calculation result of the previous layer.

[0099] The monitored target attribute of the first stage of the visual algorithm is input into the Tracker, and for the specific architecture of the Tracker, the NPU hardware supports the operators InverseSigmoid, GridSampleXDec, PosEncode and MulReduceSum.

[0100] Through the optimization and compatibility of the intelligent algorithm, it can run efficiently on the HiVision embedded platform, and continuously output the feature information, boundary information and trajectory information of various targets in the blind area range through the obtained continuous video information.

[0101] After the mobile target and the mobile parameter of the mobile target are obtained through the step S4, a pre-collision time is calculated according to the mobile parameter in step S5 to obtain pre-collision information, and the pre-collision information includes a collision trigger condition. The pre-collision time (TTC) has a mature algorithm, and its execution principle is generally to analyze the future motion direction and running speed of the target from the boundary information of the blind area target and the running trajectory of the target. The BSD blind area detection system obtains the vehicle running data from the CAN bus of the motor vehicle, calculates the boundary information of the motor vehicle and the relative speed of the blind area target, finally calculates the TTC (Time to Collision) based on the kinematic model, and continuously captures different blind area target attributes (such as pedestrians, vehicles, non-motor vehicles, etc.), blind area target trajectory prediction weight coefficient, and TTC of different targets. The corresponding collision warning risk level is obtained through comprehensive analysis.

[0102] After obtaining the collision warning risk level, in the blind area warning, step S6 is further needed in the embodiment to obtain the running state of the vehicle, and the vehicle running state includes the vehicle speed and the vehicle operation.

[0103] Since no pre-warning is needed when the driving state of the vehicle does not cause the collision trigger condition, S7 is performed after the driving state of the vehicle is obtained. When the driving state of the vehicle meets the collision trigger condition, the pre-warning information is output.

[0104] The blind area monitoring method of the embodiment of the application can first accurately obtain the actual driving position blind area visual field range according to the man-machine engineering parameters of the driving position, avoid capturing and calculating all moving targets in the side rear image, limit the blind area monitoring range to an accurate and reasonable range, avoid false positives, reduce the algorithm requirement, further, in outputting the pre-warning information, whether to output is judged in combination with the driving state of the vehicle, invalid prompt information is reduced, and the influence on the normal driving of the vehicle is avoided.

[0105] In a second aspect, the embodiment of the application further provides a BSD blind area monitoring system based on visual image, which comprises:

[0106] A man-machine engineering processing module 1 obtains the driving position side rear visual field range corresponding to the man-machine engineering parameters of the driving position according to the man-machine engineering parameters of the driving position.

[0107] A man-machine engineering acquisition module 2 obtains the current man-machine engineering parameters of the driving position and sends them to the man-machine engineering processing module.

[0108] A video acquisition module 3 obtains the side rear image of the vehicle.

[0109] A blind area acquisition module 4 obtains the current driving position side rear visual field range from the man-machine engineering processing module, obtains the side rear image of the vehicle from the video acquisition module, and reversely calibrates the driving position side rear visual field range in the side rear image to obtain the driving position blind area range.

[0110] A blind area analysis module 5 captures the moving target in the driving position blind area range and obtains the moving parameters of the moving target.

[0111] A pre-collision module 6 performs pre-collision time calculation according to the moving parameters to obtain pre-collision information, and the pre-collision information comprises a collision trigger condition.

[0112] A driving state monitoring module 7 monitors the driving state of the vehicle, and the driving state of the vehicle comprises the vehicle speed and the vehicle operation.

[0113] A pre-warning module 8 obtains the pre-collision information from the pre-collision module and obtains the driving state of the vehicle from the driving state monitoring module, and outputs the pre-warning information when the driving state of the vehicle meets the collision trigger condition.

[0114] The above modules communicate data and instructions through a vehicle CAN bus.

[0115] In terms of system hardware deployment, the system of the present example is deployed in a HiSilicon chip system (SOC) environment, and supports implementation in any combination of environments including a data server containing background components, a middleware application server, and a graphical user interface front-end device. The system components are interconnected through a digital communication network, including local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet, among other connection methods. This architecture is particularly suitable for intelligent application scenarios that require multi-modal interaction. Through standardized interfaces, the architecture enables data exchange and collaborative work between subsystems, ensuring compatibility and scalability of the system in different deployment environments.

[0116] In the execution conditions of the HiSilicon chip system, the scalar unit (SPU) uses reduced instruction set (RISC-V extended instructions) to support conditional branch prediction and dynamic instruction scheduling. The vector unit (VPU) integrates 128 parallel ALUs. The tensor unit (TPU) uses a mixed precision computing array (512 INT8 MACs + 256 FP16 MACs) to support a dynamic precision switching mechanism. The computing units are interconnected through a bidirectional ring bus, supporting convolution layer and pooling layer pipeline parallel processing. The weight prefetch mechanism uses a DMA controller to achieve timing overlap between computation and data transfer.

[0117] In terms of system interaction, the present example includes a display unit, an input device, and a multi-modal interaction interface. The display unit can use a cathode ray tube or a liquid crystal display to achieve information output functions. The input device includes a keyboard and a mouse, a trackball, and other pointing devices for receiving user operation instructions. The multi-modal interaction interface supports multiple feedback forms including vision, hearing, and touch, and is compatible with voice input, sound control instructions, and tactile sensing, among other interaction methods. Through the collaborative work of hardware modules, intelligent human-computer interaction is achieved.

[0118] In a third aspect, the embodiments of the present application provide a storage medium for storing an executable program for executing the above-mentioned first aspect of the BSD blind area monitoring method based on visual images.

[0119] The computer readable storage medium described in the present application includes but is not limited to the following implementation forms: solid state storage carrier (such as RAM / ROM / Flash memory based on semiconductor process), magnetic recording carrier (such as hard disk, magnetic tape) and optical recording carrier (such as CD / DVD / Blu-ray medium); Signal transmission implementation includes wired transmission medium (such as coaxial cable, twisted pair, optical fiber) and near field coupling medium (such as electromagnetic induction / infrared system); It also includes composite storage architecture (such as SSD+HDD hybrid system) and new storage technology implementation (such as phase change memory PCM, resistive random access memory RRAM). All technical solutions can be independently implemented, or can be combined in any effective way to form a new storage medium implementation.

[0120] The above is only the preferred embodiment of the embodiment of the present application, and does not limit the disclosure range of the embodiment of the present application. Any equivalent structure or equivalent flow transformation made by using the content of the specification and drawings of the embodiment of the present application, or directly or indirectly applied in other related technical fields, is also included in the patent protection range supported by the embodiment of the present application.

Claims

1. A BSD blind spot detection method based on visual images, characterized in that, Includes the following steps: Based on the driver's seat ergonomic parameters, obtain the driver's seat side and rear field of view corresponding to the driver's seat ergonomic parameters; Based on the current driver's seat ergonomic parameters, obtain the current driver's seat side and rear field of vision range; Acquire a rear-side image, and inversely mark the driver's rear-side field of view range in the rear-side image to obtain the driver's blind spot range; The moving target within the blind spot of the driver's seat is captured, and the movement parameters of the moving target are obtained; The pre-collision time is calculated based on the movement parameters to obtain pre-collision information, which includes collision triggering conditions. The vehicle's driving status is obtained, including vehicle speed and vehicle operation. When the vehicle's driving state meets the collision triggering conditions, a warning message is output.

2. The BSD blind spot monitoring method based on visual images as described in claim 1, characterized in that, The step of obtaining the driver's side and rear field of view range corresponding to the driver's side ergonomic parameters includes: Obtain ergonomic samples, which include ergonomic parameters and driver body shape data; A human factors engineering calculation model is established, the human factors engineering samples are input into the human factors engineering calculation model, and the human factors engineering calculation model is trained.

3. The BSD blind spot monitoring method based on visual images as described in claim 1 or 2, characterized in that, The ergonomic parameters include the seat's fore-and-aft distance, seat's vertical height, seat back angle, steering wheel angle, and rearview mirror angle.

4. The BSD blind spot monitoring method based on visual images as described in claim 2, characterized in that, The step of reading the current driver's seat ergonomic parameters and obtaining the current driver's seat side and rear field of view includes: The ergonomic calculation model is loaded after the vehicle is started; Read the vehicle's current ergonomic parameters, input them into the ergonomic calculation model, and obtain the current driver's body shape data; Based on the current driver's body shape data, obtain the driver's side and rear field of vision range.

5. The BSD blind spot monitoring method based on visual images as described in claim 1, characterized in that, The step of reverse-calibrating the driver's side rearward field of view range in the side rearward image includes: Displays images of the vehicle's side and rear within the field of view of the camera; The vehicle's rear-side image is delineated using the driver's side rear-side field of view. The vehicle's side and rear view outside the driver's side rear view range is defined as the driver's side blind spot range. Highlight the driver's blind spot area.

6. The BSD blind spot monitoring method based on visual images as described in claim 1, characterized in that, The step of capturing moving targets within the driver's blind spot and obtaining the movement parameters of the moving targets includes: The image within the driver's blind spot area in the side and rear view of the vehicle is preprocessed to obtain a preprocessed image; The preprocessed image is separated into consecutive frame images; The moving target is obtained from the consecutive frame images, and the movement parameters of the moving target are obtained.

7. The BSD blind spot monitoring method based on visual images as described in claim 1, characterized in that, The BSD blind spot prediction algorithm includes temporal and spatial information on the target type and the target's continuous trajectory within the BSD detection area; The method of predicting the future trajectory of a target using the BSD blind spot prediction algorithm includes: The target spatial location and boundary information are determined based on the spatial trajectory information of the BSD. The target velocity and acceleration information are determined based on the continuous time information of the BSD. The future trajectory information of the target is predicted based on the aforementioned spatial and temporal information.

8. The BSD blind spot monitoring method based on visual images as described in claim 1, characterized in that, The collision risk warning level of the BSD blind spot monitoring system is obtained by combining blind spot monitoring information based on BSD and vehicle driving status data, including: Based on the spatial information of the trajectory prediction of the multiple targets and the temporal information of the trajectory prediction of the multiple targets, the collision detection weight coefficient and the multi-target TTC weight coefficient are obtained. The collision risk level of the BSD blind spot monitoring system is determined by the target attribute weight coefficient, trajectory prediction weight coefficient, and TTC weight coefficient, wherein the TTC weight coefficient is greater than the trajectory prediction weight coefficient and the attribute weight coefficient.

9. A visual image-based BSD blind spot monitoring system, comprising: The ergonomics processing module obtains the driver's side and rear field of view range corresponding to the driver's side ergonomics parameters based on the driver's side ergonomics parameters. The ergonomics acquisition module obtains the current ergonomics parameters of the driver's seat and sends them to the ergonomics processing module; The video capture module acquires side and rear images of the vehicle; The blind spot acquisition module obtains the current driver's side and rear field of view from the ergonomics processing module, obtains the vehicle's side and rear image from the video acquisition module, and reverse-calibrates the driver's side and rear field of view in the side and rear image to obtain the driver's blind spot range. The blind spot analysis module captures moving targets within the driver's seat blind spot area and obtains the movement parameters of the moving targets. The pre-collision module calculates the pre-collision time based on the movement parameters and obtains pre-collision information, which includes collision triggering conditions. The driving status monitoring module monitors the vehicle's driving status, which includes vehicle speed and vehicle operation. The warning module obtains the pre-collision information from the pre-collision module and the vehicle driving status from the driving status monitoring module. When the vehicle driving status meets the collision triggering conditions, it outputs warning information.

10. A storage medium, characterized in that, The storage medium is used to store an executable program for performing the visual image-based BSD blind spot monitoring method described in the first aspect above.

Images