Detection method and device for dead zone of carrier, electronic equipment and storage medium

Abstract

The invention relates to the field of intelligent driving, in particular to a detection method and device for a dead zone of a carrier, electronic equipment and a storage medium. According to the method provided by the invention, when the carrier is in the target scene in which the visual blind area is easy to appear, the target monitoring equipment related to the carrier can be accurately determined. Therefore, video data of blind areas in all directions can be acquired in real time, and the monitoring range is comprehensively expanded to the periphery of the carrier instead of being limited to the two sides. By capturing videos of blind areas in all directions in real time, the moving target in the videos can be effectively detected, and the collision risk degree between the moving target and the carrier is analyzed. And on the basis of the analysis result, control operation is executed according to the corresponding risk level, effective monitoring and response to the omni-directional blind area of the carrier are achieved, the defect that the monitoring range of existing auxiliary equipment is limited is overcome, the driving safety of the carrier is improved, and the occurrence probability of collision accidents caused by the view blind area is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving, and particularly to a method, device, electronic device and storage medium for detecting blind spots of a vehicle. Background Art

[0002] In the vehicle driving scenario, the driver's field of vision blind spot has always been an important factor inducing traffic accidents. For example, when a vehicle turns or drives out of an intersection, due to the certain length of the vehicle's head and tail, the driver needs to lean out a corresponding distance to see the situations on both sides clearly. However, at this time, it is difficult to detect other vehicles or pedestrians moving quickly in the blind spots of the vehicle's field of vision on both sides. At present, some auxiliary devices such as radars equipped on vehicles mainly target the rear of the vehicle and cannot solve the problem that the blind spots on both sides of the vehicle cannot be perceived in some specific scenarios. Even if some vehicles adopt panoramic imaging systems, there are also defects in quickly detecting moving targets, accurately calculating the moving direction and speed, and making effective early warnings and countermeasures. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, device, electronic device and storage medium for detecting blind spots of a vehicle, so as to solve the problem that some auxiliary devices such as radars equipped on vehicles mainly target the rear of the vehicle and cannot solve the problem that the blind spots on both sides of the vehicle cannot be perceived in some specific scenarios.

[0004] In a first aspect, an embodiment of the present invention provides a method for detecting blind spots of a vehicle, the method including:

[0005] If the current driving scenario of the vehicle hits a target scenario category where blind spots are likely to appear, determine a target monitoring device in the vehicle associated with the target scenario category;

[0006] Control the target monitoring device to continuously obtain video data of the current blind spot of the vehicle;

[0007] Based on the video data, detect a moving target located in the blind spot, and analyze the risk degree of the risk of collision between the moving target and the vehicle;

[0008] Execute corresponding control operations on the vehicle according to the control strategy corresponding to the risk degree.

[0009] Further, the determining the target monitoring device in the vehicle associated with the target scenario category includes:

[0010] Determine the orientation of the blind spot relative to the vehicle according to the target scenario category;

[0011] Obtain the installation positions of various monitoring devices in the vehicle;

[0012] Determine the monitoring device corresponding to the installation position and the orientation as the target monitoring device.

[0013] Further, determining the monitoring device corresponding to the installation position and the orientation as the target monitoring device includes:

[0014] When there are multiple monitoring devices corresponding to the installation position and the orientation, determine all the monitoring devices corresponding to the installation position and the orientation as candidate monitoring devices;

[0015] Detect whether multiple candidate monitoring devices are in an enabled state;

[0016] If multiple candidate monitoring devices are all in an enabled state, determine the priority of each candidate monitoring device by using the load data and performance parameters of each candidate monitoring device;

[0017] Take the candidate monitoring device with the highest priority as the target monitoring device.

[0018] Further, analyzing the risk level of the collision risk between the moving target and the vehicle includes:

[0019] Obtain the vehicle motion parameters of the vehicle, and analyze the vehicle motion parameters and the video data to obtain the target motion parameters of the moving target;

[0020] Based on the target motion parameters and the vehicle motion parameters, calculate the relative speed and relative distance between the moving target and the vehicle;

[0021] Based on the relative speed and the relative distance, calculate the estimated encounter time between the moving target and the vehicle;

[0022] Map the relative speed, the relative distance, and the estimated encounter time to a fuzzy set to obtain a fuzzy result, and perform reasoning on the fuzzy result to obtain a fuzzy value;

[0023] Perform defuzzification on the fuzzy value to obtain the risk level of the collision risk.

[0024] Further, analyzing the vehicle motion parameters and the video data to obtain the target motion parameters of the moving target includes:

[0025] Obtain the displacement data and displacement direction of the moving target in the video data;

[0026] Based on the displacement data, the device acquisition parameters of the target monitoring device, and the vehicle motion parameters, calculate the moving speed of the moving target;

[0027] Determine the moving direction of the moving target based on the displacement direction;

[0028] Determine the moving speed and the moving direction as the target motion parameters.

[0029] Further, performing corresponding control operations on the vehicle according to the control strategy corresponding to the risk level includes:

[0030] If the risk level is greater than a first preset value, send an alarm message to the driver in the vehicle and monitor whether the driver triggers a feedback operation;

[0031] If the feedback operation is not monitored within a set time, obtain the change of the risk level within the set time;

[0032] If the change is an increase in the risk level and the risk level reaches a second preset value, control the vehicle to perform an emergency braking.

[0033] Further, after monitoring whether the driver triggers a feedback operation, the method further includes:

[0034] Predict the change of the risk level according to the vehicle motion parameters of the vehicle, the target motion parameters of the moving target, and the environmental data of the environment where the vehicle is located;

[0035] If the predicted change is an increase in the risk level and does not reach the second preset value, control the vehicle to perform a deceleration operation.

[0036] In a second aspect, an embodiment of the present invention provides a detection device for a vehicle blind area, and the device includes:

[0037] A determination module, configured to determine a target monitoring device in the vehicle associated with the target scene category if the current driving scene of the vehicle hits a target scene category where a vision blind area is likely to appear;

[0038] An acquisition module, configured to control the target monitoring device to acquire a blind area image of the current vision blind area of the vehicle in real time;

[0039] An analysis module, configured to detect a moving target located in the vision blind area based on the blind area image and analyze the risk level of the risk of collision between the moving target and the vehicle;

[0040] A control module, configured to perform corresponding control operations on the vehicle according to the control strategy corresponding to the risk level.

[0041] In a third aspect, an embodiment of the present invention provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the method according to the first aspect or any corresponding embodiment thereof.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which computer instructions are stored. The computer instructions are used to cause a computer to execute the method according to the first aspect or any corresponding embodiment thereof.

[0043] When the method provided in this application is applied to a target scenario where a vehicle is prone to blind spots, it can accurately determine the target monitoring devices related thereto. Thus, video data in blind spots in all directions can be obtained in real time, and the monitoring range can be comprehensively extended to the surrounding of the vehicle, rather than being limited to only two sides. By capturing video in blind spots in all directions in real time, moving targets therein can be effectively detected, and the collision risk level between the moving targets and the vehicle can be analyzed. Based on the analysis results, control operations are performed according to the corresponding risk levels, realizing effective monitoring and response to the all-round blind spots of the vehicle, making up for the deficiency of the limited monitoring range of existing auxiliary devices, improving the driving safety of the vehicle, and effectively reducing the probability of collision accidents caused by blind spots. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0045] Figure 1 is a schematic flowchart of a method for detecting blind spots of a vehicle according to some embodiments of the present invention;

[0046] Figure 2 is a schematic flowchart of another method for detecting blind spots of a vehicle according to some embodiments of the present invention;

[0047] Figure 3 is a schematic flowchart of yet another method for detecting blind spots of a vehicle according to some embodiments of the present invention;

[0048] Figure 4 is a schematic flowchart of yet another method for detecting blind spots of a vehicle according to some embodiments of the present invention;

[0049] Figure 5 is a structural block diagram of a device for detecting blind spots of a vehicle according to an embodiment of the present invention;

[0050] Figure 6 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present invention. Specific embodiments

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0052] According to an embodiment of the present invention, there is provided a method, device, electronic device, and storage medium for detecting blind spots of a vehicle. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0053] In this embodiment, a method for detecting blind spots of a vehicle is provided. Figure 1 It is a flowchart of a method for detecting blind spots of a vehicle according to an embodiment of the present invention. As Figure 1 shown, the process includes the following steps:

[0054] Step S101, if the current driving scenario of the vehicle hits the target scenario category where a vision blind spot is likely to occur, determine the target monitoring device in the vehicle associated with the target scenario category.

[0055] In the embodiments of the present application, the vehicle covers various types of manned devices, including but not limited to various vehicles traveling on land, such as cars, buses, trucks, etc.; aircraft capable of flying in the air, and flying cars with both land-traveling and air-flying capabilities; and also includes water vehicles sailing on water, such as yachts.

[0056] In the embodiments of the present application, taking a vehicle as an example for illustration, the vehicle can use cameras deployed on the vehicle body to identify the surrounding environment of the vehicle, and use image recognition algorithms and scene analysis techniques to accurately determine the driving scenario where the vehicle is located. Among them, the target scenario categories where vision blind spots are likely to occur cover specific scenarios such as vehicle turning, exiting an intersection, and vehicle reversing. Due to the change in the driving direction of the vehicle and the complexity of the surrounding environment, vision blind spots are likely to occur on both sides of the front or rear of the vehicle.

[0057] When it is confirmed that the current driving scenario of the vehicle matches the target scenario category where the field of vision blind area is likely to appear, relevant target monitoring devices can be filtered out through the matching of key parameters such as device functional attributes, installation positions, and perspective coverage ranges. The target monitoring devices can be in-vehicle cameras or in-vehicle radars.

[0058] In the embodiment of the present application, determining the target monitoring device associated with the target scenario category in the vehicle, such as Figure 2 shown, includes the following steps A1 - A3:

[0059] Step A1, determine the orientation of the field of vision blind area relative to the vehicle according to the target scenario category.

[0060] Specifically, first, obtain a database covering various target scenario categories and their corresponding field of vision blind area characteristics. This database contains information on the positions and ranges where the field of vision blind area usually appears in different scenarios such as turning, exiting an intersection, and tunnel entrances and exits. When it is detected that the vehicle is in a certain target scenario category, the field of vision blind area characteristic data of the corresponding scenario is retrieved from the database. For example, if it is determined to be a turning scenario, according to the database information, the field of vision blind area is likely to form inside the vehicle's turning side and outside the vehicle's head. Then, in combination with the real-time motion state information of the vehicle such as the current driving direction and steering angle, the retrieved field of vision blind area characteristics are dynamically adjusted. If the vehicle is turning left at a certain angle, the range and orientation of the field of vision blind area in the left front of the vehicle head and the left front of the vehicle body will change due to the turning angle.

[0061] Step A2, obtain the installation positions of each monitoring device in the vehicle.

[0062] Specifically, in the vehicle scenario, to obtain the installation positions of each monitoring device in the vehicle, taking a vehicle as an example, it mainly refers to obtaining the specific installation positions of each camera on the vehicle. These cameras are distributed in different parts of the vehicle. For example, the camera installed on the left side of the vehicle head is clearly located on the left side of the vehicle head and can monitor the area in front of the vehicle on the left; there is also a camera on the right side of the vehicle tail, installed on the right side of the vehicle tail, responsible for capturing the image of the right rear of the vehicle.

[0063] Step A3, determine the monitoring device with the installation position corresponding to the orientation as the target monitoring device.

[0064] Specifically, determining the monitoring device corresponding to the installation position and orientation includes: when there are multiple monitoring devices corresponding to the installation position and orientation, determining all the monitoring devices corresponding to the installation position and orientation as candidate monitoring devices; detecting whether the multiple candidate monitoring devices are in an enabled state; if all the multiple candidate monitoring devices are in an enabled state, determining the priority of each candidate monitoring device by using the load data and performance parameters of each candidate monitoring device; and taking the candidate monitoring device with the highest priority as the target monitoring device.

[0065] First, after determining the orientation of the field of view blind area relative to the vehicle and the installation positions of each monitoring device through the previous steps, there may be a situation where the installation positions of multiple monitoring devices correspond to this orientation. At this time, automatically mark all the multiple monitoring devices corresponding to these installation positions and orientations as candidate monitoring devices. For example, in a complex intersection scenario, the field of view blind area is in the front right of the vehicle, and there are three installation positions and viewing angles of monitoring devices in the front right of the vehicle that can cover this area, then these three monitoring devices are all determined as candidate monitoring devices.

[0066] Second, query the status information of each candidate monitoring device in turn. For example, send a status query instruction to each candidate monitoring device through the network communication protocol. After receiving the instruction, the monitoring device returns its own enabled status information (enabled or not enabled). If a certain candidate monitoring device is currently performing other tasks or in a fault state, it may be in a non-enabled state; while a normal and ready-to-use monitoring device is in an enabled state. Then record the enabled state of each candidate monitoring device and determine whether all candidate monitoring devices are in an enabled state.

[0067] Furthermore, for the candidate monitoring devices that are all in an enabled state, collect their current load data. The load data can include the network bandwidth occupancy rate of the device, the complexity of image encoding, and the number of video streams processed simultaneously, etc. At the same time, obtain the performance parameters of each candidate monitoring device, such as resolution, frame rate, viewing angle range, image sensor sensitivity, etc. Different models of cameras have different resolutions and frame rates, and these performance differences will affect their monitoring effect on the field of view blind area.

[0068] Next, using a pre-set priority calculation model, the collected load data and performance parameters are used as inputs. This model assigns corresponding weights according to the importance of different parameters. For example, in scenarios with high real-time requirements, the weight of the frame rate may be set higher; in scenarios where clear image details are needed, the weight of the resolution may be higher. The model calculates the priority scores of each candidate monitoring device through a specific algorithm (such as the weighted summation algorithm). For example, assuming the formula for calculating the priority score is: Priority score = 0.4×(1 - network bandwidth occupancy rate) + 0.3×resolution + 0.2×frame rate + 0.1×viewing angle range, the priority scores of each candidate monitoring device are calculated through this formula.

[0069] Finally, compare the priority scores of all candidate monitoring devices and find the candidate monitoring device with the highest score. The candidate monitoring device with the highest priority is determined as the target monitoring device, and this device will be used for subsequent monitoring of the blind spot of the field of view. For example, after calculation, the priority score of candidate monitoring device A is 85 points, the score of candidate monitoring device B is 78 points, and the score of candidate monitoring device C is 82 points, then candidate monitoring device A is used as the target monitoring device.

[0070] The method provided by the embodiment of the present application first determines the orientation of the blind spot of the field of view relative to the vehicle according to the target scene category, and can accurately identify the area that needs to be key monitored by the vehicle in a specific scene. Then, obtain the installation positions of the monitoring devices in the vehicle, providing basic information for screening suitable monitoring devices. The monitoring device corresponding to the installation position and the orientation is determined as the target monitoring device, ensuring that the monitoring resources can be used targeted. Especially when there are multiple corresponding monitoring devices, first determine them as candidate monitoring devices, and then detect their enabled status to ensure that the devices put into use are in a normal working state. If multiple candidate devices are all enabled, use the load data and performance parameters to determine the priority and select the device with the highest priority as the target monitoring device, which can give full play to the device performance, reasonably allocate resources, effectively monitor the blind spot of the field of view with the optimal monitoring device, and improve the monitoring effect and safety of the blind spot of the field of view during the vehicle driving process.

[0071] Step S102, control the target monitoring device to continuously obtain the video data of the current blind spot of the field of view of the vehicle.

[0072] In the embodiment of the present application, in the vehicle scenario, after the target monitoring device is determined, an instruction is issued to it to start working. Taking a vehicle as an example, if the target monitoring device is the left front camera of the vehicle head, it will continuously collect the picture information of the left front blind spot of the vehicle according to the set parameters and frequencies, forming continuous video data; if the target monitoring device is the right rear camera of the vehicle tail, it will capture the dynamics of the right rear blind spot of the vehicle in real time and convert the images of these areas into video data.

[0073] Alternatively, in the scenario of a vehicle with both land driving and air flight capabilities such as a flying car, after determining the target monitoring device according to specific rules, an instruction is immediately sent to it to start real-time data collection. For example, if the target monitoring device is a camera installed in the upper left front of the flying car's head, when driving on land, it will continuously focus on the blind spot in the ground area directly in front of the vehicle's left, and continuously capture the dynamic images of this area during the vehicle's driving process to form continuous video data; when the flying car is in the air flight state, this camera will, according to the flight attitude and preset viewing angle, obtain the blind spot images in the airspace directly in front of the flight direction in real time and convert them into video data.

[0074] Step S103: Detect moving targets located in the blind spot based on the video data, and analyze the risk level of the risk of collision between the moving targets and the vehicle.

[0075] In the embodiment of the present application, analyzing the risk level of the risk of collision between the moving target and the vehicle, as Figure 3 shown, includes the following steps B1 - B5:

[0076] Step B1: Obtain the vehicle motion parameters of the vehicle, and analyze the vehicle motion parameters and the video data to obtain the target motion parameters of the moving target.

[0077] Specifically, obtaining the vehicle motion parameters of the vehicle is to collect data reflecting the vehicle motion state such as the driving speed, driving direction, and possibly involved acceleration and angular velocity of the vehicle through corresponding sensors.

[0078] Specifically, analyzing the vehicle motion parameters and the video data to obtain the target motion parameters of the moving target includes: obtaining the displacement data and displacement direction of the moving target in the video data; calculating the moving speed of the moving target based on the displacement data, the device acquisition parameters of the target monitoring device, and the vehicle motion parameters; determining the moving direction of the moving target based on the displacement direction; and determining the moving speed and the moving direction as the target motion parameters.

[0079] First, use object detection algorithms (such as YOLO, FasterR - CNN, etc.) to process the video data frame by frame, identify the moving target and mark its position. Subsequently, use object tracking algorithms (such as KCF, CSRT, etc.) to continuously track the moving target and record the position coordinates of the target in different video frames. By comparing the position coordinates of the moving target in adjacent video frames, calculate the displacement vector of the moving target in the image plane. The displacement data is the modulus of this vector, and the displacement direction is the direction of this vector.

[0080] Secondly, the device acquisition parameters of the target monitoring device mainly include the focal length, viewing angle, frame rate, etc. of the camera. The motion parameters of the vehicle are obtained through vehicle-mounted sensors (such as speed sensors, gyroscopes, etc.), including the driving speed v1 and the driving direction.

[0081] Then, according to the focal length and viewing angle of the camera, the displacement data on the image plane is converted into the displacement distance in the actual scene. Assume that the displacement data on the image plane is d1, and the proportional relationship between the image and the actual scene obtained through calibration is k. Then the calculation process of the actual displacement distance d2 is as follows: d1×k. According to the video frame rate f, the time interval t between two adjacent frames is determined as t = 1 / f.

[0082] Considering the movement of the vehicle itself, if the movement directions of the vehicle and the moving target are the same, the actual displacement of the moving target relative to the ground also needs to consider the displacement of the vehicle within the time interval. The calculation process of the moving speed v2 of the moving target is as follows: Among them, subtraction is used for the same direction, and addition is used for the opposite direction.

[0083] Finally, the displacement direction determined on the image plane is converted into the actual geographical coordinate system. The installation angle and direction of the camera can be pre-calibrated, and combined with the driving direction of the vehicle, the image coordinate system is associated with the geographical coordinate system. After coordinate transformation, the displacement direction of the moving target relative to the geographical coordinate system is obtained, which is used as the moving direction of the moving target. Then, the moving speed and the moving direction are determined as the target motion parameters.

[0084] Step B2, based on the target motion parameters and the vehicle motion parameters, calculate the relative speed and relative distance between the moving target and the vehicle.

[0085] Specifically, first, the speeds of the moving target and the vehicle are both represented by vectors. Through trigonometric function knowledge, the speed is decomposed into horizontal and vertical directions. For example, given the magnitude of the speed of the moving target and the angle of its moving direction, the speeds in the horizontal and vertical directions can be calculated respectively, thus obtaining the speed vector of the moving target. Similarly, the speed vector of the vehicle is obtained.

[0086] Next, according to the rules of vector operations, subtract the speed vector of the vehicle from the speed vector of the moving target to obtain the relative speed vector. This vector contains the components of the relative speed in the horizontal and vertical directions. Finally, using the Pythagorean theorem, take the square root of the sum of the squares of the horizontal and vertical components of the relative speed vector to obtain the magnitude of the relative speed.

[0087] Finally, through the object detection and tracking algorithms, combined with the calibration parameters of the camera and the positioning information of the vehicle, the position coordinates of the moving object and the vehicle in the same coordinate system are determined. Using the Euclidean distance formula, that is, the formula for calculating the straight-line distance between two points, the squares of the differences between the position coordinates of the moving object and the vehicle are added respectively, and then the square root is calculated. The result obtained is the relative distance between them.

[0088] Step B3, based on the relative speed and the relative distance, calculate the predicted encounter time between the moving object and the vehicle.

[0089] Specifically, first, determine whether the moving object and the vehicle have a tendency to approach each other. For example, check whether the direction of the relative velocity vector points to the line connecting the vehicle and the moving object. If so, it indicates that there is a possibility of approaching; if not, it is considered that they will not meet. When it is determined that the two have a tendency to approach each other, divide the relative distance by the magnitude of the relative speed, and the result obtained is the predicted encounter time.

[0090] Step B4, map the relative speed, the relative distance, and the predicted encounter time to the fuzzy sets to obtain the fuzzy results, and perform reasoning on the fuzzy results to obtain the fuzzy values.

[0091] Specifically, preset the fuzzy sets corresponding to the relative speed, the relative distance, and the predicted encounter time respectively. For example, the relative speed can be divided into fuzzy sets such as very slow, slow, moderate, fast, and very fast; the relative distance is divided into very close, close, moderate, far, and very far; the predicted encounter time is divided into very short, short, moderate, long, and very long. At the same time, define the membership function for each fuzzy set. This function can convert specific numerical values into the membership degrees in the fuzzy sets. For example, for the fuzzy set of "fast" relative speed, when the relative speed is within a certain specific range, its membership degree in the "fast" set is 1, and the membership degree will gradually decrease outside this range.

[0092] Substitute the specific numerical values of the relative speed, the relative distance, and the predicted encounter time calculated previously into the corresponding membership functions respectively. In this way, the membership degrees of them in each fuzzy set can be obtained, that is, the degrees to which these numerical values belong to each fuzzy set. For example, if the relative speed is a specific value, after calculation by the membership function, it can be known what its membership degree is in the fuzzy set of "fast" and what its membership degree is in the fuzzy set of "moderate".

[0093] The preset fuzzy rules are summarized based on a large amount of actual collision experience data. For example, if the relative speed is "fast", the relative distance is "very close", and the predicted encounter time is "very short", then the collision risk is "extremely high".

[0094] For each preset fuzzy rule, according to the previously obtained fuzzy results, that is, the membership degrees of the relative speed, relative distance, and predicted encounter time in each fuzzy set, through fuzzy logic operations, usually by taking the minimum value, the excitation strength of this rule is obtained.

[0095] For each fuzzy set of collision risks, such as extremely low, low, medium, high, and extremely high, the excitation strengths of all the rules that can excite this fuzzy set are combined, generally by taking the maximum value, and finally the membership degrees of the collision risk in each fuzzy set are obtained, which are the fuzzy values.

[0096] Step B5, defuzzify the fuzzy values to obtain the risk level of the collision risk.

[0097] Specifically, first, a suitable defuzzification method needs to be selected. Common methods include the centroid method, the maximum membership degree method, etc. Here, the centroid method is taken as an example. The calculation process of the centroid method is to regard each fuzzy set of the collision risk as a point with a corresponding representative value, and at the same time, each fuzzy set also has the membership degree calculated previously. Multiply the representative value of each fuzzy set by its membership degree, then add up all the products, and divide by the sum of all membership degrees. The result obtained is the defuzzified collision risk level. This level is a specific value that can intuitively reflect the magnitude of the collision risk between the moving target and the vehicle.

[0098] In the embodiment of the present application, by obtaining the vehicle motion parameters and combining with the video data to analyze and obtain the target motion parameters of the moving target, the motion state of the moving target can be accurately grasped. On this basis, the relative speed, relative distance, and predicted encounter time between the moving target and the vehicle are calculated, providing key quantitative indicators for judging the collision possibility between the two. Using fuzzy sets and preset fuzzy rules to process these data fully considers the complexity and uncertainty of the actual collision situation, making the risk assessment more scientific and reasonable. The preset fuzzy rules are obtained based on actual collision experience data, ensuring the reliability of the assessment results. Defuzzifying the fuzzy values to obtain the risk level of the collision risk converts the fuzzy assessment results into clear risk levels, facilitating the subsequent adoption of targeted measures.

[0099] Step S104, perform corresponding control operations on the vehicle according to the control strategy corresponding to the risk level.

[0100] In the embodiment of the present application, perform corresponding control operations on the vehicle according to the control strategy corresponding to the risk level, such as Figure 4 shown, including the following steps C1 - C3:

[0101] Step C1, if the risk level is greater than the first preset value, send an alarm message to the driver in the vehicle and monitor whether the driver triggers a feedback operation.

[0102] Specifically, continuously monitor the collision risk level calculated previously. Compare this risk level with a pre-set first preset value. The first preset value is a critical value determined based on a large number of actual tests and safety standards. When the risk level exceeds this value, it means there is a certain collision danger in the current situation, and a warning needs to be issued to the driver.

[0103] Once it is determined that the risk level is greater than the first preset value, immediately activate the warning mechanism. The form of the warning information can be diversified to ensure that it can attract the driver's attention. The warning methods include audible warnings, such as emitting a sharp alarm sound; visual warnings, such as displaying prominent warning icons and text on the instrument panel or head-up display (HUD); and tactile warnings can also be used, for example, making the seat vibrate. These warning methods can be used alone or in combination.

[0104] While sending the warning information, monitor whether the driver triggers a feedback operation. The feedback operation can be various actions taken by the driver related to dealing with the danger, such as stepping on the brake, turning the steering wheel, pressing a specific button, etc. Detect these operations through in-vehicle sensors. For example, the brake pedal position sensor can detect whether the driver steps on the brake, and the steering wheel angle sensor can detect whether the steering wheel is turned.

[0105] Step C2, if no feedback operation is monitored within the set time, obtain the change of the risk level within the set time.

[0106] Specifically, preset a specific time range and start timing from the moment the warning information is sent. Continuously monitor whether the driver triggers a feedback operation within this set time. If no feedback operation is detected at the end of the set time, it means the driver may not have effectively responded to the warning information.

[0107] At this time, continuously collect the risk level data at different moments within the set time. These data can be obtained by continuously repeating the previous steps of calculating the collision risk level. Then, analyze these risk level data, compare the risk levels at different moments, and judge whether the risk level is increasing, decreasing, or remaining unchanged. The change trend can be determined by calculating the difference in the risk levels at adjacent moments. If the difference is positive, it means the risk level is increasing; if the difference is negative, it means the risk level is decreasing; if the difference is zero, it means the risk level remains unchanged.

[0108] Step C3, if the change situation shows that the risk level is increasing and the risk level reaches the second preset value, control the vehicle to perform an emergency brake.

[0109] Specifically, the obtained change in the risk level is evaluated. If the analysis result shows that the risk level shows an upward trend within the set time, it means that the collision risk is increasing continuously. Then, the current risk level is compared with a preset second preset value. The second preset value is usually higher than the first preset value, representing a higher level of collision risk. When the risk level rises and reaches or exceeds the second preset value, it indicates that the situation is very critical and immediate measures need to be taken to avoid a collision.

[0110] When the conditions that the risk level rises and reaches the second preset value are met, a control instruction is sent to the brake of the vehicle. After receiving the instruction, the brake will quickly perform an emergency braking operation. By increasing the braking force, the vehicle is decelerated and stopped as soon as possible to reduce the possibility and severity of a collision. During the execution of the emergency braking, it will also be dynamically adjusted according to the driving state of the vehicle and the surrounding environment to ensure the safety and stability of the braking process.

[0111] It should be noted that when considering the impact of different driving environments on the risk, the first preset value and the second preset value are dynamically changed. By obtaining environmental information in real time, such as weather conditions, road surface slipperiness, etc. When in a high-risk environment such as rainy days or slippery roads, in order to respond to potential dangers earlier, the first preset value and the second preset value will be reduced. In this way, the risk level is more likely to exceed the first preset value, and the system can send an alarm message to the driver in the vehicle earlier and monitor the feedback operation; and the risk level is also more likely to reach the reduced second preset value, so as to control the vehicle to perform emergency braking in time. In a low-risk environment such as good weather and dry roads, the preset values will be appropriately increased to reduce unnecessary interventions and avoid the system sending alarms and performing emergency braking operations too frequently, ensuring the smoothness and efficiency of the vehicle driving.

[0112] In the embodiment of the present application, after monitoring whether the driver triggers a feedback operation, the method further includes: predicting the change in the risk level according to the vehicle motion parameters of the vehicle, the target motion parameters of the moving target, and the environmental data of the environment where the vehicle is located; if the predicted change is that the risk level rises and does not reach the second preset value, then controlling the vehicle to perform a deceleration operation.

[0113] Specifically, a variety of environmental sensors are used to collect the environmental data of the environment where the vehicle is located. The meteorological sensor can obtain weather conditions, such as whether it is raining, snowing, fogging, etc.; the road condition sensor can detect information such as the flatness and slope of the road; the traffic sensor can count the surrounding traffic flow, the distribution of other vehicles and pedestrians, etc.

[0114] Machine learning or deep learning models are used to predict the risk level, such as recurrent neural networks (RNN) and their variants (LSTM, GRU), support vector machines (SVM), etc. A large amount of historical data is collected, including the motion parameters of the vehicle, the motion parameters of the moving target, environmental data, and the corresponding risk level data. These data are divided into a training set and a test set, and the selected model is trained using the training set. During the training process, the parameters of the model are continuously adjusted to minimize the error between the prediction result and the actual risk level.

[0115] The currently collected vehicle motion parameters, target motion parameters, and environmental data are input into the trained prediction model. The model calculates based on the input data and outputs the prediction result of the change in the risk level over a period of time in the future. The prediction result will show whether the risk level is increasing, decreasing, or remaining unchanged.

[0116] The predicted change in the risk level is analyzed. If the prediction result shows that the risk level is increasing, it means that the collision risk is increasing. The predicted risk level is compared with a second preset value. The second preset value is a preset critical value representing a relatively high collision risk level. If the risk level is increasing but has not reached the second preset value, it means that the situation is deteriorating but has not reached the level that requires emergency braking.

[0117] When the conditions of increasing risk level and not reaching the second preset value are met, the vehicle-mounted control system sends a deceleration control instruction to the power system or braking system of the vehicle. After receiving the instruction, the power system reduces the output power of the engine and the driving force of the vehicle, so that the vehicle decelerates naturally. The braking system can apply braking force moderately as needed to assist the vehicle in decelerating faster. During the deceleration process, the system will continuously monitor the speed and driving state of the vehicle to ensure that the deceleration operation is smooth and safe, and to avoid other dangerous situations caused by excessive deceleration. At the same time, the prediction result of the risk level will be continuously updated, and the deceleration strategy will be dynamically adjusted according to the new situation.

[0118] The method provided in this application can accurately determine the target monitoring device related to it when the vehicle is in a target scenario where blind spots are likely to appear. Thus, video data of blind spots in all directions can be obtained in real time, and the monitoring range is comprehensively extended to all around the vehicle, not just limited to both sides. By capturing videos of blind spots in all directions in real time, moving targets in them can be effectively detected, and the collision risk level between the moving targets and the vehicle can be analyzed. Based on the analysis results, control operations are performed according to the corresponding risk levels, realizing the effective monitoring and response to all-round blind spots of the vehicle, making up for the deficiency of the limited monitoring range of existing auxiliary devices, improving the driving safety of the vehicle, and effectively reducing the probability of collision accidents caused by blind spots in the field of vision.

[0119] In this embodiment, a detection device for a vehicle blind area is further provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" may be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0120] This embodiment provides a detection device for a vehicle blind area. As Figure 5 shown, it includes:

[0121] A determination module 501, configured to determine a target monitoring device in the vehicle associated with the target scenario category if the current driving scenario of the vehicle hits a target scenario category where a visual blind area is likely to occur;

[0122] An acquisition module 502, configured to control the target monitoring device to acquire in real time a blind area image of the current visual blind area of the vehicle;

[0123] An analysis module 503, configured to detect a moving target located in the visual blind area based on the blind area image and analyze the risk level of the risk of collision between the moving target and the vehicle;

[0124] A control module 504, configured to perform corresponding control operations on the vehicle according to the control strategy corresponding to the risk level.

[0125] In an embodiment of the present application, the determination module 501 is configured to determine the orientation of the visual blind area relative to the vehicle according to the target scenario category; obtain the installation positions of each monitoring device in the vehicle; and determine the monitoring device corresponding to the installation position and the orientation as the target monitoring device.

[0126] In an embodiment of the present application, the determination module 501 is configured to, when there are multiple installation positions of monitoring devices corresponding to the orientation, determine all the monitoring devices corresponding to the installation position and the orientation as candidate monitoring devices; detect whether the multiple candidate monitoring devices are in an enabled state; if the multiple candidate monitoring devices are all in an enabled state, determine the priority of each candidate monitoring device by using the load data and performance parameters of each candidate monitoring device; and use the candidate monitoring device with the highest priority as the target monitoring device.

[0127] In an embodiment of the present application, the analysis module 503 is configured to obtain the vehicle motion parameters of the vehicle, analyze the vehicle motion parameters and the video data to obtain the target motion parameters of the moving target; calculate the relative speed and relative distance between the moving target and the vehicle based on the target motion parameters and the vehicle motion parameters; calculate the predicted encounter time between the moving target and the vehicle based on the relative speed and the relative distance; map the relative speed, the relative distance, and the predicted encounter time to a fuzzy set to obtain a fuzzy result, and perform reasoning on the fuzzy result according to a preset fuzzy rule to obtain a fuzzy value, where the preset fuzzy rule is obtained according to actual collision experience data; perform defuzzification processing on the fuzzy value to obtain the risk level of the collision risk.

[0128] In an embodiment of the present application, the analysis module 503 is configured to obtain the displacement data and the displacement direction of the moving target in the video data; calculate the moving speed of the moving target based on the displacement data, the device acquisition parameters of the target monitoring device, and the vehicle motion parameters; determine the moving direction of the moving target based on the displacement direction; and determine the moving speed and the moving direction as the target motion parameters.

[0129] In an embodiment of the present application, the control module 504 is configured to, if the risk level is greater than a first preset value, send an alarm message to the driver in the vehicle and monitor whether the driver triggers a feedback operation; if no feedback operation is monitored within a set time, obtain the change of the risk level within the set time; if the change is that the risk level increases and the risk level reaches a second preset value, control the vehicle to perform an emergency braking.

[0130] In an embodiment of the present application, the device further includes: a prediction module, configured to predict the change of the risk level according to the vehicle motion parameters of the vehicle, the target motion parameters of the moving target, and the environmental data of the environment where the vehicle is located; if the predicted change is that the risk level increases and does not reach the second preset value, control the vehicle to perform a deceleration operation.

[0131] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of an electronic device provided by an alternative embodiment of the present invention, as Figure 6As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if needed, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system).

[0132] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field-programmable gate array, a generic array logic, or any combination thereof.

[0133] Among them, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiments.

[0134] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the electronic device presented by a kind of landing page of a small program, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The memory 20 can include a volatile memory, such as a random access memory; the memory can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive; the memory 20 can also include a combination of the above types of memories.

[0136] The electronic device further includes a communication interface 30 for the electronic device to communicate with other devices or communication networks.

[0137] Embodiments of the present invention also provide a computer-readable storage medium. The methods according to the embodiments of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the methods described herein can be stored in such software processes on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.

[0138] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for detecting a vehicle blind spot, characterized in that: The method comprises: If the current driving scene of the vehicle hits a target scene category that is prone to blind spots, determining a target monitoring device in the vehicle that is associated with the target scene category; Control the target monitoring device to obtain the video data of the current blind spot of the vehicle in real time; Detecting a moving target located in the blind spot of the field of view based on the video data, and analyzing the risk level of a collision risk between the moving target and the vehicle; Perform corresponding control operations on the vehicle according to the control strategy corresponding to the risk level.

2. The method according to claim 1, characterized in that The determining of the target monitoring device in the vehicle associated with the target scene category includes: Determining the position of the blind spot relative to the vehicle according to the target scene category; Obtaining the installation location of each monitoring device in the vehicle; The monitoring device whose installation position corresponds to the orientation is determined as the target monitoring device.

3. The method according to claim 2, characterized in that The step of determining the monitoring device corresponding to the installation position and the orientation as the target monitoring device comprises: When there are multiple monitoring devices whose installation positions correspond to the orientation, all the multiple monitoring devices whose installation positions correspond to the orientation are determined as candidate monitoring devices; Detecting whether a plurality of candidate monitoring devices are in an enabled state; If the plurality of candidate monitoring devices are all in an enabled state, determining the priority of each candidate monitoring device by using the load data and performance parameters of each candidate monitoring device; The candidate monitoring device with the highest priority is used as the target monitoring device.

4. The method according to claim 1, characterized in that: The analyzing the risk level of collision between the mobile target and the vehicle includes: Acquiring vehicle motion parameters of the vehicle, and analyzing the vehicle motion parameters and the video data to obtain target motion parameters of the moving target; Calculating the relative speed and relative distance between the moving target and the vehicle based on the target motion parameters and the vehicle motion parameters; Calculating an estimated encounter time between the moving target and the vehicle based on the relative speed and the relative distance; Mapping the relative speed, the relative distance and the estimated meeting time to a fuzzy set to obtain a fuzzy result, and reasoning the fuzzy result to obtain a fuzzy value; The fuzzy value is defuzzified to obtain the risk level of the collision risk.

5. The method according to claim 4, characterized in that The analyzing the vehicle motion parameters and the video data to obtain the target motion parameters of the moving target includes: Acquire displacement data and displacement direction of the moving target in the video data; Calculating the moving speed of the moving target based on the displacement data, the device acquisition parameters of the target monitoring device, and the vehicle motion parameters; determining a moving direction of the moving target based on the displacement direction; The moving speed and the moving direction are determined as the target motion parameters.

6. The method according to claim 1, characterized in that The performing corresponding control operations on the vehicle according to the control strategy corresponding to the risk level includes: If the risk level is greater than a first preset value, a warning message is sent to the driver in the vehicle, and a feedback operation is monitored whether the driver triggers the feedback operation; If the feedback operation is not monitored within the set time, obtaining the change of the risk level within the set time; If the change is that the risk level increases, and the risk level reaches a second preset value, the vehicle is controlled to perform emergency braking.

7. The method according to claim 6, characterized in that After monitoring whether the driver triggers a feedback operation, the method further includes: Predicting a change in the risk level based on a vehicle motion parameter of the vehicle, a target motion parameter of the moving target, and environmental data of an environment where the vehicle is located; If the predicted change is that the risk level increases and does not reach the second preset value, the vehicle is controlled to perform a deceleration operation.

8. A detection device for a vehicle blind spot, characterized in that: The device comprises: A determination module, configured to determine a target monitoring device in the vehicle that is associated with the target scene category if the current driving scene of the vehicle hits a target scene category that is prone to blind spots in the field of vision; An acquisition module, used for controlling the target monitoring device to acquire a blind spot image of the current blind spot of the vehicle in real time; An analysis module, configured to detect a moving target located in the blind spot based on the blind spot image, and analyze a risk level of a collision risk between the moving target and the vehicle; A control module is used to perform corresponding control operations on the vehicle according to a control strategy corresponding to the risk level.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images