Method and system for judging blind area collision based on 360-degree camera image analysis

Abstract

The invention discloses a blind area collision judgment method and system based on 360-degree camera image analysis, and relates to the technical field of blind area collision judgment, and the method comprises the following specific steps: collecting image data of a 360-degree camera, and carrying out the preprocessing of the image data; acquiring the current motion state of the vehicle; comprising motion direction, speed and steering wheel angle; dividing the vehicle into three motion states according to the current motion parameters; performing feature recognition on the image data by using different strategies according to different motion states, judging a vehicle collision risk level according to a recognition result, and performing early warning; according to the invention, the obstacle in the image is identified, and the predicted route image area is established in the visual area of the image, so that the calculated amount of the system is reduced; according to the method, the collision risk of the vehicle is judged according to the specific value of the area of the predicted route image region occupied by the obstacle, and the early warning information is output through the early warning system after accurate judgment is carried out according to different preset strategies, so that the safety of blind area collision during vehicle driving is greatly improved, and the collision risk is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of blind spot collision judgment, and in particular to a blind spot collision judgment method and system based on 360 camera image analysis. Background Art

[0002] The 360 ​​camera forms a panoramic image by combining images from multiple angles, with the aim of providing the driver with a wider field of view and reducing blind spots. In the prior art, when a driver is driving a vehicle, there are blind spots at the front, rear and both sides of the vehicle. Obstacles in the blind spots can easily cause vehicle collisions, resulting in loss of life and property. The application and popularization of 360 cameras have greatly improved the field of view of blind spots. However, due to the distorted images and image quality, it is difficult for the driver to directly observe obstacles in the blind spots from the images with the naked eye. Therefore, how to use the collected image data in combination with deep learning algorithms to identify obstacles in blind spots and make collision judgments and early warnings has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present invention is proposed in view of the above-mentioned background problems.

[0004] Therefore, the problem to be solved by the present invention is how to combine the deep learning algorithm to identify obstacles in the image data collected by the camera and issue an early warning.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: The first aspect of the present invention provides a blind spot collision judgment method based on 360 camera image analysis, comprising the following specific steps: S1, collects image data from the 360 ​​camera and performs preprocessing; S2, obtain the current motion state of the vehicle, including motion direction, speed, and steering wheel angle; S3, dividing the vehicle into three motion states according to the current motion parameters; including low-speed forward state A1, high-speed forward state A2, and reverse state B; S4, using different strategies to perform feature recognition on the image data according to different motion states, judging the vehicle collision risk level based on the recognition results, and issuing a warning.

[0006] Preferably, the 360 ​​camera in step S1 includes at least one front-view camera, at least one left-view camera, at least one right-view camera, and at least one rear-view camera; the front-view camera, left-view camera, right-view camera, rear-view camera, and processing unit constitute a surround-view platform; and the processing unit pre-processes the received image information.

[0007] Preferably, the preprocessing step includes distortion correction processing, grayscale processing, denoising and data enhancement.

[0008] Preferably, in step S2, the motion state information is obtained through the ECU of the vehicle; Among them, the vehicle movement direction is divided into forward and reverse; The movement speed is limited to 15km / h. A speed lower than 15km / h is a low speed state, and a speed greater than or equal to 15km / h is a high speed state. The angle of the steering wheel is recorded according to the actual rotation angle.

[0009] Preferably, in step S3, the judgment is first made according to the direction of the vehicle's movement; if the vehicle is in the forward state A, the judgment is made according to the movement speed, with 15 km / h as the limit, a speed lower than 15 km / h is a low-speed forward state A1, and a speed greater than or equal to 15 km / h is a high-speed forward state A2; If the vehicle is in reverse state, it is reverse state B.

[0010] Preferably, the low-speed forward state A1 corresponds to the first judgment strategy; the high-speed forward state A2 corresponds to the second judgment strategy; and the reverse state B corresponds to the third judgment strategy; The first judgment strategy establishes an estimated route image area within the visual range of the front camera; the collision risk is judged by detecting the ratio of the pixel area of ​​the obstacle to the pixel area of ​​the estimated route image area through a feature recognition algorithm; The second judgment strategy adds the image content of the left-view camera and the right-view camera on the basis of the first judgment strategy, detects obstacles in the left blind spot and the right blind spot at the same time, and judges the collision risk; The third judgment strategy establishes an expected route image area within the field of view of the rear view camera, the left view camera, and the right view camera; the collision risk is judged by detecting the pixel area of ​​the obstacle and the pixel area of ​​the expected route image area through a feature recognition algorithm.

[0011] Preferably, the first judgment strategy is based on the time point of entering the strategy. T A1 The image data pre-processed by the front-view camera is collected as the starting frame, and a coordinate system is established with the center of the bottom edge of the image as the origin; the expected route image area is established in the image in combination with the steering angle of the vehicle; the YOLO algorithm is used for feature recognition in the expected route image area to identify the pixel area of ​​obstacles in the area S 0 The pixel area of ​​the region S A1 Ratio F A1 ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region S A1Ratio F 1 ; Calculate the ratio F 1 Ratio F A1 The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; Among them, T A1 is the starting frame, （T A1 +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ; In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T A1 +t) for （T A1 +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.5-2s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit and has a value between 1 and 2; Unit time t If no obstacle target is detected within （T A1 +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

[0012] Preferably, the second judgment strategy is based on the time point of entering the strategy. T A2 As the starting frame, collect the pre-processed image data of the front camera, left camera, and right camera; For the front-view camera, a coordinate system is established based on the data obtained, with the center of the bottom edge of the image as the origin; an estimated route image area is established in the image in combination with the steering angle of the vehicle; the YOLO algorithm is used for feature recognition in the estimated route image area to identify the pixel area of ​​obstacles in the area S 0 The pixel area of ​​the region S A2 Ratio F A2 ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the regionS A2 Ratio F 2 ; Calculate the ratio F 2 Ratio F A2 The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; by T A2 is the starting frame, （T A2 +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ; In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T A2 +t) for （T A2 +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.1-0.5s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit and has a value between 0.01 and 0.05; For left-view or right-view cameras; establish a coordinate system with the midpoint of the image as the origin based on the acquired data, determine the warning area in the image, and establish the expected route image area in the image to identify the pixel area of ​​obstacles in the area S 0 ; At the detection time t k within the range of t k Take 0.1-0.2s to detect the vehicle steering wheel rotation angle; if it exceeds the threshold, calculate the pixel area of ​​the obstacle at this moment S 01 Pixel area relative to the initial obstacle S 0 When the rate of change exceeds the preset value, an alarm is triggered; Unit time t If no obstacle target is detected within （T A2 +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

[0013] Preferably, the third judgment strategy is based on the time point of entering the strategy. TB As the starting frame, collect the pre-processed image data of the rear view camera, the left view camera, and the right view camera; The image data collected by the three cameras at the same time are combined into a panoramic image in a time sequence; a coordinate system is established with the center of the bottom edge of the panoramic image as the origin; an estimated route image area is established in the image in combination with the steering angle of the vehicle; the YOLO algorithm is used for feature recognition in the estimated route image area to identify the pixel area of ​​obstacles in the area S 0 The pixel area of ​​the region S B Ratio F B ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region S B Ratio F 3 ; Calculate the ratio F 3 Ratio F B The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; Among them, T B is the starting frame, （T B +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ; In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T B +t) for （T B +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.5-1s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit and has a value between 3 and 5; Unit time t If no obstacle target is detected within （T B +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

[0014] A second aspect of the present invention provides a blind spot collision determination system based on 360 camera image analysis, which performs the steps of the above method: comprising a 360 camera, a processing unit, a communication unit, a feature recognition unit, a control output unit, and an early warning system; The 360 ​​camera includes at least one front-view camera, at least one left-view camera, at least one right-view camera, and at least one rear-view camera; the front-view camera, the left-view camera, the right-view camera, the rear-view camera, and the processing unit constitute a surround-view platform; the processing unit pre-processes the received image information; The communication unit is connected to the vehicle ECU through the CAN bus to collect vehicle operation data; The feature recognition unit is pre-written with the recognition algorithm and supports OTA upgrades. It is used to identify obstacles in the picture using the trained model. The control output unit sends control instructions to the early warning system according to the results of the three judgment strategies, and controls the early warning system to send out corresponding early warning signals.

[0015] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned blind spot collision judgment method based on 360 camera image analysis are implemented.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned blind spot collision judgment method based on 360 camera image analysis.

[0017] The beneficial effects of the present invention are as follows: the present invention sets three different judgment strategies for different operating conditions; by establishing an image-based depth recognition algorithm to identify obstacles in an image, by establishing an estimated route image area in the field of view of the image, the calculation amount of the system is reduced; the risk of a vehicle collision is judged by the ratio of the area occupied by the obstacle to the area of ​​the estimated route image area, and after accurate judgment according to different preset strategies, warning information is output through the warning system, which greatly improves the safety of blind spot collision when driving a vehicle and reduces the risk of collision. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1The figure is a flow chart of a blind spot collision judgment method based on 360 camera image analysis.

[0020] Figure 2 This is a structural diagram of the blind spot collision judgment system based on 360 camera image analysis.

[0021] Figure 3 : is a system structure diagram of the 360 ​​camera in this embodiment. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0025] Example 1

[0026] This embodiment provides a blind spot collision judgment method based on 360 camera image analysis, including the following specific steps: S1, collecting image data of the 360 ​​camera and preprocessing; in step S1, the 360 ​​camera includes at least one front view camera, at least one left view camera, at least one right view camera, and at least one rear view camera; the front view camera, the left view camera, the right view camera, the rear view camera, and the processing unit constitute a surround view platform; the processing unit preprocesses the received image information. The preprocessing steps include distortion correction processing, grayscale processing, denoising, and data enhancement.

[0027] S2, obtaining the current motion state of the vehicle; including motion direction, speed, and steering wheel angle; the motion state information in step S2 is obtained through the vehicle's ECU; wherein the vehicle's motion direction is divided into forward and reverse; The movement speed is limited to 15km / h. A speed lower than 15km / h is a low speed state, and a speed greater than or equal to 15km / h is a high speed state. The angle of the steering wheel is recorded according to the actual rotation angle.

[0028] S3, dividing the vehicle into three motion states according to the current motion parameters; including low-speed forward state A1, high-speed forward state A2, and reverse state B; In step S3, the vehicle is first judged according to its moving direction; if the vehicle is in the forward state A, the moving speed is judged, with 15 km / h as the limit, and a speed lower than 15 km / h is a low-speed forward state A1, and a speed greater than or equal to 15 km / h is a high-speed forward state A2; If the vehicle is in reverse state, it is in reverse state B. The low speed forward state A1 corresponds to the first judgment strategy; the high speed forward state A2 corresponds to the second judgment strategy; the reverse state B corresponds to the third judgment strategy; The following uses a specific case to introduce this solution in detail: a. The first judgment strategy establishes an estimated route image area within the field of view of the front camera; the collision risk is determined by detecting the ratio of the pixel area of ​​the obstacle to the pixel area of ​​the estimated route image area through a feature recognition algorithm; The first judgment strategy is the time point of entering the strategy T A1 The image data pre-processed by the front-view camera is collected as the starting frame, and a coordinate system is established with the center of the bottom edge of the image as the origin; the expected route image area is established in the image in combination with the steering angle of the vehicle; the YOLO algorithm is used for feature recognition in the expected route image area to identify the pixel area of ​​obstacles in the area S 0 The pixel area of ​​the region S A1 Ratio F A1 In this embodiment, in addition to the YOLO algorithm, other visual recognition algorithms can also be used, such as a deep learning algorithm based on a convolutional network, by using the aforementioned algorithm to perform machine learning training models on common obstacles on the road to identify obstacles; common obstacles include various types of vehicles, width-limited piers, street lamp poles, fire hydrants, etc. By using the deep learning algorithm to identify the aforementioned obstacles, obstacles within the visual area can be quickly detected and early warning judgments can be made as soon as possible.

[0029] Specific: The pixel area of ​​obstacles in the identified area S 0 The pixel area of ​​the region S A1 Ratio F A1 Then, calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region SA1 Ratio F 1 ; Calculate the ratio F 1 Ratio F A1 The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; Among them, T A1 is the starting frame, （T A1 +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ; In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T A1 +t) for （T A1 +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.5-2s; v is the current speed; k The adjustment coefficient is between 1 and 2 according to the computing speed of the processing unit. For example, a video recorded by a camera at 30 frames per second is taken as an example. Assuming that the unit time is t 2 seconds (the t value is negatively correlated with the speed and positively correlated with the range of the camera's field of view; the faster the speed, the t The smaller the value, the slower the speed t The larger the value, the larger the camera's field of view. t The larger the value, the smaller it is; t The value can be manually calibrated according to different models, preferably t The speed is in the range of 1-5 seconds. v is 10km / h, calculated by the above formula Q The frame extraction frequency; N(T A1 +t) is 60, k is 2, then the calculated Q value is 2.7, which is equal to 3 when taken as an integer; that is, from T A1 The time is the starting frame, and every three frames are taken for recognition, that is, 1, 4, 7, 10 frames... are taken for recognition; calculate the ratio F 1 Ratio F A1 If the relative change rate exceeds the threshold, it is determined that there is a collision risk and an early warning is issued through the early warning system.

[0030] If in unit time t If no obstacle target is detected within （T A1 +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

[0031] In the first low-speed forward condition, the forward speed is less than 15km / h, which is generally considered to be a low-speed following vehicle or entering a parking lot or other area that requires low-speed driving. In the low-speed following vehicle condition, the forward obstacle (the vehicle in front) is generally more obvious, and the steering wheel generally does not change. The target in front is relatively fixed and occupies a large area of ​​the camera's field of view (for example, when the vehicle in front enters the field of view, it occupies 60% of the pixel area, and as the vehicle approaches, the pixel area will slowly increase to 80% (within 2-5 seconds)). In the set unit time, the ratio F 1 Ratio F A1 The relative rate of change is not too large. In addition, the driver can see the vehicle in front and can autonomously control the speed to reduce the speed or use the intelligent driving system to control the speed, making collision less likely. When entering a parking lot at a low speed, especially when turning, there may be obstacles such as short fire hydrants and pillars within the driving range, which are not easy to be found by the driver; however, in the camera's field of view, the obstacle will suddenly enter the field of view, resulting in a ratio of F 1 Ratio F A1 The relative change rate suddenly increases (the obstacle area suddenly increases from 0 to 20%), and as the distance between the obstacle and the vehicle gets closer, the relative change rate increases (the obstacle area increases from 20% to 30%). The relative change rate becomes 15% (within 0.5-1 second), which indicates that there is a risk of collision. The early warning system is used to issue early warnings to reduce the possibility of blind spot collisions.

[0032] b. The second judgment strategy adds the image content of the left-view camera and the right-view camera on the basis of the first judgment strategy, detects obstacles in the left and right blind spots at the same time, and judges the collision risk; The second judgment strategy is based on the time point of entering the strategy T A2 As the starting frame, collect the pre-processed image data of the front camera, left camera, and right camera; For the front-view camera, a coordinate system is established based on the data obtained, with the center of the bottom edge of the image as the origin; an estimated route image area is established in the image in combination with the steering angle of the vehicle; the YOLO algorithm is used for feature recognition in the estimated route image area to identify the pixel area of ​​obstacles in the areaS 0 The pixel area of ​​the region S A2 Ratio F A2 ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region S A2 Ratio F 2 ; Calculate the ratio F 2 Ratio F A2 The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; by T A2 is the starting frame, （T A2 +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ; In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T A2 +t) for （T A2 +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.1-0.5s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit, and the value is between 0.01-0.05; the front camera part of the second judgment strategy is similar to the first judgment strategy. Since only the speed changes from low speed to high speed, only t Unit time and adjustment coefficient k The value range of is revised, and the rest is the same as the first judgment strategy. For details, please refer to the relevant content in the first judgment strategy, which will not be repeated here; The specific solution for the left or right camera in the second judgment strategy is as follows: When a vehicle is traveling forward at high speed, blind spots generally exist on the left and right sides of the vehicle. When the speed difference of vehicles in different lanes in the same direction is large, it can be observed through the rearview mirror and radar system. However, if the speeds of vehicles traveling in the same direction are almost the same, there will be blind spots in the vehicle's rearview mirror and radar system. At this time, if the driver changes lanes, it is very easy to collide with other vehicles in the blind spot, posing a safety hazard. Based on this, in this embodiment, the data obtained by the left or right camera is preprocessed; a coordinate system is established with the center point of the image as the origin, a warning area is determined in the image, and an estimated route image area is established in the image to identify the pixel area of ​​obstacles in the area. S 0 ; At the detection time t k within the range of t k Take 0.1-0.2s to detect the vehicle steering wheel rotation angle; if it exceeds the threshold, calculate the pixel area of ​​the obstacle at this moment S 01 Pixel area relative to the initial obstacle S 0 When the rate of change exceeds the preset value, an alarm is triggered; Specifically: When the obstacle vehicle is in the blind spot of the vehicle, the relative position of the obstacle in the left or right view camera remains basically unchanged due to the close speed of the two vehicles. S 0 Basically unchanged; when the vehicle's steering wheel deflects, that is, when it starts to change lanes, the expected route image area in the image will change from a straight line to a curve; at this time, the pixel area occupied by the obstacle S 0 Changes are bound to occur. When the detection exceeds the threshold, it is considered that there is a risk of collision and an early warning system is used to warn you.

[0033] If the unit time t If no obstacle target is detected within （T A2 +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

[0034] In this embodiment, if the vehicle is traveling on a curved road, in order to improve the recognition accuracy, the change in the steering wheel rotation angle per unit time is recognized; the detection time t kTake 0.1-0.2s; within the detection time, if the vehicle is traveling on a curve, the change in the steering wheel angle is small, generally within 1-5%; when it is necessary to change lanes, the steering wheel will change significantly in a short period of time, generally from 0-45%. The specific data calibration can be confirmed by experiments on different models; when the change in the steering wheel angle is large, blind spot monitoring is started; if the change in the steering wheel angle is small, the system is on standby to reduce the amount of calculation and power consumption.

[0035] c. The third judgment strategy establishes an estimated route image area within the viewing area of ​​the rear view camera, the left view camera, and the right view camera; the collision risk is determined by detecting the pixel area of ​​the obstacle and the pixel area of ​​the estimated route image area through a feature recognition algorithm; The third judgment strategy is mainly used in the reversing scene. In this scene, the vehicle speed is slow, generally not exceeding 5km / h, and the steering wheel turns at a large angle. In this case, blind spots are prone to occur, resulting in vehicle scratches. Based on this, the third judgment strategy is proposed, as follows: The third judgment strategy is based on the time point of entering the strategy T B As the starting frame, collect the pre-processed image data of the rear view camera, the left view camera, and the right view camera; The image data collected by three cameras at the same time are synthesized into a panoramic image in a time sequence; there are many examples of algorithms for synthesizing panoramic images in the prior art, and the present invention does not make any improvements to this part, so it will not be repeated; a coordinate system is established with the center of the bottom edge of the panoramic image as the origin; an estimated route image area is established in the image in combination with the steering wheel angle of the vehicle; the YOLO algorithm is used to perform feature recognition in the estimated route image area to identify the pixel area of ​​obstacles in the area S 0 The pixel area of ​​the region S B Ratio F B ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region S B Ratio F 3 ; Calculate the ratio F 3 Ratio F B The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; Among them, T B is the starting frame,（T B +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ; In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T B +t) for （T B +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.5-1s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit and has a value between 3 and 5; For example, taking 30 frames per second as an example, t Take 1 second, assuming the speed v 5km / h, k Take 5; calculate according to the above formula, then （T B +t） is 30, vk+t is 26, the calculated Q The value is rounded to an integer, which is 1; it is necessary to perform feature recognition on each frame of the image to determine the pixel area of ​​the obstacle; the pixel area change degree when the obstacle appears is the same as the first judgment strategy. The threshold is set through experiments. When the detection ratio F 3 Ratio F B When the relative change rate shows a large change rate, it is considered that there is a collision risk and an early warning is issued through the early warning system.

[0036] Unit time t If no obstacle target is detected within （T B +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

[0037] S4, using different strategies to perform feature recognition on the image data according to different motion states, judging the vehicle collision risk level based on the recognition results, and issuing a warning.

[0038] In this embodiment, the vehicle can establish the estimated route image area by referring to the AR navigation module in the prior art, and plan the path in the real-time collected image screen; in addition, the following strategies can be used for optimization to reduce system power consumption; 1. Establish a recognition model for the user. When the vehicle enters the working conditions of the first judgment strategy or the third judgment strategy, the image content is identified. For users with strong periodicity, the vehicle operation is relatively regular and generally travels in fixed parking lots or routes. At this time, deep recognition is performed on the image content with a high frequency of occurrence in the video recording to establish a unique motion model for the user. When the vehicle enters the same position as the motion model scene again, such as the same parking space in the same parking lot, since the motion model records the obstacle situation of the parking space, it can be directly identified and compared, and the obstacle position can be quickly determined, thereby improving system efficiency and reducing system load.

[0039] 2. In the high-speed working condition of the second judgment strategy, in order to reduce the system load, when the vehicle is traveling at a constant speed, the blind spots are mainly concentrated on the left and right sides, and obstacles in the forward direction can be identified by other auxiliary systems on the vehicle, such as laser radar, binocular camera and other equipment; the frequency of front camera detection can be reduced; for example, with 5 seconds as a detection cycle, within the continuous 5 seconds, the judgment method of the front camera in the second judgment strategy is used to detect obstacles; if no obstacles are detected in 5 consecutive detection cycles, one detection cycle is extended to 6 seconds; and so on, the maximum detection cycle is extended to 15 seconds; if no obstacles are detected for more than 10 consecutive 15-second cycles, the detection function of the front camera can be intermittently turned on; that is, detect the first 15-second cycle, pause one detection cycle, and wait for the third 15-second cycle to start again; and so on, a maximum of 2 cycles are allowed to pause the detection; in this way, the amount of calculation in the forward direction when the vehicle is traveling at high speed can be further reduced, and the system load and power consumption can be reduced.

[0040] Example 2

[0041] A second aspect of the present invention provides a blind spot collision judgment system based on 360 camera image analysis, which performs the steps of the method in Example 1: including a 360 camera, a processing unit, a communication unit, a feature recognition unit, a control output unit, and an early warning system; The 360 ​​camera includes at least one front-view camera, at least one left-view camera, at least one right-view camera, and at least one rear-view camera; the front-view camera, the left-view camera, the right-view camera, the rear-view camera, and the processing unit constitute a surround-view platform; the processing unit pre-processes the received image information; The communication unit is connected to the vehicle ECU through the CAN bus to collect vehicle operation data; The feature recognition unit is pre-written with the recognition algorithm and supports OTA upgrades. It is used to identify obstacles in the picture using the trained model. The control output unit sends control instructions to the early warning system according to the results of the three judgment strategies, and controls the early warning system to send out corresponding early warning signals.

[0042] Example 3

[0043] This embodiment also provides a computer device, which is suitable for the steps of the blind spot collision judgment method based on 360 camera image analysis, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the steps of the blind spot collision judgment method based on 360 camera image analysis as proposed in Example 1.

[0044] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0045] Example 4

[0046] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the blind spot collision judgment method based on 360 camera image analysis described in Embodiment 1 are implemented.

[0047] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination of them, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0048] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A blind spot collision judgment method based on 360 camera image analysis, characterized in that: The specific steps include: S1, collects image data from the 360 ​​camera and performs preprocessing; S2, obtain the current motion state of the vehicle, including motion direction, speed, and steering wheel angle; S3, classifying the vehicle into three motion states according to the current motion parameters; It includes a low-speed forward state A1, a high-speed forward state A2, and a reverse state B; S4, using different strategies to perform feature recognition on the image data according to different motion states, judging the vehicle collision risk level based on the recognition results, and issuing a warning.

2. The blind spot collision judgment method based on 360 camera image analysis according to claim 1 is characterized in that: In step S1, the 360 ​​camera includes at least one front view camera, at least one left view camera, at least one right view camera, and at least one rear view camera; the front view camera, the left view camera, the right view camera, the rear view camera, and the processing unit constitute a surround view platform; The processing unit pre-processes the received image information.

3. The blind spot collision judgment method based on 360 camera image analysis according to claim 2 is characterized in that: The preprocessing steps include distortion correction, grayscale processing, denoising and data enhancement.

4. The blind spot collision judgment method based on 360 camera image analysis according to claim 1 is characterized in that: In step S2, the motion state information is obtained through the vehicle's ECU; Among them, the vehicle movement direction is divided into forward and reverse; The movement speed is limited to 15km / h. A speed lower than 15km / h is a low speed state, and a speed greater than or equal to 15km / h is a high speed state. The angle of the steering wheel is recorded according to the actual rotation angle.

5. The blind spot collision judgment method based on 360 camera image analysis according to claim 1 is characterized in that: In step S3, the vehicle is first judged according to its moving direction; if the vehicle is in the forward state A, the moving speed is judged, with 15 km / h as the limit, and a speed lower than 15 km / h is a low-speed forward state A1, and a speed greater than or equal to 15 km / h is a high-speed forward state A2; If the vehicle is in reverse state, it is reverse state B.

6. The blind spot collision judgment method based on 360 camera image analysis according to claim 2 is characterized in that: The low-speed forward state A1 corresponds to the first judgment strategy; The high-speed forward state A2 corresponds to the second judgment strategy; The reversing state B corresponds to the third judgment strategy; The first judgment strategy establishes an estimated route image area within the visual range of the front camera; The collision risk is determined by the ratio of the pixel area of ​​the obstacle detected by the feature recognition algorithm to the pixel area of ​​the image area of ​​the expected route; The second judgment strategy adds the image content of the left-view camera and the right-view camera on the basis of the first judgment strategy, detects obstacles in the left blind spot and the right blind spot at the same time, and judges the collision risk; The third judgment strategy establishes an estimated route image area within the viewing area of ​​the rear view camera, the left view camera, and the right view camera; The collision risk is determined by the ratio of the pixel area of ​​the obstacle detected by the feature recognition algorithm to the pixel area of ​​the expected route image area.

7. The blind spot collision judgment method based on 360 camera image analysis according to claim 6 is characterized in that: The first judgment strategy is the time point of entering the strategy T A1 As the starting frame, collect the image data pre-processed by the front-view camera, establish a coordinate system with the center of the bottom edge of the image as the origin; and establish the expected route image area in the image in combination with the steering angle of the vehicle; The YOLO algorithm is used to perform feature recognition in the estimated route image area to identify the pixel area of ​​obstacles in the area. S 0 The pixel area of ​​the region S A1 Ratio F A1 ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region S A1 Ratio F 1 ; Calculate the ratio F 1 Ratio F A1 The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; Among them, T A1 is the starting frame, （T A1 +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ； In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T A1 +t) for （T A1 +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.5-2s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit and has a value between 1 and 2; Unit time t If no obstacle target is detected within （T A1 +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

8. The blind spot collision judgment method based on 360 camera image analysis according to claim 6 is characterized in that: The second judgment strategy is based on the time point of entering the strategy T A2 As the starting frame, collect the pre-processed image data of the front camera, left camera, and right camera; For the front-view camera, a coordinate system is established based on the data obtained, with the center of the bottom edge of the image as the origin; an estimated route image area is established in the image in combination with the steering angle of the vehicle; the YOLO algorithm is used for feature recognition in the estimated route image area to identify the pixel area of ​​obstacles in the area S 0 The pixel area of ​​the region S A2 Ratio F A2 ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region S A2 Ratio F 2 ; Calculate the ratio F 2 Ratio F A2 The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; by T A2 is the starting frame, （T A2 +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ； In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T A2 +t) for （T A2 +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.1-0.5s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit and has a value between 0.01 and 0.05; For left-view or right-view cameras; establish a coordinate system with the center of the image as the origin based on the acquired data, determine the warning area in the image, and establish the expected route image area in the image to identify the pixel area of ​​obstacles in the area S 0 ; At the detection time t k within the range of t k Take 0.1-0.2s to detect the vehicle steering wheel rotation angle; If it exceeds the threshold, the pixel area of ​​the obstacle at this moment is calculated S 01 Pixel area relative to the initial obstacle S 0 When the rate of change exceeds the preset value, an alarm is triggered; Unit time t If no obstacle target is detected within （T A2 +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

9. The blind spot collision judgment method based on 360 camera image analysis according to claim 6, characterized in that: The third judgment strategy is based on the time point of entering the strategy T B As the starting frame, collect the pre-processed image data of the rear view camera, the left view camera, and the right view camera; The image data collected by three cameras at the same time are combined into a panoramic image in time sequence; A coordinate system is established with the bottom center of the panoramic image as the origin; an image area of ​​the predicted route is established in the image in combination with the steering angle of the vehicle; The YOLO algorithm is used to perform feature recognition in the estimated route image area to identify the pixel area of ​​obstacles in the area. S 0 The pixel area of ​​the region S B Ratio F B ; and calculate the unit time according to the movement speed t The pixel area of ​​the obstacle in the image frame S t The pixel area of ​​the region S B Ratio F 3 ; Calculate the ratio F 3 Ratio F B The relative rate of change of the collision is used to determine the risk of collision and respond to different warning signals according to the preset risk level; Among them, T B is the starting frame, （T B +t） When calculating the obstacle pixel area ratio for the termination frame, the frame extraction object is determined according to the following formula: ； In the above formula, Q is the frame extraction frequency, which is a positive integer; N(T B +t) for （T B +t） The total number of image frames in the moment; t It is the unit time, and the value range is between 0.5-1s; v is the current speed; k The adjustment coefficient is based on the computing speed of the processing unit and has a value between 3 and 5; Unit time t If no obstacle target is detected within （T B +t） The time is the new starting frame and the unit time is continued t Detect obstacles inside.

10. A blind spot collision judgment system based on 360 camera image analysis, executing the steps of the method according to any one of claims 1 to 9, characterized in that: It includes 360 camera, processing unit, communication unit, feature recognition unit, control output unit and early warning system; The 360 ​​camera includes at least one front-view camera, at least one left-view camera, at least one right-view camera, and at least one rear-view camera; the front-view camera, the left-view camera, the right-view camera, the rear-view camera, and the processing unit constitute a surround-view platform; The processing unit pre-processes the received image information; The communication unit is connected to the vehicle ECU through the CAN bus to collect vehicle operation data; The feature recognition unit is pre-written with the recognition algorithm and supports OTA upgrades. It is used to identify obstacles in the picture using the trained model. The control output unit sends control instructions to the early warning system according to the results of the three judgment strategies, and controls the early warning system to send out corresponding early warning signals.

Images