Intelligent identification and detection method for starting of preceding vehicle

Abstract

The invention relates to the technical field of image recognition, in particular to an intelligent recognition and detection method for preceding vehicle starting. The method comprises the following steps: acquiring an image of an automobile data recorder, and judging whether an automobile is in a static or moving state; extracting a plurality of view angle regions and carrying out region separation according to texture density; detecting the area smoothness to determine a motion detection sensitivity; and according to the state and sensitivity of the vehicle, front vehicle starting identification is carried out on each area, starting early warning response is sent out based on an identification result, and the detection accuracy and real-time performance are improved. According to the method, the driving video and the motion sensing data are fused, and multi-view region segmentation, adaptive sensitivity adjustment and a dynamic early warning mechanism are combined, so that high-robustness and high-precision recognition of the starting behavior of the front vehicle under various illumination and working conditions is realized, and the method has good universality and compatibility.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to an intelligent recognition and detection method for a preceding vehicle starting. BACKGROUND

[0002] The preceding vehicle starting function based on machine vision mainly functions to remind the driver to continue driving when the preceding vehicle starts to move in a stopped state of the ego vehicle, such as when waiting for a red-green traffic signal or when the vehicle is stuck in traffic. The general process is as follows: an image is obtained through a camera, the position information of the preceding vehicle is detected through a vision algorithm such as machine learning or deep learning, the position information is counted, and an alarm is sounded when the preceding vehicle is far away. There are some shortcomings in the preceding vehicle starting reminder function that is realized by simply detecting the distance of the preceding vehicle: 1. If the driver has already discovered that the preceding vehicle has started and has started the vehicle, the preceding vehicle starting function should be terminated to reduce unnecessary alarms and interfere with the normal driving of the driver. 2. Another scenario is when reversing, if there is a vehicle in front, since it is not possible to determine whether the ego vehicle is moving or the preceding vehicle is moving, the preceding vehicle starting alarm will also be triggered.

[0003] The current prior art generally uses GPS information or an acceleration sensor to obtain the motion state of the ego vehicle to solve the above-mentioned defects. However, a high-precision GPS module is costly, a low-precision GPS module still outputs 0 KM / h at a very slow speed, such as below 5 KM / h, and cannot provide accurate speed, and the use of an acceleration sensor to obtain the motion state of the ego vehicle also has precision problems, and the conditions of vehicles on the market are different, such as some vehicles have large engine vibrations that interfere with the judgment of the acceleration sensor. SUMMARY

[0004] Therefore, it is necessary to provide an intelligent recognition and detection method for a preceding vehicle starting to solve at least one of the above-mentioned technical problems.

[0005] To achieve the above-mentioned purpose, an intelligent recognition and detection method for a preceding vehicle starting, the method comprising the following steps: Step S1: obtaining a driving picture of a vehicle driving recorder; confirming the running state of the ego vehicle based on the driving picture, wherein the running state includes a static state and a moving state; Step S2: extracting a plurality of perspective regions of the driving picture, and regionally separating the perspective regions according to the texture density of each perspective region to obtain a first separated texture region and a second separated texture region; Step S3: detecting the regional smoothness of the first separated texture region and the second separated texture region to determine the current motion detection sensitivity setting; Step S4: Based on the running state of the vehicle, the first and second separated texture regions in each view area are respectively subjected to front vehicle start intelligent recognition according to the set motion detection sensitivity, and a front vehicle start warning response is performed based on the recognition result.

[0006] The present application has the following advantages: I. The real-time picture obtained by the driving recorder is segmented into multiple views in the front area of the vehicle, and the picture is separated into first and second separated texture regions based on texture density. In the two running states of the vehicle being stationary or moving, pixel density, region time difference and background feature map construction and analysis are used to achieve comprehensive coverage of the two behaviors of "significant motion" and "small start" of the front vehicle. Whether in the "dark field" condition of dim light or in the case of picture shaking during vehicle driving, the front vehicle start change signal can be accurately extracted through multi-level sensitivity adjustment and low pixel compensation strategy, ensuring the robustness and reliability of the detection.

[0007] II. In the stationary state, the mean and variance of the background feature map are first evaluated, and then the scene brightness and dynamic amplitude are judged according to the preset threshold, and a pixel compensation factor is introduced in the dark field condition to weight the original motion; in the high dynamic scene, the detection threshold can be adjusted in time to avoid misjudgment or omission. This adaptive sensitivity setting makes the front vehicle start detection neither too sensitive to small changes in light nor able to quickly capture the motion trend at the initial stage of vehicle start.

[0008] III. For different front vehicle motion states (forward, backward), the present application designs two types of warning modes, far warning and collision warning, and divides them into three warning levels according to the distance between the vehicle tail and the front end of the vehicle, the distance change rate, the acceleration and the relative speed, etc. - Level 1 prompt, Level 2 reminder, Level 3 strong reminder. In this way, not only can a slight prompt be given when the front vehicle starts slowly to avoid fatigue of the driver due to excessive warning, but also a higher level of warning can be provided when the front vehicle starts quickly or backs up, thereby minimizing the risk of rear-end collision accidents caused by delayed judgment.

[0009] IV. During the steering of the vehicle, the front vehicle existing area will change in position and shape with the shift of the field of view. The present application fuses the vehicle-mounted acceleration sensor and the wheel speed sensor to obtain the motion state of the vehicle and the steering angle of the field of view in real time, and dynamically selects and "changes" several sub-front vehicle existing areas, thereby realizing accurate re-calibration of the front vehicle area. This mechanism effectively reduces the false detection rate caused by steering, so that the front vehicle start detection is still accurate and reliable under complex road conditions.

[0010] Five, the method in the case not only considers pure video image processing, but also takes into account the acceleration sensor and the physical quantity measurement of wheel speed sensor, the measurement range and the resolution design meet the daily driving demand. The fusion of video features and motion sensing data not only enhances the system's ability to identify different scenes (such as rapid acceleration, slow start, and car following), but also provides good compatibility and scalability for deploying algorithms on various vehicle models, different brands of commercial or passenger vehicles. BRIEF DESCRIPTION OF DRAWINGS

[0011] Fig. 1 It is a step flowchart of an intelligent identification and detection method for the start of a front vehicle. Fig. 2 It is a step flowchart of an intelligent identification and detection method for the start of a front vehicle provided by an embodiment of the application. Fig. 3 It is an identification and detection diagram of an intelligent identification and detection method for the start of a front vehicle. The implementation of the object of the application, functional features and advantages will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0012] The technical method of the application will be described below in detail with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0013] 1In addition, the accompanying drawings are only schematic illustrations of the application, and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated descriptions thereof will be omitted. Some block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0014] It should be understood that although the terms "first", "second" and the like can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the exemplary embodiments, a first element can be called a second element, and similarly a second element can be called a first element. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0015] To achieve the above-mentioned object, please refer toFigs. 1 to 3 The application discloses a method for intelligently identifying and detecting a front vehicle start, and the method comprises the following steps: Step S1: acquiring a driving picture of a vehicle driving recorder; and confirming a running state of a host vehicle based on the driving picture, wherein the running state comprises a static state and a moving state; In an embodiment, a high-definition driving recorder is installed at an inner side of a front windshield of a new energy vehicle, and the driving recorder is integrated with a camera, a storage module and a data transmission module. During driving of the vehicle, the camera collects a picture of a road and target objects in a field of view in front of the vehicle at a frame rate of 30 frames per second, and the picture is recorded as driving picture data. The driving picture data is uploaded in real time to a vehicle-mounted computing unit or a cloud platform through the data transmission module connected with the camera, so as to facilitate subsequent running state analysis.

[0016] It should be noted that the collection frequency and the uploading frequency of the driving picture data are the same, and are consistent in all monitoring periods, so as to ensure time continuity and synchronization consistency of the video data.

[0017] Step S2: extracting a plurality of perspective regions of the driving picture, and performing region separation on the perspective regions according to texture density of each perspective region, to obtain a first separated texture region and a second separated texture region; In an embodiment, the driving picture data is first divided into a plurality of perspective regions, such as a left perspective region, a central perspective region and a right perspective region, by a pre-trained semantic segmentation network. The size of each perspective region can be set to, for example, 640x360 pixels, so as to facilitate subsequent processing. For each perspective region, a gray level co-occurrence matrix (GLCM) method is used to calculate its texture density feature. Specifically, the perspective region is first converted into a gray scale image, and then a gray level co-occurrence matrix is generated in two directions (horizontal and vertical) and at different gray level distances (for example, distance 1 and distance 2), and statistical features such as contrast, energy and entropy are calculated, and the texture density value of the perspective region is obtained by comprehensive calculation.

[0018] The texture density value is compared with a preset threshold T: if the texture density is greater than or equal to T, the perspective region is classified into the first separated texture region (high texture density region); and if the texture density is less than T, the perspective region is classified into the second separated texture region (low texture density region).

[0019] The threshold T can be determined by offline statistics of texture density distribution of the perspective region in a large number of road scenes, for example, a median value or an optimized value after cross-validation. If dynamic adaptation is required, T can be updated according to the mean value of the texture density of each perspective region in each monitoring period.

[0020] Finally, all the view regions belonging to the first separated texture region are combined to form a high-texture-density view set for fine detection of vehicle surrounding obstacles and road features; all the view regions belonging to the second separated texture region are combined to form a low-texture-density view set for robust processing of driving state judgment and target tracking.

[0021] In another embodiment, for example, referring to Fig. 3 , there can be vehicle occlusions in the left and right view regions, such as a high bus or truck. The middle region can be a less-textured sky, such as a blue sky or a dark night sky. Therefore, the respective states need to be analyzed to accurately determine the ego vehicle state, and thus each field of view region can be divided into two parts according to the texture region, as the first separated texture region and the second separated texture region, to further improve the sensitivity, such as the upper half being a non-textured sky.

[0022] Step S3: detecting the region smoothness of the first separated texture region and the second separated texture region to determine the current motion detection sensitivity setting; In an embodiment, the grayscale image of each separated texture region is pre-smoothed by a Gaussian filter (σ = 1.0, kernel size 3x3) to remove noise interference; then, the Laplacian operator is applied to the pre-smoothed region respectively to obtain the edge response map of the region; finally, the pixel value variance of the edge response map of the region is calculated, denoted as the smoothness index , The smaller it is, the smoother the region is, and the larger it is, the coarser the texture of the region is.

[0023] The smoothness index of the first separated texture region and the smoothness index of the second separated texture region are compared with a preset smoothness threshold to determine the current motion detection sensitivity level : if and , the region is overall smooth, and a high sensitivity = "high" is set; if and (or and ), the texture region is mixed, and a medium sensitivity = "medium" is set; if and , the region is coarse, and a low sensitivity = "low" is set. Wherein, the experience value 0.005 (based on the normalized result of the Laplacian response variance) can be taken, 0.02; also can be obtained by offline statistics of a large number of driving scenes distribution and cross-validation optimization.

[0024] In an implementation of the embodiment, the correspondence between the motion detection sensitivity and the sensitivity parameter : wherein, = 0.8, = 0.5, = 0.2, for subsequent background modeling and target extraction algorithms in the motion detection module, to ensure more sensitive capture of small motions in smooth scenes, and to suppress false alarms in complex texture scenes.

[0025] It should be noted that, in order to cope with different road conditions and light changes, the historical and sequences can be statistically analyzed by a sliding window at the end of each monitoring period, and the and parameters can be dynamically updated to achieve adaptive motion detection sensitivity setting.

[0026] Step S4: Based on the running state of the ego vehicle, the first and second separated texture regions in each perspective region are respectively subjected to pre-vehicle start intelligent recognition according to the set motion detection sensitivity, and pre-vehicle start warning response is performed based on the recognition result.

[0027] In an embodiment, for each separated texture region, Gaussian Mixture Model (GMM) background modeling and frame difference method can be combined to generate a motion mask ; according to the sensitivity parameter , a corresponding motion pixel threshold (e.g. , , ) is selected, and is binarized to obtain a final binary motion region . For the binary motion region , the motion pixel ratio = (number of motion pixels / total number of region pixels) is calculated. And is compared with a preset start detection threshold (such as = 0.03): if , it is determined that there is pre-vehicle start motion in the separated texture region; if , it is considered that there is no start motion in the region.

[0028] For each viewpoint region, the detection results of its first separated texture region are statistically analyzed. Detection results of the second separated texture region The following rules are used to comprehensively judge the starting of the vehicle in front: That is, as long as any texture area detects starting motion, it can be determined that there is a vehicle starting in front in that view area.

[0029] After completing the start-up recognition of all viewing areas, the system makes a final warning decision based on the vehicle's operating status: when the vehicle is stationary or traveling at low speed (speed ≤ 5 km / h), and any viewing area R_{view} = true, the preceding vehicle start-up warning response is triggered, including: displaying a visual warning on the instrument panel or head-up display (HUD); emitting a buzzer warning sound; and sending the "preceding vehicle start-up" signal to the upper-level driver assistance system via the vehicle's CAN bus to assist in achieving automatic stop / go control.

[0030] As an example of the present invention, reference is made to... Fig. 2 As shown, in this example, step S1 includes: Step S11: Based on the driving screen, confirm the geometric vertices of the front vehicle frame, and connect each geometric vertices in sequence to obtain the area where the front vehicle exists; Step S12: Filter the horizon of the driving image, and separate the non-front vehicle view area of ​​the driving image based on the vertical center line of the area where the horizon and the front vehicle exist, to obtain the sky view area image. Step S13: For each pixel in each sky view area image, determine the pixel field of each pixel based on the parity of the sum of the pixel's row number and column number, where the pixel field includes an odd field and an even field. Step S14: Calculate the current image feature value of each pixel in the sky view region image based on the pixel field; Step S15: Construct a background feature map using the current image feature values, and confirm the vehicle's operating status by the degree of change in the background feature map.

[0031] In an embodiment, since the vehicle's driving recorder is obtained, and in the driving recorder picture, the sky usually occupies about half of the picture, that is, the horizon is approximately in the middle of the picture, so the left, middle and right regions of the sky can be intercepted as blank visual field regions, and the left and right regions may be blocked by vehicles such as high buses and trucks. The middle region may be a less textured sky, such as a blue sky or a dark night sky. Therefore, the respective states need to be analyzed to accurately determine the state of the vehicle, and whether each region is moving is determined by background modeling. However, traditional background modeling, such as optical flow method, assumes constant illumination, but on the road, the light changes complexly, and there are also raindrops and windshield wiper interference in rainy days. The present scheme adapts a template background modeling of odd and even fields to construct a background feature map for such complex scenes.

[0032] In another embodiment, for large illumination changes, raindrops and windshield wiper interference, 16 groups of 3x3 operators can be set The operators are used alternately in odd and even fields to obtain the current image feature in_feat. Wherein The operator parameters are as follows: The above alternately using operators in odd and even fields can be specifically demonstrated by code logic, for example, X in (0, w) y in (0, h); if (x+y)&0b01==False determine odd and even fields, that is, when x is an even field, y is an even field. When x is an odd field, y is an odd field. Wherein, Y&0b11 has 0, 1, 2, 3 four results, and x&0b11 has 0, 1, 2, 3 four results. One common 16 combinations, alternately use the above parameters. The code logic of the calculation formula is as follows: if (y&0b11) == 0: if (x&0b11) == 0: in_feat = ; if (x&0b11) == 1: in_feat = ; if (x&0b11) == 2: in_feat = ; if (x&0b11) == 3: in_feat = ; if (y&0b11) == 1: if (x&0b11) == 0: in_feat = ; if (x&0b11) == 1: in_feat = ; if (x&0b11) == 2: in_feat = ; if (x&0b11) == 3: in_feat = ; if (y&0b11) == 2: if (x&0b11) == 0: in_feat = ; if (x&0b11) == 1: in_feat = ; if (x&0b11) == 2: in_feat = ; if (x&0b11) == 3: in_feat = ; if (y&0b11) == 3: if (x&0b11) == 0: in_feat = ; if (x&0b11) == 1: in_feat = ; if (x&0b11) == 2: in_feat = ; if (x&0b11) == 3: in_feat = .

[0033] In another embodiment, in the front vehicle detection module, the outer contour of the front vehicle in the driving picture is first extracted, and an edge detection algorithm is used to obtain key geometric vertices of the frame of the front vehicle. Then, according to the clockwise or counterclockwise order of the vertices in the image coordinate system, the geometric vertices are sequentially connected to form a front vehicle existing area, denoted as area P. A straight line detection method based on Hough transform is used to screen the horizon in the driving picture. After the horizon L is determined, the vertical middle line M of the horizon L and the front vehicle existing area P is calculated, and the middle line M is used as a segmentation boundary to separate the front vehicle area in the driving picture from the sky and distant view area, to obtain a sky view area image, denoted as image S.

[0034] For each pixel in the image S, the sum of its row number i and column number j in the image coordinate system, i.e. (i+j), if (i+j) is even, the pixel belongs to even field; if (i+j) is odd, the pixel belongs to odd field. Through this parity field division method, the pixel field attribute of each pixel in the image S is labeled. Based on the pixel field attribute combined with the multi-dimensional information such as pixel gray scale, color saturation and gradient amplitude, the current image feature value F(i,j) of each pixel in the sky view area image S is calculated, and a feature matrix F is formed. The feature matrix F is time-series accumulated and fused to construct a background feature map B; by analyzing the change degree of the background feature map B in the continuous monitoring period, the stability or dynamic fluctuation of the feature map B is identified, so as to determine the running state of the ego vehicle, including static, slow or acceleration state.

[0035] Preferably, the background feature map is constructed by the current image feature value, and the running state of the ego vehicle is confirmed by the change degree of the background feature map, including: The region time-series difference map of the sky view area image is obtained, and the background feature map is constructed by the current image feature value; According to the sum of the effective difference values of all pixels in the background feature map, and dividing the sum by the total number of pixels in the region, the result is recorded as the first mean value of the sky view area, wherein the first mean value is used to represent the average level of image difference value of the region; The difference between the effective difference value of each pixel in the background feature map and the first mean value is calculated, the square of the difference is calculated, and the square values of all pixel differences are accumulated and then divided by the total number of pixels in the sky view area image, and the result is recorded as the second mean value, wherein the second mean value is used to represent the fluctuation degree of the image difference value of the region; The second mean value is compared with the preset pixel mean value, when the second mean value is greater than or equal to the preset pixel mean value, the running state of the ego vehicle is confirmed based on the pixel change difference of the region time-series difference map.

[0036] In one embodiment, the system continuously captures two images of the sky view region in two adjacent time periods, and performs pixel-level difference between the two images to generate a region time-difference image. In the difference image, the value of each pixel reflects the change of brightness or color at that position between the two adjacent frames. Then, the system fuses the feature values calculated in the previous step with the difference image to obtain a complete background feature map. The background feature map contains both the change intensity of each pixel over time and the field attribute and multi-dimensional image feature information of the pixel. Then, the system sums all the pixel change values in the background feature map, and divides the sum by the number of pixels in the sky view region to obtain a value that reflects the average change level of the entire region in the two time periods. In order to characterize the fluctuation degree of pixel change, the system also squares the difference between each pixel change value and the average change level, and then divides the sum of all pixel squared differences by the total number of pixels to obtain a value that represents the fluctuation strength of the region. The larger the value, the more intense the fluctuation of pixel brightness or color in the sky region over time. Finally, the system compares this fluctuation strength value with a pre-set threshold value. When the fluctuation value is greater than or equal to the threshold value, it indicates that there is a significant difference in pixel change in the sky region, for example, due to the rapid change of camera view or background environment caused by the vehicle in motion, and the system can determine that the vehicle is in motion; otherwise, it is considered that the vehicle is in a stationary or slow-moving state. Through this method based on statistical analysis of background feature map values, the system can robustly identify the running state of the vehicle under various lighting and background conditions.

[0037] In another embodiment, for example, there is a three-row and three-column pixel region, and the difference values of the adjacent two frames of images are as follows: the three difference values of the first row are 2, 3, and 1 in turn; the three difference values of the second row are 0, 2, and 4 in turn; and the three difference values of the third row are 3, 1, and 2 in turn. First, the nine difference values are added together to obtain a total sum of 18, and the total sum 18 is divided by the number of nine pixels 9 to obtain an average difference of 2.0 per pixel. Next, the square of the difference between each pixel and the average value is calculated, and the square values are added together and then divided by 9: the first pixel of the first row: the difference 2 is the same as the average value 2, so the difference between the two is 0, and after squaring, it is still 0; the second pixel of the first row: the difference 3 is 1 greater than the average value 2, and after squaring, it is 1; the third pixel of the first row: the difference 1 is 1 less than the average value 2, and after squaring, it is 1; the first pixel of the second row: the difference 0 is 2 less than the average value 2, and after squaring, it is 4; the second pixel of the second row: the difference 2 is the same as the average value 2, and after squaring, it is 0; the third pixel of the second row: the difference 4 is 2 greater than the average value 2, and after squaring, it is 4; the first pixel of the third row: the difference 3 is 1 greater than the average value 2, and after squaring, it is 1; the second pixel of the third row: the difference 1 is 1 less than the average value 2, and after squaring, it is 1; and the third pixel of the third row: the difference 2 is the same as the average value 2, and after squaring, it is 0. All the square values are added together to obtain 0+1+1+4+0+4+1+1+0=12, and 12 is divided by the number of nine pixels 9, and the result is about 1.33. Assuming that the set fluctuation threshold is 1.0, when the calculated 1.33 is greater than or equal to 1.0, it is determined that the vehicle is in a moving state; if it is less than 1.0, it is determined that the vehicle is in a stationary or slow-moving state.

[0038] Preferably, the pixel change difference based on the region time sequence difference map confirms the running state of the vehicle, including: calculating the sum of all pixel values in the region time sequence difference map, and dividing by the total number of pixels to obtain the original motion amount; comparing the original motion amount with the pre-set motion detection threshold: if the original motion amount is greater than the motion detection threshold, it is determined that significant motion is detected in the corresponding sky view angle region, and it is further determined that the vehicle is in a driving state; otherwise, it is determined that no significant motion is detected in the corresponding sky view angle region, and the vehicle is in a stationary state.

[0039] In an embodiment, the vehicle-mounted computing unit receives driving picture data from the vehicle recorder, and generates a region time sequence difference map according to a pre-set time window. The region time sequence difference map is obtained by performing gray difference on pixel points in a specified view angle region of two continuous frames of video images, to obtain the gray change value of each pixel point; and the gray change values of all pixel points are superimposed, to obtain the difference map in the time window.

[0040] In another embodiment, the time window can be set as two frame intervals (i.e. corresponding to 33 ms in a 30fps video stream), and the image resolution output by the dashcam is 1920x1080. The region-wise temporal difference is only for the sky view region, which is framed by a pre-calibrated geometric vertex in the driving picture, ensuring that only sky background motion is detected.

[0041] According to the region-wise temporal difference map, the original motion amount is calculated : ; wherein, is the total number of pixels in the sky view region, is the gray scale difference value of the i-th pixel. A preset motion detection threshold is set . In this embodiment, the experiment calibration sets the to 8 (gray scale difference mean value). After each time window ends, the original motion amount calculated is compared with the threshold : if , it is determined that the sky view region detects significant motion, and it is further confirmed that the ego vehicle is in the driving state; , it is determined that the sky view region does not detect significant motion, and it is confirmed that the ego vehicle is in the stationary state.

[0042] To improve the stability of the decision, in this embodiment, the motion amount results of the continuous three time windows are simply filtered: when at least 2 times in 3 frames are judged as "driving", the final output is "driving state"; when at least 2 times in 3 frames are judged as "stationary", the final output is "stationary state".

[0043] Preferably, when the second mean value is less than the preset pixel mean value, it further comprises: When the second mean value is less than the preset pixel mean value, the number of all effective pixels with a difference value below the average level in the background feature map is counted, and divided by the total number of pixels to obtain a sensitivity coefficient under dark field conditions; According to the sensitivity coefficient, the pixel compensation amount of the second mean value confirming motion is obtained; The pixel compensation amount is used to weight the original motion amount.

[0044] ​In an embodiment, when the second average value is less than the preset pixel average value, it indicates that the vehicle is in a low brightness environment, such as a tunnel, and thus the method further provides a pixel compensation amount based on the sensitivity to calculate the motion, so as to value the original motion amount, so as to reduce the missed detection of dark scenes. Specifically, the number of pixels with a gray scale difference value lower than the average value in the sky view area is counted, and the number is subjected to ratio operation with the total number of pixels in the area, to obtain a sensitivity coefficient reflecting the influence of the current scene brightness deficiency on the motion detection.

[0045] The calculated sensitivity coefficient is used to determine the pixel motion amount that needs to be compensated in low light conditions. Specifically, based on the sensitivity coefficient, the vehicle-mounted computing unit dynamically determines a pixel compensation amount, which is positively correlated with the sensitivity coefficient: the greater the sensitivity coefficient, the higher the compensation amount. Finally, the pixel compensation amount is applied to the weighting processing of the original motion amount, for enhancing the underestimation of the motion amount caused by image noise or uneven brightness in dark field conditions. The weighted motion amount is used for subsequent motion detection judgment, so as to improve the running state recognition accuracy under different light conditions.

[0046] Preferably, based on the stationary state of the vehicle, the first separated texture region and the second separated texture region in each view angle region are respectively subjected to intelligent identification of the start of the preceding vehicle according to the set motion detection sensitivity, and the start of the preceding vehicle is warned and responded based on the identification result, including: When the vehicle is in a stationary state, the pixel density of the first separated texture region and the second separated texture region in each view angle region is calculated according to the set motion detection sensitivity, so as to screen the optimal texture change region; The linear situation of the movement change of the preceding vehicle is confirmed based on the pixel density of the texture change region and the area area change degree of the region where the preceding vehicle exists; When the linear situation of the movement change of the preceding vehicle is positively correlated, it is confirmed that the running state of the preceding vehicle is the forward state; When the linear situation of the movement change of the preceding vehicle is negatively correlated, it is confirmed that the running state of the preceding vehicle is the backward state; The start of the preceding vehicle is respectively subjected to hierarchical warning according to the running state of the preceding vehicle, wherein the hierarchical warning includes distance warning and collision warning.

[0047] In an embodiment, the vehicle-mounted computing unit first judges that the vehicle is in a stationary state, and then calls the start detection module of the preceding vehicle. The module divides the driving picture into a plurality of view angle regions in advance, and extracts two types of separated texture regions in each view angle region, which are respectively referred to as the first separated texture region and the second separated texture region. For the above two texture regions, the pixel density is counted in real time according to the set motion detection sensitivity.

[0048] The pixel density statistics are performed at a frequency of twice per second, and the specific process includes: scanning all pixel points in the target texture region, and counting the number of pixel points with significant differences from the background; performing ratio operation on the number and the total number of pixels in the perspective region to filter out the texture change region with the most prominent changes as the event trigger region.

[0049] For the filtered texture change region, the vehicle-mounted computing unit synchronously tracks the area change of the front vehicle existing region, and judges the linear change condition through the area increase / decrease trend of continuous multiple frames. If the texture region density change and the front vehicle region area present a same-direction increase / decrease trend, it is determined that the front vehicle moves forward; if it presents an opposite trend, it is determined that the front vehicle moves backward.

[0050] According to the moving direction of the front vehicle and the relative distance from the ego vehicle, the system triggers a hierarchical warning response: 1. When the front vehicle is determined to move forward and the relative distance is within the warning threshold, a warning of moving away is generated, and the driver is reminded to pay attention through the instrument panel text prompt and voice broadcast; 2. When the front vehicle continues to move and the distance reaches the collision warning threshold, a collision warning is generated, in addition to the instrument panel and voice prompt, the steering wheel vibration or seat shaking can also be triggered synchronously to cause the driver to brake immediately.

[0051] In one implementation manner of the embodiment, the above motion detection sensitivity can be automatically corrected according to the environmental light and the road conditions, and the sensitivity of all perspective regions and texture types is ensured to be consistent, so as to avoid misjudgment caused by a single region or a single texture feature.

[0052] Preferably, the front vehicle start intelligent identification of the first separated texture region and the second separated texture region in each perspective region based on the moving state of the ego vehicle according to the set motion detection sensitivity further includes: When the ego vehicle is in a moving state, the motion state of the ego vehicle is acquired; Based on the motion state of the ego vehicle, the area change degree of the first separated texture region and the second separated texture region in the perspective region is calculated according to the set motion detection sensitivity; The ego vehicle running steering condition is confirmed according to the area change degree of the first separated texture region and the second separated texture region in each perspective region; The front vehicle existing region is updated through the ego vehicle running steering condition; The front vehicle start hierarchical warning is closed by using the updated front vehicle existing region and the area change degree of the front vehicle existing region.

[0053] In an embodiment, when the ego vehicle is in a moving state, the vehicle-mounted computing unit obtains the current motion state data including the vehicle speed, acceleration and lateral steering angle through the vehicle-mounted sensors (such as wheel speed sensors or inertial measurement units), denoted as motion state information; the motion detection sensitivity is determined by the driver or system preset parameters. According to the obtained motion state information and motion detection sensitivity, the area change degree of the first separated texture region and the second separated texture region in each perspective region is calculated respectively. Specifically, the vehicle-mounted computing unit tracks the pixel boundary of the texture region in the consecutive two image frames, and counts the increase and decrease percentage of the total number of region pixels to reflect the expansion or contraction of the texture region.

[0054] The area change degrees of the first separated texture region and the second separated texture region of each perspective region are compared, and the running steering condition of the ego vehicle is confirmed in combination with the lateral steering angle and vehicle speed information of the ego vehicle. When the expansion direction of the texture region is consistent with the steering direction of the ego vehicle and the change degree exceeds the preset threshold, it is determined as same direction steering; otherwise, it is determined as straight running or reverse steering. According to the confirmed running steering condition of the ego vehicle, the front vehicle existing region is dynamically updated: when the same direction steering, the system translates the front vehicle existing region in the corresponding side in the current frame and matches the moderate enlargement; when the straight running or reverse steering, the front vehicle existing region is kept or shrunk to ensure the tracking accuracy.

[0055] The updated front vehicle existing region and the area change degree of the region in the continuous monitoring period thereof are used to make a closing judgment on the triggered front vehicle start grading early warning: when the area change rate after updating is continuously lower than the closing threshold and the motion state of the ego vehicle is continuously stable, the system automatically terminates the away warning and collision warning response.

[0056] Preferably, the detection of the perspective region is specifically: whether the perspective region is a vehicle is detected by a machine learning method, and if it is a vehicle, the corresponding perspective region is cancelled.

[0057] In an embodiment, for the perspective region to be detected, image or video frame data in the region is collected, and the image resolution is ensured to be not less than 1280x720 pixels to ensure the accuracy of the detection. In the preprocessing stage, the image is normalized and noise is filtered, a Gaussian filter is used to remove random noise in the image, and the brightness and contrast are automatically adjusted to make the image quality meet the input requirements of the machine learning model.

[0058] Subsequently, vehicle detection can be performed using a trained deep convolutional neural network (CNN) model. The CNN model is trained based on a public vehicle detection dataset (such as COCO or KITTI) and includes multiple convolutional layers, pooling layers, and fully connected layers that can automatically extract vehicle features in the perspective region. After the image input to the model is subjected to multi-layer feature extraction, the corresponding detection result is output, including the probability value of the vehicle class and the bounding box coordinates thereof in the image.

[0059] According to the output probability value, the vehicle detection threshold is set to 0.7 (i.e., the probability that the model judges that the region contains a vehicle needs to be greater than or equal to 0.7), and it is determined whether the perspective region contains a vehicle. If the probability value meets or exceeds the threshold, the perspective region is marked as a vehicle region. Then, the system automatically cancels the subsequent processing and analysis of the perspective region, and removes it from the list of valid perspective regions, avoiding interference of the vehicle region with subsequent tasks.

[0060] The entire detection process supports batch processing, and vehicle detection is performed sequentially for multiple perspective regions, with a detection speed maintained at more than 30 frames per second, meeting real-time detection requirements. The system records all detection results and state changes of the corresponding perspective regions through logs, facilitating subsequent auditing and performance analysis.

[0061] Preferably, the front vehicle existing region is updated by the running steering condition of the ego vehicle, including: a plurality of sub-front vehicle existing regions are screened by the running steering condition of the ego vehicle; a field of view deviation calculation is performed on the front vehicle existing region and the plurality of sub-front vehicle existing regions to obtain a steering angle of the field of view of the ego vehicle; the plurality of sub-front vehicle existing regions are changed according to the relative position change trend of the steering angle of the field of view of the ego vehicle and the front vehicle existing region, and the front vehicle existing region is re-calibrated using the change result to update the front vehicle existing region.

[0062] In an embodiment, the vehicle-mounted computing unit obtains the running steering condition of the ego vehicle in real time based on the steering information collected by the steering wheel angle sensor and the gyroscope of the vehicle, and dynamically updates the original front vehicle existing region in the front image in combination with the current driving picture. First, according to the spatial position of the front vehicle existing region in the current frame image, a plurality of possible sub-front vehicle existing regions are divided in the image as new position candidate regions where the front vehicle may appear. Second, the original front vehicle existing region and each sub-front vehicle existing region are subjected to image field of view deviation calculation to obtain the steering angle of the field of view of the ego vehicle. The deviation calculation is based on the pixel displacement between consecutive frame images and the imaging geometry of the camera to determine whether the image perspective has been horizontally deviated, and is calibrated in combination with the sensor steering data.

[0063] Then, according to the acquired self-vehicle visual field turning angle and the position change trend of the original front vehicle existing region, a sub-front vehicle existing region most conforming to the current turning behavior is selected as a new front vehicle existing region. Finally, the vehicle-mounted computing unit uses the selected sub-front vehicle existing region to perform region re-labeling on the front vehicle region in the current image, updates the position and contour information of the front vehicle existing region, so that the system can still accurately track and identify the front vehicle target during the self-vehicle visual field deviation.

[0064] In one implementation manner of the embodiment of the application, the image update frequency is consistent with the camera frame rate, and the region update delay is controlled within 100 milliseconds to meet the real-time requirement. The updated front vehicle existing region can be used for subsequent front vehicle running state judgment and early warning processing.

[0065] Especially important is that analyzing the relative position change trend of the self-vehicle visual field turning angle and the front vehicle existing region also includes: extracting the visual field boundary of the driving picture to obtain the self-vehicle visual field turning angle; performing inter-frame tracking on the front vehicle existing region to generate time sequence position information data of the front vehicle existing region; based on the self-vehicle visual field turning angle and the time sequence position information data of the front vehicle existing region, calculating the relative position change curve between them to generate visual field-front vehicle relative position change trend data; performing change mode recognition on the visual field-front vehicle relative position change trend data, extracting the front vehicle relative motion trend feature, and obtaining the front vehicle relative position change trend.

[0066] In one embodiment, in order to realize accurate updating of the front vehicle existing region, the vehicle-mounted computing unit needs to further analyze the relative position change trend between the self-vehicle visual field turning angle and the front vehicle existing region. The left and right visual field boundaries are extracted from the current frame image of the driving recorder, and the self-vehicle visual field turning angle corresponding to the current image frame is calculated in combination with the vehicle-mounted gyroscope and steering wheel turning angle data. The angle reflects the degree of deviation of the current driving direction of the self-vehicle. Secondly, the system performs image tracking processing on the front vehicle existing region in the continuous frame images. The feature point matching and target boundary tracking algorithm is used to extract the continuous spatial position change of the front vehicle in the image sequence, and time sequence position information data of the front vehicle existing region is generated.

[0067] Based on the extracted time-series position information data of the self-vehicle field of view steering angle and the front vehicle existing area, the relative position change between the two is calculated, a field of view-front vehicle relative position change curve changing over time is constructed, and corresponding change trend data is generated. The change trend data is processed by change pattern recognition. By analyzing the slope change, trend direction and continuity of the curve, the relative motion characteristics of the front vehicle in the self-vehicle field of view are extracted, including stable keeping, gradually deviating, gradually approaching, etc. Thus, the trend conclusion of the relative position change of the front vehicle is obtained.

[0068] The front vehicle relative position change trend data is an important input for subsequent front vehicle existing area dynamic adjustment and early warning judgment, ensuring that the system still has high robustness and high recognition accuracy in complex driving states such as vehicle turning and lane changing.

[0069] Especially important is that the inter-frame tracking of the front vehicle existing area also includes: Based on the front vehicle existing area, the frame image set of the driving picture is screened to obtain continuous video frame images, wherein the continuous video frame images include a front frame image and a rear frame image; The front frame image is grayed, and the bounding box coordinates of the front vehicle existing area are extracted to generate front vehicle initial bounding box data; Based on the front vehicle initial bounding box data, a feature descriptor of the front vehicle area is constructed to generate front vehicle area feature template data; The rear frame image is matched by a sliding window, and the similarity is calculated based on the front vehicle area feature template data to generate inter-frame matching similarity data; According to the inter-frame matching similarity data, the optimal matching position is selected, and the bounding box coordinates of the front vehicle existing area in each row driving picture are recorded to generate time-series position information data of the front vehicle existing area.

[0070] In an embodiment, from the continuous driving picture, the front frame and the rear frame image with a time interval of less than 50 milliseconds are selected to ensure the target motion continuity. The front frame image is first grayed, and the cv2.cvtColor() function in the OpenCV library is used to convert the color image to a gray image. Then, the target detection algorithm (such as YOLOv5 or a custom lightweight detection network) is used to locate the front vehicle position and extract its bounding box coordinates (such as the upper left corner and the lower right corner points), which are recorded as initial bounding box data.

[0071] After obtaining the initial bounding box, the system extracts image features in the region, and uses the ORB (Oriented FAST and Rotated BRIEF) algorithm to obtain the key points and descriptors in the front vehicle region. In the specific operation, the feature extractor is created by calling the cv2.ORB_create() function, and the feature descriptors of the front vehicle region are extracted by using the detectAndCompute() function as the feature template data of the front vehicle region.

[0072] A sliding window is constructed in the region adjacent to the position where the front vehicle appears in the rear frame image, and the window size is equal to the size of the front vehicle bounding box. The sliding step is set to 4 pixels to ensure matching accuracy. In each window position, the ORB feature descriptors of the region are extracted and matched with the front vehicle feature template data, and the Hamming distance is used as the similarity measure. The similarity is performed by the BFMatcher function in OpenCV, and the higher the matching score, the more similar the two regions.

[0073] The similarity scores of all sliding window positions are sorted, and the position with the highest matching degree is selected as the new position of the front vehicle in the current frame, and its bounding box coordinates are recorded. This position information is cached in the memory through the queue method to form a set of trajectory data of the front vehicle position changing over time in a series of consecutive frames, i.e., the time sequence position information data of the front vehicle existing region.

[0074] Therefore, regardless of the point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the application being defined by the appended claims rather than the above description, and it is intended to encompass all variations falling within the meaning and range of equivalents of the elements of the application file. The scope of the application is therefore defined by the appended claims rather than the above description, and it is intended to encompass all variations falling within the meaning and range of equivalents of the elements of the application file.

[0075] The above description is only a specific implementation of the present application, enabling those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligently identifying and detecting a front vehicle start-up, characterized in that, The method comprises the following steps: Step S1: acquiring a driving picture of a vehicle-mounted camera; determining a running state of the ego vehicle based on the driving picture, wherein the running state comprises a static state and a moving state; Step S2: extracting a plurality of perspective regions from the driving picture, and performing region separation on each perspective region according to the texture density of the perspective region to obtain a first separated texture region and a second separated texture region; Step S3: detecting the region smoothness of the first separated texture region and the second separated texture region to determine a current motion detection sensitivity setting; Step S4: performing intelligent identification of a preceding vehicle start based on the running state of the ego vehicle and the set motion detection sensitivity for the first separated texture region and the second separated texture region in each perspective region, and performing a preceding vehicle start warning response based on the identification result.

2. The method of claim 1, wherein, The determination of the running state of the ego vehicle based on the driving picture comprises: determining geometric vertices of a preceding vehicle frame based on the driving picture, and connecting the geometric vertices in sequence to obtain a preceding vehicle existing region; screening a horizon line of the driving picture, and separating a non-preceding vehicle perspective region of the driving picture based on the horizon line and a vertical middle line of the preceding vehicle existing region to obtain a sky perspective region image; for each pixel in each sky perspective region image, determining a pixel field of each pixel according to the parity of the sum of the row number and the column number of the pixel, wherein the pixel field comprises an odd field and an even field; calculating a current image feature value of each pixel in the sky perspective region image according to the pixel field; constructing a background feature map through the current image feature value, and determining the running state of the ego vehicle through the change degree of the background feature map.

3. The method of claim 2, wherein the method further comprises: The determination of the running state of the ego vehicle through the current image feature value and the background feature map comprises: obtaining a region time sequence difference map of the sky perspective region image, and constructing a background feature map through the current image feature value; calculating a first mean value of the sky perspective region by calculating the sum of the effective difference values of all pixels in the background feature map, dividing the sum by the total number of pixels in the region, wherein the first mean value is used to represent the average level of image difference values of the region; calculating the difference between the effective difference value of each pixel in the background feature map and the first mean value, squaring the difference, and then accumulating the difference square values of all pixels, and dividing the accumulated value by the total number of pixels in the sky perspective region image to obtain a second mean value, wherein the second mean value is used to represent the fluctuation degree of image difference values of the region; comparing the second mean value with a preset pixel mean value, and determining the running state of the ego vehicle based on the pixel change difference of the region time sequence difference map when the second mean value is greater than or equal to the preset pixel mean value.

4. The method of claim 3, wherein the method further comprises: The determination of the running state of the ego vehicle based on the pixel change difference of the region time sequence difference map comprises: calculating the sum of all pixel values in the region time sequence difference map, and dividing the sum by the total number of pixels to obtain an original motion amount; comparing the original motion amount with a preset motion detection threshold value: if the original motion amount is greater than the motion detection threshold value, it is determined that significant motion is detected in the corresponding sky perspective region, and then it is determined that the ego vehicle is in a moving state; otherwise, it is determined that no significant motion is detected in the corresponding sky perspective region, and the ego vehicle is in a static state.

5. The method of claim 4, wherein, When the second average value is less than the preset pixel average value, further comprising: When the second average value is less than the preset pixel average value, counting the number of pixels with effective difference value below the average level in the background feature map, and dividing by the total number of pixels to obtain a sensitivity coefficient under dark field conditions; According to the sensitivity coefficient, confirming the pixel compensation amount of the second average value of the moving pixels; Using the pixel compensation amount to weight the original motion amount.

6. The method of claim 1, wherein, Based on the stationary state of the ego vehicle, the first and second separated texture regions in each visual angle region are respectively identified as the front vehicle starting intelligent identification based on the set motion detection sensitivity, and the front vehicle starting warning response based on the identification result includes: When the ego vehicle is in a stationary state, the first and second separated texture regions in each visual angle region are respectively calculated based on the set motion detection sensitivity to screen the optimal texture change region; Based on the pixel density of the texture change region and the area change degree of the front vehicle existing region, the moving change linearity of the front vehicle is confirmed; When the moving change linearity of the front vehicle is positively correlated, it is confirmed that the running state of the front vehicle is forward; When the moving change linearity of the front vehicle is negatively correlated, it is confirmed that the running state of the front vehicle is backward; According to the running state of the front vehicle, the front vehicle starting graded warning is carried out, wherein the graded warning includes distance warning and collision warning.

7. The method of claim 6, wherein the method further comprises: Based on the stationary state of the ego vehicle, the first and second separated texture regions in each visual angle region are respectively identified as the front vehicle starting intelligent identification based on the set motion detection sensitivity, and the front vehicle starting warning response based on the identification result includes: When the ego vehicle is in a moving state, the motion state of the ego vehicle is obtained; Based on the motion state of the ego vehicle, the area change degree of the first and second separated texture regions in the visual angle region is calculated based on the set motion detection sensitivity; According to the area change degree of the first and second separated texture regions in each visual angle region, the ego vehicle running steering situation is confirmed; The front vehicle existing region is updated through the ego vehicle running steering situation; The front vehicle starting graded warning is closed by using the updated front vehicle existing region and the area change degree of the front vehicle existing region.

8. The method of claim 1, wherein, The detection of the visual angle region is specifically: whether the visual angle region is a vehicle is detected by a machine learning method, and if it is a vehicle, the corresponding visual angle region is cancelled.

9. The method of claim 8, wherein, The front vehicle existing region is updated through the ego vehicle running steering situation, including: A number of sub-front vehicle existing regions are screened through the ego vehicle running steering situation; The visual field deviation of the front vehicle existing region and the number of sub-front vehicle existing regions is calculated to obtain the ego vehicle visual field steering angle; According to the ego vehicle visual field steering angle and the relative position change trend of the front vehicle existing region, the current front vehicle existing region is changed for the number of sub-front vehicle existing regions, and the region is re-calibrated using the change result to update the front vehicle existing region.

Images