Image recognition-based vehicle blind area warning method and system, and storage medium

Abstract

The application discloses an image recognition-based automobile blind area early warning method and system and a storage medium, and relates to the technical field of automobile blind area recognition based on internet security. The image recognition-based automobile blind area early warning method comprises the following steps: data acquisition, complete occlusion evaluation, partial occlusion detection and partial occlusion evaluation. The application obtains image data by performing image preprocessing on a blind area image, performs complete occlusion judgment on the image data and reference data interval, and takes a first-level early warning measure; a partial occlusion influence coefficient is obtained according to the image data and reference image data, and whether to perform partial occlusion evaluation is judged according to the partial occlusion influence coefficient and a partial occlusion threshold value; all partial occlusion influence coefficients in a preset time period are obtained to obtain a partial occlusion similarity coefficient, and whether to take a second-level early warning measure is judged according to the partial occlusion similarity coefficient and a similarity threshold value, so that the effect of improving the accuracy of the image-based automobile blind area early warning is achieved, and the problem of insufficient image analysis in the automobile blind area early warning in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet security-based automobile blind area identification, and in particular to an automobile blind area early warning method and system based on image recognition and a storage medium. BACKGROUND

[0002] With the increasing complexity of road traffic, blind areas have become an important hidden danger for driving safety. Traditional physical rearview mirrors have limited coverage and are difficult to effectively monitor the blind areas around the vehicle. Blind area early warning methods based on image recognition technology use cameras and artificial intelligence algorithms (such as target detection, deep learning) to achieve accurate identification and real-time warning of obstacles in the blind area. This system not only enhances the driver's perception, but also provides technical support for intelligent transportation systems such as autonomous driving and ADAS (Advanced Driver Assistance Systems), further improving road safety.

[0003] Existing image recognition-based automobile blind area early warning technology mainly relies on cameras installed on the sides or rear of the vehicle to collect image data around the vehicle in real time. Through image processing and target detection algorithms, the system can identify potential dangerous objects such as vehicles and pedestrians in the blind area, and improve recognition accuracy with deep learning models. At the same time, computer vision technology is used to estimate the distance and direction of the target object from the vehicle. When obstacles are detected in the blind area, the system will issue warnings to the driver through sound, vision, or vibration. Existing technology has been widely applied in advanced driver assistance systems (ADAS), significantly improving driving safety.

[0004] For example, the patent application with publication number CN118212775A discloses a road detection and warning system based on image recognition analysis, which includes: detecting the road traffic scene through a millimeter wave radar and a visual sensor to obtain traffic scene data, pre-processing the traffic scene data, training the pre-processed data and complex road scene data set for deep learning, extracting features from the deep learning information, predicting road traffic accidents and detecting automobile blind areas to obtain fusion features, classifying the fusion features to obtain classification results, analyzing the classification results to complete road traffic accident warning, and using the millimeter wave radar and the time sensor to detect and identify road targets in time and space, ensuring the consistency of the two sensors in time and space.

[0005] For example, the invention patent announcement No. CN117373248B: a car blind area intelligent early warning method, system and cloud platform based on image recognition, comprising: determining the vehicle-mounted road condition object space conversion matrix based on the vehicle-mounted road condition object cluster, and then analyzing the object space position characteristics between the random two target vehicle-mounted road condition objects based on the vehicle-mounted road condition object space conversion matrix.

[0006] But in the process of implementing the technical scheme of the embodiments of the present application, it is found that the above-mentioned technology at least has the following technical problems:

[0007] In the prior art, due to the complexity of the vehicle condition and the road condition during the driving of the car, and in extreme weather conditions (such as heavy rain, heavy snow, thick fog, etc.), the camera collecting images is easy to be blocked, and there is a problem of insufficient image analysis in car blind area early warning. SUMMARY

[0008] The embodiments of the present application provide a car blind area early warning method, system and storage medium based on image recognition, which solves the problem of insufficient image analysis in car blind area early warning in the prior art, and improves the accuracy of image-based car blind area early warning.

[0009] The embodiments of the present application provide a car blind area early warning method based on image recognition, comprising the following steps: S1, obtaining blind area images in a preset time period during the driving of the vehicle through a blind area camera, and performing image preprocessing on the blind area images to obtain image data; S2, performing complete blocking judgment on the image data and reference data interval obtained from a preset database and taking corresponding first-level early warning measures; S3, obtaining a partial blocking influence coefficient according to the image data and the reference image data, and judging whether to perform S4 based on the partial blocking influence coefficient and a partial blocking threshold, the partial blocking influence coefficient being used to quantify the possibility of the blind area camera being partially blocked; S4, obtaining a partial blocking similarity coefficient based on the partial blocking influence coefficient obtained in the preset time period, and judging whether to take corresponding second-level early warning measures based on the partial blocking similarity coefficient and a similarity threshold, the partial blocking similarity coefficient representing the probability of the blind area camera being continuously blocked.

[0010] Further, the specific process of the image preprocessing is as follows: extracting feature points in the blind area image through a feature point detection algorithm and counting to obtain a number of key feature points, and extracting texture features of the blind area image through a local binary pattern and counting to obtain a number of texture feature points; performing denoising processing and gray scale conversion on the blind area image to obtain a gray scale image; obtaining image basic data from the gray scale image, the image basic data including brightness mean value, image contrast and edge line number; the image data includes the image basic data, the number of key feature points and the number of texture feature points.

[0011] Further, the specific steps of the complete occlusion judgment of the image data and the reference data interval obtained from the preset database are as follows: A1, obtaining the reference data interval from the preset database, the reference data interval including a reference brightness interval and a reference contrast interval; A2, comparing the brightness average value with the reference brightness interval, if the brightness average value belongs to the reference brightness interval, it is recorded that the blind area camera is completely occluded, otherwise, A3 is executed; A3, comparing the image contrast with the reference contrast interval, if the image contrast belongs to the reference contrast interval, it is recorded that the blind area camera is completely occluded, otherwise, A4 is executed; A4, judging whether the number of edge lines is 0, if the number of edge lines is 0, it is recorded that the blind area camera is completely occluded, otherwise, A5 is executed; A5, judging whether the number of key feature points is 0, if the number of key feature points is 0, it is recorded that the blind area camera is completely occluded, otherwise, A6 is executed; A6, judging whether the number of texture feature points is 0, if the number of texture feature points is 0, it is recorded that the blind area camera is completely occluded, otherwise, it is recorded that the blind area camera is not completely occluded.

[0012] Further, the specific content of the first early warning measure is as follows: if the blind area camera is completely occluded, the image data in the next preset time period is continuously obtained and the complete occlusion judgment is performed, if the blind area camera is still completely occluded, the radar blind area early warning method is automatically switched to; if the blind area camera is not completely occluded, the partial occlusion evaluation is performed.

[0013] Further, the specific process of deriving the partial occlusion influence coefficient according to the image data and the reference image data and judging based on the partial occlusion influence coefficient and the partial occlusion threshold is as follows: the blind area image is numbered and the image data is obtained, the partial occlusion influence coefficient is obtained based on the image data; the partial occlusion influence coefficient is compared with the partial occlusion threshold obtained from the preset database: if the partial occlusion influence coefficient is not less than the partial occlusion threshold, it is recorded that the blind area camera is not occluded; if the partial occlusion influence coefficient is less than the partial occlusion threshold, all the partial occlusion influence coefficients in the preset time period are obtained to derive the partial occlusion similarity coefficient.

[0014] Further, the acquisition process of the partial occlusion similarity coefficient is as follows: all the partial occlusion influence coefficients in the preset time period are obtained, the partial occlusion influence coefficient is compared with the previous partial occlusion influence coefficient to obtain the occlusion change ratio; the cumulative sum of all the occlusion change ratios in the preset time period is obtained to obtain the total occlusion change value, and the average occlusion change value is obtained by ratio operation of the total occlusion change value and the total number of blind area images, based on the average occlusion change value, the partial occlusion similarity coefficient is obtained.

[0015] Further, the specific process of determining whether to take corresponding secondary early warning measures based on the partial occlusion similarity coefficient and the similarity threshold is as follows: the partial occlusion similarity coefficient is compared with the similarity threshold obtained from the preset database: if the partial occlusion similarity coefficient is less than the similarity threshold, it is recorded that the blind area camera is not occluded; if the partial occlusion similarity coefficient is not less than the similarity threshold, it is recorded that the blind area camera is partially occluded, and a secondary early warning measure is taken, which includes instrument panel warning, sound warning, occlusion detection, and vehicle networking warning; the instrument panel warning is used to display the position of the occluded blind area camera on the instrument panel of the automobile; the occlusion detection means that the occluded blind area camera is detected by a cleaning sensor to detect the occluded material and take corresponding occlusion cleaning measures, and the cleaning sensor includes a laser sensor, a humidity sensor, and a pressure sensor; the vehicle networking warning is used to prompt other vehicles to maintain a distance from the risk vehicle, and the risk vehicle means that the blind area camera is partially occluded.

[0016] Further, the vehicle networking warning further includes: transmitting the basic features and position information of the risk vehicle to other vehicles through vehicle networking and prompting other vehicles to maintain a distance, and the other vehicles are determined by a vehicle positioning system; when the risk vehicle interacts with other vehicles for blind area occlusion warning through vehicle networking, a vehicle networking security communication method is enabled, which includes a transmission security method and an anti-intrusion method, the transmission security method is used to ensure the transmission security of data when the risk vehicle interacts with other vehicles, and the anti-intrusion method is used to cope with various network attacks.

[0017] The embodiment of the present application provides a kind of based on image recognition's automobile blind area early warning system, the based on image recognition's automobile blind area early warning system includes: data acquisition module, completely occluded evaluation module, partial occlusion detection module and partial occlusion evaluation module;Wherein, the completely occluded evaluation module is used to judge completely occluded to image data and reference data interval obtained from the preset database and take corresponding first early warning measure;The partial occlusion detection module is used to obtain partial occlusion influence coefficient according to image data and reference image data, whether the partial occlusion evaluation module is executed based on partial occlusion influence coefficient and partial occlusion threshold, and the partial occlusion influence coefficient is used to quantify the possibility of blind area camera being partially occluded;The partial occlusion evaluation module is used to obtain partial occlusion similarity coefficient by obtaining all partial occlusion influence coefficients in preset time period, whether to take corresponding secondary early warning measures based on partial occlusion similarity coefficient and similarity threshold, and the partial occlusion similarity coefficient indicates the probability of blind area camera being continuously occluded.

[0018] The embodiment of the present application provides a kind of computer readable storage medium, for storing program, the program is executed by processor and realizes the based on image recognition's automobile blind area early warning method.

[0019] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0020] 1. Image data is obtained by image preprocessing on the blind area image, complete occlusion judgment is performed on the image data and reference data interval, and a first warning measure is taken, a partial occlusion influence coefficient is derived according to the image data and the reference image data, and whether to perform partial occlusion evaluation is judged according to the partial occlusion influence coefficient and a partial occlusion threshold, a partial occlusion similarity coefficient is derived by obtaining all partial occlusion influence coefficients in a preset time period, and whether to take a second warning measure is judged according to the partial occlusion similarity coefficient and a similarity threshold, thereby reducing the influence of the blind area camera being occluded, and further improving the accuracy of the image-based automobile blind area warning, thereby effectively solving the problem of insufficient image analysis in the prior art automobile blind area warning.

[0021] 2. The blind area image is numbered and image data is obtained, and a partial occlusion influence coefficient is derived based on the image data, thereby more comprehensively and objectively quantifying the possibility of the blind area camera being partially occluded, and further achieving more accurate judgment of whether the blind area camera is partially occluded.

[0022] 3. All partial occlusion influence coefficients in a preset time period are obtained, a ratio operation is performed on the partial occlusion influence coefficient and the previous partial occlusion influence coefficient to obtain an occlusion change ratio, all occlusion change ratios in the preset time period are accumulated and summed to obtain an occlusion change total value, an average occlusion change value is obtained by ratio operation of the occlusion change total value and the total number of blind area images, and a partial occlusion similarity coefficient is derived based on the average occlusion change value, thereby further judging whether the blind area camera is partially occluded, and further achieving timely warning when the blind area camera is partially occluded. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A flowchart of the image recognition-based automobile blind area warning method provided in the embodiments of the present application is provided.

[0024] Figure 2 A change schematic diagram of the partial occlusion influence coefficient provided in the embodiments of the present application is provided.

[0025] Figure 3 A structure schematic diagram of the image recognition-based automobile blind area warning system provided in the embodiments of the present application is provided. DETAILED DESCRIPTION

[0026] The embodiment of the application provides a vehicle blind area early warning method and system based on image recognition and a storage medium, solves the problem of insufficient image analysis in the prior art vehicle blind area early warning, acquires a blind area image through a blind area camera, and performs image preprocessing on the blind area image to obtain image data; the image data and reference data interval are subjected to complete occlusion judgment, and a corresponding first-level early warning measure is taken; the blind area image is numbered and image data is acquired, a partial occlusion influence coefficient is obtained based on the image data, whether to perform partial occlusion evaluation is judged based on the partial occlusion influence coefficient and a partial occlusion threshold value, all partial occlusion influence coefficients in a preset time period are acquired, a ratio operation is performed on the partial occlusion influence coefficient and a previous partial occlusion influence coefficient to obtain an occlusion change ratio; all occlusion change ratios in the preset time period are accumulated and summed to obtain an occlusion change total value, and an average occlusion change value is obtained by performing a ratio operation on the occlusion change total value and a total number of blind area images, a partial occlusion similarity coefficient is obtained based on the average occlusion change value, whether to take a corresponding second-level early warning measure is judged based on the partial occlusion similarity coefficient and a similarity threshold value, and the accuracy of the vehicle blind area early warning based on the image is improved.

[0027] The technical solution in the embodiment of the application is used for solving the problem of insufficient image analysis in the vehicle blind area early warning, and the general idea is as follows:

[0028] The image data and reference data interval are subjected to complete occlusion judgment, and a first-level early warning measure is taken, a partial occlusion influence coefficient is obtained according to the image data and reference image data, and whether to perform partial occlusion evaluation is judged accordingly, partial occlusion similarity coefficients are obtained by acquiring all partial occlusion influence coefficients in a preset time period, and whether to take a second-level early warning measure is judged, and the accuracy of the vehicle blind area early warning based on the image is improved.

[0029] In order to better understand the above technical solution, the above technical solution will be described in detail in combination with the description of the drawings and the specific embodiments.

[0030] As Figure 1As shown, a flowchart of the image recognition-based vehicle blind area early warning method provided by the embodiment of the application is shown, and the method comprises the following steps: S1, acquiring a blind area image in a preset time period through a blind area camera during vehicle driving, and performing image preprocessing on the blind area image to obtain image data; S2, performing complete occlusion judgment on the image data and reference data interval obtained from a preset database and taking corresponding first-level early warning measures, the first-level early warning measures being used to eliminate the influence of complete occlusion on vehicle blind area early warning; S3, obtaining a partial occlusion influence coefficient according to the image data and reference image data, and judging whether to perform S4 based on the partial occlusion influence coefficient and a partial occlusion threshold, the partial occlusion influence coefficient being used to quantify the possibility of the blind area camera being partially occluded; S4, obtaining a partial occlusion similarity coefficient based on the partial occlusion influence coefficient obtained in the preset time period, and judging whether to take corresponding second-level early warning measures based on the partial occlusion similarity coefficient and a similarity threshold, the partial occlusion similarity coefficient representing the probability of the blind area camera being continuously occluded.

[0031] In the embodiment, the preset time period represents the time length of one data acquisition cycle of the blind area early warning method, for example, 10 minutes, 30 minutes or 60 minutes; the blind area camera is usually arranged below the left and right side mirrors of the vehicle to monitor the blind area on the side of the vehicle, in addition, some vehicles may arrange a camera in the front bumper area to monitor the blind area in front of the vehicle, for example, in the BMW X5 series of BMW, a high-definition camera is installed on the front bumper, which works together with the rear camera and the side camera to form a bird's eye view; by distinguishing the complete occlusion and the partial occlusion of the blind area image, whether the field of view of the blind area camera is occluded can be effectively identified, the false alarm can be reduced, the reliability of the early warning system can be improved, and thus the accuracy of the image-based vehicle blind area early warning is improved.

[0032] Further, the specific process of image preprocessing is as follows: feature points in the blind area image are extracted through a feature point detection algorithm, and the number of key feature points is counted, texture features of the blind area image are extracted through a local binary pattern, and the number of texture feature points is counted, the feature point detection algorithm is used to obtain the number of key feature points, and the local binary pattern is used to obtain the number of texture feature points; the blind area image is subjected to denoising processing and gray scale conversion to obtain a gray scale image, the denoising processing is used to reduce the noise of the blind area image, and the gray scale processing is used to convert the color image into the gray scale image; image basic data is obtained through the gray scale image, the image basic data comprises a brightness mean value, an image contrast and an edge line number, the brightness mean value represents a mean value of pixel values of the gray scale image, the image contrast represents a standard deviation of pixel values of the gray scale image, and the edge line number is obtained by extracting edges of the gray scale image through an edge detection algorithm and counting the number of non-zero pixel edges; the image data comprises the image basic data, the number of key feature points and the number of texture feature points.

[0033] In this embodiment, the pixel values of the grayscale image are read, all pixel values of the grayscale image are processed as a two-dimensional array, and the average value of all pixel values of the two-dimensional array is calculated to obtain the brightness average value; the data in the blind area image can be more efficiently extracted and processed, the perception and processing capability of the vehicle to the shielding condition are improved, and thus the reliability and precision of the vehicle blind area warning system are enhanced.

[0034] The local binary pattern is a method for texture feature extraction, which generates a binary number by comparing the neighborhood of each pixel, representing the relationship between the pixel and its neighborhood pixels; the specific steps are as follows: selecting a center pixel and considering the neighborhood pixels (usually a 3x3 window) around it; comparing the neighborhood pixels with the center pixel, if the value of the neighborhood pixel is greater than or equal to the center pixel, it is marked as 1, otherwise it is marked as 0; converting the obtained binary number to decimal number to form a new feature value, which is a texture feature point.

[0035] The feature point detection algorithm is used to extract significant points from the image, and the specific feature point detection algorithm is SURF (Speeded-Up Robust Features): SURF is an improved version of SIFT (Scale-Invariant Feature Transform), focusing on improving the calculation speed. It uses Hessian matrix for fast feature point detection, and uses approximate method to speed up the feature point calculation and matching process.

[0036] The denoising process aims to reduce the noise in the image and improve the image quality, and the specific denoising technique is Gaussian filtering, that is, using Gaussian function to weight the neighborhood pixels to reduce high frequency noise while preserving the basic structure of the image.

[0037] The grayscale conversion is the process of converting a color image to a grayscale image, which converts the RGB (red, green, blue) value of each pixel to a single grayscale value, and the specific method is weighted average method, that is, according to the sensitivity of human eyes to different colors, using different weights to calculate the grayscale value.

[0038] Further, the specific steps of the complete occlusion judgment of the image data and the reference data interval obtained from the preset database are as follows: A1, obtaining the reference data interval from the preset database, the reference data interval including a reference brightness interval and a reference contrast interval; A2, comparing the brightness mean value with the reference brightness interval, if the brightness mean value belongs to the reference brightness interval, it is recorded that the blind area camera is completely occluded, otherwise, A3 is executed; A3, comparing the image contrast with the reference contrast interval, if the image contrast belongs to the reference contrast interval, it is recorded that the blind area camera is completely occluded, otherwise, A4 is executed; A4, judging whether the number of edge lines is 0, if the number of edge lines is 0, it is recorded that the blind area camera is completely occluded, otherwise, A5 is executed; A5, judging whether the number of key feature points is 0, if the number of key feature points is 0, it is recorded that the blind area camera is completely occluded, otherwise, A6 is executed; A6, judging whether the number of texture feature points is 0, if the number of texture feature points is 0, it is recorded that the blind area camera is completely occluded, otherwise, it is recorded that the blind area camera is not completely occluded.

[0039] In the embodiment, by sequentially checking the brightness mean value, the image contrast, the number of edge lines, the number of key feature points and the number of texture feature points, it is ensured that multiple blind area image features are comprehensively analyzed, and false judgment caused by a single feature is avoided; the intelligence and the fineness of the overall blind area early warning are improved, and the complete occlusion of the blind area camera can be more accurately identified.

[0040] Specifically, the reference brightness interval is obtained from the preset database. In a specific embodiment, the reference brightness interval is obtained by comparing the brightness mean value interval. The interval in which the image presents black is [0, 50], and the interval in which the image presents white is [231, 255], so the reference brightness interval is [0, 50] U [231, 255].

[0041] Specifically, the reference contrast interval is obtained from the preset database. In a specific embodiment, the reference contrast interval is obtained by comparing the contrast. The interval in which the difference between the light and dark areas of the image is not obvious is [0, 50], so the reference contrast interval is [0, 50].

[0042] Further, the specific content of the first level early warning measure is as follows: if the blind area camera is completely occluded, the image data in the next preset time period is continuously obtained and the complete occlusion judgment is performed, if the blind area camera is still completely occluded, the radar blind area early warning method is automatically switched to, and the driver of the automobile is prompted that the blind area early warning method has been switched, the radar blind area early warning method means that the blind area is detected by using the radar; if the blind area camera is not completely occluded, the partial occlusion evaluation is performed.

[0043] In the embodiment, the continuity and reliability of the blind area detection function are ensured by automatically switching to the radar blind area warning method when the blind area camera is completely blocked. Specifically, the radar blind area warning method aims to identify and warn potential obstacles or other vehicles on the side and rear of the vehicle, ensuring that the driver can obtain sufficient safety information when changing lanes, overtaking, and the like. Even if the blind area camera cannot work normally, the vehicle can still obtain warning information through the radar, and the driver is prompted to switch the detection method, thereby enhancing the driving safety and intelligent response capability of the system.

[0044] Further, the specific process of judging based on the partial blocking influence coefficient and the partial blocking threshold is as follows: the blind area images are numbered and image data is obtained, and the partial blocking influence coefficient is obtained based on the image data; the partial blocking influence coefficient is compared with the partial blocking threshold obtained from the preset database: if the partial blocking influence coefficient is not less than the partial blocking threshold, it is recorded that the blind area camera is not blocked; if the partial blocking influence coefficient is less than the partial blocking threshold, the partial blocking influence coefficient of all preset time periods is obtained to obtain a partial blocking similarity coefficient, and the specific limit expression of the partial blocking influence coefficient is:

[0045]

[0046] In the formula, n represents the number of blind area images, n = 1, 2,..., N, N represents the total number of blind area images, AB n represents the average brightness of the nth blind area image, AB n-1 represents the average brightness of the (n-1)th blind area image, IC n represents the image contrast of the nth blind area image, IC n-1 represents the image contrast of the (n-1)th blind area image, LN n represents the number of edge lines of the nth blind area image, KP n represents the number of key feature points of the nth blind area image, FP n represents the number of texture feature points of the nth blind area image, PEC n represents the partial blocking influence coefficient of the nth blind area image, and e represents a natural constant.

[0047] In this embodiment, the algorithm combines the average brightness, image contrast, number of edge lines, number of key feature points, and number of texture feature points to comprehensively analyze and derive the partial occlusion impact coefficient. In the algorithm, the larger the absolute value of the difference between the ratio of the average brightness to the previous average brightness and 1, the larger the partial occlusion impact coefficient becomes. Similarly, the larger the absolute value of the difference between the ratio of the image contrast to the previous image contrast and 1, the larger the partial occlusion impact coefficient becomes, indicating that the presence of image changes in the blind spot suggests a high probability that the camera in the blind spot is not occluded. Furthermore, the number of edge lines, key feature points, and texture feature points are positively correlated with the partial occlusion impact coefficient. Figure 2 The figure shows a schematic diagram of the change in the partial occlusion influence coefficient provided in this embodiment. As can be seen from the figure, the image shows an upward trend as the number of edge lines, key feature points, and texture feature points increases. This indicates that the more data elements there are in the blind spot image, the greater the possibility that the blind spot camera is not occluded. The algorithm provided in this embodiment effectively quantifies the possibility that the blind spot camera may be occluded as reflected in the blind spot image, which is conducive to timely detection of the blind spot camera's status and avoidance of traffic safety problems.

[0048] Specifically, the partial occlusion threshold is obtained from a preset database. In a specific embodiment, the partial occlusion threshold represents the average value of the corresponding dataset obtained by substituting historical image data into the partial occlusion influence coefficient limitation expression.

[0049] Furthermore, the process of obtaining the partial occlusion similarity coefficient is as follows: All partial occlusion influence coefficients within a preset time period are obtained; the ratio of each partial occlusion influence coefficient to the previous partial occlusion influence coefficient is calculated to obtain the occlusion change ratio; all occlusion change ratios within the preset time period are accumulated and summed to obtain the total occlusion change value; the average occlusion change value is obtained by ratioing the total occlusion change value to the total number of blind zone images; the partial occlusion similarity coefficient is derived based on the average occlusion change value, and the specific constraint expression for the partial occlusion similarity coefficient is as follows:

[0050]

[0051] In the formula, n represents the number of the blind spot image, n = 1, 2, ..., N, and N represents the total number of blind spot images. n PEC represents the partial occlusion influence coefficient of the nth blind zone image. n+1 represents the partial occlusion influence coefficient of the (n+1)th blind zone image, and PSC represents the partial occlusion similarity coefficient.

[0052] In this embodiment, the algorithm combines the total number of blind zone images and the partial occlusion influence coefficient to derive the partial occlusion similarity coefficient. To simplify the analysis, let's call it... PJ is the average occlusion change value, denoted as PJ. FC is the variance occlusion change value, the average occlusion change value and the variance occlusion change value are combined to obtain the partial occlusion similarity coefficient change table, which is shown in Table 1:

[0053] Table 1 Partial occlusion similarity coefficient change table

[0054] Average occlusion change value PJ Variance occlusion change value FC Partial occlusion similarity coefficient PSC 2.0 3.0 1.06 8.0 3.0 1.10 4.0 2.0 1.27 4.0 10.0 1.00 5.0 5.0 1.01 ...... ...... ......

[0055] As can be seen from Table 1, when the variance occlusion change value is constant, the value of the partial occlusion similarity coefficient increases with the increase of the average occlusion change value, for example, the first row of data and the second row of data, when the variance occlusion change value is constant and is 3.0, the average occlusion change value increases from 2.0 to 8.0, and the partial occlusion similarity coefficient increases from 1.06 to 1.10; similarly, when the average occlusion change value is constant, the value of the partial occlusion similarity coefficient decreases with the increase of the variance occlusion change value, for example, the third row and the fourth row of data, the average occlusion change value is constant and is 4.0, the variance occlusion change value increases from 2.0 to 10.0, and the partial occlusion similarity coefficient decreases from 1.27 to 1.00; therefore, when the average occlusion change value is larger and the variance occlusion change value is smaller, the partial occlusion similarity coefficient is larger, which indicates that when it is detected that a blind area image has the possibility of being partially occluded, all partial occlusion influence coefficients in a preset time period are analyzed, accidental false positives are avoided, and the false positive rate of the blind area warning of the vehicle is effectively reduced.

[0056] Further, the specific process of judging whether to take corresponding secondary warning measures based on the partial occlusion similarity coefficient and the similarity threshold value is as follows: the partial occlusion similarity coefficient is judged with the similarity threshold value obtained from the preset database: if the partial occlusion similarity coefficient is less than the similarity threshold value, it is recorded that the blind area camera is not occluded; if the partial occlusion similarity coefficient is not less than the similarity threshold value, it is recorded that the blind area camera is partially occluded, and secondary warning measures are taken, which include instrument panel warning, sound warning, occlusion detection, and vehicle networking warning; the instrument panel warning is used to display the position of the occluded blind area camera on the instrument panel of the vehicle; the sound warning means that the vehicle driver is informed through sound that the blind area camera is occluded and should drive carefully; the occlusion detection means that the occlusion material occluding the blind area camera is detected through a cleaning sensor, and corresponding occlusion cleaning measures are taken, the cleaning sensor includes a laser sensor, a humidity sensor, and a pressure sensor, and the occlusion cleaning measure is used to remove the occlusion material on the blind area camera; the vehicle networking warning is used to prompt other vehicles to maintain a distance from the risk vehicle, and the risk vehicle means the vehicle whose blind area camera is partially occluded.

[0057] In this embodiment, by introducing the judgment mechanism of partial occlusion similarity coefficient and similarity threshold, the vehicle can evaluate different degrees of occlusion, avoid triggering excessive warning due to instantaneous occlusion, and improve the accuracy and rationality of the warning; At the same time, the instrument panel warning, sound warning and secondary warning measure of occlusion detection are adopted, which provides visual and auditory automatic feedback to ensure that the vehicle driver can timely detect the occlusion of the blind area camera, effectively improve the driving safety; In addition, combined with laser, humidity and pressure sensors, the vehicle can automatically detect and identify the occlusion on the blind area camera, and take corresponding cleaning measures, which reduces human intervention, improves the automation level of the system, and ensures that the camera has a clear view at the critical moment; The reliability and automation level of the image-based blind area warning method are improved.

[0058] Specifically, when the humidity sensor continuously detects the presence of water molecules on the blind area camera within a preset time period, the heating mode is automatically enabled to eliminate water on the blind area camera; When the laser sensor detects clothing fibers or plastic products, the camera occlusion removal instruction is issued to the vehicle driver, reminding the vehicle driver to clean the obstacles; When the pressure sensor continuously detects pressure on the mirror surface of the blind area camera within a preset time period, the cleaning mode is automatically started to remove the occlusion attached to the mirror surface.

[0059] Specifically, the similarity threshold is obtained from a preset database. In one specific embodiment, the similarity threshold represents the average value of the corresponding data set obtained by substituting the partial occlusion similarity coefficient into the partial occlusion similarity coefficient limit expression.

[0060] Further, the vehicle networking warning further includes: transmitting the basic features and position information of the risk vehicle to other vehicles through vehicle networking and reminding other vehicles to maintain a distance, the other vehicles are determined by a vehicle positioning system; When the risk vehicle interacts with other vehicles for blind area occlusion warning through vehicle networking, a vehicle networking security communication method is enabled, the vehicle networking security communication method includes a transmission security method and an anti-intrusion method, the transmission security method is used to ensure the transmission security of data when the risk vehicle interacts with other vehicles, and the anti-intrusion method is used to deal with various network attacks.

[0061] In this embodiment, the basic features include the model, brand and color of the vehicle, and the other vehicles represent the vehicles whose distance from the risk vehicle gradually becomes not greater than the brake distance through the vehicle positioning system, and the brake distance is obtained by the following numerical expression:

[0062]

[0063] In the formula, V represents the speed of the risk vehicle, μ represents the friction coefficient of the road surface, g represents the acceleration of gravity, and XD represents the braking distance; the vehicle networking early warning method provided by the embodiment is beneficial to ensuring data transmission security and preventing network attacks, thereby improving driving safety.

[0064] Specifically, the transmission security method uses Internet security technology to ensure that the interactive data (basic features and position information of the risk vehicle) is not stolen or tampered with during transmission, such as the SSL (Secure Sockets Layer) / TLS (Transport Layer Security) encryption protocol; at the same time, a secure communication protocol is used to ensure the security of the interactive data transmission, for example, using V2X (Vehicle to Everything) communication technology to ensure the security of the communication protocol, to prevent hackers from disrupting the normal use of the automobile blind area early warning method by falsifying or tampering with communication data; in addition, the intrusion prevention method enables the automobile blind area early warning method to have intrusion detection and defense capabilities to deal with various network attacks, such as deploying firewalls, intrusion detection systems (IDS, Intrusion Detection System) and intrusion prevention systems (IPS, Intrusion Prevention System) to discover and prevent existing Internet security threats in a timely manner.

[0065] As shown in Figure 3 The structure of the automobile blind area early warning system based on image recognition provided by the embodiment of the application is shown in the structure diagram. The automobile blind area early warning system based on image recognition provided by the embodiment of the application comprises a data acquisition module, a complete occlusion evaluation module, a partial occlusion detection module and a partial occlusion evaluation module. The data acquisition module is used to acquire blind area images in a preset time period during vehicle driving through a blind area camera, and to obtain image data by image preprocessing of the blind area images. The complete occlusion evaluation module is used to make a complete occlusion judgment on the image data and a reference data interval obtained from a preset database and to take corresponding first-level early warning measures. The first-level early warning measures are used to eliminate the influence of complete occlusion on automobile blind area early warning. The partial occlusion detection module is used to derive a partial occlusion influence coefficient according to the image data and reference image data, and to judge whether to execute the partial occlusion evaluation module based on the partial occlusion influence coefficient and a partial occlusion threshold. The partial occlusion influence coefficient is used to quantify the possibility of partial occlusion of the blind area camera. The partial occlusion evaluation module is used to obtain the partial occlusion influence coefficient derived in the preset time period to derive a partial occlusion similarity coefficient, and to judge whether to take corresponding second-level early warning measures based on the partial occlusion similarity coefficient and a similarity threshold. The partial occlusion similarity coefficient represents the possibility of continuous occlusion of the blind area camera.

[0066] In the embodiment, the grading measures of the first warning and the second warning avoid over-warning and ensure timely response in the case of serious occlusion; through the automatic blind area camera occlusion detection and warning, the system can perform real-time monitoring and processing without relying on the attention of the driver, thereby reducing the workload of the driver and improving the driving experience.

[0067] In the embodiment, the computer readable storage medium stores the program, and the processor executes the program to realize the automatic blind area detection and warning, and the program executed by the processor can analyze the image data in real time, continuously monitor the blind area of the vehicle, reduce human intervention, automatically identify the blind area camera occlusion, and improve the driving safety. The computer readable storage medium realizes an efficient and intelligent blind area monitoring and warning system, and significantly improves the safety and convenience of driving.

[0068] In the embodiment, the computer readable storage medium stores the program, and the processor executes the program to realize the automatic blind area detection and warning, and the program executed by the processor can analyze the image data in real time, continuously monitor the blind area of the vehicle, reduce human intervention, automatically identify the blind area camera occlusion, and improve the driving safety. The computer readable storage medium realizes an efficient and intelligent blind area monitoring and warning system, and significantly improves the safety and convenience of driving.

[0069] Specifically, in the hardware design of the readable storage medium, the physical port (such as USB, Universal Serial Bus, debugging interface) of the readable storage medium needs to be prevented from becoming a breakthrough point of an attacker, for example, by closing the physical port of the readable storage medium and setting access permissions to prevent physical access. For example, the vehicle blind area warning system allows the OBD-II (On-Board Diagnostics II) interface to be locked to prevent unauthorized access.

[0070] In summary, the embodiment of the present application obtains image data by performing image preprocessing on the blind area image, performs complete occlusion judgment on the image data and the reference data interval and takes a first warning measure, obtains a partial occlusion influence coefficient according to the image data and the reference image data, and judges whether to perform partial occlusion evaluation according to the partial occlusion influence coefficient and a partial occlusion threshold value, obtains a partial occlusion similarity coefficient by obtaining all partial occlusion influence coefficients in a preset time period, and judges whether to take a second warning measure according to the partial occlusion similarity coefficient and a similarity threshold value, thereby reducing the influence of the blind area camera being occluded, and improving the accuracy of the vehicle blind area warning based on images, and effectively solving the problem of insufficient image analysis in the vehicle blind area warning in the prior art.

[0071] Those skilled in the art will appreciate that embodiments of the present application can be devised for a variety of applications. It is therefore intended that the present application cover all such modifications and variations of the application disclosed herein provided they come within the scope of the appended claims and their equivalents. It is intended to

[0072] The present application is described in reference to the drawings using a flowchart and / or a block diagram of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0073] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0075] While the preferred embodiments of the application have been described, additional variations and modifications can be employed by those skilled in the art. Therefore, the appended claims are intended to cover all such modifications and variations as fall within the scope of the present application.

[0076] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for image recognition-based blind area warning for a vehicle, characterized in that, The method comprises the following steps: S1, obtaining blind area images in a preset time period during vehicle driving through a blind area camera, and performing image preprocessing on the blind area images to obtain image data; S2, performing complete occlusion judgment on the image data and reference data obtained from a preset database, and taking corresponding first-level warning measures; S3, obtaining a partial occlusion influence coefficient according to the image data and reference image data, and judging whether to perform S4 based on the partial occlusion influence coefficient and a partial occlusion threshold, wherein the partial occlusion influence coefficient is used to quantify the possibility of the blind area camera being partially occluded; S4, obtaining a partial occlusion similarity coefficient by obtaining all partial occlusion influence coefficients in a preset time period, and judging whether to take corresponding second-level warning measures based on the partial occlusion similarity coefficient and a similarity threshold, wherein the partial occlusion similarity coefficient represents the probability of the blind area camera being continuously occluded; The partial occlusion influence coefficient is obtained according to the image data and the reference image data, specifically: the partial occlusion influence coefficient is obtained by comprehensively analyzing the brightness mean value, the image contrast, the number of edge lines, the number of key feature points and the number of texture feature points; The process of obtaining the partial occlusion similarity coefficient is as follows: Obtain all partial occlusion influence coefficients in a preset time period, and perform ratio operation on the partial occlusion influence coefficients and the previous partial occlusion influence coefficient to obtain an occlusion change ratio; The partial occlusion similarity coefficient is obtained by accumulating and summing all occlusion change ratios in a preset time period to obtain an occlusion change total value, and performing ratio operation on the occlusion change total value and the total number of blind area images to obtain an average occlusion change value; The specific process of judging whether to take corresponding second-level warning measures based on the partial occlusion similarity coefficient and the similarity threshold is as follows: The partial occlusion similarity coefficient is compared with the similarity threshold obtained from the preset database: If the partial occlusion similarity coefficient is less than the similarity threshold, it is recorded that the blind area camera is not occluded; If the partial occlusion similarity coefficient is not less than the similarity threshold, it is recorded that the blind area camera is partially occluded, and second-level warning measures are taken, wherein the second-level warning measures include instrument panel warning, sound warning, occlusion detection and vehicle networking warning; The instrument panel warning is used to display the position of the occluded blind area camera on the instrument panel of the vehicle; The occlusion detection means that the occluded blind area camera is detected by a cleaning sensor to detect the occluded material and take corresponding occlusion cleaning measures, wherein the cleaning sensor comprises a laser sensor, a humidity sensor and a pressure sensor; The vehicle networking warning is used to prompt other vehicles to maintain a distance from the risk vehicle, wherein the risk vehicle represents a vehicle with a partially occluded blind area camera.

2. The image recognition based blind area warning method for vehicle as claimed in claim 1, wherein: The specific process of the image preprocessing is as follows: Feature points in the blind area image are extracted by a feature point detection algorithm, and the number of key feature points is counted, and the texture features of the blind area image are extracted by a local binary pattern, and the number of texture feature points is counted; The blind area image is denoised and converted to a grayscale image; Image basic data is obtained from the grayscale image, wherein the image basic data comprises a brightness mean value, an image contrast and a number of edge lines; The image data comprises image basic data, key feature point quantity and texture feature point quantity.

3. The image recognition based blind zone warning method for vehicle as claimed in claim 2, wherein: The specific steps of the complete occlusion judgment of the image data and the reference data interval obtained from the preset database are as follows: A1, obtaining the reference data interval from the preset database, the reference data interval comprising a reference brightness interval and a reference contrast interval; A2, comparing the brightness mean value with the reference brightness interval, if the brightness mean value belongs to the reference brightness interval, recording that the blind area camera is completely occluded, otherwise, executing A3; A3, comparing the image contrast with the reference contrast interval, if the image contrast belongs to the reference contrast interval, recording that the blind area camera is completely occluded, otherwise, executing A4; A4, judging whether the edge line quantity is 0, if the edge line quantity is 0, recording that the blind area camera is completely occluded, otherwise, executing A5; A5, judging whether the key feature point quantity is 0, if the key feature point quantity is 0, recording that the blind area camera is completely occluded, otherwise, executing A6; A6, judging whether the texture feature point quantity is 0, if the texture feature point quantity is 0, recording that the blind area camera is completely occluded, otherwise, recording that the blind area camera is not completely occluded.

4. The image recognition based blind zone warning method for vehicle as claimed in claim 3, wherein: The specific content of the first early warning measure is as follows: If the blind area camera is completely occluded, continue to obtain the image data in the next preset time period and perform complete occlusion judgment, if the blind area camera is still completely occluded, automatically switching to the radar blind area early warning method; If the blind area camera is not completely occluded, performing partial occlusion evaluation.

5. The image recognition based blind zone warning method for vehicle as claimed in claim 2, wherein: The specific process of deriving the partial occlusion influence coefficient according to the image data and the reference image data and judging based on the partial occlusion influence coefficient and the partial occlusion threshold is as follows: numbering the blind area image and obtaining the image data; comparing the partial occlusion influence coefficient with the partial occlusion threshold obtained from the preset database: If the partial occlusion influence coefficient is not less than the partial occlusion threshold, recording that the blind area camera is not occluded; If the partial occlusion influence coefficient is less than the partial occlusion threshold, obtaining all the partial occlusion influence coefficients in the preset time period to derive the partial occlusion similarity coefficient.

6. The image recognition based blind zone warning method for vehicle as claimed in claim 1, wherein: The vehicle networking early warning further comprises: transmitting the basic features and position information of the risk vehicle to other vehicles through vehicle networking and reminding other vehicles to maintain a distance, the other vehicles being determined by a vehicle positioning system; when the risk vehicle interacts with other vehicles for blind area occlusion early warning through vehicle networking, enabling a vehicle networking security communication method, the vehicle networking security communication method comprising a transmission security method and an anti-intrusion method, the transmission security method being used to ensure the transmission security of data when the risk vehicle interacts with other vehicles, and the anti-intrusion method being used to cope with various network attacks.

7. A system for applying the image recognition-based blind zone warning method for a vehicle according to any one of claims 1 to 6, characterized in that, comprises: a data acquisition module, a complete occlusion evaluation module, a partial occlusion detection module and a partial occlusion evaluation module; wherein the data acquisition module is used to obtain blind area images in a preset time period during vehicle driving through a blind area camera, and obtain image data by image preprocessing of the blind area images; The complete occlusion evaluation module is configured to make a complete occlusion judgment on the image data and reference data obtained from a preset database and take corresponding first-level warning measures; The partial occlusion detection module is configured to derive a partial occlusion influence coefficient from the image data and the reference image data, and judge whether to execute a partial occlusion evaluation module based on the partial occlusion influence coefficient and a partial occlusion threshold value, the partial occlusion influence coefficient being used to quantify the possibility of the blind area camera being partially occluded; The partial occlusion evaluation module is configured to derive a partial occlusion similarity coefficient from all partial occlusion influence coefficients in a preset time period, and judge whether to take corresponding second-level warning measures based on the partial occlusion similarity coefficient and a similarity threshold value, the partial occlusion similarity coefficient representing the probability of the blind area camera being continuously occluded. 8.A computer readable storage medium, configured to store a program, the program being executed by a processor to implement the image recognition-based vehicle blind area warning method according to any one of claims 1 to 6.

Images