Driving blind spot monitoring method, device, equipment and readable storage medium

Abstract

The present application provides a driving blind spot monitoring method, device, equipment and readable storage medium, the method comprising: determining the background area and obstacle monitoring area from the area photographed by the target camera; preprocessing the video data collected by the target camera to obtain the target video data; determining the target image group from the target video data; for each pixel point in the obstacle monitoring area, taking the grayscale average value of the same pixel point in the target image group as the grayscale standard value of the pixel point; outputting the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale value of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value. The present application does not rely on feature matching and is not limited to the feature database, thereby reducing missed detection.

Description

Technical Field

[0001] The present application relates to the field of vehicle early warning technology, and in particular to a driving blind spot monitoring method, device, equipment and readable storage medium. Background Art

[0002] Blind spots refer to areas that the driver cannot directly see or observe while driving. These areas are located on the side and rear of the vehicle and are often prone to accidents. Blind spot monitoring is an important part of the assisted driving function. It can help the driver monitor obstacles in the blind spots, thereby reducing the potential dangers caused by them. The current blind spot monitoring method uses a feature matching algorithm to identify obstacles from the video data collected by the camera. The feature matching algorithm needs to be trained on a model to form a feature database before use. During use, it is limited by the feature database and can only identify a limited number of obstacles, which makes it easy for the blind spot monitoring method to miss detections. Summary of the invention

[0003] The present application provides a driving blind spot monitoring method, device, equipment and readable storage medium, which can solve the technical problem that the driving blind spot monitoring method in the prior art is prone to missed detection.

[0004] In a first aspect, an embodiment of the present application provides a driving blind spot monitoring method, the driving blind spot monitoring method comprising:

[0005] Determine a background area and an obstacle monitoring area from the area photographed by the target camera, wherein the background area includes the obstacle monitoring area;

[0006] Preprocess the video data collected by the target camera to obtain target video data;

[0007] Determine a target image group from the target video data, wherein a grayscale change reference value of the target image group is less than a grayscale change threshold value, the grayscale change reference value of the target image group is calculated based on grayscale change quantization values ​​of all two adjacent frames of images contained in the group, the grayscale change quantization value of two adjacent frames of images is determined based on inter-frame grayscale changes in a background area, and the greater the grayscale change, the greater the corresponding grayscale change quantization value;

[0008] For each pixel in the obstacle monitoring area, the grayscale average value of the same pixel in the target image group is used as the grayscale standard value of the pixel;

[0009] According to the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value, the obstacle recognition result in the obstacle monitoring area is output.

[0010] Furthermore, in one embodiment, the step of determining the target image group from the target video data includes:

[0011] Obtain N consecutive frames of images from the target video data;

[0012] The grayscale change reference value of the N frames of image is calculated according to the first formula group, and the first formula group is:

[0013]

[0014]

[0015]

[0016] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, T f is the grayscale difference threshold of the same pixel in the background area of ​​two adjacent frames of images, A k (i, j) is the two-dimensional value of the grayscale difference between the pixel (i, j) in the kth frame image and the k-1th frame image, D is the background area, A k is the grayscale change quantization value between the k-th frame image and the k-1-th frame image, and A is the grayscale change reference value;

[0017] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are determined to be the target image group.

[0018] Furthermore, in one embodiment, the step of outputting the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard values ​​comprises:

[0019] Acquire L consecutive frames of images from subsequent images of the target image group;

[0020] The first difference reference value of the L frames of images is calculated according to the second formula group, and the second formula group is:

[0021]

[0022]

[0023]

[0024] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T b is the grayscale difference threshold between the grayscale value of the same pixel in the obstacle monitoring area and the grayscale standard value, Bk (i, j) is the pixel point (i, j) in the k-th frame image based on T b The obtained grayscale difference two-dimensional value, d is the obstacle monitoring area, B k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the obstacle monitoring area of ​​the k-th frame image, and B is the first difference reference value;

[0025] If the first difference reference value of the L frames of images is less than the first difference threshold, it is determined that there is no obstacle in the obstacle monitoring area;

[0026] If the first difference reference value of the L frames of images is greater than or equal to the first difference threshold, it is determined that an obstacle exists in the obstacle monitoring area.

[0027] Furthermore, in one embodiment, after the step of determining the target image group from the target video data, the following steps are further included:

[0028] For each pixel in the background area except the obstacle monitoring area, the average grayscale value of the same pixel in these N frames of images is used as the grayscale standard value of the pixel;

[0029] After the step of determining that there is no obstacle in the obstacle monitoring area, the method further includes:

[0030] The second difference reference value of the L frames of images is calculated according to the third formula group, and the third formula group is:

[0031]

[0032]

[0033]

[0034] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T L is the grayscale difference threshold between the grayscale value of the same pixel in the background area and the grayscale standard value, C k (i, j) is the pixel point (i, j) in the k-th frame image based on T L The grayscale difference two-dimensional value obtained, D is the background area, C is the k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the background area of ​​the k-th frame image, and C is the second difference reference value;

[0035] If the second difference reference value of the L frames of images is greater than or equal to the second difference threshold, then taking the first frame of the L frames of images as the starting frame, obtaining N consecutive frames of images from the target video data, and calculating the grayscale change reference value of the N frames of images, where N is equal to the number of image frames in the target image group;

[0036] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

[0037] Furthermore, in one embodiment, after the step of taking the grayscale average value of each pixel in the target image group as the grayscale standard value of the pixel for each pixel in the obstacle monitoring area, the following step is further performed:

[0038] Obtaining N consecutive frames of images from the target video data regularly according to an update cycle, and calculating grayscale change reference values ​​of the N frames of images, where N is equal to the number of image frames in the target image group;

[0039] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

[0040] Furthermore, in one embodiment, before the step of determining the background area and the obstacle monitoring area from the area photographed by the target camera, the following step is further included:

[0041] Determine the lane the vehicle is in based on the vehicle positioning system and high-precision map;

[0042] If there are adjacent lanes on the left and / or right side of the lane where the vehicle is located, the camera on the corresponding side of the vehicle is used as the target camera.

[0043] Furthermore, in one embodiment, the background area includes all drivable areas in the area photographed by the target camera, and the obstacle monitoring area is located in the adjacent lane area corresponding to the target camera.

[0044] In a second aspect, an embodiment of the present application further provides a driving blind spot monitoring device, the driving blind spot monitoring device comprising:

[0045] An area determination module is used to determine a background area and an obstacle monitoring area from the area photographed by the target camera, wherein the background area includes the obstacle monitoring area;

[0046] A preprocessing module is used to preprocess the video data collected by the target camera to obtain target video data;

[0047] An image determination module is used to determine a target image group from target video data, wherein a grayscale change reference value of the target image group is less than a grayscale change threshold value, the grayscale change reference value of the target image group is calculated based on grayscale change quantization values ​​of all two adjacent frames of images contained in the group, the grayscale change quantization value of two adjacent frames of images is determined based on inter-frame grayscale changes in a background area, and the greater the grayscale change, the greater the corresponding grayscale change quantization value;

[0048] A standard value calculation module is used to take the grayscale average value of each pixel in the obstacle monitoring area as the grayscale standard value of the pixel in the target image group;

[0049] The obstacle recognition module is used to output the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale value of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value.

[0050] In a third aspect, an embodiment of the present application also provides a blind spot monitoring device, which includes a processor, a memory, and a blind spot monitoring program stored in the memory and executable by the processor, wherein when the blind spot monitoring program is executed by the processor, the steps of the above-mentioned blind spot monitoring method are implemented.

[0051] In a fourth aspect, an embodiment of the present application further provides a readable storage medium, on which a blind spot monitoring program is stored, wherein when the blind spot monitoring program is executed by a processor, the steps of the above-mentioned blind spot monitoring method are implemented.

[0052] This application obtains a target image group by analyzing the inter-frame grayscale changes in the background area of ​​the target video data, calculates the grayscale standard value based on the target image group, and outputs the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale value of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value. This application does not rely on feature matching and is not limited to the feature database, thereby reducing missed detections. In addition, this application does not require model training before use, and has lower requirements for computing power and processing speed during use, which helps to reduce implementation costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a method for monitoring a vehicle blind spot in one embodiment of the present application;

[0054] Figure 2 This is a schematic diagram of the process of obstacle recognition and road surface fluctuation recognition in one embodiment of the present application;

[0055] Figure 3This is a schematic diagram of a process of attempting to update a grayscale standard value in an embodiment of the present application;

[0056] Figure 4 This is a schematic diagram of the functional modules of a driving blind spot monitoring device in one embodiment of the present application;

[0057] Figure 5 This is a schematic diagram of the hardware structure of the blind spot monitoring device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0060] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0061] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0062] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0063] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0064] In a first aspect, an embodiment of the present application provides a method for monitoring a driving blind spot.

[0065] Figure 1 A flow chart of a method for monitoring a vehicle's blind spot in an embodiment of the present application is shown.

[0066] Reference Figure 1 In one embodiment, the driving blind spot monitoring method includes the following steps:

[0067] S11, determining a background area and an obstacle monitoring area from the area photographed by the target camera, wherein the background area includes the obstacle monitoring area;

[0068] Blind spot detection systems usually monitor the area around the vehicle by using sensors, cameras or radars. When other vehicles or objects enter the blind spot, the system will sound an alarm to alert the driver. The vision-based blind spot warning system currently on the market includes the following steps:

[0069] 1. Install the camera: First, you need to install the camera at an appropriate position on the side and rear of the vehicle. Generally speaking, the camera will be installed under the rearview mirror or on the side of the vehicle.

[0070] 2. Camera angle adjustment: Adjust the camera angle to cover the blind spot of the vehicle. Make sure the camera can clearly capture the situation on the side and rear.

[0071] 3. Camera connection: Connect the camera to the vehicle's electrical system. The camera is usually connected to the display screen or navigation system on the vehicle.

[0072] 4. Image processing and analysis: The camera captures images around the vehicle in real time. These images are then processed by image processing and analysis algorithms to detect and identify obstacles in blind spots.

[0073] 5. Blind spot alarm: Once the camera detects an obstacle in the blind spot, the system will trigger the alarm mechanism and remind the driver through the vehicle's display screen, sound or vibration.

[0074] 6. Visual display: The display screens on some vehicles will show images captured by the camera to help the driver understand the blind spot more intuitively.

[0075] It should be noted that the performance and quality of the camera have a great impact on the accuracy and reliability of blind spot monitoring. A high-quality camera can provide clear and stable images and work properly under various lighting conditions.

[0076] The target camera in this embodiment belongs to the above-mentioned camera for blind spot monitoring. It can be understood that the area photographed by the target camera includes more than just the obstacle monitoring area that needs to be paid attention to during driving. Therefore, it is necessary to determine the obstacle monitoring area from the area photographed by the target camera, and its demarcation rules are determined according to vehicle characteristics and driving safety requirements. The background area is an area including the obstacle monitoring area, which also includes other areas adjacent to the obstacle monitoring area, and is used to characterize the overall situation of the road surface where the obstacle monitoring area is located.

[0077] It is particularly noted that it is not necessary to monitor the video data of the two cameras on the left and right sides of the vehicle throughout the driving process. For example, if there is no obstacle risk on one side of the vehicle, it is not necessary to monitor the video data of the camera on that side. In this embodiment, the target camera refers to the camera that needs to monitor video data.

[0078] Furthermore, in one embodiment, before the step of determining the background area and the obstacle monitoring area from the area photographed by the target camera, the following step is further included:

[0079] Determine the lane the vehicle is in based on the vehicle positioning system and high-precision map;

[0080] If there are adjacent lanes on the left and / or right side of the lane where the vehicle is located, the camera on the corresponding side of the vehicle is used as the target camera.

[0081] In this embodiment, the target camera is determined according to the lane the vehicle is in. For example, if the vehicle is in the leftmost lane, there is no obstacle risk on its left side, and there is no need to monitor the video data of the left camera. There is an adjacent lane on its right side, and there is an obstacle risk, so it is necessary to monitor the video data of the right camera, and the right camera is used as the target camera.

[0082] Furthermore, in one embodiment, the background area includes all drivable areas in the area photographed by the target camera, and the obstacle monitoring area is located in the adjacent lane area corresponding to the target camera.

[0083] In this embodiment, the background area is the entire road area (excluding trees, guardrails, etc.) in the area captured by the target camera, and the obstacle monitoring area is a part of the adjacent lane area corresponding to the target camera. For example, there are three lanes on the left side of the vehicle, all of which are included in the shooting area of ​​the target camera. Then, these three lane areas are all included in the background area, and only part of the adjacent lane on the left side is used as the obstacle monitoring area.

[0084] Optionally, the drivable area involved in the background area can be determined based on high-precision map data, or edge detection can be performed based on a visual segmentation algorithm.

[0085] Optionally, the obstacle monitoring area is a rectangular area, the length of which is determined by vehicle characteristics and driving safety requirements, and the width of which is determined by the width of an adjacent lane.

[0086] S12, preprocessing the video data collected by the target camera to obtain target video data;

[0087] In this embodiment, the preprocessing process mainly includes time filtering and spatial filtering. The purpose of time filtering is to obtain continuous video frames, and the purpose of spatial filtering is to eliminate camera noise and environmental noise such as rain and snow. Optionally, the preprocessing process can also convert the video data into a form that is easy to process by subsequent algorithms, such as a gray matrix form.

[0088] S13, determining a target image group from the target video data, wherein a grayscale change reference value of the target image group is less than a grayscale change threshold value, the grayscale change reference value of the target image group is calculated based on grayscale change quantization values ​​of all two adjacent frames of images contained in the group, the grayscale change quantization value of two adjacent frames of images is determined based on inter-frame grayscale changes in the background area, and the greater the grayscale change, the greater the corresponding grayscale change quantization value;

[0089] In this embodiment, the grayscale change quantization value is determined based on the inter-frame grayscale change of the background area of ​​two adjacent frames of images, and the grayscale change reference value of a group of continuous images is determined based on the grayscale change quantization values ​​of all two adjacent frames of images of this group of images, and the image group whose grayscale change reference value is less than the grayscale change threshold is used as the target image group.

[0090] It can be understood that, assuming that there are no obstacles and only the road surface in the background area of ​​the continuous multi-frame images taken by the camera over a period of time, and the appearance of the road surface during this period is basically stable, then the grayscale of the background area in these images is also basically stable. The grayscale change reference value of this embodiment is used to reflect the grayscale change of the background area of ​​the continuous multi-frame images. The smaller the grayscale change reference value, the smaller the grayscale change and the lower the possibility of the existence of obstacles. The larger the grayscale change reference value, the larger the grayscale change and the higher the possibility of the existence of obstacles. The grayscale change threshold is used to define whether there are obstacles in the background area. The target image group is a group of continuous multi-frame images that are screened from the target video data based on the above judgment logic, which is considered that there are no obstacles in the background area.

[0091] Furthermore, in one embodiment, the step of determining the target image group from the target video data includes:

[0092] Obtain N consecutive frames of images from the target video data;

[0093] The grayscale change reference value of the N frames of image is calculated according to the first formula group, and the first formula group is:

[0094]

[0095]

[0096]

[0097] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, T f is the grayscale difference threshold of the same pixel in the background area of ​​two adjacent frames of images, A k (i, j) is the two-dimensional value of the grayscale difference between the pixel (i, j) in the kth frame image and the k-1th frame image, D is the background area, A k is the grayscale change quantization value between the k-th frame image and the k-1-th frame image, and A is the grayscale change reference value;

[0098] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are determined to be the target image group.

[0099] In this embodiment, when the target image group has not been successfully acquired in the initial stage of vehicle startup, if the grayscale change reference value of the N frames of images is greater than or equal to the grayscale change threshold, the first frame of the N frames of images is discarded, and the next frame of the Nth frame of images is acquired, and the grayscale change reference value of the new N frames of images is calculated until the target image group is successfully acquired. That is, a sliding window with a sliding step of 1 frame and a size of N frames is created to continuously acquire N consecutive frames of images until the target image group is successfully acquired.

[0100] S14, for each pixel point in the obstacle monitoring area, taking the grayscale average value of the same pixel point in the target image group as the grayscale standard value of the pixel point;

[0101] In this embodiment, the grayscale standard value is used to characterize the grayscale value when there is no obstacle in the obstacle monitoring area. The appropriate expansion of the background area relative to the obstacle monitoring area helps to improve the reliability of the grayscale standard value. Assuming that the target image group is determined and the grayscale standard value is calculated directly based on the grayscale change of the obstacle monitoring area, it is possible that an obstacle will be identified as a stable road surface.

[0102] Specifically, the grayscale standard value is calculated according to the following formula:

[0103]

[0104] Among them, R(i, j) is the grayscale standard value of pixel (i, j), f k (i, j) is the grayscale value of pixel (i, j) in the k-th frame image.

[0105] In the initial stage of vehicle startup, if the target image group has not been successfully acquired, the grayscale standard value cannot be calculated, and the subsequent obstacle recognition operation cannot be performed. At this time, obstacle recognition can be performed with the help of other detection methods, such as radar detection, or using the preset grayscale standard value. After the target image group has been successfully acquired and the grayscale standard value has been calculated, the update of the two will be involved, which will be elaborated in detail later.

[0106] S15. Outputting obstacle recognition results in the obstacle monitoring area according to differences between grayscale values ​​of all pixels in the obstacle monitoring area in subsequent images of the target image group and corresponding grayscale standard values.

[0107] In this embodiment, the greater the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent image and the corresponding grayscale standard value, the higher the possibility of the existence of an obstacle, and the smaller the difference, the lower the possibility of the existence of an obstacle.

[0108] Furthermore, in one embodiment, the step of outputting the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard values ​​comprises:

[0109] Acquire L consecutive frames of images from subsequent images of the target image group;

[0110] The first difference reference value of the L frames of images is calculated according to the second formula group, and the second formula group is:

[0111]

[0112]

[0113]

[0114] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T b is the grayscale difference threshold between the grayscale value of the same pixel in the obstacle monitoring area and the grayscale standard value, B k (i, j) is the pixel point (i, j) in the k-th frame image based on T b The obtained grayscale difference two-dimensional value, d is the obstacle monitoring area, B k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the obstacle monitoring area of ​​the k-th frame image, and B is the first difference reference value;

[0115] If the first difference reference value of the L frames of images is less than the first difference threshold, it is determined that there is no obstacle in the obstacle monitoring area;

[0116] If the first difference reference value of the L frames of images is greater than or equal to the first difference threshold, it is determined that an obstacle exists in the obstacle monitoring area.

[0117] In this embodiment, the first difference reference value is used to reflect the overall average difference between the grayscale of all pixels in the obstacle monitoring area of ​​L consecutive frames of images and the corresponding grayscale standard value. The first difference threshold is used to determine whether there is an obstacle in the background area. Compared with the method of monitoring each image individually, the missed detection rate is lower.

[0118] Therefore, this embodiment obtains a target image group by analyzing the inter-frame grayscale changes of the background area in the target video data, calculates the grayscale standard value based on the target image group, and outputs the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value. This embodiment does not rely on feature matching and is not limited to the feature database, thereby reducing missed detection. In addition, this embodiment does not require model training before use, and has lower requirements for computing power and processing speed during use, which helps to reduce implementation costs.

[0119] It should be noted that the accuracy of the obstacle recognition algorithm in this application depends on the stability of the road surface appearance. For scenes where the road surface appearance changes frequently, its accuracy is difficult to guarantee. The highway area is a typical area with a stable road surface appearance. The application works well in the highway area.

[0120] Figure 2A schematic diagram of the process of obstacle recognition and road surface fluctuation recognition in an embodiment of the present application is shown; Figure 3 A schematic diagram of a process of attempting to update a grayscale standard value in an embodiment of the present application is shown.

[0121] Furthermore, in one embodiment, after the step of determining the target image group from the target video data, the following steps are further included:

[0122] For each pixel in the background area except the obstacle monitoring area, the average grayscale value of the same pixel in these N frames of images is used as the grayscale standard value of the pixel;

[0123] After the step of determining that there is no obstacle in the obstacle monitoring area, the method further includes:

[0124] The second difference reference value of the L frames of images is calculated according to the third formula group, and the third formula group is:

[0125]

[0126]

[0127]

[0128] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T L is the grayscale difference threshold between the grayscale value of the same pixel in the background area and the grayscale standard value, C k (i, j) is the pixel point (i, j) in the k-th frame image based on T L The grayscale difference two-dimensional value obtained, D is the background area, C is the k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the background area of ​​the k-th frame image, and C is the second difference reference value;

[0129] If the second difference reference value of the L frames of images is greater than or equal to the second difference threshold, then taking the first frame of the L frames of images as the starting frame, obtaining N consecutive frames of images from the target video data, and calculating the grayscale change reference value of the N frames of images, where N is equal to the number of image frames in the target image group;

[0130] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

[0131] This embodiment provides a method for updating a target image group and a grayscale standard value. Figure 2 and Figure 3After calculating the first difference reference value of the L-frame image and judging that there is no obstacle, the second difference reference value of the L-frame image will be further calculated. The second difference reference value is used to reflect the overall average difference between the grayscale of all pixels in the background area of ​​the continuous L-frame image and the corresponding grayscale standard value. The second difference threshold is used to determine whether there is road surface fluctuation in the background area. For example, due to factors such as road surface material or external lighting, the overall color of the road surface in the image changes, and there may be obstacles in the area outside the obstacle monitoring area. When it is judged that there is road surface fluctuation according to the second difference reference value, the target image group and the grayscale standard value will be tried to be updated.

[0132] It should be noted that when it is determined that there is road surface fluctuation, only one attempt operation is performed, that is, N consecutive frames of images are obtained, and the grayscale change reference value is calculated. If the grayscale change reference value meets the requirements, the target image group and the grayscale standard value are updated, otherwise no update is performed.

[0133] Furthermore, in one embodiment, after the step of taking the grayscale average value of each pixel in the target image group as the grayscale standard value of the pixel for each pixel in the obstacle monitoring area, the following step is further performed:

[0134] Obtaining N consecutive frames of images from the target video data regularly according to an update cycle, and calculating grayscale change reference values ​​of the N frames of images, where N is equal to the number of image frames in the target image group;

[0135] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

[0136] This embodiment provides another method for updating the target image group and the grayscale standard value. Figure 3 When the update cycle is reached, a trial operation is performed, that is, N consecutive frames of images are obtained, and the grayscale change reference value is calculated. If the grayscale change reference value meets the requirements, the target image group and the grayscale standard value are updated, otherwise no update is performed. For example, the vehicle performs an update attempt every 30 minutes of driving.

[0137] It should be noted that the above two updating methods can be used in combination. The first real-time fluctuation update strategy ensures that the grayscale standard value is updated in time when the road surface changes to a certain extent. The second periodic update strategy ensures that a certain update frequency can be guaranteed even when the road surface changes slightly, thereby comprehensively improving the reliability of the grayscale standard value and ensuring the accuracy of the obstacle recognition algorithm.

[0138] In a second aspect, an embodiment of the present application also provides a driving blind spot monitoring device.

[0139] Figure 4A schematic diagram of functional modules of a driving blind spot monitoring device in one embodiment of the present application is shown.

[0140] Reference Figure 4 In one embodiment, the driving blind spot monitoring device includes:

[0141] The area determination module 10 is used to determine the background area and the obstacle monitoring area from the area photographed by the target camera, wherein the background area includes the obstacle monitoring area;

[0142] The preprocessing module 20 is used to preprocess the video data collected by the target camera to obtain target video data;

[0143] An image determination module 30 is used to determine a target image group from target video data, wherein a grayscale change reference value of the target image group is less than a grayscale change threshold value, the grayscale change reference value of the target image group is calculated based on grayscale change quantization values ​​of all two adjacent frames of images contained in the group, and the grayscale change quantization value of two adjacent frames of images is determined based on inter-frame grayscale changes in the background area, and the greater the grayscale change, the greater the corresponding grayscale change quantization value;

[0144] The standard value calculation module 40 is used to use the grayscale average value of each pixel in the obstacle monitoring area as the grayscale standard value of the pixel in the target image group;

[0145] The obstacle recognition module 50 is used to output the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard values.

[0146] Furthermore, in one embodiment, the image determination module 30 is used to:

[0147] Obtain N consecutive frames of images from the target video data;

[0148] The grayscale change reference value of the N frames of image is calculated according to the first formula group, and the first formula group is:

[0149]

[0150]

[0151]

[0152] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, T f is the grayscale difference threshold of the same pixel in the background area of ​​two adjacent frames of images, A k(i, j) is the two-dimensional value of the grayscale difference between the pixel (i, j) in the kth frame image and the k-1th frame image, D is the background area, A k is the grayscale change quantization value between the k-th frame image and the k-1-th frame image, and A is the grayscale change reference value;

[0153] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are determined to be the target image group.

[0154] Furthermore, in one embodiment, the obstacle identification module 50 is used to:

[0155] Acquire L consecutive frames of images from subsequent images of the target image group;

[0156] The first difference reference value of the L frames of images is calculated according to the second formula group, and the second formula group is:

[0157]

[0158]

[0159]

[0160] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T b is the grayscale difference threshold between the grayscale value of the same pixel in the obstacle monitoring area and the grayscale standard value, B k (i, j) is the pixel point (i, j) in the k-th frame image based on T b The obtained grayscale difference two-dimensional value, d is the obstacle monitoring area, B k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the obstacle monitoring area of ​​the k-th frame image, and B is the first difference reference value;

[0161] If the first difference reference value of the L frames of images is less than the first difference threshold, it is determined that there is no obstacle in the obstacle monitoring area;

[0162] If the first difference reference value of the L frames of images is greater than or equal to the first difference threshold, it is determined that an obstacle exists in the obstacle monitoring area.

[0163] Furthermore, in one embodiment, the standard value calculation module 40 is also used for:

[0164] For each pixel in the background area except the obstacle monitoring area, the average grayscale value of the same pixel in these N frames of images is used as the grayscale standard value of the pixel;

[0165] The blind spot monitoring device also includes an update module for:

[0166] The second difference reference value of the L frames of images is calculated according to the third formula group, and the third formula group is:

[0167]

[0168]

[0169]

[0170] Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T L is the grayscale difference threshold between the grayscale value of the same pixel in the background area and the grayscale standard value, C k (i, j) is the pixel point (i, j) in the k-th frame image based on T L The grayscale difference two-dimensional value obtained, D is the background area, C is the k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the background area of ​​the k-th frame image, and C is the second difference reference value;

[0171] If the second difference reference value of the L frames of images is greater than or equal to the second difference threshold, then taking the first frame of the L frames of images as the starting frame, obtaining N consecutive frames of images from the target video data, and calculating the grayscale change reference value of the N frames of images, where N is equal to the number of image frames in the target image group;

[0172] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

[0173] Furthermore, in one embodiment, the driving blind spot monitoring device further includes an updating module, which is used to:

[0174] Obtaining N consecutive frames of images from the target video data regularly according to an update cycle, and calculating grayscale change reference values ​​of the N frames of images, where N is equal to the number of image frames in the target image group;

[0175] If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

[0176] Furthermore, in one embodiment, the driving blind spot monitoring device further includes a camera determination module, which is used to:

[0177] Determine the lane the vehicle is in based on the vehicle positioning system and high-precision map;

[0178] If there are adjacent lanes on the left and / or right side of the lane where the vehicle is located, the camera on the corresponding side of the vehicle is used as the target camera.

[0179] Furthermore, in one embodiment, the background area includes all drivable areas in the area photographed by the target camera, and the obstacle monitoring area is located in the adjacent lane area corresponding to the target camera.

[0180] Among them, the functional implementation of each module in the above-mentioned blind spot monitoring device corresponds to the various steps in the above-mentioned blind spot monitoring method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0181] In a third aspect, an embodiment of the present application provides a driving blind spot monitoring device, which may be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0182] Figure 5 A schematic diagram of the hardware structure of a blind spot monitoring device involved in an embodiment of the present application is shown.

[0183] Reference Figure 5 In an embodiment of the present application, the driving blind spot monitoring device may include a processor, a memory, a communication interface, and a communication bus.

[0184] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0185] The communication interface includes input / output (I / O) interface, physical interface and logical interface, etc., which are used to realize the interconnection of devices inside the blind spot monitoring device, and the interface used to realize the interconnection between the blind spot monitoring device and other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.

[0186] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0187] The processor may be a general-purpose processor, and the general-purpose processor may call the blind spot monitoring program stored in the memory and execute the blind spot monitoring method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the blind spot monitoring program is called may refer to the various embodiments of the blind spot monitoring method of the present application, which will not be repeated here.

[0188] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0189] In a fourth aspect, an embodiment of the present application also provides a readable storage medium.

[0190] The readable storage medium of the present application stores a driving blind spot monitoring program, wherein when the driving blind spot monitoring program is executed by the processor, the steps of the driving blind spot monitoring method as described above are implemented.

[0191] Among them, the method implemented when the blind spot monitoring program is executed can refer to the various embodiments of the blind spot monitoring method of the present application, and will not be repeated here.

[0192] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0193] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0194] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A driving blind spot monitoring method, characterized in that: The driving blind spot monitoring method comprises: Determine a background area and an obstacle monitoring area from the area photographed by the target camera, wherein the background area includes the obstacle monitoring area; Preprocess the video data collected by the target camera to obtain target video data; Determine a target image group from the target video data, wherein a grayscale change reference value of the target image group is less than a grayscale change threshold value, the grayscale change reference value of the target image group is calculated based on grayscale change quantization values ​​of all two adjacent frames of images contained in the group, the grayscale change quantization value of two adjacent frames of images is determined based on inter-frame grayscale changes in a background area, and the greater the grayscale change, the greater the corresponding grayscale change quantization value; For each pixel in the obstacle monitoring area, the grayscale average value of the same pixel in the target image group is used as the grayscale standard value of the pixel; According to the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value, the obstacle recognition result in the obstacle monitoring area is output.

2. The method for monitoring blind spots in driving according to claim 1, characterized in that: The step of determining a target image group from the target video data comprises: Obtain N consecutive frames of images from the target video data; The grayscale change reference value of the N frames of image is calculated according to the first formula group, and the first formula group is: Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, T f is the grayscale difference threshold of the same pixel in the background area of ​​two adjacent frames of images, A k (i, j) is the two-dimensional value of the grayscale difference between the pixel (i, j) in the kth frame image and the k-1th frame image, D is the background area, A k is the grayscale change quantization value between the k-th frame image and the k-1-th frame image, and A is the grayscale change reference value; If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are determined to be the target image group.

3. The method for monitoring blind spots in driving as claimed in claim 1, characterized in that: The step of outputting the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale values ​​of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value comprises: Acquire L consecutive frames of images from subsequent images of the target image group; The first difference reference value of the L frames of images is calculated according to the second formula group, and the second formula group is: Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T b is the grayscale difference threshold between the grayscale value of the same pixel in the obstacle monitoring area and the grayscale standard value, B k (i, j) is the pixel point (i, j) in the k-th frame image based on T b The obtained grayscale difference two-dimensional value, d is the obstacle monitoring area, B k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the obstacle monitoring area of ​​the k-th frame image, and B is the first difference reference value; If the first difference reference value of the L frames of images is less than the first difference threshold, it is determined that there is no obstacle in the obstacle monitoring area; If the first difference reference value of the L frames of images is greater than or equal to the first difference threshold, it is determined that an obstacle exists in the obstacle monitoring area.

4. The method for monitoring blind spots in driving as claimed in claim 3, characterized in that: After the step of determining the target image group from the target video data, the method further includes: For each pixel in the background area except the obstacle monitoring area, the average grayscale value of the same pixel in these N frames of images is used as the grayscale standard value of the pixel; After the step of determining that there is no obstacle in the obstacle monitoring area, the method further includes: The second difference reference value of the L frames of images is calculated according to the third formula group, and the third formula group is: Among them, f k (i, j) is the gray value of pixel (i, j) in the k-th frame image, R(i, j) is the gray standard value of pixel (i, j), T L is the grayscale difference threshold between the grayscale value of the same pixel in the background area and the grayscale standard value, C k (i, j) is the pixel point (i, j) in the k-th frame image based on T L The grayscale difference two-dimensional value obtained, D is the background area, C k is the sum of the grayscale difference two-dimensional values ​​of all pixels in the background area of ​​the k-th frame image, and C is the second difference reference value; If the second difference reference value of the L frames of images is greater than or equal to the second difference threshold, then taking the first frame of the L frames of images as the starting frame, obtaining N consecutive frames of images from the target video data, and calculating the grayscale change reference value of the N frames of images, where N is equal to the number of image frames in the target image group; If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

5. The method for monitoring blind spots in driving according to claim 1, characterized in that: After the step of taking the grayscale average value of each pixel in the target image group as the grayscale standard value of the pixel for each pixel in the obstacle monitoring area, the method further includes: Obtaining N consecutive frames of images from the target video data regularly according to an update cycle, and calculating grayscale change reference values ​​of the N frames of images, where N is equal to the number of image frames in the target image group; If the grayscale change reference value of the N frames of images is less than the grayscale change threshold, the N frames of images are used as a new target image group, and the grayscale standard value is updated based on the new target image group.

6. The method for monitoring blind spots according to any one of claims 1 to 5, characterized in that: Before the step of determining the background area and the obstacle monitoring area in the area photographed by the target camera, the method further includes: Determine the lane the vehicle is in based on the vehicle positioning system and high-precision map; If there are adjacent lanes on the left and / or right side of the lane where the vehicle is located, the camera on the corresponding side of the vehicle is used as the target camera.

7. The method for monitoring blind spots in driving as claimed in claim 6, characterized in that: The background area includes all drivable areas in the area photographed by the target camera, and the obstacle monitoring area is located in the adjacent lane area corresponding to the target camera.

8. A driving blind spot monitoring device, characterized in that: The driving blind spot monitoring device comprises: An area determination module is used to determine a background area and an obstacle monitoring area from the area photographed by the target camera, wherein the background area includes the obstacle monitoring area; A preprocessing module is used to preprocess the video data collected by the target camera to obtain target video data; An image determination module is used to determine a target image group from target video data, wherein a grayscale change reference value of the target image group is less than a grayscale change threshold value, the grayscale change reference value of the target image group is calculated based on grayscale change quantization values ​​of all two adjacent frames of images contained in the group, the grayscale change quantization value of two adjacent frames of images is determined based on inter-frame grayscale changes in a background area, and the greater the grayscale change, the greater the corresponding grayscale change quantization value; A standard value calculation module is used to take the grayscale average value of each pixel in the obstacle monitoring area as the grayscale standard value of the pixel in the target image group; The obstacle recognition module is used to output the obstacle recognition result in the obstacle monitoring area according to the difference between the grayscale value of all pixels in the obstacle monitoring area in the subsequent images of the target image group and the corresponding grayscale standard value.

9. A driving blind spot monitoring device, characterized in that: The blind spot monitoring device includes a processor, a memory, and a blind spot monitoring program stored in the memory and executable by the processor, wherein when the blind spot monitoring program is executed by the processor, the steps of the blind spot monitoring method as described in any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a blind spot monitoring program, wherein when the blind spot monitoring program is executed by the processor, the steps of the blind spot monitoring method according to any one of claims 1 to 7 are implemented.

Images