Blind spot warning method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a blind spot warning method, device, electronic device and storage medium, the method comprising: obtaining an image to be processed, the image to be processed is an image captured by an image acquisition device installed on a vehicle, performing region recognition on the image to be processed, determining a drivable area included in the image to be processed, the drivable area is an area of ​​the road where a motor vehicle can drive, performing target recognition on the image to be processed, determining an alternative alarm target included in the image to be processed, the alternative alarm target is a target whose area has an intersection with the drivable area, filtering an alarm target from the alternative alarm targets based on the positional relationship between the alternative alarm target and the drivable area, and issuing an alarm for the alarm target. Through this solution, the drivable area and the alternative alarm targets where the drivable area may appear can be determined, and then the alarm target can be determined and an alarm can be issued, which can reduce false alarms and improve the accuracy of blind spot warnings.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle safe driving, and in particular to a blind spot warning method, device, electronic equipment and storage medium. Background Art

[0002] The Safe Driver Assistant System (SDAS) uses sensors installed on the vehicle to collect real-time environmental parameters inside and outside the vehicle, identify, detect and track static and dynamic objects, and alert the driver to possible dangers in the shortest possible time, thereby reducing the frequency of traffic accidents and alleviating the harm of traffic accidents.

[0003] The current blind spot warning method is to detect obstacles in the vehicle's surrounding environment in real time through the safe driving assistance system, such as non-motor vehicles, non-motor vehicles or pedestrians. When the distance between the obstacle and the vehicle is less than the preset threshold, the safe driving assistance system will sound an alarm to remind the driver that danger may occur.

[0004] However, the above-mentioned blind spot warning method may result in a large number of false alarms. For example, for pedestrians at a bus stop, when a vehicle passes by, the distance between these pedestrians and the vehicle may be less than a preset threshold. Therefore, the safe driving assistance system will sound an alarm to remind the driver, but the possibility of a traffic accident at this time is extremely low. It can be seen that the above-mentioned blind spot warning method is prone to false alarms and the accuracy of blind spot warnings is low. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a blind spot warning method, device, electronic device and storage medium to reduce false alarms and improve the accuracy of blind spot warnings. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a blind spot warning method, the method comprising:

[0007] Acquire an image to be processed, wherein the image to be processed is an image captured by an image acquisition device installed on a vehicle;

[0008] Performing region recognition on the image to be processed to determine a drivable region included in the image to be processed, wherein the drivable region is a road region on which a motor vehicle can travel;

[0009] Performing target recognition on the image to be processed to determine candidate alarm targets included in the image to be processed, wherein the candidate alarm targets are targets whose areas intersect with the drivable area;

[0010] Based on the positional relationship between the candidate alarm targets and the drivable area, an alarm target is obtained by filtering the candidate alarm targets, and an alarm is issued for the alarm target.

[0011] Optionally, the step of filtering the candidate alarm targets to obtain the alarm targets based on the positional relationship between the candidate alarm targets and the drivable area includes:

[0012] According to the size of the overlapping portion between the identification frame used to identify the area where each candidate alarm target is located and the drivable area, an alarm target is screened from the candidate alarm targets.

[0013] Optionally, the step of selecting an alarm target from the candidate alarm targets according to the size of the overlapping portion between the identification frame for identifying the area where each candidate alarm target is located and the drivable area includes:

[0014] For each identification frame used to identify the area where the candidate alarm target is located, calculating the proportion of the drivable area and the non-drivable area in a portion of the preset size of the identification frame;

[0015] If the proportion of the drivable area is not less than the proportion of the non-drivable area, determining the candidate alarm target corresponding to the identification frame as the alarm target;

[0016] If the proportion of the drivable area is lower than the proportion of the non-drivable area, the candidate alarm target corresponding to the identification box is determined to be a discarded target.

[0017] Optionally, the step of performing region recognition on the image to be processed and determining a drivable region included in the image to be processed includes:

[0018] Performing pixel segmentation on the image to be processed based on the segmentation mask to obtain the image to be segmented, wherein the segmentation mask at least includes a first pixel value for identifying a drivable area and a second pixel value for identifying a non-drivable area;

[0019] Scan the image to be segmented, and determine the drivable area included in the image to be segmented based on the relationship between the pixel values ​​of the pixels in the image to be segmented and the first pixel value and the second pixel value, wherein the pixel values ​​of the pixels included in the drivable area are the first pixel values.

[0020] Optionally, the step of scanning the image to be segmented and determining the drivable area included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image to be segmented and the first pixel value and the second pixel value includes:

[0021] Scanning the image to be segmented with a target scanning step length in the reverse direction of the longitudinal axis of the image coordinate system of the image to be segmented;

[0022] For each scanning process, when a pixel point having a pixel value equal to the second pixel value is scanned, the scanning is stopped to obtain a current scanning line, wherein the current scanning line is composed of the pixel points scanned this time;

[0023] Based on the connected domain formed by the scan lines obtained by scanning, the drivable area included in the image to be segmented is determined.

[0024] Optionally, the segmentation mask further includes a third pixel value for identifying a transition area;

[0025] The step of scanning the image to be segmented and determining the drivable area included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image to be segmented and the first pixel value and the second pixel value comprises:

[0026] Scanning the image to be segmented with a target scanning step length in the reverse direction of the longitudinal axis of the image coordinate system of the image to be segmented;

[0027] For each scanning process, when a pixel point whose pixel value is the third pixel value is scanned, the height of the pixel point whose pixel value is the third pixel value corresponding to the vertical axis is recorded;

[0028] If the recorded height is not less than the preset height, the scanning is stopped to obtain the current scanning line, wherein the current scanning line is composed of the scanned pixel points before the pixel point whose pixel value is the third pixel value is scanned in the current scanning process;

[0029] If the recorded height is less than the preset height, continue scanning, and when a pixel point whose pixel value is the second pixel value is scanned, stop scanning to obtain a current scan line, wherein the current scan line is composed of the pixels scanned this time;

[0030] Based on the connected domain formed by the scan lines obtained by scanning, the drivable area included in the image to be segmented is determined.

[0031] Optionally, the step of determining the drivable area included in the image to be segmented based on the connected domain composed of scan lines obtained by scanning includes:

[0032] For every two adjacent scan lines whose distance in the horizontal direction of the image coordinate system is not greater than the target scan step length, determining whether a difference between the heights corresponding to the two adjacent scan lines in the vertical direction exceeds a preset threshold;

[0033] The drivable area included in the image to be segmented is determined based on a connected domain composed of a plurality of adjacent scan lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold.

[0034] Optionally, the step of determining the drivable area included in the image to be segmented based on a connected domain composed of a plurality of adjacent scan lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold value comprises:

[0035] Determine a connected region consisting of a plurality of adjacent scan lines whose height differences corresponding to the vertical axis do not exceed the preset threshold as a candidate region;

[0036] Calculating the width of all scan lines included in each candidate area in the horizontal axis direction;

[0037] The candidate area whose width is greater than the preset width is determined as the drivable area included in the image to be segmented.

[0038] Optionally, the step of filtering the candidate alarm targets to obtain the alarm targets based on the positional relationship between the candidate alarm targets and the drivable area includes:

[0039] For each of the drivable areas, calculating the distance between the drivable area and the midline of the horizontal axis of the image to be segmented;

[0040] Determine the drivable area that is closest to the midline of the horizontal axis of the image to be segmented as the current drivable area;

[0041] Based on the positional relationship between the candidate alarm targets and the current drivable area, an alarm target is obtained by filtering from the candidate alarm targets.

[0042] In a second aspect, an embodiment of the present invention provides a blind spot warning device, the device comprising:

[0043] An acquisition module, used for acquiring an image to be processed, wherein the image to be processed is an image acquired by an image acquisition device installed on a vehicle;

[0044] A first recognition module is used to perform region recognition on the image to be processed, and determine a drivable region included in the image to be processed, wherein the drivable region is a road region on which a motor vehicle can travel;

[0045] A second recognition module is used to perform target recognition on the image to be processed, and determine a candidate alarm target included in the image to be processed, wherein the candidate alarm target is a target whose area has an intersection with the drivable area;

[0046] The filtering module is used to filter the candidate alarm targets to obtain the alarm targets based on the positional relationship between the candidate alarm targets and the drivable area, and to issue an alarm for the alarm targets.

[0047] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0048] Memory, used to store computer programs;

[0049] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0050] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0051] Beneficial effects of the embodiments of the present invention:

[0052] In the solution provided by the embodiment of the present invention, the electronic device can obtain an image to be processed, wherein the image to be processed is an image captured by an image acquisition device installed on a vehicle, perform region recognition on the image to be processed, determine the drivable area included in the image to be processed, wherein the drivable area is an area of ​​the road where a motor vehicle can drive, perform target recognition on the image to be processed, determine the candidate alarm targets included in the image to be processed, wherein the candidate alarm targets are targets whose area has an intersection with the drivable area, filter the alarm targets from the candidate alarm targets based on the positional relationship between the candidate alarm targets and the drivable area, and issue an alarm for the alarm targets. In this way, the electronic device can determine the road area where a motor vehicle can drive and the candidate alarm targets that may appear in the road area where the motor vehicle can drive, and then determine the alarm targets that need to be alarmed based on the positional relationship between the road area where the motor vehicle can drive and the candidate alarm targets that may appear in the road area where the motor vehicle can drive, and issue an alarm for the alarm targets. That is, each target in the processed image is filtered to obtain the alarm target that needs to be alarmed. Compared with the current method of alarming by the distance between the obstacle and the vehicle, it can greatly reduce false alarms and improve the accuracy of blind spot warning. Of course, any product or method implementing the present invention does not necessarily need to achieve all the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0054] Figure 1 A flowchart of a blind spot warning method provided by an embodiment of the present invention;

[0055] Figure 2 Based on Figure 1 A flow chart of screening alarm targets implemented as shown;

[0056] Figure 3 for Figure 1 A specific flow chart of step S102 in the illustrated embodiment;

[0057] Figure 4 Based on Figure 3 A schematic diagram of an image to be segmented according to the illustrated embodiment;

[0058] Figure 5 for Figure 3 A specific flow chart of step S302 in the illustrated embodiment;

[0059] Figure 6 for Figure 3 Another specific flow chart of step S302 in the illustrated embodiment;

[0060] Figure 7 for Figure 5 A specific flow chart of step S503 in the illustrated embodiment;

[0061] Figure 8 Based on Figure 7 A schematic diagram of a drivable area included in the image to be segmented in the illustrated embodiment;

[0062] Fig. 9 for Figure 7 A specific flow chart of step S702 in the illustrated embodiment;

[0063] Fig.10 for Figure 1 A specific flow chart of step S104 in the illustrated embodiment;

[0064] Fig.11 Based on Fig.10 A schematic diagram of the embodiment shown in which the image to be segmented includes an alarm target A and an alarm target B;

[0065] Fig.12A structural schematic diagram of a blind spot warning device provided by an embodiment of the present invention;

[0066] Fig.13 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention belong to the scope of protection of the present invention.

[0068] In order to reduce false alarms and improve the accuracy of blind spot warning, the embodiments of the present invention provide a blind spot warning method, device, electronic device, computer readable storage medium and computer program product. First, a blind spot warning method provided by the embodiments of the present invention is introduced.

[0069] A blind spot warning method provided in an embodiment of the present invention is applied to any electronic device that needs to perform blind spot warning, for example, it can be a vehicle-mounted computer, a processor of SDAS, etc., which is not specifically limited here. For the sake of clear description, it is referred to as an electronic device hereinafter.

[0070] like Figure 1 As shown, a blind spot warning method may include:

[0071] S101, obtaining an image to be processed;

[0072] The image to be processed is an image captured by an image capture device installed on a vehicle.

[0073] S102, performing region recognition on the image to be processed to determine a drivable region included in the image to be processed;

[0074] The drivable area is a road surface area on which motor vehicles can travel.

[0075] S103, performing target recognition on the image to be processed to determine candidate alarm targets included in the image to be processed;

[0076] The candidate alarm target is a target whose area intersects with the drivable area.

[0077] S104, based on the positional relationship between the candidate alarm targets and the drivable area, filter the alarm targets from the candidate alarm targets and generate an alarm for the alarm targets.

[0078] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can obtain an image to be processed, wherein the image to be processed is an image captured by an image acquisition device installed on a vehicle, perform region recognition on the image to be processed, determine the drivable area included in the image to be processed, wherein the drivable area is an area of ​​the road where a motor vehicle can drive, perform target recognition on the image to be processed, determine the candidate alarm targets included in the image to be processed, wherein the candidate alarm targets are targets whose area has an intersection with the drivable area, filter the alarm targets from the candidate alarm targets based on the positional relationship between the candidate alarm targets and the drivable area, and alarm the alarm targets. In this way, the electronic device can determine the road area where a motor vehicle can drive and the candidate alarm targets that may appear in the road area where a motor vehicle can drive, and then determine the alarm targets that need to be alarmed based on the positional relationship between the road area where a motor vehicle can drive and the candidate alarm targets that may appear in the road area where a motor vehicle can drive, and alarm the alarm targets. That is, each target in the processed image is filtered to obtain the alarm target that needs to be alarmed. Compared with the current method of alarming by the distance between the obstacle and the vehicle, it can greatly reduce false alarms and improve the accuracy of blind spot warnings.

[0079] When the vehicle is driving, the blind spot warning function can be activated, the image acquisition device installed on the vehicle can collect images in real time, and the electronic device can obtain the images collected by the image acquisition device as images to be processed. According to actual warning needs, the image acquisition device can be installed in one or more positions of the front, rear, left, right, etc. of the vehicle, which is not specifically limited here.

[0080] In an actual driving environment, there are some areas where vehicles cannot drive. When a target is located in such an area, even if the target is close to the vehicle, there will be no danger. At this time, sounding an alarm will result in a false alarm. Therefore, in order to avoid such a false alarm, after acquiring the image to be processed, the electronic device can execute the above step S102, that is, perform area recognition on the image to be processed and determine the drivable area included in the image to be processed.

[0081] The non-drivable area may include, for example, a bus stop, a green belt, etc. The drivable area may be a road area where a motor vehicle can travel. In one embodiment, the electronic device may use image recognition technology to perform area recognition on the image to be processed, determine the road area where the motor vehicle can travel in the image to be processed, and obtain the drivable area.

[0082] In order to determine the target that may need to be alarmed in the image to be processed, the electronic device can perform the above step S103, that is, to identify the target in the image to be processed and determine the candidate alarm target in the image to be processed. The candidate alarm target may include people, non-motor vehicles, animals, etc., which are not specifically limited here.

[0083] In one embodiment, the electronic device can perform target recognition on the above-mentioned image to be processed to obtain the target to be processed, and the target to be processed can be all the targets contained in the image to be processed. Since only the targets appearing in the drivable area are likely to be dangerous, that is, only the targets whose area has an intersection with the drivable area are likely to be dangerous, the targets to be processed can be roughly filtered to obtain the candidate alarm targets.

[0084] Specifically, the electronic device can perform target recognition on the image to be processed, determine the identification frame corresponding to the area where each target to be processed is located, and the identification frame is used to identify the area where the corresponding target to be processed is located. Then, the electronic device can filter out the targets to be processed that have an intersection between the identification frame and the drivable area as candidate alarm targets.

[0085] After determining the drivable area and the candidate alarm targets, the electronic device may execute the above step S104, that is, based on the positional relationship between the candidate alarm targets and the drivable area, filter the alarm targets from the candidate alarm targets and issue an alarm for the alarm targets.

[0086] Since the area where the alternative alarm target is located is generally represented by an identification box, the identification box is an area larger than the actual range of the alternative alarm target. Therefore, the target that intersects with the drivable area may be a target that is not actually located in the road area where motor vehicles can drive.

[0087] Therefore, in order to reduce false alarms and improve the accuracy of blind spot warnings, the electronic device can further filter the alternative alarm targets. In one embodiment, the electronic device can select the alternative alarm target actually located in the road area where the motor vehicle can drive as the alarm target based on the positional relationship between the alternative alarm target and the drivable area, and then can alarm the alarm target.

[0088] For example, if the candidate alarm target is person 1, the drivable area is the main road surface, and the identification frame of person 1 and the main road surface area have an intersection, the electronic device can determine whether person 1 is an alarm target that needs to be alarmed based on the specific position relationship between person 1 and the main road surface. For example, if person 1 is actually on the auxiliary road outside the main road railing, then it can be determined that there is no need to alarm person 1.

[0089] In this way, the electronic device can filter all the targets in the image to be processed twice, and then filter out the alarm targets that really need to be alarmed, and then alarm. Compared with the current method of alarming by the distance between the obstacle and the vehicle, it can reduce false alarms and improve the accuracy of blind spot warnings.

[0090] As an implementation manner of an embodiment of the present invention, the step of filtering the alarm target from the candidate alarm targets based on the positional relationship between the candidate alarm target and the drivable area may include:

[0091] According to the size of the overlapping portion between the identification frame used to identify the area where each candidate alarm target is located and the drivable area, an alarm target is screened from the candidate alarm targets.

[0092] Since the larger the overlap between the identification box used to identify the area where each candidate alarm target is located and the drivable area, the higher the possibility that the candidate alarm target is actually located in the drivable area. Therefore, in order to select an alarm target with a higher possibility of being located in the road area where the motor vehicle can drive and improve the alarm accuracy, the electronic device can select the alarm target from each candidate alarm target according to the size of the overlap between the identification box corresponding to each candidate alarm target and the drivable area.

[0093] In one embodiment, the electronic device can calculate the first area of ​​the drivable area in the identification box corresponding to each alternative alarm target, and calculate the second area of ​​the identification box corresponding to each alternative alarm target. For each alternative alarm target, the ratio of the first area to the second area is calculated. When the ratio is greater than a preset ratio, it means that the overlapping part between the identification box of the area where the alternative alarm target is located and the drivable area is large, then the possibility that the alternative alarm target is actually located in the drivable area is very high, and the electronic device can use the alternative alarm target as an alarm target.

[0094] For example, the preset ratio is 0.4, the first area of ​​the drivable area in the identification box used to identify the area where the alternative alarm target 1 is located is 500, the second area of ​​the identification box used to identify the areas where each alternative alarm target is located is 1000, and the ratio of the first area to the second area is 0.5, which is greater than the preset ratio of 0.4, then alternative alarm target 1 is used as alarm target 1.

[0095] It can be seen that in this embodiment, the electronic device can select the alarm target from each candidate alarm target based on the size of the overlap between the identification frame used to identify the area where each candidate alarm target is located and the drivable area. In this way, the electronic device can select the candidate alarm target that is more likely to be located in the road area where the motor vehicle can drive, i.e., the alarm target, based on the size of the overlap between the identification frame used to identify the area where each candidate alarm target is located and the drivable area, thereby reducing false alarms and improving the accuracy of blind spot warnings.

[0096] As an implementation method of the embodiment of the present invention, Figure 2 As shown, the step of selecting an alarm target from the candidate alarm targets according to the size of the overlapping portion between the identification frame for identifying the area where the candidate alarm targets are located and the drivable area may include:

[0097] S201, for each identification frame used to identify the area where the candidate alarm target is located, calculate the proportion of the drivable area and the non-drivable area in a preset size portion of the identification frame; if the proportion of the drivable area is not less than the proportion of the non-drivable area, execute step S202; if the proportion of the drivable area is less than the proportion of the non-drivable area, execute step S203;

[0098] Since the specific position of the overlapping part between the identification frame of the area where the candidate alarm target is located and the drivable area may be different, and the specific position can identify whether the candidate alarm target is actually located in the drivable area. For example, a person is generally presented in the image to be processed with the feet at the bottom, and the position of the feet is exactly where the person is actually located. Therefore, in order to accurately select the alarm target, the electronic device can calculate the proportion of the drivable area and the non-drivable area in the preset size part of each identification frame used to identify the area where the candidate alarm target is located.

[0099] In the case where the person is presented with the feet at the bottom in the image to be processed, the portion of the preset size of the identification frame may be a portion of the preset size in the lower part of the identification frame, for example, 30%, 50%, 70% of the lower part, etc.

[0100] When the proportion of the drivable area in the preset size part of the identification frame is not less than the proportion of the non-drivable area, it means that the alternative alarm target is actually likely to be located in the drivable area, and an alarm needs to be issued for the alternative alarm target, then the electronic device can execute step S202.

[0101] When the proportion of the drivable area in the preset size part of the identification frame is lower than the proportion of the non-drivable area, it means that the alternative alarm target is actually likely to be located in the non-drivable area, so there is no need to alarm the alternative alarm target, and the electronic device can execute step S203.

[0102] S202, determining that the candidate alarm target corresponding to the identification box is the alarm target;

[0103] For example, the drivable area is the motor vehicle lane, and the non-drivable area is the railing. For the alternative alarm target 2, the electronic device can calculate the proportion of the motor vehicle lane and the railing in the lower half of the identification frame corresponding to the alternative alarm target 2, and obtain that the motor vehicle lane accounts for 85% of the lower half of the identification frame, and the railing accounts for 15% of the lower half of the identification frame. The proportion of the motor vehicle lane is higher than that of the railing. Therefore, it can be determined that the alternative alarm target 2 is located within the railing relative to the vehicle, and there may be danger. It is necessary to alarm the alternative alarm target 2, and then the electronic device can determine the alternative alarm target 2 as the alarm target.

[0104] S203: Determine that the candidate alarm target corresponding to the identification box is a discard target.

[0105] For example, the drivable area is the motor vehicle lane, and the non-drivable area is the railing. For the alternative alarm target 3, the electronic device can calculate the proportion of the motor vehicle lane and the railing in the lower half of the identification frame corresponding to the alternative alarm target 3, and obtain the proportion of the motor vehicle lane in the lower half of the identification frame is 25%, and the proportion of the railing in the lower half of the identification frame is 75%. The proportion of the motor vehicle lane is lower than the proportion of the railing. Therefore, it can be determined that the alternative alarm target 3 is outside the railing relative to the vehicle and will not be dangerous, so there is no need to alarm the alternative alarm target 3. Then the electronic device can determine the alternative alarm target 3 as a discarded target, that is, a target that does not need to be alarmed.

[0106] It can be seen that in this embodiment, the electronic device can calculate the proportion of the drivable area and the non-drivable area in the preset size part of each identification frame used to identify the area where the alternative alarm target is located. If the proportion of the drivable area is not lower than the proportion of the non-drivable area, the alternative alarm target corresponding to the identification frame is determined to be the alarm target. If the proportion of the drivable area is lower than the proportion of the non-drivable area, the alternative alarm target corresponding to the identification frame is determined to be a discarded target. In this way, the electronic device can accurately select the alarm target by calculating the proportion of the drivable area and the non-drivable area in the preset size part of the identification frame, thereby further improving the accuracy of blind spot warning.

[0107] As an implementation method of the embodiment of the present invention, Figure 3 As shown, the above step of performing region recognition on the image to be processed and determining the drivable area included in the image to be processed may include:

[0108] S301, performing pixel segmentation on the image to be processed based on the segmentation mask to obtain an image to be segmented;

[0109] In order to accurately identify the area of ​​the image to be processed, the electronic device can perform pixel segmentation on the image to be processed based on the segmentation mask to obtain the image to be segmented, wherein the segmentation mask includes at least a first pixel value for identifying a drivable area and a second pixel value for identifying a non-drivable area.

[0110] The electronic device can perform pixel segmentation on the image to be processed according to the first pixel value and the second pixel value at least included in the segmentation mask and the pixel value of each pixel point in the image to be processed to obtain the image to be segmented. The regions with different pixel values ​​in the obtained image to be segmented represent different regions in the image to be processed, that is, at least include a drivable region and a non-drivable region.

[0111] In one embodiment, the segmentation mask may also include a third pixel value for identifying a transition area and a fourth pixel value for identifying a background, etc. Furthermore, when the electronic device performs pixel segmentation on the processed image based on the segmentation mask, the image to be segmented also includes a transition area identified by the third pixel value and a background identified by the fourth pixel value, etc.

[0112] For example, the segmentation mask 1 includes a first pixel value for identifying a drivable area, a second pixel value for identifying a non-drivable area, and a fourth pixel value for identifying a background, wherein the first pixel value is 1, the second pixel value is 2, and the fourth pixel value is 3. The electronic device performs pixel segmentation on the image to be processed 1 based on the segmentation mask 1 to obtain the image to be segmented 1, wherein the pixel value of the drivable area is 1, the pixel value of the non-drivable area is 2, and the pixel value of the background is 3.

[0113] In another embodiment, the first pixel value may include multiple first-category sub-pixel values ​​for identifying drivable areas such as motor vehicle lanes, bus lanes and / or cargo lanes, and the second pixel value may include multiple second-category sub-pixel values ​​for identifying non-drivable areas such as railings, bus stops and / or green belts.

[0114] Furthermore, the electronic device can perform pixel segmentation on the image to be processed based on the segmentation mask to obtain an image to be segmented, wherein the image to be segmented may include multiple drivable areas marked with different first-category sub-pixel values, multiple non-drivable areas marked with different second-category sub-pixel values, and a background marked with a fourth pixel value.

[0115] For example, the segmentation mask may include two different first-category sub-pixel values ​​for identifying a main road lane and a secondary road lane, and three different second-category sub-pixel values ​​for identifying a guardrail, a bus stop, and a green belt.

[0116] S302, scanning the image to be segmented, and determining a drivable area included in the image to be segmented based on a relationship between pixel values ​​of pixels in the image to be segmented and the first pixel value and the second pixel value;

[0117] After obtaining the image to be segmented, the electronic device can scan the image to be segmented, and then determine the drivable area included in the image to be segmented based on the relationship between the pixel values ​​of the pixel points in the image to be segmented and the first pixel value and the second pixel value, wherein the pixel values ​​of the pixel points included in the drivable area are the first pixel values.

[0118] For example, in the image a to be segmented, the pixel value of the drivable area is 1, the pixel value of the non-drivable area is 2, and the pixel value of the background is 3. The electronic device can scan the image a to be segmented, and then determine the drivable area, non-drivable area and background included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image a to be segmented and the first pixel value 1, the second pixel value 2 and the fourth pixel value 3.

[0119] like Figure 4 The image a to be segmented shown includes a drivable area 410 marked with a first pixel value, a non-drivable area 420 marked with a second pixel value, and a background 430 marked with a fourth pixel value.

[0120] As an implementation mode, in the case where the sizes of the segmentation mask and the image to be processed are different, the electronic device can obtain the size of the image to be processed and the size of the segmentation mask, and then calculate the scaling factor between the segmentation mask and the image to be processed. According to the scaling factor, the mapping relationship between the pixel points of the image to be segmented obtained by pixel segmentation and the image to be processed can be determined. Then, after determining the above-mentioned drivable area, non-drivable area, etc., the electronic device can determine the corresponding drivable area, non-drivable area, etc. in the image to be processed according to the mapping relationship.

[0121] It can be seen that in this embodiment, the electronic device can perform pixel segmentation on the image to be processed based on the segmentation mask to obtain the image to be segmented, and then scan the image to be segmented, and determine the drivable area included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image to be segmented and the first pixel value and the second pixel value. In this way, the electronic device can use the segmentation mask to perform pixel segmentation on the image to be processed, so as to accurately determine the drivable area in the image to be segmented by scanning the image to be segmented, so as to further improve the accuracy of blind spot warning.

[0122] As an implementation method of the embodiment of the present invention, Figure 5 As shown, the step of scanning the image to be segmented and determining the drivable area included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image to be segmented and the first pixel value and the second pixel value may include:

[0123] S501, scanning the image to be segmented with a target scanning step length in the reverse direction of the longitudinal axis of the image coordinate system of the image to be segmented;

[0124] To determine the specific scanning mode, the electronic device may obtain configuration parameters, wherein the configuration parameters may include the number of scanning lines. Further, the electronic device may calculate the target scanning step length based on the size of the image to be segmented and the number of scanning lines, wherein the target scanning step length = the size of the image to be segmented / the number of scanning lines.

[0125] After determining the scanning step length, the electronic device can use the coordinate point corresponding to the lower left corner of the image to be segmented in the image coordinate system as the starting point, and scan the pixels in the image to be segmented in the opposite direction of the vertical axis of the image coordinate system of the image to be segmented using the target scanning step length. That is, using the coordinate point corresponding to the lower left corner of the image to be segmented in the image coordinate system as the starting point, scan each column of pixels in the image to be segmented from bottom to top according to the target scanning step length.

[0126] S502, for each scanning process, when a pixel point whose pixel value is the second pixel value is scanned, stop scanning to obtain a current scanning line;

[0127] For this scanning process, the electronic device can start from the starting point corresponding to the lower boundary of the image to be segmented in the image coordinate system, and scan upward the column corresponding to the starting point in the image to be segmented. Each time a pixel point is scanned, it is determined whether the pixel value of the pixel point is the first pixel value or the second pixel value. If the pixel value of the pixel point is the first pixel value, it means that the pixel point is the pixel point corresponding to the drivable area, so the electronic device can continue to scan the next pixel point.

[0128] If the pixel value of the pixel point is the second pixel value, it means that the pixel point is a pixel point corresponding to the non-drivable area, so the electronic device can stop scanning. Then, the current scan line composed of the scanned pixels is obtained, and the current scan line is composed of the pixel points with the first pixel value.

[0129] After the current scan is completed, for the next scan process, the electronic device can use the coordinate point that is at the target scan step length from the starting point of the current scan process as the starting point for the next scan of the image to be segmented, until the scan of the image to be segmented is completed, at which point the image to be segmented contains multiple scan lines.

[0130] For example, the size of the image 2 to be segmented is n*b, the first pixel value is 1, the second pixel value is 2, and the target scanning step is a. For the first scanning process, the electronic device takes the coordinates (a, b) of the lower boundary of the image 2 to be segmented in the image coordinate system as the starting point, and scans the image 2 to be segmented in the opposite direction of the vertical axis of the image coordinate system of the image to be segmented. When a pixel point with a pixel value of 1 is scanned, the next pixel point is scanned until a pixel point with a pixel value of 2 and a coordinate of (a, b / 2) is scanned, and the scanning is stopped. Then, the electronic device can use the scanned pixel points as the current scanning line, that is, the line segment with endpoints (a, b) and (a, b / 2).

[0131] S503: Determine a drivable area included in the image to be segmented based on a connected domain formed by scan lines obtained by scanning.

[0132] After the electronic device completes scanning the image to be segmented, the image to be segmented contains multiple scan lines representing the drivable area. The electronic device can determine the connected domain composed of multiple scan lines with a close distance based on the scan lines obtained by scanning, and then determine the drivable area included in the image to be segmented based on the connected domain.

[0133] It can be seen that in this embodiment, the electronic device can scan the image to be segmented in the opposite direction of the longitudinal axis of the image coordinate system of the image to be segmented using the target scanning step size. For each scanning process, when a pixel point with a pixel value of the second pixel value is scanned, the scanning is stopped to obtain the current scanning line, and the drivable area included in the image to be segmented is determined based on the connected domain composed of the scanned scanning lines. In this way, the drivable area included in the image to be segmented can be accurately determined to ensure the accuracy of subsequent blind spot warnings.

[0134] As an implementation of an embodiment of the present invention, the segmentation mask may further include a third pixel value for identifying a transition area, wherein the transition area may be an area in the image to be processed that may represent both a drivable area and a non-drivable area. For example, the image to be processed is a panoramic image of a vehicle, and the transition area may be a main vehicle body located in the drivable area, where the main vehicle body is the vehicle body corresponding to the panoramic image. The transition area may also be a target that does not require blind spot warning, such as other vehicle bodies that are normally traveling in the drivable area.

[0135] The transition area can ensure the integrity of the connected domain. For example, the image to be processed is a panoramic view of a vehicle, and the transition area is the rearview mirror of the vehicle. When scanning the image to be segmented, the electronic device can pass through a smaller transition area, i.e., the rearview mirror, until it reaches the truly non-drivable area, and then stop scanning, thereby obtaining a complete connected domain.

[0136] like Figure 6 As shown, the step of scanning the image to be segmented and determining the drivable area included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image to be segmented and the first pixel value and the second pixel value may include:

[0137] S601, scanning the image to be segmented with a target scanning step length in the reverse direction of the longitudinal axis of the image coordinate system of the image to be segmented;

[0138] Since step S601 is the same as the above step S501, please refer to the description of the above step S501 and will not be described again here.

[0139] S602, for each scanning process, when a pixel point whose pixel value is the third pixel value is scanned, record the height of the pixel point whose pixel value is the third pixel value corresponding to the vertical axis; if the recorded height is not less than the preset threshold, execute step S603; if the recorded height is less than the preset threshold, execute step S604.

[0140] For this scanning process, the electronic device can start from the starting point corresponding to the lower boundary of the image to be segmented in the image coordinate system, and scan upward the column corresponding to the starting point in the image to be segmented. Each time a pixel point is scanned, it is determined whether the pixel value of the pixel point is the first pixel value, the second pixel value, or the third pixel value. If the pixel value of the pixel point is the first pixel value, it means that the pixel point is the pixel point corresponding to the drivable area, so the electronic device can continue to scan the next pixel point.

[0141] If the pixel value of the pixel point is the second pixel value, it means that the pixel point is a pixel point corresponding to the non-drivable area, so the electronic device can stop scanning. Then, the current scan line composed of the scanned pixels is obtained, and the current scan line is composed of the pixel points with the first pixel value.

[0142] If the pixel value of the pixel point is the third pixel value, it means that the pixel point is the pixel point corresponding to the transition area. Since the transition area can ensure the integrity of the connected domain, when the transition area is large, it means that the transition area is represented as a non-drivable area. Therefore, the electronic device can execute step S603.

[0143] When the transition area is small, it means that the transition area may be a rearview mirror of the vehicle or some discrete points with segmentation errors, and the electronic device may continue scanning, so the electronic device may execute step S604.

[0144] S603, if the recorded height is not less than the preset height, stop scanning and obtain the current scanning line;

[0145] If the height recorded by the electronic device is not less than the preset height, it means that the transition area corresponding to the recorded height is larger, and further it can be said that the transition area represents a non-drivable area. Then the electronic device can stop scanning to obtain the current scan line, wherein the preset height is set according to the height of the image to be segmented and the actual situation, and is not specifically limited here. The current scan line is composed of the scanned pixel points before the pixel point with the third pixel value is scanned during this scanning process, that is, the current scan line is composed of pixel points with the first pixel value.

[0146] S604, if the recorded height is less than the preset height, continue scanning, and when a pixel point with a pixel value equal to the second pixel value is scanned, stop scanning to obtain a current scan line;

[0147] If the height recorded by the electronic device is less than the preset height, it means that the transition area corresponding to the recorded height is small, which further indicates that the transition area may be a vehicle rearview mirror or some discrete points with segmentation errors, and the electronic device can continue scanning.

[0148] When a pixel point with a second pixel value is scanned, scanning is stopped to obtain the current scan line, wherein the current scan line is composed of the pixel points scanned this time, that is, the current scan line is composed of pixel points with the first pixel value and the third pixel value.

[0149] After the current scan is completed, for the next scan process, the electronic device can use the coordinate point that is at the target scan step length from the starting point of the current scan process as the starting point for the next scan of the image to be segmented, until the scan of the image to be segmented is completed, at which point the image to be segmented contains multiple scan lines.

[0150] For example, the size of the image 3 to be segmented is m*b, the first pixel value is 1, the second pixel value is 2, the third pixel value is 3, the target scanning step is a, and the preset height is c. For the first scanning process, the electronic device scans the image 3 to be segmented from the coordinates (a, b) of the lower boundary of the image 3 to be segmented in the image coordinate system as the starting point. When a pixel point with a pixel value of 1 is scanned, the next pixel point is scanned. When a pixel point with a pixel value of 3 and a coordinate of (a, b / 2) is scanned, the height d corresponding to the pixel point with the third pixel value on the vertical axis is recorded.

[0151] If the height d is not less than the preset height c, it means that the transition area corresponding to the recorded height is larger, and it can be explained that the transition area indicates a non-drivable area. Then the electronic device can stop scanning and obtain the current scanning line, that is, the line segment with endpoints (a, b) and (a, b / 2+1).

[0152] If the height d is less than the preset height c, it means that the recorded height corresponds to a transition area that is too small, which means that the transition area may be a vehicle rearview mirror or some discrete points with segmentation errors, and the scan line can pass through to continue searching for a drivable area. Therefore, the electronic device can continue scanning until it scans a pixel point (a, b / 4) with a pixel value of 2, and then stop scanning. The electronic device can use the scanned pixel points as the current scan line, that is, a line segment with endpoints (a, b) and (a, b / 4).

[0153] S605 , determining a drivable area included in the image to be segmented based on a connected domain formed by scan lines obtained by scanning.

[0154] Since step S605 is the same as the above step S503, please refer to the description of the above step S503 and will not be repeated here.

[0155] It can be seen that in this embodiment, the image to be segmented is scanned in the opposite direction of the vertical axis of the image coordinate system of the image to be segmented using the target scanning step length. For each scanning process, when a pixel point with a pixel value of the third pixel value is scanned, the height of the pixel point with the pixel value of the third pixel value on the vertical axis is recorded. If the recorded height is not less than the preset height, the scanning is stopped to obtain the current scanning line. If the recorded height is less than the preset height, the scanning is continued. When a pixel point with a pixel value of the second pixel value is scanned, the scanning is stopped to obtain the current scanning line. Based on the connected domain composed of the scanned scanning lines, the drivable area included in the image to be segmented is determined. In this way, the electronic device can scan the image to be segmented, and when the pixel value is the third pixel value used to identify the transition area, it does not stop immediately, but stops scanning according to the stop conditions in the above two cases. In the image to be segmented with a relatively complex scene, the drivable area can be accurately determined to ensure the accuracy of the subsequent blind spot warning.

[0156] As an implementation method of the embodiment of the present invention, Figure 7 As shown, the step of determining the drivable area included in the image to be segmented based on the connected domain composed of the scan lines obtained by scanning may include:

[0157] S701, for every two adjacent scan lines whose distance in the horizontal direction of the image coordinate system is not greater than the target scan step length, determining whether a difference between the heights corresponding to the two adjacent scan lines in the vertical direction exceeds a preset threshold;

[0158] Since the scan lines obtained by scanning are obtained with the target scan step length as the interval, if the distance between every two adjacent scan lines in the horizontal axis direction of the image coordinate system is not greater than the target scan step length, it means that there is no un-drivable area between the two adjacent scan lines, so the two adjacent scan lines belong to the same connected domain.

[0159] When two adjacent scan lines suddenly change in height corresponding to the vertical axis, it means that the drivable area to which the two adjacent scan lines belong has changed dramatically, and the drivable areas to which the two adjacent scan lines belong are likely not the same drivable area.

[0160] Therefore, after scanning the obtained scan lines, the electronic device can determine, for each two adjacent scan lines in the image to be segmented, whether the distance between the two scan lines in the horizontal axis direction of the image coordinate system is not greater than the target scan step length. After determining that the distance between the two scan lines in the horizontal axis direction of the image coordinate system is not greater than the target scan step length, it is further determined whether the difference in height corresponding to the two adjacent scan lines in the vertical axis exceeds a preset threshold.

[0161] S702, determining a drivable area included in the image to be segmented based on a connected domain formed by a plurality of adjacent scan lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold.

[0162] Next, the electronic device can select a connected domain composed of multiple adjacent scan lines whose corresponding height difference on the vertical axis does not exceed a preset threshold as the drivable area included in the image to be segmented, wherein the scan lines that constitute the same connected domain are scan lines whose distance between every two adjacent scan lines in the horizontal axis direction of the image coordinate system is not greater than the target scanning step length and whose corresponding height difference on the vertical axis does not exceed the preset threshold.

[0163] For example, Figure 8 As shown, the electronic device selects every two adjacent scanning lines whose distance in the horizontal direction of the image coordinate system is not greater than the target scanning step length and whose corresponding height difference in the vertical axis does not exceed a preset threshold, and can form two drivable areas, namely the first drivable area 810 and the second drivable area 820.

[0164] It can be seen that in this embodiment, the electronic device can determine whether the height difference between the two adjacent scan lines in the vertical axis exceeds the preset threshold for each two adjacent scan lines whose distance in the horizontal axis direction of the image coordinate system is not greater than the target scan step length, and determine the drivable area included in the image to be segmented based on the connected domain composed of multiple adjacent scan lines whose height difference in the vertical axis does not exceed the preset threshold. In this way, the electronic device can obtain an accurate drivable area, further improving the accuracy of blind spot warning.

[0165] As an implementation method of the embodiment of the present invention, Fig. 9 As shown, the step of determining the drivable area included in the image to be segmented based on the connected domain composed of a plurality of adjacent scan lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold value may include:

[0166] S901, determining a connected region consisting of a plurality of adjacent scan lines whose height differences corresponding to the vertical axis do not exceed the preset threshold as a candidate region;

[0167] After determining the connected domain in the image to be segmented, since there may be multiple connected domains and there may be connected domains that are misdetected due to detection errors, the electronic device may filter the connected domains in order to more accurately determine the drivable area.

[0168] Specifically, the electronic device may determine a connected domain consisting of a plurality of adjacent scan lines whose height differences corresponding to the vertical axis do not exceed a preset threshold as a candidate region, that is, all connected domains are regarded as candidate regions.

[0169] S902, calculating the width of all scan lines included in each candidate area in the horizontal axis direction;

[0170] If the width of all scan lines included in the candidate area in the horizontal direction is very small, then its area is likely to be small, and the candidate area is likely to be a connected domain detected by mistake. Therefore, the electronic device can calculate the width of all scan lines included in each candidate area in the horizontal direction.

[0171] In one embodiment, the width between the first scan line and the last scan line included in the candidate area can be determined as the width of all the scan lines included in the candidate area in the horizontal axis direction. In another embodiment, the electronic device can calculate the product of the number of all the scan lines included in each candidate area and the target scan step length, and use the product as the width of all the scan lines included in the candidate area in the horizontal axis direction.

[0172] For example, the drivable area included in the image to be segmented is Figure 8As shown, the two drivable areas include a first drivable area 810 and a second drivable area 820. The electronic device can calculate Figure 8 The width of the scan line in the first drivable area 810 in the horizontal direction can also be calculated. Figure 8 The width of the scanning line in the second drivable area 820 in the horizontal direction.

[0173] S903: Determine the candidate area whose width is greater than a preset width as the drivable area included in the image to be segmented.

[0174] In order to more accurately select the drivable area, the electronic device can filter out the drivable area with a smaller area in the image to be segmented, so the electronic device can determine the candidate area with a width greater than a preset width as the drivable area included in the image to be segmented. The preset width can be set according to the width of the image to be segmented and the actual situation, and is not specifically limited here.

[0175] It can be seen that in this embodiment, the electronic device can determine the connected domain composed of multiple adjacent scan lines whose height difference on the vertical axis does not exceed the preset threshold as the candidate area, and then calculate the width of all scan lines included in each candidate area in the horizontal axis direction, and determine the candidate area with a width greater than the preset width as the drivable area included in the image to be segmented. In this way, the electronic device can filter out the misdetected candidate areas with smaller areas in the image to be segmented, so as to determine a more accurate drivable area, thereby reducing false alarms and further improving the accuracy of blind spot warnings.

[0176] As an implementation method of the embodiment of the present invention, Fig.10 As shown, the above step of filtering the alarm target from the candidate alarm targets based on the positional relationship between the candidate alarm targets and the drivable area may include:

[0177] S1001, for each of the drivable areas, calculating the distance between the drivable area and the midline of the horizontal axis of the image to be segmented;

[0178] Since the image to be segmented may include multiple drivable areas, but the blind spot warning is aimed at the drivable area where the vehicle is currently traveling, and the image acquisition device is generally installed in the middle position of the vehicle, that is, the vehicle is actually located at the position corresponding to the center of the image to be processed. Therefore, in order to determine the drivable area where the vehicle is currently traveling, the electronic device can determine the center line of the horizontal axis of the image to be segmented, and then, for each drivable area, calculate the distance between the drivable area and the center line of the horizontal axis of the image to be segmented.

[0179] S1002, determining the drivable area closest to the midline of the horizontal axis of the image to be segmented as the current drivable area;

[0180] After calculating the distance between each drivable area and the midline of the horizontal axis of the image to be segmented, the electronic device can determine the drivable area that is closest to the midline of the horizontal axis of the image to be segmented as the current drivable area, where the current drivable area is the drivable area where the vehicle is currently traveling.

[0181] For example, if the width of the image b to be segmented is 400 pixels, then the center line is the vertical line perpendicular to the horizontal axis corresponding to the coordinate of 200 pixels, and the electronic device can then select the drivable area closest to the vertical line as the current drivable area.

[0182] For example Figure 8 For the image to be segmented shown in FIG. 8 , the electronic device can select the drivable area 810 closest to the vertical line as the current drivable area.

[0183] S1003: Filter the candidate alarm targets to obtain an alarm target based on the positional relationship between the candidate alarm targets and the current drivable area.

[0184] After acquiring the current drivable area, ie, the drivable area in which the vehicle is currently traveling, the electronic device can filter the alarm target from the candidate alarm targets based on the positional relationship between the candidate alarm targets in the image to be segmented and the current drivable area.

[0185] The method of determining the alarm target based on the positional relationship between the alternative alarm target and the current drivable area is the same as the above-mentioned method of determining the alarm target based on the positional relationship between the alternative alarm target and the drivable area, that is, the electronic device can calculate the proportion of the current drivable area and the non-drivable area in a preset size part of each identification box used to identify the area where the alternative alarm target is located; if the proportion of the current drivable area is not lower than the proportion of the non-drivable area, the alternative alarm target corresponding to the identification box is determined to be the alarm target; if the proportion of the current drivable area is lower than the proportion of the non-drivable area, the alternative alarm target corresponding to the identification box is determined to be a discarded target.

[0186] For example, Fig.11In the image to be segmented shown, the current drivable area 1110 is a motor vehicle lane, and the non-drivable area 1120 is a railing. For the candidate alarm target A 1130, the electronic device can calculate the proportion of the motor vehicle lane and the railing in the 70% portion of the lower part of the identification frame corresponding to the candidate alarm target A 1130, and obtain that the motor vehicle lane accounts for 15% of the lower half of the identification frame, and the railing accounts for 55% of the lower half of the identification frame. The proportion of the motor vehicle lane is lower than the proportion of the railing, so it can be considered that the candidate alarm target A 1130 is located outside the railing relative to the vehicle and will not cause danger, so there is no need to alarm the candidate alarm target A 1130, and then the electronic device can determine the candidate alarm target A 1130 as a discarded target, that is, a target that does not need to be alarmed.

[0187] For the candidate alarm target B 1140, the electronic device can calculate the proportion of the motor vehicle lane and the railing in the 70% portion of the lower portion of the identification frame corresponding to the candidate alarm target B 1140, and obtain that the motor vehicle lane accounts for 55% of the lower half of the identification frame, and the railing accounts for 45% of the lower half of the identification frame. The proportion of the motor vehicle lane is not less than the proportion of the railing, so it can be determined that the candidate alarm target B 1140 is located within the railing relative to the vehicle, and danger may occur, and the candidate alarm target B 1140 needs to be alarmed, and then the electronic device can determine the candidate alarm target B1140 as the alarm target.

[0188] It can be seen that in this embodiment, the electronic device can calculate the distance between each drivable area and the center line of the horizontal axis of the image to be segmented, determine the drivable area with the closest distance to the center line of the horizontal axis of the image to be segmented as the current drivable area, and filter the alarm target from the candidate alarm targets based on the positional relationship between the candidate alarm targets and the current drivable area. The electronic device can calculate the proportion of the current drivable area and the non-drivable area in the portion of the preset size of the identification frame, and then more accurately select the alarm target, thereby further improving the accuracy of the blind spot warning.

[0189] Corresponding to the above-mentioned blind spot warning method, an embodiment of the present invention further provides a blind spot warning device. The following is an introduction to the blind spot warning device provided by the embodiment of the present invention.

[0190] like Fig.12 As shown, a blind spot warning device may include:

[0191] An acquisition module 1210 is used to acquire an image to be processed;

[0192] The image to be processed is an image captured by an image capture device installed on a vehicle.

[0193] A first recognition module 1220, configured to perform region recognition on the image to be processed, and determine a drivable region included in the image to be processed;

[0194] The drivable area is a road surface area on which motor vehicles can travel.

[0195] The second recognition module 1230 is used to perform target recognition on the image to be processed and determine the candidate alarm targets included in the image to be processed;

[0196] The candidate alarm target is a target whose area intersects with the drivable area.

[0197] The filtering module 1240 is used to filter the candidate alarm targets to obtain an alarm target based on the positional relationship between the candidate alarm target and the drivable area, and to issue an alarm for the alarm target.

[0198] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can obtain an image to be processed, wherein the image to be processed is an image captured by an image acquisition device installed on a vehicle, perform region recognition on the image to be processed, determine the drivable area included in the image to be processed, wherein the drivable area is an area of ​​the road where a motor vehicle can drive, perform target recognition on the image to be processed, determine the candidate alarm targets included in the image to be processed, wherein the candidate alarm targets are targets whose area has an intersection with the drivable area, filter the alarm targets from the candidate alarm targets based on the positional relationship between the candidate alarm targets and the drivable area, and alarm the alarm targets. In this way, the electronic device can determine the road area where a motor vehicle can drive and the candidate alarm targets that may appear in the road area where a motor vehicle can drive, and then determine the alarm targets that need to be alarmed based on the positional relationship between the road area where a motor vehicle can drive and the candidate alarm targets that may appear in the road area where a motor vehicle can drive, and alarm the alarm targets. That is, each target in the processed image is filtered to obtain the alarm target that needs to be alarmed. Compared with the current method of alarming by the distance between the obstacle and the vehicle, it can greatly reduce false alarms and improve the accuracy of blind spot warnings.

[0199] As an implementation of the embodiment of the present invention, the filtering module 1240 may include:

[0200] The screening unit is used to screen the alarm targets from the candidate alarm targets according to the size of the overlapping part between the identification frame used to identify the area where the candidate alarm targets are located and the drivable area.

[0201] As an implementation of an embodiment of the present invention, the screening unit may include:

[0202] A first calculation subunit is used to calculate the proportion of the drivable area and the non-drivable area in a portion of a preset size of each identification frame used to identify the area where the candidate alarm target is located;

[0203] A first determination subunit is used to determine the candidate alarm target corresponding to the identification frame as the alarm target if the proportion of the drivable area is not less than the proportion of the non-drivable area;

[0204] The second determining subunit is used to determine the candidate alarm target corresponding to the identification box as a discarded target if the proportion of the drivable area is lower than the proportion of the non-drivable area.

[0205] As an implementation of the embodiment of the present invention, the first identification module 1220 may include:

[0206] A segmentation unit, configured to perform pixel segmentation on the image to be processed based on the segmentation mask to obtain an image to be segmented;

[0207] The segmentation mask at least includes a first pixel value for identifying a drivable area and a second pixel value for identifying a non-drivable area.

[0208] A scanning unit is used to scan the image to be segmented, and determine the drivable area included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image to be segmented and the first pixel value and the second pixel value.

[0209] Among them, the pixel values ​​of the pixel points included in the drivable area are the first pixel values.

[0210] As an implementation of an embodiment of the present invention, the scanning unit may include:

[0211] A first scanning subunit, configured to scan the image to be segmented in a direction opposite to the longitudinal axis of the image coordinate system of the image to be segmented with a target scanning step length;

[0212] A second scanning subunit is used for, in each scanning process, when a pixel point having a pixel value equal to the second pixel value is scanned, stopping scanning to obtain a current scanning line, wherein the current scanning line is composed of the pixels scanned this time;

[0213] The third determining subunit is used to determine the drivable area included in the image to be segmented based on the connected domain composed of the scan lines obtained by scanning.

[0214] As an implementation of an embodiment of the present invention, the scanning unit may include:

[0215] A third scanning subunit is used to scan the image to be segmented in the reverse direction of the longitudinal axis of the image coordinate system of the image to be segmented with a target scanning step length;

[0216] A fourth scanning subunit is configured to record, during each scanning process, a height of the pixel point having the third pixel value corresponding to the vertical axis when a pixel point having the third pixel value is scanned;

[0217] A fifth scanning subunit, configured to stop scanning if the recorded height is not less than a preset height, and obtain a current scanning line, wherein the current scanning line is composed of scanned pixel points before a pixel point having a pixel value of the third pixel value is scanned during this scanning process;

[0218] a sixth scanning subunit, configured to continue scanning if the recorded height is less than the preset height, and stop scanning when scanning a pixel point whose pixel value is the second pixel value, to obtain a current scanning line, wherein the current scanning line is composed of the pixels scanned this time;

[0219] The fourth determining subunit determines the drivable area included in the image to be segmented based on the connected domain formed by the scan lines obtained by scanning.

[0220] As an implementation manner of an embodiment of the present invention, the third determining subunit may include:

[0221] a fifth determination subunit, configured to determine, for each two adjacent scan lines whose distance in the horizontal direction of the image coordinate system is not greater than the target scan step length, whether a difference between the heights corresponding to the two adjacent scan lines in the vertical direction exceeds a preset threshold;

[0222] The sixth determination subunit is used to determine the drivable area included in the image to be segmented based on a connected domain composed of a plurality of adjacent scanning lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold.

[0223] As an implementation manner of an embodiment of the present invention, the sixth determining subunit may include:

[0224] A seventh determination subunit is used to determine a connected region formed by a plurality of adjacent scanning lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold as a candidate region;

[0225] A second calculation subunit, used for calculating the width of all scan lines included in each candidate area in the horizontal axis direction;

[0226] The eighth determination subunit is used to determine the candidate area with a width greater than a preset width as the drivable area included in the image to be segmented.

[0227] As an implementation of the embodiment of the present invention, the filtering module 1240 may include:

[0228] A calculation unit, used for calculating, for each of the drivable areas, a distance between the drivable area and a midline of the horizontal axis of the image to be segmented;

[0229] A determination unit, configured to determine a drivable area that is closest to a midline of a horizontal axis of the image to be segmented as a current drivable area;

[0230] A filtering unit is used to filter the alarm target from the candidate alarm targets based on the positional relationship between the candidate alarm target and the current drivable area.

[0231] The embodiment of the present invention further provides an electronic device, such as Fig.13 As shown, it includes a processor 1301, a communication interface 1302, a memory 1303 and a communication bus 1304, wherein the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304.

[0232] Memory 1303, used for storing computer programs;

[0233] The processor 1301 is used to implement the blind spot warning method steps described in any of the above embodiments when executing the program stored in the memory 1303.

[0234] It can be seen that in the solution provided by the embodiment of the present invention, the electronic device can obtain an image to be processed, wherein the image to be processed is an image captured by an image acquisition device installed on a vehicle, perform region recognition on the image to be processed, determine the drivable area included in the image to be processed, wherein the drivable area is an area of ​​the road where a motor vehicle can drive, perform target recognition on the image to be processed, determine the candidate alarm targets included in the image to be processed, wherein the candidate alarm targets are targets whose area has an intersection with the drivable area, filter the alarm targets from the candidate alarm targets based on the positional relationship between the candidate alarm targets and the drivable area, and alarm the alarm targets. In this way, the electronic device can determine the road area where a motor vehicle can drive and the candidate alarm targets that may appear in the road area where a motor vehicle can drive, and then determine the alarm targets that need to be alarmed based on the positional relationship between the road area where a motor vehicle can drive and the candidate alarm targets that may appear in the road area where a motor vehicle can drive, and alarm the alarm targets. That is, each target in the processed image is filtered to obtain the alarm target that needs to be alarmed. Compared with the current method of alarming by the distance between the obstacle and the vehicle, it can greatly reduce false alarms and improve the accuracy of blind spot warnings.

[0235] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0236] The communication interface is used for communication between the above electronic device and other devices.

[0237] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0238] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0239] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned blind spot warning methods are implemented.

[0240] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any blind spot warning method in the above embodiments.

[0241] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0242] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusions, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0243] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for devices, electronic devices, computer-readable storage media, and computer program products, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0244] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A blind spot warning method, characterized in that: The method comprises: Acquire an image to be processed, wherein the image to be processed is an image captured by an image acquisition device installed on a vehicle; Performing region recognition on the image to be processed to determine a drivable region included in the image to be processed, wherein the drivable region is a road region on which a motor vehicle can travel; Performing target recognition on the image to be processed to determine candidate alarm targets included in the image to be processed, wherein the candidate alarm targets are targets whose identification boxes of the areas where they are located intersect with the drivable area; Based on the positional relationship between the candidate alarm targets and the drivable area, filtering the alarm targets from the candidate alarm targets, and issuing an alarm for the alarm targets; The step of performing region recognition on the image to be processed and determining the drivable area included in the image to be processed includes: Performing pixel segmentation on the image to be processed based on the segmentation mask to obtain the image to be segmented, wherein the segmentation mask at least includes a first pixel value for identifying a drivable area, a second pixel value for identifying a non-drivable area, and a third pixel value for identifying a transition area; Scan the image to be segmented, determine the drivable area included in the image to be segmented based on the relationship between the pixel values ​​of the pixels in the image to be segmented and the first pixel value and the second pixel value, and use the height of the pixel whose scanned pixel value is the third pixel value on the vertical axis of the image coordinate system of the image to be segmented as a judgment criterion for whether to stop scanning, wherein the pixel values ​​of the pixels included in the drivable area are the first pixel values.

2. The method according to claim 1, characterized in that The step of filtering the candidate alarm targets to obtain the alarm targets based on the positional relationship between the candidate alarm targets and the drivable area comprises: According to the size of the overlapping portion between the identification frame used to identify the area where each candidate alarm target is located and the drivable area, an alarm target is screened from the candidate alarm targets.

3. The method according to claim 2, characterized in that The step of selecting an alarm target from the candidate alarm targets according to the size of the overlapping portion between the identification frame for identifying the area where the candidate alarm targets are located and the drivable area comprises: For each identification frame used to identify the area where the candidate alarm target is located, calculating the proportion of the drivable area and the non-drivable area in a portion of the preset size of the identification frame; If the proportion of the drivable area is not less than the proportion of the non-drivable area, determining the candidate alarm target corresponding to the identification frame as the alarm target; If the proportion of the drivable area is lower than the proportion of the non-drivable area, the candidate alarm target corresponding to the identification box is determined to be a discarded target.

4. The method according to claim 1, characterized in that The step of scanning the image to be segmented, determining the drivable area included in the image to be segmented based on the relationship between the pixel value of the pixel point in the image to be segmented and the first pixel value and the second pixel value, and taking the height of the pixel point whose scanned pixel value is the third pixel value on the vertical axis of the image coordinate system of the image to be segmented as a criterion for determining whether to stop scanning, comprises: Scanning the image to be segmented with a target scanning step length in the reverse direction of the longitudinal axis of the image coordinate system of the image to be segmented; For each scanning process, when a pixel point whose pixel value is the third pixel value is scanned, the height of the pixel point whose pixel value is the third pixel value corresponding to the vertical axis is recorded; If the recorded height is not less than the preset height, the scanning is stopped to obtain the current scanning line, wherein the current scanning line is composed of the scanned pixel points before the pixel point whose pixel value is the third pixel value is scanned in the current scanning process; If the recorded height is less than the preset height, continue scanning, and when a pixel point whose pixel value is the second pixel value is scanned, stop scanning to obtain a current scan line, wherein the current scan line is composed of the pixels scanned this time; Based on the connected domain formed by the scan lines obtained by scanning, the drivable area included in the image to be segmented is determined.

5. The method according to claim 4, characterized in that The step of determining the drivable area included in the image to be segmented based on the connected domain composed of the scan lines obtained by scanning comprises: For every two adjacent scan lines whose distance in the horizontal direction of the image coordinate system is not greater than the target scan step length, determining whether a difference between the heights corresponding to the two adjacent scan lines in the vertical direction exceeds a preset threshold; The drivable area included in the image to be segmented is determined based on a connected domain composed of a plurality of adjacent scan lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold.

6. The method according to claim 5, characterized in that The step of determining the drivable area included in the image to be segmented based on the connected domain composed of a plurality of adjacent scan lines whose height differences corresponding to the longitudinal axis do not exceed the preset threshold value comprises: Determine a connected region consisting of a plurality of adjacent scan lines whose height differences corresponding to the vertical axis do not exceed the preset threshold as a candidate region; Calculating the width of all scan lines included in each candidate area in the horizontal axis direction; The candidate area whose width is greater than the preset width is determined as the drivable area included in the image to be segmented.

7. The method according to claim 5, characterized in that The step of filtering the candidate alarm targets to obtain the alarm targets based on the positional relationship between the candidate alarm targets and the drivable area comprises: For each of the drivable areas, calculating the distance between the drivable area and the midline of the horizontal axis of the image to be segmented; Determine the drivable area that is closest to the midline of the horizontal axis of the image to be segmented as the current drivable area; Based on the positional relationship between the candidate alarm targets and the current drivable area, an alarm target is obtained by filtering from the candidate alarm targets.

8. A blind spot warning device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be processed, wherein the image to be processed is an image acquired by an image acquisition device installed on a vehicle; A first recognition module is used to perform region recognition on the image to be processed, and determine a drivable region included in the image to be processed, wherein the drivable region is a road region on which a motor vehicle can travel; A second recognition module is used to perform target recognition on the image to be processed, and determine a candidate alarm target included in the image to be processed, wherein the candidate alarm target is a target whose identification frame of the area where the target is located intersects with the drivable area; A filtering module, configured to filter an alarm target from the candidate alarm targets based on a positional relationship between the candidate alarm targets and the drivable area, and to generate an alarm for the alarm target; Wherein, the first identification module is specifically used for: Performing pixel segmentation on the image to be processed based on the segmentation mask to obtain the image to be segmented, wherein the segmentation mask at least includes a first pixel value for identifying a drivable area, a second pixel value for identifying a non-drivable area, and a third pixel value for identifying a transition area; Scan the image to be segmented, determine the drivable area included in the image to be segmented based on the relationship between the pixel values ​​of the pixels in the image to be segmented and the first pixel value and the second pixel value, and use the height of the pixel whose scanned pixel value is the third pixel value on the vertical axis of the image coordinate system of the image to be segmented as a judgment criterion for whether to stop scanning, wherein the pixel values ​​of the pixels included in the drivable area are the first pixel values.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images