Vehicle lens smudginess warning method

Abstract

The invention relates to a vehicle lens smudginess warning method, which comprises the following steps of: when a vehicle is in a backing state, acquiring continuous images of a shooting range through a vehicle lens; at least three angular points are obtained from the Nth frame of image of the continuous image through the processing module according to the bounding box information, so that a specific area is formed, the specific area does not cover the four corners of the Nth frame of image, and the area of the specific area is smaller than that of the Nth frame of image; through the processing module, according to the image identification model, smudginess image feature detection is carried out on the Nth frame image of the continuous images to obtain smudginess information, and the smudginess information comprises a smudginess area; and through the processing module, according to the smudginess information, judging the proportion between the smudginess area of the smudginess area overlapped in the specific area and the area of the specific area, and when the proportion exceeds a specific proportion, generating a smudginess warning.

Description

Technical Field

[0001] The present invention relates to a vehicle lens auxiliary method, more specifically, a vehicle lens captures an environmental image when the vehicle is in a reverse state, and warns the user after determining in real time whether the lens is dirty, thereby achieving the purpose of improving driving safety. Background Art

[0002] In recent years, thanks to the shrinking size of various camera lenses or sensors, many applications based on images or various environmental information have become more and more widespread. Nowadays, the use of camera lenses for driving records or driving safety has almost become a standard feature of cars, motorcycles and even bicycles. For cars, the currently known electronic rearview mirror uses a camera lens installed at the rear of the car with a pedestrian recognition function to provide environmental detection at the rear of the car when reversing, send the coordinates and relative speed of pedestrians behind the car, and cooperate with the active emergency braking system (Autonomous Emergency Braking, AEB) to automatically brake the vehicle in dangerous situations, effectively reducing the occurrence of driving accidents.

[0003] However, these camera lenses are mainly exposed to the outside of the vehicle and will be affected by various pollution when driving under some extreme road conditions. For example, water droplets, mud, ice and snow on the road surface may splash onto the camera surface, causing the camera to become dirty, which will reduce the image gradient and affect pedestrian recognition. It cannot reflect the real road condition information, causing the control unit to make incorrect judgments, reducing the safety of vehicle reversing, and having a great impact on car driving safety.

[0004] In the prior art, the means of identifying camera dirtiness is poor. Although only a single frame of image is needed for judgment, it is easily affected by factors such as light changes and background interference. The fault tolerance is too poor and the recognition accuracy is low. In addition, because of the deformation caused by shooting at the edge of the camera lens, false detection and missed detection are prone to occur. Summary of the invention

[0005] The present invention provides a method for warning a dirty vehicle lens, which aims to avoid false detection caused by edge deformation by dividing the entire image into specific areas, and because the area to be processed for identification is smaller, the dirty area can be judged more quickly and accurately.

[0006] The present invention provides a method for warning a dirty vehicle lens, which is applicable to a vehicle. The vehicle comprises a vehicle lens, a processing module, and a warning module. The method comprises the following steps: when the vehicle is in a reversing state, obtaining at least one continuous image of a shooting range through the vehicle lens; obtaining at least three corner points from an N-th frame image of at least one of the continuous images through the processing module according to a boundary box information, so as to form a specific area on the N-th frame image, wherein the specific area does not cover the four corners of the N-th frame image, and the area of ​​the specific area is smaller than the area of ​​the N-th frame image; performing dirty image feature detection on the N-th frame image of at least one of the continuous images according to an image recognition model through the processing module to obtain dirty information, wherein the dirty information comprises a dirty area; and generating a dirty warning through the processing module according to the dirty information when it is determined that the ratio between a dirty area overlapped in the specific area by the dirty area and the area of ​​the specific area exceeds a specific ratio.

[0007] In one embodiment of the present invention, the vehicle further comprises at least one memory unit for storing the bounding box information and the image recognition model. The memory unit further comprises a bounding box database and an image recognition model database.

[0008] In one embodiment of the present invention, the vehicle camera is a reversing camera, which is disposed on the center line of the vehicle and located behind the vehicle.

[0009] In one embodiment of the present invention, the specific area of ​​the Nth frame image is a first pedestrian recognition area located in an image coordinate system, which is used to detect whether there is a pedestrian at a specific position of a second pedestrian recognition area located in a world coordinate system after coordinate conversion when the vehicle is in a reversing state.

[0010] In one embodiment of the present invention, the above-mentioned step of obtaining at least three corner points in an N-th frame image of at least one of the continuous images according to a bounding box information through the processing module to form a specific area on the N-th frame image also includes: selecting at least three of the corner points as reference points according to the bounding box information, and taking the closed area surrounded by the lines connecting the at least three corner points as the specific area.

[0011] In one embodiment of the present invention, the specific area of ​​the Nth frame image is a first pedestrian recognition area located in an image coordinate system. After coordinate conversion, the first pedestrian recognition area corresponds to a second pedestrian recognition area located in a world coordinate system. In the world coordinate system with the position of the vehicle lens as the origin of the X-axis and Y-axis coordinates, the second pedestrian recognition area is located in the area of ​​-400 cm to 400 cm in the X-axis direction of the vehicle lens, and the second pedestrian recognition area is located in the area of ​​0 cm to 900 cm in the Y-axis direction of the vehicle lens.

[0012] In one embodiment of the present invention, the area of ​​the specific region is 1 / 2 to 2 / 3 of the area of ​​the Nth frame image.

[0013] In one embodiment of the present invention, the above-mentioned image recognition model is a semantic segmentation model, which is used to detect dirt image features to obtain the dirt information. The dirt information also includes a dirt, which is located at a coordinate position and a size of the specific area.

[0014] In one embodiment of the present invention, the semantic segmentation model is a fully convolutional neural network model (FCN model), a U-net model or an Enet model (efficient neural network model).

[0015] In an embodiment of the present invention, the specific ratio is one of 10%, 15%, 20% and 30%.

[0016] The effect of the present invention is that by dividing the full image obtained by the lens into specific areas, the distortion and low resolution of the picture taken by the edge of the lens are avoided, and the specific area is used as the identification area that actually needs to be detected when the vehicle is in a reversing state. Through the AI ​​recognition model, it is accurately identified whether the dirt area in the specific area exceeds a specific ratio, and a warning is issued, which effectively reduces the time required for traditional dirt judgment using the full screen, and improves the recognition accuracy, thereby achieving the purpose of improving false detection and missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart showing a method for warning a dirty vehicle lens according to the present invention;

[0018] Figure 2 A schematic diagram showing a dirty area of ​​a dirty warning method for a vehicle lens of the present invention;

[0019] Figure 3 A schematic diagram showing a first pedestrian recognition area of ​​a method for warning a dirty vehicle lens according to the present invention;

[0020] Figure 4 A schematic diagram showing a second pedestrian recognition area of ​​a method for warning a dirty vehicle lens according to the present invention;

[0021] Figure 5 A schematic diagram showing a second pedestrian recognition area calibration method of a vehicle lens dirt warning method of the present invention.

[0022] Among them: S110~S140 step flow, 1. Nth frame image, 11. specific area, 12. dirty area, 21. first pedestrian recognition area, 22. second pedestrian recognition area, 2. pedestrian, 3. car. DETAILED DESCRIPTION

[0023] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description.

[0024] Figure 1 The flowchart of the steps of the vehicle lens dirt warning method of the present invention is shown. The steps are as follows:

[0025] Step S110: when the vehicle is in a reverse state, obtaining at least one continuous image of a shooting range through the vehicle lens;

[0026] Step S120: obtaining at least three corner points from an N-th frame image of at least one of the continuous images through the processing module according to a boundary box information, so as to form a specific area on the N-th frame image, wherein the specific area does not cover the four corners of the N-th frame image, and the area of ​​the specific area is smaller than the area of ​​the N-th frame image;

[0027] Step S130: performing dirt image feature detection on the Nth frame image of at least one of the continuous images according to an image recognition model through the processing module to obtain dirt information, wherein the dirt information includes a dirt area;

[0028] Step S140: generating a dirt warning through the processing module according to the dirt information when it is determined that the ratio between a dirt area of ​​the dirt region overlapping in the specific region and the area of ​​the specific region exceeds a specific ratio.

[0029] In this embodiment, the vehicle further includes at least one memory unit for storing the bounding box information and the image recognition model. The memory units further include a bounding box database and an image recognition model database.

[0030] In this embodiment, the vehicle camera is a reversing camera, which is disposed on the center line of the vehicle and located at the rear of the vehicle.

[0031] In this embodiment, the specific area of ​​the Nth frame image is a first pedestrian recognition area located in an image coordinate system, which is used to detect whether there is a pedestrian at a specific position of a second pedestrian recognition area located in a world coordinate system after coordinate conversion when the vehicle is in a reversing state.

[0032] In this embodiment, at least three corner points are selected as reference points according to the boundary box information, and a closed area surrounded by a line connecting at least three corner points is used as the specific area. The specific area may be, but is not limited to, a circular area, an elliptical area, a triangular area, a quadrilateral area, or a polygonal area.

[0033] In this embodiment, the specific area of ​​the Nth frame image is a first pedestrian recognition area located in an image coordinate system. After coordinate conversion, the first pedestrian recognition area corresponds to a second pedestrian recognition area located in a world coordinate system. In the world coordinate system with the position of the vehicle lens as the origin of the X-axis and Y-axis coordinates, the second pedestrian recognition area is located in the area of ​​-400cm to 400cm in the X-axis direction of the vehicle lens, and the second pedestrian recognition area is located in the area of ​​0cm to 900cm in the Y-axis direction of the vehicle lens. The second pedestrian recognition area is a quadrilateral area in the world coordinate system. Preferably, the quadrilateral area is 9m×8m (length×width).

[0034] In this embodiment, the area of ​​the specific region is 1 / 2 to 2 / 3 of the area of ​​the Nth frame image.

[0035] In this embodiment, the image recognition model is a semantic segmentation model, which is used to detect dirt image features to obtain the dirt information, and the dirt information further includes dirt, a coordinate position and a size of the dirt located in the specific area. The semantic segmentation model is used to obtain a plurality of semantic segmentation areas of the Nth frame image of each of the continuous images to perform dirt image feature detection.

[0036] In this embodiment, the semantic segmentation model is a fully convolutional neural network model (FCN model), a U-net model or an Enet model (efficient neural network model).

[0037] In this embodiment, the specific ratio is one of 10%, 15%, 20% and 30%. If it is determined that the dirt area is greater than or equal to the specific ratio of the total area of ​​the specific area (for example, the specific ratio is 30%), an alarm unit is used to present the dirt alarm as one of a light signal or an alarm sound.

[0038] Figure 2 A schematic diagram of a dirty area of ​​a dirty lens warning method of the present invention is shown. Through the processing module, at least three corner points are obtained from an N-th frame image 1 of at least one of the continuous images according to a boundary box information to form a specific area 11 on the N-th frame image 1, wherein the specific area 11 does not cover the four corners of the N-th frame image 1, and the area of ​​the specific area 11 is smaller than the area of ​​the N-th frame image 1; and dirty image feature detection is performed on the N-th frame image 1 of at least one of the continuous images according to an image recognition model to obtain dirt information, wherein the dirt information includes a dirty area 12, such as Figure 2 Wherein, through the processing module, according to the dirt information, when it is determined that the ratio between a dirt area of ​​the dirt area 12 overlapping in the specific area 11 and the area of ​​the specific area 11 exceeds a specific ratio (for example, the specific ratio is 30%), a dirt warning is generated.

[0039] In this embodiment, at least three corner points are selected as reference points according to the boundary box information, and a closed area surrounded by lines connecting at least three corner points is the specific area 11. The specific area 11 may be, but is not limited to, a circular area, an elliptical area, a triangular area, a quadrilateral area or a polygonal area.

[0040] Please also refer to Figure 3 and Figure 4 , Figure 3 A schematic diagram showing a first pedestrian recognition area of ​​a method for warning a dirty vehicle lens according to the present invention; Figure 4 The diagram is a second pedestrian recognition area diagram of a dirty vehicle lens warning method according to the present invention.

[0041] In this embodiment, the specific area 11 is a first pedestrian recognition area 21 located in an image coordinate system, which is used to detect whether there is a pedestrian at a specific position of a second pedestrian recognition area 22 located in a world coordinate system after coordinate conversion when the vehicle is in a reversing state.

[0042] The Autonomous Emergency Braking (AEB) system is currently being vigorously promoted by new car safety assessment associations in various countries. For example, in the rear-view test method specified by the European New Car Assessment Program (Euro-NCAP), when the reversing speed is 4km / h and 8km / h, it must be able to identify adults and children stationary behind the vehicle. In addition, when the reversing speed is 4km / h and 8km / h, it must be able to identify adults and children moving at 5km / h behind the vehicle.

[0043] In the present embodiment, the specific area 11 is a first pedestrian recognition area 21 located in an image coordinate system. Coordinate registration is performed through a calibration method. After coordinate conversion, the first pedestrian recognition area 21 corresponds to a second pedestrian recognition area 22 located in a world coordinate system. In the world coordinate system with the position of the vehicle lens as the origin of the X-axis and Y-axis coordinates, the second pedestrian recognition area 22 is located in the area of ​​-400 cm to 400 cm in the X-axis direction of the vehicle lens, and the second pedestrian recognition area 22 is located in the area of ​​0 cm to 900 cm in the Y-axis direction of the vehicle lens. Figure 2 , Figure 3 and Figure 4 shown.

[0044] Wherein, the reverse speed is 8 km / h and the pedestrian movement speed is 5 km / h, and then the second pedestrian recognition area 22 in the world coordinate system is deduced to be a quadrilateral area of ​​9m×8m (length×width), such as Figure 4 shown.

[0045] The specific area 11 may be, but is not limited to, a circular area, an elliptical area, a triangular area, a quadrilateral area or a polygonal area.

[0046] Figure 5 A schematic diagram showing a second pedestrian recognition area calibration method of a vehicle lens dirt warning method of the present invention.

[0047] In this embodiment, according to the second pedestrian recognition area calibration method specified by the European New Car Assessment Program (Euro-NCAP), when the reversing speed is 8 km / h, only a child is required for testing. At this time, the rear 50% of the car 3 is the collision position. When the pedestrian 2 starts walking from the side of the car 3 at a distance of 4 m from the point L, the speed is 5 km / h, and F=1.5 m is the acceleration distance.

[0048] Assume that the time taken by pedestrian 2 in the acceleration section F=1.5m is t1, the acceleration is a, and the time taken to reach point L is t.

[0049]

[0050] t1=2×F÷5000=0.0006h=2.16s

[0051] t=[(4-1.5)÷5000]×3600+2.16=1.8+2.16=3.96s

[0052] Pedestrian 2 needs 3.96 seconds to reach point L;

[0053] Assume that the acceleration of car 3 is a2 = 2 m / s 2 , the time taken to accelerate from 0 to 8 km / h is t2, and the acceleration distance is s;

[0054]

[0055] Braking distance of car 3 Where μ is the friction coefficient, which is usually about 0.8, and g is the acceleration due to gravity, which is approximately 9.8 m / s 2 .

[0056] When v = 8 km / h = 2.22 m / s, s1 = (2.22) 2 / (2×0.8×9.8)=0.31m;

[0057] If the reaction time of AEB is 0.5s, pedestrian 2 walks to point L, and the distance traveled by car 3 is s2 = 1.23 + (8000 / 3600) × (3.96-1.11 + 0.5) + 0.31 = 9m (rounded off);

[0058] Therefore, in order for the pedestrian 2 to reach the point L without being hit by the car 3, the car must start reversing at least 9 meters away from the point L. Based on this deduction, at least the second pedestrian recognition area 22 is defined as having a length×width=9m×8m.

[0059] In summary, the present invention divides the full image obtained by the lens into specific areas to avoid the disadvantages of image distortion and low resolution captured by the edge of the lens, and uses the specific areas as the identification areas that actually need to be detected when the vehicle is in a reversing state. The AI ​​identification model is used to accurately identify whether the dirt area in the specific area exceeds a specific ratio, and a warning is issued, which effectively reduces the time required for traditional dirt judgment using the full screen, and improves the identification accuracy, thereby achieving the purpose of improving false detection and missed detection.

[0060] Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any equivalent substitutions made by persons skilled in the art without departing from the spirit and scope of the present invention are still within the scope of patent protection of the present invention.

Claims

1. A vehicle lens dirt warning method, applicable to a vehicle, characterized in that: The vehicle comprises a vehicle lens, a processing module, and a warning module. The steps of the vehicle lens dirt warning method include: When the vehicle is in a reverse state, at least one continuous image of a shooting range is obtained through the vehicle lens; By means of the processing module, at least three corner points are obtained from an N-th frame image of at least one of the continuous images according to a boundary box information, so as to form a specific area on the N-th frame image, wherein the specific area does not cover the four corners of the N-th frame image, and the area of ​​the specific area is smaller than the area of ​​the N-th frame image; Through the processing module, according to an image recognition model, dirt image feature detection is performed on the Nth frame image of at least one of the continuous images to obtain dirt information, wherein the dirt information includes a dirt area; and Through the processing module, according to the dirt information, when it is determined that the ratio between a dirt area of ​​the dirt region overlapping in the specific region and the area of ​​the specific region exceeds a specific ratio, a dirt warning is generated.

2. A method for warning a dirty vehicle lens as claimed in claim 1, characterized in that: The vehicle also includes at least one memory unit for storing the boundary box information and the image recognition model. The memory units also include a boundary box database and an image recognition model database.

3. The method for warning a dirty vehicle lens as claimed in claim 1, characterized in that: The vehicle lens is a reversing lens, which is arranged on the center line of the vehicle and located at the rear of the vehicle.

4. The method for warning a dirty vehicle lens as claimed in claim 1, characterized in that: The specific area of ​​the Nth frame image is a first pedestrian recognition area located in an image coordinate system, which is used to detect whether there is a pedestrian at a specific position of a second pedestrian recognition area located in a world coordinate system after coordinate conversion when the vehicle is in a reversing state.

5. The method for warning a dirty vehicle lens as claimed in claim 1, characterized in that: The step of obtaining at least three corner points from an N-th frame image of at least one of the continuous images according to a boundary box information through the processing module to form a specific area on the N-th frame image also includes: selecting at least three of the corner points as reference points according to the boundary box information, and taking the closed area surrounded by the lines connecting the at least three corner points as the specific area.

6. The method for warning a dirty vehicle lens as claimed in claim 1, characterized in that: The specific area of ​​the Nth frame image is a first pedestrian recognition area located in an image coordinate system. After coordinate conversion, the first pedestrian recognition area corresponds to a second pedestrian recognition area located in a world coordinate system. In the world coordinate system with the position of the vehicle lens as the origin of the X-axis and Y-axis coordinates, the second pedestrian recognition area is located in the area of ​​-400 cm to 400 cm in the X-axis direction of the vehicle lens, and the second pedestrian recognition area is located in the area of ​​0 cm to 900 cm in the Y-axis direction of the vehicle lens.

7. The method for warning a dirty vehicle lens as claimed in claim 1, characterized in that: The area of ​​the specific region is 1 / 2 to 2 / 3 of the area of ​​the Nth frame image.

8. The vehicle lens dirt warning method according to claim 1, characterized in that: The image recognition model is a semantic segmentation model, which is used to detect dirt image features to obtain the dirt information. The dirt information also includes a dirt, a coordinate position and a size of the dirt located in the specific area.

9. A method for warning a dirty vehicle lens as claimed in claim 8, characterized in that: The semantic segmentation model is a fully convolutional neural network model. Networks model, FCN model), U-net model or Enet model (efficient neural network model).

10. The vehicle lens dirt warning method according to claim 1, characterized in that: The specific ratio is one of 10%, 15%, 20% and 30%.

Images