Fixture detection unit, fixture detection method, and program

By designing a fixed object detection unit in the vehicle camera system, the problem of high error detection rate is solved by shooting, storing and comparing the fixed object coordinate information in the image, the problem of high error detection rate is improved, and the detection accuracy is expanded to detect raindrops and lenses. defect.

CN119096553APending Publication Date: 2024-12-06NUVOTON TECH CORP JAPAN NAGAOKAKYO CITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380030386.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-31
Filing Date
2023-03-22
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art is prone to incorrect detection when detecting attached dirt in vehicle camera images, resulting in a low detection rate.

Method used

A fixed object detection unit is designed, including a shooting device, a fixed object area information acquisition unit, a storage unit and an observation unit. By searching for fixed object candidates in the image, storing their coordinate information, and by comparing with past stored information, it is determined. Whether it is a persistent fixed object, the judgment result will be output to reduce false detection.

Benefits of technology

It effectively reduces the occurrence of false detection, improves the detection rate of attached fixtures, and can detect cracks and scratches on raindrops and lenses or window glass.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119096553A_ABST
    Figure CN119096553A_ABST
Patent Text Reader

Abstract

A stationary object detection unit (102) is provided with: an imaging device (101); a fixture region information acquisition unit (dirt region information acquisition unit (201)) that searches for, as a fixture candidate, a candidate of a fixture in the image captured by the imaging device (101), and acquires fixture region information indicating coordinates of the fixture candidate in the image, the fixture being a fixture having a fixed relative position to the imaging device (101); a storage means (204) for storing the acquired fixed object area information; and an observation means (205) that determines, as the coordinates of the fixture in the image, the coordinates indicated by the fixture region information in which the coordinates coincide with the coordinates of the fixture region information previously stored in the storage means (204), and outputs the determination result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] An object of the present invention is to improve the detection rate in a system for detecting attached dirt in an image via a camera mounted on a vehicle. Background Art

[0002] In the use of camera monitoring systems installed in vehicles, the importance of ensuring their field of view is increasing. For example, dirt reflected in the image by the camera will form a blind spot in the driver's field of view. In addition, in ADAS, dirt can be a major cause of damage to vehicle safety. Therefore, there is an increasing demand for detecting attached dirt and warning the operator or automatically cleaning the dirt.

[0003] Several mechanisms have been proposed to detect dirt reflected in an image via a camera. For example, in Patent Document 1, a fixed object is determined based on the adjacent small areas of brightness information of each small area obtained by dividing a predetermined area in a captured image and a brightness threshold as determination conditions.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-51381 Summary of the invention

[0007] Problem that the invention aims to solve

[0008] However, in the conventional method described above, false detection may easily occur.

[0009] The purpose of the present invention is to reduce the occurrence of false detection and improve the detection rate of attached fixed objects.

[0010] Means used to solve problems

[0011] In order to solve the above-mentioned problems and achieve the purpose, the fixed object detection unit of the present invention comprises: a photographing device; a fixed object area information acquisition unit, which searches for a candidate of a fixed object reflected in an image photographed by the photographing device as a fixed object candidate, and acquires fixed object area information representing the coordinates of the fixed object candidate in the image, wherein the fixed object is a fixed object whose relative position with respect to the photographing device is fixed; a storage unit, which stores the acquired fixed object area information; and an observation unit, which determines the coordinates indicated by the fixed object area information that are consistent with the coordinates of the fixed object area information stored in the past in the storage unit as the coordinates of the fixed object in the image, and outputs the determination result.

[0012] Effects of the Invention

[0013] According to the present invention, the occurrence of erroneous detection can be reduced and the detection rate of attached fixed objects can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a configuration diagram of a fixed object detection unit according to Embodiment 1 of the present invention.

[0015] Figure 2 It is an explanatory diagram of the contamination area information according to the first embodiment of the present invention.

[0016] Figure 3 It is an explanatory diagram of the coupling unit according to the first embodiment of the present invention.

[0017] Figure 4 The flow of the observation unit in the first embodiment of the present invention is Figure 1 .

[0018] Figure 5 The flow of the observation unit in the first embodiment of the present invention is Figure 2 .

[0019] Figure 6 The flow of the observation unit in the first embodiment of the present invention is Figure 3 .

[0020] Figure 7 The flow of the observation unit in the first embodiment of the present invention is Figure 4 .

[0021] Figure 8 The flow of the observation unit in the first embodiment of the present invention is Figure 5 .

[0022] Fig. 9 This is an explanation of the contamination matching determination process according to the first embodiment of the present invention. Figure 1 .

[0023] Fig.10 This is an explanation of the contamination matching determination process according to the first embodiment of the present invention. Figure 2 .

[0024] Fig.11 This is an explanation of the contamination matching determination process according to the first embodiment of the present invention. Figure 3 .

[0025] Fig.12 This is a diagram showing the structure of a contamination region information acquisition unit according to Embodiment 1 of the present invention.

[0026] Fig.13 It is an explanatory diagram of the area division of the image according to the first embodiment of the present invention.

[0027] Fig.14This is an explanatory diagram of the exclusion range of the observation unit according to the first embodiment of the present invention.

[0028] Fig.15 This is an explanatory diagram of the excluded colors of the observation unit according to the first embodiment of the present invention.

[0029] Fig.16 It is a configuration diagram of a fixed object detection unit according to Embodiment 2 of the present invention.

[0030] Fig.17 It is a configuration diagram of a fixed object detection unit according to Embodiment 3 of the present invention.

[0031] Fig.18 It is a configuration diagram of a fixed object detection unit according to a fourth embodiment of the present invention.

[0032] Fig.19 It is a configuration diagram of a fixed object region information acquisition unit according to a fifth embodiment of the present invention. DETAILED DESCRIPTION

[0033] (The process of obtaining the technical solution)

[0034] In the use of camera monitoring systems installed in vehicles, the importance of ensuring their field of view is increasing. For example, dirt reflected in the image by the camera will form a blind spot in the driver's field of view. In addition, in ADAS, dirt can be a major cause of damage to vehicle safety. Therefore, there is an increasing demand for detecting attached dirt and warning the operator or automatically cleaning the dirt.

[0035] Several mechanisms have been proposed to detect dirt in an image via a camera. For example, in Patent Document 1, a fixed object is determined based on the brightness information of each small area obtained by dividing a predetermined area in a captured image and the brightness threshold of adjacent small areas.

[0036] However, although this method can detect fixed objects even in an environment such as dusk, when an image with little brightness change such as the sky or road in an image captured by a camera is input, sometimes an erroneous detection is made even if there is no fixed object.

[0037] The object of the present invention is to reduce erroneous detection and improve the detection rate of attached dirt by means of a detection unit for detecting multiple dirts with unchanged brightness, a distance calculation unit for calculating the distance between the detected coordinates, and a combining unit for treating close dirts as one.

[0038] According to the present invention, the false detection rate of mistakenly determining that a portion of the image, such as the sky and the road, which have little change but are not dirt attached to the camera, is dirt is reduced.

[0039] Furthermore, by applying the dirt detection of the present invention, not only dirt but also attached raindrops can be detected, or cracks and scratches existing on the lens or window glass of a camera mounted on a vehicle can be detected.

[0040] (Summary of the Invention)

[0041] The present invention is summarized as follows. That is, the fixed object detection unit of the first invention comprises: a photographing device; a fixed object region information acquisition unit, which searches for a candidate of a fixed object reflected in an image photographed by the photographing device as a fixed object candidate, and acquires fixed object region information indicating the coordinates of the fixed object candidate in the image, wherein the fixed object is a fixed object whose relative position relative to the photographing device is fixed; a storage unit, which stores the acquired fixed object region information; and an observation unit, which determines the coordinates indicated by the fixed object region information that are consistent with the coordinates of the fixed object region information previously stored in the storage unit as the coordinates of the fixed object in the image, and outputs the determination result.

[0042] In such a fixed object detection unit, it is possible to determine whether the relative position between the fixed object candidate and the camera device has been continuously fixed since the past, based on whether the coordinates of the fixed object region information obtained are consistent with the coordinates of the fixed object region information obtained and stored in the past. Then, if the relative position between the fixed object candidate and the camera device has been continuously fixed since the past, for example, it can be determined that the fixed object candidate is a fixed object. Since the fixed object is likely to be dirt that should be removed or cracks and scratches that should be dealt with, the existence of the fixed object can be notified to the manager of the camera device by outputting the determination result. On the other hand, according to the above method, if the fixed object region information obtained is inconsistent with the fixed object region information obtained and stored in the past, it can be determined that the relative position between the fixed object candidate and the camera device is only temporarily fixed, so for example, it can be determined that the fixed object candidate is not a fixed object. In this case, the fixed object candidate may be misdetected. In this way, by determining whether it is the same fixed object candidate continuously in the time domain, the misdetected candidate fixed object is not determined as a fixed object, so that the occurrence of erroneous detection can be reduced, and the detection rate of attached fixed objects can be relatively improved.

[0043] In addition, for example, the fixed object detection unit of the second invention may be the fixed object detection unit described in the first invention, and further comprises: a distance calculation unit, which calculates the distance between the coordinates indicated by a plurality of fixed object area information, the plurality of fixed object area information respectively corresponding to each fixed object candidate among the fixed object candidates simultaneously projected in the image; and a combining unit, which updates the coordinates indicated by the fixed object area information by setting the fixed object area information whose coordinates calculated by the distance calculation unit have a distance less than a threshold as one fixed object area information, and the fixed object area information stored in the storage unit is the fixed object area information after the coordinates are updated.

[0044] Thus, a plurality of pieces of fixed object region information which may be derived from one fixed object candidate and whose distances from each other are smaller than the occupied area can be processed as one piece of fixed object region information.

[0045] Furthermore, for example, the fixed object detection unit of the third invention may be the fixed object detection unit of the first or second invention, and further include an attached object notification device for notifying a user of the determination result output by the observation unit.

[0046] Thus, the determination result can be notified to the user (for example, the imaging device or the manager of the mounted medium on which the imaging device is mounted. The mounted medium includes a moving object such as a vehicle) through the attached object notification device.

[0047] In addition, for example, it can also be that the fixed object detection unit of the fourth invention is the fixed object detection unit described in any one of the first to third inventions, and the observation unit performs: consistency judgment processing, judging whether the coordinates shown in the fixed object area information are consistent with the coordinates shown in the fixed object area information stored in the past; detection number recording processing, adding the number of consistent detections of the fixed object area information judged to be consistent in the consistency judgment processing, and subtracting the number of consistent detections of the fixed object area information judged to be inconsistent in the consistency judgment processing; fixed object area information determination processing, judging the coordinates shown in the fixed object area information whose number of consistent detections exceeds the first threshold number as the coordinates of the fixed object in the image; and fixed object area information deletion processing, deleting the fixed object area information whose number of consistent detections is less than the second threshold number.

[0048] Thus, it is possible to manage how long in the past the fixed object candidate has been in a fixed relative position to the camera device based on the number of consistent detections. Furthermore, if the number of consistent detections exceeds the first threshold number, it is possible to determine that the fixed object candidate is a fixed object and output a determination result. In addition, if the coordinates of the fixed object region information are inconsistent, the number of consistent detections is subtracted, and when the number of consistent detections is below the second threshold number, it can be considered as a fixed object candidate temporarily generated due to misdetection, etc., and the fixed object region information is deleted. The second threshold number can be set to a relatively small number of about 0 to 10 times. If so, there are the following advantages: it is less likely to cause an undesirable situation in which the fixed object region information actually corresponding to the fixed object is deleted only once due to a missed detection and is inconsistent with the past fixed object region information.

[0049] In addition, for example, the fixed object detection unit of the fifth invention may be the fixed object detection unit described in the fourth invention, and at least one of the first threshold number and the second threshold number may be different in each of the plurality of regions when dividing the image.

[0050] Thus, in each of the plurality of regions when dividing a video, a first threshold number of times according to the likelihood of erroneous detection and a second threshold number of times according to the likelihood of missed detection can be set.

[0051] In addition, for example, the fixed object detection unit of the sixth invention may be the fixed object detection unit described in the first invention, and the observation unit may estimate the coordinates of the fixed object in the image based on past fixed object area information stored in the storage unit, and output the estimated result as the judgment result.

[0052] Thereby, it is possible to output, as the determination result, an estimation result indicating the coordinates of the fixed object in the image estimated based on the past fixed object region information stored in the storage unit.

[0053] In addition, for example, the fixed object detection unit of the seventh invention may be the fixed object detection unit described in any one of the first to sixth inventions, and the observation unit performs coordinate information determination processing to determine whether the coordinates shown in the fixed object area information are included in a pre-set exclusion range in the image, and for the fixed object area information whose coordinates are determined to be not included in the exclusion range in the coordinate information determination processing, it is determined whether it is the coordinates of the fixed object in the image.

[0054] Thus, it is possible to individually set whether or not to determine whether the coordinates of the fixed object in the image are to be determined for the fixed object region information, depending on whether or not the fixed object region information is included in the exclusion range pre-set in the image. For example, for the fixed object region information included in the exclusion range pre-set in the image, the determination of whether or not the coordinates of the fixed object in the image are to be determined is not performed, thereby reducing the processing resources required for the determination.

[0055] In addition, for example, the fixed object detection unit of the eighth invention may be the fixed object detection unit described in any one of the first to seventh inventions, and the observation unit performs color judgment processing to judge whether the display color in the image of the coordinates shown by the fixed object area information matches the preset exclusion color; in the color judgment processing, for the fixed object area information whose display color in the image is judged to be coordinates that do not match the exclusion color, a judgment is made as to whether it is the coordinates of the fixed object in the image.

[0056] Thus, it is possible to individually set whether to determine whether the coordinates of the fixed object in the image are to be determined for the fixed object region information, depending on whether the color matches the preset exclusion color. For example, by not determining whether the coordinates of the fixed object in the image are to be determined for the fixed object region information whose color matches the preset exclusion color, it is possible to reduce the processing resources required for the determination.

[0057] In addition, for example, it may also be that the fixed object detection unit of the 9th invention is the fixed object detection unit described in any one of the 1st to 8th inventions, the coordinates shown in the fixed object area information include one end coordinates as the coordinates of one end side of the fixed object candidate in the prescribed direction and the other end coordinates as the coordinates of the other end side of the fixed object candidate in the prescribed direction, and when the distance between the one end coordinates shown in one fixed object area information and the one end coordinates shown in another fixed object area information is within the one end threshold, and when the distance between the other end coordinates shown in one fixed object area information and the other end coordinates shown in another fixed object area information is within the other end threshold, it is determined that the coordinates of one fixed object area information are consistent with the coordinates of another fixed object area information.

[0058] Thus, whether or not two pieces of fixed object region information are consistent with each other can be determined using each of the one end coordinates and the other end coordinates.

[0059] In addition, for example, it may also be that the fixed object detection unit of the tenth invention is the fixed object detection unit described in any one of the first to ninth inventions, and the coordinates shown in the fixed object area information include center coordinates, and the center coordinates are the coordinates of the center of a line segment that bisects a point on one end side and a point on the other end side in a specified direction of the fixed object candidate, and when the distance between the center coordinates shown in one fixed object area information and the center coordinates shown in another fixed object area information is within a center threshold, it is determined that the coordinates of one fixed object area information are consistent with the coordinates of the other fixed object area information.

[0060] Thus, whether or not two pieces of fixed object region information coincide with each other can be determined using the center coordinates.

[0061] In addition, for example, it may also be that the fixed object detection unit of the 11th invention is the fixed object detection unit described in any one of the first to eighth inventions, and the coordinates indicated by the fixed object area information include one end coordinates as coordinates of one end side of the fixed object candidate in the prescribed direction, the other end coordinates as coordinates of the other end of the fixed object candidate in the prescribed direction, and the center coordinates, wherein the center coordinates are the coordinates of the center of a line segment bisecting a point on the one end side and a point on the other end side in the prescribed direction of the fixed object candidate, and when the distance between the one end coordinates indicated by one fixed object area information and the one end coordinates indicated by another fixed object area information is within the one end threshold, and the distance between the other end coordinates indicated by one fixed object area information and the other end coordinates indicated by another fixed object area information is within the other end threshold, and when the distance between the center coordinates indicated by one fixed object area information and the center coordinates indicated by another fixed object area information is within the center threshold, it is determined that the coordinates of one fixed object area information are consistent with the coordinates of another fixed object area information.

[0062] Thus, whether or not two fixed object region information are consistent with each other can be determined using the one end coordinates, the other end coordinates, and the center coordinates.

[0063] In addition, for example, the fixed object detection unit of the twelfth invention may be the fixed object detection unit described in any one of the first to eleventh inventions, and further include an image quality adjustment device, which receives an input of an original image captured by a shooting device and outputs an image with adjusted image quality to increase the search probability of fixed object candidates.

[0064] This can improve the search probability of fixed object candidates.

[0065] Furthermore, for example, the fixed object detection unit according to the 13th invention may be the fixed object detection unit according to any one of the first to 12th inventions, and may further include a remover that removes the fixed object based on the output determination result.

[0066] Thereby, the fixing material which persists can be removed by the removal machine.

[0067] In addition, for example, the fixed object detection unit of the 14th invention may be the fixed object detection unit described in any one of the 1st to 13th inventions, comprising: a reduction unit that reduces an image; a feature quantity detection unit that detects a feature quantity based on the reduced image reduced by the reduction unit; a feature quantity storage unit that stores the feature quantity; a cumulative addition unit that adds the feature quantity detected by the feature quantity detection unit and the feature quantity already stored in the feature quantity storage unit, and stores the addition result in the feature quantity storage unit as the feature quantity; a binarization unit that binarizes the stored feature quantity; a noise removal unit that removes noise contained in the binarized feature quantity data by reducing and expanding the binarized feature quantity data; and a labeling unit that generates fixed object area information based on the binarized feature quantity data from which the noise has been removed.

[0068] Thus, fixed object region information can be generated from an image through a reduction unit, a feature quantity detection unit, a feature quantity storage unit, a cumulative addition unit, a binarization unit, a noise removal unit, and a labeling unit.

[0069] In addition, for example, the fixed object detection unit of the 15th invention may be the fixed object detection unit described in any one of the 1st to 14th inventions, the fixed object is dirt attached to the light-transmitting component within the imaging range of the imaging device, and the fixed object candidate is a candidate for dirt.

[0070] This can reduce the occurrence of erroneous detection and relatively improve the detection rate of attached dirt.

[0071] In addition, for example, the fixed object detection unit of the 16th invention may be the fixed object detection unit described in any one of the 1st to 14th inventions, the fixed object is a raindrop attached to a light-transmitting component within the shooting range of the shooting device, and the fixed object candidate is a raindrop candidate.

[0072] This can reduce the occurrence of erroneous detection and relatively improve the detection rate of attached raindrops.

[0073] In addition, for example, the fixed object detection unit of the seventeenth invention may be the fixed object detection unit according to the sixteenth invention, and further include a remover that removes the fixed object based on the output determination result, and the remover may be a wiper device that removes raindrops.

[0074] Thus, the lingering raindrops can be removed by the wiper device.

[0075] In addition, for example, the fixed object detection unit of the 18th invention may be the fixed object detection unit described in any one of the 1st to 14th inventions, the fixed object is a defect including at least one of a crack and a scratch generated in a light-transmitting component within the imaging range of the imaging device, and the fixed object candidate is a candidate for the defect.

[0076] This can reduce the occurrence of erroneous detection and relatively improve the detection rate of defects that have occurred.

[0077] In addition, for example, the fixed object detection unit of the 19th invention may be the fixed object detection unit described in any one of the 1st to 18th inventions, comprising: a mark attached to a position corresponding to the specified coordinates on the image; a position tilt detection unit, which uses the mark as a fixed object and measures the difference between the coordinates of the mark shown by the judgment result output by the observation unit and the specified coordinates; a position tilt correction device, which corrects at least one of the position and tilt of the image based on the difference measured by the position tilt detection unit; and a display device, which displays the image after at least one of the position and tilt is corrected by the position tilt correction device.

[0078] Thus, when a marker attached to a position corresponding to the designated coordinates is detected as a fixed object, the offset between the coordinates indicated by the fixed object region information of the marker generated and the designated coordinates can be calculated by taking the difference. Then, by correcting the offset by an amount corresponding to the difference in at least one of the position and tilt of the image, the image can be displayed on the display device in an appropriate state.

[0079] In addition, for example, the fixed object detection unit of the 20th invention may be the fixed object detection unit of any one of the first to nineteenth inventions, comprising: a prediction unit that detects the coordinates of a portion whose contour shape reflected in an image captured by the camera is different from a prescribed shape; and a determination unit that determines that the determination result is correct when the coordinates of the fixed object in the determination result determined by the observation unit are consistent with the coordinates detected by the prediction unit, and the observation unit outputs the determination result determined by the determination unit to be correct.

[0080] Thus, when the coordinates indicated by the fixed object region information corresponding to the fixed object in the determination result are consistent with the coordinates predicted to correspond to the fixed object based on the case where the contour shape of the fixed object projected in the image is different from the prescribed shape, the determination result is assumed to be correct and can be output. In other words, according to this method, the determination result is verified based on the coordinates predicted to correspond to the fixed object based on the case where the contour shape of the fixed object projected in the image is different from the prescribed shape, so that the accuracy of the determination result can be guaranteed.

[0081] In addition, for example, the fixed object detection unit of the 21st invention may be the fixed object detection unit described in any one of the 1st to 20th inventions, comprising: a reduction unit that reduces an image; a feature quantity detection unit that detects a feature quantity based on the reduced image reduced by the reduction unit; a feature quantity storage unit that stores the feature quantity; a cumulative addition unit that adds the feature quantity detected by the feature quantity detection unit and the feature quantity already stored in the feature quantity storage unit, and stores the addition result in the feature quantity storage unit as the feature quantity; a binarization unit that binarizes the stored feature quantity; and a straight line detection unit that detects the coordinates of a linear fixed object candidate in the image using the Hough transform as a transformation algorithm based on the data of the binarized feature quantity.

[0082] Thus, the coordinates in the image of the linear fixed object candidate can be generated from the image through the reduction unit, the feature amount detection unit, the feature amount storage unit, the cumulative addition unit, the binarization unit and the line detection unit.

[0083] In addition, in the fixed object detection method of the 22nd invention, it is a fixed object detection method executed by a computer, and the fixed object detection method searches for candidates of fixed objects reflected in an image captured by a shooting device as fixed object candidates, and obtains fixed object area information representing the coordinates of the fixed object candidates in the image, wherein the fixed object is a fixed object whose relative position with respect to the shooting device is fixed, stores the obtained fixed object area information, determines the coordinates indicated by the fixed object area information that are consistent with the coordinates of the fixed object area information stored in the past as the coordinates of the fixed object in the image, and outputs the determination result.

[0084] In such a fixed object detection method, the same effects as those of the above-mentioned fixed object detection unit can be achieved.

[0085] Furthermore, a program according to the twenty-third invention is a program for causing a computer to execute the above-mentioned fixed object detection method.

[0086] Such a program can achieve the same effect as the above-mentioned fixed object detection unit using a computer.

[0087] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

[0088] (Implementation Method 1)

[0089] use Figure 1 The structural diagram is explained in detail. Figure 1 The overall configuration of a camera monitoring system mounted on a vehicle including a fixed object detection unit according to Embodiment 1 of the present invention is shown.

[0090] As shown in the figure, the camera monitoring system is composed of the following structure: a shooting device 101, which is installed in a posture to shoot the rear or side of the vehicle; a fixed object detection unit 102, which detects dirt as an example of a fixed object from the image captured by the shooting device 101; and a fixed object notification device 103, which notifies the operator of the vehicle of the presence of the dirt detected by the fixed object detection unit 102.

[0091] Here, the imaging device 101 is an analog camera or a digital camera, includes an image sensor having a light receiving element, and transmits an image signal in the form of an analog signal or a digital signal.

[0092] The destination of the notification of the fixed object notification device 103 may not be the operator, but a cleaning machine for cleaning the lens part attached to the camera 101. Thus, when dirt or other attachments (in other words, fixed objects) are attached to the lens, the attachments can be automatically cleaned. Similarly, when the camera is installed across a window, the window cleaning machine can be notified to automatically clean the window to remove the dirt attached to the window.

[0093] The fixed object detection unit 102 is implemented by executing a predetermined program using a processor and a memory. The fixed object detection unit 102 is composed of the following structures: a dirt region information acquisition unit 201, which searches for dirt candidates from the image input by the imaging device 101 and obtains its coordinate information; a distance calculation unit 202, which calculates the distance between dirt and dirt based on the dirt region information (an example of fixed object region information) obtained by the dirt region information acquisition unit 201; a combination unit 203, which treats the close dirt as one dirt based on the distance obtained (calculated) by the distance calculation unit 202, and calculates new dirt region information; a storage unit 204, which stores the new dirt region information obtained by the combination unit 203 by temporarily storing it in a storage device; and an observation unit 205, which compares the past dirt region information stored by the storage unit 204 with the newly detected dirt region information, and determines the dirt region information that persists as the final existing dirt region information.

[0094] use Figure 2 The contamination area information obtained by the contamination area information acquisition unit 201 will be described.

[0095] The dirt region information is represented by the X and Y coordinates of points 301 and 302 on a virtual XY coordinate plane parallel to the image plane, and a rectangular range that can be drawn by these two points is represented as the dirt attachment range.

[0096] That is, when the coordinates of point 301 are set to (X 301 , Y 301 ), the coordinates of point 302 are set to (X 302 , Y 302 ), we can connect the four points with coordinates (X 301 , Y 301 )、(X 301 , Y 302 )、(X 302 , Y 301 )、(X 302 , Y 302 ) and the drawn rectangle is defined as the dirt area.

[0097] use Figure 3 The combining unit 203 will be described.

[0098] The distance calculation unit 202 is used to calculate the distance between the dirt and the dirt. Figure 3 (a)), dirt partially overlaps with another dirt ( Figure 3 (b)), or the distance D between dirt and dirt is less than a certain threshold, that is, there is an adjacent ( Figure 3 In the case of the (c) relationship, the regions are combined to become one dirt region information.

[0099] The distance between the dirt may be measured again for the new dirt region information obtained by the combination, and based on the result, the combination may be further performed by the combination unit 203. This series of processing may be repeated multiple times.

[0100] use Figure 4 The flowchart shown is used to explain the determination of the contamination that persists in the observation unit 205.

[0101] A contamination region information detected in the past is taken out, and a matching determination process is performed to determine whether there is contamination region information matching with the newly detected contamination region information (S501). Specifically, it is determined whether there is matching contamination (S511).

[0102] When it is determined that the number of contamination detections is consistent in this process (Yes in S511), a contamination detection number addition update process is performed to increase the number of contamination detections by 1 (S502). Then, the number of contamination detections is compared with the threshold value (S512). As a result, when the number of contamination detections is greater than the predetermined threshold value (Yes in S512), a contamination region information determination process is performed to determine that there is contamination region information (that is, it is not a candidate for contamination but is contamination) (S503). On the other hand, when the number of contamination detections is less than the predetermined threshold value (No in S512), step S503 is skipped.

[0103] On the other hand, when the result of the determination in the contamination consistency determination process is determined to be inconsistent (Yes in S511), a contamination detection number subtraction update process is performed to subtract 1 from the contamination detection number (S504). Then, the contamination detection number is compared with a threshold value (S514). The threshold value here is 0. As a result, when the contamination detection number is 0 (Yes in S514), it is regarded as contamination that does not exist in the contamination region information, and a contamination region information deletion process is performed to delete the contamination region information (S505). In addition, when the contamination detection number is greater than the prescribed 0 (No in S514), step S505 is skipped.

[0104] The above process is performed on all the contaminated area information. That is, it is determined whether all the contaminated area information has been confirmed (S510). If it is not determined to be confirmed (No in S510), the process returns to step S501 and the same process is repeated again for another contaminated area information. If it is determined to be confirmed (Yes in S510), the process ends.

[0105] In addition, the threshold for determining fouling does not need to be constant. Fig.13 As shown, the image can be segmented into a plurality of regions (eg, six regions from region 801 to region 806), and the threshold value can be changed in each region.

[0106] For example, by increasing the threshold value of the area 802 including more sky, it is possible to reduce the time spent on detecting dirt in this area and reduce false detections. On the other hand, dirt is easily detected in the areas 801 and 803 with more buildings, so by reducing the threshold value, it is possible to shorten the time required to detect dirt in these areas.

[0107] use Figure 5 The flowchart shown is used to explain the determination of another contamination in the determination observation unit 205.

[0108] Based on the past contamination region information accumulated in the storage unit 204, the coordinate information of the contamination is estimated, and contamination prediction processing is performed on the contamination determined to be present (S506).

[0109] use Figure 6 The flowchart shown and Fig.14 The determination of another type of dirt that persists in the observation unit 205 is described below. Figure 6 In the flowchart shown, Figure 4 The same processing is given the same figure mark and repeated description is omitted.

[0110] Implement coordinate information determination processing (S507), take out a dirt area information detected in the past, and determine whether the Y coordinate (coordinate in the up-down direction of the paper) of the lower right point of the dirt area information exists outside the range of the upper exclusion range 901, and the Y coordinate of the upper left point exists outside the range of the lower exclusion range 902. That is, determine whether both coordinates are outside the range of the exclusion range (S517). When it is determined in the determination of the coordinate information determination processing that the dirt area information is outside the exclusion range (yes in S517), implement the dirt consistency determination processing (S501) to determine whether there is dirt area information that is consistent with the newly detected dirt area information. The subsequent processing is as follows Figure 4 As shown in the description of . If it is determined that it is consistent in this process, the dirt detection number is added and updated by 1 (S502). As a result, if the dirt detection number is greater than the specified threshold, the dirt area information is determined as the dirt area information having dirt (S503).

[0111] On the other hand, if the result of the contamination consistency determination process (S501) is inconsistent, a contamination detection number subtraction update process (S504) is performed to subtract 1 from the contamination detection number. As a result, if the contamination detection number is 0, it is considered that there is no contamination in the contamination region information, and a contamination region information deletion process (S505) is performed to delete the contamination region information.

[0112] Repeat the above steps for all dirt area information.

[0113] use Figure 7 The flowchart shown and Fig.15 The determination of another type of dirt that persists in the observation unit 205 is described below. Figure 7 In the flowchart shown, Figure 4 The same processing is given the same figure mark and repeated description is omitted.

[0114] Here, the color information of the dirt is added to the dirt region information.

[0115] Take out the dirt area information detected in the past and obtain the color information of the dirt. Implement the dirt color determination process (S508) to determine whether the color information of the dirt matches the exclusion color 1001 and the exclusion color 1002. In other words, determine whether the display color in the image of the coordinates does not match the pre-set exclusion color (S518). In the case where it is determined in the dirt color determination process that the color information of the dirt is not the exclusion color (yes in S518), implement the dirt consistency determination process (S501) to determine whether there is dirt area information that is consistent with the newly detected dirt area information. The subsequent processing is as follows Figure 4 As shown in the description of . If it is determined that it is consistent in this process, a dirt detection number addition update process (S502) is performed to add 1 to the dirt detection number. As a result, if the dirt detection number is greater than a predetermined threshold, a dirt area information determination process (S503) is performed to determine that the dirt area information indicates that there is dirt.

[0116] On the other hand, if the result of the contamination consistency determination process (S501) is inconsistent, a contamination detection number subtraction update process (S504) is performed to subtract 1 from the contamination detection number. As a result, if the contamination detection number is 0, it is considered that there is no contamination in the contamination region information, and a contamination region information deletion process (S505) is performed to delete the contamination region information.

[0117] Repeat the above steps for all dirt area information.

[0118] Here, the excluded colors 1001 and 1002 are assumed to be two colors, namely, the average sky color and the average asphalt color on a clear day. However, various colors may be assumed according to the weather and time, and the excluded colors may be set to three or more colors.

[0119] use Figure 8 The flowchart shown and Fig.14 , Fig.15 The determination of another type of dirt that persists in the observation unit 205 is described below. Figure 8In the flowchart shown, Figure 4 The same processing is given the same figure mark and repeated description is omitted.

[0120] Here, the color information of the dirt is also added to the dirt region information.

[0121] Implement coordinate information determination processing (S507), take out a dirt area information detected in the past, and determine whether the Y coordinate of the lower right point of the dirt area information exists outside the range of the upper exclusion range 901, and the Y coordinate of the upper left point exists outside the range of the lower exclusion range 902. In the case where the coordinate information determination processing determines that the dirt area information is outside the exclusion range, further obtain the color information of the dirt. Implement a dirt color determination processing (S508) to determine whether the color information of the dirt is consistent with the exclusion color 1001 and the exclusion color 1002. In the case where the dirt color determination processing determines that the color information of the dirt is not an exclusion color, implement a dirt consistency determination processing (S501) to determine whether there is dirt area information that is consistent with the newly detected dirt area information. The subsequent processing is as follows Figure 4 As shown in the description of . If it is determined that it is consistent in this process, a dirt detection number addition update process (S502) is performed to add 1 to the dirt detection number. As a result, if the dirt detection number is greater than a predetermined threshold, a dirt area information determination process (S503) is performed to determine that the dirt area information indicates that there is dirt.

[0122] On the other hand, if the result of the contamination consistency determination process (S501) is inconsistent, a contamination detection number subtraction update process (S504) is performed to subtract 1 from the contamination detection number. As a result, if the contamination detection number is 0, it is considered that there is no contamination in the contamination region information, and a contamination region information deletion process (S505) is performed to delete the contamination region information.

[0123] Repeat the above steps for all dirt area information.

[0124] use Fig. 9 The contamination matching determination process will be described.

[0125] As described above, the dirt region information is composed of coordinate information of two points. The point 601 indicated by the coordinates (X1, Y1) as the dirt region information detected in the past has a consistent determination area 603, which ranges from (X1-X1 threshold) to (X1+X1 threshold) and from (Y1-Y1 threshold) to (Y1+Y1 threshold). In addition, the point 602 indicated by the coordinates (X2, Y2) has a consistent determination area 604, which ranges from (X2-X2 threshold) to (X2+X2 threshold) and from (Y2-Y2 threshold) to (Y2+Y2 threshold). In the case where the coordinate information of the newly detected dirt region information is within the range of the consistent determination areas 603 and 604 (that is, the difference is within the threshold), it is determined to be the same dirt.

[0126] In addition, use Fig.10 To illustrate another dirt consistent determination.

[0127] As described above, the contamination region information is composed of coordinate information of one point. The center coordinates (X1, Y1) between the point 611 indicated by the coordinates (X1, Y1) as the contamination region information detected in the past and the point 612 indicated by the coordinates (X2, Y2) are c , Y c ) has a consistent determination area 614, and the consistent determination area 614 is taken from (X c -X c Threshold) to (X c +X c Threshold) and from (Y c -Y c Threshold) to (Y c +Y c When the center points of the two points of the newly detected dirt region information are within the range of the coincidence determination area 614 (that is, the difference is within the threshold), it is determined that the dirt is the same.

[0128] In addition, use Fig.11 To illustrate another dirt consistent determination.

[0129] As described above, the dirt region information is composed of the coordinate information of three points. The point 621 indicated by the coordinates (X1, Y1) as the dirt region information detected in the past has a consistent determination area 623, and the consistent determination area 623 ranges from (X1-X1 threshold) to (X1+X1 threshold) and from (Y1-Y1 threshold) to (Y1+Y1 threshold). In addition, the point 622 indicated by the coordinates (X2, Y2) has a consistent determination area 624, and the consistent determination area 624 ranges from (X2-X2 threshold) to (X2+X2 threshold) and from (Y2-Y2 threshold) to (Y2+Y2 threshold). In addition, the center coordinates (X1, Y1) between the two points 621 and 622 are c , Y c ) has a consistent determination area 626, and the consistent determination area 626 is taken from (X c -X c Threshold) to (X c +X c Threshold) and from (Y c -Y c Threshold) to (Y c +Y c When the coordinate information of the newly detected dirt region information is within the range of the coincidence determination regions 623 and 624 or within the range of the coincidence determination region 626, it is determined to be the same dirt.

[0130] use Fig.12 The dirt region information acquisition unit 201 will be described.

[0131] The dirt area information acquisition unit 201 is composed of the following structure: an image reduction unit 701, which reduces the image captured by the shooting device 101; a feature detection unit 702, which detects the feature based on the image data reduced by the reduction unit; a cumulative addition unit 703, which adds the result of the feature detection unit 702 and the feature stored by the feature storage unit 704; the feature storage unit 704, which stores the cumulative feature calculated by the cumulative addition unit 703 in the storage device; a binarization unit 705, which binarizes the cumulative feature stored by the feature storage unit 704 in the storage device; a noise removal unit 706, which removes noise by reducing / expanding the data binarized by the binarization unit 705; and a labeling unit 707, which obtains the dirt area information from the data obtained by the noise removal unit 706.

[0132] As described above, according to the first embodiment, when detecting dirt, it is possible to obtain an effect of reducing the false detection rate of erroneously determining that parts other than dirt such as the sky and the road are dirt.

[0133] Furthermore, by utilizing these mechanisms, not only the dirt exemplified above but also fixed objects such as raindrops and even cracks and scratches on lenses and window glasses can be detected as fixed objects.

[0134] (Implementation Method 2)

[0135] The overall structure of the camera monitoring system mounted on a vehicle according to the second embodiment of the present invention is shown in FIG. Fig.16 In addition, Fig.16 The dirt detection unit shown in the figure has the same structure as that used in the description of the first embodiment. Figure 1 The same reference numerals are given to components having the same functions, and duplicate descriptions are omitted.

[0136] As shown in the figure, the camera monitoring system is composed of the following structure: a shooting device 101, which is installed in a posture to shoot the rear or side of the vehicle; an image quality adjustment device 104, which changes the image quality of the image shot by the shooting device 101; a fixed object detection unit 102, which detects dirt from the image output by the image quality adjustment device 104; and a fixed object notification device 103, which notifies the operator of the existence of attached dirt detected by the fixed object detection unit 102.

[0137] The image quality adjustment device 104 has functions such as tone correction, color balance adjustment, contrast correction, tone curve adjustment, gamma correction, and contour enhancement for adjusting the quality of the image input from the imaging device 101. Using these functions, the image is posterized, the contour is enhanced, and the image is corrected to make it easier to detect dirt.

[0138] As described above, according to Embodiment 2, the accuracy of dirt detection performed by the dirt region information acquisition unit 201 is improved, and the effect of reducing the false detection rate of mistakenly determining that dirt attached to the camera, such as the sky and the road, is dirt can be further obtained compared to Embodiment 1. In addition, conversely, it is also possible to detect dirt that cannot be detected.

[0139] (Implementation method 3)

[0140] The overall structure of the camera monitoring system mounted on a vehicle according to the third embodiment of the present invention is shown in FIG. Fig.17 In addition, Fig.17 The dirt detection unit shown in the figure has the same structure as that used in the description of the first embodiment. Figure 1 The same reference numerals are given to components having the same functions, and duplicate descriptions are omitted.

[0141] In the figure, the structure is composed of: a mark 105, including coordinates of two or more points intentionally attached in a manner fixed to specific coordinates; a shooting device 101, which is mounted in a posture of shooting the rear or side of the vehicle and shooting an image including the mark 105; a fixed object detection unit 102, which detects the mark as a fixed object based on the image captured by the shooting device 101; a position tilt correction device 106, which corrects the position and angle of the input image based on the position and tilt information notified from the fixed object detection unit 102; and a display device 107, which displays the image obtained by correcting the image from the shooting device 101 by the position tilt correction device 106.

[0142] The fixed object detection unit 102 is composed of the following structure: a dirt area information acquisition unit 201, which searches for dirt candidates from the image input by the shooting device 101 and obtains its coordinate information; a distance calculation unit 202, which calculates the distance between dirt and dirt based on the dirt area information obtained by the dirt area information acquisition unit 201; a combination unit 203, which treats close dirt as one dirt based on the distance obtained by the distance calculation unit 202, and calculates new dirt area information; a storage unit 204, which temporarily stores the new dirt area information obtained by the combination unit 203 in the storage device; an observation unit 205, which compares the past dirt area information stored by the storage unit 204 with the newly detected dirt area information, and determines the dirt area information that continues to exist as the final dirt area information; and a position tilt detection unit 206, which compares the dirt area information of the marker 105 as a fixed object detected by the observation unit 205 with the assumed dirt area information of the marker 105, and calculates the difference in their position and angle.

[0143] As described above, according to Embodiment 3, the tilt and position of the camera generated when the camera is installed in a factory, etc. can be automatically corrected. Alternatively, the tilt and position deviation of the camera generated when the operator uses the camera can be corrected by attaching a fixture to a specified position by the operator.

[0144] (Implementation 4)

[0145] The overall structure of the camera monitoring system mounted on a vehicle according to the fourth embodiment of the present invention is shown in FIG. Fig.18 The structure diagram is shown in Figure 1. Fig.18 The dirt detection unit shown in the figure has the same structure as that used in the description of the first embodiment. Figure 1 The same reference numerals are given to components having the same functions, and duplicate descriptions are omitted.

[0146] As shown in the figure, the camera monitoring system is composed of the following structure: a shooting device 101, which is mounted in a posture to shoot the rear or side of the vehicle; a fixed object detection unit 102, which detects dirt from the image captured by the shooting device 101; and a fixed object notification device 103, which notifies the operator of the presence of dirt detected by the fixed object detection unit 102.

[0147] The fixed object detection unit 102 is composed of the following structures: a dirt region information acquisition unit 201, which searches for dirt candidates from the image input by the shooting device 101 and obtains its coordinate information; a distance calculation unit 202, which calculates the distance between dirt and dirt based on the dirt region information obtained by the dirt region information acquisition unit 201; a combination unit 203, which treats the adjacent dirt as one dirt based on the distance obtained by the distance calculation unit 202, and calculates new dirt region information; a storage unit 204, which temporarily stores the new dirt region information obtained by the combination unit 203 in a storage device; an observation unit 205, The past dirt region information stored in the storage unit 204 is compared with the newly detected dirt region information, and the dirt region information that persists is determined to be the final dirt region information. The prediction unit 207 detects the coordinate information of the portion of the image captured by the shooting device 101 where the contour of the image is not the assumed shape (for example, a portion where a part of the contour line of the vehicle is missing); and the determination unit 208 compares the dirt region information obtained by the observation unit 205 with the coordinate information obtained by the prediction unit 207, and when the same coordinate information is detected in the two units, the coordinate is determined to be dirt.

[0148] As described above, according to Embodiment 4, the accuracy of dirt detection by the dirt region information acquisition unit 201 is improved, and compared with Embodiment 1, the false detection rate of mistakenly judging parts other than dirt attached to the camera such as the sky and the road as dirt can be further reduced.

[0149] (Implementation method 5)

[0150] The overall structure of the camera monitoring system mounted on a vehicle according to the fifth embodiment of the present invention is shown in FIG. Fig.19 The structure diagram is shown in Figure 1. Fig.19 The fixed object region information acquisition unit 201a shown in the figure has the same Figure 1 The same reference numerals are given to components having the same functions, and duplicate descriptions are omitted.

[0151] The fixed object area information acquisition unit 201a is composed of the following structure: an image reduction unit 701, which reduces the image captured by the imaging device 101; a feature detection unit 702, which detects feature values ​​based on the image data reduced by the reduction unit; a feature storage unit 704, which stores feature values; an accumulation addition unit 703, which adds the feature values ​​stored in the feature storage unit 704 and the feature values ​​detected by the feature detection unit 702; a binarization unit 705, which binarizes the feature values ​​stored in the feature storage unit 704; and a straight line detection unit 708, which detects straight lines using the Hough transform as a transformation algorithm based on the data binarized by the binarization unit 705.

[0152] As described above, according to the fifth embodiment, a part of the function of the dirt detection unit can be used to detect white lines dividing lanes on the road, as well as fixed objects such as dirt.

[0153] (Other embodiments)

[0154] The above describes the fixed object detection unit of the present disclosure based on various embodiments and variations, but the present disclosure is not limited to the above embodiments. For example, various variations that can be thought of by a person skilled in the art on each embodiment, and any combination of structural elements and functions of each embodiment within the scope of the present disclosure are also included in the present disclosure.

[0155] For example, the structural elements of the processing unit such as the fixed object detection unit or the dirt area information acquisition unit may also be composed of one or more electronic circuits. One or more circuits may be general-purpose circuits or special-purpose circuits. For example, one or more electronic circuits may include semiconductor devices, ICs (Integrated Circuits) or LSIs (Large Scale Integrations), etc. ICs or LSIs may be integrated on one chip or on multiple chips. Here, they are called ICs or LSIs, but the names are changed according to the degree of integration, and may be called system LSIs, VLSIs (Very Large Scale Integrations) or ULSIs (Ultra Large Scale Integrations). In addition, FPGAs (Field Programmable Gate Arrays) programmed after manufacturing the LSIs may also be used for the same purpose.

[0156] In addition, the general or specific aspects of the present disclosure may be implemented by a system, device, method, integrated circuit, or computer program. Alternatively, it may be implemented by a computer-readable non-temporary recording medium such as an optical disk, HDD, or semiconductor memory storing the computer program. In addition, the present invention may be implemented by any combination of a system, device, method, integrated circuit, computer program, and recording medium.

[0157] Industrial Applicability

[0158] The present invention can improve the reliability of the device by reducing the false detection rate of the fixed object that can be detected in the image projected by the imaging device, and enables the imaging device to be cleaned accurately.

[0159] In addition, the fixture is independent of dirt and can also detect raindrops, etc.

[0160] Furthermore, by intentionally attaching a fixture having fixed coordinates, the mounting position and angle of the imaging device can be detected and corrected.

[0161] Description of Reference Numerals

[0162] 101 Filming Devices

[0163] 102 Fixed object detection unit

[0164] 103 Fixed object notification device

[0165] 104 Image quality adjustment device

[0166] 105 Mark

[0167] 106 position tilt correction device

[0168] 107 Display device

[0169] 201 Dirt area information acquisition unit

[0170] 201a Fixed object area information acquisition unit

[0171] 202 distance calculation unit

[0172] 203 Combination Unit

[0173] 204 storage units

[0174] 205 Observation Unit

[0175] 206 position tilt detection unit

[0176] 207 prediction unit

[0177] 208 judgment unit

[0178] 301, 302, 601, 602, 611, 612, 613, 621, 622, 625 points

[0179] 603, 604, 614, 623, 624, 626 consistent judgment area

[0180] 701 Image Reduction Unit

[0181] 702 Feature detection unit

[0182] 703 Accumulation and addition unit

[0183] 704 Feature storage unit

[0184] 705 Binarization Unit

[0185] 706 Noise Removal Unit

[0186] 707 Annotation Unit

[0187] 708 Linear Detection Unit

[0188] Areas 801, 802, 803, 804, 805, 806

[0189] 901 Upper Exclusion Range

[0190] 902 Lower Exclusion Range

[0191] 1001, 1002 exclusion color

Claims

1. A fixed object detection unit, wherein: have: Filming device; a fixed object region information acquisition unit that searches for a candidate of a fixed object that appears in the image captured by the camera as a fixed object candidate and acquires fixed object region information indicating the coordinates of the fixed object candidate in the image, the fixed object being a fixed object whose relative position to the camera is fixed; A storage unit for storing the obtained fixed object area information; as well as The observation unit determines that the coordinates indicated by the fixed object region information, which coordinates match the coordinates of the fixed object region information previously stored in the storage unit, are the coordinates of the fixed object in the image, and outputs the determination result.

2. The fixed object detection unit according to claim 1, wherein: Also available: a distance calculation unit for calculating distances between coordinates indicated by a plurality of the fixed object region information, the plurality of the fixed object region information respectively corresponding to each of the fixed object candidates simultaneously projected in the image; as well as The combining unit updates the coordinates indicated by the fixed object region information by setting the fixed object region information whose distances between the coordinates calculated by the distance calculating unit are smaller than a threshold value as one fixed object region information, The fixed object region information stored in the storage unit is the fixed object region information after coordinates are updated.

3. The fixed object detection unit according to claim 1, wherein: The device further includes an attached matter notification device for notifying a user of the determination result output by the observation unit.

4. The fixed object detection unit according to claim 1, wherein: The observation unit performs: a consistency determination process of determining whether the coordinates indicated by the fixed object region information are consistent with the coordinates indicated by the fixed object region information stored in the past; a detection number recording process of adding the number of detections of the fixed object region information determined to be consistent in the consistency determination process and subtracting the number of detections of the fixed object region information determined to be inconsistent in the consistency determination process; a fixed object region information determination process of determining the coordinates indicated by the fixed object region information for which the number of coincidence detections exceeds a first threshold number of times as the coordinates of the fixed object in the image; and The fixed object region information deletion process deletes the fixed object region information for which the number of coincidence detections is less than or equal to a second threshold number of times.

5. The fixed object detection unit according to claim 4, wherein: At least one of the first threshold number and the second threshold number is different in each of a plurality of regions when the image is divided.

6. The fixed object detection unit according to claim 1, wherein: The observation unit estimates the coordinates of the fixed object in the image based on past fixed object region information stored in the storage unit, and outputs an estimation result as the determination result.

7. The fixed object detection unit according to claim 1, wherein: The observation unit, executing coordinate information determination processing to determine whether the coordinates indicated by the fixed object region information are included in a pre-set exclusion range in the image, For the fixed object region information whose coordinates are determined not to be included in the exclusion range in the coordinate information determination process, it is determined whether or not the fixed object region information is the coordinates of the fixed object in the image.

8. The fixed object detection unit according to claim 1, wherein: The observation unit, executing a color determination process to determine whether a display color in the image at the coordinates indicated by the fixed object region information matches a preset exclusion color; In the color determination process, for the fixed object region information whose coordinates are determined to have a display color in the image that does not match the exclusion color, it is determined whether or not they are the coordinates of the fixed object in the image.

9. The fixed object detection unit according to claim 1, wherein: The coordinates indicated by the fixed object region information include one end coordinates which are coordinates of one end side of the fixed object candidate in a predetermined direction and another end coordinates which are coordinates of the other end side of the fixed object candidate in the predetermined direction. When the distance between the one end coordinate indicated by one of the fixed object area information and the one end coordinate indicated by the other of the fixed object area information is within the one end threshold, and the distance between the other end coordinate indicated by one of the fixed object area information and the other end coordinate indicated by the other of the fixed object area information is within the other end threshold, it is determined that the coordinates of one of the fixed object area information are consistent with the coordinates of the other of the fixed object area information.

10. The fixed object detection unit according to claim 1, wherein: The coordinates indicated by the fixed object region information include center coordinates, which are coordinates of the center of a line segment bisecting a point on one end side and a point on the other end side in a predetermined direction of the fixed object candidate. When the distance between the center coordinates indicated by one of the fixed object region information and the center coordinates indicated by the other of the fixed object region information is within a center threshold, it is determined that the coordinates of the one of the fixed object region information coincide with the coordinates of the other of the fixed object region information.

11. The fixed object detection unit according to claim 1, wherein: The coordinates shown in the fixed object region information include one end coordinates which are coordinates of one end side of the fixed object candidate in a specified direction, the other end coordinates which are coordinates of the other end side of the fixed object candidate in the specified direction, and the center coordinates, wherein the center coordinates are the coordinates of the center of a line segment bisecting a point on one end side and a point on the other end side in the specified direction of the fixed object candidate, When the distance between the one end coordinate indicated by one of the fixed object area information and the one end coordinate indicated by another of the fixed object area information is within the one end threshold, and the distance between the other end coordinate indicated by one of the fixed object area information and the other end coordinate indicated by another of the fixed object area information is within the other end threshold, and the distance between the center coordinate indicated by one of the fixed object area information and the center coordinate indicated by another of the fixed object area information is within the center threshold, it is determined that the coordinates of one of the fixed object area information are consistent with the coordinates of another of the fixed object area information.

12. The fixed object detection unit according to claim 1, wherein: The image quality adjustment device is further provided, the image quality adjustment device receiving an input of an original image captured by the imaging device and outputting the image with the image quality adjusted so as to increase the search probability of the fixed object candidate.

13. The fixed object detection unit according to claim 1, wherein: The device further includes a removing device for removing the fixed object based on the outputted determination result.

14. The fixed object detection unit according to claim 1, wherein: have: A reduction unit, which reduces the image; a feature quantity detecting unit, which detects a feature quantity based on the reduced image reduced by the reducing unit; A feature quantity storage unit for storing feature quantities; a cumulative adding unit that adds the feature amount detected by the feature amount detecting unit and the feature amount already stored in the feature amount storing unit, and stores the addition result in the feature amount storing unit as the feature amount; A binarization unit, binarizing the stored feature quantity; a noise removal unit that removes noise included in the data of the binarized feature quantity by reducing and expanding the data of the binarized feature quantity; as well as The labeling unit generates the fixed object region information based on the binary feature quantity data from which noise has been removed.

15. The fixed object detection unit according to claim 1, wherein: The fixed object is dirt attached to a light-transmitting member within the imaging range of the imaging device. The fixed object candidate is a candidate for the dirt.

16. The fixed object detection unit according to claim 1, wherein: The fixed object is a raindrop attached to a light-transmitting member within the shooting range of the shooting device, The fixed object candidate is a candidate for the raindrop.

17. The fixed object detection unit according to claim 16, wherein: further comprising a removing machine for removing the fixed object based on the output determination result, The remover is a wiper device that removes the raindrops.

18. The fixed object detection unit according to claim 1, wherein: The fixed object is a defect including at least one of a crack and a scratch generated in the light-transmitting member within the imaging range of the imaging device, The fixture candidate is a candidate for the defect.

19. The fixed object detection unit according to claim 1, wherein: have: Markers, attached at locations corresponding to specified coordinates on the image; a position tilt detection unit that uses the marker as the fixed object and measures the difference between the coordinates of the marker indicated by the determination result output by the observation unit and the designated coordinates; a position tilt correction device for correcting at least one of the position and tilt of the image according to the difference measured by the position tilt detection unit; as well as The display device displays an image after at least one of the position and tilt is corrected by the position tilt correction device.

20. The fixed object detection unit according to claim 1, wherein: have: a prediction unit that detects coordinates of a portion whose contour shape is different from a predetermined shape and is reflected in the image captured by the imaging device; and a determination unit that determines that the determination result is correct when the coordinates of the fixed object in the determination result determined by the observation unit are consistent with the coordinates detected by the prediction unit, The observation unit outputs the determination result determined to be correct by the determination unit.

21. The fixed object detection unit according to any one of claims 1 to 20, wherein: have: A reduction unit, which reduces the image; a feature quantity detecting unit, which detects a feature quantity based on the reduced image reduced by the reducing unit; A feature quantity storage unit for storing feature quantities; a cumulative adding unit that adds the feature amount detected by the feature amount detecting unit and the feature amount already stored in the feature amount storing unit, and stores the addition result in the feature amount storing unit as the feature amount; A binarization unit, binarizing the stored feature quantity; as well as The straight line detection unit detects the coordinates of the straight line fixed object candidate in the image using Hough transform as a transformation algorithm based on the binary feature quantity data.

22. A method for detecting a fixed object, executed using a computer, wherein: Searching for a candidate of a fixed object reflected in an image captured by a camera as a fixed object candidate, and obtaining fixed object region information indicating coordinates of the fixed object candidate in the image, wherein the fixed object is a fixed object whose relative position to the camera is fixed. storing the obtained fixed object area information, The coordinates indicated by the fixed object region information, whose coordinates match the coordinates of the fixed object region information stored in the past, are determined to be the coordinates of the fixed object in the image, and the determination result is output.

23. A program, wherein: The program is for causing a computer to execute the fixed object detection method according to claim 22.