Fixed object detection means, fixed object detection method, and program

Abstract

A fixed object detection means (102) comprises: an imaging device (101); a fixed object area information acquisition means (dirty area information acquisition means (201)) for searching for a fixed object candidate which is a candidate of a fixed object that is included in an image captured by the imaging device (101) and that has a position relative to the imaging device (101) fixed, and for acquiring fixed object area information indicating coordinates of the fixed object candidate in the image; a storage means (204) for storing therein the acquired fixed object area information; and an observation means (205) for determining that coordinates, which are indicated by the fixed object area information and which match the coordinates of fixed object area information stored in the past in the storage means (204), are coordinates of the fixed object in the image and for outputting the determination result.

Description

Fixed object detection means, fixed object detection method, and program

[0001] The present invention aims to improve the detection rate in a system that detects dirt adhering to a vehicle and that is captured on an image via a camera mounted on the vehicle.

[0002] Ensuring visibility is becoming increasingly important for camera monitor systems installed in vehicles. For example, dirt visible on the camera image can create blind spots in the driver's field of view. Furthermore, dirt can compromise vehicle safety in ADAS. Therefore, there is a growing demand for systems that can detect dirt and warn the driver or automatically clean it.

[0003] Several mechanisms for detecting dirt captured on video via a camera have been proposed. For example, Patent Document 1 (JP-A-2005-102666) divides a predetermined area in a captured video into small areas, and determines fixed objects based on the brightness information of each small area and the brightness threshold and determination conditions for adjacent small areas.

[0004] Japanese Patent Application Laid-Open No. 2021-51381

[0005] However, the conventional methods described above may be prone to false detection.

[0006] An object of the present invention is to reduce the occurrence of false detections and improve the detection rate of attached fixed objects.

[0007] In order to solve the above problems and achieve the object, the fixed object detection means of the present invention comprises a photographing device, a fixed object area information acquisition means for searching for fixed object candidates that appear in an image photographed by the photographing device and are candidates for fixed objects whose relative position relative to the photographing device is fixed, and acquiring fixed object area information indicating the coordinates of the fixed object candidates in the image, a storage means for storing the acquired fixed object area information, and an observation means for determining that the coordinates indicated by the fixed object area information that match the coordinates of the fixed object area information previously stored in the storage means are the coordinates of the fixed object in the image and outputting the determination result.

[0008] According to the present invention, the occurrence of erroneous detection is reduced, and the detection rate of attached fixed objects can be improved.

[0009] FIG. 1 is a configuration diagram of a fixed object detection means according to embodiment 1 of the present invention. FIG. 2 is an explanatory diagram of dirt area information according to embodiment 1 of the present invention. FIG. 3 is an explanatory diagram of the observation means according to embodiment 1 of the present invention. FIG. 4 is an explanatory diagram of the observation means according to embodiment 1 of the present invention. FIG. 5 is an explanatory diagram of the dirt match determination process according to embodiment 1 of the present invention. FIG. 1 is an explanatory diagram of the dirt match determination process according to embodiment 1 of the present invention. FIG. 2 is an explanatory diagram of the dirt match determination process according to embodiment 1 of the present invention. FIG. 3 is an explanatory diagram of the dirt area information acquisition means according to embodiment 1 of the present invention. FIG. 4 is an explanatory diagram of the dirt area information acquisition means according to embodiment 1 of the present invention. FIG. 5 is an explanatory diagram of the dirt area division process according to embodiment 1 of the present invention. FIG. 6 is an explanatory diagram of the exclusion range of the observation means according to embodiment 1 of the present invention. FIG. 7 is an explanatory diagram of the color exclusion of the observation means according to embodiment 1 of the present invention. FIG. 7 is a configuration diagram of a fixed object detection means according to embodiment 2 of the present invention. FIG. 8 is a configuration diagram of a fixed object detection means according to embodiment 3 of the present invention. FIG. 9 is a configuration diagram of a fixed object area information acquisition means according to embodiment 5 of the present invention.

[0010] (Background to the Invention) Ensuring visibility is becoming increasingly important in the use of camera monitor systems mounted on vehicles. For example, dirt visible in the camera image can create blind spots in the driver's field of view. Furthermore, dirt can be a factor in compromising vehicle safety in ADAS. Therefore, there is a growing demand for systems that can detect attached dirt and warn the driver or automatically clean it.

[0011] Several mechanisms for detecting dirt captured on video via a camera have been proposed. For example, Patent Document 1 (JP-A-2005-102666) divides a predetermined area in a captured video into small areas, and determines fixed objects based on the brightness information of each small area and the brightness threshold and determination conditions for adjacent small areas.

[0012] However, while this method can detect fixed objects even in environments such as twilight, when images captured by a camera with little change in brightness, such as of the sky or road, are input, it may erroneously detect fixed objects even though they do not exist.

[0013] The present invention aims to reduce erroneous detections and improve the detection rate of attached stains by using a detection means that detects multiple stains whose brightness does not change, a distance calculation means that calculates the distance between detected coordinates, and a combination means that groups nearby stains together.

[0014] According to the present invention, the rate of false detection, in which areas other than dirt adhering to the camera, such as the sky or road, which have little image variation, are mistakenly determined to be dirt, is reduced.

[0015] Furthermore, by applying the dirt detection method of the present invention, it is possible to detect not only dirt but also raindrops adhering to the surface, and to detect cracks and scratches on the lens or window glass of a camera mounted on a vehicle.

[0016] (Summary of the Invention) The summary of the present invention is as follows: That is, a fixed object detection means according to a first invention includes a photographing device, fixed object area information acquisition means for searching for fixed object candidates that appear in an image photographed by the photographing device and are candidates for fixed objects whose relative position with respect to the photographing device is fixed, and acquiring fixed object area information indicating the coordinates of the fixed object candidates in the image, storage means for storing the acquired fixed object area information, and observation means for determining that coordinates indicated by the fixed object area information that match coordinates of fixed object area information previously stored in the storage means are coordinates of a fixed object in the image, and outputting the determination result.

[0017] Such fixed object detection means can determine whether the relative position of a fixed object candidate to the image capture device has been continuously fixed since the past, based on whether the coordinates of the acquired fixed object area information match those of previously acquired and stored fixed object area information. If the relative position of the fixed object candidate to the image capture device has been continuously fixed since the past, the fixed object candidate can be determined to be a fixed object, for example. Since fixed objects are likely to be stains that need to be removed or cracks or scratches that need to be addressed, the determination result can be output to notify the administrator of the image capture device, etc., of the presence of a fixed object. On the other hand, according to the above aspect, if the acquired fixed object area information does not match the previously acquired and stored fixed object area information, the fixed object candidate can be determined to have only temporarily become fixed in its relative position to the image capture device, and therefore, for example, the fixed object candidate can be determined not to be a fixed object. In this case, the fixed object candidate may be erroneously detected. By determining whether the fixed object candidate is the same in the time domain in this way, erroneously detected fixed object candidates are not determined to be fixed objects, thereby reducing the occurrence of erroneous detections and improving the detection rate of relatively attached fixed objects.

[0018] Also, for example, the fixed object detection means of the second invention is the fixed object detection means described in the first invention, and further comprises a distance calculation means for calculating the distance between coordinates indicated by multiple fixed object area information, each of the multiple fixed object area information corresponding to each of the fixed object candidates that appear simultaneously in the image, and a combination means for combining fixed object area information whose coordinate distance calculated by the distance calculation means is less than a threshold into one fixed object area information, and updating the coordinates indicated by the fixed object area information, and the fixed object area information stored in the memory means may be the fixed object area information after the coordinates have been updated.

[0019] This allows multiple pieces of fixed object area information that are potentially derived from one fixed object candidate and are located at a distance less than the site area to be treated as one piece of fixed object area information.

[0020] Also, for example, the fixed object detection means of the third invention may be the fixed object detection means described in the first or second invention, and may further include an attachment notification device that notifies the user of the determination result output by the observation means.

[0021] According to this, the attachment notification means can notify the user (for example, the manager of the photographing device or the mounted medium that has the photographing device mounted thereon; an example of the mounted medium is a moving body such as a vehicle) of the determination result.

[0022] Furthermore, for example, the fixed object detection means of the fourth invention may be the fixed object detection means described in any one of the first to third inventions, and the observation means may execute a match determination process to determine whether the coordinates indicated by the fixed object area information match the coordinates indicated by fixed object area information previously stored; a detection count recording process to add the number of match detections for fixed object area information determined to match in the match determination process and subtract the number of match detections for fixed object area information determined not to match in the match determination process; a fixed object area information confirmation process to determine that the coordinates indicated by fixed object area information whose match detection count exceeds a first threshold count are the coordinates of a fixed object in the image; and a fixed object area information deletion process to delete fixed object area information whose match detection count falls below a second threshold count.

[0023] This allows the number of match detections to be used to manage how long the fixed object candidate has been continuously positioned relative to the imaging device. If the number of match detections exceeds a first threshold number, the fixed object candidate is determined to be a fixed object, and a determination result can be output. Furthermore, the number of match detections is decremented if the coordinates of the fixed object area information do not match. If the number of match detections falls below a second threshold number, the fixed object area information can be deleted as a temporary fixed object candidate due to erroneous detection or the like. The second threshold number can be set to a relatively small number, such as 0 to 10. This configuration has the advantage of reducing the likelihood of a problem such as a missed detection, where fixed object area information actually corresponding to a fixed object is deleted because it does not match past fixed object area information just once.

[0024] Also, for example, the fixed object detection means according to the fifth invention is the fixed object detection means according to the fourth invention, and at least one of the first threshold number and the second threshold number may be different for each of the multiple regions into which the image is divided.

[0025] This allows a first threshold number of times to be set according to the likelihood of false detection, and a second threshold number of times to be set according to the likelihood of missed detection, for each of the multiple areas into which the video is divided.

[0026] Also, for example, the fixed object detection means of the sixth invention may be the fixed object detection means described in the first invention, and the observation means may estimate the coordinates of the fixed object within the image based on past fixed object area information stored in the memory means, and output the estimated result as the judgment result.

[0027] This makes it possible to output, as a determination result, an estimated result indicating the coordinates of the fixed object in the image, estimated based on the past fixed object area information stored in the storage means.

[0028] Also, for example, the fixed object detection means of the seventh invention is the fixed object detection means described in any one of the first to sixth inventions, and the observation means executes a coordinate information determination process to determine whether the coordinates indicated by the fixed object area information are included in a predetermined exclusion range within the image, and for fixed object area information whose coordinates are determined not to be included in the exclusion range in the coordinate information determination process, the observation means may determine whether they are the coordinates of a fixed object within the image.

[0029] This allows for individual settings to be made as to whether or not to determine whether or not the fixed object area information is the coordinates of a fixed object in the video, depending on whether or not the fixed object area information is included in a preset exclusion range in the video. For example, by not determining whether or not the fixed object area information is the coordinates of a fixed object in the video for fixed object area information included in a preset exclusion range in the video, it is possible to reduce the processing resources required for the determination.

[0030] Also, for example, the fixed object detection means of the eighth invention is a fixed object detection means described in any one of the first to seventh inventions, and the observation means performs a color judgment process to determine whether the display color in the image of the coordinates indicated by the fixed object area information corresponds to a predetermined excluded color, and for fixed object area information for which the display color in the image of the coordinates is determined not to correspond to an excluded color in the color judgment process, a judgment is made as to whether the coordinates are the coordinates of a fixed object in the image.

[0031] This allows for individual settings to be made as to whether or not to determine whether or not the fixed object area information is the coordinates of a fixed object in the video, depending on whether or not it corresponds to a preset exclusion color. For example, by not determining whether or not the fixed object area information of a color that corresponds to a preset exclusion color is the coordinates of a fixed object in the video, it is possible to reduce the processing resources required for the determination.

[0032] Also, for example, a fixed object detection means according to a ninth invention is a fixed object detection means described in any one of the first to eighth inventions, wherein the coordinates indicated by the fixed object area information include one end coordinates and other end coordinates, which are the coordinates of one end side and the other end side of the fixed object candidate in a predetermined direction, and if the distance between the one end coordinates indicated by one fixed object area information and the one end coordinates indicated by the other fixed object area information is within a one end threshold, and the distance between the other end coordinates indicated by one fixed object area information and the other end coordinates indicated by the other fixed object area information is within an other end threshold, the one fixed object area information may be determined to have coordinates that match those of the other fixed object area information.

[0033] This makes it possible to determine whether two pieces of fixed object area information match each other using the coordinates of one end and the coordinates of the other end.

[0034] Also, for example, a fixed object detection means according to a tenth invention is a fixed object detection means described in any one of the first to ninth inventions, wherein the coordinates indicated by the fixed object area information include central coordinates which are the coordinates of the center that bisects a line segment connecting a point on one end side and a point on the other end side in a predetermined direction of the fixed object candidate, and if the distance between the central coordinates indicated by one fixed object area information and the central coordinates indicated by another fixed object area information is within a central threshold, the one fixed object area information may be determined to have coordinates that match those of the other fixed object area information.

[0035] This makes it possible to determine whether two pieces of fixed object area information match using the center coordinates.

[0036] Furthermore, for example, a fixed object detection means according to an eleventh invention is a fixed object detection means according to any one of the first to eighth inventions, wherein the coordinates indicated by the fixed object area information include one end coordinates and other end coordinates, which are the coordinates of one end side and the other end side of the fixed object candidate in a predetermined direction, and a center coordinate, which is the coordinate of the center that bisects a line segment connecting a point on one end side and a point on the other end side of the fixed object candidate in a predetermined direction, and if the distance between the one end coordinate indicated by one fixed object area information and the one end coordinate indicated by the other fixed object area information is within a one end threshold, and the distance between the other end coordinate indicated by one fixed object area information and the other end coordinate indicated by the other fixed object area information is within a other end threshold, and the distance between the center coordinate indicated by one fixed object area information and the center coordinate indicated by the other fixed object area information is within a center threshold, then the one fixed object area information may be determined to have coordinates that match those of the other fixed object area information.

[0037] This makes it possible to determine whether two pieces of fixed object area information match each other using the one end coordinate, the other end coordinate, and the center coordinate.

[0038] Furthermore, for example, the fixed object detection means of the 12th invention may be the fixed object detection means described in any one of the first to 11th inventions, and may further include an image quality adjustment device that accepts input of original video captured by a camera device and outputs video whose image quality has been adjusted so as to improve the probability of detecting fixed object candidates.

[0039] This can improve the probability of finding fixed object candidates.

[0040] Also, for example, the fixed object detection means according to the thirteenth invention may be the fixed object detection means according to any one of the first to twelfth inventions, and may further include a removal machine that removes the fixed object based on the output judgment result.

[0041] This allows the removal of any continuously existing fixed objects by the removal machine.

[0042] Furthermore, for example, the fixed object detection means of the fourteenth invention may be the fixed object detection means described in any one of the first to thirteenth inventions, and may include: a reduction means for reducing an image; a feature detection means for detecting features from the reduced image reduced by the reduction means; a feature storage means for storing the features; an accumulative addition means for adding the features detected by the feature detection means to features already stored in the feature storage means and storing the result of the addition as a feature in the feature storage means; a binarization means for binarizing the stored features; a noise removal means for removing noise contained in the binarized feature data by reducing and expanding the binarized feature data; and a labeling means for generating fixed object area information based on the binarized feature data from which noise has been removed.

[0043] According to this, fixed object area information can be generated from the video by the reduction means, feature detection means, feature storage means, cumulative addition means, binarization means, noise removal means, and labeling means.

[0044] Also, for example, the fixed object detection means of the 15th invention is the fixed object detection means described in any one of the 1st to 14th inventions, wherein the fixed object is dirt adhering to a translucent member within the shooting range of the shooting device, and the fixed object candidate may be a dirt candidate.

[0045] This reduces the occurrence of erroneous detection and makes it possible to relatively improve the detection rate of attached dirt.

[0046] Also, for example, the fixed object detection means of the 16th invention is the fixed object detection means described in any one of the 1st to 14th inventions, wherein the fixed object is a raindrop attached to a translucent member within the shooting range of the shooting device, and the fixed object candidate is a raindrop candidate.

[0047] This reduces the occurrence of false detections and makes it possible to relatively improve the detection rate of attached raindrops.

[0048] Also, for example, the fixed object detection means of the 17th invention is the fixed object detection means described in the 16th invention, and further includes a removal machine that removes fixed objects based on the output judgment result, and the removal machine may be a wiper device that removes raindrops.

[0049] This allows the wiper device to remove raindrops that are continuously present.

[0050] Also, for example, the fixed object detection means of the 18th invention is a fixed object detection means described in any one of the 1st to 14th inventions, wherein the fixed object is a defect including at least one of a crack and a scratch that has occurred in a translucent member within the shooting range of the shooting device, and the fixed object candidate may be a candidate for the defect.

[0051] This reduces the occurrence of erroneous detection, making it possible to relatively improve the detection rate of defects that have occurred.

[0052] Furthermore, for example, the fixed object detection means according to a nineteenth aspect of the present invention may be the fixed object detection means according to any one of the first to eighteenth aspects of the present invention, and may further include: a marker attached to a position corresponding to designated coordinates on the image; a position and tilt detection means that measures, using the marker as a fixed object, a difference between the coordinates of the marker indicated in the determination result output by the observation means and the designated coordinates; a position and tilt correction device that corrects at least one of the position and tilt of the image based on the difference measured by the position and tilt detection means; and a display device that displays the image whose position and at least one of the tilt has been corrected by the position and tilt correction device.

[0053] According to this, when a marker attached to a position corresponding to a specified coordinate is detected as a fixed object, the coordinates indicated by the fixed object area information of the generated marker can be calculated by taking the difference, and the deviation equivalent to the difference can be corrected by correcting at least one of the position and tilt of the image, so that the image can be displayed on the display device in an appropriate state.

[0054] Furthermore, for example, the fixed object detection means according to the twentieth invention is the fixed object detection means according to any one of the first to nineteenth inventions, and includes a prediction means for detecting the coordinates of a portion of the contour shape shown in the image captured by the imaging device that differs from a predetermined shape, and a determination means for determining that the determination result is correct if the coordinates determined to be the coordinates of the fixed object in the determination result determined by the observation means match the coordinates detected by the prediction means, and the observation means may output the determination result determined to be correct by the determination means.

[0055] According to this, when the coordinates indicated by the fixed object area information corresponding to the fixed object in the determination result match the coordinates predicted to correspond to the fixed object based on the fact that the contour shape shown in the video is different from the predetermined shape, the determination result can be output as being correct. In other words, according to this aspect, the accuracy of the determination result can be guaranteed by verifying the determination result using the coordinates predicted to correspond to the fixed object based on the fact that the contour shape shown in the video is different from the predetermined shape.

[0056] Furthermore, for example, the fixed object detection means according to a 21st aspect of the present invention may be the fixed object detection means according to any one of the first to 20th aspects of the present invention, and may include: reduction means for reducing an image; feature detection means for detecting features from the reduced image reduced by the reduction means; feature storage means for storing the features; cumulative addition means for adding the features detected by the feature detection means to features already stored in the feature storage means and storing the result of the addition as features in the feature storage means; binarization means for binarizing the stored features; and straight line detection means for detecting the coordinates of straight-line fixed object candidates within the image from the binarized feature data using a Hough transform, which is a transformation algorithm.

[0057] According to this, the reduction means, feature detection means, feature storage means, cumulative addition means, binarization means, and straight line detection means can generate coordinates within the image of straight-line fixed object candidates from the image.

[0058] In addition, a fixed object detection method according to the 22nd invention is a fixed object detection method executed using a computer, which searches for fixed object candidates that are fixed objects that appear in an image captured by a camera and are candidates for fixed objects whose relative position to the camera is fixed, obtains fixed object area information indicating the coordinates of the fixed object candidates in the image, stores the obtained fixed object area information, determines that coordinates indicated by fixed object area information that match coordinates of fixed object area information previously stored are the coordinates of the fixed object in the image, and outputs the determination result.

[0059] Such a fixed object detection method can achieve the same effects as the fixed object detection means described above.

[0060] A twenty-third aspect of the present invention is a program for causing a computer to execute the above-described fixed object detection method.

[0061] Such a program can achieve the same effect as the fixed object detection means described above by using a computer.

[0062] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

[0063] (First Embodiment) A description will be given with reference to the configuration diagram of FIG. 1 showing the overall configuration of a camera monitor system mounted on a vehicle, including a fixed object detection means according to a first embodiment of the present invention.

[0064] As shown in the figure, the camera monitor system is composed of an imaging device 101 mounted in a position that allows it to photograph the rear or side of the vehicle, a fixed object detection means 102 that detects dirt, which is an example of a fixed object, from the image captured by the imaging device 101, and a fixed object notification device 103 that notifies the vehicle driver of the presence of dirt detected by the fixed object detection means 102.

[0065] Here, the image capturing device 101 is an analog video camera or a digital video camera that includes an image sensor having a light receiving element and transmits a video signal as an analog signal or a digital signal.

[0066] The notification destination of the fixed object notification device 103 may not be the operator, but may be a cleaning machine that cleans the lens portion attached to the image capture device 101. This makes it possible to automatically clean any dirt or other attachments (in other words, fixed objects) that are attached to the lens. Similarly, if the image capture device is installed behind a window, it is possible to notify a window washer to automatically clean the window in order to remove dirt that has adhered to the window.

[0067] The fixed object detection means 102 is realized by executing a predetermined program using a processor and a memory. The fixed object detection means 102 includes a dirt area information acquisition means 201 that searches for dirt candidates in the image input from the imaging device 101 and acquires their coordinate information, a distance calculation means 202 that calculates the distance between each dirt spot based on the dirt area information (an example of fixed object area information) acquired by the dirt area information acquisition means 201, a combination means 203 that groups nearby dirt spots into one dirt spot based on the distance acquired (calculated) by the distance calculation means 202 and calculates new dirt area information, a storage means 204 that temporarily stores the new dirt area information acquired by the combination means 203 in a storage device, and an observation means 205 that compares the past dirt area information stored in the storage means 204 with the newly detected dirt area information and determines that the continuously existing dirt area information is the finally existing dirt area information.

[0068] The dirty area information obtained by the dirty area information obtaining means 201 will be described with reference to FIG.

[0069] The dirty area information is expressed by the X and Y coordinates of points 301 and 302 on a virtual XY coordinate plane parallel to the image plane, and the range of the rectangle that can be drawn by these two points indicates the area where dirt has adhered.

[0070] That is, the coordinates of the point 301 are (X ３０１ , Y ３０１), and the coordinates of point 302 are (X ３０２ ,Y ３０２ ), the coordinates of the four points (X ３０１ , Y ３０１ ), (X ３０１ , Y ３０２ ), (X ３０２ , Y ３０１ ), (X ３０２ ,Y ３０２ ) is defined as the dirty area.

[0071] The coupling means 203 will be described with reference to FIG.

[0072] The distance calculation means 202 calculates the distance between stains, and if a stain contains another stain (FIG. 3(a)), a stain partially overlaps another stain (FIG. 3(b)), or the stains are adjacent and the separation distance D between them is less than a certain threshold (FIG. 3(c)), the areas are combined to form a single stain area information.

[0073] The distance between stains may be measured again for the new stain area information obtained by the merging, and based on the result, further merging may be performed by the merging means 203. This series of processes may be repeated multiple times.

[0074] The process of determining whether stains continue to exist in the observation means 205 will be described with reference to the flowchart shown in FIG.

[0075] A stain matching determination process (S501) is performed to extract one piece of previously detected stain area information and determine whether there is any stain area information that matches the newly detected stain area information. Specifically, it is determined whether there is any matching stain (S511).

[0076] If this process determines a match (Yes in S511), a stain detection count addition / update process (S502) is performed to increment the stain detection count by one. Then, the stain detection count is compared with a threshold (S512). If the stain detection count is greater than a specified threshold (Yes in S512), a stain area information confirmation process (S503) is performed to confirm the stain area information as existing stain (i.e., actual stain, not a stain candidate). If the stain detection count is equal to or less than the specified threshold (No in S512), step S503 is skipped.

[0077] On the other hand, if the result of the stain match determination process is a mismatch (Yes in S511), a stain detection count subtraction update process (S504) is performed to subtract one from the number of stain detections.The stain detection count is then compared with a threshold value (S514).The threshold value here is 0.As a result, if the number of stain detections becomes 0 (Yes in S514), the stain area information is considered to be non-existent stain, and a stain area information deletion process (S505) is performed to delete the stain area information.Also, if the number of stain detections is greater than the specified 0 (No in S514), step S505 is skipped.

[0078] This process is performed for all the dirty area information. That is, it is determined whether all the dirty area information has been checked (S510), and if it is determined that the checking has not been completed (No in S510), the process returns to step S501 and the same process is repeated for other dirty area information. If it is determined that the checking has been completed (Yes in S510), the process ends.

[0079] The threshold for determining whether a stain is present does not need to be constant. For example, the image may be divided into a plurality of regions (for example, six regions 801 to 806) as shown in FIG. 13, and the threshold may be different for each region.

[0080] For example, by increasing the threshold value for region 802, which contains a large amount of sky, it is possible to take more time to detect dirt in this region and reduce false detections. On the other hand, since dirt is easier to detect in regions 801 and 803, which contain many buildings, it is possible to shorten the time it takes to detect dirt in these regions by decreasing the threshold value.

[0081] Another method of determining whether or not there is dirt in the observation means 205 will be described with reference to the flowchart shown in FIG.

[0082] Coordinate information where dirt exists is estimated from past dirt area information accumulated in the storage means 204, and dirt prediction processing (S506) is performed to determine that dirt exists.

[0083] The determination of another continuously existing stain in the observation means 205 will be described with reference to the flowchart shown in Fig. 6 and Fig. 14. In the flowchart shown in Fig. 6, the same processes as those in Fig. 4 are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0084] One piece of previously detected dirty area information is extracted, and a coordinate information determination process (S507) is performed to determine whether the Y coordinate (the coordinate in the vertical direction on the page) of the bottom right point of the dirty area information is outside the upper exclusion range 901 and whether the Y coordinate of the top left point is outside the lower exclusion range 902. That is, a determination is made as to whether both coordinates are outside the exclusion ranges (S517). If the coordinate information determination process determines that the dirty area information is outside the exclusion range (Yes in S517), a dirt match determination process (S501) is performed to determine whether there is any dirt area information that matches the newly detected dirty area information. The subsequent processes are as described in FIG. 4. If a match is determined in this process, a dirt detection count addition / update process (S502) is performed to increment the number of dirt detections by one. If the result of this process is that the number of dirt detections is equal to or greater than a specified threshold, a dirt area information confirmation process (S503) is performed to confirm that the dirty area information is an existing dirt.

[0085] On the other hand, if the result of the stain match determination process (S501) is that the stains do not match, a stain detection count subtraction update process (S504) is performed to subtract 1 from the stain detection count. If the stain detection count becomes 0 as a result, the stain area information is determined to be non-existent stain, and a stain area information deletion process (S505) is performed to delete the stain area information.

[0086] This is performed for all the dirt area information.

[0087] The determination of another continuously existing stain in the observation means 205 will be described with reference to the flowchart shown in Fig. 7 and Fig. 15. In the flowchart shown in Fig. 7, the same processes as those in Fig. 4 are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0088] Here, stain color information is added to the stain area information.

[0089] One piece of previously detected dirt area information is extracted, and dirt color information is acquired. A dirt color determination process (S508) is performed to determine whether the dirt color information matches the exclusion color 1001 and the exclusion color 1002. That is, a determination is made as to whether the display color in the image of the coordinates corresponds to a preset exclusion color (S518). If the dirt color determination process determines that the dirt color information is not an exclusion color (Yes in S518), a dirt match determination process (S501) is performed to determine whether there is dirt area information that matches the newly detected dirt area information. The subsequent processes are as described in FIG. 4 . If a match is determined in this process, a dirt detection count addition / update process (S502) is performed to increment the dirt detection count by one. If the result of this process is that the dirt detection count is equal to or greater than a specified threshold, a dirt area information confirmation process (S503) is performed to confirm the dirt area information as existing dirt.

[0090] On the other hand, if the result of the stain match determination process (S501) is that the stains do not match, a stain detection count subtraction update process (S504) is performed to subtract 1 from the stain detection count. If the stain detection count becomes 0 as a result, the stain area information is determined to be non-existent stain, and a stain area information deletion process (S505) is performed to delete the stain area information.

[0091] This is performed for all the dirt area information.

[0092] Here, the exclusion colors 1001 and 1002 are assumed to be two colors, the average color of the sky on a sunny day and the average color of asphalt, but various colors may be assumed depending on the weather and time, and the exclusion colors may be three or more colors.

[0093] The determination of another continuously existing stain in the observation means 205 will be described with reference to the flowchart shown in Fig. 8 and Figs. 14 and 15. In the flowchart shown in Fig. 8, the same processes as those in Fig. 4 are denoted by the same reference numerals, and duplicated explanations will be omitted.

[0094] Here too, stain color information is added to the stain area information.

[0095] One piece of previously detected stained area information is extracted, and a coordinate information determination process (S507) is performed to determine whether the Y coordinate of the bottom right point of the stained area information is outside the upper exclusion range 901 and whether the Y coordinate of the top left point is outside the lower exclusion range 902. If the coordinate information determination process determines that the stained area information is outside the exclusion range, stain color information is further acquired. A stain color determination process (S508) is performed to determine whether the stain color information matches the exclusion colors 1001 and 1002. If the stain color determination process determines that the stain color information is not an exclusion color, a stain match determination process (S501) is performed to determine whether there is stain area information that matches the newly detected stained area information. The subsequent processes are as described in FIG. 4 . If a match is determined in this process, a stain detection count addition / update process (S502) is performed to increment the stain detection count by one. If the stain detection count becomes equal to or exceeds a specified threshold as a result, a stain area information confirmation process (S503) is performed to confirm that the stained area information is an existing stain.

[0096] On the other hand, if the result of the stain match determination process (S501) is that the stains do not match, a stain detection count subtraction update process (S504) is performed to subtract 1 from the stain detection count. If the stain detection count becomes 0 as a result, the stain area information is determined to be non-existent stain, and a stain area information deletion process (S505) is performed to delete the stain area information.

[0097] This is performed for all the dirt area information.

[0098] The stain matching determination process will be described with reference to FIG.

[0099] As described above, the dirt area information is composed of coordinate information of two points. １ , Y１ ) are the points 601 indicated by (X １ -X １ threshold) to (X １ +X １ threshold) and (Y １ -Y １ threshold) to (Y １ +Y １ The coordinates (X ２ , Y ２ ) are points 602 respectively (X ２ -X ２ threshold) to (X ２ +X ２ threshold) and (Y ２ -Y ２ threshold) to (Y ２ +Y ２ If the coordinate information of the newly detected stain area information is within the range of the match determination regions 603 and 604 (i.e., the difference is within the threshold), it is determined that the stains are the same.

[0100] In addition, another method of determining whether or not a stain coincides will be described with reference to FIG.

[0101] As described above, the dirt area information is composed of coordinate information of one point. １ , Y １ ) and the point 611 indicated by the coordinate (X ２ , Y ２ The center coordinate (X ｃ , Y ｃ The point 613 indicated by (X ｃ -X ｃ threshold) to (X ｃ +X ｃ threshold) and (Y ｃ -Y ｃ threshold) to (Y ｃ +Y ｃ If the center point of the two points of the newly detected stain area information is within the range of the match determination region 614 (i.e., the difference is within the threshold), it is determined that the stains are the same.

[0102] Another method for determining whether a stain coincides will be described with reference to FIG.

[0103] As described above, the dirt area information is composed of coordinate information of three points. １ , Y １ ) are the points 621 indicated by (X １ -X １ threshold) to (X １ +X １ threshold) and (Y １ -Y １ threshold) to (Y １ +Y １ The coordinates (X ２ , Y ２ ) are points 622 respectively (X ２ -X ２ threshold) to (X ２ +X ２ threshold) and (Y ２ -Y ２ threshold) to (Y ２ +Y ２ In addition, the coordinates of the center (X , Y ) between the two points 621 and 622 are ｃ , Y ｃ ) are points 625 respectively (X ｃ -X ｃ threshold) to (X ｃ +X ｃ threshold) and (Y ｃ -Y ｃ threshold) to (Y ｃ +Y ｃ If the coordinate information of the newly detected stain area information is within the range of the match determination regions 623 and 624 or within the range of the match determination region 626, it is determined that the stains are the same.

[0104] The stain area information acquisition means 201 will be described with reference to FIG.

[0105] The dirty area information acquisition means 201 is composed of an image reduction means 701 that reduces the image captured by the imaging device 101, a feature detection means 702 that detects features from the image data reduced by the reduction means, an accumulative addition means 703 that adds the results of the feature detection means 702 and the features stored in the feature storage means 704, a feature storage means 704 that saves the accumulated features calculated by the accumulative addition means 703 in a storage device, a binarization means 705 that binarizes the accumulated features saved in the storage device by the feature storage means 704, a noise removal means 706 that removes noise by reducing and expanding the data binarized by the binarization means 705, and a labeling means 707 that acquires dirty area information from the data acquired by the noise removal means 706.

[0106] As described above, the first embodiment can provide the effect of reducing the rate of false detections, in which areas other than dirt, such as the sky or road, are mistakenly determined to be dirt when detecting dirt.

[0107] Furthermore, by utilizing these mechanisms, it is possible to detect fixed objects such as raindrops, as well as the dirt exemplified above, and even cracks and scratches on lenses and window glass.

[0108] (Embodiment 2) The overall configuration of a camera monitor system mounted on a vehicle according to embodiment 2 of the present invention will be described with reference to the configuration diagram of Fig. 16. In the dirt detection means shown in Fig. 16, components having the same functions as those in Fig. 1 used in the description of embodiment 1 are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0109] As shown in the figure, the camera monitor system is composed of an imaging device 101 mounted in a position that allows it to capture images of the rear or side of the vehicle, an image quality adjustment device 104 that changes the image quality of the image captured by the imaging device 101, a fixed object detection means 102 that detects dirt from the image output by the image quality adjustment device 104, and a fixed object notification device 103 that notifies the driver of the presence of dirt detected by the fixed object detection means 102.

[0110] The image quality adjustment device 104 has functions such as color correction, color balance adjustment, contrast correction, tone curve adjustment, gamma correction, and contour emphasis to adjust the quality of the image input from the imaging device 101, and uses these functions to posterize the image and emphasize its contours, thereby correcting the image so that dirt is easier to detect.

[0111] As described above, the second embodiment improves the accuracy of dirt detection by the dirt area information acquisition means 201, and has the effect of further reducing the rate of false detections, in which areas other than dirt on the camera, such as the sky or road, are mistakenly determined to be dirty, compared to the first embodiment. Conversely, it is also possible to detect dirt that cannot be detected.

[0112] Third Embodiment The overall configuration of a camera monitor system mounted on a vehicle according to a third embodiment of the present invention will be described with reference to the configuration diagram of Fig. 17. In the dirt detection means shown in Fig. 17, components having the same functions as those in Fig. 1 used in the description of the first embodiment are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0113] In the figure, the system comprises a marker 105 including two or more coordinate points that are intentionally attached so as to be fixed at specific coordinates, an imaging device 101 that is mounted in a position that captures the rear or side of the vehicle and captures video including the marker 105, fixed object detection means 102 that detects the marker as a fixed object from the video captured by the imaging device 101, a position and tilt correction device 106 that corrects the position and angle of the input video in accordance with position and tilt information notified from the fixed object detection means 102, and a display device 107 that displays the video from the imaging device 101 corrected by the position and tilt correction device 106.

[0114] The fixed object detection means 102 is composed of a dirt area information acquisition means 201 that searches for dirt candidates from the image input from the imaging device 101 and acquires their coordinate information, a distance calculation means 202 that calculates the distance between stains from the dirt area information acquired by the dirt area information acquisition means 201, a combination means 203 that groups nearby stains into one stain based on the distance acquired by the distance calculation means 202 and calculates new dirt area information, a storage means 204 that temporarily saves the new dirt area information acquired by the combination means 203 in a storage device, an observation means 205 that compares the past dirt area information saved in the storage means 204 with the newly detected dirt area information and determines that the dirt area information that continues to exist is the dirt area information that will finally exist, and a position and tilt detection means 206 that compares the dirt area information of the marker 105 as a fixed object detected by the observation means 205 with the assumed dirt area information of the marker 105 and calculates the difference in position and angle.

[0115] As described above, according to the third embodiment, it is possible to automatically correct the tilt and position of the camera that occurs when the camera is installed in a factory, etc. Alternatively, it is possible to correct the tilt and position deviation of the camera that occurs while the operator is using the camera by attaching a fixed object to a specified position.

[0116] (Embodiment 4) The overall configuration of a camera monitor system mounted on a vehicle according to embodiment 4 of the present invention will be described with reference to the configuration diagram of Fig. 18. In the dirt detection means shown in Fig. 18, components having the same functions as those in Fig. 1 used in the description of embodiment 1 are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0117] As shown in the figure, the camera monitor system is composed of an imaging device 101 mounted in a position that allows it to photograph the rear or side of the vehicle, a fixed object detection means 102 that detects dirt from the image captured by the imaging device 101, and a fixed object notification device 103 that notifies the driver of the presence of dirt detected by the fixed object detection means 102.

[0118] The fixed object detection means 102 includes a dirt area information acquisition means 201 for searching for dirt candidates from the image input from the image capture device 101 and acquiring coordinate information thereof, a distance calculation means 202 for calculating the distance between dirt particles from the dirt area information acquired by the dirt area information acquisition means 201, a combining means 203 for grouping adjacent dirt particles into one dirt particle from the distance acquired by the distance calculation means 202 and calculating new dirt area information, a storage means 204 for temporarily storing the new dirt area information acquired by the grouping means 203 in a storage device, and a storage means 205 for temporarily storing the old dirt area information stored in the storage means 204. The system is composed of an observation means 205 that compares previous dirt area information with newly detected dirt area information and determines that the dirt area information that continues to exist is the dirt area information that finally exists, a prediction means 207 that detects coordinate information of parts of the image captured by the imaging device 101 where the outline of the image does not have the expected shape (for example, a part where part of the vehicle outline is missing), and a determination means 208 that compares the dirt area information obtained by the observation means 205 with the coordinate information obtained by the prediction means 207 and, when the same coordinate information is detected by both means, determines that the coordinates in question are dirt.

[0119] As described above, embodiment 4 improves the accuracy of dirt detection by the dirt area information acquisition means 201, and has the effect of further reducing the false detection rate, in which areas other than dirt on the camera, such as the sky or road, are mistakenly determined to be dirt, compared to embodiment 1.

[0120] (Embodiment 5) The overall configuration of a camera monitor system mounted on a vehicle according to embodiment 5 of the present invention will be described with reference to the configuration diagram of Fig. 19. In the fixed object area information acquisition means 201a shown in Fig. 19, components having the same functions as those in Fig. 12 used in the description of embodiment 1 are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0121] The fixed object area information acquisition means 201a is composed of an image reduction means 701 that reduces the image captured by the imaging device 101, a feature detection means 702 that detects feature amounts from the image data reduced by the reduction means, a feature storage means 704 that stores the feature amounts, a cumulative addition means 703 that adds the feature amounts stored in the feature storage means 704 and the feature amounts detected by the feature detection means 702, a binarization means 705 that binarizes the feature amounts stored in the feature storage means 704, and a line detection means 708 that detects line amounts from the data binarized by the binarization means 705 using a Hough transform, which is a conversion algorithm.

[0122] As described above, according to the fifth embodiment, it is possible to detect white lines dividing lanes on a road in the same way as fixed objects such as dirt, by using part of the function of the dirt detection means.

[0123] While the fixed object detection means and the like according to the present disclosure have been described above based on the embodiments and modifications, the present disclosure is not limited to the above-described embodiments. For example, the present disclosure also includes forms obtained by applying various modifications to the embodiments that would occur to a person skilled in the art, and forms realized by arbitrarily combining the components and functions of the embodiments within the scope of the present disclosure.

[0124] For example, components of the processing unit, such as the fixed object detection unit and the dirty area information acquisition unit, may be composed of one or more electronic circuits. Each of the one or more electronic circuits may be a general-purpose circuit or a dedicated circuit. The one or more electronic circuits may include, for example, a semiconductor device, an integrated circuit (IC), or a large-scale integration (LSI). The IC or LSI may be integrated on a single chip or on multiple chips. Although the IC or LSI is referred to here as an IC or LSI, the term may be changed depending on the degree of integration, and may be called a system LSI, a very large-scale integration (VLSI), or an ultra-large-scale integration (ULSI). Also, a field programmable gate array (FPGA), which is programmed after the LSI is manufactured, can be used for the same purpose.

[0125] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, or a computer program. Alternatively, the general or specific aspects may be realized as a computer-readable non-transitory recording medium such as an optical disk, a HDD, or a semiconductor memory on which the computer program is stored. Alternatively, the general or specific aspects of the present disclosure may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0126] The present invention increases the reliability of the device and enables accurate cleaning of the imaging device by reducing the false detection rate of fixed objects that can be detected in images captured through the imaging device.

[0127] Furthermore, this fixed object can detect raindrops and the like regardless of dirt.

[0128] Furthermore, by attaching an intentional fixed object with fixed coordinates, it is possible to detect and correct the installation position and angle of the imaging device.

[0129] REFERENCE SIGNS LIST 101 Photographing device 102 Fixed object detection means 103 Fixed object notification device 104 Image quality adjustment device 105 Marker 106 Position and tilt correction device 107 Display device 201 Dirt area information acquisition means 201a Fixed object area information acquisition means 202 Distance calculation means 203 Combining means 204 Storage means 205 Observation means 206 Position and tilt detection means 207 Prediction means 208 Determination means 301, 302, 601, 602, 611, 612, 613, 621, 622, 625 Points 603, 604, 614, 623, 624, 626 Match determination area 701 Image reduction means 702 Feature amount detection means 703 Accumulation and addition means 704 Feature amount storage means 705 Binarization means 706 Noise removal means 707 Labeling means 708 Line detection means 801, 802, 803, 804, 805, 806 Area 901 Upper exclusion range 902 Lower exclusion range 1001, 1002 Exclusion color

Claims

1. A photographing device; a fixed object area information acquisition means for searching for a fixed object candidate that appears in an image captured by the image capture device and is a candidate for a fixed object whose relative position with respect to the image capture device is fixed, and acquiring fixed object area information indicating the coordinates of the fixed object candidate in the image; A storage means for storing the acquired fixed object area information; an observation means for determining that the coordinates indicated by the fixed object area information, which coordinates match those of the fixed object area information previously stored in the storage means, are the coordinates of the fixed object in the image, and outputting a determination result. Fixed object detection means.

2. a distance calculation means for calculating a distance between coordinates indicated by a plurality of pieces of fixed object area information, each of the plurality of pieces of fixed object area information corresponding to each of the fixed object candidates simultaneously captured in an image; a combining unit that combines the fixed object area information pieces whose coordinates are calculated by the distance calculating unit and whose distance is less than a threshold value and updates the coordinates indicated by the fixed object area information pieces, The fixed object area information stored in the storage means is the fixed object area information after the coordinates have been updated. The fixed object detection means according to claim 1 .

3. The monitoring means may further include a fixed object notification device that notifies a user of the determination result output by the monitoring means. The fixed object detection means according to claim 1 .

4. The observation means is a matching determination process for determining whether the coordinates indicated by the fixed object area information match the coordinates indicated by the fixed object area information previously stored; a detection count recording process for adding a match detection count of the fixed object area information determined to match in the match determination process and subtracting a match detection count of the fixed object area information determined to not match in the match determination process; a fixed object area information determination process for determining that the coordinates indicated by the fixed object area information in which the number of coincidence detections exceeds a first threshold number are the coordinates of the fixed object in the video; a fixed object area information deleting process for deleting the fixed object area information in which the number of coincidence detections has become equal to or less than a second threshold number of times. The fixed object detection means according to claim 1 .

5. At least one of the first threshold number of times and the second threshold number of times is different for each of a plurality of regions obtained by dividing the image.

5. The fixed object detection means according to claim 4.

6. The observation means estimates coordinates of the fixed object in the image based on past fixed object area information stored in the storage means, and outputs the estimated result as the determination result. The fixed object detection means according to claim 1 .

7. The observation means is execute a coordinate information determination process to determine whether or not the coordinates indicated by the fixed object area information are included in a preset exclusion range within the video; For the fixed object area information whose coordinates are determined not to be included in the exclusion range in the coordinate information determination process, it is determined whether or not the coordinates are the coordinates of the fixed object in the image. The fixed object detection means according to claim 1 .

8. The observation means is execute a color determination process for determining whether or not a display color of the coordinates indicated by the fixed object area information in the image corresponds to a preset exclusion color; In the color determination process, for the fixed object area information whose display color in the image of the coordinates is determined not to correspond to the excluded color, a determination is made as to whether or not the fixed object area information corresponds to the coordinates of the fixed object in the image. The fixed object detection means according to claim 1 .

9. The coordinates indicated by the fixed object area information include one end coordinates and other end coordinates, which are the coordinates of one end side and the other end side of the fixed object candidate in a predetermined direction. When the distance between the one end coordinate indicated by one of the fixed object area information and the one end coordinate indicated by the other of the fixed object area information is within a one end threshold, and the distance between the other end coordinate indicated by the one of the fixed object area information and the other end coordinate indicated by the other of the fixed object area information is within an other end threshold, it is determined that the one of the fixed object area information has coordinates that match the other of the fixed object area information. The fixed object detection means according to claim 1 .

10. The coordinates indicated by the fixed object area information include center coordinates that are the center coordinates that bisect a line segment connecting a point on one end side and a point on the other end side of the fixed object candidate in a predetermined direction. When the distance between the central coordinates indicated by one of the fixed object area information and the central coordinates indicated by the other fixed object area information is within a central threshold, the one of the fixed object area information is determined to have coordinates that match those of the other fixed object area information. The fixed object detection means according to claim 1 .

11. The coordinates indicated by the fixed object area information include one end coordinates and other end coordinates, which are the coordinates of one end side and the other end side of the fixed object candidate in a predetermined direction, and a center coordinate, which is the center coordinate that bisects a line segment connecting a point on one end side and a point on the other end side of the fixed object candidate in the predetermined direction. When the distance between the one end coordinate indicated by one of the fixed object area information and the one end coordinate indicated by the other of the fixed object area information is within a one end threshold, and the distance between the other end coordinate indicated by the one of the fixed object area information and the other end coordinate indicated by the other of the fixed object area information is within an other end threshold, and the distance between the center coordinate indicated by the one of the fixed object area information and the center coordinate indicated by the other of the fixed object area information is within a center threshold, it is determined that the coordinates of the one of the fixed object area information match those of the other of the fixed object area information. The fixed object detection means according to claim 1 .

12. The image processing apparatus further includes an image quality adjustment device that receives an input of an original image captured by the image capturing device, and outputs the image whose image quality has been adjusted so as to improve the probability of searching for the fixed object candidate. The fixed object detection means according to claim 1 .

13. The apparatus further includes a remover for removing the fixed object based on the outputted determination result. The fixed object detection means according to claim 1 .

14. a reduction means for reducing the image; a feature amount detection means for detecting a feature amount from the reduced image reduced by the reduction means; A feature amount storage means for storing the feature amount; an accumulating means for adding the feature quantity detected by the feature quantity detecting means and the feature quantity already stored in the feature quantity storing means, and storing the result of the addition in the feature quantity storing means as the feature quantity; a binarization means for binarizing the stored feature quantity; a noise removing means for removing noise contained in the binarized feature data by shrinking and expanding the binarized feature data; and a labeling means for generating the fixed object area information based on the binarized feature amount data from which noise has been removed. The fixed object detection means according to claim 1 .

15. the fixed object is dirt adhering to a light-transmitting member within a photographing range of the photographing device, The fixed object candidate is a stain candidate. The fixed object detection means according to claim 1 .

16. The fixed object is a raindrop attached to a translucent member within a shooting range of the photographing device, The fixed object candidate is the raindrop candidate. The fixed object detection means according to claim 1 .

17. a remover that removes the fixed object based on the output determination result, The remover is a wiper device that removes the raindrops. The fixed object detection means according to claim 16.

18. the fixed object is a defect including at least one of a crack and a scratch that has occurred in a translucent member within a photographing range of the photographing device, The fixed object candidate is a candidate for the defect. The fixed object detection means according to claim 1 .

19. A marker attached to a position corresponding to a specified coordinate on the video; a position and tilt detection means for measuring a difference between the coordinates of the marker indicated in the determination result output by the observation means and the designated coordinates, with the marker being the fixed object; a position and tilt correction device that corrects at least one of the position and tilt of the image based on the difference measured by the position and tilt detection means; a display device that displays an image whose position and / or tilt has been corrected by the position and tilt correction device. The fixed object detection means according to claim 1 .

20. a prediction means for detecting coordinates of a portion of an image captured by the imaging device, the portion having a contour shape that differs from a predetermined shape; a determination means for determining that the determination result is correct when the coordinates determined to be the coordinates of the fixed object in the determination result determined by the observation means match the coordinates detected by the prediction means, The observation means outputs the judgment result that is judged to be correct by the judgment means. The fixed object detection means according to claim 1 .

21. a reduction means for reducing the image; a feature amount detection means for detecting a feature amount from the reduced image reduced by the reduction means; A feature amount storage means for storing the feature amount; an accumulating means for adding the feature quantity detected by the feature quantity detecting means and the feature quantity already stored in the feature quantity storing means, and storing the result of the addition in the feature quantity storing means as the feature quantity; a binarization means for binarizing the stored feature quantity; and a straight line detection means for detecting the coordinates of the fixed object candidate in the image in a straight line shape by using a Hough transform, which is a transformation algorithm, from the binarized feature data. The fixed object detection means according to any one of claims 1 to 20.

22. 1. A computer-implemented method for detecting fixed objects, comprising: searching for a fixed object candidate that is a fixed object that appears in an image captured by an image capture device and has a fixed position relative to the image capture device, and acquiring fixed object area information that indicates coordinates of the fixed object candidate in the image; storing the acquired fixed object area information; The coordinates indicated by the fixed object area information that match the coordinates of the fixed object area information stored in the past are determined to be the coordinates of the fixed object in the image, and the determination result is output. Fixed object detection method.

23. A method for causing a computer to execute the fixed object detection method according to claim 22. program.