Object detection device, object detection method, and recording medium

Abstract

One purpose of the present invention is to provide an object detection device capable of detecting not only a specific object but also a foreign body. An object detection device according to one aspect of the present disclosure comprises: an acquisition means for acquiring a captured image in which a road is captured; a division means for dividing the captured image into a plurality of blocks; a calculation means for calculating the degree of similarity between the blocks; and a determination means for determining, as a block in which a foreign body appears, a block for which the number of other blocks considered to be similar according to the degree of similarity is smaller than a prescribed number.

Description

Object detection device, object detection method, and recording medium

[0001] The present disclosure relates to an object detection device, an object detection method, and a recording medium.

[0002] There are technologies that use image analysis technology to detect objects on the road. For example, Patent Literature 1 discloses a technology that uses thermal images acquired by an infrared camera to detect pedestrians around a vehicle.

[0003] Japanese Patent Application Laid-Open No. 2006-350699

[0004] For example, the presence of foreign objects on roads, such as runways and roads, may impede the safe passage of aircraft and vehicles. To address this issue, detecting foreign objects on the road is considered. However, there is a possibility that various objects may exist on the road. Therefore, it is necessary to detect not only specific objects but also various foreign objects that may exist on the road.

[0005] One object of the present disclosure is to provide an object detection device or the like that is capable of detecting foreign objects, not limited to specific objects.

[0006] An object detection device according to one aspect of the present disclosure includes an acquisition means for acquiring an image of a road, a division means for dividing the image into a plurality of blocks, a calculation means for calculating the similarity between each block, and a determination means for determining that a block in which the number of other blocks deemed to be similar based on the similarity is less than a predetermined number is a block in which a foreign object is captured.

[0007] An object detection method according to one aspect of the present disclosure acquires an image of a road, divides the image into multiple blocks, calculates the similarity between each block, and determines that a block contains a foreign object if the number of other blocks deemed similar based on the similarity is less than a predetermined number.

[0008] A recording medium according to one aspect of the present disclosure non-temporarily records a program that causes a computer to execute the following processes: acquiring an image of a road; dividing the image into a plurality of blocks; calculating the similarity between each block; and determining that a block in which the number of other blocks deemed similar based on the similarity is less than a predetermined number is a block in which a foreign object is captured.

[0009] According to the present disclosure, foreign matter can be detected not only for specific objects but also for other objects.

[0010] 1 is a diagram schematically illustrating an example of a configuration including an object detection device of the present disclosure. FIG. 2 is a first block diagram illustrating an example of a functional configuration of an object detection device of the present disclosure. FIG. 3 is a first flowchart illustrating an example of an operation of an object detection device of the present disclosure. FIG. 4 is a diagram schematically illustrating an example of a configuration of an object detection system of the present disclosure. FIG. 5 is a second block diagram illustrating an example of a functional configuration of an object detection device of the present disclosure. FIG. 6 is a first diagram illustrating an example of a captured image of the present disclosure. FIG. 7 is a first diagram illustrating an example of a block of the present disclosure. FIG. 8 is a diagram illustrating an example of clustering of the present disclosure. FIG. 9 is a diagram illustrating an example of information showing a determination result of the present disclosure. FIG. 10 is a diagram illustrating an example of output information including an imaging range of the present disclosure. FIG. 11 is a second flowchart illustrating an example of an operation of an object detection device of the present disclosure. FIG. 12 is a second diagram illustrating an example of a captured image of the present disclosure. FIG. 13 is a second diagram illustrating an example of a block of the present disclosure. FIG. 14 is a block diagram illustrating an example of a functional configuration of an object detection system of the present disclosure. FIG. 15 is a diagram illustrating an example of inspection information of the present disclosure. FIG. 16 is a first diagram illustrating an example of output information of the present disclosure. FIG. 17 is a diagram illustrating an example of an input form of the present disclosure. FIG. 18 is a second diagram illustrating an example of output information of the present disclosure. FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer device that realizes an object detection device according to the present disclosure

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] First Embodiment An outline of an object detection device according to a first embodiment will be described.

[0013] Fig. 1 is a diagram schematically illustrating an example of a configuration including an object detection device 100. The object detection system 1000 includes the object detection device 100. In the example of Fig. 1, the object detection system 1000 further includes an imaging device 200. Note that the configuration of the object detection system 1000 is not limited to this example. For example, the object detection system 1000 may include further components.

[0014] The object detection device 100 is communicably connected to the image capture device 200 via a wireless or wired network. The object detection device 100 and the image capture device 200 may be communicable with other devices (not shown). For example, the object detection device 100 may be communicably connected to a device that outputs information, such as a terminal device. The object detection device 100 is, for example, a server device, but is not limited to this example. The object detection device 100 may be realized as a cloud server. Furthermore, the object detection device 100 may be constructed by multiple devices.

[0015] The imaging device 200 is a device that takes photographs. The imaging device 200 is located in a position where it can take photographs of the road. The road is, for example, a traffic route such as a runway or a road. The imaging device 200 may be a stationary device. The imaging device 200 may also be a device mounted on a moving object such as an aircraft or a vehicle. There may be multiple imaging devices 200. The imaging device 200 transmits the captured image to the object detection device 100.

[0016] An example of the object detection device 100 of the present disclosure is a device that detects foreign objects on a road using images captured by an imaging device 200. In the present disclosure, a foreign object refers to an object whose presence in a particular location is determined to be unusual. For example, a foreign object on a road is an object that may interfere with the safe passage of aircraft, vehicles, etc. Examples of foreign objects on a road include, but are not limited to, animals, machine parts, and garbage.

[0017] Next, an example of the functional configuration of the object detection device 100 will be described.

[0018] 2 is a block diagram showing an example of the functional configuration of the object detection device 100. The object detection device 100 includes an acquisition unit 110, a division unit 120, a calculation unit 130, and a determination unit 140.

[0019] The acquisition unit 110 acquires a captured image. For example, the image capturing device 200 captures an image of a road. The acquisition unit 110 then acquires the captured image of the road from the image capturing device 200.

[0020] In this way, the acquisition unit 110 acquires the captured image of the road. The acquisition unit 110 is an example of an acquisition means.

[0021] The dividing unit 120 performs a dividing process on the captured image. Specifically, the dividing unit 120 divides the captured image into a plurality of blocks. That is, a block indicates a partial area within the captured image. All the blocks within the captured image may be the same size. However, this example is not limiting, and the size of the block may be determined for each area within the captured image.

[0022] In this way, the dividing unit 120 divides the captured image into a plurality of blocks. The dividing unit 120 is an example of a dividing means.

[0023] The calculation unit 130 calculates the similarity for each block. Specifically, the calculation unit 130 calculates the similarity for each block with other blocks. For example, the calculation unit 130 extracts a feature amount for each block. Then, the calculation unit 130 calculates the similarity based on the extracted feature amount. For example, the calculation unit 130 may calculate the distance between the blocks based on the feature amount as the similarity. Note that the method for calculating the similarity is not limited to this example.

[0024] In this way, the calculation unit 130 calculates the similarity between each block. The calculation unit 130 is an example of a calculation means.

[0025] The determination unit 140 determines the presence or absence of a foreign object in a block based on the similarity. Specifically, the determination unit 140 determines, among multiple blocks, a block that has no or few other similar blocks as a block that reflects a foreign object. For example, the determination unit 140 identifies other similar blocks for each block based on the similarity. Then, the determination unit 140 determines, as a block that reflects a foreign object, a block for which the number of other blocks considered to be similar is less than a predetermined number. Note that the determination method is not limited to this example. For example, the determination unit 140 may cluster blocks based on the similarity. Then, the determination unit 140 may determine, as a block that reflects a foreign object, a block for which the number of other blocks included in the same cluster is less than a predetermined number.

[0026] For example, if the image is of a road, the surface portion of the roadway will be visible in the image. In this case, it is considered that there are many blocks that show the surface portion. Therefore, it is highly likely that similar blocks exist for blocks that mainly show the surface portion. On the other hand, it is less likely that similar blocks exist for blocks that contain foreign objects than for blocks that mainly show the surface portion. Therefore, the determination unit 140 can determine, among multiple blocks, a block that does not have or has only a few similar blocks as a block that contains a foreign object. Note that the method for determining the presence or absence of a foreign object is not limited to this example.

[0027] In this way, the determining unit 140 determines that a block in which the number of other blocks that are deemed to be similar based on the similarity is less than a predetermined number is a block in which a foreign object appears. The determining unit 140 is an example of a determining means.

[0028] Next, an example of the operation of the object detection device 100 will be described with reference to Fig. 3. Fig. 3 is a first flowchart illustrating an example of the operation of the object detection device 100. Note that in this disclosure, each step in the flowchart will be represented by a number assigned to each step, such as "S1."

[0029] The acquisition unit 110 acquires an image of a road (S1).

[0030] The dividing unit 120 divides the captured image into a plurality of blocks (S2).

[0031] The calculation unit 130 calculates the similarity between each block (S3).

[0032] The determining unit 140 determines that a block in which the number of other blocks that are considered to be similar based on the similarity is less than a predetermined number is a block in which a foreign object is reflected (S4).

[0033] In this way, the object detection device 100 of the first embodiment acquires a captured image of a road and divides the captured image into multiple blocks. The object detection device 100 then calculates the similarity between each block and determines that a block contains a foreign object if the number of other blocks deemed similar based on the similarity is less than a predetermined number. This allows the object detection device 100 to detect foreign objects, regardless of the specific object.

[0034] There are methods for detecting objects that use a model that has learned the characteristics of the object to be detected. Such methods require the definition of the object to be detected. However, various types of foreign objects are expected to be present on roads, etc. Therefore, it can be difficult to define the object to be detected. In contrast, the object detection device 100 can detect the object to be detected without any special definition.

[0035] Furthermore, the object detection device 100 detects foreign objects from the captured image. That is, the object detection device 100 can detect foreign objects without using various sensors other than a device that captures the captured image.

[0036] Second Embodiment Next, an object detection device according to a second embodiment will be described. In the second embodiment, a further example of the object detection device 100 described in the first embodiment will be described. Note that some of the description overlapping with the first embodiment will be omitted.

[0037] In addition, in this embodiment, an example will be mainly described in which the object detection device 100 detects foreign objects on the road at an airport. However, the application of the object detection device 100 is not limited to this example.

[0038] Fig. 4 is a diagram schematically illustrating an example of the configuration of an object detection system 1000. The object detection system 1000 includes an object detection device 100. In the example of Fig. 4, the object detection system 1000 further includes an image capture device 200 and a terminal device 300. The object detection device 100 is communicably connected to the image capture device 200 and the terminal device 300 via a wireless or wired network.

[0039] The image capturing device 200 captures images of roads within an airport. For example, the image capturing device 200 may capture images of a runway or an airplane parking area. The image capturing device 200 may be an apparatus capable of capturing images of a wide range, including roads within an airport. The image capturing device 200 may be capable of performing operations such as panning, tilting, and zooming. The image capturing device 200 may also be a telephoto camera.

[0040] The terminal device 300 is a device capable of outputting information. For example, the terminal device 300 outputs information acquired from the object detection device 100. The terminal device 300 may be a personal computer. The terminal device 300 may also be a portable terminal such as a smartphone or a tablet terminal. The terminal device 300 is not limited to this example. It should be noted that there may be a plurality of terminal devices 300. For example, one terminal device 300 may be installed in a control tower. Furthermore, other terminal devices 300 may be carried by workers working on runways, parking areas, etc.

[0041] Fig. 5 is a block diagram showing an example of the functional configuration of the object detection device 100. As shown in Fig. 5, the object detection device 100 includes an acquisition unit 110, a division unit 120, a calculation unit 130, a determination unit 140, and a display control unit 150. Furthermore, in the example of Fig. 5, the object detection device 100 includes a storage device 190. The storage device 190 may be an external device capable of communicating with the object detection device 100.

[0042] The acquisition unit 110 acquires photographed images of the road from the camera device 200. Fig. 6 is a diagram showing an example of a photographed image. The photographed image in Fig. 6 is an image of a portion of a runway. For example, assume that the subject being photographed is the entire runway shown in Fig. 4. In this case, the camera device 200 sequentially captures images of the entire runway. The acquisition unit 110 may acquire photographed images of the entire runway photographed by the camera device 200. In other words, the acquisition unit 110 may acquire multiple photographed images from the camera device 200.

[0043] The acquisition unit 110 may associate the acquired photographed image with detailed information about the photographed image and store the associated information in the storage device 190. The detailed information about the photographed image includes the time of photographing, location information about the photographed location, and photographing parameters of the photographing device 200 when the photograph was taken. The location information about the photographed location and the photographing parameters will be described later.

[0044] The dividing unit 120 divides the captured image into a plurality of blocks. A block corresponds to a partial area within the captured image. FIG. 7 is a diagram showing an example of a block. More specifically, FIG. 7 is a diagram showing an example in which the captured image of FIG. 6 is divided into a plurality of blocks. In the example of FIG. 7, the captured image is divided into 42 blocks. In addition, in this example, block numbers are assigned to the blocks. Specifically, block numbers a1 to g6 are assigned to each block. For example, a foreign object is captured in block number c3. When a plurality of captured images are acquired by the acquiring unit 110, the dividing unit 120 may divide each of the plurality of captured images into a plurality of blocks.

[0045] In the example of FIG. 7, all the blocks have the same size. The block size indicates the size of the block. However, the block size and the number of divisions are not limited to this example. The block size may be changed. Furthermore, the block size may be different for each predetermined area in the captured image. Furthermore, only a portion of the area in the captured image may be divided into blocks. When there are multiple captured images, the block size may be different for each captured image.

[0046] The block size can be changed. For example, the farther the distance between the camera device 200 and the subject, the smaller the subject will appear in the captured image. In this case, by setting the block size smaller as the distance between the camera device 200 and the subject increases, it becomes easier to extract the features of the object. The dividing unit 120 may change the block size depending on the distance between the camera device 200 and the shooting location. Hereinafter, in this disclosure, a location that appears in the captured image will be referred to as the shooting location. Furthermore, the distance between the camera device 200 and the shooting location will be referred to as the shooting distance. For example, the dividing unit 120 may calculate the block size as shown in the following equation 1.

[0047] (Equation 1) Block size = α × shooting distance, where α is a coefficient. Information indicating the shooting distance may be stored in advance in a storage device (not shown). Alternatively, the dividing unit 120 may calculate the shooting distance from the position information of the camera device 200 and the position information of the shooting location. The position information of the camera device 200 and the position information of the shooting location may be stored in advance in a storage device. Note that the storage device may be a device included in the object detection device 100, or may be an external device capable of communicating with the object detection device 100. The position information of the shooting location may be calculated from the position information of the camera device 200 and the shooting parameters of the camera device 200. The shooting parameters are, for example, information indicating settings such as the zoom level, angle of view, pan, tilt, and roll of the camera device 200.

[0048] Alternatively, the information indicating the shooting distance, or the position information of the image capturing device 200 and the position information of the image capturing location may be information acquired from an external device. For example, the acquisition unit 110 may acquire the information indicating the shooting distance, or the position information of the image capturing device 200 and the position information of the image capturing location from the terminal device 300. In this case, the position information may be information input by the user on the terminal device 300, for example.

[0049] In this way, the dividing section 120 may set the block size, which is the size of the block, depending on the distance between the photographing device 200 that photographs the image and the photographing location.

[0050] The way a subject appears also changes depending on the zoom level of the image capture device 200. For example, the higher the zoom level, the larger the subject appears in the captured image. Therefore, the dividing unit 120 may change the block size taking into account the zoom level of the image capture device 200. The block size may also be changed depending on the size of the object to be detected. For example, the smaller the object to be detected, the smaller the block size may be. For example, the dividing unit 120 may calculate the block size using the following equation 2.

[0051] (Equation 2) Block size = α × shooting distance + β × zoom level + γ × size of detection target, where β and γ are coefficients. Information indicating the zoom level and information indicating the size of the detection target may be stored in advance in a storage device (not shown). Alternatively, information indicating the zoom level and information indicating the size of the detection target may be information acquired from an external device. For example, the acquisition unit 110 may acquire information indicating the zoom level from the imaging device 200. Alternatively, the acquisition unit 110 may acquire information indicating the zoom level and information indicating the size of the detection target from the terminal device 300.

[0052] In this way, the dividing section 120 may set the block size according to the distance between the image capturing device 200 and the image capturing location, the zoom level of the image capturing device 200, and the size of the detection target.

[0053] The calculation unit 130 includes an extraction unit 1301 and a similarity calculation unit 1302. The extraction unit 1301 extracts features from an image. More specifically, the extraction unit 1301 extracts features from each block. At this time, the extraction unit 1301 may extract the features using a method that utilizes deep learning, such as a convolutional neural network (CNN). For example, the extraction unit 1301 may extract the features by performing a convolution operation on the block and performing a downsampling operation. The present invention is not limited to this example, and the extraction unit 1301 may extract the features using various machine learning-based methods. Alternatively, the extraction unit 1301 may extract features by an artificial method such as HOG (Histograms of Oriented Gradients) features, SIFT (Scale-Invariant Feature Transform) features, and Haar-Like features. In this case, the calculation unit 130 may further compress the extracted features by principal component analysis or the like.

[0054] Furthermore, the extraction unit 1301 may extract feature amounts for each block after performing noise removal processing on the blocks or the entire captured image. Examples of noise removal methods include smoothing methods such as a median filter, a Gaussian filter, and a bilateral filter. The noise removal method is not limited to these examples. In this way, the extraction unit 1301 may perform smoothing processing on the captured image and extract feature amounts from each block of the captured image after the smoothing processing.

[0055] The similarity calculation unit 1302 calculates the similarity between each block based on the feature amounts extracted by the extraction unit 1301. For example, the similarity calculation unit 1302 calculates the distance between the feature amount of one block and each of the feature amounts of the other blocks as the similarity. That is, the similarity may be the distance between the feature amounts extracted from each block. In this case, the greater the distance between the feature amount of one block and the feature amount of the other blocks, the greater the similarity. The method of calculating the similarity is not limited to this example. For example, the similarity may be a value that increases as the distance between the feature amount of one block and the feature amount of the other blocks decreases. In this case, the similarity may be expressed as 1-distance (0≦distance≦1). Furthermore, for example, the similarity calculation unit 1302 compares the feature amount of one block with each of the feature amounts of the other blocks. Then, the similarity calculation unit 1302 may calculate the similarity between the one block and each of the other blocks based on the comparison results. In this case, the similarity calculation unit 1302 may set a value that increases as the difference between the feature amounts increases. In this way, the similarity calculation unit 1302 may calculate the distance between each block based on the extracted feature amounts as the similarity.

[0056] When multiple captured images are acquired, the extraction unit 1301 may extract feature amounts for each block of each captured image, and the similarity calculation unit 1302 may calculate the similarity between each block for each captured image.

[0057] The determination unit 140 includes a classification unit 1401 and a foreign substance determination unit 1402. The classification unit 1401 classifies blocks based on their similarity. For example, the classification unit 1401 performs clustering on the blocks using the similarity between the blocks. That is, the classification unit 1401 classifies the blocks into clusters.

[0058] The foreign substance determination unit 1402 determines the presence or absence of a foreign substance based on the block classification results. For example, the foreign substance determination unit 1402 identifies a cluster that includes less than a predetermined number of blocks from among the classified clusters. The foreign substance determination unit 1402 then determines that a foreign substance is present in a block included in the identified cluster. Hereinafter, a block determined to contain a foreign substance may also be referred to as an abnormal block.

[0059] FIG. 8 is a diagram illustrating an example of clustering. More specifically, FIG. 8 is a tree diagram showing the distance between blocks or clusters. The vertical axis indicates the distance, and the horizontal axis indicates the block number. The block number is identification information of the block. The lines connecting individual blocks indicate the distance between the blocks. The lines connecting multiple blocks to other blocks indicate the distance between the representative value of the multiple blocks and the other blocks. The representative value may be a feature value of one of the multiple blocks, or may be information obtained by performing a predetermined calculation on the feature values ​​of the multiple blocks. For example, the representative value may be the average value, median, mode, etc. of the feature values ​​of the multiple blocks.

[0060] In the example of FIG. 8 , a threshold is set. For example, a group of branches with a distance equal to or less than the threshold is considered to be one cluster. Blocks within one cluster are considered to be similar blocks. In other words, blocks with a similarity equal to or less than the threshold are considered to be similar blocks. On the other hand, blocks in other clusters are considered to be dissimilar blocks to one block. In the example of FIG. 8 , 42 blocks are classified into four clusters. More specifically, blocks a1, b1, ..., g1 are the first cluster. Blocks a2, b2, ..., g2, block a3, block g3, blocks a4, b4, ..., g4, blocks a5, b5, ..., g5, and blocks a6, b6, ..., g6 are the second cluster. Blocks b3, d3, e3, and f3 are the third cluster. Block c3 is the fourth cluster. In this example, the foreign substance determination unit 1402 identifies a cluster from among the four clusters that includes less than a predetermined number of blocks. For example, the predetermined number is set to 3. In this case, the foreign substance determination unit 1402 identifies a fourth cluster that includes only one block. The foreign substance determination unit 1402 then determines that a foreign substance is present in block c3 of the fourth cluster. Note that the threshold value and the predetermined number are not limited to this example.

[0061] When multiple captured images are acquired, the classification unit 1401 may perform clustering on the blocks for each captured image. Then, the foreign substance determination unit 1402 may identify, from among the classified clusters for each captured image, a cluster that includes less than a predetermined number of blocks. Furthermore, the foreign substance determination unit 1402 determines that a foreign substance is captured in a block included in the identified cluster for each captured image.

[0062] The display control unit 150 outputs various information. Specifically, the display control unit 150 causes the terminal device 300 to display the captured image. For example, the display control unit 150 causes the terminal device 300 to display the captured image of Fig. 6. The display control unit 150 may also cause the terminal device 300 to display a captured image divided into blocks as shown in Fig. 7. The display control unit 150 is an example of a display control means.

[0063] The display control unit 150 may also cause the terminal device 300 to display information indicating the determination result. FIG. 9 is a diagram illustrating an example of information indicating the determination result. In the example of FIG. 9, the abnormal block c3 is highlighted. In this manner, the display control unit 150 may display abnormal blocks, which are blocks in the captured image that have been determined to contain a foreign object, in a different display mode from the other blocks. Note that the method of displaying abnormal blocks is not limited to this example. For example, the display control unit 150 may add color to the abnormal blocks or cause the abnormal blocks to blink. In the example of FIG. 9, each block is assigned a number indicating a cluster. For example, blocks with block numbers b3, d3, e3, and f3 are indicated as belonging to the third cluster. Furthermore, a block with block number c3 is indicated as belonging to the fourth cluster. In this manner, the display control unit 150 may display information indicating the cluster to which each block belongs.

[0064] The display control unit 150 may also display information indicating the location of the photographed location. For example, the display control unit 150 may display information indicating which area of ​​the photographed range (i.e., the entire runway) corresponds to the location on the road shown in the photographed image. FIG. 10 is a diagram illustrating an example of output information including the photographed range. In the example of FIG. 10, a map including the photographed runway is displayed. The map may be an image photographed by an artificial satellite. The output information in FIG. 10 includes a rectangle superimposed on the map along with the sentence, "A foreign object was detected at the following location." The rectangle indicates the photographed location of the photographed image where the foreign object was detected. In other words, the display control unit 150 may indicate the location on the road within the photographed range where the foreign object was detected. In this way, the display control unit 150 may superimpose information indicating the area of ​​the photographed location corresponding to the photographed image including the abnormal block on map information indicating the photographed range of the photographing device 200 that captured the photographed image.

[0065] Furthermore, the display control unit 150 may display a captured image corresponding to the shooting location by accepting input of the shooting location. Specifically, the user inputs the shooting location on the terminal device 300. Then, the display control unit 150 displays the captured image corresponding to the input shooting location on the terminal device 300. For example, assume that the output information of FIG. 10 is displayed on the terminal device 300. On the terminal device 300, the user selects a rectangle shown in the output information. At this time, the display control unit 150 displays the captured image of the shooting location corresponding to the selected rectangle on the terminal device 300. For example, the display control unit 150 may display a captured image as shown in FIG. 9. In this way, when an area of ​​the shooting location is selected, the display control unit 150 may display the captured image corresponding to the selected shooting location.

[0066] [Example of Operation of Object Detection Device 100] Next, an example of operation of the object detection device 100 will be described with reference to FIG.

[0067] 11 is a second flowchart illustrating an example of the operation of the object detection device 100. In this operation example, when an airport runway is photographed by the imaging device 200 and multiple photographed images are generated, the presence or absence of a foreign object is determined for each of the photographed images.

[0068] The acquisition unit 110 acquires captured images (S101). At this time, the acquisition unit 110 may sequentially acquire the captured images from the imaging device 200 as soon as the imaging device 200 has captured an image. Alternatively, the acquisition unit 110 may acquire multiple captured images from the imaging device 200 after the imaging device 200 has completed capturing an image. Alternatively, the captured images may be stored in a device (not shown) that can communicate with the imaging device 200 and the object detection device 100. The acquisition unit 110 may acquire the captured images from the device. The acquisition unit 110 may associate the acquired captured images with detailed information about the captured images and store them in the storage device 190.

[0069] The dividing unit 120 divides the captured image into a plurality of blocks (S102). For example, the dividing unit 120 divides one of the acquired plurality of captured images into a plurality of blocks. At this time, the dividing unit 120 may set the block size according to the distance between the position of the image capture device 200 and the capture location. In this case, the capture location is a partial area of ​​the runway shown in the captured image. Furthermore, the dividing unit 120 may set the block size using information on at least one of the zoom level of the image capture device 200 and the size of the detection target.

[0070] The extraction unit 1301 extracts features from each divided block (S103). At this time, the extraction unit 1301 may store information in a storage device (not shown) that associates the extracted features with identification information of the block and identification information of the captured image.

[0071] The similarity calculation unit 1302 calculates the similarity between each block (S104). The classification unit 1401 performs clustering on the blocks based on the similarity (S105). Then, the foreign substance determination unit 1402 determines whether or not a foreign substance is present (S106). For example, the foreign substance determination unit 1402 identifies a cluster that includes fewer than a predetermined number of blocks from among the classified clusters. Then, the foreign substance determination unit 1402 may determine that a foreign substance is present in a block included in the identified cluster.

[0072] If the determination has not been performed for all captured images ("No" in S107), the object detection device 100 returns to the process of S102. That is, the object detection device 100 performs the processes of S102 to S106 for the captured images for which the determination has not been performed.

[0073] When the determination has been made for all the photographed images ("Yes" in S107), the display control unit 150 outputs the determination result (S108). For example, the display control unit 150 causes the terminal device 300 to display output information such as that shown in Fig. 10. At this time, the display control unit 150 may display photographed images corresponding to the photographed locations in response to an input operation of the photographed locations by the user.

[0074] This operation example is merely an example and is not limited to the above example. For example, if multiple captured images are acquired in the process of S101, the process of S102 may be performed on each of the multiple captured images. If the answer to S107 is "No," the object detection device 100 may perform the processes of S103 to S106 on the other captured images for which no judgment has been performed.

[0075] In this way, the object detection device 100 of the second embodiment acquires a captured image of a road and divides the captured image into multiple blocks. The object detection device 100 then calculates the similarity between each block and determines that a block contains a foreign object if the number of other blocks deemed similar based on the similarity is less than a predetermined number. This allows the object detection device 100 to detect foreign objects, regardless of the specific object.

[0076] Furthermore, the object detection device 100 may change the block size, which is the size of a block, depending on the distance between the imaging device 200 that captures the image and the imaging location. This allows the object detection device 100 to set an appropriate block size that takes into account the distance between the imaging device 200 and the imaging location.

[0077] [Modification 1] The object detection device 100 may calculate an abnormality degree according to the similarity degree. The abnormality degree is an index indicating the degree to which the characteristics of an abnormal block differ from those of other blocks or other clusters.

[0078] Specifically, the foreign object determination unit 1402 calculates the degree of abnormality based on the degree of similarity. In this modification, the closer the distance between the feature amounts of the blocks, the larger the degree of similarity. For example, the foreign object determination unit 1402 identifies a block in another cluster that is closest to the abnormal block. In other words, the foreign object determination unit 1402 identifies the block that has the highest degree of similarity to the abnormal block among blocks in another cluster other than the cluster that includes the abnormal block. The other cluster is a cluster that is dissimilar to the abnormal block. In other words, the foreign object determination unit 1402 identifies the block that is most similar to the abnormal block among blocks in a cluster that is deemed dissimilar to the abnormal block.

[0079] The foreign object determination unit 1402 then calculates the degree of abnormality based on whether the similarity between the abnormal block and the block identified as most similar to the abnormal block exceeds a predetermined value. For example, if the similarity is equal to or greater than a predetermined value, the foreign object determination unit 1402 calculates the degree of abnormality as "low." Furthermore, if the similarity is less than a predetermined value, the abnormal block and the block identified as most similar to the abnormal block are significantly different from each other compared to when the similarity is equal to or greater than the predetermined value. Therefore, the foreign object determination unit 1402 calculates the degree of abnormality as "high." This allows the object detection device 100 to indicate the distance between the abnormal block and other blocks. Note that the method for calculating the degree of abnormality is not limited to this example. For example, the degree of abnormality may be expressed as a value in three or more stages, or may be a numerical value in which the closer the abnormal block is to other blocks, the smaller the value.

[0080] In this way, the foreign substance determination unit 1402 may identify a block that has the closest distance between its feature and the abnormal block among the blocks that are deemed dissimilar to the abnormal block, and may calculate the degree of abnormality according to the degree of similarity between the abnormal block and the identified block.

[0081] The display control unit 150 may cause the terminal device 300 to display information indicating the calculated abnormality level. For example, when displaying the information indicating the determination result of Fig. 9, the display control unit 150 may add text, color, or symbol corresponding to the abnormality level to the abnormal block. In this way, the display control unit 150 may change the display mode of the abnormal block depending on the abnormality level.

[0082] [Modification 2] The object detection device 100 may set a block for each area in the captured image.

[0083] Fig. 12 is a diagram showing an example of a captured image. More specifically, the example of Fig. 12 shows a captured image of a road sandwiched between two lawns. A guardrail is present on one side of the lawn.

[0084] The dividing unit 120 may set regions within the captured image. Specifically, the dividing unit 120 may set regions within the captured image by identifying objects within the captured image. For example, the dividing unit 120 may identify a road, a lawn, and a guardrail within the captured image of FIG. 12 . At this time, the dividing unit 120 may use a segmentation technique. Then, the dividing unit 120 sets a road region, a lawn region, and a guardrail region within the captured image. In this way, the dividing unit 120 may identify objects within the captured image.

[0085] Then, the dividing unit 120 sets blocks for the set regions. At this time, the dividing unit 120 may set a block size for each region. For example, suppose there is a need to detect foreign objects of about several centimeters in size, such as machine parts, on roads. In contrast, suppose there is a need to detect larger foreign objects, such as animals, on grass. In this case, the dividing unit 120 may set blocks with a smaller block size for road regions than for grass regions. In this way, the dividing unit 120 may divide each region of the identified subject into blocks with a block size set for each region.

[0086] The smaller the block size, the higher the possibility of detecting a small object, but the larger the number of blocks. A large number of blocks increases the processing load in the similarity calculation process and the process of determining whether or not a foreign object is present. On the other hand, a large block size reduces the processing load because the number of blocks is small. As described above, by setting a block size for each area of ​​a subject identified in a captured image, the object detection device 100 can detect a desired object while suppressing an increase in the processing load.

[0087] Furthermore, the dividing unit 120 does not need to set blocks for all areas of the identified subject. For example, the dividing unit 120 may set blocks only in road areas. FIG. 13 is a diagram showing an example of blocks. More specifically, FIG. 13 is a diagram showing an example in which blocks are set for the captured image of FIG. 12. In the example of FIG. 13, blocks are set by dividing the road area in the captured image. The dividing unit 120 may set blocks according to the areas of the identified subject. By setting blocks in this manner, the determination of the presence or absence of a foreign object is performed only for the road area. This allows the object detection device 100 to reduce the processing load compared to uniformly dividing the entire captured image into blocks, for example, when there is a need to determine the presence or absence of a foreign object only in the road area.

[0088] [Modification 3] The object detection device 100 may process the captured image depending on the capturing environment.

[0089] The shooting environment is the weather, time, etc. when the image is taken. For example, if it is raining or at night, the captured image may be dark. Therefore, the dividing unit 120 may sharpen the captured image.

[0090] Furthermore, when it is snowing, snow may be scattered throughout the captured image, making it difficult to make an appropriate determination. Therefore, when it is snowing, the extraction unit 1301 may extract features using areas of the block other than the areas of white pixels.

[0091] The object detection device 100 acquires environmental information indicating the shooting environment in advance. The environmental information may be acquired, for example, from an external device. For example, the acquisition unit 110 may acquire weather information as environmental information from a server that manages weather information. Alternatively, the acquisition unit 110 may acquire environmental information input by a user to the terminal device 300. Alternatively, the object detection device 100 may determine the weather from the captured image. In this case, the object detection device 100 may determine the weather of the captured image using a learning model based on training data including the captured image and weather information as ground truth data.

[0092] In this way, the acquisition unit 110 may acquire environmental information including at least one of the weather and the time when the photographed image was taken. At this time, if the environmental information satisfies a predetermined condition, the division unit 120 may perform a sharpening process on the photographed image.

[0093] [Modification 4] The object detection device 100 may change the block size depending on the road conditions.

[0094] Assume that the road is a runway. For example, immediately after an airplane takes off, the division unit 120 may divide the captured image showing the runway into blocks with a smaller block size than before takeoff. This allows the object detection device 100 to check in detail whether any parts have fallen after takeoff. On the other hand, immediately before takeoff, the division unit 120 may divide the captured image into blocks with a larger block size than after takeoff. Increasing the block size reduces the processing load until determining the presence or absence of a foreign object. In other words, the processing time until determining the presence or absence of a foreign object is shortened. If the processing time is long, there is a risk that the processing will not be completed before the airplane takes off. In this case, there is a possibility that the takeoff time of the airplane will be affected. Therefore, by dividing the captured image into blocks with a larger block size than after takeoff, the object detection device 100 can reduce the possibility of affecting the takeoff of the airplane.

[0095] Furthermore, an airport may have multiple runways. The object detection device 100 may detect foreign objects for each runway. In this case, the object detection device 100 may divide the captured image showing the runway into blocks of different block sizes for each runway.

[0096] For example, the distances of the runways may differ. If the captured images of each runway are processed using the same block size, the processing time for a runway with a longer distance may be longer than for other runways. Therefore, the dividing unit 120 may divide the captured images of a runway with a longer distance into blocks with a larger block size than for the captured images of the other runways.

[0097] [Modification 5] The object detection device 100 may set the block size according to the processing time for a plurality of captured images.

[0098] For example, suppose there are multiple captured images of a traffic road that is the subject of a foreign object detection. In this case, the dividing unit 120 calculates the block size for each captured image using, for example, Equation 1 or Equation 2. The dividing unit 120 also calculates the number of divisions (i.e., the number of blocks) according to the block size for each captured image. The dividing unit 120 then calculates the processing time required to determine the presence or absence of a foreign object for each captured image from the processing time per unit block (e.g., one block) and the number of divisions.

[0099] For example, the dividing unit 120 can calculate the processing time for the n-th captured image using the following equation 3.

[0100] (Equation 3) Processing time of captured image n = θ Ｓｎ ×D Ｓｎ S is the block size. Ｓ indicates the number of divisions of the captured image when the block size is S. Ｓ indicates the processing time per block for block size S.

[0101] The dividing unit 120 calculates the processing time for each of the plurality of captured images, and if the total processing time exceeds the target time, increases the block size of the blocks in at least one of the captured images.

[0102] In this way, the dividing unit 120 may calculate the processing time for each captured image based on the number of blocks corresponding to the block size indicating the size of the block and the processing time required for the determination process per unit block. Then, the dividing unit 120 may set the block size for each captured image so that the total processing time for each captured image is less than the target time.

[0103] At this time, the dividing unit 120 may set the block size by solving an optimization problem that finds a solution that makes the block size of the blocks in each captured image as small as possible, within a range where the total processing time does not exceed a target time.

[0104] In this way, the division unit 120 may calculate the block size of each captured image by solving an optimization problem that calculates a solution that minimizes the block size for each captured image, within a range where the total processing time for each captured image does not exceed the target time.

[0105] Third Embodiment Next, an object detection device according to a third embodiment will be described. In the third embodiment, an example of responding to various inputs by a user will be described. Note that some of the content overlapping with the first and second embodiments will not be described.

[0106] Also, in this embodiment, an example in which the object detection device detects foreign objects on the road at an airport will be mainly described.

[0107] Fig. 14 is a block diagram showing an example of the functional configuration of an object detection system 1001. As shown in Fig. 14, the object detection system 1001 includes an object detection device 101. In the example of Fig. 14, the object detection system 1001 further includes an image capture device 200 and a terminal device 300. The object detection device 101 is communicably connected to the image capture device 200 and the terminal device 300 via a wireless or wired network.

[0108] The object detection device 101 includes an acquisition unit 110, a division unit 120, a calculation unit 130, a determination unit 140, a display control unit 150, and an information registration unit 160. Furthermore, in the example of FIG.

[0109] The information registration unit 160 registers information indicating the determination result in the database. For example, suppose that the entire runway is taken as the shooting range and multiple photographed images corresponding to the runway are acquired. Then, suppose that the determination unit 140 performs a determination process on the multiple photographed images. At this time, the information registration unit 160 registers information indicating the determination result for the multiple photographed images in the database as inspection information.

[0110] FIG. 15 is a diagram showing an example of inspection information. The example of FIG. 15 shows information indicating the determination results of a single inspection. Here, inspection refers to determining the presence or absence of a foreign object in a target road area. For example, if the entire runway is the target for determining the presence or absence of a foreign object, performing the determination process on multiple photographed images corresponding to the entire runway is referred to as one inspection. The example of FIG. 15 shows items such as photographed image, photographed location, photographed time, abnormal block, and status. The "Photographed Image" item shows identification information of the photographed image. The "Photographed Location" item shows the location shown in the photographed image. The "Photographed Time" item shows the time the photographed image was taken. The "Abnormal Block" item shows the presence or absence of an abnormal block. The "Status" item shows whether a foreign object has been removed. In other words, the inspection information includes information associated with information indicating the photographed location, photographed time, the presence or absence of an abnormal block, and status for each photographed image in a single inspection. For example, the captured image "001" is an image captured at point A, and indicates that the capture time is 17:40:03 on June 30, 2024. Furthermore, the captured image "001" does not contain any abnormal blocks. Therefore, no information is displayed in the status. The inspection information may be stored in the storage device 190.

[0111] In this way, the information registration unit 160 registers the result of the determination on the photographed image as examination information. The information registration unit 160 is an example of an information registration means.

[0112] The display control unit 150 may display output information including information indicated in the inspection information. FIG. 16 is a diagram showing an example of the output information. For example, the display control unit 150 causes the terminal device 300 to display output information such as that shown in FIG. 16. In the example of FIG. 16, information indicating the determination result including the captured image, the time of capture, the location of the image, the number of abnormal blocks, and the status are displayed. In this example, it is shown that one abnormal block has been detected. Furthermore, the status is displayed as "not addressed." In other words, it is shown that no action, such as removal, has been taken against the detected foreign object.

[0113] For example, suppose that the display control unit 150 displays output information such as that shown in Fig. 10 on the terminal device 300. In this case, when the display control unit 150 receives a selection of an area within the shooting range from the user, the display control unit 150 may display output information such as that shown in Fig. 16.

[0114] Furthermore, the information registration unit 160 may update the inspection information using information acquired from the terminal device 300. For example, assume that a maintenance worker near a runway removes a detected foreign object. At this time, the object detection device 100 acquires status information indicating the terminal device 300's response status regarding the foreign object. The information registration unit 160 may update the inspection information in accordance with the acquired status information.

[0115] For example, a maintenance worker checks information indicating the determination result through the terminal device 300. If a foreign object is detected, the maintenance worker heads to the location on the runway where the foreign object is located. The maintenance worker then removes the foreign object. The maintenance worker inputs situation information into the terminal device 300. At this time, the terminal device 300 carried by the maintenance worker may be a portable terminal such as a smartphone.

[0116] The display control unit 150 may cause the terminal device 300 to display an input form for inputting status information. For example, assume that the output information shown in the example of FIG. 16 is displayed. A maintenance worker, who is an example of a user, selects the "Update" button in FIG. 16. At this time, the display control unit 150 may cause the terminal device 300 to display an input form. FIG. 17 is a diagram showing an example of the input form.

[0117] FIG. 17 shows the block number of the abnormal block and the location where the foreign object is believed to exist. In this example, there are input fields for whether or not a maintenance person has responded to the foreign object, who responded, an image, and a comment. In the "Response" field, there are radio buttons for inputting whether or not a maintenance person has responded. For example, if a maintenance person has removed the foreign object, the maintenance person selects the "Completed" radio button. In the "Response" field, for example, information identifying the person who removed the foreign object or the person entering information into the input form is entered. In the "Comment" field, a comment is entered.

[0118] An image is input in the "Image" field. For example, a maintenance worker carries the terminal device 300 and heads to a location on the runway where a foreign object is present. At this time, the terminal device 300 takes an image in response to the maintenance worker's operation. Specifically, the terminal device 300 may take an image of the detected foreign object. Alternatively, the terminal device 300 may take an image of the road surface after the foreign object has been removed. That is, an image taken by the terminal device 300 may be input in the "Image" field. The image of the detected foreign object and the image of the road surface after the foreign object has been removed can be used as confirmation that the foreign object has been removed.

[0119] When the terminal device 300 accepts input of situation information through a user operation, it transmits the situation information to the object detection device 101. The information registration unit 160 updates the inspection information based on the transmitted situation information. For example, the information registration unit 160 updates the status of the photographed image of point X to "dealt with." Furthermore, the information registration unit 160 may register information about the photographed image of point X in association with information such as the responder, image, and comment included in the situation information.

[0120] Furthermore, the display control unit 150 may display output information based on updated inspection information. FIG. 18 is a diagram showing an example of output information. For example, the display control unit 150 causes the terminal device 300 to display output information such as that shown in FIG. 18. In the example of FIG. 18, output information different from the output information of FIG. 16 is displayed, except for the status. At this time, the status has been updated to "addressed." In addition, the example of FIG. 18 also shows status information. The status information shows information indicating the time of shooting, the location of the shooting, the block determined to contain a foreign object, the person who addressed the issue, and comments, along with the image transmitted from the terminal device 300. In this way, the display control unit 150 may display information based on the input image.

[0121] [Example of Operation of Object Detection Device 101] Next, an example of operation of the object detection device 101 will be described with reference to FIG.

[0122] 19 is a third flowchart illustrating an example of the operation of the object detection device 101. In this operation example, when an airport runway is photographed by the imaging device 200 and a plurality of photographed images are generated, the presence or absence of a foreign object is determined for each of the photographed images.

[0123] The processes of S201 to S207 are the same as those of S101 to S107, and therefore will not be described again. When a determination has been made for all captured images, the information registration unit 160 registers the determination results as examination information (S208). Then, the display control unit 150 outputs the determination results (S209). At this time, for example, the display control unit 150 causes the terminal device 300 to display output information such as that shown in FIG. 16 .

[0124] If there is a request for update from the user ("Yes" in S210), the object detection device 101 performs the processes from S211 onwards. The request for update may be information transmitted from the terminal device 300. For example, when output information such as that shown in FIG. 16 is displayed on the terminal device 300, the object detection device 101 may determine that the user has selected the "Update" button as a request for update from the user.

[0125] The display control unit 150 displays an input form on the terminal device 300 (S211). For example, the display control unit 150 displays the input form shown in the example of FIG. 17. Then, when the terminal device 300 receives input from the user into the input form, it transmits situation information to the object detection device 101. The acquisition unit 110 acquires the transmitted situation information (S212).

[0126] The information registration unit 160 updates the examination information based on the acquired status information (S213). Then, the display control unit 150 displays output information including the updated examination information on the terminal device 300 (S214). For example, the display control unit 150 displays output information such as that shown in FIG. 18.

[0127] In addition, in the process of S210, if there is no request for updating ("No" in S210), the object detection device 101 may end the process.

[0128] This operation example is merely an example, and is not limited to the above example.

[0129] In this way, the object detection device 101 of the third embodiment also achieves the same effects as the first and second embodiments.

[0130] Furthermore, the object detection device 101 may acquire status information indicating the status of response to a foreign object from the terminal device 300 operated by the user. Then, the object detection device 101 may associate the status information with a captured image including a block determined to contain a foreign object, and register the information as inspection information.

[0131] This allows the object detection device 101 to reflect the response status regarding the foreign object in the system when a foreign object is detected.

[0132] [Variation 6] The object detection device of the present disclosure can be applied in various situations. For example, the object detection device can be applied to situations where foreign objects on a road are detected. The object detection device can also be applied to situations where objects are detected in terrain with uniform areas, such as grasslands, deserts, snowy plains, oceans, and lakes.

[0133] The object detection device of the present disclosure also acquires captured images. The captured images may be images captured by a device mounted on a mobile object such as an aircraft or a vehicle. For example, the captured images may be moving images captured while the mobile object is moving. In this case, the object detection device may acquire the moving images and location information from the mobile object. The location information may be information measured by a positioning device mounted on the mobile object. The positioning device may be a device capable of receiving signals transmitted from positioning satellites of the Global Navigation Satellite System (GNSS), such as Global Positioning System (GPS) satellites, and acquiring location information indicating the location of the object based on the received signals.

[0134] <Example of Hardware Configuration of Object Detection Device> The hardware constituting the object detection device of the first, second, and third embodiments described above will be described. Fig. 20 is a block diagram showing an example of the hardware configuration of a computer device constituting the object detection device in each embodiment. The object detection device and object detection method described in each embodiment and each modified example are realized in a computer device 90. For example, the object detection device described in each embodiment and each modified example may have the hardware configuration shown in Fig. 20.

[0135] 20, a computer device 90 includes a processor 91, a RAM (Random Access Memory) 92, a ROM (Read Only Memory) 93, a storage device 94, an input / output interface 95, a bus 96, and a drive device 97. Note that the object detection device and the like may be realized by a plurality of electric circuits.

[0136] The storage device 94 stores a program (computer program) 98. The processor 91 executes the program 98 of the object detection device using the RAM 92. Specifically, the program 98 includes, for example, a program that causes a computer to execute processes shown in Figures 3, 11, 19, etc. The functions of each component of the object detection device are realized in response to the processor 91 executing the program 98. The program 98 may be stored in the ROM 93. Alternatively, the program 98 may be recorded on the recording medium 80 and read out using the drive device 97, or may be transmitted to the computer device 90 from an external device (not shown) via a network (not shown).

[0137] The input / output interface 95 exchanges data with peripheral devices (such as a keyboard, a mouse, and a display device) 99. The input / output interface 95 functions as a means for acquiring or outputting data. The bus 96 connects each component.

[0138] There are various variations in the implementation of the object detection device. For example, each component included in the object detection device can be implemented as a dedicated device. Furthermore, the object detection device can be implemented based on a combination of multiple devices.

[0139] The scope of each embodiment also includes a processing method for non-temporarily recording a program for realizing each configuration of the function of each embodiment on a recording medium, reading the program recorded on the recording medium as code, and executing it on a computer. In other words, a computer-readable non-temporary recording medium is also included in the scope of each embodiment. Furthermore, a recording medium on which the above-mentioned program is recorded and the program itself are also included in each embodiment.

[0140] The recording medium may be, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a compact disc (CD)-ROM, a magnetic tape, a non-volatile memory card, or a ROM, but is not limited to these examples. Furthermore, the program recorded on the recording medium is not limited to a program that executes processing on its own, but also includes a program that runs on an operating system (OS) and executes processing in cooperation with other software or functions of an expansion board, within the scope of each embodiment.

[0141] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0142] Furthermore, the above-described embodiments and modifications can be combined as appropriate.

[0143] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0144] <Additional Notes>

[0145] [Supplementary Note 1] An object detection device comprising: an acquisition means for acquiring an image of a road; a division means for dividing the image into a plurality of blocks; a calculation means for calculating a similarity between each block; and a determination means for determining that a block in which the number of other blocks deemed to be similar based on the similarity is less than a predetermined number is a block in which a foreign object is captured.

[0146] [Supplementary Note 2] The object detection device according to Supplementary Note 1, wherein the dividing unit sets a block size, which is a size of a block, depending on a distance between an image capturing device that captures the captured image and an image capturing location.

[0147] [Supplementary Note 3] The object detection device described in Supplementary Note 1, wherein the acquisition means acquires a plurality of the captured images, and the division means calculates a processing time for each of the captured images based on the number of blocks corresponding to a block size indicating the size of the block and the processing time required for the determination process per unit block, and sets the block size for each of the captured images so that the total processing time for each of the captured images is less than a target time.

[0148] [Supplementary Note 4] The object detection device according to Supplementary Note 3, wherein the dividing means calculates a block size for each of the captured images by solving an optimization problem that calculates a solution that minimizes a block size for each of the captured images within a range in which a total processing time for each of the captured images does not exceed a target time.

[0149] [Supplementary Note 5] The object detection device according to Supplementary Note 1, wherein the acquisition means acquires environmental information including at least one of weather and time when the photographed image was taken, and the division means performs a sharpening process on the photographed image if the environmental information satisfies a predetermined condition.

[0150] [Supplementary Note 6] The object detection device according to Supplementary Note 1, wherein the dividing means identifies a subject in the captured image, and divides a predetermined area in each area of ​​the identified subject into blocks.

[0151] [Supplementary Note 7] The object detection device according to Supplementary Note 1, wherein the dividing means identifies a subject in the captured image, and divides each area of ​​the identified subject into blocks having a block size set for each area.

[0152] [Supplementary Note 8] An object detection device as described in Supplementary Note 1, further comprising a display control means for displaying an abnormal block, which is a block determined to contain a foreign object, in a display manner different from that of other blocks, wherein the determination means identifies a block that is closest to the abnormal block among blocks that are deemed not to be similar to the abnormal block, calculates a degree of abnormality according to the degree of similarity between the abnormal block and the identified block, and the display control means changes the display manner of the abnormal block according to the degree of abnormality.

[0153] [Supplementary Note 9] The object detection device according to Supplementary Note 1, further comprising: an information registration means for registering the result of the judgment on the photographed image as inspection information; wherein the acquisition means acquires status information indicating a response status regarding the foreign object from a terminal device operated by a user; and the information registration means associates the status information with the photographed image including a block determined to contain the foreign object, and registers the status information as the inspection information.

[0154] [Supplementary Note 10] The object detection device according to Supplementary Note 2, wherein the dividing means sets a block size according to the distance, a zoom level of the image capturing device, and a size of the detection target.

[0155] [Supplementary Note 11] The object detection device according to Supplementary Note 1, further comprising a display control means for displaying an abnormal block, which is a block determined to contain a foreign object, in a display mode different from that of other blocks.

[0156] [Supplementary Note 12] The object detection device according to Supplementary Note 11, wherein the display control means displays information indicating an area of ​​a photographing location corresponding to the photographed image including the abnormal block superimposed on map information indicating a photographing range of a photographing device that captured the photographed image.

[0157] [Supplementary Note 13] The object detection device according to Supplementary Note 12, wherein when an area of ​​the photographing location is selected, the display control means displays the photographed image corresponding to the selected photographing location.

[0158] [Supplementary Note 14] The object detection device according to Supplementary Note 1, wherein the calculation means extracts a feature from each of the plurality of blocks, and calculates a distance between each of the blocks based on the extracted feature as the similarity.

[0159] [Supplementary Note 15] The object detection device according to Supplementary Note 14, wherein the calculation means performs a smoothing process on the captured image and extracts a feature amount from each block of the captured image that has been subjected to the smoothing process.

[0160] [Supplementary Note 16] An object detection method comprising: acquiring an image of a road; dividing the image into a plurality of blocks; calculating a similarity between each block; and determining a block in which the number of other blocks deemed similar based on the similarity is less than a predetermined number as a block in which a foreign object is captured.

[0161] [Supplementary Note 17] A recording medium that non-temporarily records a program that causes a computer to execute the following processes: acquiring an image of a road; dividing the image into a plurality of blocks; calculating the similarity between each block; and determining that a block in which the number of other blocks that are deemed similar based on the similarity is less than a predetermined number is a block in which a foreign object is captured.

[0162] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 14, which are dependent on Supplementary Note 1, may also be dependent on Supplementary Notes 15 and 16 in the same dependent relationship as Supplementary Notes 2 to 14. Furthermore, within the scope of each of the above-described embodiments, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems.

[0163] REFERENCE SIGNS LIST 100, 101 Object detection device 110 Acquisition unit 120 Division unit 130 Calculation unit 140 Determination unit 150 Display control unit 160 Information registration unit 190 Storage device 200 Imaging device 300 Terminal device 1000, 1001 Object detection system

Claims

1. An object detection device comprising: an acquisition means for acquiring an image of a road; a division means for dividing the image into a plurality of blocks; a calculation means for calculating the similarity between each block; and a determination means for determining that a block in which the number of other blocks deemed to be similar based on the similarity is less than a predetermined number is a block in which a foreign object is captured.

2. The object detection device according to claim 1, wherein the dividing means sets a block size, which is the size of a block, according to the distance between the photographing device that photographs the photographed image and the photographed location.

3. The object detection device of claim 1, wherein the acquisition means acquires a plurality of the captured images, and the division means calculates a processing time for each of the captured images based on the number of blocks corresponding to a block size indicating the size of the block and the processing time required for the determination process per unit block, and sets the block size for each of the captured images so that the total processing time for each of the captured images is less than a target time.

4. The object detection device according to claim 3, wherein the dividing means calculates the block size of each of the captured images by solving an optimization problem that calculates a solution that minimizes the block size for each of the captured images, within a range where the total processing time for each of the captured images does not exceed a target time.

5. The object detection device according to claim 1, wherein the acquisition means acquires environmental information including at least one of the weather and the time when the photographed image was taken, and the division means performs a sharpening process on the photographed image if the environmental information satisfies a predetermined condition.

6. The object detection device according to claim 1, wherein the dividing means identifies the subject in the captured image and divides a predetermined area in each area of ​​the identified subject into blocks.

7. The object detection device according to claim 1, wherein the dividing means identifies the subject in the captured image and divides each area of ​​the identified subject into blocks of a block size set for each area.

8. An object detection device as described in claim 1, further comprising a display control means for displaying abnormal blocks, which are blocks determined to contain foreign objects, in a different display mode from other blocks, the similarity being the distance between features extracted from each block, the determination means identifying the block that is closest in distance to the abnormal block between features among the blocks determined to be dissimilar to the abnormal block, calculating the degree of abnormality according to the similarity between the abnormal block and the identified block, and the display control means changing the display mode of the abnormal block according to the degree of abnormality.

9. An object detection device as described in claim 1, further comprising an information registration means for registering the result of the judgment on the captured image as inspection information, wherein the acquisition means acquires status information indicating the response status regarding the foreign object from a terminal device operated by a user, and the information registration means associates the status information with the captured image including a block that has been judged to contain the foreign object, and registers it as the inspection information.

10. The object detection device according to claim 2, wherein the dividing means sets a block size according to the distance, the zoom level of the image capture device, and the size of the detection target.

11. The object detection device according to claim 1, further comprising a display control means for displaying an abnormal block, which is a block determined to contain a foreign object, in a display mode different from that of other blocks.

12. The object detection device according to claim 11, wherein the display control means displays map information indicating the photographing range of the photographing device that captured the photographed image, superimposed with information indicating the area of ​​the photographed location corresponding to the photographed image that includes the abnormal block.

13. The object detection device according to claim 12, wherein when an area of ​​the photographing location is selected, the display control means displays the photographed image corresponding to the selected photographing location.

14. The object detection device according to claim 1, wherein the calculation means extracts features from each of the plurality of blocks, and calculates the distance between each block based on the extracted features as the similarity.

15. The object detection device according to claim 14, wherein the calculation means performs a smoothing process on the captured image and extracts a feature amount from each block of the captured image that has been subjected to the smoothing process.

16. An object detection method comprising: acquiring an image of a road; dividing the image into a plurality of blocks; calculating the similarity between each block; and determining a block in which the number of other blocks deemed similar based on the similarity is less than a predetermined number as a block in which a foreign object is captured.

17. A recording medium that non-temporarily records a program that causes a computer to execute the following processes: acquiring an image of a road; dividing the image into multiple blocks; calculating the similarity between each block; and determining that a block in which the number of other blocks deemed similar based on the similarity is less than a predetermined number is a block in which a foreign object is captured.

Images