Information processing device, information processing method, and program

Abstract

To accurately determine whether or not a field of view of an imaging device is blocked in accordance with a situation of imaging.SOLUTION: An information processing device divides a captured image captured by an imaging unit into a plurality of blocks, extracts respective feature values of the plurality of blocks, determines a block to be compared with a reference image from the plurality of blocks in the captured image on the basis of the extracted feature values, calculates a similarity between the determined block from the plurality of blocks in the captured image and the corresponding block in the reference image, and determines whether or not a field of view of the imaging unit is blocked on the basis of the result of calculating the similarity.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, a technique is known in which an image captured by an imaging device is subjected to image analysis to extract features of the captured image, and a state in which the field of view of the imaging device is obstructed (for example, a state associated with a tampering act) is detected based on a change in the features. In such detection, there are problems with detection failures, which are when the field of view of the imaging device is obstructed but the state is not detected, and with false detections, which are when the field of view is not obstructed but the state is detected as obstructed. In order to solve such problems, a technique has been proposed that reduces the occurrence of detection failures and false detections and improves detection accuracy.

[0003] Patent Document 1 discloses a method of dividing a captured image into a plurality of blocks, comparing the feature amount of each block with a threshold value to determine whether each block is an obstructed block, and determining whether the field of view of an imaging device is obstructed according to the ratio of obstructed blocks in the captured image. Patent Document 2 discloses a method of switching between detecting whether the field of view of an imaging device is obstructed based on the difference between a captured image and a reference image or comparing the feature amount of the captured image with a threshold value for each of a plurality of blocks divided from the captured image. Patent Document 3 discloses a method of converting a captured image into an edge image, generating a similarity value between the edge image and a reference edge image, and determining whether the field of view of an imaging device has been tampered with based on the similarity value. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-182682 A [Patent Document 2] Patent Publication No. 2021-182216 [Patent Document 3] JP 2008-167429 A Summary of the Invention [Problem to be solved by the invention]

[0005] In a method for determining whether the field of view of an imaging device is obstructed based on a comparison between the feature amount of each block of a captured image and a threshold value as disclosed in Patent Document 1, the relationship between missed detection and false detection may be in a trade-off relationship depending on the level of the threshold value. Also, in a method for determining whether the field of view of an imaging device is obstructed based on a comparison between a captured image and a reference image as disclosed in Patent Documents 2 and 3, the accuracy of the determination is easily affected by environmental changes such as brightness and color temperature.

[0006] The present invention has been made in consideration of the above problems, and has an object to determine whether the field of view of an imaging device is obstructed with higher accuracy depending on the imaging situation. [Means for solving the problem]

[0007] In order to achieve the above object, an information processing device according to one aspect of the present invention has the following configuration: a division means for dividing an image captured by an imaging unit into a plurality of blocks, an extraction means for extracting a feature value of each of the plurality of blocks, a determination means for determining which of the plurality of blocks in the captured image should be compared with a reference image based on the feature value extracted by the extraction means, a calculation means for calculating a similarity between the block determined by the determination means among the plurality of blocks in the captured image and a corresponding block in the reference image, and a determination means for determining whether the field of view of the imaging unit is obstructed based on a result of the calculation of the similarity value by the calculation means. Effect of the Invention

[0008] According to the present invention, it is possible to determine with higher accuracy whether the field of view of an imaging device is obstructed depending on the imaging conditions. [Brief description of the drawings]

[0009] [Figure 1] 1 illustrates an example of the configuration of a monitoring system according to an embodiment. [Diagram 2] 1A shows an example of a functional configuration of an imaging device according to an embodiment, and FIG. 1B shows an example of a hardware configuration of an imaging device according to an embodiment. [Diagram 3] 11 is a flowchart of a process executed by the imaging device to determine whether the field of view of an imaging unit is obstructed. [Figure 4] 11 is a flowchart of a determination process using a reference image according to the first embodiment. [Diagram 5] 1 shows a conceptual diagram of a plurality of captured images captured at different times. [Figure 6] 10 is a flowchart of a determination process using a reference image according to a second embodiment. [Figure 7] 1 shows an example of different reference images corresponding to different time periods. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the embodiment described below is one example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the embodiment described below.

[0011] [First embodiment] First, a first embodiment of the present invention will be described. In this embodiment, the term "image" is understood to include at least one of a still image and a video (moving image). In this embodiment, the term "image" may be understood to be synonymous with "image data."

[0012] (Monitoring system configuration) 1 shows an example of the configuration of a monitoring system 1 according to this embodiment. The monitoring system 1 includes imaging devices 101-1, 101-2, and 101-3 and a management device 102, and the imaging devices 101-1, 101-2, and 101-3 and the management device 102 are connected via a network 103. The network 103 may be, for example, the Internet, a local area network (LAN), a wide area network (WAN), or a public network. The network 103 may be a wireless network or a wired network. Communications between the imaging devices 101-1, 101-2, and 101-3 and the management device 102 may be relayed by another communication device.

[0013] The imaging devices 101-1, 101-2, and 101-3 are surveillance cameras that transmit captured images to the management device 102, and in this embodiment, function as information processing devices that determine whether the field of view of the imaging unit of each imaging device is obstructed based on the captured images. When it is determined that the field of view of the imaging unit is obstructed, the imaging devices 101-1, 101-2, and 101-3 output an alert (warning) to the management device 102. Furthermore, the imaging devices 101-1, 101-2, and 101-3 can receive setting information related to settings of imaging and other functions from the management device 102 and operate based on the setting information.

[0014] The management device 102 manages the imaging devices 101-1, 101-2, and 101-3. The management may be performed by an operator 104 or according to a predetermined setting. For example, the management device 102 generates and transmits setting information related to settings of imaging and other functions to the imaging devices 101-1, 101-2, and 101-3. The management device 102 also receives captured images from the imaging devices 101-1, 101-2, and 101-3 and records them. Furthermore, the management device 102 receives an alert from one or more imaging devices that it has determined to have an obstructed view among the imaging devices 101-1, 101-2, and 101-3.

[0015] When an image management application is installed on the management device 102, the management device 102 may manage the imaging devices 101-1, 101-2, and 101-3 via the application. For example, when the management device 102 has a display unit and the application runs on a web browser displayed on the display unit, the operator 104 may operate the display and recording of images received from the imaging devices 101-1, 101-2, and 101-3 on the web browser. Furthermore, when the management device 102 receives an alert from the imaging devices 101-1, 101-2, and 101-3, the management device 102 may display information indicating the alert on the web browser. The information indicating the alert may be output by sound.

[0016] [Configuration of imaging device] Next, the configurations of the image pickup devices 101-1, 101-2, and 101-3 will be described with reference to Fig. 2. Since the image pickup devices 101-1, 101-2, and 101-3 have the same configuration, they will be collectively referred to as the image pickup device 101 in the following description.

[0017] 2(A) shows an example of the functional configuration of the imaging device 101. As an example of the functional configuration, the imaging device 101 has an imaging unit 201, an image compression unit 202, a formatting unit 203, a communication unit 204, a block division unit 205, an extraction unit 206, a determination unit 207, a calculation unit 208, a judgment unit 209, and a setting unit 210. The imaging unit 201, the image compression unit 202, the formatting unit 203, the communication unit 204, the block division unit 205, the extraction unit 206, the determination unit 207, the calculation unit 208, and the judgment unit 209 configure a functional unit group 211.

[0018] The imaging unit 201 creates an image by converting an analog signal corresponding to light imaged on a light receiving surface via a lens (not shown) that is an optical system, into a digital signal. The image is created according to a predetermined frame rate. In this embodiment, an image created (captured) by the imaging unit 201 is also referred to as a captured image. The image compression unit 202 performs compression encoding on the captured image. The formatting unit 203 converts the image compressed and encoded by the image compression unit 202 into a file format suitable for transmission to the management device 102 via the network 103.

[0019] The block division unit 205 divides the captured image captured by the imaging unit 201 into a plurality of blocks (partial regions). The blocks may be rectangular. The extraction unit 206 extracts a feature amount of each block divided by the block division unit 205. An example of the feature amount is an edge amount (edge ​​power). The extraction unit 206 can extract the feature amount using an edge amount extraction filter for extracting an edge amount, such as a Sobel filter. Other examples of the feature amount may be quantitative values ​​related to an image, such as lightness, saturation, hue, and black area amount.

[0020] The determining unit 207 determines a block to be compared with the reference image from among the multiple blocks divided by the block dividing unit 205, based on the feature amount extracted by the extracting unit 206. The determining unit 207 can also determine that there are zero blocks to be compared with the reference image (i.e., there are no blocks), based on the feature amount. Note that the reference image is assumed to be captured by the imaging device 101 in a state where the subject is not within the field of vision (angle of view) (unobstructed state) and stored in the imaging device 101.

[0021] The calculation unit 208 calculates a similarity between one or more blocks determined by the determination unit 207 among the multiple blocks divided by the block division unit 205 and a corresponding block in the reference image. The determination unit 209 determines whether the field of view of the imaging unit 201 is obstructed based on the calculation result of the similarity by the calculation unit 208. Furthermore, the determination unit 209 can determine that the field of view of the imaging unit 201 is not obstructed when the number of blocks (blocks determined by the determination unit 207) to be compared with the reference image is less than a predetermined threshold. Furthermore, when the determination unit 209 determines that the field of view of the imaging unit 201 is obstructed, it outputs an alert to an external device via the communication unit 204. In this embodiment, the determination unit 209 outputs the alert to the management device 102.

[0022] The setting unit 210 receives setting information from the management device 102 via the communication unit 204, and transmits the setting information to any one of the functional units of the functional unit group 212 based on the content of the setting information. For example, when the setting information is a threshold for decision or judgment processing, the setting unit 210 can transmit information on the threshold to the extraction unit 206, the determination unit 207, the calculation unit 208, or the judgment unit 209. Furthermore, when the setting information is the number of consecutive frames (or duration) that triggers issuing an alert, the setting unit 210 can transmit information on the number of frames (or duration) to the judgment unit 209. The communication unit 204 controls communication with an external device. For example, the communication unit 204 is configured to transmit a captured image and an alert to the management device 102 connected via the network 103, and to receive setting information from the management device 102.

[0023] 2B shows an example of the hardware configuration of the imaging device 101. The imaging device 101 includes, as an example of the hardware configuration, a processor 221, a ROM (Read Only Memory) 222, a RAM (Random Access Memory) 223, a HDD (Hard Disc Drive) 224, a communication I / F (interface) 225, and an image sensor 226.

[0024] The processor 221 is configured by, for example, a CPU (Central Processing Unit) and controls the imaging device 101. The imaging device 101 may include one or more processors 221. The ROM 222 stores programs and the like for the processor 221 to control the imaging device 101. Note that a secondary storage device may be provided instead of the ROM 222. The RAM 223 is a memory for the processor 221 to expand the programs read from the ROM 222 and execute the processes. The RAM 223 may also be used as a storage area for temporarily storing data to be subjected to various processes. The HDD 224 is a non-volatile memory and holds programs and data. The communication I / F 225 is controlled by the communication unit 204 and executes communication control processing with an external device via the network 103. The image sensor 226 is an image sensor for capturing images and is used for processing by the imaging unit 201.

[0025] (Processing flow) A flow of a process of determining whether or not the field of view of the imaging unit 201 is obstructed according to this embodiment will be described with reference to FIG. 3 and FIG. 4. FIG. 3 is a flowchart of a process executed by the imaging device 101 according to this embodiment, and FIG. 4 is a flowchart of a process of S307 in FIG. 3. The process shown in FIG. 3 is performed every time a frame of an image is captured by the imaging unit 201 and input to the block division unit 205. In this example, the determination unit 207 acquires a predetermined first threshold value in advance from the setting unit 211, and the determination unit 209 acquires predetermined second and third threshold values ​​in advance from the setting unit 211. The first to third threshold values ​​may be values ​​equal to or greater than 1. The determination unit 209 sets the count number of consecutive frames for determining whether or not to issue an alert to zero. The determination unit 207 and the determination unit 209 may be configured to acquire threshold values ​​by another method.

[0026] First, the block division unit 205 divides the captured image input from the imaging unit 201 into a plurality of blocks (S301). Next, the extraction unit 206 extracts a feature amount of each of the plurality of blocks (S302). Next, the determination unit 207 determines a block to be compared with the reference image from among the plurality of blocks based on the extracted amount (S303). Specifically, first, the determination unit 207 determines a block whose view is obstructed from among the plurality of blocks based on the feature amount extracted in S302. For example, the determination unit 207 determines a block whose view is obstructed from among the plurality of blocks by comparing the feature amount with a predetermined threshold. For the sake of explanation, a block whose view is obstructed from among the plurality of blocks in S303 is referred to as a primary obstructed block. Then, the determination unit 207 determines the primary obstructed block from among the plurality of blocks as a block to be compared with the reference image.

[0027] Next, the determination unit 207 determines whether the number of primary occlusion blocks is equal to or greater than the first threshold value set by the setting unit 210 (S304). If the number of primary occlusion blocks is less than the first threshold value (No in S304), the determination unit 209 determines that the field of view of the imaging unit 201 is not blocked (S305) and lowers the alert (S306). If the alert was originally lowered, the determination unit 209 maintains that state. On the other hand, if the number of primary occlusion blocks is equal to or greater than the first threshold value (Yes in S304), the imaging device 101 uses the reference image to determine whether the field of view of the imaging unit 201 is blocked (S307). The process of S307 will be described later with reference to FIG. 4. After the processes of S306 and S307, the imaging device 101 returns to S301 to determine whether the field of view of the imaging unit 201 is blocked based on the captured image of the next frame.

[0028] Fig. 4 shows a flowchart of the process of judging whether the field of view of the imaging unit 201 is obstructed using a reference image (S307 in Fig. 3) according to this embodiment. In this process, the calculation unit 208 calculates the similarity between a block (primarily obstructed block) determined by the determination unit 207 to be compared with the reference image and a corresponding block in the reference image, and the judgment unit 209 judges whether the field of view of the imaging unit 201 is obstructed based on the similarity. In this embodiment, the similarity of an edge image is used as the similarity. An edge image is an image obtained by extracting an edge portion (for example, a portion where pixel values ​​change suddenly) in an image.

[0029] First, the calculation unit 208 creates an edge image (target edge image) of the captured image and an edge image (reference edge image) of the reference image (S401). The calculation unit 208 can create the edge image by using, for example, a Sobel filter. Next, the calculation unit 208 calculates the similarity between the target edge image and the reference edge image created in S401 for each primary occlusion block. For example, the calculation unit 208 calculates a correlation value between both images (S402). Next, the determination unit 209 determines a block having a low similarity among the primary occlusion blocks based on the calculation result of the similarity in S402 (S403). In this embodiment, the determination unit 209 determines a block having a similarity less than a predetermined threshold as a block having a low similarity. For the sake of explanation, the block determined to have a low similarity in S403 is referred to as a secondary occlusion block.

[0030] Next, the determination unit 209 determines whether the number of secondary occlusion blocks is equal to or greater than the second threshold set by the setting unit 210 (S404). If the number of secondary occlusion blocks is less than the second threshold (No in S404), the determination unit 209 determines that the field of view of the imaging unit 201 is not blocked (S405) and lowers the alert (S406). If the alert was originally lowered, the determination unit 209 maintains that state. On the other hand, if the number of secondary occlusion blocks is equal to or greater than the second threshold (Yes in S404), the determination unit 209 determines that the captured image is an image in which the field of view of the imaging unit 201 is blocked (S407). For the sake of explanation, a captured image determined in S403 to be blocked is referred to as a blocked image. Then, the determination unit 209 counts up the number of consecutive frames of the blocked image (S407). The determination unit 209 further determines whether the number of consecutive frames of the masked image is equal to or greater than a predetermined third threshold (S408). If the number of consecutive frames of the masked image is equal to or greater than the third threshold (Yes in S408), the determination unit 209 determines that the field of view of the imaging unit 201 is blocked (S409), and the process proceeds to S410. That is, the determination unit 209 determines that the field of view of the imaging unit 201 is blocked when a block determined to have a similarity to the reference image less than a predetermined threshold is equal to or greater than a threshold (second threshold in this example) for a plurality of captured images that are consecutive for a predetermined number of frames or more (continuous for the third threshold in this example). In S410, the determination unit 209 issues an alert. For example, the determination unit 209 outputs an alert to the management device 102. After S410, and if the number of consecutive frames of the masked image is less than the third threshold (No in S408), the process shown in FIG. 4 (S307 in FIG. 3) is terminated.

[0031] 3, the determination unit 207 determines whether the number of primary occlusion blocks is equal to or greater than a first threshold, but the threshold may be expressed as a percentage, and the determination unit 207 may be configured to determine whether the percentage of the primary occlusion blocks in the captured image is equal to or greater than the first threshold. Similarly, in S404 in FIG. 4, the determination unit 209 determines whether the number of secondary occlusion blocks is equal to or greater than a second threshold, but the threshold may be expressed as a percentage, and the determination unit 209 may determine whether the percentage of the secondary occlusion blocks in the primary occlusion blocks is equal to or greater than the second threshold.

[0032] In this way, the imaging device 101 according to the present embodiment determines a block to be compared with the reference image based on the feature amount of each block into which the captured image is divided, and determines whether the field of view of the imaging unit 201 is obstructed based on the similarity between the determined block and the reference image. This two-stage determination makes it possible to determine with higher accuracy whether the field of view of the imaging unit 201 is obstructed depending on the imaging situation. For example, if the determination is made based only on the similarity between the entire captured image and the entire reference image, when the inside of a train or a store crowded with customers is the imaging target, there is a possibility that the field of view of the imaging unit 201 is erroneously detected as being obstructed based on changes in the image caused by passengers, passersby, or customers. In this embodiment, it is possible to avoid such erroneous detection by specifying the block to be compared with the reference image. In addition, this two-stage determination makes it possible to reduce a decrease in detection accuracy even in places where the subject and background change from moment to moment, such as inside a train or bus, or inside a crowded store, or in places where the background, such as the ceiling or walls, is uniform.

[0033] In addition, in the imaging device 101 installed in an environment where there is vibration, the vibration may affect the accuracy of the calculation result of the similarity by the calculation unit 208. For example, when the imaging device 101 is an in-vehicle camera, the vibration of the vehicle may affect the accuracy of the calculation result of the similarity by the calculation unit 208. In order to reduce such an effect, a captured image having the highest correlation value with the reference image is specified from captured images of a plurality of frames captured around the same time, and the specified captured image can be used to compare the similarity with the reference image. FIG. 5 shows a conceptual diagram of a plurality of captured images captured around the same time. FIG. 5 shows an example in which a subject vibrates in the lateral direction. Here, the calculation unit 208 of the imaging device 101 calculates the correlation values ​​between five captured images, including captured images of two frames captured around the same time, including the captured image of the Nth frame, and the reference image. Then, the calculation unit 208 specifies the image having the highest correlation value with the reference image among the five captured images as the captured image to be compared with the reference image for the similarity, and calculates the similarity with the captured image using the specified reference image. By using such a method, it is possible to reduce the effect of vibration on the calculation result of the similarity.

[0034] [Second embodiment] In the first embodiment, the similarity between the edge image of the captured image and the edge image of the reference image is used to compare the similarity between the imaging device and the reference image. In this embodiment, the color histogram of the image is used as the similarity. The color histogram represents the frequency corresponding to each of a plurality of predetermined color types. The explanation of the first embodiment can be applied to the second embodiment, and the following will explain the features that are different from the first embodiment, and will omit the explanation of the common features.

[0035] Fig. 6 shows a flowchart of a process of judging whether the field of view of the imaging unit 201 is obstructed using a reference image (S307 in Fig. 3) according to this embodiment. In this process, the calculation unit 208 calculates a similarity between a block (primarily obstructed block) determined by the determination unit 207 to be compared with the reference image and a corresponding block in the reference image, and the judgment unit 209 judges whether the field of view of the imaging unit 201 is obstructed based on the similarity. In this embodiment, the similarity is determined by using a similarity of color histograms. Note that the same processes as those shown in Fig. 4 and described in the first embodiment are denoted by the same reference symbols and will not be described.

[0036] First, the calculation unit 208 creates a color histogram of the captured image (object color histogram) and a color histogram of the reference image (reference color histogram) (S601). The calculation unit 208 can create a color histogram based on a color space such as RGB or YCrCb. Next, the calculation unit 208 calculates the similarity between the object color histogram created in S601 and the reference color histogram for each primary occlusion block (S602). Next, the determination unit 209 determines a block having a low similarity among the primary occlusion blocks based on the similarity calculation result in S602 as a secondary occlusion block (S603). In this embodiment, the determination unit 209 determines a block having a similarity of the feature amount less than a predetermined threshold as a block having a low similarity. The process from S404 onwards is the same as that in FIG. 4.

[0037] Note that the color histogram of an image is affected by changes in the color temperature of the ambient light. Taking this into consideration, different reference images may be used for comparing the similarity between daytime (for example, when there is sunlight coming in through a window) and nighttime (for example, when there is only interior lighting). FIG. 7 shows an example of a plurality of different reference images corresponding to a plurality of different time periods. FIG. 7 shows an example of two types of reference images, a daytime reference image 71 and a nighttime reference image 72. For example, the daytime reference image 71 is a reference image captured in advance during the day, and the nighttime reference image 72 is a reference image captured in advance at night. The calculation unit 208 selects the daytime reference image 71 or the nighttime reference image 72 according to the time period when the captured image is captured.

[0038] 6, for example, a color correlation value can be calculated between the color histograms of captured images 73 and 74 captured during the day and the color histogram of daytime reference image 71. Similarly, the calculation unit 208 can calculate a color correlation value between the color histogram of captured image 75 captured at night and the color histogram of nighttime reference image 72. In this way, the calculation unit 208 can reduce the influence of changes in environmental light by switching the reference image used for comparing the similarity with the captured image depending on the time of day when the captured image is captured.

[0039] In this way, the imaging device 101 according to this embodiment can determine with higher accuracy whether the field of view of the imaging unit 201 is blocked depending on the imaging situation, as in the first embodiment, by two-stage determination. In this embodiment, with reference to FIG. 7, a method of preparing different reference images depending on the time of day has been described as a method of reducing the influence of changes in environmental light, but the reference image may be selected in more detail depending on the lighting situation. For example, two types of reference images, one for daytime and one for nighttime, are prepared when the lighting is turned on and the other when it is turned off. That is, a total of four types of reference images are prepared. Then, the calculation unit 208 of the imaging device 101 selects one of the four reference images depending on the time of day when the captured image is captured and the lighting situation, and uses it for calculating the similarity, thereby making it possible to further reduce the influence of changes in environmental light.

[0040] [Variations] In the above embodiment, when the number of the primary occluding blocks (blocks determined to be compared with the reference image) is equal to or greater than the first threshold (Yes in S304 in FIG. 3), the imaging device 101 continuously compares the similarity between the captured image and the reference image (S307). Alternatively, the imaging device 101 may be configured to compare the similarity between the captured image and the reference image every few frames when the number of the primary occluding blocks is equal to or greater than the first threshold (Yes in S304 in FIG. 3). In this case, for example, the determination unit 209 calculates the elapsed time from when it is determined that the number of the primary occluding blocks is equal to or greater than the first threshold (Yes in S304) to when it is determined that the number of the secondary occluding blocks is equal to or greater than the second threshold (S404 in FIG. 4). Then, when the elapsed time is equal to or greater than a predetermined threshold time, the determination unit 209 may determine that the field of view of the imaging unit 201 is obstructed (S407) and may raise an alert (S410). Alternatively, the imaging device 101 may be configured to proceed to a determination process using a reference image when the number of primary occluded blocks is equal to or greater than the first threshold (Yes in S304 in FIG. 3) after a predetermined number of consecutive determinations. In this manner, by reducing the frequency of performing the determination process using the reference image or delaying the timing of starting the determination process, it is possible to reduce the processing load while maintaining the detection accuracy.

[0041] In the above embodiment, the imaging device 101 is an information processing device having an imaging unit, and it is determined whether the field of view of the imaging unit is obstructed based on an image captured by the imaging unit. Alternatively, the information processing device may be configured to perform the determination based on an image acquired from another device having an imaging unit. In this case, for example, the information processing device can determine whether the field of view of each imaging unit is obstructed for multiple images captured by multiple imaging units.

[0042] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0043] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) A dividing means for dividing an image captured by the imaging unit into a plurality of blocks; an extraction means for extracting a feature amount of each of the plurality of blocks; a determining unit that determines a block to be compared with a reference image from among the plurality of blocks in the captured image based on the feature amount extracted by the extracting unit; a calculation means for calculating a similarity between a block determined by the determination means among the plurality of blocks in the captured image and a corresponding block in the reference image; a determination means for determining whether a field of view of the imaging unit is obstructed based on a result of the calculation of the similarity by the calculation means; 13. An information processing device comprising:

[0044] (Configuration 2) The information processing device according to configuration 1, characterized in that the determination means determines, among the plurality of blocks in the captured image, a block that is determined to have an obstructed field of view based on the feature extracted by the extraction means, as a block to be compared with the reference image.

[0045] (Configuration 3) The information processing device according to configuration 1 or 2, characterized in that the calculation means calculates the similarity of the blocks determined by the determination means to be compared with the reference image when the number of the blocks determined by the determination means to be compared with the reference image is equal to or greater than a threshold value.

[0046] (Configuration 4) 4. The information processing device according to any one of configurations 1 to 3, wherein the feature amount is an edge amount of each of the plurality of blocks.

[0047] (Configuration 5) a generating unit for generating a target edge image which is an edge image of each of the primary occlusion blocks determined as blocks to be compared with the reference image among the plurality of blocks in the captured image, and a reference edge image which is an edge image of the corresponding block in the reference image, the calculation means calculates a correlation value between the target edge image and the reference edge image as the degree of similarity. 5. The information processing device according to any one of configurations 1 to 4.

[0048] (Configuration 6) The method further includes a specification unit that specifies a captured image having a highest correlation value with the reference image from among the captured image and one or more captured images captured by the imaging unit at a time before or after the captured image, The creation means generates the target edge image of a block corresponding to each of the primary occlusion blocks in the identified captured image. 6. The information processing device according to configuration 5.

[0049] (Configuration 7) a generation unit for generating a target color histogram, which is a color histogram of each of the primary occlusion blocks determined as blocks to be compared with the reference image among the plurality of blocks in the captured image, and a reference color histogram, which is a color histogram of the corresponding block in the reference image; the calculation means calculates a correlation value between the object color histogram and the reference color histogram as the degree of similarity. 5. The information processing device according to any one of configurations 1 to 4.

[0050] (Configuration 8) The imaging device further includes a specification unit that specifies a captured image having a highest correlation value with the reference image from among the captured image and one or more captured images captured by the imaging unit at a time before or after the captured image, the creation means generates the object color histogram of a block corresponding to each of the primary occlusion blocks in the identified captured image. 8. The information processing device according to configuration 7.

[0051] (Configuration 9) 9. The information processing device according to configuration 7 or 8, wherein the calculation means selects the reference image from a plurality of different reference images corresponding to a plurality of different time periods in accordance with a time period when the captured image was captured.

[0052] (Configuration 10) The information processing device according to any one of claims 1 to 9, characterized in that the determination means determines that the field of view of the imaging unit is obstructed when the number of blocks for which the similarity is determined to be less than the threshold value is equal to or greater than a threshold value for a plurality of consecutive captured images of a predetermined number of frames or more captured by the imaging unit.

[0053] (Claim 11) 11. The information processing apparatus according to claim 1, wherein the determining means outputs an alert to an external device when it determines that the field of view of the imaging unit is obstructed.

[0054] (Method 1) A dividing step of dividing an image captured by an imaging unit into a plurality of blocks; an extraction step of extracting a feature amount of each of the plurality of blocks; a determining step of determining a block to be compared with a reference image from among the plurality of blocks in the captured image based on the feature amount extracted in the extracting step; a calculation step of calculating a similarity between a block determined in the determination step among the plurality of blocks in the captured image and a corresponding block in the reference image; a determination step of determining whether the field of view of the imaging unit is obstructed based on a result of the calculation of the similarity in the calculation step; 13. An information processing method comprising: [Explanation of symbols]

[0055] 1: monitoring system, 101-1; 101-2; 101-3: imaging device, 102: management device, 103: network, 104: operator, 201: imaging unit, 202: image compression unit, 203: formatting unit, 204: communication unit, 205: block division unit, 206: extraction unit, 207: determination unit, 208: calculation unit, 209: judgment unit, 210: setting unit, 221: processor, 222: ROM, 223: RAM, 224: HDD, 225: communication I / F, 226: image sensor

Claims

1. A dividing means for dividing an image captured by the imaging unit into a plurality of blocks; an extraction means for extracting a feature amount of each of the plurality of blocks; a determining unit that determines a block to be compared with a reference image from among the plurality of blocks in the captured image based on the feature amount extracted by the extracting unit; a calculation means for calculating a similarity between a block determined by the determination means among the plurality of blocks in the captured image and a corresponding block in the reference image; a determination means for determining whether a field of view of the imaging unit is obstructed based on a result of the calculation of the similarity by the calculation means; 13. An information processing device comprising:

2. The information processing device according to claim 1, characterized in that the determination means determines, among the plurality of blocks in the captured image, a block that is determined to have an obstructed field of view based on the features extracted by the extraction means, as a block to be compared with the reference image.

3. 3. The information processing apparatus according to claim 1, wherein the calculation means calculates the similarity of the blocks determined by the determination means to be compared with the reference image when the number of the blocks determined by the determination means to be compared with the reference image is equal to or greater than a threshold value.

4. The information processing apparatus according to claim 1 , wherein the feature amount is an edge amount of each of the plurality of blocks.

5. a generating unit for generating a target edge image which is an edge image of each of the primary occlusion blocks determined as blocks to be compared with the reference image among the plurality of blocks in the captured image, and a reference edge image which is an edge image of the corresponding block in the reference image, the calculation means calculates a correlation value between the target edge image and the reference edge image as the degree of similarity.

2. The information processing apparatus according to claim 1,

6. The imaging device further includes a specification unit that specifies a captured image having a highest correlation value with the reference image from among the captured image and one or more captured images captured by the imaging unit at a time before or after the captured image, The creation means generates the target edge image of a block corresponding to each of the primary occlusion blocks in the identified captured image.

6. The information processing apparatus according to claim 5,

7. a generation unit for generating a target color histogram, which is a color histogram of each of the primary occlusion blocks determined as blocks to be compared with the reference image among the plurality of blocks in the captured image, and a reference color histogram, which is a color histogram of the corresponding block in the reference image; the calculation means calculates a correlation value between the object color histogram and the reference color histogram as the degree of similarity.

2. The information processing apparatus according to claim 1,

8. The imaging device further includes a specification unit that specifies a captured image having a highest correlation value with the reference image from among the captured image and one or more captured images captured by the imaging unit at a time before or after the captured image, the creation means generates the object color histogram of a block corresponding to each of the primary occlusion blocks in the identified captured image.

8. The information processing apparatus according to claim 7,

9. 8. The information processing apparatus according to claim 7, wherein the calculation means selects the reference image from a plurality of different reference images corresponding to a plurality of different time periods in accordance with a time period in which the captured image was captured.

10. The information processing device according to claim 1, characterized in that the determination means determines that the field of view of the imaging unit is obstructed when the number of blocks for which the similarity is determined to be less than a threshold value is equal to or greater than a threshold value for a plurality of consecutive captured images of a predetermined number of frames or more captured by the imaging unit.

11. The information processing apparatus according to claim 1 , wherein the determining means outputs an alert to an external device when it determines that the field of view of the imaging section is obstructed.

12. A dividing step of dividing an image captured by an imaging unit into a plurality of blocks; an extraction step of extracting a feature amount of each of the plurality of blocks; a determining step of determining a block to be compared with a reference image from among the plurality of blocks in the captured image based on the feature amount extracted in the extracting step; a calculation step of calculating a similarity between a block determined in the determination step among the plurality of blocks in the captured image and a corresponding block in the reference image; a determination step of determining whether the field of view of the imaging unit is obstructed based on a result of the calculation of the similarity in the calculation step; 13. An information processing method comprising:

13. A program for causing a computer to function as the information processing device according to claim 1.

Images