Information processing device and information processing method

Abstract

To reduce failure in detecting a human body in a look-down image captured by a fish eye camera.SOLUTION: An information processing apparatus comprises: a storage part which stores color information on a human body as reference color information; a detection part which detects a human body candidate in an image captured by a fish eye camera; and a human body determination part which acquires reference color information corresponding to a detection region in which the human body candidate is detected from the storage part, and then determines whether the human body candidate is a human body based upon the similarity between the acquired reference color information on the detection region and the color information on the human body candidate.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing method.

Background Art

[0002] In recent years, in the factory automation (FA) market, applications for analyzing human movement using a fisheye downward-looking camera to improve processes in line production methods and cell production methods have been utilized. To analyze human movement, improvement in human body detection accuracy is required. Patent Document 1 discloses a technique for identifying a face candidate region that is likely to be a human face as a face region by determining whether the hue of the face candidate region extracted from the captured image is skin color.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a downward-looking image captured by a fisheye camera installed on a ceiling or the like, the appearance of a person changes depending on the position within the image, and the color information of the person to be detected varies depending on the detected position. Therefore, even when using color information in an image captured by a fisheye camera, it may be difficult to reduce false detection.

[0005] One aspect of the present invention aims to provide a technique for reducing false detection of the human body in a downward-looking image captured by a fisheye camera.

Means for Solving the Problems

[0006] To achieve the above object, the present invention employs the following configuration.

[0007] The first aspect of this disclosure is an information processing device comprising: a storage unit that stores color information of a human body as reference color information; a detection unit that detects human body candidates from images captured by a fisheye camera; and a human body determination unit that acquires reference color information corresponding to the detection area where a human body candidate is detected from the storage unit, and determines whether or not a human body candidate is a human body based on the similarity between the acquired reference color information of the detection area and the color information of the human body candidate.

[0008] The information processing device can accurately determine whether a detected human body candidate is a human body or not by comparing it with reference color information corresponding to the detected location (detection area) of the human body candidate, thereby reducing false detections of human bodies in overhead images captured by a fisheye camera.

[0009] The memory unit may store the color information of the human body detected in each of the multiple regions obtained by dividing the image captured by the fisheye camera as reference color information. By providing reference color information for each region, the information processing device can accurately determine whether or not a candidate human body detected within a region is a human body.

[0010] The information processing device may further include a generation unit that generates reference color information for a region from the color information of a human body detected in that region and stores it in a memory unit. By including a generation unit, the information processing device can generate and update reference color information while performing human body detection processing. The information processing device can continuously update the reference color information, and time-dependent information such as background and person characteristics can be stored. This can suppress the decrease in accuracy of human body detection that occurs due to changes in the environment.

[0011] The generation unit may generate reference color information corresponding to each region based on the color information of the human body detected in each region of multiple captured images. The information processing device can create a more average color information map by generating reference color information corresponding to each region from multiple training captured images.

[0012] The generation unit updates the reference color information of a region based on the newly detected human body color information if the correlation coefficient between the newly detected human body color information in the region and the reference color information corresponding to the region is greater than or equal to the first threshold. If the correlation coefficient between the newly detected human body color information in the region and the reference color information of the region is less than the second threshold (less than or equal to the first threshold), the generation unit may store the newly detected human body color information in the region as the new reference color information for the region in the storage unit. The information processing device stores new reference color information in the storage unit from the captured images used for training, using color information with different characteristics. Each region is associated with multiple reference color information sets, and the information processing device can accurately detect human bodies even when detecting people with different color characteristics.

[0013] Reference color information is generated based on pixel values ​​within a frame surrounding a human body detected in a region, and color information for a candidate human body may be generated based on pixel values ​​within a frame surrounding the candidate human body. The information processing device can suppress false detections based on the difference in pixel values ​​between a human body and a candidate human body.

[0014] Reference color information is generated based on pixel values ​​in the portion of the captured image excluding the background, within the frame surrounding the detected human body. Color information for candidate human bodies may be generated based on pixel values ​​in the portion of the captured image excluding the background, within the frame surrounding the candidate human body. By generating color information while excluding the background, the information processing device can generate basic color information and color information only from the actual human body and candidate human body parts, enabling accurate detection of human bodies.

[0015] The reference color information is a histogram of pixel values ​​within the frame surrounding the human body detected in the region, and the color information of the human body candidate may be a histogram of pixel values ​​within the frame surrounding the human body candidate. The information processing device can suppress false detections based on the difference in the distribution of pixel values ​​between the human body and the human body candidate.

[0016] The human body detection unit may determine that a candidate is a human body if the correlation coefficient between the histogram of color information of the candidate human body and the histogram of reference color information corresponding to the detection area is greater than or equal to a predetermined threshold. The information processing device can determine whether or not a candidate is a human body according to the correlation coefficient (similarity) between the histograms. The information processing device can adjust the detection accuracy for differences in color characteristics by changing a predetermined threshold.

[0017] The pixel value histogram may be a histogram for each RGB component. Furthermore, the human body detection unit may determine whether a candidate is a human body based on the average, maximum, or minimum correlation coefficients between the RGB histogram of the color information of the candidate human body and the RGB histogram of the reference color information corresponding to the detection area. The information processing device can suppress false detections when the color characteristics of a human body and a candidate human body differ.

[0018] The reference color information is the mode or mean of the RGB values ​​of the pixels within the frame surrounding the detected human body in the area, and the color information of the human body candidate may be the mode or mean of the RGB values ​​of the pixels within the frame surrounding the human body candidate. The information processing device can accurately detect a human body by comparing the color information of the human body candidate with the reference color information using a simple calculation.

[0019] The reference color information may be information obtained by averaging the color information of the human body detected in a region in multiple captured images. The color information may be used as training data to train a pre-trained model, which then outputs whether the input color information of a candidate human body matches the color information of a human body detected in the detection region. The information processing device can create a more average color information map by generating reference color information corresponding to each region from multiple training images.

[0020] The region may be associated with multiple reference color information. The information processing device can accurately detect the human body even when detecting individuals with different color characteristics.

[0021] The information processing apparatus may further include an output unit that presents, to the user, a human body candidate determined to be a human body by the human body determination unit as a detection result of the human body. The information processing apparatus can present the user with a detection result of the human body in consideration of the color information.

[0022] The storage unit may store reference color information corresponding to the distance from the center position of the captured image by the fisheye camera. By preparing the reference color information corresponding to the distance from the center position of the captured image, the information processing apparatus can accurately determine whether the human body candidate detected at that distance is a human body.

[0023] A second aspect of the present invention is an information processing method including: a storage step in which a computer stores, in a storage unit, color information of a human body detected in a region as reference color information of the region for each of a plurality of regions obtained by dividing a captured image by a fisheye camera; a detection step of detecting a human body candidate from the captured image; and a human body determination step of obtaining, from the storage unit, reference color information corresponding to a detection region in which the human body candidate is detected among the regions, and determining whether the human body candidate is a human body based on the obtained reference color information of the detection region and the color information of the human body candidate.

[0024] The present invention can also be regarded as a program for realizing such a method by a computer and a recording medium on which the program is non-temporarily recorded. Each of the above processes can be combined with each other as much as possible to constitute the present invention.

Effect of the Invention

[0025] According to the present invention, false detection of a human body can be reduced in a downward-looking image captured by a fisheye camera.

Brief Description of the Drawings

[0026] [Figure 1] FIG. 1 is a diagram for explaining an application example of an information processing apparatus according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating the hardware configuration of the information processing apparatus. [Figure 3] Figure 3 is a diagram illustrating the functional configuration of an information processing device. [Figure 4] Figure 4 is a flowchart illustrating the human body detection process. [Figure 5] Figure 5 is a diagram illustrating the color information related to Modification Example 1. [Figure 6] Figure 6 is a diagram illustrating the color information related to Modification Example 2. [Figure 7] Figure 7 is a diagram illustrating the color information map related to Modification Example 3. [Figure 8] Figure 8 is a diagram illustrating the color information map related to Modification 4. [Modes for carrying out the invention]

[0027] Hereinafter, embodiments relating to one aspect of the present invention will be described based on the drawings.

[0028] <Examples of application> Figure 1 is a diagram illustrating an example of the application of an information processing device according to the embodiment. The information processing device detects a subject recognized as a human body (hereinafter referred to as a human body candidate) from a camera image (captured image) captured by a camera, and uses color information to determine whether or not the human body candidate is a human body.

[0029] The information processing device divides the captured image into multiple regions and acquires (generates) color information of the human body detected in each region. In the example in Figure 1, the training image 1 is divided into 4x4 rectangular regions. The information processing device generates a histogram for each RGB component for the pixels within the frame surrounding the human body detected in region A1, as color information of the human body. The generated color information is stored in the memory unit as reference color information (hereinafter referred to as reference color information) for determining whether a candidate human body is a human body or not. The reference color information is the color information of the human body detected in each region and is generated for each of the multiple regions into which the captured image is divided.

[0030] The information processing device detects potential human body candidates in the captured image and generates color information for those candidates. Figure 1 shows an example where the information processing device detects a personal computer (PC) as a potential human body candidate in region A1 of captured image 2. The information processing device generates histograms for each RGB component for the pixels within the frame surrounding the detected PC, as color information for the potential human body candidate.

[0031] The information processing device compares the color information of the PC detected in region A1 of the captured image 2 with the reference color information of region A1, and determines whether the PC detected as a candidate for a human body is a human body based on the similarity between them. For example, the information processing device calculates the correlation coefficient between the histogram of the color information of the candidate human body (PC) and the histogram of the reference color information as the similarity, and if the correlation coefficient is above a predetermined threshold, it can determine that the candidate human body is a human body.

[0032] The correlation coefficient is calculated using various known methods, for example, as a value between 0 and 1. In the example in Figure 1, the information processing device calculates the correlation coefficient between the histograms for each RGB channel, and can use, for example, the average of the three correlation coefficients as the correlation coefficient for determining whether or not it is a human body. The predetermined threshold can be, for example, a value between 0.6 and 0.9. By raising the predetermined threshold, the information processing device can determine whether or not it is a human body with greater accuracy.

[0033] The camera used for imaging for human body detection is an ultra-wide-angle camera equipped with a fisheye lens that can acquire image information over a wide area. Cameras equipped with a fisheye lens are also called fisheye cameras, omnidirectional cameras, or 360-degree cameras, but in this specification, the term "fisheye camera" will be used.

[0034] Images captured with a fisheye camera exhibit distortion in the appearance of the subject depending on its position within the captured image. For example, if a fisheye camera is positioned to look down from the ceiling onto the floor, the image of a person will have their feet facing the center and their head facing outwards. The human body appears as a frontal, rear, or side view at the edges of the captured image, and as a top view in the center of the image.

[0035] Therefore, even for the same person or a person wearing the same uniform, the color information generated will differ depending on the detected area. The information processing device can accurately determine whether a candidate is a human body or not by comparing the color information of the human body candidate with reference color information corresponding to the area where the human body candidate is detected (detection area). Thus, the information processing device can reduce false detections of human bodies in overhead images captured by a fisheye camera.

[0036] When detecting potential human bodies from images captured by a camera installed in a specific location, the information processing device primarily detects potential human bodies from background subtraction, which is the background image removed from the captured image. In this case, objects included in the background subtraction are more likely to be detected as potential human bodies even if they are not human bodies. In particular, when detecting human bodies using a model trained to detect human bodies in a specific environment, objects that are not included in the background and have a different color may be detected as human bodies. The information processing device can reduce false detections of objects from background subtraction by using color information to determine whether or not an object is a human body.

[0037] <Embodiment> (Hardware configuration) Referring to Figure 2, an example of the hardware configuration of the information processing device 10 will be described. Figure 2 is a diagram illustrating the hardware configuration of the information processing device 10. The information processing device 10 comprises a processor 101, a main memory unit 102, an auxiliary memory unit 103, a communication interface (I / F) 104, and a display unit 105.

[0038] The processor 101 reads programs stored in the auxiliary storage unit 103 into the main memory unit 102 and executes them, thereby realizing the functions of each functional configuration described in Figure 3. The main memory unit 102 is, for example, a semiconductor memory such as RAM (Random Access Memory) or ROM (Read Only Memory). The auxiliary storage unit 103 is, for example, a non-volatile memory such as a hard disk drive or a solid-state drive.

[0039] The communication interface 104 is an interface for communication via wired (USB cable, LAN cable, etc.) or wireless (WiFi, etc.). The display unit 105 is a display or the like for displaying the results of human body detection.

[0040] The information processing device 10 may be a general-purpose computer such as a personal computer, server computer, tablet terminal, or smartphone, or it may be an embedded computer such as an onboard computer. The information processing device 10 may implement some of the processing of each functional unit using a cloud server. Furthermore, some of the processing of each functional unit of the information processing device 10 may be implemented using dedicated hardware devices such as FPGAs or ASICs.

[0041] The information processing device 10 is connected to the camera 20 by wire or wireless connection and receives image data (captured images) captured by the camera 20. The camera 20 is an imaging device having an optical system including a lens and an image sensor (such as a CCD or CMOS image sensor).

[0042] Furthermore, some of the processing performed by the information processing device 10 may be performed by the camera 20. Also, the results of human body detection by the information processing device 10 may be transmitted to an external device and presented to the user. In addition, the information processing device 10 may be configured as an integral part of the camera 20.

[0043] (Functional Configuration) Referring to Figure 3, an example of the functional configuration of the information processing device 10 will be described. Figure 3 is a diagram illustrating the functional configuration of the information processing device 10. The information processing device 10 includes a color information generation unit 11, a detection unit 12, a human body determination unit 13, an output unit 14, and a color information database 15 (color information DB 15).

[0044] The color information generation unit 11 acquires the captured image captured by the camera 20 and generates reference color information for each of the multiple regions into which the captured image has been divided, from the color information of the human body detected in that region. The reference color information data associated with each region into which the captured image has been divided is also called a color information map.

[0045] Color information is generated based on the pixel values ​​within the frame surrounding the detected human body. For example, the color information generation unit 11 generates information representing the pixel values ​​(luminance values) of pixels within the frame surrounding the human body as a histogram, and uses this as reference color information for the area where the human body is detected. The color information generation unit 11 may also generate histograms for each RGB component as reference color information. The color information generation unit 11 stores the generated reference color information in the color information database 15, associating it with the area where the human body is detected.

[0046] The color information generation unit 11 can generate color information for human body candidates detected from the captured image, similar to the reference color information. The human body candidate color information is used by the human body determination unit 13 to determine whether or not the human body candidate is a human body. The color information generation unit 11 is an example of a "generation unit".

[0047] The detection unit 12 acquires the captured image from the camera 20 and detects human body candidates from the captured image. The detection unit 12 can detect human body candidates using general object recognition algorithms. For example, the detection unit 12 can detect human body candidates using a classifier that combines image features such as HoG or Haar-like with boosting. Alternatively, the detection unit 12 may detect human body candidates using human body recognition algorithms based on deep learning (e.g., R-CNN, Faster R-CNN, YOLO, SSD, etc.).

[0048] The human body determination unit 13 determines whether the human body candidate detected by the detection unit 12 is a human body or not. The human body determination unit 13 determines whether the human body candidate is a human body or not based on the similarity between the reference color information corresponding to the detection region in which the human body candidate was detected among multiple regions of the captured image and the color information of the human body candidate. The detection region can be, for example, the region in which the center position of the rectangular frame surrounding the human body candidate is detected.

[0049] The output unit 14 outputs human body candidates that have been determined to be human bodies by the human body determination unit 13 as human body detection results. The output unit 14 can present the human body detection results to the user, for example, by superimposing a rectangle surrounding the detected human body onto the captured image.

[0050] The color information database 15 stores a color information map previously created by the color information generation unit 11. The color information map includes regions into which the captured image has been divided, and reference color information data associated with those regions. The reference color information in the color information map may be updated while the human body detection process is being performed. The color information database 15 is an example of a "storage unit".

[0051] (Human body detection process) Referring to Figure 4, the overall flow of the human body detection process will be explained. Figure 4 is a flowchart illustrating the human body detection process. The human body detection process starts, for example, when the camera 20 is powered on and the information processing device 10 receives the captured image from the camera 20. The human body detection process shown in Figure 4 is executed for each frame (captured image) of the image data received from the camera 20. Furthermore, the color information map is assumed to have been created in advance by the color information generation unit 11 and stored in the color information database 15.

[0052] In step S101, the detection unit 12 acquires the captured image. The detection unit 12 can acquire the captured image from the camera 20 via the communication interface 104. If the information processing device 10 is configured integrally with the camera (imaging unit), the color information generation unit 11 acquires the captured image captured by the imaging unit.

[0053] In step S102, the detection unit 12 detects human body candidates from the image acquired in step S101. The detection unit 12 can detect human body candidates using known techniques such as deep learning. If multiple human body candidates are detected in step S102, the human body determination process L1 from steps S103 to S106 is repeated for each human body candidate.

[0054] In step S103, the human body determination unit 13 determines whether the human body candidate detected in step S102 is a human body or not. First, the human body determination unit 13 determines multiple regions obtained by dividing the captured image. From this, the system obtains reference color information corresponding to the detection area where a human body candidate is detected from the color information database 15. Next, the human body determination unit 13 determines whether or not the human body candidate is a human body based on the similarity between the obtained reference color information of the detection area and the color information of the detected human body candidate. The color information of the human body candidate is obtained by the color information generation unit 11.

[0055] The color information is represented as a histogram for each RGB color, as illustrated in Figure 1. In this case, the human body detection unit 13 calculates the correlation coefficient between the RGB histogram of the color information of the candidate human body and the RGB histogram of the reference color information.

[0056] In step S104, the human body determination unit 13 determines whether the correlation coefficient calculated in step S103 is above a predetermined threshold. For example, the human body determination unit 13 determines that the human body candidate is a human body if the mean, maximum, or minimum value of each correlation coefficient of the RGB histogram is above a predetermined threshold. Alternatively, the human body determination unit 13 may determine that the human body candidate is a human body if all of the correlation coefficients of the RGB histogram are above a predetermined threshold.

[0057] If the correlation coefficient is above a predetermined threshold and the candidate for human body is determined to be a human body (Step S104: YES), the process proceeds to Step S105. If the correlation coefficient is below a predetermined threshold and the candidate for human body is determined not to be a human body (Step S104: NO), the process proceeds to Step S106.

[0058] In step S105, the human body detection unit 13 adopts the human body candidate to be determined as a human body detection result and stores it in the auxiliary storage unit 103, etc. In step S106, the human body detection unit 13 removes the human body candidate to be determined as a false detection.

[0059] When the human body determination process L1 is executed for each human body candidate detected in step S102, the process proceeds to step S107. In step S107, the output unit 14 outputs information about the human body adopted as the detection result in step S105. The output unit 14 may also display the detection results on the display unit 105. The output unit 14 can present the human body detection results to the user, for example, by superimposing a rectangle surrounding the detected human body onto the captured image. The output unit 14 may also output the detection results to an external device and display them on the external device's display or the like.

[0060] Once the human body detection process for the current frame (captured image) is completed, the human body detection process for the next frame begins. The information processing device 10 repeatedly executes the human body detection process shown in Figure 4 until the imaging process by the camera 20 is stopped.

[0061] In the above embodiment, the information processing device 10 detects a human body candidate from the captured image and determines whether or not the human body candidate is a human body based on the similarity between the reference color information corresponding to the detection area where the human body candidate is detected and the color information of the human body candidate. In images captured by a fisheye camera, the appearance differs depending on the position where the image was taken, but the information processing device 10 can accurately determine whether or not the detected human body candidate is a human body by comparing it with reference color information corresponding to the position (detection area) where the human body candidate is detected. Therefore, the information processing device 10 can reduce false detections of human bodies in overhead images captured by a fisheye camera.

[0062] <Example 1> The above embodiment shows an example where the color information is a histogram of pixel values. Modification 1 is an example where the color information is the mode of the RGB values ​​of the pixels within the frame surrounding the human body or a candidate human body. However, the color information is not limited to the mode, and may be the average value of the RGB values ​​of the pixels within the frame surrounding the human body or a candidate human body.

[0063] Figure 5 is a diagram illustrating the color information related to Modification 1. Captured image 1 is a training image for generating reference color information, and captured image 2 is a captured image that is the target of human body detection.

[0064] The color information generation unit 11 acquires the most frequent RGB values ​​(r1, g1, b1) for pixels within the frame surrounding the human body detected in region A1 of the captured image 1, and stores them in the color information database 15 as reference color information for region A1. The color information generation unit 11 also acquires the most frequent RGB values ​​(r2, g2, b2) for pixels within the frame surrounding the PC (human body candidate) detected in region A1 of the captured image 2, as color information for the human body candidate. The mode is the pixel value that appears most frequently among the pixels within the frame surrounding the human body or human body candidate. r1, g1, b 1, r², g², and b² are represented by values ​​ranging from 0 to 255.

[0065] The human body determination unit 13 calculates the distance d between the most frequent RGB values ​​(r1, g1, b1) of the reference color information and the most frequent RGB values ​​(r2, g2, b2) of the color information of the human body candidate using the following equation (Equation 1).

number

[0066] According to Modification 1, the information processing device 10 can accurately detect a human body by comparing the color information of a candidate human body with reference color information using a simple calculation.

[0067] <Modification 2> Modification 2 is an example in which color information is generated based on pixel values ​​in the area within the frame surrounding the human body or a candidate human body, excluding the background. By excluding the pixels in the background area, the color information generation unit 11 can generate color information for the human body or a candidate human body more accurately.

[0068] Figure 6 illustrates the color information related to Modification 2. Figure 6 illustrates an example of generating color information for a detected human body candidate from the captured image 60 to be detected. The color information generation unit 11 can similarly generate reference color information for a human body from a training captured image.

[0069] The detection unit 12 detects human body candidates from the captured image 60. Since the rectangular frame 61 surrounding the human body candidate includes the background image, if color information is generated from the pixel values ​​within the frame 61, the generated color information will include color information of the background area other than the human body candidate. Therefore, the color information generation unit 11 acquires a background difference 63, which is the difference between the captured image 60 and the background image 62, and generates color information from the human body candidate 64 in the background difference 63.

[0070] According to Modification 2, the information processing device 10 generates color information excluding the background portion, thereby generating basic color information and color information only from the actual human body and candidate human body parts, and can detect the human body with high accuracy.

[0071] <Variation 3> The above embodiment shows an example in which reference color information corresponding to each region is generated from a single training image. Modification 3 is an example in which reference color information corresponding to each region is generated from multiple training images and a color information map is created.

[0072] Figure 7 is a diagram illustrating the color information map according to the modified example 3. The color information generation unit 11 generates reference color information 74 corresponding to region A1 by, for example, averaging the color information 71 and color information 73 of the human body detected in region A1 of the captured image 70 and captured image 72.

[0073] The color information generation unit 11 is not limited to the two captured images 70 and 72, but may generate reference color information from three or more captured images. Furthermore, the color information generation unit 11 may generate reference color information from a trained model that has been trained using the color information of a human body detected in region A1 as training data across multiple training captured images. This trained model outputs whether or not the input color information of a candidate human body is the color information of a human body detected in the detection region.

[0074] According to Modification 3, the information processing device 10 can create a more average color information map by generating reference color information corresponding to each region from multiple training images.

[0075] <Modification 4> Modification 4, like Modification 3, is an example of generating reference color information corresponding to each region from multiple training images and creating a color information map. While Modification 3 is an example of generating one reference color information from multiple training images, Modification 4 is an example of generating multiple reference color information from multiple training images. In Modification 4, if color information is generated from any of the training images whose correlation coefficient with the current reference color information is less than a predetermined threshold, that color information is retained as new reference color information.

[0076] Figure 8 is a diagram illustrating the color information map related to the modified example 4. First, the color information generation unit 11 adopts the color information 81 of the human body detected in the captured image 80 as the reference color information 81 in region A1.

[0077] Next, the color information generation unit 11 generates color information 83 of the human body newly detected in the captured image 82 and calculates a correlation coefficient with the reference color information 81. If the correlation coefficient is greater than or equal to a predetermined threshold (first threshold), the color information generation unit 11 updates the reference color information 81 based on the color information 83. The color information generation unit 11 updates the reference color information 84 by, for example, averaging the reference color information 81 with the color information 83. The first threshold can be, for example, a value between 0.6 and 0.9.

[0078] Furthermore, the color information generation unit 11 generates color information 86 of the human body newly detected in the captured image 85 and calculates a correlation coefficient with the reference color information 84. If the correlation coefficient is less than a predetermined threshold (second threshold), the color information generation unit 11 adopts the color information 86 as the new reference color information 86 in area A1 and stores it in the color information database 15. The second threshold is a value less than or equal to the first threshold, and can be, for example, a value between 0.3 and 0.6.

[0079] Similarly, if there are other training images, the color information generation unit 11 calculates the correlation coefficients between the newly detected human body color information and the reference color information 84 and reference color information 86. If the correlation coefficient is equal to or greater than the first threshold, the color information generation unit 11 updates the reference color information based on the newly detected human body color information. If the correlation coefficient for any of the reference color information falls below the second threshold, the color information generation unit 11 adopts the newly detected human body color information as the new reference color information for region A1 and stores it in the color information database 15.

[0080] According to Modification 4, when the information processing device 10 generates color information with different characteristics from the captured image used for training, it stores this information in the color information database 15 as new reference color information. By associating each region with multiple reference color information, the information processing device 10 can accurately detect human bodies even when detecting people with different uniform colors.

[0081] Furthermore, by adopting multiple reference color information in each area, it is possible to obtain information such as the frequency with which individuals with different characteristics are detected. This information can be used as useful information for process improvement in factory automation.

[0082] <Modification 5> In the above embodiment, an example was described in which a color information map containing reference color information for each region is prepared in advance. However, the color information map may be generated and updated while the human body detection process is being performed. In step S105 of the human body detection process shown in Figure 4, if the human body determination unit 13 determines that a candidate is a human body, it updates the reference color information by averaging the color information of the human body candidate determined to be a human body with the current reference color information.

[0083] According to Modification 5, the information processing device 10 can continuously update the reference color information, thereby suppressing the decrease in accuracy of human body detection due to changes over time in the background and the characteristics of the person.

[0084] <Other> The embodiments and variations described above are merely illustrative examples illustrating the configuration of the present invention. The present invention is not limited to the specific forms described above, and various combinations and modifications are possible within the scope of its technical idea.

[0085] In the above embodiments and their variations, the explanation was given using an example where the captured image is divided into a 4x4 rectangular region and human body detection processing is performed. However, the method of dividing the captured image is not limited to a 4x4 rectangular region. The number of divided regions may be more or less than 4x4. Also, the shape of the divided regions is not limited to rectangles; they may be concentric circles. Furthermore, the information processing device 10 is not limited to preparing reference color information for each divided region, but may also prepare reference color information according to the distance from the center position of the captured image.

[0086] <Note> (1) A memory unit (15) that stores the color information of the human body as reference color information, A detection unit (12) detects potential human body candidates from images captured by a fisheye camera, A human body determination unit (13) acquires the reference color information corresponding to the detection area in which the human body candidate is detected from the storage unit, and determines whether or not the human body candidate is a human body based on the similarity between the acquired reference color information of the detection area and the color information of the human body candidate, An information processing device (10) equipped with the following.

[0087] (2) The computer A memory step in which the color information of the human body is stored in the memory unit as reference color information, A detection step (step S102) for detecting potential human body candidates from images captured by a fisheye camera, A human body determination step (steps S103 to S106) is performed to obtain the reference color information corresponding to the detection area in which the human body candidate is detected from the storage unit, and to determine whether or not the human body candidate is a human body based on the similarity between the obtained reference color information of the detection area and the color information of the human body candidate, Information processing methods including [Explanation of Symbols]

[0088] 10: Information processing device, 11: Color information generation unit, 12: Detection unit, 13: Human body detection unit, 14: Output unit, 15: Color information database, 20: Camera 101: Processor, 102: Main memory, 103: Auxiliary memory, 104: Communication interface, 105: Display unit

Claims

1. A memory unit that stores the color information of the human body as reference color information, A detection unit that detects potential human bodies from images captured by a fisheye camera positioned to look down from above, A human body determination unit acquires reference color information from the storage unit corresponding to the detection area in which the human body candidate is detected, and determines whether or not the human body candidate is a human body based on the similarity between the acquired reference color information of the detection area and the color information of the human body candidate. An information processing device equipped with the following features.

2. The storage unit stores, for each of the multiple regions obtained by dividing the image captured by the fisheye camera, the color information of the human body detected in that region as the reference color information. The information processing apparatus according to claim 1.

3. The system further includes a generation unit that generates reference color information for the region from the color information of the human body detected in the region and stores it in the storage unit. The information processing apparatus according to claim 2.

4. The generation unit generates the reference color information corresponding to the region based on the color information of the human body detected in each of the regions of the plurality of captured images. The information processing apparatus according to claim 3.

5. The generating unit is If the correlation coefficient between the color information of the human body newly detected in the region and the reference color information corresponding to the region is greater than or equal to the first threshold, the reference color information of the region is updated based on the color information of the human body newly detected in the region. If the correlation coefficient between the color information of the human body newly detected in the region and the reference color information of the region is less than the second threshold (which is less than or equal to the first threshold), the color information of the human body newly detected in the region is stored in the storage unit as the new reference color information of the region. The information processing apparatus according to claim 3.

6. The aforementioned reference color information is generated based on the pixel values ​​within the frame surrounding the human body detected in the region. The information processing device according to any one of claims 2 to 5, wherein the color information of the candidate human body is generated based on the pixel values ​​within the frame surrounding the candidate human body.

7. The aforementioned reference color information is generated based on the pixel values ​​in the portion of the captured image excluding the background, within the frame surrounding the human body detected in the region. The color information of the human body candidate is generated based on the pixel values ​​in the portion of the captured image excluding the background portion from within the frame surrounding the human body candidate. The information processing apparatus according to claim 6.

8. The aforementioned reference color information is a histogram of pixel values ​​within the frame surrounding the human body detected in the region. The information processing device according to claim 6, wherein the color information of the candidate human body is a histogram of pixel values ​​within the frame surrounding the candidate human body.

9. The human body determination unit determines that the human body candidate is a human body if the correlation coefficient between the histogram of the color information of the human body candidate and the histogram of the reference color information corresponding to the detection area is greater than or equal to a predetermined threshold. The information processing apparatus according to claim 8.

10. The aforementioned histogram of pixel values ​​is a histogram for each RGB channel. The information processing apparatus according to claim 8.

11. The human body determination unit determines whether the human body candidate is a human body or not based on the average, maximum, or minimum value of the correlation coefficient between the RGB histogram of the color information of the human body candidate and the RGB histogram of the reference color information corresponding to the detection area. The information processing apparatus according to claim 10.

12. The aforementioned reference color information is the mode or mean value of the RGB values ​​of the pixels within the frame surrounding the human body detected in the region. The color information of the candidate human body is the mode or mean value of the RGB values ​​of the pixels within the frame surrounding the candidate human body. The information processing apparatus according to claim 6.

13. The aforementioned reference color information is information obtained by averaging the color information of the human body detected in the region in multiple captured images. The information processing apparatus according to any one of claims 2 to 5.

14. The aforementioned reference color information is a trained model that has been trained using the color information of human bodies detected in the region in multiple captured images as training data, and outputs whether or not the input color information of the candidate human body is the color information of a human body detected in the detection region. The information processing apparatus according to any one of claims 2 to 5.

15. The region is associated with a plurality of the reference color information. The information processing apparatus according to any one of claims 2 to 5.

16. The system further includes an output unit that presents the human body candidate determined to be a human body by the human body determination unit to the user as a human body detection result. The information processing apparatus according to any one of claims 1 to 5.

17. The memory unit stores the reference color information corresponding to the distance from the center position of the image captured by the fisheye camera. The information processing apparatus according to claim 1.

18. Computers A memory step in which color information of the human body is stored in the memory unit as reference color information, A detection step involves detecting potential human body candidates from images captured by a fisheye camera positioned to look down from above, A human body determination step involves obtaining the reference color information corresponding to the detection area in which the human body candidate is detected from the storage unit, and determining whether or not the human body candidate is a human body based on the similarity between the obtained reference color information of the detection area and the color information of the human body candidate. Information processing methods including

19. On the computer, A memory step in which color information of the human body is stored in the memory unit as reference color information, A detection step involves detecting potential human body candidates from images captured by a fisheye camera positioned to look down from above, A human body determination step involves obtaining the reference color information corresponding to the detection area in which the human body candidate is detected from the storage unit, and determining whether or not the human body candidate is a human body based on the similarity between the obtained reference color information of the detection area and the color information of the human body candidate. A program to execute.

Images