Attitude estimation device, attitude estimation system, attitude estimation method and program

Abstract

A posture inference device (10) has a target person detection unit (11), a size determination unit (12), a size adjustment processing unit (13), and a posture detection unit (14). The target person detection unit (11) detects a target person who is a person included in a captured image IMG. The size determination unit (12) determines the size of the target person. The size adjustment processing unit (13), when the size of the target person is out of a predetermined range, performs a process for adjustment such that the size of the target person falls within the predetermined range. The posture detection unit (14) detects the posture of the target person on the basis of an image in which the size of the target person falls within the predetermined range.

Description

[Technical Field]

[0001] The present disclosure relates to a posture estimation device, a posture estimation system, a posture estimation method, and a non-transitory computer-readable medium having a program stored thereon. [Background technology]

[0002] In recent years, methods have been widely used that detect whether a person is captured in an image or video captured by a camera, and then automatically detect the posture of the detected person (Patent Documents 1 to 5).

[0003] For example, a method has been proposed to improve posture detection accuracy by extracting a person's head region from an image, rotating the image so that the orientation of the head is constant, and then detecting the posture (Patent Document 1). Also, a method has been proposed to improve posture detection accuracy by performing super-resolution processing on images with low posture detection accuracy (Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-121045 [Patent Document 2] Japanese Patent Publication No. 2020-201558 [Patent Document 3] Japanese Patent Application Publication No. 2019-110525 [Patent Document 4] Japanese Patent Application Laid-Open No. 2019-29998 [Patent Document 5] Japanese Patent Application Publication No. 2017-73722 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the general posture detection method described above, if the size of the subject reflected in the camera image is too large, the size deviates from the size at which the posture can be suitably detected, and the posture cannot be accurately detected. Also, if the size of the subject reflected in the camera image is too small, the number of pixels required to detect the posture cannot be obtained, and the posture cannot be accurately detected.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to suitably adjust the size of a subject appearing in an image and accurately detect the subject's posture. [Means for solving the problem]

[0007] A posture estimation device according to one aspect of the present disclosure includes a subject detection means for detecting a subject who appears in a captured image, a size determination means for determining the size of the subject, a size adjustment processing means for performing processing to adjust the size of the subject so that it falls within a predetermined range if the size of the subject is outside the predetermined range, and a posture detection means for detecting the posture of the subject based on an image in which the size of the subject falls within the predetermined range.

[0008] A posture estimation system that is one aspect of the present disclosure comprises an imaging device that outputs an image of an area to be monitored, and a posture estimation device that detects the posture of a subject that is a person reflected in the image, wherein the posture estimation device comprises a subject detection means that detects the subject, a size determination means that determines the size of the subject, a size adjustment processing means that, if the size of the subject is outside a predetermined range, performs processing to adjust the size of the subject so that it falls within the predetermined range, and a posture detection means that detects the posture of the subject based on an image in which the size of the subject falls within the predetermined range.

[0009] A posture estimation system that is one aspect of the present disclosure comprises an imaging device that outputs an image of an area to be monitored, and a posture estimation device incorporated in the imaging device that detects the posture of a subject that is a person reflected in the image, wherein the posture estimation device comprises subject detection means that detects the subject, size determination means that determine the size of the subject, size adjustment processing means that, if the size of the subject is outside a predetermined range, performs processing to adjust the size of the subject so that it falls within the predetermined range, and posture detection means that detects the posture of the subject based on an image in which the size of the subject falls within the predetermined range.

[0010] A posture estimation method that is one aspect of the present disclosure detects a subject who is a person reflected in a captured image, determines the size of the subject, and if the size of the subject is outside a predetermined range, performs a process to adjust the size of the subject so that it falls within the predetermined range, and detects the posture of the subject based on an image in which the size of the subject falls within the predetermined range.

[0011] A non-transitory computer-readable medium storing a program that is one aspect of the present disclosure causes a computer to perform the following processes: detecting a subject who is a person reflected in a captured image; determining the size of the subject; adjusting the size of the subject so that it falls within a predetermined range if the size of the subject is outside the predetermined range; and detecting the posture of the subject based on an image in which the size of the subject falls within the predetermined range. [Effects of the Invention]

[0012] According to the present disclosure, the size of a subject reflected in an image can be suitably adjusted, and the posture of the subject can be detected with high accuracy. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a posture estimation system according to a first embodiment. [Figure 2] FIG. 1 is a diagram schematically illustrating a configuration of a posture estimation device according to a first embodiment. [Figure 3] 4 is a flowchart of an attitude detection operation of the attitude estimation system according to the first embodiment. [Figure 4] FIG. 2 is a diagram illustrating in more detail the configuration of a size determining unit and a size adjusting unit according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating a configuration of a posture estimation device according to a second embodiment. [Figure 6] 10 is a flowchart of an attitude detection operation in the attitude estimation system according to the second embodiment. [Figure 7] 10 is a flowchart of an attitude detection operation in the attitude estimation system according to the second embodiment. [Figure 8] FIG. 10 is a diagram showing an example in which the detection area of ​​the current image is not similar to the detection area of ​​the previous image. [Figure 9] FIG. 10 is a diagram showing an example in which the detection area of ​​the current image is similar to the detection area of ​​the previous image. [Figure 10] FIG. 10 is a diagram illustrating a configuration of a posture estimation system according to a third embodiment. [Figure 11] FIG. 1 illustrates an example of the configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same elements are designated by the same reference numerals, and redundant explanations will be omitted as necessary.

[0015] Embodiment 1 A posture estimation system according to a first embodiment will be described. FIG. 1 schematically illustrates the configuration of a posture estimation system 100 according to the first embodiment. The posture estimation system 100 includes a camera 110 and a posture estimation device 10. The camera 110 is configured as an imaging device that continuously captures images of a monitoring target area Z, for example, as a video, and outputs the captured images IMG to the posture estimation device 10. The camera 110 may be capable of capturing images in a non-visible light range such as infrared light, not limited to visible light, and may also be capable of projecting illumination light onto the monitoring target area Z as appropriate. The posture estimation device 10 detects a target person who appears in the captured images IMG, performs image processing as necessary, and detects the posture of the detected target person. The camera 110 and the posture estimation device 10 are connected by various communication means, including wired and wireless communication.

[0016] The posture estimation device 10 will now be described. Fig. 2 schematically shows the configuration of the posture estimation device 10 according to the first embodiment. The posture estimation device 10 includes a subject detection unit 11, a size determination unit 12, a size adjustment processing unit 13, and a posture detection unit 14.

[0017] The subject detection unit 11 refers to the image IMG received from the camera 110, determines whether a subject is reflected in the image IMG, and detects the subject if the subject is reflected. Determining whether a subject is reflected in the image IMG may be done using predetermined conditions that determine whether an object reflected in the image is a person, or may be done by inputting the image IMG into a trained model constructed by various types of machine learning to detect an object that is estimated to be the subject. Specifically, for example, the YOLO (You Look Only Once) method, the SSD (Single Shot multibox Detector) method, or the Faster-RCNN (Region-based Convolutional Neural Networks) method may be used to detect the subject. The subject detection unit 11 may detect the subject's outline, or may detect the subject as a collection of images of the area occupied by the subject. The subject detection unit 11 outputs information INF indicating the detected subject and the image IMG in which the detected subject is reflected to the size determination unit 12.

[0018] The size determination unit 12 identifies an area corresponding to the detected subject H1 reflected in the image IMG based on the information INF indicating the detected subject. Thereafter, the size determination unit 12 determines the size of the identified subject H1 and outputs the determination result RES to the size adjustment processing unit 13 together with the information INF indicating the detected subject and the image IMG.

[0019] Based on the determination result, the size adjustment processing unit 13 performs processing to change the size of the subject H1 reflected in the image IMG as necessary. Note that, depending on the determination result, the size adjustment processing unit 13 may not perform processing to change the size of the subject H1 reflected in the image IMG. Thereafter, the size adjustment processing unit 13 outputs information INF indicating the subject and the size-adjusted image or the size-unadjusted image to the posture detection unit 14.

[0020] The posture detection unit 14 detects the posture of the identified subject H1 based on the information INF indicating the detected subject. The posture detection may be performed based on predetermined conditions that determine the posture of the subject reflected in the image, or the posture of the subject may be estimated by inputting the image of the subject into a trained model constructed by various types of machine learning.

[0021] Next, a description will be given of the posture detection operation in posture estimation system 100. Fig. 3 is a flowchart of the posture detection operation in posture estimation system 100 according to the first embodiment. Fig. 4 shows the configurations of size determination unit 12 and size adjustment processing unit 13 in more detail.

[0022] Step S11 The camera 110 is positioned at a location where the camera 110 is installed, and the area to be monitored is the area to which the camera 110 is directed. Z The image IMG is acquired and output to the subject detection unit 11 of the posture estimation device 10.

[0023] Step S12 The subject detection unit 11 performs a process of detecting a subject reflected in the image IMG received from the camera 110. If a subject is reflected in the image IMG, the subject detection unit 11 outputs information INF indicating the detected subject and the image IMG to the size determination unit 12.

[0024] Step S13 When the subject detection unit 11 detects the subject H1, the first threshold determination unit 12A of the size determination unit 12 determines whether the size L of the subject H1 in the image IMG is equal to or smaller than the first threshold L. TH1 It is determined whether the size L of the subject H1 in the image IMG is greater than the first threshold L TH1 If the size L of the object H1 in the image IMG is larger than the first threshold L, the first threshold determination unit 12A outputs the image IMG and information INF indicating the detected object to the reduction processing unit 13A of the size adjustment processing unit 13. TH1In the following cases, the first threshold determination unit 12A outputs the image IMG and information INF indicating the detected subject to the second threshold determination unit 12B of the size determination unit 12.

[0025] Step S14 In step S13, the size L of the target person H1 is set to a first threshold L TH1 If it is determined that the size L of the target person H1 is larger than the maximum image size L MAX The image IMG including the subject H1 is reduced in size as follows: MAX is the first threshold L TH1 It can be set to any of the following values: Then, the reduction processing unit 13A outputs the image IMG′ after the reduction processing and information INF′ indicating the subject H1 in the image IMG′ to the posture detection unit 14.

[0026] Step S15 In step S13, the size L of the target person H1 is set to a first threshold L TH1 If it is determined that the size L of the subject H1 is equal to or smaller than the second threshold value L, the second threshold value determination unit 12B further determines whether the size L of the subject H1 is equal to or smaller than the second threshold value L TH2 It is determined whether the size L of the subject H1 is smaller than the second threshold L TH2 If the size L of the target person H1 is smaller than the second threshold value L, the second threshold value determination unit 12B outputs the determination result DET to the camera control unit 13B of the size adjustment processing unit 13. TH2 In the above cases, the second threshold determination unit 12B outputs the image IMG and information INF indicating the detected subject to the posture detection unit .

[0027] Step S16 In step S15, the size L of the target person is set to a second threshold L TH2 If it is determined that the size L of the subject H1 is smaller than the minimum image size L MIN The PTZ control of the camera 110 is performed so that the minimum image size L MIN is the second threshold L TH2The above values ​​can be set as any of the above. Specifically, the camera control unit 13B commands the camera 110 to zoom in on the subject, i.e., optically enlarge the subject, and then capture the subject again so that the subject H1 has a desired size in the image captured by the camera 110. Furthermore, if the subject H1 extends beyond the image as a result of the camera 110 zooming, the camera control unit 13B may command the camera 110 to change P (pan, angle of view) and T (tilt, elevation / depression angle) as appropriate. Thereafter, the recaptured image IMG is output to the posture detection unit 14.

[0028] Here, the significance of re-imaging the subject after optically enlarging the subject in camera 110 will be explained. Generally, when the subject is simply enlarged by image processing or the like (for example, so-called digital zoom), the contours and shading of the subject's area become unclear. Therefore, if posture detection is performed based on such an image, the accuracy of posture detection decreases. Therefore, in this embodiment, the subject is optically enlarged and then re-imaged. This allows the subject to be captured as a high-definition image, and posture detection can be performed based on this, thereby achieving highly accurate posture detection.

[0029] Meanwhile, in this embodiment, if the subject is excessively large, the subject is simply reduced in size by image processing, etc. This is because, in general, simply reducing the image does not blur the contours or shading of the subject's area, and there is no risk of a decrease in the accuracy of posture detection, or the risk is extremely low.

[0030] Step S17 If the subject is not detected in step S12, after step S14, if the judgment result in step S15 is NO, or after step S16, the posture detection unit 14 performs posture detection processing of the subject H1 reflected in the image based on the image and information indicating the subject received from the size adjustment processing unit 13.

[0031] As described above, when detecting the posture of a subject reflected in an image, if the subject's size is excessively large, the image can be reduced so that the subject's size falls within a desired range.

[0032] Furthermore, if the size of the subject is too small, the camera's zoom function can be used to obtain an enlarged image of the detected subject. Then, by performing the posture detection operation shown in Fig. 3 again based on the obtained image, the posture of the subject can be detected using an image in which the size of the subject falls within a desired range.

[0033] As described above, according to this configuration, when the size of the subject in the image is equal to or smaller than the first threshold L TH1 and the second threshold L TH2 If the image size does not fall within the range specified by and, the minimum image size L MIN and maximum image size L MAX The size of the image can be adjusted to a desired range suitable for attitude control, as defined by the formula (1). If no size adjustment is required, the image with its size unadjusted can be used as is, and if size adjustment is required, the image with its size adjusted appropriately can be used for attitude detection. This allows for more accurate attitude detection.

[0034] Embodiment 2 A posture estimation device according to a second embodiment will now be described. Fig. 5 schematically shows the configuration of posture estimation device 20 according to the second embodiment. Posture estimation device 20 has a configuration in which a detection area selection unit 21, a score evaluation unit 22, a detection result integration unit 23, and a storage unit 24 are added to posture estimation device 10.

[0035] The detection area selection unit 21 compares the detection area in which the subject is detected in the image IMG acquired by the camera 110 with the detection area in the image IMG_P acquired immediately before the image IMG, and selects one of them as the detection area to be subjected to posture detection.

[0036] The score evaluation unit 22 evaluates whether the detected object is a target person or not based on the posture detection result, and assigns a score S to the object.

[0037] As will be described later, the detection result integration unit 23 integrates two posture detection results acquired under different conditions.

[0038] The memory unit 24 stores in advance information about the previous image IMG_P and its detection area. The score evaluation unit 22 also stores the score S assigned to the current image IMG and its detection area A in the memory unit 24. The stored score S and detection area A will be used as the previous image IMG_P and its detection area for the image captured by the camera 110 the next time.

[0039] Next, a description will be given of an attitude detection operation in the attitude estimation system according to the embodiment 2. Figures 6 and 7 are flowcharts of the attitude detection operation in the attitude estimation system 200 according to the embodiment 2. In the attitude detection operation in the attitude estimation system 200 according to the embodiment 2, steps S21 to S29 are added to steps S11 to S17 in Figure 3.

[0040] Step S11 As in the first embodiment (FIG. 3), camera 110 acquires image IMG of monitoring area Z toward which camera 110 is directed at the installation position of camera 110, and outputs it to subject detection unit 11 of posture estimation device 20.

[0041] Step S12 As in the first embodiment (FIG. 3), the subject detection unit 11 performs a process of detecting a subject reflected in the image IMG received from the camera 110. If a subject is reflected in the image IMG, the subject detection unit 11 outputs information INF indicating the detected subject and the image IMG to the size determination unit 12 and the posture detection unit 14.

[0042] Step S21 If no target person is detected in step S12, the posture detection unit 14 performs posture detection processing on the image IMG acquired by the camera 110 to detect the posture of the target person captured in the image.

[0043] Step S22 If a subject is detected in step S12, the detection area selection unit 21 determines whether the detection area A in which the subject H1 is detected in the current image IMG is similar to the detection area A_P in which the subject H1 is detected in the previous image IMG_P, i.e., the image IMG_P acquired most recently before the current image IMG. Determining whether two detection areas are similar can be achieved, for example, by comparing the positions and sizes of the detection areas. More specifically, comparison may be made based on, for example, the position of the detection area on the image, the ratio of the detection area to the image, or the number of vertical and horizontal pixels of the detection area on the image. Note that the method for determining and comparing the positions of the detection areas is not limited to the above, and various other methods may be used as appropriate, such as comparing the center coordinates of the detection areas or the coordinates of the upper left corners of the detection areas. Note that, hereinafter, the current image IMG will be referred to as the first image, the previous image IMG_P as the second image, the detection area A in the current image IMG as the first detection area, and the detection area A_P in the previous image IMG_P as the second detection area.

[0044] A specific example will be used to explain a case where two detection areas are dissimilar. Fig. 8 shows an example where detection area A of the current image IMG is dissimilar to detection area A_P of the previous image IMG_P. In this example, a subject H1 reflected in the previous image IMG_P is detected as the subject, and detection area A_P is set. In contrast, in the current image IMG, an object OBJ, rather than subject H1, is mistakenly detected as the subject, and detection area A is set. Since the positions of detection area A and detection area A_P are significantly different, detection area selection unit 21 determines that they are dissimilar.

[0045] Next, a specific example will be used to explain a case where two detection areas are similar. FIG. 9 shows an example where detection area A of the current image IMG is similar to detection area A_P of the previous image IMG_P. In this example, a subject H1 who appears in the previous image IMG_P is detected as the subject, and detection area A_P is set. Also, in the current image IMG, subject H1 is detected as the subject, and detection area A is set. Note that since the current image IMG was acquired later in time than the previous image IMG_P, the position of subject H1 has changed, but the timing (sampling rate) for acquiring images when posture detection is performed is sufficiently frequent. Therefore, detection area A and detection area A_P in this case are set at positions sufficiently close to each other. Therefore, in this case, detection area selection unit 21 determines that the positions of detection area A and detection area A_P are similar.

[0046] Step S23 If the detection area A of the current image IMG is not similar to the detection area A_P of the previous image IMG_P, the detection area selection unit 21 determines whether the value of the score SP assigned to the previous image IMG_P is positive (+). As will be described later, if the object in the detection area is recognized as a person, the score SP of the previous image IMG_P is a positive (+) value, and if it is not recognized as a person, the score SP of the previous image IMG_P is a negative (-) value. If the value of the score SP is negative (-), the process proceeds to step S24, and if the value of the score SP is positive (+), the process proceeds to step S25.

[0047] Step S24 If it is determined that the detection area A of the current image IMG is similar to the detection area A_P of the previous image IMG_P (S22: YES), or if the value of the score SP is negative (-) (S23: NO), the detection area selection unit 21 uses the detection area A as is as the detection area of ​​the subject of the image IMG. If the value of the score SP is negative (-) (S23: NO), there is a high possibility that the subject in the detection area A_P of the previous image IMG_P is not the subject, so here, the detection area A newly set in the current image IMG is used.

[0048] Step S25 If the value of the score SP is positive (+) (S23: YES), the detection area selection unit 21 uses the detection area A_P in the previous image IMG_P instead of the detection area A as the detection area of ​​the subject in the image IMG. Here, the significance of changing the detection area will be explained. As shown in FIG. 8, if the detection area A and the detection area A_P are dissimilar, it is assumed that the subjects inside the two are different. Also, while it is not evaluated whether the subject in the detection area of ​​the current image IMG is a subject or not, the subject in the detection area A_P of the previous image IMG_P is recognized as a person based on the score SP, so here, the detection area A Instead, a detection area A_P in which a person has been detected with high reliability is set in the image IMG, and the subsequent processing is performed.

[0049] Steps S13 to S17 After step S24 or S25, the processes of steps S13 to S17 are carried out in the same manner as in FIG.

[0050] Step S26 After step S17, the score evaluation unit 22 evaluates whether the object whose posture has been detected corresponds to a human subject. Here, the score evaluation unit 22 determines whether, for example, a human skeleton can be found in the object whose posture has been detected. Various methods can be used to find the human skeleton.

[0051] Here, the score will be explained. In this embodiment, the size determination unit 12 detects a subject in an image. As described above, the size determination unit 12 detects an "object" subject reflected in the image based on predetermined conditions and a trained model constructed by machine learning. However, it is conceivable that an object other than a person reflected in the image may be mistakenly detected as a subject. Therefore, it is determined whether the detected subject has characteristics specific to a human, for example, skeletal characteristics of a human body. Then, based on the determination result, a score indicating the degree to which the subject is estimated to be a human is assigned to express the reliability of the subject detection result.

[0052] Step S27 If a human skeleton is found in the object for which posture detection has been performed, i.e., if the object for which posture detection has been performed is determined to be a person, the score evaluation unit 22 assigns a positive (+) value as the score S to the current image IMG.

[0053] Step S28 If a human skeleton cannot be found for the object for which posture detection has been performed, that is, if it is determined that the object for which posture detection has been performed is not a person, the score evaluation unit 22 assigns a score S to the current image IMG. negative A value of (-) is assigned.

[0054] Step S29 Detection result integration unit 23 integrates the posture detection result RES1 obtained in step S17 and the posture detection result RES2 obtained in step S21. The integration of the detection results may be performed by simply taking the sum of posture detection result RES1 and posture detection result RES2, or may include additional processing to combine similar skeletons between posture detection result RES1 and posture detection result RES2 into one.

[0055] As described above, with this configuration, it is possible to detect the posture of the target person based on the case where it is estimated that the person has been correctly detected as the target person, thereby further improving the accuracy of detecting the posture of the target person.

[0056] Embodiment 3 In the first embodiment, the pose estimation system 100 has been described as being configured with a camera 110 and a pose estimation device 10 configured as a separate device from the camera 110. In contrast, in the present embodiment, an example will be described in which the camera and the pose estimation device are configured as a single system.

[0057] A posture estimation system 300 according to the third embodiment will be described. Fig. 10 schematically shows the configuration of the posture estimation system 300 according to the third embodiment. The posture estimation system 300 has a configuration in which the posture estimation apparatus 10 is built into a camera 310.

[0058] Recent imaging devices capture images using a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. Therefore, imaging devices are equipped with processing devices with high computing power. Therefore, in this embodiment, the functions of the posture estimation device 10 are realized by a processing device installed in the imaging device or a processing unit that can be additionally implemented in the processing device, so that the posture estimation device 10 is built into the camera 310 itself.

[0059] This makes it possible to provide a posture estimation system 300 that includes a camera 310 and a posture estimation device 10 built into the camera 310. This makes it possible to realize a more compact posture estimation system.

[0060] Other embodiments The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, in the third embodiment, the posture estimation device 10 is built into the camera 310, but the posture estimation device 20 according to the second embodiment may be built into the camera 310.

[0061] In the above-described embodiment, the present invention has been described as being configured using hardware, but the present invention is not limited to this. The present invention can also be implemented by having a central processing unit (CPU) execute a computer program to perform processing in a processing device. The above-described program can be stored on various types of non-transitory computer-readable media and supplied to a computer. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and random access memories (RAMs)). The program may also be supplied to a computer via various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0062] An example of a computer will be described below. The computer can be any of a variety of computers, such as a dedicated computer or a personal computer (PC). However, the computer does not need to be a single physical computer; multiple computers may be used when performing distributed processing.

[0063] An example of the configuration of a computer is shown in Fig. 11. The computer 1000 in Fig. 11 has a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003, which are interconnected via a bus 1004. Note that although explanation of the OS software for operating the computer will be omitted, it is assumed that the computer that constitutes this information processing device also has these.

[0064] An input / output interface 1005 is also connected to the bus 1004. To the input / output interface 1005, for example, an input unit 1006 including a keyboard, mouse, sensor, etc., a display including a CRT, LCD, etc., an output unit 1007 including headphones, speakers, etc., a storage unit 1008 including a hard disk, etc., and a communication unit 1009 including a modem, terminal adapter, etc. are connected.

[0065] The CPU 1001 executes various processes in accordance with various programs stored in the ROM 1002 or various programs loaded from the storage unit 1008 to the RAM 1003. In the above-described embodiment, the CPU 1001 executes various processes, for example, the processes of the various units of the information processing device 100 described below. A graphics processing unit (GPU) may be provided to execute various processes, similar to the CPU 1001, in accordance with various programs stored in the ROM 1002 or various programs loaded from the storage unit 1008 to the RAM 1003. In the present embodiment, the GPU executes various processes, for example, the processes of the various units of the information processing device 100 described below. The GPU is suitable for performing routine processing in parallel, and by applying it to neural network processing, for example, as described below, it is possible to improve processing speed compared to the CPU 1001. The RAM 1003 also stores data necessary for the CPU 1001 and the GPU to execute various processes.

[0066] The communication unit 1009 performs communication processing via the Internet (not shown), for example, transmits data provided by the CPU 1001, and outputs data received from a communication partner to the CPU 1001, RAM 1003, and storage unit 1008. The storage unit 1008 exchanges data with the CPU 1001 and stores and erases information. The communication unit 1009 also performs communication processing of analog or digital signals with other devices.

[0067] The input / output interface 1005 is also connected to a drive 1010 as needed, and, for example, a magnetic disk 1011, an optical disk 1012, a flexible disk 1013, or a semiconductor memory 1014 is appropriately attached, and computer programs read from these are installed in the memory unit 1008 as needed.

[0068] In the above-described embodiment, a determination of the magnitude of two values ​​has been described. However, this is merely an example, and cases where the two values ​​are equal in a determination of the magnitude of two values ​​may be handled as needed. That is, either the determination of whether a first value is greater than or equal to a second value or less than the second value, or the determination of whether a first value is greater than or equal to the second value, may be adopted as needed. Either the determination of whether a first value is less than or equal to a second value or greater than the second value, or the determination of whether a first value is less than or equal to the second value or greater than or equal to the second value, may be adopted. In other words, when two values ​​are determined to be greater than or equal to each other to obtain two determination results, a case where the two values ​​are equal may be included in either of the two determination results as needed. [Explanation of symbols]

[0069] 10, 20 Posture estimation device 11. Target detection unit 12 Size determination section 12A First threshold judgment unit 12B Second threshold determination unit 13 Size adjustment processing section 13A Reduction processing section 13B Camera control unit 14 Attitude detection unit 21 Detection area selection section 22 Score Evaluation Section 23 Detection result integration unit 24 Memory section 100, 300 Pose estimation system 110, 310 camera 1000 computers 1001 CPU 1002 ROM 1003 RAM 1004 Bus 1005 Input / Output Interface 1006 Input section 1007 Output section 1008 Storage section 1009 Communications Department 1010 Drive 1011 Magnetic Disk 1012 Optical disc 1013 Flexible Disk 1014 Semiconductor Memory A, A_P detection area DET judgment result H1 Target Audience IMG, IMG_P images INF information L MAX Maximum image size L MIN Minimum image size L TH1 First Threshold L TH2 Second Threshold OBJ object RES judgment result S, SP score Z Surveillance area

Claims

1. a subject detection means for detecting a subject who is a person reflected in a captured image; a size determination means for determining the size of the subject; a size adjustment processing means for performing a process of adjusting the size of the target person so that the size of the target person falls within a predetermined range when the size of the target person is outside the predetermined range; a posture detection means for detecting a posture of the subject based on an image in which the size of the subject falls within a predetermined range; a storage means for storing a score indicating the degree to which a subject detected in a second image captured immediately before the first image, which is the captured image, is estimated to be a person, and information indicating a second detection area in which the subject detected in the second image is detected; a detection area selection means for determining whether the second detection area of ​​the second image is similar to a first detection area in which the subject is detected in the first image, and selecting either the first or second detection area as a detection area to be applied to the first image based on the determination result and the score indicating the degree to which the subject detected in the second image is estimated to be a person; a score evaluation means for calculating a score indicating a degree to which the subject is estimated to be a human after the posture detection means detects the posture of the subject in the first image, the size determination means determines the size of the subject in the detection area selected by the detection area selection means; Posture estimation device.

2. the size determination means determines whether the size of the subject is larger than an upper limit value of the predetermined range; When the size of the subject is larger than the upper limit of the predetermined range, the size adjustment processing means reduces the image so that the size of the subject becomes smaller than the upper limit of the predetermined range. The posture estimation device according to claim 1 .

3. the size determination means determines whether the size of the subject is smaller than a lower limit value of the predetermined range; When the size of the subject is smaller than the lower limit of the predetermined range, the size adjustment processing means controls the imaging device that captured the image so as to optically enlarge the subject and capture the image. The posture estimation device according to claim 1 or 2.

4. When the size adjustment processing means controls the imaging device that captured the image so as to enlarge the image of the subject, the posture detection means detects the posture of the subject based on the image captured after the control. The posture estimation device according to claim 3 .

5. The detection area selection means If the second detection area and the first detection area are similar, select the second detection area as the detection area to be applied to the first image; If the second detection area and the first detection area are dissimilar, select the first detection area as the detection area to be applied to the first image. The posture estimation device according to claim 1 .

6. an imaging device that outputs an image of an area to be monitored; a posture estimation device for detecting a posture of a subject who is a person reflected in the image, The posture estimation device includes: a subject detection means for detecting the subject; a size determination means for determining the size of the subject; a size adjustment processing means for performing a process of adjusting the size of the target person so that the size of the target person falls within a predetermined range when the size of the target person is outside the predetermined range; a posture detection means for detecting a posture of the subject based on an image in which the size of the subject falls within a predetermined range; a storage means for storing a score indicating the degree to which a subject detected in a second image captured immediately before the first image, which is the captured image, is estimated to be a person, and information indicating a second detection area in which the subject detected in the second image is detected; a detection area selection means for determining whether the second detection area of ​​the second image is similar to a first detection area in which the subject is detected in the first image, and selecting either the first or second detection area as a detection area to be applied to the first image based on the determination result and the score indicating the degree to which the subject detected in the second image is estimated to be a person; a score evaluation means for calculating a score indicating a degree to which the subject is estimated to be a human after the posture detection means detects the posture of the subject in the first image, the size determination means determines the size of the subject in the detection area selected by the detection area selection means; Pose estimation system.

7. an imaging device that outputs an image of an area to be monitored; a posture estimation device incorporated in the imaging device that detects a posture of a subject who is a person reflected in the image, The posture estimation device includes: a subject detection means for detecting the subject; a size determination means for determining the size of the subject; a size adjustment processing means for performing a process of adjusting the size of the target person so that the size of the target person falls within a predetermined range when the size of the target person is outside the predetermined range; a posture detection means for detecting a posture of the subject based on an image in which the size of the subject falls within a predetermined range; a storage means for storing a score indicating the degree to which a subject detected in a second image captured immediately before the first image, which is the captured image, is estimated to be a person, and information indicating a second detection area in which the subject detected in the second image is detected; a detection area selection means for determining whether the second detection area of ​​the second image is similar to a first detection area in which the subject is detected in the first image, and selecting either the first or second detection area as a detection area to be applied to the first image based on the determination result and the score indicating the degree to which the subject detected in the second image is estimated to be a person; a score evaluation means for calculating a score indicating a degree to which the subject is estimated to be a human after the posture detection means detects the posture of the subject in the first image, the size determination means determines the size of the subject in the detection area selected by the detection area selection means; Pose estimation system.

8. Detecting a person who is a target person reflected in the captured image, Based on information indicating a second detection area in which a subject detected in a second image captured one image prior to the first image, which is the captured image, that is, the captured image, it is determined whether the second detection area is similar to the first detection area in which the subject detected in the first image; selecting one of the first and second detection areas as a detection area to be applied to the first image based on the determination result and a predetermined score indicating the degree to which the subject detected in the second image is estimated to be a human; determining a size of the subject in the selected detection area; If the size of the subject is outside a predetermined range, performing a process of adjusting the size of the subject so that it falls within the predetermined range; Detecting a posture of the subject based on the first image in which the size of the subject falls within a predetermined range; After detecting the posture of the subject in the first image, calculate a score indicating the degree to which the subject is estimated to be a human. Pose estimation method.

9. A process of detecting a target person who is a person reflected in the captured image; a process of determining whether the second detection area and the first detection area in which the subject is detected in the first image are similar based on information indicating a second detection area in which the subject is detected in a second image captured one image prior to the first image, which is the captured image, are similar to each other; a process of selecting one of the first and second detection areas as a detection area to be applied to the first image based on the determination result and a predetermined score indicating the degree to which the subject detected in the second image is estimated to be a person; determining the size of the subject in the selected detection area; If the size of the subject is outside a predetermined range, adjusting the size of the subject so that it falls within the predetermined range; detecting a posture of the subject based on the first image in which the size of the subject falls within a predetermined range; and calculating a score indicating the degree to which the subject is estimated to be a human after detecting the posture of the subject in the first image. program.

Images