Information processing device, information processing method, and program

Abstract

To properly estimate the number of objects being counting targets even when there exists a dead angle region in a region of the counting target.SOLUTION: An information processing device comprises: detection means which detects an object being a counting target from an input image; storage means which stores the behavior of the object detected by the detection means; determination means which determines whether or not the object that becomes non-detection has moved to a dead angle region from a non-dead angle region on the basis of the behavior of the object stored by the storage means when the object detected by the detection means becomes non-detection; and counting means which counts the number of objects in the region of the counting target in the input image on the basis of the detection result of the object by the detection means and the determination result by the determination means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for counting the number of people based on images captured by a surveillance camera installed on the ceiling. Patent Document 1 discloses an elevator system that detects the number of passengers in an elevator hall, and estimates the number of passengers in a blind spot of a surveillance camera based on elevator control information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-166066 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology described in Patent Document 1, the accuracy of estimating the number of passengers in the blind spot area decreases over time. Therefore, an object of the present invention is to appropriately estimate the number of objects to be counted even when a blind spot area exists in the area to be counted. [Means for solving the problem]

[0005] In order to solve the above problem, one aspect of the information processing device according to the present invention includes a detection unit that detects objects to be counted from an input image, a storage unit that stores behavior of the objects detected by the detection unit, and a processing unit that stores the behavior of the objects detected by the detection unit. was When the object becomes undetected, the object that became undetected is detected based on the behavior of the object stored in the storage means. SpecificThe apparatus further comprises a determination means for determining whether or not an object has moved into a certain area, and a counting means for counting the number of objects in the area to be counted in the input image based on the detection result of the object by the detection means and the determination result by the determination means. Effect of the Invention

[0006] According to the present invention, even if a blind spot area exists in the area to be counted, the number of objects to be counted can be appropriately estimated. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing an example of the arrangement of an imaging apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of installation of an imaging device. [Diagram 3] FIG. [Figure 4] 4 is a flowchart of a process executed by the imaging apparatus according to the first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of human body detection information. [Figure 6] FIG. 4 is a diagram showing a coordinate system of a captured image. [Figure 7] FIG. 4 is a diagram showing an example of human body behavior information. [Figure 8] FIG. 4 is a diagram showing an example of human body detection information. [Figure 9] FIG. 4 is a diagram showing an example of human body detection information. [Figure 10] 5 is a flowchart of the process executed in S4 of FIG. 4. [Figure 11] 1A and 1B are diagrams illustrating movement of a human body from a non-blind spot area to a blind spot area. [Figure 12] 10 is a flowchart of a process executed by an imaging apparatus according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the embodiment described below is one example of a means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the embodiment described below.

[0009] In this embodiment, a camera system is described that counts the number of objects to be counted that exist in a region to be counted in an input image, using an image (video) captured by an imaging device as an input image. The camera system in this embodiment includes an information processing device that counts the number of passengers in a seating region, where the objects to be counted are human bodies and the region to be counted is a seating region in a vehicle such as a railway car.

[0010] FIG. 1 is a block diagram showing an example of the configuration of a camera system 1000 according to this embodiment. The camera system 1000 includes an imaging device 100. The imaging device 100 can be a network camera that captures image data or video data through a lens. The imaging device 100 can be a lens-interchangeable imaging device in which an imaging unit 200 having an imaging optical system and a driving unit 300 are interchangeable. The imaging device 100 can also be a lens-integrated imaging device in which the imaging unit 200 and the driving unit 300 are integrally formed. In response to a request from a client, the imaging device 100 can transmit captured image data, etc. to the client device 500 via the network 400. In addition, the imaging device 100 may actively transmit image data, etc. to the client device 500 that is connected in advance.

[0011] The network 400 is composed of, for example, a plurality of routers, switches, cables, etc. that comply with the Ethernet (registered trademark) communication standard. Note that the network 400 may be configured in any communication standard, scale, or configuration as long as it is capable of communicating between the imaging device 100 and the above-mentioned client devices. The network 400 may be realized by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or a combination of these. The client device 500 can be configured with a general terminal device such as a personal computer (PC), a smartphone, a tablet PC, etc. The client device 500 may be a server device or a dedicated controller device for operating a remote camera.

[0012] FIG. 2 is a diagram showing an example of installation of the imaging device 100. As shown in FIG. The imaging device 100 is installed on the ceiling of the vehicle 600, and counts the number of passengers present in a seating area 601 to grasp the degree of congestion of seats A to H, for example. The vehicle 600 is provided with a seating area 601 and a non-seat area 602. Here, the non-seat area 602 can be an aisle provided between seats A to D and seats E to H. Passengers can enter and exit the vehicle 600 through an entrance / exit 603 or an entrance / exit 604, move around the non-seat area 602, and take or leave seats A to H provided in the seating area 601. The imaging device 100 is installed at an end inside the vehicle 600, specifically, on the ceiling near the entrance / exit 603.

[0013] As shown in FIG. 3, each of the seats A to H is provided with a backrest 610. Therefore, when the imaging device 100 is installed at the edge of the ceiling of the vehicle 600, a blind spot 620 that cannot be imaged by the imaging device 100 occurs in the seat area 601 depending on the positional relationship between the imaging device 100 and the backrest 610. When a passenger sits down and enters the blind spot 620, the imaging device 100 cannot image the passenger, and the passenger cannot be detected from the captured image. Therefore, if the number of human bodies detected in the area corresponding to the seat area 601 in the captured image is counted as the number of passengers seated in the seats, it may be erroneously determined that the seats are vacant even though the passengers are seated, and accurate counting of the number of people is not possible.

[0014] In this embodiment, the imaging device 100 stores the behavior of the human body detected from the captured image. Then, when a human body that was detected in a non-blind spot area that can be imaged by the imaging device 100 becomes undetected, it is determined whether the undetected human body has moved into the blind spot area based on the behavior of the human body before it became undetected. Then, the imaging device 100 counts the number of people in the area to be counted, taking into account the determination result. In this way, even if a human body moves into the blind spot area, accurate counting of the number of people is performed.

[0015] The following describes the configuration of each unit shown in Fig. 1. First, the imaging unit 200 and the driving unit 300 will be described. The imaging unit 200 includes a zoom lens 201, a focus lens 202, an aperture 203, and an image sensor 204, which are an imaging optical system. The driving unit 300 includes a lens driving unit 301 and an image sensor driving unit 302. The lens driving unit 301 moves the positions of the zoom lens 201 and the focus lens 202 along the optical axis based on the focus / zoom setting positions instructed by a zoom / focus control unit 101a (described later). The lens driving unit 301 also drives the aperture 203.

[0016] The image sensor driver 302 tilts the image sensor 204 based on the set position of the flap angle instructed by the zoom / focus controller 101a, which will be described later. Specifically, the rotation axis for tilting the image sensor 204 is located at the center of the imaging screen, and the image sensor 204 tilts around the rotation axis. The image sensor 204 photoelectrically converts light that has passed through the zoom lens 201, the focus lens 202, and the aperture 203 to generate an analog image signal. The generated analog image signal is subjected to amplification processing by sampling processing such as correlated double sampling, and then output to the A / D converter 106, which will be described later.

[0017] The imaging device 100 includes a CPU 101, a ROM 102, a RAM 103, a storage device 104, an I / F 105, an A / D conversion unit 106, a camera signal processing unit 107, an image analysis unit 108, and a compression / decompression unit 109. The CPU 101 includes a zoom / focus control unit 101a. The image analysis unit 108 further includes a human body detection unit 111, a human body behavior storage unit 112, a blind spot area movement determination unit 113, a seat area detection unit 114, and a people counting unit 115.

[0018] The CPU 101 performs overall control of the operation of the imaging apparatus 100. The zoom / focus control unit 101a performs focus control using autofocus (AF) or manual focus (MF). ROM 102 is a non-volatile memory such as an EEPROM or a flash memory. ROM 102 stores programs and data necessary for CPU 101 to execute processing. The programs may be stored in storage device 104 or a removable storage medium (not shown). RAM 103 is a volatile memory such as SRAM or DRAM, and functions as a main memory, a work area, etc. of CPU 101. When executing processing, CPU 101 loads necessary programs, etc. from ROM 102 into RAM 103 via internal bus 110, and executes the programs, etc. to realize various functional operations. The storage device 104 is a storage device such as an HDD (hard disk drive), an SSD (solid state drive), or an eMMC (embedded multimedia card).

[0019] The I / F 105 is a network interface for connecting to the network 400 . The A / D conversion unit 106 converts the analog image signal output from the imaging unit 200 into a digital image signal, and outputs it to the camera signal processing unit 107 . The camera signal processor 107 performs various image processing on the digital image signal to generate a captured image (video). The various image processing includes, for example, offset processing, gamma correction processing, gain processing, RGB interpolation processing, noise reduction processing, contour correction processing, color correction processing, light source type determination processing, and the like.

[0020] The image analysis unit 108 performs image analysis such as human body detection and moving object detection using the captured image generated by the camera signal processing unit 107 as an input image. The human body detection unit 111 detects a specific human body from a captured image based on image analysis processing according to an existing algorithm. A method for detecting a human body includes a method of extracting at least the external shape information of an object as a feature amount and performing a pattern matching process. The external shape information is trajectory information indicating the external shape of an object, and can be Ω-shaped trajectory information from the head to both shoulders. Alternatively, human body detection may be performed using image analysis processing such as face recognition. Furthermore, the human body detection unit 111 detects and identifies the human body at the same time, and assigns an ID to each detected human body. For example, the human body may be identified by extracting a color histogram of the detected human body and identifying the human body from the color histogram. Note that the methods of human body detection and human body identification are not limited to those described above, and any known method may be appropriately adopted.

[0021] The human body behavior storage unit 112 stores the behavior of the human body detected by the human body detection unit 111. Specifically, the human body behavior storage unit 112 stores in the RAM 103, for each frame, a history of the position of the human body detected by the human body detection unit 111 in the captured image, for example, a history of the central coordinates of the outline of the human body. When a human body detected by the human body detection unit 111 becomes undetected, the blind spot area movement determination unit 113 determines whether the undetected human body has moved from a non-blind spot area to a blind spot area. Specifically, the blind spot area movement determination unit 113 estimates a moving position of the human body after becoming undetected based on the behavior of the human body stored before becoming undetected, and determines whether the moving position is a blind spot area.

[0022] The seat area detection unit 114 detects a seat area in a captured image. The seat area detection unit 114 may detect the seat area by acquiring information indicating a seat area specified by a user, or may detect the seat area in a captured image by image analysis. The people counting unit 115 counts the number of human bodies present in a region to be counted in the captured image. In this embodiment, the region to be counted is the seating region 601. The region to be counted may be a predetermined region in the vehicle 600, and may be a part or the whole of the region including the seating region 601 and the non-seat region 602.

[0023] The image analysis unit 108 notifies the CPU 101 of the results of the above processing via the internal bus 110. Note that some or all of the functions of the image analysis unit 108 can be realized by the CPU 101 executing a program. However, at least some of the elements of the image analysis unit 108 may be configured to operate as dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 101. The compression / decompression unit 109 performs compression processing on the captured image to generate compressed data in accordance with a control instruction from the CPU 101 via the internal bus 110. The compressed data is transmitted from the I / F 105 to the client device 500 via the network 400.

[0024] The client device 500 includes a CPU 501 , a ROM 502 , a RAM 503 , an I / F 504 , an input / output I / F 505 , an input device 506 , and a display device 507 . The CPU 501, the ROM 502, the RAM 503, and the I / F 504 have the same functions as the CPU 101, the ROM 102, the RAM 103, and the I / F 105 of the camera 10 described above. The input / output I / F 505 is an interface related to input / output. The input / output I / F 505 is connected to an input device 506, receives instruction information from the input device 506, and notifies the CPU 501 via an internal bus 508. Here, the input device 506 includes operation keys including a release switch and a power switch, a cross key, a joystick, a touch panel, a keyboard, and a pointing device (e.g., a mouse), etc. The input / output I / F 505 is also connected to a display device 507 including a monitor such as an LCD display, and displays information such as captured images and operation menus transmitted from the imaging device 100 and temporarily stored in the RAM 503.

[0025] In this embodiment, a case will be described in which the imaging device 100 operates as an information processing device including the image analysis unit 108. However, a client device 500, a general PC, a cloud server, or the like communicably connected to the imaging device 100 may operate as the information processing device. In this case, the information processing device acquires a captured image captured by the imaging device 100 as an input image via the network 400, and performs processing equivalent to that of the image analysis unit 108.

[0026] Next, the operation of the imaging device 100 in this embodiment will be specifically described. FIG. 4 is a flowchart showing the procedure of the people counting process executed by the image capture device 100 in this embodiment. The process in Fig. 4 is started, for example, based on a user's instruction, and is repeated at regular intervals. However, the start timing of the process in Fig. 4 is not limited to the above timing. The process in Fig. 4 may be started automatically, for example, when the imaging operation starts after the imaging device 100 is started. The imaging device 100 can realize each process shown in Fig. 4 by the CPU 101 in Fig. 1 reading and executing a necessary program. Hereinafter, the letter S means a step in a flowchart.

[0027] In this embodiment, it is assumed that the start-up of the imaging device 100 has already been completed, and the image analysis unit 108 is in a state in which it can analyze the captured image. It is also assumed that the seat area detection unit 114 has detected the seat area 601 shown in FIG. 2, and information on the area in the captured image corresponding to the seat area 601 has been taken into the RAM 103.

[0028] In S1, the imaging device 100 detects a human body from a captured image. Then, the imaging device 100 assigns an ID to the detected human body, associates the position coordinates of the detected human body with the ID, and stores the ID as human body detection information in the RAM 103. For example, when four human bodies are detected in S1, the IDs (001 to 004) of the detected human bodies and the position coordinates of the detected human bodies are stored in the RAM 103 as shown in FIG. Here, the position coordinates of the human body can be the central coordinates of the outline of the detected human body. As shown in Fig. 6, when the image size of the captured image is 1920 x 1080 pixels, the position coordinates of the human body are expressed as (0, 0) to (1920, 1080). Note that Fig. 5 shows human body detection information when all four human bodies with IDs 001 to 004 are present in the non-seat area 602 in Fig. 2.

[0029] In S2, the imaging device 100 stores the behaviors of the N human bodies detected in S1 in the RAM 103. Specifically, the imaging device 100 stores the position coordinates of each human body in the captured image for each frame in the RAM 103. For example, as shown in Fig. 7, the imaging device 100 stores the position coordinates of the detected human bodies for up to 10 frames as human body behavior information, and when the position coordinates are updated, the position coordinates of the oldest frame are discarded and updated.

[0030] In S3, the imaging device 100 judges whether or not there is an undetected human body K among the N human bodies detected in S1. For example, when the human body with ID=001 and the human body with ID=002 shown in FIG. 5 move from the aisle to the seating area to sit in a seat and enter the blind spot area of ​​the seat back, the human body with ID=001 and the human body with ID=002 become undetected. In this case, it is judged that there is an undetected human body K. In this way, if it is judged that there is an undetected human body K, the process proceeds to S4. On the other hand, if it is judged that there is no undetected human body K, the process proceeds to S7.

[0031] In S4, the imaging device 100 judges whether the position to which the human body K moves after non-detection is a seat area. That is, the imaging device 100 judges whether the position to which the human body K moves after non-detection is a region in which a blind spot area exists. The process in S4 will be described in detail later. In S4, if it is determined that the position to which the human body K moves after non-detection is not a seat area, the process proceeds to S5, and if it is determined that the position to which the human body K moves after non-detection is a seat area, the process proceeds to S6. In S5, the imaging device 100 erases information about the human body K whose movement position after non-detection is determined to be not the seat area from the human body detection information, and proceeds to S7. For example, when it is determined that the human body with ID=001 and the human body with ID=002 shown in Fig. 5 are non-detected due to a reason other than movement to the seat area, the information about the human body with ID=001 and the human body with ID=002 is erased, and the human body detection information is updated, as shown in Fig. 8.

[0032] In S6, the imaging device 100 updates the position coordinates of the human body K determined to be undetected to the coordinates of the seat area. For example, it is assumed that the human body with ID=001 shown in Fig. 5 has moved to seat F and become undetected, and the human body with ID=002 has moved to seat C and become undetected. In this case, as shown in Fig. 9, the position coordinates of the human body with ID=001 and the human body with ID=002 are updated to the center coordinates of the area corresponding to seat F and the center coordinates of the area corresponding to seat C, respectively.

[0033] In S7, the image capture device 100 counts the number of people in the seating area. Specifically, the image capture device 100 counts the number of human bodies whose position coordinates are the coordinates of the seating area, based on the human body detection information. For example, in the case of the human body detection information shown in Fig. 9, the position coordinates of the human bodies with ID=001 and ID=002 are the coordinates of the seating area, and the position coordinates of the human bodies with ID=003 and ID=004 are the coordinates of the non-seat area (aisle). Therefore, in this case, the number of people in the seating area is counted as two people.

[0034] The process flow of S4 in FIG. 4 will be specifically described below with reference to FIG. When it is determined in S3 of Fig. 4 that a human body K that has become undetected is present, the image capturing device 100 estimates the position coordinates of the human body K after the undetection in S4a of Fig. 10. The image capturing device 100 estimates the position coordinates of the human body K in the frame where the human body K became undetected, based on human body behavior information that records the position coordinates of the human body K for 10 consecutive frames before the undetection as shown in Fig. 7. As an estimation method, for example, there is a method of calculating the slope a and intercept b of the regression equation Y=aX+b (X is elapsed time, Y is the position coordinate of the human body K) by simple regression analysis from the position coordinates of 10 frames, and calculating the position coordinate Y by setting the elapsed time of the frame after non-detection as X. Since the method of estimating position coordinates by simple regression analysis is publicly known, a detailed description will be omitted. For example, if a human body exhibiting the behavior shown in FIG. 7 is not detected in frame ID=511, the position coordinate of the human body in frame ID=511 is estimated to be (850,700).

[0035] In S4b, the image capturing apparatus 100 estimates a movement vector indicating the movement direction of the human body K that has become undetected. The image capturing apparatus 100 calculates the movement vector based on human body behavior information that records the position coordinates of the human body K for 10 consecutive frames before it became undetected as shown in Fig. 7. For example, as shown in Fig. 7, if the position coordinates of frame ID = 501 are taken as the start point and the position coordinates of frame ID = 510 are taken as the end point, the movement vector of the human body exhibiting the behavior shown in Fig. 7 is (-90, 0). The method of calculating the movement vector is well known, so a detailed description will be omitted. In S4c, the imaging device 100 determines whether the position coordinates estimated in S4a are within the seat area. If the imaging device 100 determines that the estimated position coordinates are within the seat area, the imaging device 100 proceeds to S4d, and if the imaging device 100 determines that the estimated position coordinates are not within the seat area, the imaging device 100 ends the process in Fig. 10 and proceeds to S5 in Fig. 4.

[0036] In S4d, the imaging device 100 determines whether the human body K has moved vertically downward based on the movement vector calculated in S4b. Specifically, when the imaging device 100 captures an image as shown in FIG. 6, if the Y component of the movement vector calculated in S4b is negative, the imaging device 100 determines that the human body K has moved vertically downward. Then, when the imaging device 100 determines that the human body K has moved vertically downward, it determines that the human body K has moved vertically downward from a state where it was detected in the seat area, such as passenger P1 shown in FIG. 11, to a blind spot area 620 by the backrest 610 and has become undetectable. Therefore, when the imaging device 100 determines that the human body K has moved vertically downward in S4d of FIG. 10, it ends the process of FIG. 10 and proceeds to S6 of FIG. 4, and when it determines that the human body K has not moved vertically downward, it proceeds to S4e. In S4d, the image capturing apparatus 100 may determine whether or not the human body K has moved vertically downward above the seat area, thereby making it possible to more accurately determine movement into the blind spot area of ​​the seat.

[0037] In S4e, the imaging device 100 determines whether the human body K has moved horizontally toward the seating area based on the movement vector calculated in S4b. Specifically, if the imaging device 100 determines in S4c that the human body K has moved to one of the seats A to D shown in FIG. 6 and the X component of the movement vector calculated in S4b is positive, the imaging device 100 determines that the human body K has moved horizontally toward the seating area. Similarly, if the imaging device 100 determines in S4c that the human body K has moved to one of the seats E to H shown in FIG. 6 and the X component of the movement vector calculated in S4b is negative, the imaging device 100 determines that the human body K has moved horizontally toward the seating area. If the imaging device 100 determines that the human body K has moved horizontally toward the seating area, the imaging device 100 ends the process of FIG. 10 and proceeds to S6 of FIG. 4, and if the imaging device 100 determines that the human body K has not moved horizontally toward the seating area, the imaging device 100 ends the process of FIG. 10 and proceeds to S5 of FIG. 4.

[0038] As described above, the imaging device 100 in this embodiment is installed on the ceiling of a vehicle. The imaging device 100 captures an image of the inside of the vehicle, and detects objects to be counted from the input image using the captured image as an input image. The imaging device 100 also stores the history of the position of the detected object in the captured image as the behavior of the object. Here, the object to be counted can be a human body. When a detected human body becomes undetected, the imaging device 100 determines whether the undetected human body has moved from a non-blind spot area to a blind spot area based on the recorded behavior of the human body. Here, the blind spot area can be a blind spot area of ​​a seat that occurs due to the positional relationship between the imaging direction of the imaging device 100 and the backrest of the seat. Then, the imaging device 100 counts the number of human bodies in the area to be counted in the captured image based on the detection result of the human body and the determination result of the movement from the non-blind spot area to the blind spot area. Here, the area to be counted can be a seat area in a vehicle.

[0039] Specifically, when a human body is detected from a captured image, the imaging device 100 records information linking the position coordinates of the detected human body with an ID for identifying the detected human body as human body detection information. When the imaging device 100 determines that the undetected human body has moved from a non-blind spot area to a blind spot area, the imaging device 100 updates the position coordinates of the human body recorded as the human body detection information to the position coordinates of the blind spot area to which the human body has moved. The imaging device 100 can easily grasp the number of human bodies present in the seating area by counting the number of human bodies whose position coordinates recorded in the human body detection information are position coordinates within the seating area.

[0040] In this way, the imaging device 100 can determine the number of human bodies present in the seating area by adding up the number of human bodies detected in the area corresponding to the seating area in the captured image and the number of human bodies determined to have moved from a non-blind spot area to a blind spot area present in the seating area. Therefore, even if a passenger moves into the blind spot area of ​​the seat, it is possible to accurately count the number of people in the seat area. In other words, when a passenger moves into the blind spot area of ​​the seat, the human body detection is not performed, but it is correctly determined that a passenger is present in the blind spot area of ​​the seat, and the number of people count can be maintained without being decremented.

[0041] In addition, since the image capturing device 100 stores the history of the position in the captured image as the behavior of the human body, when the human body becomes undetected, the moving position of the human body can be estimated based on the human body behavior information stored before the undetection. Therefore, when the moving position of the undetected human body is estimated to be the seat area, the image capturing device 100 can appropriately determine that the human body has moved from a non-blind spot area to the blind spot area of ​​the seat. Similarly, when a human body becomes undetected, the image capture device 100 can estimate the moving direction of the human body based on the human body behavior information stored before the undetection. Therefore, when the image capture device 100 estimates that the moving direction of the undetected human body is toward the seat area, it can appropriately determine that the human body has moved from a non-blind spot area to the blind spot area of ​​the seat.

[0042] In this way, in this embodiment, the movement of a human body into a blind spot of a seat can be determined by combining the behavior of the human body detected from the captured image and the detection / non-detection determination. Therefore, even if a human body moves from a non-blind spot to a blind spot, the number of people count is not decremented. Furthermore, even if a human body subsequently moves from a blind spot area to a non-blind spot area and the human body is detected, the number of people count is not erroneously incremented. For example, the human body with ID=001 shown in FIG. 9 is a human body present in the blind spot area of ​​the seat. When the human body with ID=001 stands up from the seat and moves to a non-blind spot area, the human body is imaged by the imaging device 100 and the human body is detected. In this case, the detected human body is identified as the human body with ID=001, and the position coordinates of the human body with ID=001 are updated to the position coordinates that have actually been detected. Therefore, a new human body is not erroneously determined to have been detected in the seat area, and the number of people count is maintained without being incremented.

[0043] As described above, even if passengers move between the non-blind spot area and the blind spot area in the vehicle, if there is no increase or decrease in the number of passengers in the vehicle, the number of people count can be prevented from increasing or decreasing, thereby enabling accurate counting of the number of people.

[0044] (Modification) In the above embodiment, the case where it is determined whether the moving direction of the human body K is a direction toward an area where a blind spot area exists in S4d and S4e in Fig. 10 has been described, but the present invention is not limited to the above. If it is determined in S4c that the moving position of the human body K is a seat area, the process may proceed to S6 in Fig. 4 without performing the processes of S4d and S4e.

[0045] In the above embodiment, the case where the position coordinates of the human body K in the next frame are estimated by simple regression analysis in S4a of Fig. 10 has been described, but the present invention is not limited to the above. For example, the position coordinates of the next frame may be estimated by using multiple regression analysis or the like, using not only the position coordinates before non-detection but also the movements of the human body such as the face, arms, and legs as input. Furthermore, although the position coordinates of the next frame are estimated in S4a, the position coordinates after F frames (F is a real number equal to or greater than 0) or the position coordinates after a predetermined time has elapsed may be estimated.

[0046] In the above embodiment, if the human body K is not detected for a predetermined time or longer after the human body K has become non-detectable, an alert may be issued as an abnormality has been detected. In this case, the imaging device 100 executes the process shown in Fig. 12 instead of the process shown in Fig. 4. In Fig. 12, the steps performing the same processes as those in Fig. 4 are given the same step numbers as those in Fig. 4, and the following description will focus on the differences in the processes. In S11, the imaging device 100 determines whether the non-detection state of the human body K continues for a predetermined time or more after the human body K becomes non-detected. Then, if the imaging device 100 determines that the non-detection state of the human body K continues for a predetermined time or more, the imaging device 100 proceeds to S12. On the other hand, if the imaging device 100 determines that the non-detection state of the human body K has not continued for a predetermined time or more, the imaging device 100 proceeds to S3. In S12, the imaging device 100 notifies that an abnormality has been detected. For example, the imaging device 100 transmits information indicating the abnormality detection to the client device 500 via the network 400, and causes the display device 500 included in the client device 500 to display the information indicating the abnormality detection.

[0047] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0048] 100...imaging device, 101...CPU, 102...ROM, 103...RAM, 104...storage device, 105...I / F, 106...A / D conversion unit, 107...camera signal processing unit, 108...image analysis unit, 109...compression / expansion unit, 111...human body detection unit, 112...human body behavior memory unit 112, 113...blind spot area movement determination unit, 114...seat area detection unit, 115...people counting unit, 200...imaging unit, 300...driving unit, 400...network, 1000...camera system

Claims

1. A detection means for detecting objects to be counted from an input image obtained by capturing an image with an imaging device; A storage means for storing the behavior of the object detected by the detection means; a determination means for determining, when the object detected by the detection means becomes undetected, whether or not the undetected object has moved to a specific area based on the behavior of the object stored in the storage means; a counting means for counting the number of the objects in a count target area in the input image based on a detection result of the object by the detection means and a determination result by the determination means, The information processing device, wherein the specific area is a blind spot area of ​​the imaging device.

2. 2. The information processing apparatus according to claim 1, wherein the storage means stores a history of the position of the object detected by the detection means in the input image.

3. The counting means is The information processing device according to claim 1 or 2, characterized in that the number of objects in the area to be counted is counted by adding up the number of objects detected in the area to be counted by the detection means and the number of objects determined to have moved to the specific area by the determination means.

4. The determination means is When the object that has been detected by the detection means becomes undetected, The information processing device according to any one of claims 1 to 3, characterized in that when the movement position of the object that became undetected is estimated to be an area in which the specific area exists based on the behavior of the object stored by the memory means before the object became undetected, it is determined that the object has moved to the specific area.

5. The determination means is When the object that has been detected by the detection means becomes undetected, An information processing device according to any one of claims 1 to 4, characterized in that when the movement direction of the object that has become undetected is estimated to be a direction toward an area in which the specific area exists, based on the behavior of the object stored by the memory means before the object became undetected, it is determined that the object has moved toward the specific area.

6. The information processing device according to any one of claims 1 to 5, further comprising a notification means for notifying that an abnormality has been detected if the object determined by the determination means to have moved to the specific area is not detected by the detection means for a predetermined period of time or longer.

7. 7. The information processing apparatus according to claim 1, wherein the object to be counted is a human body.

8. the input image is a captured image obtained by capturing an image using the imaging device installed in the vehicle, the area to be counted is a seat area in the vehicle, The information processing apparatus according to claim 1 , wherein the specific area is an area in which a blind spot occurs due to a seat in the angle of view of the imaging device.

9. the object to be counted is a human body, The determination means is When the human body that was detected by the detection means becomes undetected, The information processing device described in claim 8, characterized in that when it is estimated that the undetected human body has moved vertically downward based on the behavior of the human body stored in the memory means before the human body became undetected, it is determined that the human body has moved into the specific area.

10. the object to be counted is a human body, The determination means is When the human body that was detected by the detection means becomes undetected, The information processing device described in claim 8 or 9, characterized in that when it is estimated that the undetected human body has moved toward the seat based on the behavior of the human body stored by the memory means before the human body became undetected, it is determined that the human body has moved toward the specific area.

11. An information processing device according to any one of claims 1 to 10; an imaging means that is installed on a ceiling of a vehicle and captures an image to obtain the input image.

12. A detection step of detecting objects to be counted from an input image obtained by capturing an image with an imaging device; a storage step of storing the behavior of the object detected in the detection step; a determination step of determining, when the object detected in the detection step becomes undetected, whether or not the undetected object has moved to a specific area based on the behavior of the object stored in the storage step; a counting step of counting the number of the objects in a count target region in the input image based on a detection result of the object in the detection step and a judgment result in the judgment step; Including, The information processing method according to claim 1, wherein the specific area is a blind spot area of ​​the imaging device.

13. A program for causing a computer to function as the information processing device according to any one of claims 1 to 10.

Images