Information processing device, information processing method, and program

Abstract

To solve the problem that it is difficult to recognize whether or not a person included in a picked-up image is within the range of a detection area.SOLUTION: An information processing device includes: input means for receiving a switching instruction to switch a screen; screen creation means for creating a detection result conformation screen including a picked-up image and information on detection results of persons included in the picked-up image, and an area confirmation screen including a detection area set corresponding to a floor surface of the picked-up image to detect persons included in the picked-up image and count the number of the person, and foot images showing positions of feet of the persons; and display control means for switching display from the detection result confirmation screen to the area confirmation screen on the basis of the switching instruction.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] When an information processing apparatus sets a detection area on a captured image and counts the number of people, there is a case where a person who should not be counted appears in the set detection area and cannot be counted as intended. For example, when the information processing apparatus sets a two-dimensional detection area in the captured image without considering the depth of the space, a person far away who appears small in the image may be unintentionally counted.

[0003] In Patent Document 1, in order to solve the above problems, a technique is disclosed in which a detection area is set on the floor surface and the number of people whose feet are present in the detection area is counted.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When a detection area is set on the floor surface, it is difficult for the user to recognize, for example, whether a person whose feet are hidden among the people included in the captured image is within the range of the detection area.

[0006] The present invention has been made in view of the above problems, and an object thereof is to enable easy recognition of whether a person included in a captured image is within the range of a detection area.

Means for Solving the Problems

[0007] To solve this problem, for example, the information processing apparatus of the present invention has the following configuration. That is, Input means for receiving a switching instruction to switch the screen, A detection result confirmation screen including a captured image and information regarding a detection result of a person included in the captured image, a detection area set corresponding to the floor surface of the captured image to detect a person included in the captured image and count the number of people, and an area confirmation screen including a foot image indicating a position of the feet of the person, and screen creation means for creating the same, Display control means for switching the display from the detection result confirmation screen to the area confirmation screen based on the switching instruction, and is provided with.

Effect of the Invention

[0008] According to the present invention, it is possible to easily recognize whether a person included in a captured image is within the range of a detection area.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and duplicate explanations are omitted.

[0011] <First Embodiment> The person counting system according to the first embodiment is a surveillance camera installed in a large space such as an office or a hospital, which captures a person, who is a subject in the large space, to generate an image and counts the number of people. Note that the image is used as a concept including the image and the image data.

[0012] [Example of System Configuration of Person Counting System] FIG. 1 shows the overall configuration and usage example of the person counting system in this embodiment. As shown in FIG. 1, the person counting system 100 in this embodiment is an example of an information processing apparatus, and includes a surveillance camera 101, a person counting server 102, and a browsing terminal 103. The surveillance camera 101, the person counting server 102, and the browsing terminal 103 are arranged on a network and connected to be able to transmit and receive information to and from each other. The network used in this embodiment assumes a high-speed network combining wired and wireless. It is assumed that the surveillance camera 101 and the person counting server 102 are connected by a high-speed wireless LAN. All configurations may be connected by a high-speed wireless network or a wired network. The person counting server 102 in this embodiment assumes a service on the cloud, but it is not limited thereto.

[0013] In the person counting system 100 of this embodiment, for example, a surveillance camera 101 installed on the side wall of a large room captures images of people present in the large room to generate images. The person counting system 100 analyzes the captured image 104 captured by the surveillance camera 101 to detect heads, and counts the number of people present in a detection area, which is a specified area, using the information of the detected heads. The area shown in the captured image 104 of FIG. 1 indicates the imaging area of the surveillance camera 101.

[0014] [Hardware Configuration of Person Counting System] FIG. 2 shows the hardware configurations of the surveillance camera 101, the person counting server 102, and the browsing terminal 103 that constitute the person counting system 100 of this embodiment.

[0015] FIG. 2(A) is a block diagram showing an example of the hardware configuration of the surveillance camera 101 in this embodiment. The surveillance camera 101 includes a CPU 200, a RAM 201, a ROM 202, a bus 203, a network I / F 204, an input I / F 205, and an RGB sensor 206. The CPU 200, the RAM 201, the ROM 202, the network I / F 204, and the input I / F 205 are connected to each other via the bus 203 so as to be able to transmit and receive information. The CPU 200, the RAM 201, and the ROM 202 function as the main components of a computer.

[0016] The CPU 200 is an abbreviation for Central Processing Unit, and comprehensively controls each device connected via the bus 203. Note that the surveillance camera 101 may have other processors such as a GPU (Graphics Processing Unit), an MPU (Micro Processing Unit), and a QPU (Quantum Processing Unit) in addition to the CPU 200.

[0017] The RAM 201 is an abbreviation for Random Access Memory. The RAM 201 functions as a main memory and a temporary storage area such as a work area when the CPU 200 executes a program.

[0018] The ROM 202 is the abbreviation of Read Only Memory. The ROM 202 stores, among other things, the operating system (OS), each processing program according to the present invention shown in the sequence diagrams and flowcharts described later, device drivers, parameters necessary for program execution, and the like. Note that, in addition to the ROM 202, the surveillance camera 101 may have a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0019] The CPU 200 reads out the programs stored in the ROM 202, expands them in the RAM 201, and executes them to realize various functions.

[0020] The network I / F 204 (I / F is the abbreviation of interface) communicates remotely with the people counting server 102 on the network. The input I / F 205 converts the RGB image captured by the RGB sensor 206 into a format processable by the surveillance camera 101. Note that I / F is the abbreviation of interface.

[0021] The RGB sensor 206 has an imaging element such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal - Oxide - Semiconductor) sensor. The RGB sensor 206 images a subject and generates an imaging image.

[0022] Figure 2(B) is a block diagram showing an example of the hardware configuration of the number-of-people measurement server 102 in the present embodiment. The number-of-people measurement server 102 includes a CPU 210, a RAM 211, a ROM 212, a bus 213, a network I / F 214, a storage I / F 215, and a storage 216. The CPU 210, the RAM 211, the ROM 212, the network I / F 214, and the storage I / F 215 are connected to each other via the bus 213 so as to be able to transmit and receive information. The CPU 210, the RAM 211, and the ROM 212 function as the main components of the computer. The CPU 210, the RAM 211, and the ROM 212 function in the same manner as the CPU 200, the RAM 201, and the ROM 202 of the surveillance camera 101.

[0023] The network I / F 214 communicates, for example, remotely with the surveillance camera 101 on the network and the browsing terminal 103. The storage I / F 215 inputs and outputs data to and from the storage 216. The storage 216 is, for example, an HDD or an SSD. The storage 216 stores captured images captured by the surveillance camera 101 on the network, the number-of-people measurement results, and programs and operating systems that the number-of-people measurement server 102 starts internally. Necessary information among the information stored in the storage 216 is appropriately read into the RAM 211 when the power is turned on.

[0024] Figure 2(C) is a block diagram showing an example of the hardware configuration of the browsing terminal 103 in the present embodiment. The browsing terminal 103 includes a CPU 220, a RAM 221, a ROM 222, a bus 223, a network I / F 224, a storage I / F 225, a storage 226, and a display I / F 227. The CPU 220, the RAM 221, the ROM 222, the network I / F 224, the storage I / F 225, and the display I / F 227 are connected to each other via the bus 223 so as to be able to transmit and receive information. The CPU 220, the RAM 221, and the ROM 222 function as the main components of the computer. The CPU 220, the RAM 221, and the ROM 222 have the same functions as the above-described surveillance camera 101.

[0025] The network I / F 224 communicates remotely with the number measurement server 102 on the network. The storage I / F 225 inputs and outputs data to and from the storage 226. The storage 226 stores UI components used for screen display, programs and operating systems that the browsing terminal 103 internally activates. The storage 226 appropriately loads the information necessary at power-on into the RAM 221. The display I / F 227 converts the screen to be displayed on the browsing terminal 103 into a signal that can be displayed on the display 228 and outputs it.

[0026] Hereinafter, when explaining the same components using other figures, the same reference numerals will be used and the explanation will be omitted.

[0027] FIG. 3(A) is a block diagram showing an example of the functional configuration of the monitoring camera 101 in the present embodiment. The monitoring camera 101 includes an imaging unit 300, an image processing unit 301, and a communication unit 302. The CPU 200 expands the program stored in the ROM 202 into the RAM 201 and executes it to realize at least part of the functions of the imaging unit 300, the image processing unit 301, and the communication unit 302.

[0028] FIG. 3(B) is a block diagram showing an example of the functional configuration of the number measurement server 102 in the present embodiment. The number measurement server 102 includes a communication unit 320, an image acquisition unit 321, a head detection unit 322, and a foot position estimation unit 323. Further, the number measurement server 102 includes a number measurement unit 324, a measurement result management unit 325, and a storage unit 326. The CPU 210 expands the program stored in the ROM 212 into the RAM 211 and executes it to realize at least part of the functions of the communication unit 320, the image acquisition unit 321, the head detection unit 322, and the foot position estimation unit 323.

[0029] FIG. 3(C) is a block diagram showing an example of the functional configuration of the browsing terminal 103 in the present embodiment. The browsing terminal 103 includes a communication unit 330, a storage unit 331, an input unit 332, a display control unit 333, a screen creation unit 334, and a region setting unit 335. Each of these functional units, the CPU 220 expands the program stored in the ROM 222 into the RAM 221 and executes it to realize at least part of the functions of the communication unit 330, the storage unit 331, the input unit 332, the display control unit 333, the screen creation unit 334, and the region setting unit 335.

[0030] Also, when configuring hardware as an alternative to software processing using a CPU, it may be realized by an arithmetic unit, a circuit (for example, an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array), etc.) corresponding to the processing of each functional unit described here.

[0031] Next, each functional block of the monitoring camera 101 will be described.

[0032] The communication unit 302 performs data communication with the people counting server 102 on the network via the network I / F 204. Specific data exchanged in the communication will be described later.

[0033] The imaging unit 300 is realized by the RGB sensor 206. The imaging unit 300 acquires an RGB image captured by the RGB sensor 206 and stores it in the RAM 201 at any time. An RGB image is an image in which the color of any one point (pixel) in the image is represented by the three primary colors of red, green, and blue, and each pixel in the image has pixel values for red, green, and blue respectively. The RGB image held in the RAM 201 is used as an input to the image processing unit 301. In the present embodiment, the period (frame rate) at which the imaging unit 300 captures an RGB image is set to 30 fps, but it is not limited to this.

[0034] The image processing unit 301 converts the RGB image held in the RAM 201 into video data. The image processing unit 301 compresses and converts the RGB images captured at a predetermined frame interval into video data called H.264. The video data is transmitted to the person counting server 102 on the network via the communication unit 302. The video data shall include the information of the captured date and time. The image processing unit 301 compresses and converts the RGB images captured at a predetermined frame interval into H.264 format video data to reduce the load on the network bandwidth, but it is not limited to this. The image processing unit 301 may compress and convert the RGB image into other standard compression formats such as MotionJPEG, MPEG-4, H.265, etc. and transmit it. Also, the image processing unit 301 can convert it into still image data instead of video data and transmit it to the person counting server 102 at predetermined time intervals.

[0035] Next, each functional block of the person counting server 102 will be described.

[0036] The communication unit 320 performs data communication with the surveillance camera 101 and the browsing terminal 103 existing on the network via the network I / F 214. The communication unit 320 receives video data, date and time, and camera information (hereinafter referred to as camera ID) for identifying the camera from the surveillance camera 101. The communication unit 320 transmits the person counting result and the background image to the browsing terminal 103. The person counting result includes the analyzed frame image and the number of people counted in the image, etc. Specific data exchanged in the communication will be described later.

[0037] The image acquisition unit 321 decodes the video data received from the surveillance camera 101 into a plurality of frame images and holds them in the RAM 211 as needed. In this embodiment, an example will be described in which the image acquisition unit 321 receives and processes the video data from one surveillance camera, but it is not limited to this. The image acquisition unit 321 may be configured to receive videos from a plurality of surveillance cameras individually, decode them into a plurality of frame images, and perform person counting on each of them.

[0038] The head detection unit 322 detects a person's head from the frame image by analyzing the frame image, and creates a head list associating the head position and the head size.

[0039] The foot position estimation unit 323 estimates the foot position corresponding to each head position from the head size in the head list created by the head detection unit 322.

[0040] The number-of-people measurement unit 324 measures the number of people by counting the number of human heads whose estimated foot positions are included in the detection area among the number of heads included in the head list created by the head detection unit 322, and generates a number-of-people measurement result as the measured result.

[0041] The measurement result management unit 325 associates the number-of-people measurement result by the number-of-people measurement unit 324 with the camera ID. The measurement result management unit 325 records and manages the number-of-people measurement result associated with the camera ID in a form that can be read out at the analyzed date and time in the storage unit 326.

[0042] The storage unit 326 is realized by the storage 216. The storage unit 326 stores the analyzed frame image and the number-of-people measurement result including the measured number of people. The storage unit 326 also stores a program and an OS that the number-of-people measurement server 102 starts up.

[0043] Next, each functional block of the browsing terminal 103 will be described.

[0044] The communication unit 330 performs data communication with the number-of-people measurement server 102 on the network via the network I / F 224. The communication unit 330 receives, for example, the number-of-people measurement result and the background image from the number-of-people measurement server 102.

[0045] The storage unit 331 is realized by the storage 226. The storage unit 331 stores data of UI components of display objects such as icons and buttons used in screen display, and a program and an OS that the browsing terminal 103 starts up.

[0046] The input unit 332 receives and obtains keyword input by the user and operation inputs from a mouse or the like, and outputs them to the area setting unit 335 and the screen creation unit 334. The input unit 332 receives and obtains operation inputs such as a switching instruction for switching the screen and a change instruction for changing the detection area.

[0047] The area setting unit 335 sets a detection area based on the input information acquired by the input unit 332. The detection area is an area for detecting the heads of people and counting the number of people. The area setting unit 335 sets the detection area corresponding to the floor surface included in the captured image or changes it based on the user's change instruction. For example, the area setting unit 335 reduces the shape of the floor surface and sets a detection area similar to the floor surface.

[0048] The screen creation unit 334 filters the number measurement results received from the number measurement server 102 in the detection area. The screen creation unit 334 creates a display screen by combining the filtered results with the UI components held in the storage unit 331. The screen creation unit 334 is assumed to generate two screens, for example, a detection result confirmation screen and an area confirmation screen. The details of the display screen will be described later.

[0049] The display control unit 333 causes the display 228 to display the screen created by the screen creation unit 334. For example, the display control unit 333 performs display control to switch between the detection result confirmation screen and the area confirmation screen based on a switching instruction by the user's operation acquired by the input unit 332. The display control unit 333 may acquire the user's operation from the input unit 332 or via the screen creation unit 334. The details of the display control will be described later.

[0050] [Explanation of the number measurement process] Next, with reference to FIG. 4, the number measurement process executed by the number measurement system 100 of the present embodiment will be described in detail.

[0051] FIG. 4 is a sequence diagram showing an example of the number-of-people measurement process executed by the number-of-people measurement system 100 of the present embodiment. In the number-of-people measurement system 100 of the present embodiment, it is assumed that a surveillance camera 101 installed in a large space, a number-of-people measurement server 102 on the cloud, and a browsing terminal 103 are pre-set to be connected to a network and communicate with each other.

[0052] When the surveillance camera 101 is activated, in step S400 of FIG. 4, the imaging unit 300 of the surveillance camera 101 captures an RGB image at a specified frame rate (30 fps) and stores it in the RAM 201.

[0053] Subsequently, in step S401, the image processing unit 301 of the surveillance camera 101 converts the RGB image stored in the RAM 201 into video data in H.264 format.

[0054] Then, in step S402, the communication unit 302 of the surveillance camera 101 transmits the converted video data to the number-of-people measurement server 102. While the surveillance camera 101 is activated, the processes of steps S400 to S402 are repeated, and the video data is continuously transmitted to the number-of-people measurement server 102.

[0055] In step S403, the image acquisition unit 321 of the number-of-people measurement server 102 acquires the video data transmitted from the surveillance camera 101 via the communication unit 320 and stores it in the RAM 211. Further, in step S403, the image acquisition unit 321 decodes the received video data into a plurality of frame images and stores them in the RAM 211.

[0056] In step S404, the head detection unit 322 of the number measurement server 102 extracts one frame image from the frame images held in the RAM 211 at a predetermined time interval (the initial value is a one-minute interval). An example of the frame image extracted at this time is shown as 104 in FIG. 1. People existing in a wide area are shown in 104 of FIG. 1. Further, in step S404, the head detection unit 322 analyzes the frame image to detect the heads of people in the image and creates a head list consisting of the positions and sizes of the heads and holds it in the RAM 211.

[0057] In step S405, when the head detection unit 322 of the number measurement server 102 cannot detect the head of a person from the frame image to be analyzed, it holds that frame image as a background image in the RAM 211. In the present embodiment, the head detection unit 322 uses one frame image in which no person is shown as the background image, but this is not the only case. For example, the head detection unit 322 may create and update the background image by collecting and averaging a plurality of frame images in which no person is shown. Also, in the present embodiment, the head detection unit 322 creates the background image during the number measurement process, but a background image created in advance may be registered in the number measurement server 102 and used.

[0058] In step S406, the foot position estimation unit 323 of the number measurement server 102 estimates the foot positions corresponding to the respective head positions from the head sizes in the head list held in the RAM 211 and holds them in the RAM 211. The foot position estimation unit 323 shall estimate the foot position corresponding to the head position from the head size. As shown in FIG. 7(A), the head size at an arbitrary position in the image can be estimated by using, in addition to the installation height, depression angle, and vertical angle of view of the surveillance camera 101, the distance from the camera of the person in the angle of view, the average height of the person, and the average total head height (the height of the head in the vertical direction). The foot position estimation unit 323 calculates and estimates the foot position corresponding to the head position by inverse calculation from the head size.

[0059] In step S407, the person count unit 324 of the person count server 102 measures the number of people by counting the number of heads of the people whose foot positions held in the RAM 211 are within the detection area among the heads included in the head list held in the RAM 211, and holds the result in the RAM 211 as the person count result.

[0060] Also, in step S407, the measurement result management unit 325 of the person count server 102 associates the person count result held in the RAM 211 with the camera ID, and records it in the storage unit 326 so that the person count result can be read out at high speed at the analyzed date and time. FIG. 6(A) shows an example of a person count result list 600 in which the recorded person count results are listed. The person count result list 600 is composed of the date and time when the person count was performed, the camera ID of the surveillance camera, the measured number of people, the head position, the estimated foot position, and the frame image to be analyzed. In the present embodiment, the information on the head position and the estimated foot position is in the form of the x coordinate of the upper left of the head, the y coordinate of the upper left of the head, the width w of the head, the height h of the head, the x coordinate of the foot, and the y coordinate of the foot, and is listed according to the number of detected and counted heads.

[0061] In step S408, the communication unit 320 of the person count server 102 transmits the person count result, the captured image, and the background image to the browsing terminal 103.

[0062] In step S409, when the person count server 102 stops, the person count process of the person count system 100 ends.

[0063] On the other hand, in step S410, when the communication unit 330 of the browsing terminal 103 receives the person count result, the captured image, and the background image from the person count server 102, it holds them in the RAM 221.

[0064] In step S411, the browsing terminal 103 executes screen display processing based on the person count result, the captured image, and the background image held in the RAM 221. The details of the screen display processing will be described in detail with reference to FIG. 5.

[0065] [Explanation of Screen Display Processing] Next, with reference to FIG. 5, the screen display processing executed by the number-of-people measurement system 100 of the present embodiment will be described in detail. FIG. 5(A) shows a flowchart of the screen display processing. FIG. 5(B) shows the screen in the corresponding step of the screen display processing. It is assumed that the detection area of the present embodiment is preset by the user who uses the browsing terminal 103 and is recorded in the storage unit 311.

[0066] In step S411 of FIG. 4, when the browsing terminal 103 executes the screen display processing, the flowchart of FIG. 5(A) starts.

[0067] First, in step S500, the screen creation unit 334 of the browsing terminal 103 filters out and excludes the people outside the detection area from the number-of-people measurement result based on the relationship between the foot position of the number-of-people measurement result held in the RAM 221 and the detection area. The screen creation unit 334 holds the head positions of the people within the detection area that remain unfiltered in the RAM 221.

[0068] Next, in step S501, the screen creation unit 334 of the browsing terminal 103 creates a detection result confirmation screen 520a including the captured image and information regarding the person detection result. For example, the screen creation unit 334 creates, as a display screen, a detection result confirmation screen 520a in which the head positions within the detection area that remain unfiltered and the detection area are superimposed on the frame image that is the captured image. The upper part of FIG. 5(B) shows the detection result confirmation screen 520a created in step S501. The detection result confirmation screen 520a includes the superimposed detection area 509. The detection frame 508 displayed on the detection result confirmation screen 520a superimposes and displays the foot existing within the detection area 509 and the head size of the detected person. In other words, the person in the detection frame 508 is the person counted in the number. Therefore, the detection frame 508 functions as an identifier for identifying whether the person is counted as the number. In step S501, the display control unit 333 causes the detection result confirmation screen 520a to be displayed on the display 228 of the browsing terminal 103.

[0069] In the detection result confirmation screen 520a, the feet of the person 510a cannot be confirmed. The fact that the detection frame of the person 510a is not superimposed on the detection result confirmation screen 520a means that the foot position of the person 510a is outside the detection area 509, but the user does not know how much the detection area 509 should be expanded to include the person 510a. That is, just by visually observing the detection result confirmation screen 520a, the user does not know how much the detection area 509 should be corrected. This embodiment solves that problem.

[0070] In step S502, when the input unit 332 determines that the user has clicked and selected the head of the person 510a on the detection result confirmation screen 520a using, for example, the mouse connected to the browsing terminal 103 (Yes in step S502), step S503 is executed. The selection of the head by the user is an example of a switching instruction for instructing the switching from the detection result confirmation screen 520a to the area confirmation screen 530a. When the browsing terminal 103 determines that the user has selected and clicked on something other than the head (No in step S502), the screen display process ends.

[0071] In step S503, the screen creation unit 334 of the browsing terminal 103 creates the area confirmation screen 530a shown in the second row from the top of FIG. 5(B) as the display screen. The area confirmation screen 530a is a screen in which the head image 510b of the detected person 510a, the foot image 511a indicating the estimated position of the feet from the head size, and the line 512 connecting the head image and the foot position are superimposed on the background image. In other words, the screen creation unit 334 deletes the image of the torso of the person 510a and replaces it with the line 512 on the area confirmation screen 530a. In this embodiment, the head image 510b is an image created by cutting out based on the head position detected from the frame image and the preset head size. In step S503, the display control unit 333 causes the display on the display 228 of the browsing terminal 103 to be switched from the detection result confirmation screen 520a to the area confirmation screen 530a.

[0072] When the area confirmation screen 530a is displayed on the display 228, the user can clearly understand the positional relationship between the foot image 511a of the person 510a and the detection area 509. As a result, the user can determine how much to expand the detection area so that the person 510a falls within the range of the detection area 509, making it possible to easily and appropriately modify the detection area 509. Note that in the area confirmation screen 530a, only the foot image showing the feet of the person outside the detection area is displayed, but it is not limited to this. That is, the area confirmation screen may be configured to also display the foot image showing the feet of the person within the detection area.

[0073] In step S504, when the input unit 332 determines that the user has expanded the detection area 509 in the direction of the arrow 513 of the area confirmation screen 530b shown in the second row from the bottom of FIG. 5(B) using the mouse connected to the browsing terminal 103 (Yes in step S504), step S505 is executed. If the input unit 332 determines that the detection area 509 has not been changed (No in step S504), step S505 is not executed.

[0074] In step S505, the area setting unit 335 of the browsing terminal 103 changes the detection area 509 and records the changed value in the storage unit 311. Also, the screen creation unit 334 creates a screen in which the display of the foot position of the person 510a changes from the foot image 511a to the foot image 511b and outputs it to the display control unit 333. The display control unit 333 reflects the change on the display screen of the display 228. Through this display change, the user can recognize that the foot image 511b of the person 510a is now included in the detection area 509.

[0075] In step S506, the input unit 332 determines whether the selection of the person 510a by the user has been canceled. For example, when the user clicks on the head image 510b on the region confirmation screen 530b, the input unit 332 determines that the selection of the person 510a has been canceled (Yes in step S506). However, the determination method is not limited to this. For example, when the user clicks on a region outside the detection region 509 on the region confirmation screen 530b, the input unit 332 may also determine that the selection has been canceled. When the input unit 332 determines that the selection of the person 510a by the user has been canceled, step S507 is executed. When the input unit 332 does not determine that the selection of the person 510a has been canceled (No in step S506), steps S503 and subsequent steps are repeated.

[0076] In step S507, the screen creation unit 334 creates a detection result confirmation screen 520b shown in the lower part of FIG. 5(B) in accordance with the change of the detection region 509, and outputs it to the display control unit 333. The display control unit 333 switches the display from the region confirmation screen 530b to the detection result confirmation screen 520b. On the detection result confirmation screen 520b, the detection frame 510c of the person 510a is superimposed and displayed due to the correction of the detection region 509 performed in steps S504 and S505. That is, the person 510a is also counted as the number of people within the detection region 509.

[0077] As described above, by switching the display to the region confirmation screen when the user selects a person on the detection result confirmation screen, even when it is crowded and the feet cannot be seen, the user can clearly grasp the positional relationship between the person to be counted and the detection region. As a result, a UI that can easily set and correct the detection region can be realized.

[0078] When the person counting system 100 of the present embodiment receives a switching instruction from a user such as a service operator, the display is switched from the detection result confirmation screen 520a including the captured image to the area confirmation screen 530a including the foot image 511a indicating the foot position of a person outside the detection area 509. In this way, the person counting system 100 can easily let the user recognize whether the person included in the captured image is within the range of the detection area 509 by showing the detection area 509 and the foot image 511a of the person outside the detection area 509.

[0079] The person counting system 100 superimposes and displays a detection frame 508 indicating whether it has been counted on the detection result confirmation screen 520a, so that the user can easily recognize the counted person and the uncounted person. The person counting system 100 receives the selection of the uncounted person from the user, generates and displays an area confirmation screen 530a including the foot image of the person. Thereby, the person counting system 100 can superimpose and display, on the area confirmation screen 530a, the foot image indicating the feet of the uncounted person selected by the user together with the detection area 509, so that the area confirmation screen 530a meeting the user's wishes can be created.

[0080] The person counting system 100 creates a detection result confirmation screen 520a with the detection area 509 superimposed on the captured image. Thus, for example, when the number of people is small, the user can be made to determine whether the number of people is accurately counted in the current detection area 509. Thereby, the person counting system 100 can further reduce the user's work.

[0081] When a person is selected, the person counting system 100 switches from the detection result confirmation screen 520a to the area confirmation screen 530a, so that the user can easily switch the screen.

[0082] The person counting system 100 creates and displays a region confirmation screen 350a that includes a head image 510b of a person corresponding to the foot image 511a. Thus, the user can associate and confirm the face and foot position of a person not counted in the number of people.

[0083] The person counting system 100 detects the head of a person and counts the number of people, and estimates the position of the feet of the person based on the head of the person not counted. Therefore, the position of the feet can be estimated with high accuracy, and the foot image 511a with high position accuracy can be superimposed on the region confirmation screen 530a.

[0084] When the person counting system 100 receives a cancellation of the selection of a person, it creates and displays a detection result confirmation screen 520b with the detection area 509 updated. As a result, in the person counting system 100, the user can easily update and switch the display of the detection result confirmation screen 520b.

[0085] The person counting system 100 creates and displays a region confirmation screen 530a in which the head image 510b and the foot image 511a are connected by a line. Thus, the user can easily recognize the head image 510b of the person in the foot image 511a.

[0086] The person counting system 100 receives an instruction to change the detection area 509 on the region confirmation screen 530a, changes the detection area 509 based on the change instruction, and changes the display of the detection area 509 on the region confirmation screen 500b. As a result, the person counting system 100 can change the detection area 509 while allowing the user to recognize the foot position of a person outside the detection area 509 by the foot images 511a, 511b. As a result, in the person counting system 100, the user can easily change and set the detection area 509 so that a person not counted is counted.

[0087] The person counting system 100 receives the selection of a person who was not counted as an instruction to switch the screen, creates and displays a region confirmation screen 530a including the foot image 511a of the selected person. As a result, since the person counting system 100 also serves as the selection of the person who displays the foot image 511a for the switching instruction, the user's operation can be simplified further.

[0088] Since the person counting system 100 creates a region confirmation screen 530a including a background image excluding people, the user can change the detection region 509 while recognizing the situation where the surveillance camera 101 is installed, especially the floor surface and the like.

[0089] In addition, in this embodiment, the description has been made on the premise that all the heads of the people shown in the frame image to be analyzed have been detected. However, in actuality, it may not be detected (due to non-detection), and the detection frame may not be displayed on the detection result confirmation screen.

[0090] For example, since the person 602 in FIG. 6(B) is not detected, the detection frame is not displayed around the person 602 on the detection result confirmation screen 601a including the detection region 607. In this case, when the user clicks and selects a position on the region including the person 602 on the detection result confirmation screen 601a, the screen creation unit 334 creates and displays a region confirmation screen 601b indicating that the person 602 is not detected in the region. For example, the screen creation unit 334 creates a region confirmation screen 601b in which a dotted line frame 603 surrounding the position selected by the user and a message indicating that it is not detected (here, "not detected") are superimposed. Note that the screen creation unit 334 may display only one of the dotted line frame 603 and the message. Here, since the head of the person 602 is not detected, there is no head image to be superimposed in the first place. As a result, the screen creation unit 334 creates a region confirmation screen 601b in which the dotted line frame 603 is superimposed on the background image.

[0091] Alternatively, the screen creation unit 334 can also express that the head of the person 602 has not been detected by creating and displaying a screen such as the area confirmation screen 601c. That is, when the user selects the position of an undetected person on the detection result confirmation screen 601a, the screen creation unit 334 superimposes the head image of the detected person, the detection frame 604 surrounding the head image of the detected person, the foot image 606 indicating the foot position of the detected person, the dotted line frame 605 surrounding the position selected by the user, and the detection area 607 on the background image to create and display the area confirmation screen 601c. The detection frame 604 is an example of a first identifier, and the dotted line frame 605 is an example of a second identifier. The screen creation unit 334 may superimpose the head image of the undetected person on the area confirmation screen 601c. In this case, since there is no head image of the undetected person, the screen creation unit 334 may use an image obtained by cropping the captured image in the area of the dotted line frame 605 surrounding the position selected by the user as the head image of the undetected person. Further, the screen creation unit 334 may superimpose the foot image 611 corresponding to the area of the dotted line frame 605 on the area confirmation screen 601c as the foot image 611 of the undetected person. In this case, the screen creation unit 334 may display the foot image 606 of the detected person and the foot image 611 of the undetected person differently.

[0092] As described above, by switching between the detection result confirmation screen and the area confirmation screen with a simple operation, it is possible to distinguish whether the person to be counted is outside the detection area and whether the person has been detected.

[0093] When an area where no person has been detected is selected by the user on the detection result confirmation screen 601a, the person counting system 100 creates an area confirmation screen 601b including a dotted line frame 603 indicating that no person has been detected. Thereby, the person counting system 100 can notify the user that no person has been detected.

[0094] When the user selects an area where no person is detected on the detection result confirmation screen 601a, the person counting system 100 creates an area confirmation screen 601c that includes a detection frame 604 indicating the detected person and a dotted line frame 605 indicating the selected area. In this way, by displaying the detection frame 604 and the dotted line frame 605 differently, the person counting system 100 can make the user recognize whether the person is detected or not even if the head image of the person is displayed regardless of the presence or absence of detection. Furthermore, since the person counting system 100 makes the foot images 606 and 611 different according to whether the detected person is detected or not, the user can recognize whether the position of the feet of the detected person is detected or not.

[0095] <Modification Example of the First Embodiment> Hereinafter, the modification example will be described in detail with reference to the drawings. Note that the same numbers are assigned to those that are common to the content described in the first embodiment, and the description thereof is omitted. In addition, the hardware configuration and the functional configuration of the person counting system according to the modification example conform to the first embodiment.

[0096] In the modification example, the functions of the screen creation unit 334 and the area setting unit 335 are different from those in the first embodiment. In the modification example, an example in which an area confirmation screen is used when setting a detection area for the first time will be described. The area setting of the present invention does not set the area two-dimensionally using a rectangle, but sets the detection area on the floor surface in consideration of the depth. Therefore, when setting the detection area for the first time, the user does not know what range to set the detection area.

[0097] Therefore, in the modification example, the area guide and the area confirmation screen are used so that the detection area can be easily set even for the first time.

[0098] FIG. 7(B) shows an area confirmation screen 700a for setting a polygonal detection area. In this modification, the detection area is a rectangle. The screen creation unit 334 causes a polygonal area guide 701 to be displayed along the floor surface of a large space. In this modification, the area guide 701 is a rectangle that conforms to the shape of the floor surface. The shape of the area guide 701 is, for example, the same as the aspect ratio of the floor surface and is similar to the shape of the floor surface. In the modification, the screen creation unit 334 analyzes the frame image and detects the floor surface, thereby causing the area guide 701 to be displayed. However, the display method is not limited to this. For example, the floor surface may be registered in the person counting system 100 in advance by a user operation, and the screen creation unit 334 may cause the area guide 701 to be displayed based on the registered floor surface.

[0099] With the area guide 701 displayed, when the user uses the mouse connected to the browsing terminal 103 to give an instruction to change the detection area by a drag-and-drop operation along the direction of the arrow 710 from the starting point 702, the area setting unit 335 freely changes the sizes of the detection areas 703a and 703b to the same shape as the area guide 701 (for example, the same aspect ratio). In other words, the area setting unit 335 changes the detection areas 703a and 703b while maintaining the similarity relationship with the area guide 701 in accordance with the user's operation.

[0100] When the user stops the mouse operation and determines the settings of the detection areas 703a and 703b, the screen creation unit 334 creates and displays the area confirmation screen 700b in FIG. 7(C). In the area confirmation screen 700b, the screen creation unit 334 displays, in addition to the newly set detection area 703b, size frames 704 indicating the sizes of the heads estimated by calculation, in association with the four corners that are the corners of the detection area 703b. The user can know in advance the head sizes detected within the detection area set by the size frames 704. The user can repeat the area setting until the intended head size is obtained. The screen creation unit 334 may display a foot image 705 indicating the foot position at the corners of the detection area 703b.

[0101] As described above, by superimposing and displaying the size of the head and the foot position estimated as the area guide at the time of area setting, it is possible to realize a UI that can easily set the detection area even for the first time.

[0102] The person counting system 100 displays a size frame 704 indicating the size of the head in association with the corners of the rectangular detection area 703b on the area confirmation screen 700b, so that the user can recognize whether the person in the captured image is included in the detection area 703b.

[0103] The person counting system 100 displays an area guide 701 similar to the floor surface on the area confirmation screen 700a, and changes the detection areas 703a and 703b based on the user's change instruction while maintaining the similar relationship with the area guide 701. As a result, the person counting system 100 can set the detection areas 703a and 703b corresponding to the shape of the floor surface, and can make the user recognize that the detection areas 703a and 703b correspond to the floor surface.

[0104] <Other Embodiments> Further, the present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or device reads and executes the program.

[0105] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0106] In the above-described embodiment, the monitoring camera 101, the number counting server 102, and the browsing terminal 103 are configured as separate devices, but the configuration of the number counting system is not limited to this. For example, the number counting system may integrate two or more of the monitoring camera 101, the number counting server 102, and the browsing terminal 103 into one device, or part of the functions of any one device may be implemented on another device.

[0107] In the above-described embodiment, the screen creation unit 334 displays the detected people, such as those counted on the detection result confirmation screen 520a, in a distinguishable manner using the detection frame 508, but the distinguishable screen is not limited to this. For example, the screen creation unit 334 may enclose the detected but uncounted people in a distinguishable frame.

[0108] In the above-described embodiment, the screen creation unit 334 changes the detection area based on the user's input, but the change method is not limited to this. For example, the screen creation unit 334 may automatically change the detection area to include the foot position of the people who were not counted.

[0109] The disclosure of this specification includes the following information processing apparatus, information processing method, and program. (Item 1) Input means for receiving a switching instruction to switch the screen, A detection result confirmation screen including the captured image and information regarding the detection result of the people included in the captured image, a detection area set corresponding to the floor surface of the captured image for detecting the people included in the captured image and counting the number of people, and an area confirmation screen including a foot image indicating the foot position of the people, Display control means for switching the display from the detection result confirmation screen to the area confirmation screen based on the switching instruction, An information processing apparatus characterized by comprising the above. (Item 2) The area confirmation screen includes a foot image indicating the foot position of the people outside the detection area The information processing apparatus according to Item 1, characterized by the above. (Item 3) The screen creation means creates the detection result confirmation screen in which an identifier for identifying whether or not the counted person is superimposed on the captured image, The input means receives a selection of the person not counted in the detection result confirmation screen, The screen creation means creates the area confirmation screen including the foot image of the selected person. The information processing apparatus according to item 1 or item 2, characterized in that. (Item 4) The screen creation means creates the detection result confirmation screen by superimposing the detection area on the captured image. The information processing apparatus according to any one of items 1 to 3, characterized in that. (Item 5) The input means receives a selection of the person as the switching instruction, When the person is selected, the display control means switches and displays from the detection result confirmation screen to the area confirmation screen. The information processing apparatus according to any one of items 1 to 4, characterized in that. (Item 6) The screen creation means creates the area confirmation screen including a head image which is an image of the head of the person corresponding to the foot image, the information processing apparatus according to any one of items 1 to 5, characterized in that. (Item 7) A number counting means for detecting the head of a person included in the captured image and counting the number of people based on the head, An estimation means for estimating the position of the feet of the person not counted based on the detected head of the person, and The screen creation means creates the area confirmation screen including the foot image indicating the estimated position of the feet of the person not counted. The information processing apparatus according to any one of items 1 to 6, characterized in that. (Item 8) The input means receives a cancellation of the selection of the person, Based on the cancellation of the selection, the screen creation means updates the detection area of the detection result confirmation screen. The display control means displays the updated detection result confirmation screen. The information processing apparatus according to item 3, characterized in that. (Item 9) The screen creation means creates the area confirmation screen in which the head image and the foot image corresponding to the head image are connected by a line. The information processing apparatus according to item 6, characterized in that. (Item 10) It includes area setting means for setting the detection area. The information processing apparatus according to any one of items 1 to 9, characterized in that. (Item 11) The input means receives an instruction to change the detection area displayed on the area confirmation screen. The area setting means changes the detection area based on the change instruction. The screen creation means updates the area confirmation screen based on the changed detection area. The information processing apparatus according to item 10, characterized in that. (Item 12) The area setting means sets the detection area in the shape of a polygon. The screen creation means creates the area confirmation screen including a size frame indicating the size of the head of the person and an image indicating the foot position, associated with the corners of the detection area. The information processing apparatus according to item 10 or item 11, characterized in that. (Item 13) The area setting means sets an area guide similar to the floor surface included in the captured image. The screen creation means creates the area confirmation screen including the area guide. The input means receives an instruction to change the detection area. When receiving the change instruction, the area setting means changes the detection area while maintaining the similarity relationship with the area guide. The information processing apparatus according to any one of items 10 to 12, characterized in that. (Item 14) The input means receives a selection of a region on the detection result confirmation screen, When the person is not detected within the region, the screen creation means creates the region confirmation screen indicating that the person is not detected in the region The information processing apparatus according to any one of Items 1 to 13, characterized in that. (Item 15) The screen creation means creates the region confirmation screen including a first identifier indicating the detected person, a second identifier indicating the selected region, and the foot image corresponding to the person and the region The information processing apparatus according to Item 14, characterized in that. (Item 16) The input means receives a selection of the person not counted as the switching instruction, The screen creation means creates the region confirmation screen including the foot image of the selected person The information processing apparatus according to any one of Items 1 to 15, characterized in that. (Item 17) The screen creation means creates the region confirmation screen including the background image excluding the person from the captured image The information processing apparatus according to any one of Items 1 to 16, characterized in that. (Item 18) An input step of receiving a switching instruction for switching the screen, A detection result confirmation screen including the captured image and information on the detection result of the person included in the captured image, a detection region set corresponding to the floor surface of the captured image for detecting the person included in the captured image and counting the number of people, and a region confirmation screen including a foot image indicating the position of the foot of the person, and a screen creation step of creating A display control step of switching the display from the detection result confirmation screen to the region confirmation screen based on the switching instruction, An information processing method characterized by comprising. (Item 19) A program for causing a computer to function as each means of the information processing apparatus according to any one of items 1 to 17.

[0110] The invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Reference Numerals

[0111] 100 ··· Person counting system, 101 ··· Surveillance camera, 102 ··· Person counting server, 103 ··· Browsing terminal, 104 ··· Captured image, 323 ··· Foot position estimation unit, 324 ··· Person counting unit, 332 ··· Input unit, 333 ··· Display control unit, 334 ··· Screen creation unit, 335 ··· Region setting unit.

Claims

1. An input means for receiving a switching instruction for switching the screen; a screen creation means for creating a detection result confirmation screen including a captured image and information related to the detection result of people included in the captured image, and an area confirmation screen including a detection area set corresponding to a floor surface of the captured image in order to detect people included in the captured image and count the number of people, and a foot image showing the position of the feet of the people; a display control means for switching a display from the detection result confirmation screen to the area confirmation screen based on the switching instruction; An information processing device comprising:

2. The area confirmation screen includes a foot image showing the position of the person's feet outside the detection area.

2. The information processing apparatus according to claim 1,

3. the screen creation means creates the detection result confirmation screen in which an identifier for identifying whether the person is counted or not is superimposed on the captured image, the input means accepts a selection of the person not counted on the detection result confirmation screen, The screen creation means creates the area confirmation screen including an image of the feet of the selected person.

2. The information processing apparatus according to claim 1,

4. The screen creation means creates the detection result confirmation screen by superimposing the detection area on the captured image.

2. The information processing apparatus according to claim 1,

5. The input means accepts a selection of the person as the switching instruction, When the person is selected, the display control means switches the display from the detection result confirmation screen to the area confirmation screen.

2. The information processing apparatus according to claim 1,

6. 2 . The information processing apparatus according to claim 1 , wherein the screen generating means generates the area confirmation screen including a head image which is an image of the head of the person corresponding to the foot image.

7. a number-of-people counting means for detecting heads of people included in the captured image and counting the number of people based on the heads; and an estimation means for estimating a position of the feet of the person not counted based on the detected head of the person, The screen generating means generates the area confirmation screen including the foot image showing the estimated foot position of the person who was not counted.

2. The information processing apparatus according to claim 1,

8. The input means accepts deselection of the person, The screen creation means updates the detection area of ​​the detection result confirmation screen based on the cancellation of the selection, The display control means displays the updated detection result confirmation screen.

4. The information processing apparatus according to claim 3.

9. The screen creation means creates the area confirmation screen in which the head image and the foot image corresponding to the head image are connected by a line.

7. The information processing apparatus according to claim 6,

10. A region setting means for setting the detection region is provided.

2. The information processing apparatus according to claim 1,

11. the input means receives an instruction to change the detection area displayed on the area confirmation screen; The area setting means changes the detection area based on the change instruction, The screen creation means updates the area confirmation screen based on the changed detection area.

11. The information processing apparatus according to claim 10,

12. The area setting means sets the detection area in a polygonal shape, The screen creation means creates the area confirmation screen including a size frame indicating the size of the person's head and an image indicating the foot position in association with a corner of the detection area.

11. The information processing apparatus according to claim 10,

13. The area setting means sets an area guide similar to a floor surface included in the captured image, the screen creation means creates the area confirmation screen including the area guide, The input means receives an instruction to change the detection area, When the area setting means receives the change instruction, the area setting means changes the detection area while maintaining a similarity relationship with the area guide.

11. The information processing apparatus according to claim 10,

14. the input means accepts a selection of an area on the detection result confirmation screen, When the person is not detected in the area, the screen creation means creates the area confirmation screen indicating that the person is not detected in the area.

2. The information processing apparatus according to claim 1,

15. The screen creation means creates the area confirmation screen including a first identifier indicating the detected person, a second identifier indicating the selected area, and the foot image corresponding to the person and the area.

15. The information processing apparatus according to claim 14,

16. The input means accepts a selection of the person who was not counted as the switching instruction, The screen creation means creates the area confirmation screen including an image of the feet of the selected person.

2. The information processing apparatus according to claim 1,

17. The screen creation means creates the area confirmation screen including a background image from which the person has been removed from the captured image.

2. The information processing apparatus according to claim 1,

18. an input step of receiving a switching instruction for switching the screen; a screen creation process for creating a detection result confirmation screen including a captured image and information related to the detection result of people included in the captured image, and an area confirmation screen including a detection area set corresponding to a floor surface of the captured image in order to detect people included in the captured image and count the number of people, and a foot image showing the position of the feet of the people; a display control step of switching a display from the detection result confirmation screen to the area confirmation screen based on the switching instruction; An information processing method comprising:

19. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 17.

Images