People flow measurement system, people flow measurement method, and people flow measurement program

Abstract

The people flow measurement system (90) includes a person detection unit (120), a people counting unit (130), and a counted number of people correction unit (140). The person detection unit (120) detects target objects that are objects that appear in an input image, which is an image of the target space, and that exist within the target space. The people counting unit (130) counts the number of detected target objects. The counted number of people correction unit (140) corrects the number of counted target objects using correction coefficients that are generated based on the three-dimensional structure of the target space and are used to correct the number of counted target objects.

Description

Technical Field

[0001] The present disclosure relates to a pedestrian flow measurement system, a pedestrian flow measurement method, and a pedestrian flow measurement program.

Background Art

[0002] In conventional pedestrian flow measurement methods, in order to improve the accuracy of counting the number of people based on images, a correction coefficient obtained based on the analysis result of the image and various conditions at the time of shooting is calculated, and the calculated correction coefficient is used to correct the number of people obtained by an existing number counting method (for example, Patent Document 1). The analysis result of the image is, as a specific example, a result showing the body parts detected from the image and the motion vectors obtained from the image. Various conditions at the time of shooting are, as specific examples, the angle of view and the shooting direction at the time of shooting the image. In Patent Document 1, in order to implement this method, there is disclosed a technique having detection means for detecting a human body from an image taken by a photographing device, counting means for counting the number of human bodies detected by the detection means, and correction means for correcting the number of human bodies counted by the counting means using a correction coefficient based on at least one of the analysis result of the image and the shooting conditions at the time of shooting the image. Here, the correction coefficient is determined by the body parts detected from the image, the motion vectors obtained from the image, the angle of view, and the shooting direction at the time of shooting.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technology disclosed in Patent Document 1, the correction coefficient is calculated based on the appearance in the captured 2D image, without considering the 3D structure of the target space being photographed. However, even if the appearance in the 2D image is the same, the 3D structure of the target space is not necessarily the same. Therefore, if the number of people present in the target space is corrected using a correction coefficient calculated without considering the 3D structure of the target space, the accuracy of the person correction may decrease. Consequently, Patent Document 1 has the problem that the accuracy of the person correction decreases because it does not use a correction coefficient calculated based on the 3D structure of the target space. This disclosure aims to correct the number of people measured within a target space with relatively high accuracy in a pedestrian flow measurement method by using a correction coefficient calculated based on the three-dimensional structure of the target space. [Means for solving the problem]

[0005] The human flow measurement system related to this disclosure is A person detection unit detects an object that is an object that appears in an input image, which is an image of the target space, and which is an object that exists within the target space. A person counting unit that measures the number of detected target objects, A measurement person correction unit corrects the number of measured target objects using a correction coefficient, which is a coefficient generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects. First human flow measuring device equipped with It is equipped with. [Effects of the Invention]

[0006] According to this disclosure, the measurement person correction unit corrects the number of target objects using a correction coefficient generated based on the three-dimensional structure of the target space. Here, the number of target objects may be the number of people measured in the target space. Therefore, according to this disclosure, in a pedestrian flow measurement system, in order to correct the number of people measured in the target space with relatively high accuracy, the number of people measured can be corrected using a correction coefficient calculated considering the three-dimensional structure of the target space. [Brief explanation of the drawing]

[0007] [Figure 1] A diagram showing an example configuration of the pedestrian flow measurement system 90 according to Embodiment 1. [Figure 2] A diagram showing an example of the hardware configuration of the pedestrian flow measurement system 90 according to Embodiment 1. [Figure 3] A flowchart illustrating the operation of the pedestrian flow measurement system 90 according to Embodiment 1. [Figure 4] A diagram illustrating the correction coefficient according to Embodiment 1. [Figure 5] A diagram illustrating the correction coefficient according to Embodiment 1. [Figure 6] This figure shows an example of the hardware configuration of a human flow measurement system 90 according to a modified example of Embodiment 1. [Modes for carrying out the invention]

[0008] In the description and drawings of the embodiments, the same elements and corresponding elements are denoted by the same reference numeral. The descriptions of elements denoted by the same reference numeral are omitted or simplified as appropriate. The arrows in the figures mainly indicate the flow of data or processing. Also, "part" may be interpreted as "circuit," "process," "procedure," "processing," or "circuitry" as appropriate.

[0009] Embodiment 1. This embodiment will now be described in detail with reference to the drawings.

[0010] ***Explanation of the structure*** Figure 1 shows an example of a block diagram of a pedestrian flow measurement system 90 according to Embodiment 1. As shown in Figure 1, the pedestrian flow measurement system 90 includes a camera 100 and a server 200. The camera 100 and the server 200 are connected in a communication manner. The pedestrian flow measurement system 90 may include multiple cameras 100. The server 200 is also called a client server. The client server is also a computer. Camera 100 comprises an imaging unit 110, a person detection unit 120, a person measurement unit 130, and a person measurement correction unit 140. Server 200 comprises an image receiving unit 210, a three-dimensional structure analysis unit 220, and a correction coefficient generation unit 230. Camera 100 is also called the first person flow measurement device. The first person flow measurement device may include a computer. Server 200 is also called the second person flow measurement device. The first person flow measurement device and the second person flow measurement device may be configured as an integrated unit as appropriate, and each part of each device may be provided by other devices.

[0011] The imaging unit 110 captures an image of the target space. The target space is the three-dimensional space that the camera 100 is capturing.

[0012] The person detection unit 120 detects a target object. The target object is an object that appears in the input image and exists within the target space, a specific example being a human body. The input image is an image of the target space. Specifically, the person detection unit 120 detects a person included in the image shown by the image data 2 output from the imaging unit 110. The image shown by image data 2 is also the input image. Image data 2 is data representing an image captured by the imaging unit 110. Image data 2 may represent a still image or a video.

[0013] The people counting unit 130 measures the number of target objects detected by the person detection unit 120. Specifically, the people counting unit 130 measures the number of people in the target space based on the people detected by the person detection unit 120.

[0014] The person measurement correction unit 140 corrects the number of target objects measured by the person measurement unit 130 using a correction coefficient. The correction coefficient is a coefficient generated based on the three-dimensional structure of the target space and is used to correct the number of target objects that have been measured. Specifically, the person measurement correction unit 140 corrects the number of people measured by the person measurement unit 130 using a correction coefficient output from the correction coefficient generation unit 230.

[0015] The image receiving unit 210 receives the image data 2 from the camera 100.

[0016] The three-dimensional structure analysis unit 220 analyzes the three-dimensional structure of the target space, which is the three-dimensional structure represented in the image output by the image receiving unit 210.

[0017] The correction coefficient generation unit 230 generates a correction coefficient based on the three-dimensional structure analyzed by the three-dimensional structure analysis unit 220. The correction coefficient is a coefficient used to correct the number of people measured by the person counting unit 130. The correction coefficient may be a single value, may be prepared for each detected person, may be a value set for each region in the target space, and may be a value set considering the number of people and population density in each region. Also, the correction coefficient may be a value set considering environmental information indicating weather, events, etc. around the target space. The correction coefficient may be a value set considering the presence or absence of umbrellas. When an occluder appears in the input image, the correction coefficient generation unit 230 may generate, as a correction coefficient, a coefficient used to estimate the number of target objects existing behind the occluder when the position of the camera 100 that captured the input image is used as a reference, based on the number of target objects appearing in the reference region. The reference region is a region in the input image and corresponds to the size of the occluder in the input image. The occluder is a three-dimensional structure that shields a part of the target space with reference to the position of the camera 100.

[0018] The corrected measured number of people 9 is data indicating the number of people corrected by the measurement number correction unit 140 and is data indicating the number of people existing in the target space at the time when the image shown by the image data 2 was captured.

[0019] The pedestrian flow measurement system 90, when its various components are integrated, includes hardware such as a processor 11, memory 12, image input interface 13, image processing processor 14, and network interface 15, as shown in Figure 2 as a specific example. These hardware components are appropriately connected via signal lines. The pedestrian flow measurement system 90 may be composed of a general-purpose computer or multiple computers.

[0020] The processor 11 is an integrated circuit (IC) that performs arithmetic operations and controls the hardware of the computer. Specific examples of the processor 11 include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit). The pedestrian flow measurement system 90 may have multiple processors that replace the processor 11. The multiple processors share the role of the processor 11.

[0021] Memory 12 typically consists of volatile and non-volatile memory. A specific example of volatile memory is RAM (Random Access Memory). A specific example of non-volatile memory is ROM (Read Only Memory), HDD (Hard Disk Drive), or flash memory.

[0022] The image input interface 13 is an interface for receiving image data 2, and a specific example is a USB (Universal Serial Bus) terminal. The image processing processor 14 is a processor that has the function of performing image processing. The processor 11 and the image processing processor 14 may implement the functions of each part of the human flow measurement system 90. The network interface 15 consists of a receiver and a transmitter, and specific examples include a communication chip or a NIC (Network Interface Card).

[0023] In this configuration, first, the image data 2 is input to the image receiving unit 210 via the image input interface 13. Next, the input image data 2 is input to the image processing processor 14. The image processing processor 14 detects people in the image and stores person data indicating the detected people in the memory 12. Next, the image processing processor 14 takes in the person data stored in memory 12, increments a counter stored in memory 12 that counts the number of people, and writes the incremented counter back to memory 12 as the number of people count result. The pedestrian flow measurement system 90 applies the above processing to all person data detected from a single image.

[0024] Furthermore, the image processing processor 14 analyzes the three-dimensional structure shown in the image data 2 and stores the three-dimensional structure information showing the analysis results in the memory 12. Next, the processor 11 determines the three-dimensional position of the person detected in the image based on the three-dimensional structural information stored in memory 12, creates a correction coefficient for the number of people measured based on the three-dimensional structural information, and stores the created correction coefficient in memory 12. This process is performed for all detected people. Next, the image processing processor 14 takes in the person count information and correction coefficients stored in the memory 12, performs a correction process using the acquired person count information and correction coefficients, and outputs the result of the correction process to the memory 12. The measured number of people 9 in the memory 12, which indicates the result of the correction process, is sent to the network via the network interface 15.

[0025] Memory 12 stores the pedestrian flow measurement program. The pedestrian flow measurement program is a program that enables the computer to implement the functions of each part of the pedestrian flow measurement system 90, and is a general term for the programs executed in each device of the pedestrian flow measurement system 90. The pedestrian flow measurement program is executed by the processor 11 and the image processing processor 14. The functions of each part of the pedestrian flow measurement system 90 may be implemented by software.

[0026] The data used when executing the pedestrian flow measurement program, and the data obtained by executing the pedestrian flow measurement program, are appropriately stored in the memory device. Each part of the pedestrian flow measurement system 90 utilizes the memory device as appropriate. The memory device consists, specifically, of memory 12, registers in the processor 11, and at least one of the cache memory in the processor 11. Note that the terms data and information may have the same meaning. The memory device may be independent of the computer. The functions of memory 12 may be implemented by other storage devices.

[0027] The pedestrian flow measurement program may be recorded on a computer-readable non-volatile recording medium. Specific examples of non-volatile recording media include optical discs or flash memory. The pedestrian flow measurement program may also be provided as a program product.

[0028] ***Explanation of operation*** The operating procedure of the pedestrian flow measurement system 90 corresponds to the pedestrian flow measurement method. Furthermore, the program that implements the operation of the pedestrian flow measurement system 90 corresponds to the pedestrian flow measurement program.

[0029] Figure 3 is a flowchart illustrating an example of the processing performed by the pedestrian flow measurement system 90 when the system has the configuration shown in Figure 1. The processing of the pedestrian flow measurement system 90 will be explained using Figure 3. It is assumed that the target camera, which is camera 100, executes the camera processing flowchart.

[0030] (Step S101) The imaging unit 110 captures an image of the target space and outputs image data 2 representing the captured image to the server 200.

[0031] (Step S102) The measurement person correction unit 140 receives the correction coefficient from the correction coefficient generation unit 230.

[0032] (Step S103) The target camera performs the following processing on each image captured by the imaging unit 110.

[0033] (Step S104) The imaging unit 110 captures an image of the target space and outputs image data 2 showing the captured image.

[0034] (Step S105) The person detection unit 120 detects people in the image data 2 output by the imaging unit 110.

[0035] (Step S106) The person counting unit 130 measures the number of people in the image shown in the image data 2 based on the people detected by the person detection unit 120.

[0036] (Step S107) The person measurement correction unit 140 corrects the number of people measured by the person measurement unit 130 using a correction coefficient corresponding to the target camera, and outputs the corrected number of people 9, which indicates the corrected number.

[0037] (Step S111) Server 200 performs the following processing for each camera 100.

[0038] (Step S112) The image receiving unit 210 receives image data 2 from the camera 100.

[0039] (Step S113) The 3D structural analysis unit 220 analyzes the 3D structure in the target space shown in the image represented by the received image data 2.

[0040] (Step S114) The correction coefficient generation unit 230 calculates a correction coefficient based on the analysis results of the 3D structure by the 3D structure analysis unit 220, and transmits the calculated correction coefficient to the measurement person correction unit 140. The transmitted correction coefficient is associated with data indicating the camera 100 corresponding to the correction coefficient.

[0041] With the above configuration and operation, the correction coefficient generation unit 230 stores correction coefficients based on the three-dimensional structure of the target being photographed, corresponding to each image data 2.

[0042] Figure 4 is a diagram illustrating the correction coefficient, showing a specific example of an input image and the frames added for explanatory purposes. In Figure 4, the white solid frame and the black solid frame represent the three-dimensional structure analyzed by the three-dimensional structure analysis unit 220, showing the three-dimensional structure of the three-dimensional space that is the target space indicated by the input image. The white solid frame corresponds to the upper part of the three-dimensional space, and the black solid frame corresponds to the lower part of the three-dimensional space. Note that in Figure 4, for explanatory purposes, the white solid frame and the black solid frame are added to only a part of the three-dimensional space, but in reality, the three-dimensional space extends across the entire input image. Therefore, the three-dimensional structure analysis unit 220 analyzes the upper and lower parts of the three-dimensional space that extends across the entire input image. The background area refers to the area located relatively behind the camera 100 in 3D space. The foreground area refers to the area located relatively in front of the camera 100 in 3D space. The person detection unit 120 detects each person in the input image. Generally, in techniques for detecting people from an image, people who appear relatively large can be detected with relatively high accuracy, while people who appear relatively small can only be detected with relatively low accuracy. Therefore, when the correction coefficient generation unit 230 calculates correction coefficients such that the correction for people in the background area is relatively large, and the correction for people in the foreground area is relatively small, the pedestrian flow measurement system 90 can measure the number of people in the target space with relatively high accuracy overall.

[0043] Figure 5 shows an example of a 3D structural analysis performed by the 3D structural analysis unit 220, in which a 3D structure that acts as an obstruction that occludes a portion of the 3D space shown by the input image, as viewed from the position of the camera 100 that captured the image data 2, was analyzed. In this example, there is a possibility that a person is behind the 3D structure, but conventional pedestrian flow measurement methods cannot detect a person behind a 3D structure. Therefore, conventional pedestrian flow measurement methods may not be able to measure the number of people in the target space with relatively high accuracy. In this example, the correction coefficient corresponds to the number of people behind the 3D structure, and it is desirable to set the correction coefficient corresponding to the 3D structure using the 3D size of the analyzed 3D structure as a clue. Therefore, the correction coefficient generation unit 230 calculates a correction coefficient such that, as a specific example, the number of people behind the 3D structure is equivalent to the number of people measured in an area that is 3D spatially equivalent to the area behind the 3D structure (as a specific example, the area indicated by the dashed frame in Figure 5). The area indicated by the dashed frame corresponds to the reference area. By using the correction coefficient calculated in this way, it is possible to calculate the number of people with a certain degree of accuracy even in areas occluded by occluders in the input image. Therefore, the pedestrian flow measurement system 90 can measure the number of people with relatively high accuracy overall. The correction factor may also be data indicating the area to be referenced when estimating the number of people behind an obstacle.

[0044] ***Explanation of the effects of Embodiment 1*** As described above, according to this embodiment, the number of people to be measured is corrected using a correction coefficient calculated considering the three-dimensional structure of the target space. Therefore, according to this embodiment, the number of people can be measured more accurately in the pedestrian flow measurement method.

[0045] ***Other configurations*** <Example 1> Figure 6 shows an example of the hardware configuration of each device in the pedestrian flow measurement system 90 according to this modified example. Figure 6 shows the server 200 as a representative of each device. Each device in the pedestrian flow measurement system 90 is equipped with a processing circuit 18 instead of a processor 11, a processor 11 and an image processing processor 14, or a processor 11 and a memory 12. The processing circuit 18 is hardware that implements at least some of the components of the human flow measurement system 90. The processing circuit 18 may be dedicated hardware, or it may be a processor that executes the program stored in memory 12.

[0046] When the processing circuit 18 is dedicated hardware, specific examples of the processing circuit 18 include a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The pedestrian flow measurement system 90 may include multiple processing circuits that replace the processing circuit 18. The multiple processing circuits share the role of the processing circuit 18.

[0047] In the pedestrian flow measurement system 90, some functions may be implemented by dedicated hardware, while the remaining functions may be implemented by software or firmware.

[0048] The processing circuit 18 can be implemented, in specific examples, by hardware, software, firmware, or a combination thereof. The processor 11, memory 12, image processing processor 14, and processing circuit 18 are collectively referred to as the "processing circuitry." In other words, the functions of each functional component of the human flow measurement system 90 are realized by the processing circuitry.

[0049] ***Other Embodiments*** Although Embodiment 1 has been described, multiple parts of this embodiment may be combined and implemented. Alternatively, this embodiment may be implemented partially. Furthermore, this embodiment may be modified in various ways as needed, and may be implemented as a whole or in parts in any combination. The embodiments described above are essentially preferred examples and are not intended to limit the scope of this disclosure, its applications, or its uses. The procedures described using flowcharts, etc., may be modified as appropriate. [Explanation of symbols]

[0050] 2 Image data, 9 Corrected number of people measured, 11 Processor, 12 Memory, 13 Image input interface, 14 Image processing processor, 15 Network interface, 18 Processing circuit, 90 People flow measurement system, 100 Camera, 110 Imaging unit, 120 Person detection unit, 130 People measurement unit, 140 Measured number correction unit, 200 Server, 210 Image receiving unit, 220 3D structure analysis unit, 230 Correction coefficient generation unit.

Claims

1. a person detection unit that detects a target object that is an object captured in an input image that is an image of a target space and exists in the target space; a people counting unit that counts the number of detected target objects; a number of people correction unit that corrects the number of measured target objects using a correction coefficient that is generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects; A first people flow measurement device comprising: A people flow measurement system comprising: A people flow measurement system in which the three-dimensional structure indicates the depth within the target space based on the position of the device that captured the input image.

2. The people flow measurement system further comprises: a three-dimensional structure analysis unit that analyzes a three-dimensional structure of the target space based on the input image; a correction coefficient generation unit that generates the correction coefficients based on the analyzed three-dimensional structure; A second people flow measurement device comprising: The people flow measurement system according to claim 1 , comprising:

3. 3. The people flow measurement system according to claim 2, wherein, when an obstructing object is captured in the input image, the correction coefficient generation unit generates the correction coefficient based on the number of target objects captured in a reference area, which is an area in the input image corresponding to the size of the obstructing object in the input image, to estimate the number of target objects present behind the obstructing object when the position of the camera that captured the input image is used as a reference.

4. the first people flow measurement device is a camera; 4. The people flow measurement system according to claim 2 or 3, wherein the second people flow measurement device is a computer.

5. The people flow measurement system according to claim 1 , wherein the target object is a human body.

6. a computer detects a target object that is an object captured in an input image that is an image of a target space and that exists in the target space; The computer counts the number of detected target objects; The computer corrects the number of measured target objects using a correction coefficient, which is a coefficient generated based on a three-dimensional structure of the target space and is used to correct the number of measured target objects, A people flow measurement method in which the three-dimensional structure indicates depth within the target space based on the position of the device that captured the input image.

7. a person detection process for detecting a target object that is an object captured in an input image that is an image of a target space and exists in the target space; a people counting process for counting the number of detected target objects; A counting number of people correction process that corrects the number of measured target objects using a correction coefficient that is a coefficient generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects. A people flow measurement program that causes a computer included in the first people flow measurement device to execute the above. A people flow measurement program in which the three-dimensional structure indicates the depth within the target space based on the position of the device that captured the input image.

Images