Human flow measurement system, human flow measurement method, and human flow measurement program
The people flow measurement system addresses the inaccuracy in conventional methods by using a correction coefficient based on the three-dimensional structure of the target space, enhancing the accuracy of people counting through a camera-server configuration with a person detection and correction unit, effectively measuring the number of people with improved precision.
Patent Information
- Application Number
- PCT/JP2023/017883
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-21
AI Technical Summary
Conventional people flow measurement methods inaccurately correct the number of people in a target space due to the lack of consideration for the three-dimensional structure of the imaging target space, leading to decreased accuracy.
A people flow measurement system that uses a correction coefficient generated based on the three-dimensional structure of the target space to correct the number of detected people, incorporating a camera and server configuration with a person detection unit, people counting unit, and a measured number correction unit to enhance accuracy.
The system effectively corrects the number of people measured in the target space with high accuracy by considering the three-dimensional structure, improving the overall precision of people flow measurement.
Smart Images

Figure JP2023017883_21112024_PF_FP_ABST
Abstract
Description
People flow measurement system, people flow measurement method, and people flow measurement program
[0001] The present disclosure relates to a people flow measurement system, a people flow measurement method, and a people flow measurement program.
[0002] Conventional people flow counting methods improve the accuracy of image-based people counting by calculating a correction coefficient based on image analysis results and various conditions at the time of capture, and then correcting the number of people counted using existing people counting methods using the calculated correction coefficient (see, for example, Patent Document 1). The image analysis results, for example, include results showing the body parts of the human body detected in the image and the motion vectors obtained from the image. The various conditions at the time of capture, for example, include the angle of view and the shooting direction when the image was captured. Patent Document 1 discloses a technology for implementing this method, which includes a detection means for detecting human bodies in images captured by a camera, a counting means for counting the number of human bodies detected by the detection means, and a correction means for correcting the number of human bodies counted by the counting means using a correction coefficient based on at least one of the image analysis results and the shooting conditions at the time of capture. Here, the correction coefficient is determined based on the body parts of the human body detected in the image, the motion vectors obtained from the image, and the angle of view and shooting direction when the image was captured.
[0003] JP 2018-151960 A
[0004] In the technology disclosed in Patent Document 1, correction coefficients are calculated based on the appearance in the captured two-dimensional image without considering the three-dimensional structure of the target space being photographed. Here, even if the appearance in the two-dimensional image is the same, the three-dimensional structure of the target space is not necessarily the same. Therefore, correcting the number of people present in the target space using a correction coefficient calculated without considering the three-dimensional structure of the target space may reduce the accuracy of the number of people correction. Therefore, Patent Document 1 has the problem of reducing the accuracy of the number of people correction because it does not use a correction coefficient calculated based on the three-dimensional structure of the target space. The present disclosure aims to correct the measured number of people in a people flow measurement method using a correction coefficient calculated based on the three-dimensional structure of the target space in order to correct the measured number of people in the target space with relatively high accuracy.
[0005] The people flow measurement system according to the present disclosure comprises a first people flow measurement device comprising: a person detection unit that detects target objects that are objects that appear in an input image, which is an image of a target space and that exist within the target space; a people counting unit that counts the number of detected target objects; and a counted number of people correction unit that corrects the number of measured target objects using a correction coefficient that is generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects.
[0006] According to the present disclosure, a counted number of people correction unit corrects the number of target objects using a correction coefficient generated based on the three-dimensional structure of the target space. Here, the number of target objects may be the number of people measured in the target space. Therefore, according to the present disclosure, in a people flow measurement method, in order to correct the number of people measured in the target space with relatively high accuracy, the counted number of people can be corrected using a correction coefficient calculated taking into account the three-dimensional structure of the target space.
[0007] A diagram showing an example of the configuration of a people flow measurement system 90 according to embodiment 1. A diagram showing an example of the hardware configuration of a people flow measurement system 90 according to embodiment 1. A flowchart showing the operation of a people flow measurement system 90 according to embodiment 1. A diagram explaining correction coefficients according to embodiment 1. A diagram explaining correction coefficients according to embodiment 1. A diagram showing an example of the hardware configuration of a people flow measurement system 90 according to a modified example of embodiment 1.
[0008] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. The description of elements given the same reference numerals will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate the flow of data or the flow of processing. Furthermore, "unit" may be read as "circuit," "step," "procedure," "process," or "circuitry" as appropriate.
[0009] First Embodiment Hereinafter, the present embodiment will be described in detail with reference to the drawings.
[0010] ***Description of Configuration*** FIG. 1 shows an example of a block diagram of a people flow measurement system 90 according to the first embodiment. As shown in FIG. 1, the people flow measurement system 90 includes a camera 100 and a server 200. The camera 100 and the server 200 are connected so that they can communicate with each other. The people flow measurement system 90 may include multiple cameras 100. The server 200 is also called a client-server. The client-server is also a computer. The camera 100 includes an imaging unit 110, a person detection unit 120, a people counting unit 130, and a counted number of people correction unit 140. The server 200 includes an image receiving unit 210, a three-dimensional structure analysis unit 220, and a correction coefficient generation unit 230. The camera 100 is also called a first people flow measurement device. The first people flow measurement device may include a computer. The server 200 is also called a second people flow measurement device. The first people flow measurement device and the second people flow measurement device may be configured integrally as appropriate, or each unit of each device may be included in the other device.
[0011] The image capturing unit 110 captures an image of a target space. The target space is a three-dimensional space that is the target of the camera 100 to capture.
[0012] The person detection unit 120 detects a target object. The target object is an object that appears in the input image and exists in the target space, and a specific example is a human body. The input image is an image of the target space. As a specific example, the person detection unit 120 detects a person included in an image represented by image data 2 output from the imaging unit 110. The image represented by the image data 2 is also the input image. The image data 2 is data representing an image captured by the imaging unit 110. The image data 2 may be data representing a still image or data representing a moving image.
[0013] The people counting unit 130 counts the number of target objects detected by the person detection unit 120. As a specific example, the people counting unit 130 counts the number of people in the target space based on the people detected by the person detection unit 120.
[0014] The counted number of people correction unit 140 uses a correction coefficient to correct the number of target objects counted by the people counting unit 130. The correction coefficient is a coefficient generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects. As a specific example, the counted number of people correction unit 140 uses the correction coefficient output from the correction coefficient generation unit 230 to correct the number of people counted by the people counting unit 130.
[0015] The image receiving unit 210 receives the image data 2 from the camera 100 .
[0016] The three-dimensional structure analysis unit 220 analyzes the three-dimensional structure of the target space, which is expressed in the image output by the image receiving unit 210 .
[0017] The correction coefficient generation unit 230 generates a correction coefficient based on the three-dimensional structure analyzed by the three-dimensional structure analysis unit 220. The correction coefficient is a coefficient used to correct the number of people counted by the people counting unit 130. The correction coefficient may be a single value, may be prepared for each detected person, or may be a value set for each area in the target space, taking into account the number of people and population density in each area. The correction coefficient may also be a value set taking into account environmental information indicating the weather, events, etc. around the target space. The correction coefficient may also be a value set taking into account the presence or absence of an umbrella. When an obstruction is captured in the input image, the correction coefficient generation unit 230 may generate a correction coefficient based on the number of target objects captured in the reference area, to estimate the number of target objects present behind the obstruction when the position of the camera 100 that captured the input image is used as the reference. The reference area is an area in the input image that corresponds to the size of the obstruction in the input image. The obstruction is a three-dimensional structure that obstructs part of the target space based on the position of the camera 100.
[0018] The corrected counted number of people 9 is data indicating the number of people corrected by the counted number of people correction unit 140, and is data indicating the number of people present in the target space at the time the image indicated by the image data 2 was captured.
[0019] When the various components of the people flow measurement system 90 are integrated, the system 90 includes hardware such as a processor 11, a memory 12, an image input interface 13, an image processing processor 14, and a network interface 15, as shown in Fig. 2. These pieces of hardware are connected as appropriate via signal lines. The people flow measurement system 90 may be configured from a general computer or from multiple computers.
[0020] The processor 11 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of the computer. Specific examples of the processor 11 include a central processing unit (CPU), a digital signal processor (DSP), or a graphics processing unit (GPU). The people flow measurement system 90 may include multiple processors that replace the processor 11. The multiple processors share the role of the processor 11.
[0021] The memory 12 typically includes a volatile storage device and a non-volatile storage device. A specific example of the volatile storage device is a random access memory (RAM). A specific example of the non-volatile storage device is a read-only memory (ROM), a hard disk drive (HDD), or a flash memory.
[0022] The image input interface 13 is an interface for receiving image data 2, and is specifically a USB (Universal Serial Bus) terminal. The image processor 14 is a processor having the function of executing image processing. The processor 11 and the image processor 14 may realize the functions of each unit included in the people flow measurement system 90. The network interface 15 is a receiver and transmitter, and is specifically a communication chip or a NIC (Network Interface Card).
[0023] In this configuration, image data 2 is first input to the image receiving unit 210 via the image input interface 13. Next, the input image data 2 is input to the image processing processor 14. The image processing processor 14 detects people in the image and stores person data indicating the detected people in the memory 12. Next, the image processing processor 14 imports the person data stored in the memory 12, increments a counter that is saved in the memory 12 and counts the number of people, and writes the incremented counter back to the memory 12 as the number of people counting result. The people flow measurement system 90 performs the above-described process for all person data detected from one image.
[0024] The image processor 14 also analyzes the three-dimensional structure indicated by the image data 2 and stores three-dimensional structure information indicating the analysis results in the memory 12. The processor 11 then determines the three-dimensional positions of people detected in the image based on the three-dimensional structure information stored in the memory 12, creates correction coefficients for the counted number of people based on the three-dimensional structure information, and stores the created correction coefficients in the memory 12. This process is performed for all detected people. The image processor 14 then loads the number of people information and the correction coefficients stored in the memory 12, performs a correction process using the loaded number of people information and the correction coefficients, and outputs the results of the correction process to the memory 12. The counted number of people 9 in the memory 12 indicating the results of the correction process are sent to the network via the network interface 15.
[0025] The memory 12 stores a people flow measurement program. The people flow measurement program is a program that causes a computer to realize the functions of each unit of the people flow measurement system 90, and is a general term for programs executed by each device of the people flow measurement system 90. The people flow measurement program is executed by the processor 11 and the image processing processor 14. The functions of each unit of the people flow measurement system 90 may be realized by software.
[0026] Data used when executing the people flow measurement program and data obtained by executing the people flow measurement program are stored in a storage device as appropriate. Each part of the people flow measurement system 90 uses a storage device as appropriate. Specific examples of the storage device include at least one of the memory 12, a register in the processor 11, and a cache memory in the processor 11. Note that the terms "data" and "information" may have the same meaning. The storage device may be independent of the computer. The function of the memory 12 may be realized by another storage device.
[0027] The people flow measurement program may be recorded on a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The people flow measurement program may be provided as a program product.
[0028] ***Explanation of Operation*** The operation procedure of the people flow measurement system 90 corresponds to a people flow measurement method. Also, the program that realizes the operation of the people flow measurement system 90 corresponds to a people flow measurement program.
[0029] Fig. 3 is a flowchart showing an example of the processing of people flow measurement system 90 when people flow measurement system 90 has the configuration shown in Fig. 1. The processing of people flow measurement system 90 will be described using Fig. 3. It is assumed that the target camera, which is camera 100, executes the camera processing flowchart.
[0030] (Step S101 ) The imaging unit 110 captures an image showing the target space, and outputs image data 2 indicating the captured image to the server 200 .
[0031] (Step S102 ) The counted number of people correction unit 140 receives the correction coefficient from the correction coefficient generation unit 230 .
[0032] (Step S103) The target camera performs the following process on each image captured by the image capturing unit 110.
[0033] (Step S104) The imaging unit 110 captures an image showing the target space, and outputs image data 2 indicating the captured image.
[0034] (Step S105 ) The person detection unit 120 detects people appearing in the image data 2 output by the imaging unit 110 .
[0035] (Step S106 ) The number of people counting unit 130 counts the number of people in the image represented by the image data 2 based on the people detected by the person detection unit 120 .
[0036] (Step S107) The counted number of people correction unit 140 corrects the number of people counted by the people counting unit 130 using the correction coefficient corresponding to the target camera, and outputs a corrected counted number of people 9 indicating the corrected number of people.
[0037] (Step S111) The server 200 executes the following process for each camera 100.
[0038] (Step S112 ) The image receiving unit 210 receives the image data 2 from the camera 100 .
[0039] (Step S113) The three-dimensional structure analysis unit 220 analyzes the three-dimensional structure in the target space shown in the image indicated by the received image data 2.
[0040] (Step S114) The correction coefficient generation unit 230 calculates a correction coefficient based on the analysis result of the three-dimensional structure by the three-dimensional structure analysis unit 220, and transmits the calculated correction coefficient to the number of people correction unit 140. Note that the transmitted correction coefficient is associated with data indicating the camera 100 corresponding to the correction coefficient.
[0041] With the above-described configuration and operation, the correction coefficient generating unit 230 stores correction coefficients based on the three-dimensional structure of the subject corresponding to each image data 2 .
[0042] FIG. 4 is a diagram illustrating correction coefficients, showing a specific example of an input image and each frame added for ease of explanation. In FIG. 4, the white solid frame and the black solid frame represent the three-dimensional structure analyzed by the three-dimensional structure analysis unit 220, which is the three-dimensional structure of the three-dimensional space that is the target space represented by the input image. The white solid frame corresponds to the upper part of the three-dimensional space, and the black solid frame corresponds to the lower part of the three-dimensional space. Note that in FIG. 4, the white solid frame and the black solid frame are added to only a portion of the three-dimensional space for ease of explanation, but in reality, the three-dimensional space extends throughout the entire input image. Therefore, the three-dimensional structure analysis unit 220 analyzes the upper and lower parts of the three-dimensional space that extends throughout the entire input image. The back area indicates an area that exists relatively farther in the three-dimensional space based on the position of the camera 100. The near area indicates an area that exists relatively closer in the three-dimensional space based on the position of the camera 100. The person detection unit 120 detects each person appearing in the input image. Generally, in technology for detecting people from an image, people who appear relatively large can be detected with relatively high accuracy, but people who appear relatively small can only be detected with relatively low accuracy. Therefore, when the correction coefficient generation unit 230 calculates correction coefficients such that the correction corresponding to people appearing in the back area is relatively large and the correction corresponding to people appearing in the front area is relatively small, the people flow measurement system 90 can measure the number of people in the target space with relatively high accuracy overall.
[0043] FIG. 5 shows an example of a three-dimensional structure analyzed by the three-dimensional structure analysis unit 220, which is an object blocking a portion of the three-dimensional space represented by the input image as viewed from the position of the camera 100 that captured the image data 2. In this example, a person may be present behind the three-dimensional structure, but conventional people flow measurement methods are unable to detect the person behind the three-dimensional structure. Therefore, conventional people flow measurement methods may be unable to measure the number of people in the target space with a relatively high degree of accuracy. In this example, a correction coefficient corresponding to the number of people behind the three-dimensional structure is preferably set based on the three-dimensional size of the analyzed three-dimensional structure. Therefore, the correction coefficient generation unit 230 calculates a correction coefficient that, as a specific example, makes the number of people behind the three-dimensional structure equal to the number of people measured in an area three-dimensionally equivalent to the area behind the three-dimensional structure (the area indicated by the dashed frame in FIG. 5 ). The area indicated by the dashed frame corresponds to the reference area. By using the correction coefficient calculated in this way, it is possible to calculate the number of people with a certain degree of accuracy even in areas in the input image that are blocked by obstructions. Therefore, the people flow measurement system 90 can measure the number of people with a relatively high degree of accuracy overall. The correction coefficient may also be data indicating the area to be referenced when estimating the number of people behind obstructions.
[0044] ***Explanation of the Effects of Embodiment 1*** As described above, according to this embodiment, the counted number of people is corrected using a correction coefficient calculated taking into account the three-dimensional structure of the target space. Therefore, according to this embodiment, the number of people can be counted more accurately in the people flow counting method.
[0045] ***Other Configurations*** <Variation 1> Fig. 6 shows an example of the hardware configuration of each device included in the people flow measurement system 90 according to this variation. Fig. 6 shows the server 200 as a representative of each device. Each device included in the people flow measurement system 90 includes a processing circuit 18 instead of the processor 11, the processor 11 and image processor 14, or the processor 11 and memory 12. The processing circuit 18 is hardware that realizes at least a portion of each unit included in the people flow measurement system 90. The processing circuit 18 may be dedicated hardware, or may be a processor that executes a program stored in the memory 12.
[0046] When the processing circuitry 18 is dedicated hardware, the processing circuitry 18 may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The people flow measurement system 90 may include multiple processing circuits that replace the processing circuitry 18. The multiple processing circuits share the role of the processing circuitry 18.
[0047] In the people flow measurement system 90, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.
[0048] The processing circuitry 18 is realized by, for example, hardware, software, firmware, or a combination of these. The processor 11, memory 12, image processor 14, and processing circuitry 18 are collectively referred to as the "processing circuitry." In other words, the functions of each functional component of the people flow measurement system 90 are realized by the processing circuitry.
[0049] ***Other Embodiments*** Although the first embodiment has been described, it is also possible to combine multiple parts of this embodiment and implement it. Alternatively, it is also possible to implement this embodiment in part. In addition, this embodiment may be modified in various ways as needed, and may be implemented in any combination, either as a whole or in part. Note that the above-described embodiments are essentially preferred examples and are not intended to limit the scope of the present disclosure, its applications, and uses. The procedures described using flowcharts, etc. may be modified as appropriate.
[0050] 2 Image data, 9 Corrected counted number of people, 11 Processor, 12 Memory, 13 Image input interface, 14 Image processing processor, 15 Network interface, 18 Processing circuit, 90 People flow measurement system, 100 Camera, 110 Imaging unit, 120 Person detection unit, 130 Number of people counting unit, 140 Counted number of people correction unit, 200 Server, 210 Image receiving unit, 220 Three-dimensional structure analysis unit, 230 Correction coefficient generation unit.
Claims
1. A people flow measurement system comprising a first people flow measurement device comprising: a person detection unit that detects target objects that are objects appearing in an input image, which is an image of a target space, and that are objects present in the target space; a people counting unit that measures the number of detected target objects; and a measured people count correction unit that corrects the number of measured target objects using a correction coefficient that is generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects.
2. The people flow measurement system according to claim 1, further comprising a second people flow measurement device comprising: a three-dimensional structure analysis unit that analyzes a three-dimensional structure of the target space based on the input image; and a correction coefficient generation unit that generates the correction coefficient based on the analyzed three-dimensional structure.
3. The people flow measurement system of claim 2, wherein the correction coefficient generation unit generates, when an obstruction is captured in the input image, a correction coefficient based on the number of target objects captured in a reference area, which is an area in the input image corresponding to the size of the obstruction in the input image, to generate a coefficient used to estimate the number of target objects present behind the obstruction when based on the position of the camera that captured the input image.
4. The people flow measuring system according to claim 2 or 3, wherein the first people flow measuring device is a camera, and the second people flow measuring device is a computer.
5. The people flow measurement system according to any one of claims 1 to 4, wherein the target object is a human body.
6. A people flow measurement method in which a computer detects target objects that are objects captured in an input image, which is an image of a target space, and that are present in the target space, the computer counts the number of detected target objects, and the computer corrects the number of measured target objects using a correction coefficient that is generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects.
7. A people flow measurement program that causes a computer equipped in a first people flow measurement device to execute the following steps: a person detection process that detects target objects that are objects that appear in an input image, which is an image of a target space and that exist in the target space; a people counting process that counts the number of detected target objects; and a measured number of people correction process that corrects the number of measured target objects using a correction coefficient that is generated based on the three-dimensional structure of the target space and is used to correct the number of measured target objects.
Citation Information
Patent Citations
Congestion monitoring system and congestion monitoring program
JP2005025328A
Information processing device, information processing method, and program
JP2021071923A
Information processing device, information processing method, and program
JP2022022874A
