Crowd management device using fixed cameras and PTZ camera, driving method therefor, and crowd management system

Abstract

The present invention relates to a crowd management device using fixed cameras and a PTZ camera, a driving method therefor, and a crowd management system. According to an embodiment of the present invention, the crowd management device using fixed cameras and a PTZ camera comprises: a communication interface unit that communicates with each of a plurality of fixed cameras for covering and imaging a first range area in an arbitrary place and a PTZ camera for circulating and imaging a second range area except for the first range area in the arbitrary place according to a preset method; and a control unit that determines whether there is an overlap in each of the captured images of the first range area and the second range area, and manages a crowd in the arbitrary place on the basis of the determination result.

Description

Crowd management device using fixed cameras and PTZ cameras, operating method thereof, and crowd management system

[0001] The present invention relates to a crowd management device utilizing a fixed camera and a PTZ camera, a method for operating the device, and a crowd management system. More specifically, for example, when each exit route is monitored by a fixed camera at the center of an intersection where a CCTV is installed, and a PTZ is installed for long-distance camera detection, the PTZ scans a limited area of ​​the CCTV detection range to count and estimate a crowd, maps the crowd all over a single global space, and detects the crowd to minimize blind spots in crowd detection. The present invention relates to a crowd management device utilizing a fixed camera and a PTZ camera, a method for operating the device, and a crowd management system.

[0002] In modern society, effectively monitoring and managing the movement of people in urban areas and transportation hubs is emerging as a critical challenge. This is directly related to public safety, smooth traffic flow, and rapid response in emergency situations.

[0003] Conventional CCTV systems have been used to continuously monitor specific areas from fixed locations. While these fixed cameras enable continuous monitoring of the surveillance area, their limited coverage and the presence of blind spots remain major drawbacks. These limitations are particularly evident in complex urban areas.

[0004] PTZ cameras were introduced as a solution. PTZ cameras can be remotely controlled to change direction and adjust zoom, offering the advantage of flexible surveillance of a wider area. However, systems using only PTZ cameras run the risk of focusing on specific areas while neglecting surveillance of other areas.

[0005] Another way to address the limitations of surveillance areas is to install additional CCTV. However, attempts to address these blind spots face two major challenges: excessive costs and privacy concerns. Privacy is a critical issue in modern society, and the proliferation of surveillance cameras exacerbates these concerns.

[0006] The purpose of an embodiment of the present invention is to provide a system configuration and detection technology for counting and estimating a crowd while the PTZ scans a limited area of ​​the CCTV detection range when each exit route is monitored by a fixed camera at the center of an intersection where CCTV is installed, and a PTZ is installed for long-distance camera detection, and to map all of this to a single global space to minimize blind spots in crowd detection, a crowd management device utilizing a fixed camera and a PTZ camera, and an operating method of the device, and a crowd management system.

[0007] A crowd management device utilizing a fixed camera and a PTZ camera according to an embodiment of the present invention includes a communication interface unit that communicates with a plurality of fixed cameras that each manage and photograph a first range of an area in an arbitrary location, and a PTZ (Pan Tilt Zoom) camera that circulates and photographs a second range of an area excluding the first range of an area in the arbitrary location according to a preset method, and a control unit that determines whether each image of the captured images of the first range of an area and the second range of an area overlaps and manages a crowd in the arbitrary location based on the determination result.

[0008] The control unit can select a plurality of image coordinates of the fixed camera, map the selected plurality of image coordinates to pan and tilt coordinates of the PTZ camera, and then convert the image of the fixed camera into the spherical coordinate system of the PTZ camera to determine whether there is an overlap.

[0009] The above control unit can manage the average of each analysis result of image data for the overlapping shooting areas when the shooting areas of the fixed camera and the PTZ camera overlap.

[0010] The above control unit can operate the PTZ camera at one of a low speed lower than a reference speed and a high speed higher than the reference speed when moving sequentially, and can temporarily stop the analysis operation of the captured image when moving at the high speed.

[0011] The control unit can move the PTZ camera at high speed to a location where an incident or accident occurs in the first range of areas and the second range of areas, and then resume the analysis operation of the captured image.

[0012] In addition, a method for operating a crowd management device using a fixed camera and a PTZ camera according to an embodiment of the present invention includes a step in which a communication interface unit communicates with a plurality of fixed cameras that each manage and photograph a first range of an area in an arbitrary location and a PTZ camera that circulates and photographs a second range of an area excluding the first range of an area according to a preset method, and a step in which a control unit determines whether each image of the captured images of the first range of an area and the second range of an area overlaps and manages a crowd in the arbitrary location based on the determination result.

[0013] The step of determining whether there is an overlap may include selecting a plurality of image coordinates of the fixed camera, mapping the selected plurality of image coordinates to pan and tilt coordinates of the PTZ camera, and then converting the image of the fixed camera into the spherical coordinate system of the PTZ camera to determine whether there is an overlap.

[0014] The step of managing the crowd above can be performed by averaging the analysis results of each image data for the overlapping shooting areas when the shooting areas of the fixed camera and the PTZ camera overlap.

[0015] The step of managing the crowd may include a step of operating the PTZ camera sequentially at one of a low speed lower than a reference speed and a high speed higher than the reference speed, and temporarily stopping the analysis operation of the captured image when moving at the high speed.

[0016] The step of managing the crowd may further include the step of moving the PTZ camera at high speed to a location where an incident or accident event occurs in the first range of areas and the second range of areas, and then resuming the analysis operation of the captured image.

[0017] Furthermore, a crowd management system utilizing a fixed camera and a PTZ camera according to an embodiment of the present invention includes a plurality of fixed cameras that each control and photograph a first range of an area in an arbitrary location, a PTZ camera that circulates and photographs a second range of an area excluding the first range of an area in the arbitrary location according to a preset method, and a crowd management device that communicates with the plurality of fixed cameras and the PTZ camera, respectively, to determine whether each image overlaps with respect to the captured images of the first range of an area and the second range of an area, and manages a crowd in the arbitrary location based on the determination result.

[0018] According to an embodiment of the present invention, the combination of a fixed camera and a PTZ camera enables continuous, comprehensive surveillance of a wide area. While fixed cameras monitor a specific area in real time, PTZ cameras periodically scan a wide area or film in a touring manner, enabling them to cover a larger area.

[0019] Furthermore, according to embodiments of the present invention, the periodic scanning and zoom-in functions of the PTZ camera enable accurate counting of crowds even over a wide area. This is particularly useful for tracking and analyzing the dynamic movement of crowds at large events or public spaces.

[0020] Furthermore, according to an embodiment of the present invention, the system integrates and processes image data captured by fixed cameras and PTZ cameras. By averaging overlapping captured areas, data consistency is maintained and more accurate results are provided.

[0021] The system according to an embodiment of the present invention can be usefully utilized in various fields, including public safety, traffic management, and large-scale event management. In particular, it enhances public safety by enabling rapid and accurate responses in emergency situations.

[0022] Furthermore, according to embodiments of the present invention, the zoom control and scanning mechanism of the PTZ camera provide flexibility to adapt to various environments and requirements. This allows users to adjust and optimize the system to suit specific situations.

[0023] The above effects are the main advantages of the embodiments of the present invention, and contribute to greatly improving efficiency and accuracy in crowd management and surveillance in various environments.

[0024] Figure 1 is a drawing illustrating a crowd management system according to an embodiment of the present invention;

[0025] Figure 2 is an example diagram showing the field of view of a fixed camera and the touring range area of ​​a PTZ camera, with a fixed camera and a PTZ camera installed in the same location, and showing the field of view of each camera.

[0026] Figure 3 is an example of a table mapping the correct hardware (HW) zoom value by zoom ratio and the order in which PTZ camera calibration is performed.

[0027] Figure 4 is a flowchart for explaining the method of linking a regular camera and a PTZ camera.

[0028] Figure 5 is a flowchart for explaining the crowd counting and analysis process using a PTZ camera.

[0029] Figure 6 is a flowchart for the crowd counting and data integration process using fixed and PTZ cameras.

[0030] Figure 7 is a block diagram illustrating the detailed structure of the crowd management device of Figure 1, and

[0031] Figure 8 is a flowchart showing the operation process of the crowd management device of Figure 1.

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0033] FIG. 1 is a drawing illustrating a crowd management system according to an embodiment of the present invention; FIG. 2 is an exemplary drawing showing a fixed camera and a PTZ camera installed at the same location, and indicating the field of view of each camera, showing the field of view of a fixed camera and a touring range area of ​​a PTZ camera; and FIG. 3 is an exemplary drawing of a table mapping the order in which PTZ camera correction is performed and the exact hardware (HW) zoom value by zoom ratio.

[0034] As illustrated in FIGS. 1 and 2, a crowd management system (90) according to an embodiment of the present invention is a crowd management system utilizing, for example, a fixed camera and a PTZ camera, and may include a fixed camera (or a first type of camera) (100), a PTZ camera (or a second type of camera, a non-fixed shooting camera) (110), a crowd management device (or an image analysis device) (120), and a communication network (e.g., a 5G network, a 6G network, a cloud communication network, a WCDMA communication network, etc.) of a communication company.

[0035] Here, “including some or all” means that the system is configured by omitting some components such as a fixed camera (100) (e.g., a configuration is possible by simply linking with a camera such as a CCTV installed by another company), or that some or all of the components constituting the crowd management device (120) can be configured by being integrated into a fixed camera (100), a PTZ camera (110), or a network device (e.g., a wireless switching device, etc.) constituting a communication network, and is described as including all in order to help a sufficient understanding of the invention.

[0036] A fixed camera (100) refers to a camera installed at a specific location and fixedly photographs a designated area of ​​an arbitrary location. For example, the fixed camera (100) may be a closed-circuit television (CCTV) camera that detects a designated area in real time, and may be installed, for example, to monitor each exit and entry route from the center of an intersection. Of course, the fixed camera (100) according to the embodiment of the present invention is preferably a plurality of CCTV cameras that continuously monitor each area of ​​an arbitrary location, and as shown in FIG. 2, a plurality of fixed cameras can each monitor and photograph a first range of an arbitrary location. Of course, the fixed camera (100) may also be an IP camera. The fixed camera (100) may also be equipped with an image analysis device (e.g., an edge device) that analyzes the captured images from the fixed camera (100) either internally or adjacently. However, in the embodiment of the present invention, for the convenience of explanation, the system will be described assuming that data analysis is performed in a separate crowd management device (120) as shown in FIG. 1. Of course, the crowd management device (120) may be, for example, an edge device or a server, but in cases where the amount of data calculation is large, data calculation processing may also be performed on the side of the fixed camera (100), and thus the embodiment of the present invention will not be particularly limited to any one form.

[0037] A PTZ camera (110) refers to a camera that can change direction and adjust zoom through remote control, etc. The PTZ camera (110) can not only zoom in on a fixed area and film, but can also film while changing the filming location. In the past, a control center's control personnel would normally remotely control the PTZ camera (110), but in the embodiment of the present invention, an area that the plurality of fixed cameras (100) above cannot monitor or a second range, which can be called a blind spot, can be automatically filmed from any location according to a preset method, and at this time, an action such as sequentially moving to a specific area, that is, filming while circulating (or touring) can be automatically performed. Of course, automatic settings can be made in advance when installing the PTZ camera (110) for such automatic filming. In other words, it can be considered that programming is performed. Of course, the PTZ camera (110) may be equipped with an artificial intelligence (AI) program inside and may perform automated operations by executing the program. Similarly to the fixed camera (100), the PTZ camera (110) may also be equipped with a separate image analysis module that analyzes the captured image. However, in the embodiment of the present invention, for the convenience of explanation, it is assumed that the image captured by the PTZ camera (110) is analyzed by the crowd management device (120) of FIG. 1.

[0038] The PTZ camera (110) can be installed and operated in the same location as the fixed camera (100) as shown in FIG. 2, and in this state, it can perform an operation to circulate and film blind spots that are not covered by the multiple fixed cameras (100). Of course, there may be multiple blind spots. The circular filming operation of the PTZ camera (110) can be set up for circular filming as described above when the camera is installed, and the circular filming operation of the PTZ camera (110) can be set and operated in various ways depending on the characteristics of the location, such as a crowded subway station or a performance hall. In particular, if the PTZ camera (110) is equipped with an artificial intelligence program, it can analyze the captured images using the program to determine the characteristics of a specific location, and it can perform the circular filming operation by periodically changing the circular method based on the determination result. For example, if circular filming is initially set in the order of abcd in the blind spots, it is also possible to perform circular filming in the order of adbc as a result of analyzing the characteristics of the location. This could be automatically changed depending on where crowds are concentrated on a particular date or time.

[0039] In summary, an embodiment of the present invention provides an efficient crowd management system configured by combining a fixed camera (100) and a PTZ camera (110). The primary purpose of this system is to monitor and count crowds in a wide area using fixed and PTZ cameras installed in the same location. To this end, camera installation and calibration operations can be performed. In the system, the fixed camera (100) and the PTZ camera (110) are installed in the same location. Both cameras undergo precise calibration to ensure that the images captured by each camera are processed as accurately as possible. The calibration (method) uses a corresponding point mapping method. This method selects, for example, four or more points from the image coordinates of the fixed camera (100) and maps them to the pan and tilt coordinates of the PTZ camera (110). This allows the image of the fixed camera (100) to be projected onto the spherical coordinate system of the PTZ camera (110). In an embodiment of the present invention, a precise calibration operation of the PTZ camera (110) is performed. The PTZ camera (110) has a focal length pre-measured based on the image coordinate system, and the zoom ratio is also precisely measured. This allows the PTZ camera (110) to accurately capture detailed images from a long distance. Image capture and processing are performed. The fixed camera (100) continuously monitors a specific area, and the PTZ camera (110) captures an area outside the surveillance range of the fixed camera. The PTZ camera (110) zooms in on this area to capture detailed images, and the captured images are separately subjected to crowd counting.

[0040] The crowd management device (120) is directly or indirectly connected to the fixed camera (100) and the PTZ camera (110), analyzes the captured images provided by the fixed camera (100) and the PTZ camera (110), and performs operations for crowd management based on the results of the image analysis. The crowd management device (120) is separate from the analysis of the captured images, that is, performs the image analysis on the side of the fixed camera (100) and the PTZ camera (110), and can also manage the crowd in a specific area by only processing the number of crowds, i.e., the count. Of course, in this process, the crowd management device (120) can perform an operation to integrate and process image data for the captured image of the fixed camera (100) and the captured image of the PTZ camera (110), and can also set a plurality of coordinates for the captured image captured by the fixed camera (100) and map the set plurality of coordinates with the captured image of the PTZ camera (110) so that the image of the fixed camera (100) is projected onto the spherical coordinate system of the PTZ camera (110).

[0041] For example, the crowd management device (120) can receive images in real time from a fixed camera (100) and a PTZ camera (110). There may be multiple fixed cameras (100) and PTZ cameras (110). As a video analysis device, the crowd management device (120) is a device that can analyze multiple channels and may be in the form of a server, edge equipment, or camera. Video analysis can analyze each input image in real time and can be divided into a method for detecting individual objects and a crowd counting method for counting a large number of people. The PTZ camera (110) or the image provided from the PTZ camera (110) can perform analysis by utilizing a preset touring function. The PTZ preset can use a function built into the PTZ, or the PTZ preset setting function and the touring function can be implemented in the crowd management device (120) that operates as a video analysis device. It can be viewed as having a program for performing the corresponding function.

[0042] Video analysis based on the touring of a PTZ camera (110) can be performed in two ways: either at maximum speed or slowly when moving to a preset. When moving at high speed, motion blur makes video analysis difficult. In this case, a mode is activated that temporarily suspends video analysis when the camera is moving at high speed. This is clearly illustrated in Fig. 5. Here, the mode can be understood as a method for performing a designated action when a specific condition is met. This method prevents objects from being detected and then disappearing when the PTZ camera moves at high speed, as the video becomes blurred, reducing the accuracy of object detection. When the PTZ camera (110) moves slowly, analysis can continue because a certain amount of analysis is possible.

[0043] PTZ presets are presets for crowd counting, and a single pass through all presets is required to collect the crowd count for the target area. Once the PTZ camera (110) passes through all presets, it obtains a crowd count for the entire region of interest. From then on, the PTZ camera continuously repeats the preset pass, updating the crowd count for each region of interest. Once images for each preset have been acquired and crowd counted, the overlap with surrounding images must be checked and the crowd count results must be merged. If the two presets are filmed from the same distance, they can be merged under equal conditions. However, if they are filmed from different distances, i.e., with different zoom ratios, the count from the more zoomed-in image is prioritized. For this crowd counting to proceed smoothly, it is crucial that they be integrated into a single coordinate system.

[0044] To do this, each camera can operate in a way that is integrated into the PTZ camera coordinate system. First, the PTZ camera (110) itself must be calibrated. First, in the case of the PTZ camera (110), the API (application programming interface) provided by default rarely provides information for image analysis. The most important calibration information here is the focal length (f). The focal length of the PTZ camera (110) is measured at 1x. The field of view (FOV) provided by the manufacturer (state) can be used for the focal length, but there may be a difference between the specified field of view and the actual field of view, so measurement is always necessary. In the case of the PTZ camera (110), measurement can be easily performed through a function that moves a specific point of the image clicked by the user to the center of the camera. A precise focal length can be obtained by testing the movement to the target position based on the initial focal length value. The horizontal distance to the clicked point can be expressed as in <Relationship 1>.

[0045] [Relationship 1]

[0046]

[0047] Here, is the horizontal coordinate of the clicked point.

[0048] Pan angle calculation can be expressed as in <Relationship 2>.

[0049] [Relationship 2]

[0050]

[0051] Additionally, the tilt angle calculation can be expressed as in <Relationship 3>.

[0052] [Relationship 3]

[0053]

[0054] Of course, the crowd management device (120) according to the embodiment of the present invention can manage a mapping table for the measured focal length and the HW zoom value for each zoom ratio in XML format, as shown in FIG. 3 (S200 to S220). Various methods can be used for the mapping method between the fixed camera (100) and the PTZ camera (110). This method can be mapped using a method already presented. For example, it is also possible to use the method presented in Korean Patent Publication No. 10-0888935 (20009.03.10). This is well shown in FIG. 4 (S300 to S370). The pan / tilt coordinates of the PTZ camera (110) corresponding to the feature point designated in the fixed camera (100) must be obtained. To this end, a method is used in which the image center of the PTZ camera (110) becomes the center of the corresponding point. However, since the PTZ camera (110) has been calibrated in advance, it is also possible to designate the corresponding points of the PTZ camera (110). After completing this process, a homography matrix representing the relationship between the fixed camera (100) and the PTZ camera (110) can be obtained. By obtaining this homography matrix, the overlapping area between the image of the fixed camera (100) and the image of the PTZ camera (110) can be obtained.

[0055] The crowd management system (90) according to an embodiment of the present invention implements a process for performing crowd counting using a fixed camera (100) and a PTZ camera (110). In the initial stage, the fixed camera (100) receives continuous video input and performs crowd counting. The captured video is used to count the number of people. Simultaneously, the PTZ camera (110) performs a separate touring process. This camera tours several preset locations, receives video input from each location, and performs crowd counting. In this manner, the PTZ camera (110) can assess the size of the crowd from various angles and locations. Subsequently, the crowd counting results collected from the two camera systems undergo an integration process. This process first collects data from each camera and determines whether the two images overlap. If the two images overlap, the results are merged and additional counting is performed. Conversely, if the two images do not overlap, each crowd counting result is added independently. In the final stage, the collected and merged crowd counting results are totaled, and the final results are transmitted and stored. This process is important to ensure that the PTZ camera (110) continuously traverses the presets to update the crowd count, providing an accurate crowd count for the entire area of ​​interest.

[0056] In summary, the crowd management device (120) performs data integration and processing operations. Images captured by each camera are integrated and processed. In particular, when the capture areas overlap, the results are averaged and managed. This maintains data consistency between the two cameras and ensures accurate crowd counting. Furthermore, the crowd management device (120) can perform crowd density measurement operations in a spherical coordinate system. The core of the system (90) according to an embodiment of the present invention is to measure crowd density expressed in a spherical coordinate system. This function, which enables crowd counting in all areas scanned by the PTZ camera (110), is very effective for crowd management in a wide area. The crowd management device (120) can perform real-time and periodic measurements. The fixed camera (100) monitors a designated area in real time, while the PTZ camera (110) periodically scans a wider area or films in a touring manner. This is a necessary procedure for the PTZ camera (110) to effectively monitor a wide area and accurately measure the number of people. The PTZ camera (110) sequentially zooms in and scans various areas according to a specific pattern or schedule, thereby counting the number of people in each area. This method enables the PTZ camera (110) to comprehensively monitor a wide area and accurately track the dynamic movements of the crowd. In addition, the crowd management device (120) performs an accurate counting operation through zoom ratio control. Through appropriate zoom ratio control, the PTZ camera (110) can more accurately count the number of people even from a long distance. This is particularly useful in large public spaces or events. The system according to an embodiment of the present invention can be seen as suggesting potential applications in various fields such as public safety and traffic management, and large-scale event surveillance.

[0057] Embodiments of the present invention can address important challenges in crowd management and monitoring in urban areas and intersections. While CCTV systems are widely used as a popular surveillance tool, these systems still have several significant limitations. First, fixed-location CCTV cameras, even when installed in large numbers, inevitably create blind spots. Public spaces such as intersections typically feature PTZ (pan-tilt-zoom) cameras alongside fixed cameras. These PTZ cameras can be remotely controlled to rotate in various directions and zoom in to focus on a specific area. Therefore, embodiments of the present invention can utilize existing camera resources to reduce installation costs and other factors. For example, by utilizing the flexible operation capabilities of the PTZ camera (110), a method can be proposed for more accurately measuring the number of people in a wide space. For example, it is possible to simply download and install a program according to an embodiment of the present invention onto a PTZ camera (110) in firmware format. This method utilizes a PTZ camera (110) to dynamically scan a wide area, including blind spots of a fixed camera (100), and enables detailed monitoring by focusing on densely populated areas. This overcomes the limitations of the fixed camera (100) and enables efficient crowd monitoring and management without the need for additional camera installation.

[0058] Figure 5 is a flowchart for explaining a crowd counting and analysis process using a PTZ camera, and Figure 6 is a flowchart for a crowd counting and data integration process using fixed and PTZ cameras.

[0059] For convenience of explanation, referring to FIGS. 5 and 6 together with FIG. 1, the crowd management device (120) or PTZ camera (110) of FIG. 1 according to an embodiment of the present invention can perform an operation for preset setting (S400). The preset setting can be viewed as a preliminary operation performed by the PTZ camera (110) to enable surveillance in order to monitor (multiple) blind spots that the fixed camera (100) cannot monitor from any location.

[0060] In addition, the crowd management device (120) can start the preset touring operation of the PTZ camera (110) and check the movement speed during the process (S410, S420). Since this operation can be performed by the PTZ camera (110) itself and then the movement data related to the operation can be provided to the crowd management device (120), the embodiment of the present invention will not be particularly limited to any one form. For example, when an incident or accident occurs in the surveillance area of ​​the fixed camera (100) or in another area currently being filmed by the PTZ camera (110), high-speed movement may occur to film the area where the incident or accident occurred.

[0061] Accordingly, the crowd management device (120) can temporarily suspend image analysis to prevent motion blur during high-speed movement, and can continue image analysis operation when the speed is lower than the reference speed (S421, S423, S425). In addition, the analysis operation can be resumed after the preset movement is completed (S430).

[0062] Next, the crowd management device (120) can perform an operation to measure the crowd count for each preset, for example, since presetting for an arbitrary location has been completed (S440), and in the process, the crowd count results can be merged (S450). In addition, the crowd count result by merging can be transmitted to a DB (or memory, storage medium, etc.) linked to the crowd management device (120) and stored and managed (S460).

[0063] Figure 6 is a flowchart of a crowd counting and data integration process using fixed and PTZ cameras. It can be seen as a diagram detailing steps S440 to S460 in Figure 5.

[0064] As can be seen in Fig. 6, the crowd management device (120) performs an operation for measuring the crowd count for each fixed camera (100), and also performs an operation for measuring the crowd count for each preset area according to the touring of the PTZ camera (110) to collect the results for each camera (S500, S501, S510, S511, S512). For example, it is possible to analyze the captured images from each camera, and receive and utilize metadata related to the crowd count, i.e., the count value, and the location or properties of the captured images. For example, the metadata may include data or information that can determine whether each image overlaps.

[0065] Next, the crowd management device (120) can collect results from each camera and determine whether each image overlaps (S430). In practice, it can be difficult to accurately separate the shooting areas of the fixed camera (100) and the PTZ camera (110) in a shooting environment so that shooting can be performed. Therefore, it is possible to determine whether there is overlap during shooting, i.e., whether there is overlap during shooting.

[0066] Accordingly, the crowd management device (120) can add a count after merging the results when an overlap occurs, and can add a crowd counting result when an overlap does not occur (S540, S545). When two images overlap, the results are merged and an additional counting is performed. On the other hand, when there is no overlap, each crowd counting result is added independently.

[0067] And the crowd management device (120) can perform total count calculation, transmission, and storage operations (S550).

[0068] In the embodiment of the present invention, for the convenience of explanation, the operation is described assuming a crowd management device (120) with reference to FIG. 5. However, since the operation or some of the operations can be performed directly by the PTZ camera (110), the embodiment of the present invention will not be particularly limited to the above.

[0069] In addition to the above, the fixed camera (100), PTZ camera (110), and crowd management device (120) of FIG. 1 can perform various operations, and other detailed information has been sufficiently explained above, so those contents will be used instead. In addition, since the contents related to the fixed camera (100), PTZ camera (110), and crowd management device (120) will be continuously discussed later, detailed information will be replaced with those contents.

[0070] Figure 7 is a block diagram illustrating a detailed structure of the crowd management device of Figure 1.

[0071] As illustrated in FIG. 7, the crowd management device (120) of FIG. 1 according to an embodiment of the present invention includes part or all of a communication interface unit (600), a control unit (610), a crowd management unit (620), and a storage unit (630).

[0072] Here, “including some or all” means that some components, such as the storage unit (630), may be omitted to configure the crowd management device (120), or some components, such as the crowd management unit (620), may be integrated into other components, such as the control unit (610), etc. In order to help a sufficient understanding of the invention, it is described as including all.

[0073] The communication interface unit (600) communicates with the fixed camera (100) and PTZ camera (110) of Fig. 1, respectively. The communication interface unit (600) can perform operations such as modulation / demodulation, muxing / demuxing, encoding / decoding, and encryption / decryption during the communication process, which are obvious to those skilled in the art, and thus further description thereof will be omitted.

[0074] The communication interface unit (600) can operate in various ways depending on the intention of the system designer. For example, this is the case where the fixed camera (100) and the PTZ camera (110) are each equipped with a video analysis module for analyzing the captured images they have taken. Also, this is the case where only the PTZ camera (110) is equipped with a video analysis module, and this is the case where the fixed camera (100) and the PTZ camera (110) analyze all the captured images they have taken in the crowd management device (120) of FIG. 1. The communication interface unit (600) can operate in various ways depending on the setup method of such a system. For example, the communication interface unit (600) can receive the captured images taken by the fixed camera (100) and the PTZ camera (110) and transmit them to the control unit (610).

[0075] In addition, the communication interface unit (600) may operate to perform circular movement, i.e., touring operation, of the PTZ camera (110) under the control of the control unit (610). For example, if a program related to circular movement is set for the PTZ camera (110) for the initial operation setting of the system, the PTZ camera (110) may perform circular movement operation according to the corresponding programming method. In other words, if the PTZ camera (110) monitors a blind spot that cannot be monitored by a plurality of fixed cameras (100) for an arbitrary location, the operation may be performed while moving in a preset manner for multiple areas. For example, if the touring of the PTZ camera (110) is performed at a high speed higher than the reference speed, the communication interface unit (600) may not receive the captured image provided by the PTZ camera (110) for image analysis.

[0076] The control unit (610) can perform the overall control operation of the communication interface unit (600), the crowd management unit (620), and the storage unit (630) of FIG. 7. As described above, the control unit (610) can perform setting operations related to the operations of the fixed camera (100) and the PTZ camera (110). For example, as shown in FIG. 2, when the surveillance setting for the first range of an area from an arbitrary location is completed for the fixed camera (100), the second range of an area not covered by the fixed camera (100) is set for the PTZ camera (110) from the same arbitrary location. Of course, the second range of an area can be a plurality of areas, and thus, the PTZ camera (110) can be set to film while circulating through the plurality of areas. Of course, during this process, the control unit (610) can also communicate with the PTZ camera (110) to transmit a program related to circulating movement. The control unit (610) can be involved in these operations.

[0077] Of course, the control unit (610) can perform various operations in conjunction with the crowd management unit (620). When the captured images are provided from the fixed camera (100) and the PTZ camera (110), the control unit (610) can temporarily store them in the storage unit (630) and then retrieve them to provide them to the crowd management unit (620) for image analysis. In this process, the control unit (610) can execute a program installed in the crowd management unit (620), and according to the execution of the program, the control unit (610) can control the communication of the communication interface unit (600) to receive a coefficient value of a crowd coefficient related to a crowd in a random location, for example, and store and manage it in a database, etc.

[0078] The crowd management unit (620) can perform operations to minimize blind spots in crowd detection by counting and estimating crowds while having the PTZ scan the limited detection range of the CCTV at the center of an intersection where CCTV is installed, for example, by monitoring each exit route with a fixed camera (100) and installing a PTZ for long-distance camera detection, and then mapping all of this to a single global space. To this end, the crowd management unit (620) can perform operations such as processing of captured video data, data integration and processing, crowd density measurement in a spherical coordinate system, real-time and periodic measurement, and accurate counting measurement through zoom ratio control. Since this has been sufficiently explained above, we will replace it with those contents.

[0079] However, briefly, the crowd management unit (620) performs an integrated operation of the crowd counting results collected from the two camera systems. In this process, data collected from each camera is first collected and whether there is an overlap between the two images is determined. If the two images overlap, the results are merged and additional counting is performed. On the other hand, if there is no overlap, each crowd counting result is added independently. In the final step, the crowd management unit (620) totals the collected and merged crowd counting results and finally transmits and stores them in a database, etc. This process can provide an accurate crowd count for the entire area of ​​interest by continuously updating the crowd count while the PTZ camera (110) tours the presets. In this process, for example, the crowd management unit (620) can check the movement speed of the PTZ camera (110) when the preset touring starts for the preset tour operation and temporarily suspend the video analysis operation when the PTZ camera is moving at a high speed.

[0080] The storage unit (630) can temporarily store various types of information or data processed under the control of the control unit (610) of FIG. 7. Here, the terms information and data may be used interchangeably in practice in the industry, but for example, identification information of a fixed camera (100) or a PTZ camera (110) may be referred to as information. On the other hand, captured images captured by cameras may be referred to as image data. When image data of captured images are transmitted from each camera, the storage unit (630) matches the data with identification information and temporarily stores it in the storage unit (630), and then retrieves the image data of a specific camera and provides it to the crowd management unit (620) for image analysis.

[0081] In addition to the above, the communication interface unit (600), control unit (610), crowd management unit (620), and storage unit (630) of FIG. 7 can perform various operations, and other detailed information has been sufficiently explained above, so it will be replaced with those contents.

[0082] According to an embodiment of the present invention, the communication interface unit (600), the control unit (610), the crowd management unit (620), and the storage unit (630) of FIG. 7 are configured as physically separate hardware modules, but each module may store and execute software for performing the above operations. However, the software is a collection of software modules, and each module may be formed of hardware, so there is no particular limitation on the configuration, such as software or hardware. For example, the storage unit (630) may be hardware, such as storage or memory. However, since it is also possible to store information (repository) in software, there is no particular limitation on the above.

[0083] Meanwhile, as another embodiment of the present invention, the control unit (610) may include a CPU and a memory, and may be formed as a single chip. The CPU may include a control circuit, an operation unit (ALU), a command interpretation unit, and a registry, and the memory may include a RAM. The control circuit may perform a control operation, the operation unit may perform an operation of binary bit information, and the command interpretation unit may perform an operation of converting a high-level language into machine language and vice versa, including an interpreter or a compiler, and the registry may be involved in software data storage. According to the above configuration, for example, at the initial operation of the crowd management device (120) of FIG. 1, a program stored in the crowd management unit (620) may be copied and loaded into memory, i.e., RAM, and then executed, thereby rapidly increasing the data operation processing speed. In the case of a deep learning model, it may be loaded into the GPU memory instead of the RAM and executed by accelerating the execution speed using the GPU.

[0084] Figure 8 is a flowchart showing the operation process of the crowd management device of Figure 1.

[0085] For convenience of explanation, referring to FIG. 8 together with FIG. 1, a crowd management device (120) according to an embodiment of the present invention can communicate with a plurality of fixed cameras (100) that control and film a first range of an area in an arbitrary location, and a PTZ camera (110) that sequentially moves in a preset manner, i.e., films a second range of an area excluding the first range of an area in an arbitrary location, i.e., while circulating or touring (S700). Here, the first range of an area may refer to a surveillance area that can be covered by a plurality of fixed cameras (100) in the range of an arbitrary location. On the other hand, the second range of an area may refer to an area excluding the first range of an area in the range of an arbitrary location. It may refer to blind spots that are not monitored by a plurality of CCTVs, etc., other than the first range of an area in an arbitrary location. Of course, the second range of an area may be divided into a plurality of areas. For example, if the second range has four blind spots that cannot be monitored by multiple fixed cameras (100), and these are referred to as areas a to d, the PTZ camera (110) can perform a filming operation by circulating through the blind spots a to d areas through pan, tilt, and zoom operations. Settings related to such circular or circular filming operations can be made in advance.

[0086] In addition, the crowd management device (120) can manage crowds at any location based on the results of the integrated analysis of the video data for the captured images of the first range and the second range (S710). The integrated analysis may refer only to an operation of determining whether each image overlaps, or may include such an operation. Of course, for data analysis efficiency, the crowd management device (120) may not merge the images from each camera, but may individually analyze the captured images for each area to count the number of crowds, and then perform an integration process. During this process, it may be performed to determine whether the captured areas overlap and increase or decrease the count value, i.e., the number of crowds, accordingly. Depending on the result of the overlap determination, whether to add or subtract the count may be determined by calculating the average. For example, if two people are counted in an overlapping area, it may be entirely possible to operate by subtracting 2 from the total count result.

[0087] In addition to the above, the crowd management device (120) of FIG. 1 can perform various operations, and other detailed information has been sufficiently explained above, so it will be replaced with that information.

[0088] Even though all components constituting the embodiments of the present invention have been described as being combined or operating in combination, the present invention is not necessarily limited to such embodiments. That is, within the scope of the present invention, all of the components may be selectively combined and operated one or more times. In addition, although all of the components may be implemented as individual hardware, some or all of the components may be selectively combined and implemented as a computer program having program modules that perform some or all of the functions of the combined hardware in one or more pieces. The codes and code segments constituting the computer program will be readily inferred by those skilled in the art. Such a computer program may be stored in a non-transitory computer-readable storage medium and read and executed by a computer, thereby implementing the embodiments of the present invention.

[0089] Here, the non-transitory readable storage medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the above-described programs may be stored and provided on a non-transitory readable storage medium, such as a CD, DVD, hard disk, Blu-ray disc, USB, memory card, or ROM.

[0090] Although embodiments of the present invention have been described with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

[0091] * Explanation of symbols

[0092] 100: Fixed camera 110: PTZ camera

[0093] 120: Crowd management device 600: Communication interface unit

[0094] 610: Control Unit 620: Crowd Management Unit

[0095] 630: Storage

Claims

1. A communication interface unit that communicates with a plurality of fixed cameras that each control and film a first range of an area in an arbitrary location and a PTZ (Pan Tilt Zoom) camera that circulates and films a second range of an area excluding the first range of an area in the arbitrary location in a preset manner; and A control unit for determining whether each image of the captured images of the first range of the area and the second range of the area overlaps and managing the crowd at the arbitrary location based on the determination result; A crowd management device utilizing fixed cameras and PTZ cameras.

2. In paragraph 1, A crowd management device utilizing a fixed camera and a PTZ camera, wherein the control unit selects a plurality of image coordinates of the fixed camera, maps the selected plurality of image coordinates to pan and tilt coordinates of the PTZ camera, and then converts the image of the fixed camera into the spherical coordinate system of the PTZ camera to determine whether there is an overlap.

3. In paragraph 2, The above control unit is a crowd management device utilizing a fixed camera and a PTZ camera, which manages the average of each analysis result of image data for the overlapping shooting areas when the shooting areas of the fixed camera and the PTZ camera overlap.

4. In paragraph 1, A crowd management device utilizing a fixed camera and a PTZ camera, wherein the control unit operates the PTZ camera at one of a low speed lower than a reference speed and a high speed higher than the reference speed when moving the PTZ camera sequentially, and temporarily stops the analysis operation of the captured image when moving at the high speed.

5. In paragraph 4, A crowd management device utilizing a fixed camera and a PTZ camera, wherein the control unit moves the PTZ camera at high speed to a location where an incident or accident event occurs in the first range of areas and the second range of areas, and then resumes analysis of the captured image.

6. A step in which the communication interface unit communicates with a plurality of fixed cameras that each control and film a first range of an area in an arbitrary location and a PTZ camera that circulates and films a second range of an area excluding the first range of an area in the arbitrary location according to a preset method; and A step for the control unit to determine whether each image of the captured images of the first range of the area and the second range of the area overlaps and manage the crowd at the arbitrary location based on the determination result; A method for operating a crowd management device using a fixed camera and a PTZ camera.

7. In paragraph 6, The step of determining whether or not there is an overlap is: A method for operating a crowd management device utilizing a fixed camera and a PTZ camera, wherein a plurality of image coordinates of the fixed camera are selected, the selected plurality of image coordinates are mapped to pan and tilt coordinates of the PTZ camera, and the image of the fixed camera is converted into a spherical coordinate system of the PTZ camera to determine whether there is an overlap.

8. In paragraph 7, The steps to manage the above crowd are: A method for operating a crowd management device utilizing a fixed camera and a PTZ camera, wherein when the shooting areas of the fixed camera and the PTZ camera overlap, the analysis results of each image data for the overlapping shooting areas are averaged and managed.

9. In paragraph 6, The steps to manage the above crowd are: A method for operating a crowd management device utilizing a fixed camera and a PTZ camera, comprising: a step of sequentially moving the PTZ camera at one of a low speed lower than a reference speed and a high speed higher than the reference speed, and temporarily stopping the analysis operation of the captured image when moving at the high speed; 10. In paragraph 9, The steps to manage the above crowd are: A method for operating a crowd management device utilizing a fixed camera and a PTZ camera, further comprising: a step of moving the PTZ camera at high speed to a location where an incident or accident event occurs in the first range of areas and the second range of areas, and then resuming an analysis operation of the captured image; 11. Multiple fixed cameras each covering an area of ​​the first range in an arbitrary location and shooting; A PTZ camera that circulates and photographs an area of ​​a second range excluding an area of ​​a first range in a preset manner at the above random location; and A crowd management device that determines whether each image overlaps with respect to the captured images of the first range of areas and the second range of areas received by communicating with the plurality of fixed cameras and the PTZ camera, and manages the crowd at the arbitrary location based on the determination result; A crowd management system utilizing fixed cameras and PTZ cameras.

Images