Apparatus and method for synchronizing a surveillance camera and a virtual camera on a 3D digital twin system
The system synchronizes actual surveillance cameras with virtual cameras on 3D digital twins using deep learning networks, addressing inefficiencies by aligning views in real-time and enabling an unmanned surveillance system.
Patent Information
- Application Number
- KR1020210106232
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-08-11
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2041-08-11
AI Technical Summary
Existing surveillance systems face challenges in synchronizing the view of actual surveillance cameras with virtual cameras on 3D digital twins, particularly for long-range and PTZ cameras, leading to inefficiencies due to the need for human intervention and time delays in events like fires or thefts.
A system that synchronizes actual surveillance cameras with virtual cameras on a 3D digital twin using a domain conversion unit, feature information extraction, position estimation, and scene selection units, leveraging deep learning networks to align the views in real-time without human intervention.
Enables real-time synchronization of actual and virtual cameras, allowing seamless projection of digital twin information onto surveillance camera images and vice versa, transitioning to an unmanned surveillance system that utilizes vast digital twin data efficiently.
Smart Images

Figure 112021092824798-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a technology for displaying in real-time in a digital twin space a scene identical to the scene being monitored by an actual CCTV in a digital twin-based surveillance system. Background Technology
[0002] In order to improve the efficiency of deep learning-based surveillance systems using cameras, a service is being attempted to provide various information by constructing a 3D digital twin of the camera's surveillance area.
[0003] For example, when a fire is detected by a deep learning-based monitoring system, the digital twin can transmit not only the exact location information of the area but also detailed related information such as the location of shelters, the location of firefighting tools, and surrounding terrain features, thereby increasing the efficiency of fire suppression.
[0004] In the case of short-range surveillance cameras, the surveillance radius is small, so relevant information can be easily obtained using only the camera's location information; however, in the case of long-range surveillance cameras installed at high positions to monitor a wide area, it is difficult to identify the location where an event occurred.
[0005] In addition, in the case of standard PTZ (pan, tilt, zoom) surveillance cameras capable of moving up, down, left, right, rotating, and zooming in / out, although camera operation is possible, they do not provide information such as absolute position and orientation, so they are not synchronized with the camera in the 3D digital twin system, making user intervention unavoidable to find the direction the camera is currently facing.
[0006] However, the time delay caused by such user intervention significantly reduces efficiency due to the nature of events such as fires, thefts, and accidents processed by the camera surveillance system.
[0007] Therefore, a system is needed that synchronizes the actual surveillance camera with the virtual camera on the 3D digital twin in real time without the need for human intervention, displaying the same location on the digital twin as the location viewed by the actual camera. The problem to be solved
[0008] The present invention aims to solve conventional problems by providing a surveillance camera and a virtual camera synchronization device on a 3D digital twin system that provides camera synchronization technology for ensuring that the scene viewed by the actual surveillance camera and the scene viewed by the virtual camera on the 3D digital twin view the same location.
[0009] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood by those skilled in the art from the description below. means of solving the problem
[0010] A virtual camera synchronization device on a surveillance camera and a 3D digital twin system according to an embodiment of the present invention for achieving the above objective comprises: a domain conversion unit that performs a domain conversion to convert a video stream captured by a surveillance camera into data converted into a 3D digital twin form; a 3D digital twin DB that stores 3D digital twin data and setting information of a virtual camera that captured the 3D digital twin data; a feature information extraction unit that extracts global feature information from the converted data converted into a 3D digital twin form and the 3D digital twin data, respectively; a position estimation unit that estimates the primary position of a scene image of the 3D digital twin data by comparing the global feature information extracted from the 3D digital twin data and the converted data converted into a 3D digital twin form, respectively; and a scene selection unit that determines the matching rate of local feature information extracted from the 3D digital twin data with the estimated primary position and the data converted into a 3D digital twin form, respectively, and selects the scene image on the 3D digital twin data with the highest matching rate. and includes a camera adjustment unit that adjusts the settings of the actual surveillance camera to correspond to the setting information of the virtual camera corresponding to the selected scene image.
[0011] The above domain transformation unit can extract feature information from data transformed into a 3D digital twin form using a deep learning network.
[0012] The above feature information includes global feature information and local feature information.
[0013] The domain transformation unit and the feature information extraction unit may use a deep learning network trained to input a real video stream and output data converted into a 3D digital twin, or input data converted into a 3D digital twin and output an image of the real video stream.
[0014] The above feature information extraction unit preferably uses a convolutional neural network to extract global feature information, local feature information, and feature map information from the actual image stream.
[0015] The present invention further includes a deep learning network learning unit that extracts feature maps, local feature information, and global feature information from a video stream captured through a real camera and a scene image on 3D digital twin data that matches the same, compares them with each other, and learns in a direction that minimizes the difference.
[0016] The above 3D digital twin DB includes scene images captured using a virtual camera in the virtual space of the 3D digital twin, and information regarding the position, attitude, and zoom of the camera when capturing with the virtual camera, and the scene images include local and global feature information extracted through a deep learning-based feature information extraction network that has been trained.
[0017] The scene selection unit can determine the final matching target by comparing the regional feature information of the scene image candidate group of the 3D digital twin data selected by the location estimation unit with the regional feature information extracted from the data converted into a 3D digital twin form.
[0019] A method for synchronizing a virtual camera on a surveillance camera and a 3D digital twin system according to an embodiment of the present invention comprises: a step of performing a domain conversion by a domain conversion unit to convert a video stream captured by a surveillance camera into data converted into a 3D digital twin form; a step of extracting global feature information from the domain-converted data converted into a 3D digital twin form and from the 3D digital twin data stored in the 3D digital twin DB, respectively, by a feature information extraction unit; a step of estimating the primary position of a scene image of the 3D digital twin data by comparing the global feature information extracted from the 3D digital twin data and the global feature information extracted from the data converted into a 3D digital twin form, respectively, by a position estimation unit; a step of determining the matching rate of the local feature information extracted from the 3D digital twin data in which the primary position is estimated and the data converted into a 3D digital twin form, respectively, and selecting the scene image on the 3D digital twin data with the highest matching rate by a scene selection unit; and a step of controlling the settings of an actual surveillance camera according to the setting information of a virtual camera corresponding to the selected scene by a virtual camera adjustment unit.
[0020] The step of converting the above domain can extract feature information from data converted into a 3D digital twin form using a deep learning network.
[0021] The above feature information includes global feature information and local feature information.
[0022] The step of extracting the above feature information may utilize a deep learning-based network trained to input a real video stream and output data converted into a 3D digital twin, or to input data converted into a 3D digital twin and output an image of the real video stream.
[0023] The step of extracting the above feature information preferably utilizes a convolutional neural network to extract global feature information, local feature information, and feature map information from the actual image stream.
[0024] The step of extracting the above feature information involves extracting feature maps, local feature information, and global feature information for a video stream captured through an actual camera and a scene image on the 3D digital twin data that matches it, comparing them with each other, and learning in a direction that minimizes the difference.
[0025] The above 3D digital twin DB includes scene images captured using a virtual camera in the virtual space of the digital twin, and information regarding the position, attitude, and zoom of the camera when capturing with the virtual camera, and the scene images include local and global feature information extracted through a deep learning-based feature information extraction network that has been trained.
[0026] The step of selecting the above scene can determine the final matching target by comparing the regional feature information of the scene image candidate group of the 3D digital twin data selected by the location estimation unit with the regional feature information extracted from the data converted into a 3D digital twin form. Effects of the invention
[0027] According to one embodiment of the present invention, a service is provided by linking a digital twin added to a surveillance system using CCTV. In order to utilize the vast amount of data of the digital twin in real time and to perform a transition to an unmanned surveillance system without user intervention, the actual surveillance camera and the virtual camera in the 3D space of the digital twin can be synchronized, and when the surveillance camera is moved, the virtual camera can also move in real time, thereby accurately aligning the 2D surveillance camera image and the 3D digital twin space so that information of the digital twin can be projected onto the surveillance camera image, and conversely, the surveillance camera image can be projected onto the digital twin to be displayed. Brief explanation of the drawing
[0028] FIG. 1 is a block diagram illustrating a virtual camera synchronization device on a 3D digital twin system and another surveillance camera in one embodiment of the present invention. FIG. 2 is a reference diagram for illustrating an example of a video stream captured by a surveillance camera in one embodiment of the present invention. FIG. 3 is a reference diagram for explaining an example of domain conversion in which a video stream captured by a surveillance camera in one embodiment of the present invention is converted into data converted into a 3D digital twin form. FIG. 4 is a reference diagram illustrating a domain conversion process for explaining the process of converting an image stream into data converted into a 3D digital twin form in one embodiment of the present invention. FIG. 5 is a reference diagram for explaining an example of a feature information detection process using CNN in a feature information extraction unit in one embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for synchronizing a surveillance camera and a virtual camera on a 3D digital twin system according to an embodiment of the present invention. Specific details for implementing the invention
[0029] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" as used in this specification do not exclude the presence or addition of one or more other components, steps, actions, and / or elements in addition to the mentioned components, steps, actions, and / or elements.
[0030] FIG. 1 is a block diagram illustrating a virtual camera synchronization device on a surveillance camera and a 3D digital twin system according to the present invention.
[0031] As illustrated in FIG. 1, a virtual camera synchronization device on a surveillance camera and a 3D digital twin system according to one embodiment of the present invention includes a domain conversion unit (100), a 3D digital twin DB (200), a feature information extraction unit (300), a position estimation unit (400), a scene selection unit (500), and a virtual camera adjustment unit (600).
[0032] The domain transformation unit (100) performs data domain transformation into the video captured by the surveillance camera shown in FIG. 2 and the 3D digital twin data shown in FIG. 3. At this time, the domain transformation unit (100) can perform domain transformation into data (Y') converted into a 3D digital twin form from a video stream (X) captured in some way through a gain (Gx) network as shown in FIG. 4. It is preferable that the gain (Gx) network be a network based on a Generative Adversarial Network (GAN).
[0033] The 3D digital twin DB (200) stores 3D digital twin data (Y) obtained through a virtual camera to estimate geometric information of the camera through the analysis of a video stream (X). The 3D digital twin DB (200) includes scene images captured using a virtual camera in the virtual space of the digital twin, and information regarding the camera's position, attitude, and zoom when capturing with the virtual camera. It is preferable that the scene images include local and global feature information extracted through a deep learning-based feature information extraction network that has completed training. At this time, global feature information can be extracted using a technique such as PCA (Principal Component Analysis), and local features can be extracted using a technique such as HoG (Histogram of Gradients). At this time, it is preferable that the feature information includes both global feature information and local feature information. Here, global feature information is extracted by increasing the number of channels in the network and reducing the horizontal and vertical resolution, and includes overall feature information of the scene.
[0034] Here, scene images include local feature information and global feature information extracted through a deep learning-based feature information extraction network that has finished training.
[0035] The feature information extraction unit (300) extracts global feature information from the actual video stream (X) captured by the surveillance camera and global feature information extracted from the data (Y') converted into a 3D digital twin form stored in the 3D digital twin DB (200). However, the feature information extraction unit (300) performs domain transformation using a deep learning network in the case of the actual captured video stream (X) and then extracts feature information from the data (Y') converted into a 3D digital twin form.
[0036] It is preferable for the domain conversion unit (100) and the feature information extraction unit (300) to use a deep learning network trained to input a real video stream (X) and output data (Y') converted into a 3D digital twin, or input data (Y') converted into a 3D digital twin and output an image of the real video stream (X).
[0037] Additionally, the feature information extraction unit (300) extracts feature information captured through a surveillance camera and feature information of an image stored in a 3D digital twin DB (200) based on a deep learning network.
[0038] In the general field of image processing, feature point extraction involves comparing images captured by the same camera using a predefined filter and extracting points with clear structural features. However, as shown in FIG. 2, this is difficult to apply to spatial data with different domains. Therefore, in the present invention, feature information is extracted from the captured image and 3D digital twin data (Y) based on a deep learning network through a feature information extraction unit (300).
[0039] Deep learning-based feature extraction can effectively represent results that reflect the characteristics of the data because it uses feature points extracted through learning with data rather than filters predefined by the user.
[0040] FIG. 5 is a reference diagram for explaining an example of a feature information detection process using CNN in a feature information extraction unit in one embodiment of the present invention.
[0041] The feature information extraction unit (300) preferably uses a convolutional neural network to extract feature information, such as global feature information, local feature information, and feature map information, from a captured image stream (X). At this time, as shown in FIG. 5, it seems necessary to provide a simple explanation for defining each filter for extracting global feature information, local feature information, and feature map information.
[0042] Here, deep learning network training extracts feature maps, local feature information, and global feature information from a video stream (X) captured by a real camera and scene images on the matching digital twin data (Y), compares them, and proceeds with training in a direction that minimizes the difference. However, in actual matching, global and local feature information are compared, and feature map information is used only during the training phase to improve training performance.
[0043] The location estimation unit (400) estimates a primary location by comparing global feature information extracted from 3D digital twin data (Y) with global feature information extracted from a surveillance camera video stream (X). That is, the location estimation unit (400) selects a candidate group of 3D digital twin data (Y) that matches by comparing global feature information extracted from the video stream (X) with global feature information extracted from 3D digital twin data (Y) of the 3D digital twin DB (200).
[0044] The scene selection unit (500) estimates a precise location using local feature information, selects a scene on the 3D digital twin data (Y) with the highest matching rate, and determines the target that is the final match. The scene selection unit (500) determines the matching rate of local feature information extracted from the 3D digital twin data (Y) with the estimated primary location and the data (Y') converted into the 3D digital twin form, respectively, and selects a scene image on the 3D digital twin data (Y) with the highest matching rate.
[0045] The virtual camera adjustment unit (600) checks the settings of the virtual camera of the actual 3D digital twin system using the position, attitude, and zoom information of the virtual camera corresponding to the scene of the selected 3D digital twin data (Y), and adjusts the virtual camera of the actual 3D digital twin system using the virtual camera setting information that has been checked.
[0046] Additionally, the virtual camera adjustment unit (600) adjusts the settings of the actual surveillance camera so as to correspond to the setting information of the virtual camera corresponding to the selected scene image.
[0047] According to one embodiment of the present invention, as a base technology for providing services by linking a digital twin added to a surveillance system using CCTV, the actual surveillance camera and the virtual camera in the 3D space of the digital twin are synchronized so that when the surveillance camera is moved, the virtual camera also moves in real time to accurately align the 2D surveillance camera image and the 3D digital twin space, thereby enabling the information of the digital twin to be projected onto the surveillance camera image, and conversely, the surveillance camera image to be projected onto the digital twin to be displayed.
[0048] In one embodiment of the present invention, there is an effect of being able to transition into an unmanned surveillance system without user intervention in order to utilize the vast data of the digital twin in real time.
[0050] As shown in Figure 2, when compared with actual CCTV footage, the digital twin data (Y) shows significant differences in structure and texture, so while approximate locations can be compared, it takes a long time to match them accurately. In particular, the matching accuracy is lowered because there are cases where the digital twin data (Y) cannot cover an area due to the surveillance area being very large. Additionally, while it is easy to match when there are specific indicators such as buildings in villages or cities, the matching rate is lower in mountainous areas.
[0051] Therefore, to narrow the difference between the actual image and the 3D digital twin data (Y), the domain of the data is changed and used.
[0052] In addition, for domain change, the present invention uses deep learning network training to extract actual feature information from data generated through bidirectional learning.
[0053] Using the deep learning network trained in this way, the domains of the input data image and the scene data image on the digital twin are transformed, and feature information is extracted using this.
[0055] A method for synchronizing a surveillance camera and a virtual camera on a 3D digital twin system according to one embodiment of the present invention will be described with reference to FIG. 6.
[0056] First, the domain conversion unit (100) performs domain conversion on the video stream (X) captured by the surveillance camera into data (Y') converted into a 3D digital twin form (S100). The step of performing domain conversion (S100) performs domain conversion of the captured video stream (X) using a deep learning network. By converting the video stream (X) into data (Y') converted into a 3D digital twin form in this way, feature information extraction and comparison become easy.
[0057] Next, the feature information extraction unit (300) extracts global feature information from the video stream (X) captured by the surveillance camera and from the 3D digital twin data (Y) stored in the 3D digital twin DB (200) (S200). The step of extracting the feature information (S200) is preferably trained using a deep learning-based network so that the actual video stream (X) is input and the output is data (Y') converted into a 3D digital twin form, or the data (Y') converted into a 3D digital twin form is input and the output is an image of the actual video stream (X).
[0058] Meanwhile, the 3D digital twin DB (200) includes scene images captured using a virtual camera in the virtual space of the digital twin, and information about the position, pose, and zoom of the virtual camera when capturing with the virtual camera, and the scene images include local and global feature information extracted through a deep learning-based feature information extraction network that has finished training.
[0059] In addition, the step (S200) of extracting the above feature information preferably uses a convolutional neural network that extracts global feature information, local feature information, and feature map information after extracting feature information from the image stream (X).
[0060] In one embodiment of the present invention, it is preferable to use a convolutional neural network (CNN) trained to minimize the difference between a scene image of a video stream (X) captured through a real camera and a 3D digital twin data (Y) that matches it, by extracting a feature map, local feature information, and global feature information through a deep learning network learning unit and comparing them.
[0061] Afterwards, the position estimation unit (400) estimates the primary position of the 3D digital twin data (Y) by comparing the global feature information extracted from the 3D digital twin data (Y) with the global feature information extracted from the video stream (X) captured by the surveillance camera (S300).
[0062] Next, the scene selection unit (500) estimates a precise location using local feature information and selects a scene on the 3D digital twin data (Y) with the highest matching rate (S400). The scene selection step (S400) determines the target that is finally matched by comparing the local feature information of the scene image candidates of the 3D digital twin data (Y) for which the primary location has been estimated.
[0063] By the virtual camera adjustment unit (600), the setting information of the virtual camera corresponding to the selected scene image is checked in the 3D digital twin DB (200) to adjust the settings of the actual surveillance camera. By the virtual camera adjustment unit (600), the virtual camera of the actual 3D digital twin system is adjusted using the position, attitude, and zoom information of the virtual camera corresponding to the selected scene.
[0064] According to one embodiment of the present invention, a service is provided by linking a digital twin added to a surveillance system using CCTV. In order to utilize the vast amount of data of the digital twin in real time and to perform a transition to an unmanned surveillance system without user intervention, the actual surveillance camera and the virtual camera in the 3D space of the digital twin can be synchronized, and when the surveillance camera is moved, the virtual camera can also move in real time, thereby accurately aligning the 2D surveillance camera image and the 3D digital twin space so that information of the digital twin can be projected onto the surveillance camera image, and conversely, the surveillance camera image can be projected onto the digital twin to be displayed.
[0065] Although the configuration of the present invention has been described in detail with reference to the accompanying drawings, this is merely illustrative, and it is obvious that those skilled in the art can make various modifications and changes within the scope of the technical concept of the present invention. Accordingly, the scope of protection of the present invention should not be limited to the aforementioned embodiments but should be determined by the description in the following claims.
Claims
Claim 1 A domain conversion unit that performs a domain conversion to convert a video stream captured by a surveillance camera into data converted into a 3D digital twin form; a 3D digital twin DB that stores 3D digital twin data (Y) and setting information of a virtual camera that captured the 3D digital twin data; a feature information extraction unit that extracts global feature information from the converted data in the 3D digital twin form and the 3D digital twin data, respectively; a position estimation unit that estimates the primary position of a scene image in the 3D digital twin data by comparing the global feature information extracted from the 3D digital twin data and the converted data in the 3D digital twin form, respectively; and a scene selection unit that determines the matching rate of the local feature information extracted from the 3D digital twin data with the estimated primary position and the data converted into the 3D digital twin form, respectively, and selects the scene image on the 3D digital twin data with the highest matching rate. A surveillance camera and a virtual camera synchronization device on a 3D digital twin system, comprising a camera adjustment unit that adjusts the settings of the actual surveillance camera so as to correspond to the setting information of the virtual camera corresponding to the selected scene image, and a deep learning network learning unit that extracts feature maps, local feature information, and global feature information for a video stream captured through the actual camera and a scene image on the 3D digital twin data that matches it, compares them with each other, and learns in a direction that minimizes the difference. Claim 2 A virtual camera synchronization device on a surveillance camera and a 3D digital twin system, wherein, in claim 1, the domain conversion unit extracts feature information from data converted into a 3D digital twin form using a deep learning network. Claim 3 In paragraph 2, the feature information comprises a virtual camera synchronization device on a surveillance camera and a 3D digital twin system, including global feature information and local feature information. Claim 4 A virtual camera synchronization device on a surveillance camera and a 3D digital twin system according to claim 1, wherein the domain conversion unit and the feature information extraction unit utilize a deep learning network trained to input an actual video stream and output data converted into a 3D digital twin form, or input 3D digital twin converted data and output an actual video stream image. Claim 5 A virtual camera synchronization device on a surveillance camera and a 3D digital twin system, wherein the feature information extraction unit in claim 1 utilizes a convolutional neural network to extract global feature information, local feature information, and feature map information from an actual video stream. Claim 6 delete Claim 7 A virtual camera synchronization device on a surveillance camera and a 3D digital twin system, characterized in that, in claim 1, the 3D digital twin DB includes scene images captured using a virtual camera in the virtual space of the 3D digital twin and information regarding the position, attitude, and zoom of the camera when capturing with the virtual camera, and the scene images include local and global feature information extracted through a deep learning-based feature information extraction network that has completed training. Claim 8 A virtual camera synchronization device on a surveillance camera and a 3D digital twin system, wherein, in claim 1, the scene selection unit determines a target that is finally matched by comparing the regional feature information of a group of scene image candidates of the 3D digital twin data selected by the position estimation unit with the regional feature information extracted from data converted into a 3D digital twin form. Claim 9 A step of performing a domain conversion by a domain conversion unit to convert a video stream captured by a surveillance camera into data converted into a 3D digital twin form; a step of extracting global feature information from the domain-converted data converted into a 3D digital twin form and from the 3D digital twin data stored in the 3D digital twin DB, respectively, by a feature information extraction unit; a step of estimating the primary position of a scene image in the 3D digital twin data by comparing the global feature information extracted from the 3D digital twin data with the global feature information extracted from the data converted into a 3D digital twin form, respectively, by a position estimation unit; and a step of determining the matching rate of the local feature information extracted from the 3D digital twin data with the estimated primary position and the data converted into a 3D digital twin form, respectively, and selecting the scene image on the 3D digital twin data with the highest matching rate, by a scene selection unit. A method for synchronizing a surveillance camera and a virtual camera on a 3D digital twin system, comprising the step of controlling the settings of an actual surveillance camera according to setting information of a virtual camera corresponding to a selected scene by a virtual camera adjustment unit, wherein the step of extracting feature information is characterized by extracting a feature map, local feature information, and global feature information for a video stream captured through an actual camera and a scene image on 3D digital twin data that matches it, comparing them with each other, and learning in a direction that minimizes the difference. Claim 10 A method for synchronizing a virtual camera on a surveillance camera and a 3D digital twin system, wherein the step of converting the domain is characterized by extracting feature information from data converted into a 3D digital twin form using a deep learning network. Claim 11 In claim 10, the above feature information is a method for synchronizing a virtual camera on a surveillance camera and a 3D digital twin system, comprising global feature information and local feature information. Claim 12 A method for synchronizing a virtual camera on a surveillance camera and a 3D digital twin system, wherein the step of extracting feature information is characterized by using a deep learning network trained to input a real video stream and output data converted into a 3D digital twin form, or input data converted into a 3D digital twin form and output an image of the real video stream. Claim 13 A method for synchronizing a virtual camera on a surveillance camera and a 3D digital twin system, wherein, in claim 12, the step of extracting feature information is characterized by using a convolutional neural network to extract global feature information, local feature information, and feature map information from the actual image stream. Claim 14 delete Claim 15 A method for synchronizing a virtual camera on a surveillance camera and a 3D digital twin system, wherein, in claim 9, the 3D digital twin DB includes scene images captured using a virtual camera in the virtual space of the digital twin and information regarding the position, attitude, and zoom of the camera when capturing with the virtual camera, and the scene images include local and global feature information extracted through a deep learning-based feature information extraction network that has completed training. Claim 16 A method for synchronizing a virtual camera on a surveillance camera and a 3D digital twin system, wherein, in claim 9, the step of selecting the scene is characterized by determining a final matching target by comparing the regional feature information of a scene image candidate group of the 3D digital twin data selected by the position estimation unit with the regional feature information extracted from the data converted into a 3D digital twin form.
Citation Information
Patent Citations
Apparatus and Method for Real Time Camera Tracking for Large Map
KR101199492B1
Three-dimensional monitoring system and method using real-time warping of surveilance videos
KR101890166B1
Method and system for detecting disaster damage information based on artificial intelligence using drone
KR102203135B1
