Method and system for arranging camera based on virtual simulation

Abstract

The present invention relates to a method and system for arranging a camera based on a virtual simulation, and may comprise the steps of: receiving, as an input, information regarding a target space and information regarding a target object; separating a virtual target space generated on the basis of the information regarding the target space, designating a plurality of camera areas, and arranging one camera in each camera area; arranging the target object in the virtual target space; calculating the rotation angle of each camera, and adjusting the rotation angle of the camera so as to have the maximum interest point and the maximum interest area; evaluating the visibility of all the cameras with respect to the target object, and deriving a camera arrangement having a minimum number of cameras and a visibility of a preset ratio or more through a simulation that has been performed a plurality of times; and providing the derived optimal camera arrangement.

Description

Camera placement method and system based on virtual simulation

[0001] The present invention relates to a camera arrangement method and system based on virtual simulation, and more specifically, to a camera arrangement method and system based on virtual simulation, which generates a target space identical to an actual space in a virtual environment by entering information about an actual target space, and derives a camera arrangement that optimally detects a target object by arranging the target object in the target space in the generated virtual environment through simulation.

[0002] The present invention was derived from research conducted with the support of the Information and Communications Technology Planning and Evaluation Institute with funding from the government (Ministry of Science and ICT) in 2023 (No. 1711193561, (Detail 3) Development of ultra-high-resolution atypical plenoptic video acquisition technology for medium and large spaces).

[0003] Mixed reality (MR) is a technology that combines virtual reality (VR) and augmented reality (AR), providing an interactive environment with graphics added to the real world. To achieve this, high-precision capture of real-world objects or spaces is required.

[0004] To realize mixed reality technology, technology is being used to reconstruct objects from 3D digital data using information such as multiple images. However, the technology is being developed with a focus on matching feature points or performing post-processing based on multiple images to accurately reconstruct actual objects, and research on how to optimally place cameras to acquire 3D digital data is insufficient.

[0005] The technology underlying the present invention is described in Republic of Korea Patent Publication No. 10-2018-0007845 (published on January 24, 2018).

[0006] Thus, according to the present invention, a method and system for camera placement based on virtual simulation are provided, which generates a target space identical to the actual space in a virtual environment by entering information about an actual target space, and derives a camera placement that optimally detects the target object by arranging the target object in the target space in the generated virtual environment through simulation.

[0007] According to an embodiment of the present invention for achieving the above technical task, a method for arranging cameras based on a virtual simulation may include: a step in which an input unit receives information on a target space and information on a target object; a step in which an area designation unit divides a virtual target space generated based on the information on the target space into voxels of a predetermined size, designates a plurality of camera areas in the divided virtual target space, and places one camera in each of the camera areas; a step in which an object placement unit places a target object in the virtual target space; a step in which a camera control unit calculates a rotation angle of each camera for recognizing the target object through information on cameras placed in each camera area, and adjusts the rotation angle of each camera so as to have a maximum point of interest and a maximum area of ​​interest; a step in which a simulation unit evaluates the visibility of all cameras with respect to the target object, and in a simulation performed multiple times, removes a preset number of cameras from among the plurality of cameras to evaluate the visibility of the remaining cameras, and then derives a camera arrangement having a minimum number of cameras and a visibility higher than a preset ratio; and a step in which a result provision unit provides the derived camera arrangement.

[0008] The above input receiving step can create an identical virtual target space based on the information of the input target space.

[0009] The step of placing the above camera includes the step of dividing the virtual target space into voxels of a predetermined size based on information of the target space, designating N camera areas; and the step of placing one camera in each of the camera areas, wherein N is a natural number.

[0010] The step of placing the target object may place the target object at a predetermined location in the virtual target space or place the target object according to the user's selection.

[0011] The step of adjusting the rotation angle of the camera may include the step of calculating the rotation angle of the camera for recognizing the target object in the virtual target space using the field of view of the camera, and rotating the camera by each of the calculated rotation angles; and the step of rotating the camera so that it has a rotation angle for photographing the maximum point of interest and the maximum area of ​​interest from the currently placed position.

[0012] The step of deriving the above camera arrangement includes the steps of: evaluating the visibility of the entire camera by calculating the detection rate of each camera for the target object; calculating the visibility of the remaining cameras by removing a preset number of cameras from among a plurality of cameras and performing a simulation according to the calculated visibility of the cameras; and deriving the camera arrangement if the minimum number of cameras (M) is among the results of performing a plurality of simulations and the visibility of the M cameras is equal to or greater than a preset ratio, wherein the detection rate is the ratio of detected detection points among a plurality of detection points designated for the target object.

[0013] The step of performing the above simulation includes removing a preset number of cameras during one simulation, calculating the visibility of the remaining cameras, and repeatedly performing the simulation while removing a preset number of cameras from the remaining cameras if the calculated visibility is greater than or equal to a preset ratio, restoring and labeling the removed preset number of cameras if the calculated visibility is less than the preset ratio, and performing the simulation by removing a preset number of cameras from among the cameras other than the restored and labeled cameras, and when the simulation is performed for all cameras, the simulation can be terminated.

[0014] According to another embodiment of the present invention, a camera arrangement system based on a virtual simulation may include: an input unit for receiving information on a target space and information on a target object; an area designation unit for dividing a virtual target space generated based on information on the target space into voxels of a predetermined size, designating a plurality of camera areas in the divided virtual target space, and placing one camera in each of the camera areas; an object arrangement unit for arranging a target object in the virtual target space; a camera control unit for calculating a rotation angle of each camera for recognizing the target object through information on cameras arranged in each camera area, and adjusting the rotation angle of each camera so as to have a maximum point of interest and a maximum area of ​​interest; a simulation unit for evaluating visibility of all cameras with respect to the target object, and for evaluating visibility of the remaining cameras by removing a preset number of cameras from among the plurality of cameras in a simulation performed multiple times, and then deriving a camera arrangement having a minimum number of cameras and a visibility greater than a preset ratio; and a result provision unit for providing the derived camera arrangement.

[0015] Thus, according to the present invention, a placement method capable of acquiring optimal 3D digital data with a minimum number of cameras can be achieved through simulation. This minimizes the economic and time costs associated with camera installation.

[0016] FIG. 1 is a configuration diagram of a camera placement system based on virtual simulation according to one embodiment of the present invention.

[0017] FIG. 2 is a flowchart of a camera placement method based on virtual simulation according to another embodiment of the present invention.

[0018] FIG. 3 is a schematic diagram of a camera placement method based on virtual simulation according to another embodiment of the present invention.

[0019] FIG. 4 is an exemplary diagram illustrating a virtual target space separated based on voxels according to another embodiment of the present invention.

[0020] FIG. 5 is a drawing showing an example of placing each camera in a camera area according to another embodiment of the present invention.

[0021] FIG. 6 is a diagram illustrating an example of deriving an optimal camera arrangement by performing a simulation according to another embodiment of the present invention.

[0022] FIG. 7 is a diagram illustrating an example of deriving optimal camera placement for multiple target objects according to another embodiment of the present invention.

[0023] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. In this process, the thickness of lines and the sizes of components depicted in the drawings may be exaggerated for clarity and convenience of explanation.

[0024] Furthermore, the terms described below are defined based on their functions within the present invention, and may vary depending on the intent or custom of the user or operator. Therefore, the definitions of these terms should be based on the overall content of this specification.

[0025] FIG. 1 is a configuration diagram of a camera placement system based on virtual simulation according to one embodiment of the present invention.

[0026] As illustrated in FIG. 1, a virtual simulation-based camera placement system (100) may include an input unit (110), an area designation unit (120), an object placement unit (130), a camera control unit (140), a simulation unit (150), and a result provision unit (160).

[0027] First, the input unit (110) can receive target space information for which 3D image data is desired to be generated. Here, the target space information can include the width, length, and height of the space and the shape of the space (e.g., a rectangular parallelepiped, a regular tetrahedron, etc.), and, if necessary, can further include features of the target space (e.g., topographic information of the space, such as the sea or mountains).

[0028] Specifically, the input unit (110) can receive target space information and create an identical virtual target space based on the input target space information.

[0029] Additionally, the input unit (110) can receive information about a camera to be placed in a virtual target space from a user or from a database (not shown).

[0030] Next, the area designation unit (120) can separate the target space into voxels of a predetermined size based on the input target space information, designate multiple camera areas in the separated voxels, and place one camera in each camera area.

[0031] Specifically, the area designation unit (120) divides the target space into voxels of a predetermined size (e.g., 50 cm × 50 cm × 50 cm, 1 m × 1 m × 1 m) based on the information of the input target space, and can designate N camera areas. Here, N is a natural number.

[0032] Additionally, the area designation unit (120) can place one camera in each camera area.

[0033] Next, the object placement unit (130) can place a target object in a virtual target space.

[0034] Specifically, the object placement unit (130) can place the target object in a virtual target space at a predetermined location (e.g., the (2,8)th voxel or (0,10,50) of the virtual target space, etc.) or place the target object according to the user's selection.

[0035] Next, the camera control unit (140) calculates the rotation angle of the camera for recognizing the target object using the camera information arranged in each camera area, and can adjust the rotation angle of each camera to have the maximum point of interest (POI) and the maximum region of interest (ROI). Here, the point of interest is a target point existing in the virtual target space and is used as a criterion for determining whether the current arrangement of the cameras arranged in each camera area has the maximum visibility. At this time, the cameras arranged in each camera area can derive a camera arrangement that maximally captures the point of interest by comparing the point of interest captured before and after rotation through the rotation angle adjustment of the camera at the current arrangement position. In addition, the region of interest is an area that includes the point of interest in the virtual target space and can include configuration information of the surrounding space in the camera field of view including the point of interest. At this time, the camera arrangement that captures the maximum point of interest in the field of view of the camera arranged in each camera area can derive maximum region of interest information from the surrounding space information of the maximum point of interest through the camera field of view.

[0036] Specifically, the camera control unit (140) can calculate the rotation angle of the camera for recognizing a target object in a virtual target space using the field of view of the camera, and rotate the camera by each of the calculated rotation angles.

[0037] Additionally, the camera control unit (140) can rotate each camera to have a rotation angle that captures the maximum point of interest and maximum area of ​​interest from the currently placed position.

[0038] Next, the simulation unit (150) evaluates the visibility of the camera for the target object, and in a simulation performed multiple times, removes a preset number of cameras (e.g., 1, 3, etc.) from among the multiple cameras, evaluates the visibility of the remaining cameras, and then derives a camera arrangement in which the detection rate for the target object is higher than a preset rate (e.g., 95%) with the minimum number of cameras.

[0039] Specifically, the simulation unit (150) can evaluate the visibility of the entire camera by calculating the detection rate of each camera for the target object. Here, the detection rate of the camera may be the ratio of detected detection points among a plurality of detection points designated for the target object.

[0040] Additionally, the simulation unit (150) can remove a preset number of cameras (e.g., 1) from among multiple cameras in a simulation performed multiple times and evaluate the visibility of the remaining cameras.

[0041] In addition, the simulation unit (150) can derive the camera arrangement when the detection rate of the camera among the results of multiple simulations is greater than or equal to a predetermined rate (e.g., 95%) and the minimum number of cameras (M units) is used.

[0042] Next, the result providing unit (160) can provide the derived camera arrangement.

[0043] Specifically, the result provision unit (160) can provide the final number of M cameras and the location of each camera.

[0044] Hereinafter, a camera placement method based on virtual simulation will be described in more detail with reference to FIGS. 2 to 7.

[0045] FIG. 2 is a flowchart for a camera placement method based on virtual simulation according to another embodiment of the present invention, and FIG. 3 is a schematic diagram for a camera placement method based on virtual simulation according to another embodiment of the present invention.

[0046] First, the input unit (110) can receive information on the target space (Space import) and information on the target object (Target object) (S210).

[0047] Here, the target spatial information may include the width, length, and height of the space and the shape of the space (e.g., rectangular parallelepiped, regular tetrahedron, etc.), and may further include features of the target space (e.g., topographic information of the space such as the sea or mountains) as needed.

[0048] Additionally, information about the target object may include gender, height, and weight if the target object is a person, species (e.g., dog, cat, parrot, shepherd, Maltese, Korean shorthair, etc.), fur color, and size if the target object is a non-human animal, and type (e.g., chest of drawers, fish tank, etc.) and size (width × length × height) if the target object is an object.

[0049] Specifically, the input unit (110) can receive information of a target space including a voxel composition and a space shape for the target space, and can receive information of a target object including a detection point and an object shape for detecting the target object.

[0050] In addition, the input unit (110) can receive target space information and generate an identical virtual target space based on the input target space information. At this time, the virtual target space can be defined as a voxel (Define voxel area).

[0051] Additionally, the input unit (110) can receive information about a camera to be placed in a virtual target space from a user or from a database (not shown).

[0052] Next, the area designation unit (120) separates the target space based on voxels of a predetermined size based on the input target space information, designates multiple camera areas in the separated target space, and can place one camera in each camera area (S220).

[0053] FIG. 4 is an exemplary diagram illustrating a virtual target space separated based on voxels according to another embodiment of the present invention, and FIG. 5 is a diagram illustrating an example of placing each camera in a camera area according to another embodiment of the present invention.

[0054] As shown in FIGS. 4 and 5, the area designation unit (120) can separate a virtual target space based on voxels of a predetermined size based on information of the input target space and designate N camera areas (Define camera placement area) (S221).

[0055] According to one embodiment of the present invention, the area designation unit (120) can divide a virtual target space based on voxels and designate 72 camera areas in the outer area of ​​the divided space.

[0056] Additionally, the area designation unit (120) can place one camera in each camera area (S222).

[0057] Next, the object placement unit (130) can place a target object in a virtual target space (S230).

[0058] Specifically, the object placement unit (130) can place the target object in a virtual target space at a predetermined location (e.g., the (2,8)th voxel or (0,10,50) of the virtual target space, etc.) or place the target object according to the user's selection.

[0059] Next, the camera control unit (140) calculates the rotation angle of the camera for recognizing the target object using the information of the cameras arranged in each camera area, and can adjust the rotation angle of each camera to have the maximum point of interest and the maximum region of interest (Camera POI, ROI setup) (S240). Here, the point of interest is a target point existing in the virtual target space and is used as a criterion for determining whether the current arrangement of the cameras arranged in each camera area has the maximum visibility. At this time, the cameras arranged in each camera area can derive a camera arrangement that maximally captures the point of interest by comparing the point of interest captured before and after rotation through the rotation angle adjustment of the camera at the current arrangement position. In addition, the region of interest is an area that includes the point of interest in the virtual target space and can include configuration information of the surrounding space in the camera field of view including the point of interest. At this time, the camera arrangement that captures the maximum point of interest in the field of view of the cameras arranged in each camera area can derive maximum region of interest information from the surrounding space information of the maximum point of interest through the camera field of view.

[0060] Specifically, the camera control unit (140) can calculate the rotation angle of the camera for recognizing a target object in a virtual target space using the field of view of the camera, and rotate the camera by the calculated rotation angle (S241).

[0061] Additionally, the camera control unit (140) can rotate each camera to have a rotation angle that captures the maximum point of interest and the maximum area of ​​interest from the currently placed position (S242).

[0062] Next, the simulation unit (150) evaluates the visibility of each camera for the target object, and in a simulation performed multiple times, a preset number of cameras (e.g., 1, 4, etc.) is removed from among the multiple cameras to derive a camera arrangement in which the detection rate for the target object is higher than the preset rate with the minimum number of cameras (S250).

[0063] Specifically, the simulation unit (150) can calculate the detection rate of each camera for the target object and evaluate the visibility of the entire camera (Verification (Camera visibility)) (S251). Here, the detection rate of the camera may be the ratio of detected detection points among a plurality of detection points designated for the target object.

[0064] In addition, the simulation unit (150) can remove a preset number of cameras (e.g., 1 camera) from among multiple cameras in a simulation performed multiple times and calculate the visibility of the remaining cameras (Analysis of results) (S252).

[0065] At this time, the simulation unit (150) performs a simulation by removing an additional preset number of cameras if the evaluated visibility is greater than a preset ratio, evaluates the visibility of all cameras, and if the evaluated visibility is less than a preset ratio, restores the deleted cameras and performs a simulation by removing cameras other than the deleted cameras, and evaluates the visibility of all cameras.

[0066] According to one embodiment of the present invention, the simulation unit (150) removes one camera during one simulation and calculates the visibility of the remaining cameras, and if the calculated visibility is greater than a preset ratio, the simulation can be performed again.

[0067] According to one embodiment of the present invention, if the visibility calculated during one simulation is less than a preset ratio, the simulation unit (150) can perform the simulation by recovering and labeling the removed camera and removing a preset number of cameras from among the other cameras except for the removed camera.

[0068] According to one embodiment of the present invention, the simulation unit (150) can terminate the simulation after performing the simulation for all cameras.

[0069] In addition, the simulation unit (150) can derive the camera arrangement when the visibility of the entire camera is greater than a predetermined ratio (e.g., 95%) and the minimum number of cameras (M) among the results of multiple simulations (Threshold<0.95) (S253).

[0070] According to one embodiment of the present invention, the simulation unit (150) can derive the camera arrangement when the minimum number of cameras (3) is used, each camera is labeled, and the total visibility of the three cameras is greater than or equal to a preset ratio.

[0071] In other words, if the detection rate of M cameras is less than a predetermined rate, the simulation unit (150) can change the location or number of cameras and perform the simulation again, and derive the camera placement by calculating the detection rate and visibility of the cameras.

[0072] According to one embodiment of the present invention, the simulation unit (150) can perform K simulations on 72 cameras to derive a camera arrangement using three cameras. Here, K is a natural number and represents the number of times the simulations are performed.

[0073] FIG. 6 is a drawing showing an example of deriving an optimal camera arrangement by performing a simulation according to another embodiment of the present invention, and FIG. 7 is a drawing showing an example of deriving an optimal camera arrangement for a plurality of target objects according to another embodiment of the present invention.

[0074] As illustrated in FIGS. 6 and 7, the simulation unit (150) can derive a camera arrangement that optimally captures at least one target object. At this time, the target object may be in a dynamic or static state.

[0075] Finally, the result providing unit (160) can provide the derived camera arrangement (S260).

[0076] Specifically, the result provision unit (160) can provide the final number of M cameras and the location of each camera.

[0077] According to the embodiments of the present invention described above, a placement method capable of acquiring optimal 3D digital data with a minimum number of cameras can be obtained through simulation. This minimizes the economic and time costs associated with camera installation.

[0078] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments are possible. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the following claims.

[0079] [Explanation of symbols]

[0080] 100: Camera placement system based on virtual simulation

[0081] 110: Input section

[0082] 120: Area designation section

[0083] 130: Object placement section

[0084] 140: Camera Control Unit

[0085] 150: Simulation Department

[0086] 160: Results Provider

Claims

1. A step in which the input unit receives information on the target space and information on the target object; A step of dividing a virtual target space based on information about the target space by a region designation unit into a virtual target space generated based on voxels of a predetermined size, designating a plurality of camera regions in the divided virtual target space, and placing one camera in each camera region; A step for placing an object in the virtual target space; A step of calculating the rotation angle of each camera for recognizing the target object through information of cameras placed in each camera area by the camera control unit, and adjusting the rotation angle of each camera to have the maximum point of interest and the maximum area of ​​interest; A step for evaluating the visibility of all cameras for the target object by the simulation unit, removing a preset number of cameras from among the multiple cameras in a simulation performed multiple times, evaluating the visibility of the remaining cameras, and deriving a camera arrangement that has the minimum number of cameras and a visibility ratio higher than the preset ratio; and A camera placement method based on a virtual simulation, comprising a step of providing a derived camera placement to a result provider.

2. In paragraph 1, The above input receiving step is, A camera placement method based on a virtual simulation that creates an identical virtual target space based on information about the input target space.

3. In paragraph 1, The steps for placing the above cameras are: A step of dividing the virtual target space into voxels of a predetermined size based on information of the target space and designating N camera areas; and Comprising the step of placing one camera in each of the above camera areas, The above N is a virtual simulation-based camera placement method where the N is a natural number.

4. In paragraph 1, The step of placing the above target object is: A camera placement method based on a virtual simulation for placing a target object at a predetermined location in the above virtual target space or placing the target object according to a user's selection.

5. In paragraph 1, The step of adjusting the rotation angle of the above camera is: A step of calculating the rotation angle of the camera for recognizing the target object in the virtual target space using the field of view of the camera and rotating the camera by the calculated rotation angle; and A camera placement method based on a virtual simulation, comprising the step of rotating each camera so that it has a rotation angle that photographs the maximum point of interest and the maximum area of ​​interest from the currently placed position.

6. In paragraph 1, The steps for deriving the above camera arrangement are: A step of evaluating the visibility of the entire camera by calculating the detection rate of each camera for the above target object; A step of removing a preset number of cameras from among multiple cameras to calculate the visibility of the remaining cameras and performing a simulation based on the calculated visibility of the cameras; and It includes a step of deriving a camera arrangement when the minimum number of cameras (M) is found among the results of multiple simulations and the visibility of the M cameras is greater than or equal to a predetermined ratio. The above detection rate is a virtual simulation-based camera placement method which is the ratio of detected detection points among multiple detection points specified for a target object.

7. In paragraph 6, The steps for performing the above simulation are: During one simulation, a preset number of cameras are removed, the visibility of the remaining cameras is calculated, and if the calculated visibility is greater than the preset ratio, a preset number of cameras are removed from the remaining cameras and the simulation is performed repeatedly. If the calculated visibility is less than a preset ratio, a preset number of cameras that were removed are restored and labeled, and a simulation is performed by removing a preset number of cameras from among the cameras other than the restored and labeled cameras. A virtual simulation-based camera placement method that terminates the simulation once the simulation has been performed for all cameras.

8. Input section for receiving information on target space and information on target object; A region designation unit that divides the virtual target space based on information about the target space into voxels of a predetermined size, designates multiple camera regions in the divided virtual target space, and places one camera in each camera region; An object placement unit that places a target object in the virtual target space; A camera control unit that calculates the rotation angle of each camera for recognizing the target object through information of cameras placed in each camera area, and adjusts the rotation angle of each camera to have the maximum point of interest and the maximum area of ​​interest; A simulation unit that evaluates the visibility of all cameras for the above target object, removes a preset number of cameras from among the multiple cameras in a simulation performed multiple times, evaluates the visibility of the remaining cameras, and derives a camera arrangement that has a minimum number of cameras and a preset ratio or higher; and A camera placement system based on virtual simulation including a result providing unit providing derived camera placements.

9. In paragraph 8, The above input section, A camera placement system based on a virtual simulation that creates an identical virtual target space based on information about the input target space.

10. In paragraph 8, The above area designation section is, Based on the information of the above target space, the virtual target space is divided into voxels of a predetermined size, N camera areas are designated, and one camera is placed in each of the camera areas. The above N is a virtual simulation-based camera placement system where the N is a natural number.

11. In paragraph 8, The above object placement section is, A camera placement system based on a virtual simulation that places a target object at a pre-designated location in the above virtual target space or places the target object according to the user's selection.

12. In paragraph 8, The above camera control unit, A camera placement system based on a virtual simulation that calculates the rotation angles of the cameras for recognizing target objects in the virtual target space using the field of view of the cameras, rotates the cameras by each of the calculated rotation angles, and rotates the cameras so that they have rotation angles that capture the maximum point of interest and the maximum area of ​​interest from their current placement positions.

13. In paragraph 8, The above simulation part, The detection rate of each camera for the above target object is calculated to evaluate the visibility of the entire camera, a preset number of cameras are removed from a plurality of cameras to calculate the visibility of the remaining cameras, and a simulation is performed according to the calculated visibility of the cameras, and if the minimum number of cameras (M) is among the results of the multiple simulations performed and the visibility of the M cameras is higher than a preset ratio, the camera arrangement is derived, The above detection rate is a virtual simulation-based camera placement system which is the ratio of detected detection points among multiple detection points specified for the target object.

14. In paragraph 13, The above simulation part, A camera placement system based on a virtual simulation, which removes a preset number of cameras during a single simulation, calculates the visibility of the remaining cameras, and if the calculated visibility is greater than a preset ratio, removes a preset number of cameras from the remaining cameras and repeatedly performs a simulation, and if the calculated visibility is less than the preset ratio, restores and labels the removed preset number of cameras, and performs a simulation by removing a preset number of cameras from other cameras excluding the restored and labeled cameras, and terminates the simulation when the simulation is performed for all cameras.

Images