System for generating 3D model based on video data

Abstract

A system to generate three-dimensional models from video data. Pose (position and orientation) information of a data collection device 100, is reconstructed based on video data. Pose information and t

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention

[0001] The present invention relates to a system for generating a 3D model based on video data. 2. Description of Related Art

[0002] In the related art, a method using red, green, blue, and depth (RGBD) information is often used for a high-density 3D reconstruction. For example, PTL 1 discloses that "a method of reconstructing a scene as a 3D model using images acquired by a 3D sensor and an omnidirectional camera is provided. The 3D sensor can be a Kinect (trademark) sensor, and the omnidirectional camera can be a wide field of view (FOV) camera. An RGB image is acquired by the camera, and the RGBD image is acquired by the 3D sensor. A 3D model of the scene is formed using both types of images." Citation List Patent Literature

[0003] PTL 1: JP2018-510419A SUMMARY OF THE INVENTION According to the technique in the related art, the high-density 3D reconstruction is possible by using the RGBD information in various scenes assumed in industrial applications, such as a large scene and a scene with little texture. The number of 3D models generated by the technique in the related art is usually one, and qualities are uniform over an entire area. However, the 3D model generated by the technique in the related art may not be suitable for a use case of a 3D model in industry.

[0005] Specifically, there is a request to list an entire target environment in order to grasp a progress of work, an arrangement status of articles, and the like, and a request to check a specific area of the target environment in detail in order to check or educate equipment. For the former request, a single low-quality 3D model (overall model) of the entire environment is required, and for the latter request, a plurality of 3D models (partial model groups) having any quality corresponding to the specific area of the environment are required. In the technique in the related art, it is not possible to generate a 3D model satisfying both requests in one measurement.

[0006] An object of an aspect of the invention is to generate and browse a plurality of 3D model groups having different target areas and qualities in a 3D reconstruction.

[0007] In order to achieve the above object, a typical example of the invention is a system for generating a 3D model based on video data. The system includes: one or more processors; and one or more storage devices. The one or more storage devices store video data from a data collection device, and the one or more processors reconstruct first pose information of the data collection device based on the video data, generate, based on the first pose information and the video data, one or more 3D partial models for one or more partial spaces determined to have an importance level higher than a threshold in an entire space in which the video data is captured, reconstruct second pose information of the data collection device based on the video data, and generate a 3D overall model for the entire space based on the second pose information and the video data.

[0008] According to an aspect of the invention, it is possible to generate and browse a plurality of 3D model groups having different target areas and qualities in a 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration diagram of an industrial high-density 3D reconstruction system; FIG. 2 is a flowchart showing a procedure of partial model group generation processing; FIG. 3 is a flowchart showing a procedure of overall model generation processing; FIG. 4 shows an example of an overall model display interface; FIG. 5 shows an example of a partial model display interface; and FIG. 6 schematically shows a computer configuration. DESCRIPTION OF EMBODIMENTS

[0010] Hereinafter, an embodiment of the invention will be described with reference to the drawings. The embodiment to be described later does not limit the invention according to the claims, and all of the various elements described in the embodiment and the combinations thereof are not necessarily essential for the solution of the invention. In addition, illustration and description of well-known components that are essential for the configuration of the invention may be omitted.

[0011] In the following description, information from which an output is obtained in response to an input may be described by an expression such as an "xxx table", and this information may be data having any structure. Therefore, the "xxx table" can be referred to as "xxx information".

[0012] In the following description, a configuration of each table is an example. One table may be divided into two or more tables, or all or some of two or more tables may be one table.

[0013] In the following description, processing may be described using a "program" as the subject. Since the program is executed by a processor unit to perform predetermined processing while appropriately using a storage unit and / or an interface unit, the subject of the processing may be the processor unit (or a device such as a controller including the processor unit).

[0014] The program may be installed in a device such as a computer, or may be on, for example, a program distribution server or a computer-readable (for example, non-transitory) recording medium. In the following description, two or more programs may be implemented as one program, or one program may be implemented as two or more programs.

[0015] The "processor unit" is one or more processors. The processor may be typically a micro-processor such as a central processing unit (CPU), and may be another type of processor such as a graphics processing unit (GPU) . The processor may be a single-core processor or a multi-core processor. In addition, the processor may be a processor in a broad sense such as a hardware circuit (for example, a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC)) that performs a part or all of the processing.

[0016] In the following description, when elements of the same type are described without being distinguished, a reference sign (or a common sign of the reference signs) may be used, and when elements of the same type are distinguished and described, identification numerals (or the reference signs) of the elements may be used. In addition, the number of elements shown in each drawing is an example and is not limited to the illustration.

[0017] FIG. 1 is a configuration diagram of an industrial high-density 3D reconstruction system. The industrial high- density 3D reconstruction system shown in FIG. 1 includes a data collection device 100, a 3D reconstruction device 110, and a 3D model display device 120.

[0018] The data collection device 100 includes an integrated sensor unit 101, a data collection processing unit 102, and a notification input unit 103.

[0019] The integrated sensor unit 101 includes a sensor capable of acquiring RGBD information and other sensors that can be added as necessary. The sensor capable of acquiring the RGBD information is, for example, an RGBD camera or a stereo camera. The other sensors are, for example, inertial measurement units.

[0020] The data collection processing unit 102 collects data obtained from the integrated sensor unit, and transmits the data to the 3D reconstruction device 110 or stores the data in a secondary storage device or the like on the data collection device 100. The data collection is, for example, to stack data sequentially transmitted from each sensor of the integrated sensor unit in time series in time synchronization .

[0021] The notification input unit 103 notifies a person of information and receives an input of the person. For example, display of the acquired RGBD information, and an input of a start and an end of the data collection are performed.

[0022] The 3D reconstruction device 110 includes a partial model group SLAM processing unit 111, a partial model group generation processing unit 112, an overall model SLAM processing unit 113, an overall model generation processing unit 114, and a notification input unit 115.

[0023] The partial model group SLAM processing unit 111 performs SLAM processing on the data transmitted or stored by the data collection processing unit 102, and reconstructs pose information (including information on a position and an orientation) of the data collection device 100 used for generating the partial model group. The reconstructed pose information of the data collection device 100 is transmitted to the partial model group generation processing unit 112 or stored in a secondary storage device or the like on the 3D reconstruction device 110.

[0024] For the partial model group SLAM processing unit 111, for example, a known SLAM technique using RGBD information such as ORB-SLAM can be used.

[0025] The partial model group generation processing unit 112 integrates the RGBD information and the pose information of the data collection device 100 for the data transmitted or stored by the data collection processing unit 102 and the partial model group SLAM processing unit 111, and generates the partial model group. The generated partial model group is transmitted to the 3D model display device 120 or stored in a secondary storage device or the like on the 3D reconstruction device 110.

[0026] The overall model SLAM processing unit 113 performs the SLAM processing on the data transmitted or stored by the data collection processing unit 102, and reconstructs the pose information of the data collection device 100 used for generating an overall model. The reconstructed pose information of the data collection device 100 is transmitted to the overall model generation processing unit 114 or stored in the secondary storage device or the like on the 3D reconstruction device 110.

[0027] For the overall model SLAM processing unit 113, for example, a known SLAM technique using the RGBD information such as the ORB-SLAM can be used.

[0028] The overall model generation processing unit 114 integrates the RGBD information and the pose information of the data collection device 100 for the data transmitted or stored by the data collection processing unit 102 and the overall model SLAM processing unit 113, and generates the overall model. The generated overall model is transmitted to the 3D model display device 120 or stored in the secondary storage device or the like on the 3D reconstruction device 110 .

[0029] The notification input unit 115 notifies a person of information and receives an input of the person. For example, an input of a start of a 3D reconstruction, an input of various parameters, and display of progress of 3D model generation are performed.

[0030] The partial model group SLAM processing unit 111 and the overall model SLAM processing unit 113 may be the same.

[0031] A processing format in the partial model group SLAM processing unit 111 and the partial model group generation processing unit 112 may be, for example, a format in which the data transmitted or stored by the data collection processing unit 102 is sequentially processed frame by frame or a format in which all data stored in the data collection processing unit 102 is batch-processed.

[0032] The 3D model display device 120 includes a notification input unit 121.

[0033] The notification input unit 121 notifies a person of information on the partial model group and the overall model transmitted or stored by the partial model group generation processing unit 112 and the overall model generation processing unit 114, and receives an input from the person. For example, a 3D model is selected, the 3D model is visualized, and a viewpoint change of the 3D model is performed.

[0034] FIG. 2 is a flowchart showing a procedure of partial model group generation processing performed by the partial model group generation processing unit 112. For example, a partial model group is generated from a video file being captured or one or a plurality of captured video files. First, for a new partial model, a TSDF volume is generated (S200) . The TSDF volume is a 3D representation obtained using a truncated signed distance function (TSDF), which is a known distance function.

[0035] Next, next data (frame) is received from the data collection processing unit 102 and the partial model group SLAM processing unit 111 (S201) . The information obtained by the SLAM processing includes information on a pose (position and orientation) of the data collection device 100 of the corresponding frame and a position (positional relationship between frames) of the frame in an entire area of the video. Next, an importance level of a subject (frame) in a frame being processed is calculated (S202).

[0036] Next, it is determined whether a state in which the importance level is high continues (S203). When the state in which the importance level is high does not continue, the process returns to S201. When the state in which the importance level is high continues, the RGBD information and the pose information of the data collection device 100 in a current frame are integrated (stored) in the partial model TSDF volume at a preset target resolution (S204). Accordingly, a resolution required for the referred partial model for detailed information can be reliably secured. The resolution is calculated based on a resolution of a texture, the number of vertices of a mesh, and the like.

[0037] Next, it is determined whether to end the processing in the current frame (S205). For example, the end of the processing may be caused by a stop of the capturing of a last frame or video in the video file. When the processing is to be ended in the current frame (S205: YES), polygon meshes (entities of 3D partial models) are respectively generated from all TSDF volumes and stored (S208), and the partial model group generation processing is ended.

[0038] When the processing is not to be ended in the current frame (S205: NO), it is determined whether a capacity of the partial model TSDF volume is larger than a calculation resource (for example, a capacity of the primary storage device) of the 3D reconstruction device 110 (S206) . For example, it is determined whether the capacity of the partial model TSDF volume is larger than a preset threshold. When the capacity of the partial model TSDF volume is larger than the calculation resource of the 3D reconstruction device 110, the partial model TSDF volume is stored and the process returns to S200. When the capacity of the partial model TSDF volume is equal to or smaller than the calculation resource of the 3D reconstruction device 110, the process returns to S201. Accordingly, the capacity of each partial model TSDF volume can be set to a value appropriate for processing or less.

[0039] The TSDF volume in the procedure of the partial model group generation processing in FIG. 2 may be a volume obtained by another distance function.

[0040] The importance level of the subject (frame) in S202 can be obtained by, for example, a moving velocity of the data collection device 100 and a frequency of the RGBD information in the frame being processed. For example, the importance level is represented by a function of the moving velocity and / or the frequency of the RGBD information of the frame, the lower the moving velocity, the higher the importance level is calculated, and the higher the frequency the higher the importance level is calculated. The frequency of the RGBD information is represented by, for example, a function of the frequency of each of R, G, B, and D, and a part thereof may be removed. Alternatively, the importance level for generating the partial model may be assigned to a partial space designated in advance.

[0041] In S203, for example, a moving average of the importance level in peripheral frames of the frame being processed can be used to determine whether the state in which the importance level is high continues. The peripheral frames may include, for example, a predetermined number of frames preceding a target frame or a predetermined number of frames preceding and following the target frame.

[0042] For the mesh generation in S208, for example, a Marching Cubes algorithm can be used.

[0043] FIG. 3 is a flowchart showing a procedure of overall model generation processing performed by the overall model generation processing unit 114. First, all data is received from the data collection processing unit 102 and the overall model SLAM processing unit 113 (S300) . Next, for the received data, one 3D bounding box is calculated based on a movement trajectory of the data collection device 100 (S301).

[0044] Next, a mesh capacity (capacity of the 3D model) after the overall model is reconstructed is estimated based on the bounding box (S302). For example, the mesh capacity is estimated based on a volume of the bounding box and a reference resolution. Next, a new overall model TSDF volume is generated (S303) . Next, the next data (frame) is extracted from all the data received from the data collection processing unit 102 and the overall model SLAM processing unit 113 (S304).

[0045] Next, it is determined whether a partial model group is present in the vicinity of the pose (position and / or orientation) of the data collection device 100 in the frame being processed (S305) . For a determination of the vicinity, the preset threshold may be referred to. When the partial model group is not present in the vicinity of the pose of the data collection device 100 in the frame being processed (S305: NO), the RGBD information and the pose information of the data collection device 100 in the current frame are integrated into the overall model TSDF volume at a resolution such that the estimated mesh capacity after the reconstruction of the overall model is equal to or smaller than (for example, equal to) a smaller one of a preset upper limit mesh capacity which is set in advance and allows comfortable browsing of the mesh and the calculation resource (for example, the capacity of the primary storage device) of the 3D reconstruction device 110 (S306) . A calculation resource limit of the 3D reconstruction device 110 may be the same as a calculation resource limit of the 3D reconstruction device 110 in the generation of the partial model. The same applies to step S307.

[0046] In the determination of the resolution here, the mesh capacity estimated in step S302 is recalculated using the resolution as a variable, the value is compared with the smaller one of the two values, and a resolution at which a difference thereof becomes a predetermined value is determined. At this time, it may be assumed that all frames have a common resolution.

[0047] When the partial model group is present in the vicinity of the pose of the data collection device 100 in the frame being processed (S305: YES), the RGBD information and the pose information of the data collection device 100 in the current frame are integrated into the overall model TSDF volume at a resolution such that the estimated mesh capacity after the reconstruction of the overall model is sufficiently smaller than the smaller one of the preset upper limit mesh capacity which is set in advance and allows comfortable browsing of the mesh and the calculation resource of the 3D reconstruction device 110 (S307) . Accordingly, it is possible to reduce the data capacity while preventing deterioration of a quality of the 3D model.

[0048] In the determination of the resolution here, the mesh capacity estimated in step S302 is recalculated using the resolution as the variable, the value is compared with the smaller one of the two values, and the resolution at which the difference thereof becomes the predetermined value is determined. The predetermined value is larger than the predetermined value in step S306. At this time, it may be assumed that all frames have a common resolution. As described above, the frames are classified into two types of frames, that is, the frame in which the partial model is present in the vicinity and the frame in which the partial model is not present in the vicinity, and a lower resolution is given to the frame in which the partial model is present in the vicinity. All the frames may be integrated at the common resolution, so that a condition of the estimated mesh capacity is satisfied.

[0049] Next, it is determined whether the current frame is the last frame (S308). When the current frame is the last frame (S308: YES), a mesh is generated from the overall model TSDF volume (S309), and the overall model generation processing is ended. When the current frame is not the last frame (S308: NO), the process returns to S304. The processing in step S308 is the same as that in step S205.

[0050] The TSDF volume in the procedure of the overall model generation processing in FIG. 3 may be a volume obtained by another distance function.

[0051] In S301, for example, principal component analysis can be used instead of calculating the bounding box of the movement trajectory of the data collection device.

[0052] In order to estimate the mesh capacity after the reconstruction of the overall model based on the bounding box in S302, for example, the volume of the bounding box can be used.

[0053] For the mesh generation in S309, for example, the Marching Cubes algorithm can be used.

[0054] FIG. 4 is a diagram showing an example of a display interface of the overall model in the 3D model display device 120. FIG. 5 is a diagram showing an example of a display interface of a partial model in the 3D model display device 120.

[0055] A 3D model display interface 400 is a user interface for selecting data to be displayed, displaying an overall model or a partial model, selecting a partial model, and the like.

[0056] When the overall model is displayed, the 3D model display interface 400 includes an information display window 410, an overall model display area 420, a data selection button 430, and a partial model selection button 440.

[0057] The information display window 410 displays an identifier of data, a data measurement date and time, a capacity of a 3D model, and the like.

[0058] The overall model display area 420 is an area for performing operations such as display of the overall model and a display viewpoint change of the overall model. In the overall model display area 420, an overlay display 421 indicating an area where the partial model is present and an identifier thereof is performed.

[0059] The data selection button 430 is a button to be pressed when a display target of the 3D model is switched to one obtained by another measurement.

[0060] The partial model selection button 440 is a button to be pressed when the display is switched to the partial model having the identifier corresponding to a label of the button.

[0061] When the partial model is displayed, the 3D model display interface 400 includes the information display window 410, the data selection button 430, the partial model selection button 440, a partial model display area 450, and an overall model display button 460.

[0062] The partial model display area 450 is an area for performing operations such as display of the partial model and a display viewpoint change of the partial model. Satellite display 451 of the overall model is performed in the partial model display area 450.

[0063] The overall model display button 460 is a button to be pressed when switching the display to the overall model.

[0064] FIG. 6 shows a hardware configuration example of a computer 600 according to an embodiment of the present specification. Each of the data collection device 100, the 3D reconstruction device 110, and the 3D model display device 120 may have a computer configuration. Some or all of these devices may be implemented in one computer, and each device may be implemented in a system including a plurality of computers.

[0065] The computer 600 includes a processor 601 that executes various programs, a primary storage device 602 that stores various programs, and a secondary storage device 603 that stores various types of data. The processor 601 may include one or more cores, and the primary storage device 602 is, for example, a DRAM including a volatile storage area. The secondary storage device 603 is, for example, a hard disk drive (HDD) or a flash memory, and can provide a nonvolatile storage area.

[0066] The computer 600 further includes an output device 604 for presenting information to a user of the device, an input device 605 for inputting instructions, images, and the like by the user, and a communication device 606 for communicating with other devices. Component elements of the computer 600 are connected to one another by a bus. The user may use a device connected to the computer 600 via a network instead of an input and output device of the computer 600.

[0067] Functional units of the computer 600 can be implemented by, for example, the processor 601 operating according to a program. The processor 601 reads and executes various programs from the primary storage device 602 as necessary. The primary storage device 602 can store programs and data used by the programs. The programs and reference data are, for example, loaded from the secondary storage device 603 to the primary storage device 602, and executed and processed by the processor 601. At least a part of the functional units may be implemented by a logic circuit.

[0068] The output device 604 may include devices such as a display, a printer, and a speaker. The input device 605 may include devices such as a keyboard, a mouse, and a microphone. An output device 204 presents an input result from the user and a processing result obtained by the computer 600. An instruction from the user is input to the computer 600 by an input device 205. When another device is used, the input and output device functions in the same manner, and the output device 604 and the input device 605 can be omitted.

[0069] For example, the communication device 606 receives data transmitted from another device connected via the network, and transmits a processing result by the computer 600 to another device. Note that some devices may be omitted

[0070] As described above, according to the industrial high-density 3D reconstruction system in the present embodiment, the partial model group SLAM processing unit, the partial model group generation processing unit, the overall model SLAM processing unit, and the overall model generation processing unit of the 3D reconstruction device each perform 3D reconstruction processing on the data collected by the data collection device, thereby generating a plurality of 3D model groups having different target areas and qualities in one measurement.

[0071] The plurality of 3D model groups can be integrated, displayed, and operated by the 3D model display device. Accordingly, it is possible to easily satisfy two needs for an industrial 3D model, that is, listing of an entire environment and definition of a specific area, and it is possible to simplify a process of generating and browsing an industrial 3D model.

[0072] The invention is not limited to the embodiment described above and includes various modifications. For example, the above embodiments have been described in detail to facilitate understanding of the invention, and the invention is not necessarily limited to those including all the configurations described above. In addition to deletion of such a configuration, it is also possible to replace or add a configuration.

[0073] A part or all of the configurations, functions, processing units, processing methods, and the like described above may be implemented by hardware by, for example, designing with an integrated circuit. The invention can also be implemented by a program code of software for implementing functions of the embodiment. In this case, a recording medium recording the program code is provided to a computer, and a processor provided in the computer reads the program code stored in the recording medium. In this case, the program code read from the recording medium implements the functions in the embodiments described above by itself, and the program code itself and the recording medium storing the program code implement the invention. Examples of the recording medium for supplying such a program code include a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, a solid state drive (SSD), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, a nonvolatile memory card, and a ROM.

[0074] The program code for implementing the functions described in the present embodiment can be implemented in a wide range of programs or script languages such as Assembler C / C++, Python, Shell, PHP, and Java (registered trademark).

[0075] Control lines and information lines considered to be necessary for description are shown in the embodiments described above, and not all control lines and information lines in a product are necessarily shown. All the configurations may be connected to each other.

Claims

1. A system for generating a 3D model from video data, the system comprising:one or more processors; andone or more storage devices, whereinthe one or more storage devices store video data from a data collection device, andthe one or more processorsreconstruct first pose information of the data collection device based on the video data,generate, based on the first pose information and the video data, one or more 3D partial models for one or more partial spaces determined to have an importance level higher than a threshold in an entire space in which the video data is captured,reconstruct second pose information of the data collection device based on the video data, andgenerate a 3D overall model for the entire space based on the second pose information and the video data.

2. The system according to claim 1, whereinthe one or more processors generate each 3D partial model based on the importance level of consecutive frames in the video data.

3. The system according to claim 1, whereinthe one or more processorsgenerate a volume of each 3D partial model using a predetermined distance function, andintegrate a frame of the video data into the volume of each 3D partial model such that a volume capacity is within the threshold.

4. The system according to claim 1, whereinthe one or more processorsgenerate a volume of each 3D partial model using a predetermined distance function, andintegrate a frame of the video data into the volume at a target resolution.

5. The system according to claim 1, whereinthe one or more processorsgenerate a volume of the 3D overall model using a predetermined distance function, anddetermine a resolution of a frame of the video data to be integrated into the volume such that an estimated capacity after reconstruction of the 3D overall model is within a threshold.

6. The system according to claim 1, whereinthe one or more processorsgenerate the 3D overall model after generating the one or more 3D partial models,generate a volume of the 3D overall model using a predetermined distance function,sequentially integrate a frame of the video data into the volume at a determined resolution, anddetermine a resolution of a frame determined to be in the vicinity of any of the one or more 3D partial models to be a resolution lower than a resolution of another frame .

7. The system according to claim 1, wherein the one or more processorsgenerate a volume of each 3D partial model using a predetermined distance function, andintegrate a frame of the video data into the volume of each 3D partial model at a target resolution.

8. The system according to claim 1, wherein the one or more processorsdisplay an image indicating an area of the one or more 3D partial models on an image of the 3D overall model, anddisplay an image of a selected 3D partial model according to user selection from the one or more 3D partial models .

9. A method for generating a 3D model from video data that is executed by a system, the method comprising:acquiring, by the system, video data from a data collection device;reconstructing, by the system, first pose information of the data collection device based on the video data;generating, by the system, based on the first pose information and the video data, one or more 3D partial models for one or more partial spaces determined to have an importance level higher than a threshold in an entire space in which the video data is captured;reconstructing, by the system, second pose information of the data collection device based on the video data; andgenerating, by the system, a 3D overall model for the entire space based on the second pose information and the video data.What is claimed is:

1. A system for generating a 3D model from video data, the system comprising:one or more processors; andone or more storage devices, whereinthe one or more storage devices store video data froma data collection device, andthe one or more processorsreconstruct first pose information of the data collection device based on the video data,generate, based on the first pose information and the video data, one or more 3D partial models for one or more partial spaces determined to have an importance level higher than a threshold in an entire space in which the video data is captured,reconstruct second pose information of the data collection device based on the video data,generate a 3D overall model for the entire space based on the second pose information and the video data,display an image indicating an area of the one or more 3D partial models on an image of the 3D overall model, anddisplay an image of a selected 3D partial modelaccording to user selection from the one or more 3D partial models.

2. The system according to claim 1, whereinthe one or more processors generate each 3D partial model based on the importance level of consecutive frames in the video data.

3. The system according to claim 1, whereinthe one or more processorsgenerate a volume of each 3D partial model using a predetermined distance function, andintegrate a frame of the video data into the volume of each 3D partial model such that a volume capacity is within the threshold.

4. The system according to claim 1, whereinthe one or more processorsgenerate a volume of each 3D partial model using a predetermined distance function, andintegrate a frame of the video data into the volume at a target resolution.

5. The system according to claim 1, whereinthe one or more processorsgenerate a volume of the 3D overall model using a predetermined distance function, anddetermine a resolution of a frame of the video data to be integrated into the volume such that an estimated capacity after reconstruction of the 3D overall model is within a threshold.

6. The system according to claim 1, whereinthe one or more processorsgenerate the 3D overall model after generating the one or more 3D partial models,generate a volume of the 3D overall model using a predetermined distance function,sequentially integrate a frame of the video data into the volume at a determined resolution, anddetermine a resolution of a frame determined to be in the vicinity of any of the one or more 3D partial models to be a resolution lower than a resolution of another frame.

7. A method for generating a 3D model from videodata that is executed by a system, the method comprising:acquiringby the system, video data from a datacollection device;reconstructing, by the system, first pose information of the data collection device based on the video data;generating, by the system, based on the first pose information and the video data, one or more 3D partial models for one or more partial spaces determined to have an importance level higher than a threshold in an entire space in which the video data is captured;reconstructing, by the system, second pose information of the data collection device based on the video data;generating, by the system, a 3D overall model for the entire space based on the second pose information and the video data;displaying an image indicating an area of the one or more 3D partial models on an image of the 3D overall model; anddisplaying an image of a selected 3D partial model according to user selection from the one or more 3D partial models.T +44(0)30 0300 2000

Images