Image information management system, image information management method, and image information management program
The image data management system addresses inefficiencies in three-dimensional reproduction by comparing and updating spatial data to maintain consistency, reducing the need for full re-acquisition.
Patent Information
- Application Number
- JP2020159967
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-09-24
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2040-09-24
AI Technical Summary
Conventional methods for three-dimensional reproduction require re-acquiring spatial data when movement or changes occur in a space, leading to increased costs and inefficiencies.
An image data management system that compares updated spatial information with stored data to determine differences, combining updated areas with existing spatial information while maintaining consistency, eliminating the need to re-acquire data for the entire space.
Enables efficient management of spatial data changes in real spaces targeted for AR content by updating only affected areas, preserving data consistency without re-acquiring the entire dataset.
Smart Images

Figure 0007581735000001 
Figure 0007581735000002
Abstract
Description
[Technical field]
[0001] The present invention is a technology related to an image data management system, which is a technology for managing and updating image data, acoustic data, shape data, depth data, etc. obtained from real space, which need to be obtained in order to display free viewpoint video in real space. [Background technology]
[0002] Free viewpoint video is realized by acquiring and stitching together image data and shape data from various directions in real space. Various technologies are used to acquire real space data, such as photographic data using a camera and 3D measurement to acquire shape by measuring distance.
[0003] A typical multi-view stereo device uses photographic data taken from various angles, estimates the camera's shooting position internally using the feature points of the subject in the image, and performs stereo processing to reconstruct the three-dimensional internal shape. Summary of the Invention [Problem to be solved by the invention]
[0004] Conventional spatial information for 3D reproduction needs to be acquired so that there is consistency across the entire acquired space to prevent any changes in movement or appearance within the space, and spatial reproduction is performed on the premise that the space is consistent. If any movement or changes in appearance occur within the space, the spatial data needs to be acquired again, which increases the cost of reacquisition.
[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an image data management system that can appropriately manage spatial data when photographing a real space that is the target of AR content, etc., even if changes occur within the space. [Means for solving the problem]
[0006] In one aspect of the present invention for solving the above problem, an image acquisition unit acquires image data, a storage unit stores the image data acquired by the image acquisition unit, and a method for determining an updated portion that is an area that has been updated by comparing the image data stored in the storage unit with acquired spatial information. Also, the image data stored in the storage unit is compared with previously acquired spatial information to determine whether or not there is a difference of a certain amount or more. A data update determination unit for determining whether If the difference is determined to be less than a certain amount, The update part determined by the data update determination unit Already acquired Spatial information and combination If it is determined that the difference is equal to or greater than a certain amount, the difference is combined with the acquired spatial information as a chronologically different update part. and a spatial information database that stores the spatial information combined by the spatial information combining section. Effect of the Invention
[0007] According to the present invention, it is possible to provide an image data management system that can appropriately manage spatial data when a real space that is the subject of AR content or the like is photographed, even if changes occur within the space. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram showing an example of a configuration of a spatial information management system according to an embodiment of the present invention. [Diagram 2] 11 is a flowchart showing an example of an operation of a spatial information update process according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] An embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram of an example of a configuration of a spatial information management system according to an embodiment of the present invention. In Fig. 1, the spatial information management system includes an image acquisition unit 101, an image determination unit 102, a storage unit 103, a data update determination unit 104, a spatial information synthesis unit 105, a spatial information database 106, and a storage unit 107. The image acquisition unit 101, the image determination unit 102, and the storage unit 107 are included in a terminal such as a smartphone owned by a client, and the storage unit 103, the data update determination unit 104, the spatial information synthesis unit 105, and the spatial information database 106 are included in a server (spatial information management server, computer).
[0010] The image acquisition unit 101 is, for example, a smartphone camera, and acquires image data by capturing an image. In the spatial information management system, spatial information is updated using the image acquired by the image acquisition unit 101. The acquired image data includes information such as location information, the time of capture, and camera parameters if a GPS positioning system is available.
[0011] The image determination unit 102 is a device that compares the image data acquired by the image acquisition unit 101 with spatial information previously stored in the memory unit 107, and determines (matches) to which position in the spatial information the acquired image data corresponds.
[0012] The memory unit 103 is a device that stores each image data acquired by the image acquisition unit 101 and used for judgment by the image judgment unit 102, and the judgment results (matching information) of the image judgment unit 102 regarding which position in the spatial information each image data corresponds to.
[0013] The data update determination unit 104 compares the image data stored in the storage unit 103 with the spatial information recorded in the spatial information database 106 to determine an updated area. The data update determination unit 104 also determines whether the number of updated locations exceeds a predetermined threshold.
[0014] The spatial information synthesis unit 105 adds the updated area determined by the data update determination unit 104 to the spatial information (combines the updated portion) to generate updated spatial information. If the data update determination unit 104 determines that the number of changed portions exceeds a predetermined threshold, the spatial information synthesis unit 105 adds the spatial information to the spatial information in a form that can be identified as, for example, data that differs in time series.
[0015] The spatial information database 106 is a device for saving the spatial information generated by the spatial information synthesis unit 105, and stores data on the shooting location and acquisition time.
[0016] The storage unit 107 is a device that downloads spatial information required for the determination by the image determination unit 102 from the spatial information database 106 and stores it in the storage area of each terminal.
[0017] Next, a method for updating spatial information will be described with reference to the flowchart of FIG.
[0018] First, the storage unit 107 acquires spatial information data in advance from the spatial information storage server (S101).
[0019] Next, image data is acquired by the image acquisition unit 101 (S102).
[0020] The image determination unit 102 performs matching between the image data acquired by the image acquisition unit 101 in S101 and the spatial information stored in the storage unit 107. The matching process compares the feature amount space in the space with the feature point space of the acquired image, and determines where in the space the image of the acquired image data corresponds. If matching fails, the image acquisition unit 101 acquires image data at a different position again (S103).
[0021] Next, the image determination unit 102 transmits the result of matching with the image data acquired by the image acquisition unit 101 (matching information) to the storage unit 103 of the spatial information management server (S104).
[0022] The storage unit 103 stores the multiple image data and the matching information transmitted in S104 (S105).
[0023] Next, the data update determination unit 104 determines obstacles (e.g., people, cars, etc.) included in the image data stored in the storage unit 103, and removes unnecessary areas in the image (S106).
[0024] Next, the data update determination unit 104 compares the image data stored in the storage unit 103 with the corresponding spatial information stored in the spatial information database 106, and performs a determination to sort the image data into two types according to the degree of match of the feature points (S107). If the feature points of the image data sufficiently match (YES in S107), the process proceeds to S108, and if they do not sufficiently match (NO in S107), it is determined that a spatial change has occurred in the real space, and the process proceeds to S109.
[0025] If it is determined in S107 that the feature points of the image data sufficiently match, the spatial information synthesis unit 105 refers to the updated area determined by the data update determination unit 104, combines the updated parts, and adopts the combined spatial information as image data for updating the latest spatial information (S108).
[0026] If it is determined in S107 that there is not sufficient match, the spatial information synthesis unit 105 combines the difference between the image data and the spatial information into the spatial information as a chronologically different update part, and adopts the combined spatial information as image data that updates the latest spatial information (S109).
[0027] Next, the spatial information synthesis unit 105 stores the image data adopted in S108 and S109 in the spatial information database 106 to update the spatial information to the latest one, and ends the process (S110).
[0028] As described above, in the image information management system according to the present invention, the image data is compared with the acquired spatial information to determine whether there is an update, and the updated portion that is considered to have changed in the space is combined with the spatial information as information of a different time series, so that the consistency of the acquired spatial information that is consistent over the entire space is not lost. Therefore, there is no need to reacquire the spatial information over the entire space in order to generate spatial information that is consistent over the entire space. [Industrial Applicability]
[0029] The present invention is useful for image information management systems. [Explanation of symbols]
[0030] 101 Image acquisition unit 102 Image Judgment Unit 103 Storage section 104 Data update determination unit 105 Spatial Information Synthesis Unit 106 Spatial Information Database 107 Storage section
Claims
1. an image acquisition unit that acquires image data; a storage unit that stores image data acquired by the image acquisition unit; a data update determination unit that compares the image data stored in the storage unit with previously acquired spatial information to determine an updated portion, which is an area that has been updated, and also compares the image data stored in the storage unit with previously acquired spatial information to determine whether or not there is a difference of a certain amount or more; a spatial information synthesis unit that combines the update portion determined by the data update determination unit with the acquired spatial information when it is determined that the difference is less than a certain amount, and combines the difference with the acquired spatial information as a chronologically different update portion when it is determined that the difference is equal to or greater than a certain amount; and a spatial information database for storing the spatial information combined by the spatial information combining unit.
2. An image information management method executed by a computer of a spatial information management server, comprising: A step of comparing newly acquired image data with previously acquired spatial information to determine an updated portion, which is an area that has been updated; A step of comparing newly acquired image data with previously acquired spatial information to determine whether or not there is a difference therebetween of a certain amount or more; a step of combining the update portion with the acquired spatial information when it is determined that the difference is less than a certain amount, and combining the difference with the acquired spatial information as a time-series different update portion when it is determined that the difference is equal to or greater than a certain amount; and storing the combined spatial information.
3. An image information management program executed by a computer of a spatial information management server, A step of comparing newly acquired image data with previously acquired spatial information to determine an updated portion, which is an area that has been updated; A step of comparing newly acquired image data with previously acquired spatial information to determine whether or not there is a difference therebetween of a certain amount or more; a step of combining the update portion with the acquired spatial information when it is determined that the difference is less than a certain amount, and combining the difference with the acquired spatial information as a time-series different update portion when it is determined that the difference is equal to or greater than a certain amount; and storing the combined spatial information.
Citation Information
Patent Citations
Image processing device, image processing method and program
JP2018136793A
Image processing apparatus, control method for image processing apparatus, and program
JP2020046960A
Method, system and apparatus for capture of image data for free viewpoint video
JP2020095717A
