Information processing apparatus and information processing method

Abstract

To assign a position corresponding to data while suppressing an increase in cost and man-hours.SOLUTION: An information processing apparatus holds registered data and information indicating a position corresponding to the registered data, acquires information indicating registration target data and the registered data that has a hierarchical relation sharing at least a part of positions with the registration target data, and assigns the position corresponding to the registration target data based on the position corresponding to the registered data that has a hierarchical relation with the registration target data and the position corresponding to the registered data that has a similar relation with the registration target data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Background art of the present invention is found in Japanese Patent Application Laid-Open No. 2017-44923, which states that "a system is provided which includes a plurality of position information devices that output position information representing absolute three-dimensional positions installed or existing in the real space of a plant, and a position information authoring device that virtually represents each piece of equipment arranged in the plant as a three-dimensional model, arranges the equipment in the virtual space, and manages the real positions of the equipment and the position information devices based on the three-dimensional plant model that marks the positions where the position information devices are installed" (see abstract). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-44923 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 acquires the absolute three-dimensional location of equipment from a location information device and registers the equipment data and location, but installing a location information device is costly. Furthermore, manually entering the location corresponding to the data increases the amount of work required. Therefore, one aspect of the present invention assigns the location corresponding to the data while suppressing increases in cost and work. [Means for solving the problem]

[0005] In order to solve the above problem, one aspect of the present invention employs the following configuration: An information processing device includes a processor and a memory, the memory holds registered data and information indicating a position corresponding to the registered data, the processor accepts a data registration request including data to be registered, acquires information indicating registered data in a hierarchical relationship that shares at least a portion of a position with the data to be registered, identifies registered data in a similarity relationship with the data to be registered based on a predetermined algorithm, assigns a position corresponding to the data to be registered based on the position corresponding to the registered data in the hierarchical relationship with the data to be registered and the position corresponding to the registered data in the similarity relationship with the data to be registered, and stores the data to be registered, the assigned position, and information indicating the registered data in the hierarchical relationship with the data to be registered in the memory. [Effects of the Invention]

[0006] According to one aspect of the present invention, it is possible to assign a position corresponding to data while suppressing increases in costs and man-hours.

[0007] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of a data management system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the data configuration of metadata stored in a metadata DB (DataBase) in the first embodiment. [Figure 3] 3 is a diagram illustrating an example of a data configuration of virtual space data stored in a virtual space DB according to the first embodiment. FIG. [Figure 4] FIG. 3 is a diagram illustrating an example of a data configuration of participant data stored in a participant DB according to the first embodiment. [Figure 5] FIG. 2 is an explanatory diagram illustrating an example of a hierarchical relationship between data in the first embodiment. [Figure 6A]FIG. 2 is an explanatory diagram showing a specific example of a hierarchical relationship between data in the first embodiment. [Figure 6B] FIG. 2 is an explanatory diagram showing a specific example of a hierarchical relationship between data in the first embodiment. [Figure 6C] FIG. 2 is an explanatory diagram showing a specific example of a hierarchical relationship between data in the first embodiment. [Figure 6D] FIG. 2 is an explanatory diagram showing a specific example of a hierarchical relationship between data in the first embodiment. [Figure 7] 10 is a flowchart illustrating an example of a data registration process according to the first embodiment. [Figure 8] 10 is a flowchart illustrating an example of a similar data selection process according to the first embodiment. [Figure 9A] FIG. 2 is an explanatory diagram illustrating an example of a hierarchical similarity measurement process in the first embodiment. [Figure 9B] FIG. 2 is an explanatory diagram illustrating an example of a hierarchical similarity measurement process in the first embodiment. [Figure 10] 10 is a flowchart illustrating an example of a data deletion process according to the first embodiment. [Figure 11] 10 is a flowchart illustrating an example of a data editing process according to the first embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a screen configuration of a query input screen in the first embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a screen configuration of a search result display screen in the first embodiment. [Figure 14] FIG. 10 is a diagram illustrating an example of a screen configuration of a data detail display screen in the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment, the same components are generally designated by the same reference numerals, and repeated explanations will be omitted. It should be noted that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention. [Example]

[0010] 1 is a block diagram showing an example of the configuration of a data management system. The data management system includes, for example, a data management device 100, participant terminals 200 used by participants, and worker terminals 300 used by on-site workers, and each participant terminal 200 and each worker terminal 300 is connected to the data management device 100 via a network 400 such as the Internet.

[0011] The data management device 100 provides a site 3D space as a 3D (Three-Dimensional) virtual space (hereinafter simply referred to as virtual space or space). Examples of sites include locations where work is performed, such as large factories and plants, and areas within large factories and plants (e.g., rooms within large factories and plants). The site 3D space is a 3D space constructed as a copy of the site space.

[0012] The participant terminal 200 may be, for example, a PC (Personal Computer), a smartphone, or a tablet terminal on which an application for realizing predetermined functions in a virtual space is installed, or may be a headset (for example, a VR (Virtual Reality) headset) including a goggle-type display device, headphones, a microphone, and a gyro sensor. The application may, for example, allow a participant to enter a virtual space such as a metaverse, move within the virtual space, and interact with other participants in the virtual space.

[0013] A participant using the participant terminal 200, for example, causes an icon representing the participant to enter the virtual space via an input device (e.g., a mouse, keyboard, touch panel, or microphone) provided in the participant terminal 200. An image corresponding to the position, orientation, and viewing angle of the icon is displayed on the output device of the participant terminal 200. For example, via the input device provided in the participant terminal 200, the participant can move the icon in the virtual space within the virtual space or to a different virtual space, or change the field of view displayed on the display device provided in the participant terminal 200 (e.g., change the viewing angle or line of sight, zoom in or out, etc.).

[0014] After an icon representing the participant enters the virtual space, the participant can input voice or text via an input device provided in the participant terminal 200, and the input voice or text can be output from the icon on an output device provided in the participant terminal 200 of the participant entering the virtual space. This allows the participant using the participant terminal 200 to hold a discussion with other participants in the virtual space. In this way, a participant is a person who can enter the virtual space.

[0015] The worker terminal 300 is carried by, for example, a worker on-site, and registers data in accordance with input by the worker in the data management device 100. The worker terminal 300 is, for example, a PC, a smartphone, or a tablet terminal.

[0016] The data management device 100 is configured by a computer having, for example, a CPU (Central Processing Unit) 101, a memory 102, an auxiliary storage device 103, a communication device 104, an input device 105, and an output device 106.

[0017] The CPU 101 is an example of a processor, and executes programs stored in the memory 102. The memory 102 includes a ROM (Read Only Memory), which is a nonvolatile storage element, and a RAM (Random Access Memory), which is a volatile storage element. The ROM stores unchanging programs (e.g., a BIOS (Basic Input / Output System)). The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by the CPU 101 and data used when the programs are executed.

[0018] The auxiliary storage device 103 is a large-capacity, non-volatile storage device such as a magnetic storage device (HDD (Hard Disk Drive)) or a flash memory (SSD (Solid State Drive)), and stores programs to be executed by the CPU 101 and data to be used when the programs are executed. That is, the programs are read from the auxiliary storage device 103, loaded into the memory 102, and executed by the CPU 101.

[0019] The input device 105 is a device that receives input from an operator, such as a keyboard, mouse, or microphone. The output device 106 is a device that outputs the results of program execution in a format that can be recognized by the operator, such as a display device, printer, or speaker.

[0020] The communication device 104 is a network interface device that controls communication with other devices in accordance with a predetermined protocol, and includes a serial interface such as a USB (Universal Serial Bus).

[0021] Some or all of the programs executed by the CPU 101 may be provided to the data management device 100 via a network from a removable medium (such as a CD-ROM or flash memory) or an external computer equipped with a non-transitory storage device, and stored in a non-volatile auxiliary storage device 103, which is also a non-transitory storage medium. For this reason, the data management device 100 should preferably have an interface for reading data from removable media. This also applies to the participant terminal 200 and the worker terminal 300.

[0022] The data management device 100 is a computer system that is configured on a single physical computer or on multiple logically or physically configured computers, and may run on separate threads on the same computer, or on a virtual computer built on multiple physical computer resources. The same applies to the participant terminal 200 and the worker terminal 300.

[0023] The CPU 101 includes, for example, a DB (DataBase) management unit 110, a data registration unit 120, a data processing unit 130, and a virtual space processing unit 140, all of which are functional units.

[0024] The DB management unit 110 manages various DBs held by the auxiliary storage device 103. The DB management unit 110 includes, for example, a DB deletion unit 111, a DB editing unit 112, a DB display unit 113, and a DB search unit 114, all of which are functional units. The DB deletion unit 111 deletes information stored in various DBs held by the auxiliary storage device 103. The DB editing unit 112 edits information stored in various DBs held by the auxiliary storage device 103. The DB display unit 113 displays information stored in various DBs held by the auxiliary storage device 103 on the output device 106. The DB search unit 114 searches for information stored in various DBs held by the auxiliary storage device 103.

[0025] The data registration unit 120 registers data in various DBs held in the auxiliary storage device 103. The data registration unit 120 includes, for example, a coordinate estimation unit 121, an automatic registration unit 124, and an input acquisition unit 125, all of which are functional units. The coordinate estimation unit 121 estimates a position (for example, three-dimensional coordinates, which are an example of a position) to be assigned to data that is to be registered or has already been registered in various DBs. The coordinate estimation unit 121 includes, for example, a similar data selection unit 122 and a coordinate setting unit 123. The similar data selection unit 122 selects data similar to the data to be registered. The coordinate setting unit 123 sets a position to be assigned to the data to be registered, based on the position assigned to the similar data selected by the similar data selection unit 122.

[0026] The automatic registration unit 124 generates and registers values ​​of items that can be automatically generated among the items of metadata to be assigned to the data to be registered. The input acquisition unit 125 acquires input of the data to be registered and the values ​​of the items of metadata to be assigned to the data to be registered.

[0027] The data processing unit 130 executes predetermined processing on the data to be registered or the registered data. The data processing unit 130 includes, for example, an analysis unit 131, a thumbnail creation unit 132, a search unit 133, a display unit 134, a feature calculation unit 135, and an editing unit 136, all of which are functional units.

[0028] The analysis unit 131 creates new registration target data from the registration target data by executing a specific process (hereinafter also referred to as an analysis process) according to the type of the registration target data. Specific examples of the analysis process according to the type of data will be described later. The thumbnail creation unit 132 generates a thumbnail corresponding to the registration target data. Specific examples of thumbnails will be described later.

[0029] The search unit 133 searches for data in the predetermined processing executed by the data processing unit 130. The display unit 134 displays data on the output device 106 in the predetermined processing executed by the data processing unit 130. The feature calculation unit 135 calculates features depending on the type of data to be registered. The editing unit 136 edits the data in the predetermined processing executed by the data processing unit 130.

[0030] The virtual space processing unit 140 executes processing related to the virtual space. The virtual space processing unit 140 includes, for example, a virtual space management unit 141 and a participant management unit 142, both of which are functional units. The virtual space management unit 141 manages the virtual space. The participant management unit 142 manages participants who participate in the virtual space.

[0031] For example, CPU 101 functions as DB management unit 110 by operating in accordance with a DB management program loaded into memory 102, and functions as data registration unit 120 by operating in accordance with a data registration program loaded into memory 102. The same relationship between programs and functional units applies to other functional units included in CPU 101.

[0032] Note that some or all of the functions of the functional units included in the CPU 101 may be realized by hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0033] The auxiliary storage device 103 includes, for example, registration information 150 and virtual space information 160, each of which is realized by a storage area for storing information. The registration information 150 includes, for example, a metadata DB 151, a raw data DB 152, and a thumbnail DB 153. The metadata DB 151 holds metadata assigned to data to be registered. The raw data DB 152 holds the data to be registered itself. Hereinafter, the data to be registered itself, i.e., raw data, will be simply referred to as data to distinguish it from metadata.

[0034] The virtual space information 160 includes, for example, a virtual space DB 161 and a participant DB 162. The virtual space DB 161 holds information for managing a virtual space including a site 3D space. The participant DB 162 holds information for managing participants.

[0035] Note that some or all of the information stored in the auxiliary storage device 103 may be stored in the memory 102, or may be stored in an external server or the like connected to the data management device 100. In particular, since the amount of data stored in the raw data DB 152 and the thumbnail DB 153 is large, it is preferable that the raw data DB 152 and the thumbnail DB 153 are included in an external server.

[0036] In this embodiment, the information used by the data management system is not dependent on the data structure and may be represented in any data structure, for example, a table, a list, a database, or a queue, as appropriate, may be used to store the information.

[0037] 2 is a diagram showing an example of the data configuration of metadata stored in the metadata DB 151. The metadata 1510 includes, for example, common items, relationship items, and special items. The common items are items that can be commonly assigned to all types of data. The common items include, for example, a data ID, a data name, a data type, a location, a site name, a user name, a registration date and time, an edit date and time, a memo, a data save destination, a thumbnail save destination, and a search tag.

[0038] The data ID is an ID that identifies the data. The data name is the name of the data. The data type is the type of data. Images, documents, text, videos, audio, sensor data, 3D models, and access data are all examples of data types. The site name is the name of the site where the data was acquired.

[0039] As the location, for example, a location (e.g., three-dimensional coordinates) linked to the data is stored. Examples of locations linked to data include the location where the data was acquired and the location of an object indicated by the data. Specifically, for example, there are locations where devices included in a video or image are located, locations where devices described in an instruction manual (document data or text data) are located, locations where devices that are the subject of discussion in a meeting indicated by audio data are located, locations where audio data was acquired, locations of objects measured by sensor data, and locations of objects photographed by a camera indicated by access data.

[0040] Note that one piece of data may be linked to multiple locations. Specifically, for example, if a certain device is installed at multiple locations, the locations of the multiple locations may be linked to the document data of the instruction manual for the device.

[0041] The username is the name of the user associated with the data (e.g., the registrant, editor, or author of the data). In addition to the user's name, the username may also include the user's attributes (e.g., the user's organization, age, gender, job responsibilities, and job title). The registration date and time indicates the date and time when the data was registered. The edit date and time indicates the date and time when the data and / or metadata was edited.

[0042] The memo indicates supplementary information about the data. For example, the details of how the data was acquired may be stored as a memo. The data storage location indicates the path (into the raw data DB 152) where the data is stored. The thumbnail storage location indicates the path (into the thumbnail DB 153) where a thumbnail of the data is stored. The search tag is a tag for searching for the data.

[0043] The relationship item is an item that indicates the relationship between data corresponding to the metadata 1510. The relationship item includes, for example, a parent data ID and a child data ID. The parent data ID is the data ID of the parent data of the data. The child data ID is the data ID of the child data of the data. Details of the hierarchical relationship between parent data and child data will be described later. The relationship item may also include an item that indicates the data ID of data that is determined to be similar in the similar data selection process described later.

[0044] Special items are items that can be assigned depending on the type of data. In other words, the type of special items included in metadata may vary depending on the type of data. Special items corresponding to an image include, for example, features extracted from the image using a predetermined algorithm (e.g., CLIP (Contrastive Language-Image Pre-training)).

[0045] Special items corresponding to documents, text, and audio include, for example, features extracted from text extracted from data using a predetermined algorithm (e.g., CLIP), and summaries of the text (e.g., the first predetermined number of characters). Special items corresponding to sensor data include, for example, access information (e.g., IP address). Special items corresponding to 3D models include, for example, the brightness and orientation of the 3D model.

[0046] In this embodiment, an example is described in which all metadata items are stored in the metadata DB 151, but the common items, relationship items, and special items may be stored in different DBs. In this case, by including a data ID in the relationship items and special items, the common items, relationship items, and special items of the same data can be linked together.

[0047] 3 is a diagram showing an example of the data configuration of virtual space data stored in the virtual space DB 161. The virtual space data 1610 indicates, for example, a space ID that identifies the virtual space, a data ID of data (e.g., 3D model data) that constitutes the virtual space, a data save destination for the data, and a display size of the data (e.g., a ratio to the size of the original 3D model). Note that coordinates within the 3D model data (virtual space) and coordinates in real space (e.g., positions indicated by metadata) are associated in advance. That is, in this embodiment, positions within the 3D model data (virtual space) and positions in real space can be treated as equivalent.

[0048] 4 is a diagram showing an example of the data configuration of participant data stored in participant DB 162. Participant data 1620 indicates, for example, a participant ID for identifying a participant, icon data of the participant displayed in the virtual space, the space ID of the virtual space in which the participant (icon) is present, a coordinate field indicating the current position of the participant (icon) in the virtual space, and the current orientation of the participant (icon) in the virtual space (the viewing angle displayed on the participant terminal 200 used by the participant).

[0049] The virtual space management unit 141 registers information in the virtual space data 1610, for example, in accordance with input from the user of the data management device 100 to the input device 105. The participant terminal 200 generates an initial registration instruction including a participant ID and icon data in accordance with the input from the participant, and transmits it to the data management device 100. The participant management unit 142 registers the user ID and icon data in the participant data 1620 in accordance with the initial registration instruction.

[0050] The participant management unit 142 also receives an entry instruction from, for example, the participant terminal 200 of a participant who is not yet in the virtual space. The entry instruction includes, for example, the participant ID of the participant, the space ID of the virtual space the participant wishes to enter, and coordinates and orientation within the virtual space. The participant management unit 142 stores the space ID, coordinates, and orientation indicated by the entry instruction in a record of the participant data 1620 that corresponds to the participant ID indicated by the entry instruction. The participant management unit 142 also places icon data corresponding to the participant ID indicated by the entry instruction in the virtual space of the space ID indicated by the entry instruction at the coordinates indicated by the entry instruction, in the orientation indicated by the entry instruction.

[0051] The participant management unit 142 also receives an exit instruction from, for example, the participant terminal 200 of a participant currently in the virtual space. The exit instruction includes, for example, the participant ID of the participant. The participant management unit 142 changes the space ID, coordinates, and orientation of the record in the participant data 1620 that corresponds to the participant ID indicated by the exit instruction to null values. The participant management unit 142 also deletes the icon data that corresponds to the participant ID indicated by the exit instruction from the virtual space. In this way, the participant can exit the virtual space.

[0052] Additionally, a participant currently in the virtual space inputs an instruction to move the icon and an instruction to change the orientation to the input device (e.g., a gyro sensor, keyboard, and / or mouse) of the participant terminal 200. At this time, the participant management unit 142 receives the movement instruction and orientation change instruction, including the participant ID of the participant, from the participant terminal 200, and reflects the post-movement coordinates and post-change orientation indicated by the instruction for the participant ID in the participant data 1620. Furthermore, the participant management unit 142 changes the coordinates and orientation of the icon for the participant ID in the virtual space to the post-movement coordinates and post-change orientation.

[0053] Figure 5 is an explanatory diagram showing an example of a hierarchical relationship between data. As shown in Figure 5, a parent-child relationship, which is an example of a hierarchical relationship, can be defined between data. For example, the parent-child relationship between data can be represented by a tree structure in which data is a node. The metadata of child data includes all of the positions included in the metadata of parent data. In other words, any data can be said to include all of the positions of all of the ancestor data of that data, and data in an ancestor-descendant relationship (hierarchical relationship) can be said to share at least some of the positions.

[0054] 5, parent data A is the parent data of child data C and child data D, and parent data B is the parent data of child data D. Therefore, the metadata of child data C includes all positions included in the metadata of parent data A, and the metadata of child data D includes all positions included in the metadata of parent data A and all positions included in the metadata of parent data B.

[0055] 5, child data D is the parent data of grandchild data E and grandchild data F. Therefore, the metadata of grandchild data E and the metadata of grandchild data F both include all positions included in the metadata of child data D. Therefore, it can be said that the metadata of grandchild data E and the metadata of grandchild data F include all positions included in the metadata of parent data A and all positions included in the metadata of parent data B.

[0056] It is assumed that parent-child relationships do not form loops. That is, for any combination of data, if one piece of data is ancestor data of another piece of data, the other piece of data is not ancestor data of the one piece of data.

[0057] In addition, when the position associated with any data is updated (for example, according to a user input to the data management device 100), the positions of all descendant data of that data are also updated in the same way. For example, when the value of the position associated with certain data is changed, the corresponding positions of all descendant data of that data are also changed to the same value. Furthermore, for example, when a new position is additionally associated with certain data, that position is additionally associated with all descendant data of that data. Furthermore, for example, when some or all positions associated with certain data are deleted, the corresponding positions of all descendant data of that data are also deleted. This eliminates the need for users to assign and update positions.

[0058] 6A, 6B, 6C, and 6D are explanatory diagrams showing specific examples of hierarchical relationships between data. In the example of FIG. 6A, "3DCGModel.glb," "Instruction Manual.pdf," "Memo A.txt," and "Memo B.txt" are all assumed to be data relating to the same object (e.g., a device). It is desirable that data relating to the same object be assigned the same position. Therefore, it is desirable that a hierarchical relationship be defined between these data (e.g., a data ID of parent data is set when child data is registered) by, for example, user input (or automatically if the object related to the data is specified).

[0059] In the example of FIG. 6A, "3DCG model.glb" is set as the parent data of "instruction manual.pdf," and "instruction manual.pdf" is set as the parent data of "memo A.txt" and "memo B.txt."

[0060] In the example of Figure 6B, "9 / 10 fixed point shooting.png," "9 / 12 fixed point shooting.png," and so on are data acquired periodically (by the same device) during fixed point observations such as maintenance work, and it is desirable for these data to be assigned the same location. Therefore, for example, it is advisable to set "access data" (data indicating the device acquiring the data) for the device acquiring data during fixed point observations (a network camera in the example of Figure 6B) as parent data, and automatically set the periodically acquired data as child data of the "access data." For example, access data for a network camera includes the IP address of the network camera, authentication information corresponding to the network camera, and so on.

[0061] In the example of Figure 6C, "transcription data.txt" and "transcription summary data.txt" are assumed to be data created from "audio data.mp3." Specifically, "transcription data.txt" is text data transcribed from the audio contained in the audio data "audio data.mp3," and "transcription summary data.txt" is text data summarizing the text transcribed from the audio contained in the audio data "audio data.mp3." Hereinafter, the process of creating other registration target data from registration target data will also be referred to as analysis processing.

[0062] As shown in the example of Figure 6C, it is desirable to assign the same position to the data that was the subject of the analysis process and the data created as a result of the analysis process. Therefore, in the example of Figure 6C, a hierarchical relationship is automatically set in which the data that was the subject of the analysis process, "audio data.mp3," is the parent data, and the data created as a result of the analysis process, "transcription data.txt" and "transcription summary data.txt," are each child data.

[0063] In the example of FIG. 6D, "weakly compressed.glb," "medium compressed.glb," and "strongly compressed.glb" are assumed to be data created by an analysis process on "3DData.glb." Specifically, "weakly compressed.glb," "medium compressed.glb," and "strongly compressed.glb" are 3D model data compressed at weak, medium, and strong compression rates, respectively, from the 3D model data "3DData.glb." Therefore, in the example of FIG. 6D, for example, a hierarchical relationship is automatically established in which "3DData.glb," the data that was the subject of the analysis process, is the parent data, and "weakly compressed.glb," "medium compressed.glb," and "strongly compressed.glb," the data created as a result of the analysis process, are the child data.

[0064] 7 is a flowchart showing an example of the data registration process. The input acquisition unit 125 receives a data registration request (S701). The input acquisition unit 125 may receive the data registration request via input to the input device 105, or may receive the data registration request from the participant terminal 200, the worker terminal 300, or a device that acquires data (for example, a sensor that acquires sensor data, a network camera that acquires images, etc.).

[0065] The data registration request includes the data to be registered. The data registration request may also specify values ​​for some or all of the metadata items of the data to be registered. For example, the values ​​for some or all of the metadata items may be specified by input to the input device 105, the participant terminal 200, or the worker terminal 300, or may be automatically specified by the participant terminal 200, the worker terminal 300, or a device that acquires data. Therefore, the data registration request may also include information such as the location of the data, the parent data ID of the data, and the child data ID of the data.

[0066] The automatic registration unit 124 creates metadata for the data to be registered (S702). Specifically, the automatic registration unit 124 creates values ​​for predetermined items of metadata for which values ​​are not specified in the data registration request. The predetermined items include, for example, a data ID, a user name, a data type, a registration date and time, a data storage destination, a thumbnail storage destination, a search tag, and special items. Furthermore, for example, if the registration request includes location information, the automatic registration unit 124 may create a site name corresponding to the location (for example, correspondence information between locations and site names is stored in advance in the auxiliary storage device 103).

[0067] A specific example of a method for creating values ​​for the predetermined items of metadata will be described below. For example, the automatic registration unit 124 determines a data ID so that the data ID does not overlap with data already registered in the registration information 150. For example, the automatic registration unit 124 acquires a user name from the login information of the terminal that uploaded the data. In addition, the automatic registration unit 124 identifies the data type from the data extension, for example.

[0068] The automatic registration unit 124 also acquires, for example, the date and time when the data is uploaded as the registration date and time. The automatic registration unit 124 also determines the data storage destination and thumbnail storage destination according to, for example, a predetermined rule (for example, a data storage destination and a thumbnail storage destination are determined in advance for each data type).

[0069] Furthermore, the automatic registration unit 124 extracts search tags from data, for example, according to a predetermined algorithm. Specifically, for example, the automatic registration unit 124 performs image recognition on data whose data type is an image and determines the name of the recognized object as a search tag, or performs topic estimation on text extracted from data whose data type is text, video, or audio and determines the name of the estimated topic as a search tag.

[0070] Next, the thumbnail creation unit 132 creates a thumbnail of the data to be registered (S703). A thumbnail is data that is displayed as a sample when displaying search results for registered data. Therefore, it is desirable that the thumbnail of the data to be registered has a smaller data volume than the data to be registered. The thumbnail creation unit 132 creates a thumbnail from the data to be registered using an algorithm that is predetermined for each data type of the data to be registered. The data type of the data to be registered and the data type of the thumbnail may be different. There may also be data types for which thumbnails are not created (in this case, information about the thumbnail save destination is deleted from the metadata).

[0071] A specific example of a thumbnail creation method will be described. For example, for data whose data type is an image, thumbnail creation unit 132 creates image data of the image with a reduced resolution as a thumbnail. Also, for data whose data type is text, thumbnail creation unit 132 creates a thumbnail of the text, up to a predetermined number of characters (e.g., 100 characters) from the beginning of the text.

[0072] For example, for data whose data type is video, the thumbnail creation unit 132 creates a thumbnail from video data that has been shortened to a predetermined length (e.g., 15 seconds).For example, for data whose data type is audio, the thumbnail creation unit 132 creates a thumbnail from audio data that has been shortened to a predetermined length (e.g., 15 seconds).

[0073] For example, for data whose data type is sensor data, the thumbnail creation unit 132 creates a thumbnail using sensor data whose sampling period for the sensor data value is increased to a predetermined time (for example, if the sensor value is acquired every second in the original data, the thumbnail creation unit 132 creates a thumbnail using the value acquired every 10 seconds). For example, for data whose data type is a 3D model, the thumbnail creation unit 132 creates a thumbnail using image data of an overhead image of the 3D model.

[0074] The analysis unit 131 determines whether to perform analysis processing on the data to be registered (S704). For example, whether or not the data is subject to analysis processing is determined in advance for each data type of the data to be registered. Furthermore, for example, the analysis unit 131 may determine whether to perform analysis processing in accordance with a user input to the input device 105, or a user input to the participant terminal 200 or the operator terminal 300 that sent the registration request.

[0075] When the analysis unit 131 determines that analysis processing should be performed on the data to be registered (S704: YES), it performs analysis processing according to the data type of the data to be registered (S705). Note that there may be data types for which the thumbnail creation processing and analysis processing are the same processing. In this case, if the processing of step S704 has been performed for that type of data, the processing of step S705 is omitted. Furthermore, the data generated as a result of the analysis processing is registered as raw data in the raw data DB 152 in the processing described below.

[0076] A specific example of the analysis process will be described. For example, for data whose data type is text or document, the analysis unit 131 creates text data as a result of the analysis process by summarizing the text included in the data using a predetermined algorithm. Also, for example, for data whose data type is video, the analysis unit 131 creates video data as a result of the analysis process by compressing the video using a predetermined algorithm.

[0077] Furthermore, for example, for data whose data type is voice, the analysis unit 131 creates text data obtained by transcribing the voice as a result of the analysis process.Furthermore, for example, for data whose data type is a 3D model, the analysis unit 131 creates 3D model data obtained by compressing the 3D model using a predetermined algorithm as a result of the analysis process.

[0078] The analysis unit 131 sets a hierarchical relationship between the data that was the subject of the analysis process in step S705 and the data created as a result of the analysis process (S706). Specifically, for example, the analysis unit 131 assigns a data ID to the data created as a result of the analysis process, sets the data ID of the data created as a result of the analysis process as a child data ID included in the metadata of the data that was the subject of the analysis process, and sets the data ID of the data that was the subject of the analysis process as a parent data ID included in the metadata of the data created as a result of the analysis process.

[0079] After the process of step S706, the process proceeds to step S707. Furthermore, if the analysis unit 131 determines not to execute analysis processing on the registration target data (S704: NO), the process proceeds to step S707.

[0080] The coordinate setting unit 123 determines whether parent data of the data to be registered exists (S707). For example, if the parent data ID is included in the registration request in step S701 and / or if the parent data ID is set in step S706, the coordinate setting unit 123 determines that parent data of the data to be registered exists.

[0081] If the coordinate setting unit 123 determines that parent data of the data to be registered exists (S707: YES), it adds all positions of all parent data of the data to be registered to the position information of the data to be registered (S708). After the processing of step S708, the process proceeds to step S709. If the coordinate setting unit 123 determines that parent data of the data to be registered does not exist (S707: NO), it proceeds to step S709.

[0082] The coordinate setting unit 123 determines whether or not location information of the data to be registered exists (S709). For example, if location information is included in the registration request in step S701, if location information is automatically created in the metadata creation process in step S702, and / or if location information is added in step S708, the coordinate setting unit 123 determines that location information of the data to be registered exists.

[0083] If the coordinate setting unit 123 determines that there is no position information of the data to be registered (S709: NO), the similar data selection unit 122 executes a similar data selection process (S710). The similar data selection process will be described in detail later. The similar data selection unit 122 adds the position acquired based on the similar data selected in the similar data selection process in step S710 to the coordinate information of the data to be registered (S711).

[0084] After the process of step S711, the process proceeds to step S712. If the coordinate setting unit 123 determines that coordinate information of the data to be registered exists (S709: YES), the process proceeds to step S712. Note that the determination of step S709 may be omitted, and the processes of steps S710 and S711 may always be executed.

[0085] The automatic registration unit 124 registers various metadata of the data to be registered in the metadata DB 151, the data to be registered itself (raw data) in the raw data DB 152, and the thumbnail in the thumbnail DB 153 (S712). At this time, the automatic registration unit 124 may include, for example, site names corresponding to the added locations (for example, correspondence information between locations and site names is stored in advance in the auxiliary storage device 103) in the metadata.

[0086] In addition, if the metadata of the data to be registered includes a child data ID (i.e., if the data to be registered is parent data of any registered data), in step S712, the automatic registration unit 124 adds all positions of the data to be registered to the positions of the registered data indicated by each of the child data IDs.

[0087] The automatic registration unit 124 determines whether any unregistered data exists among the data created as a result of the analysis processing in step S705 (S713). When the automatic registration unit 124 determines that any unregistered data exists among the data created as a result of the analysis processing (S713: YES), the automatic registration unit 124 regards the unregistered data as data to be registered, and returns to the processing of step S702. Note that the processing of steps S704 to S706 is omitted for data created as a result of the analysis processing. Furthermore, the processing of steps S704 to S706 may be omitted only when the data created as a result of the analysis processing is a specific type of data.

[0088] If the automatic registration unit 124 determines that there is no unregistered data among the data created as a result of the analysis process in step S705 (S713: NO), it ends the data registration process.

[0089] The data management device 100 of this embodiment can automatically assign a highly accurate location to data for which location information is not included in the data registration request, based on the hierarchical relationship and similar data (i.e., the location of data that has already been registered) through the above-described registration process. Therefore, even if the user of the data management device 100, participant terminal 200, or worker terminal 300 does not assign location information, the location is assigned to the data, reducing the burden on the user.

[0090] Furthermore, the data management device 100 can assign a location without using external sensors such as surveillance camera images or beacons, thereby reducing the cost of assigning location information. Furthermore, it is possible to avoid privacy issues that arise when using surveillance cameras to estimate location information, and problems such as reduced accuracy due to occlusion, and the accuracy of location estimation is improved compared to when beacons are used to estimate location information.

[0091] Furthermore, by assigning a location to data, the data management device 100 can manage the location assigned to the data by linking it to a virtual space, allowing the user to easily grasp the location of the data within the virtual space on the search result display screen described below, etc., thereby improving the user's understanding of the data.

[0092] In addition, the data management device 100 can assign not only the location of the physically existing equipment itself, but also the locations of objects related to the instruction manual or the audio data, for example.

[0093] 8 is a flowchart showing an example of the similar data selection process in step S710. The similar data selection unit 122 uses the feature amount calculation unit 135 to execute a vector similarity measurement process (S801).

[0094] A specific example of the vector similarity measurement process in step S801 will be described. For example, the feature amount calculation unit 135 calculates a feature amount (feature vector) of a predetermined type according to the data type for the data to be registered. Furthermore, the feature amount calculation unit 135 identifies, from among the registered data in the raw data DB 152 (data whose position is also registered), data of the same type as the data type and data of a different type from the data type but from which feature amounts of the same type can be calculated, and calculates feature amounts of the same type for the identified registered data.

[0095] Specific examples of feature amounts according to data type will be described. For image data, the feature amount calculation unit 135 calculates, for example, SURF (Speeded Up Robust Features) feature amounts and / or CLIP (Contrastive Language-Image Pre-training) feature amounts. For document data, the feature amount calculation unit 135 calculates, for example, Word2vec feature amounts and / or CLIP feature amounts.

[0096] For example, calculating SURF features for image data enables similarity determination between image data, and calculating CLIP features for image data and document data enables similarity determination between image data and document data. In this way, multiple types of features may be calculated for a specific type of data. There may also be types of data for which only one type of feature is calculated, and there may also be types of data for which no feature is calculated (i.e., types of data that are not subject to vector similarity measurement processing).

[0097] Furthermore, the similar data selection unit 122 calculates the similarity between the feature of the data to be registered and each of the feature of the specified registered data, based on the feature of the data to be registered and each of the feature of the same type of the specified registered data. Note that the similarity calculation algorithm may be predetermined for each type of data to be similarly determined, or may be predetermined for each type of feature.

[0098] The similar data selection unit 122 selects registered data similar to the data to be registered in the vector similarity measurement process based on the calculated similarity. Specifically, for example, the similar data selection unit 122 may select only the registered data with the highest similarity as the similar data, or may select a predetermined number of registered data in descending order of similarity as the similar data. Note that the similar data selection unit 122 may exclude registered data with similarity lower than a predetermined value from the similar data.

[0099] Furthermore, the similar data selection unit 122 registers information associating the registration target data with the data selected as the similar data in the metadata of these data (for example, the data ID of the similar data is registered as an example of a relationship item). Note that the similar data selection unit 122 also executes processing to associate the registration target data with the data selected as the similar data in each of steps S802 to S804 described below.

[0100] Next, the similar data selection unit 122 executes a metadata similarity measurement process (S802). A specific example of the metadata similarity measurement process in step S802 will be described. For example, the similar data selection unit 122 compares predetermined items, excluding the position, included in the metadata of the data to be registered with the predetermined items of the metadata of each piece of registered data (data whose position is also registered), to calculate the similarity between the data to be registered and each piece of registered data.

[0101] The predetermined item may be a plurality of items. In this case, the similar data selection unit 122, for example, calculates the similarity for each of the plurality of items and calculates a predetermined statistic (for example, an average value (which may be a weighted average value to which a predetermined weight is assigned)) of the calculated similarity as the similarity between the data to be registered and each of the registered data in the metadata similarity measurement process.

[0102] The similar data selection unit 122 selects registered data similar to the data to be registered in the metadata similarity measurement process based on the calculated similarity. Specifically, for example, the similar data selection unit 122 may select only the registered data with the highest similarity as the similar data, or may select a predetermined number of registered data in descending order of similarity as the similar data. Note that the similar data selection unit 122 may exclude registered data with similarity lower than a predetermined value from the similar data.

[0103] A specific example of a method for comparing the predetermined items of metadata will be described below: The predetermined items include, for example, a user name, a registration date and time, a memo, and a search tag.

[0104] The similar data selection unit 122 calculates the similarity for the user name, for example, between the data to be registered and each of the registered data, so that the value increases the more matches there are in the user name and each of the user's attributes (for example, the user's organization, age, gender, job responsibilities, and job title, etc.) indicated by the user name (which is an item of metadata) between the data to be registered and each of the registered data (for example, the similarity for the user name is calculated as the value obtained by multiplying the number of matches by a predetermined value).

[0105] The similar data selection unit 122 calculates the similarity regarding the registration date and time, for example, between the data to be registered and each of the registered data, by substituting the difference in the registration date and time into a predetermined function (for example, a function whose value increases as the difference in the registration date and time becomes closer). Also, for example, when the days, weeks, months, and / or days of the week indicated by the registration dates and times match, the similar data selection unit 122 may modify the similarity regarding the registration date and time to a higher value by adding a predetermined value to it, for example.

[0106] The similar data selection unit 122, for example, calculates Word2vec features from notes of the registration target data and each of the registered data, compares the features, and calculates similarity regarding the notes. Note that, for example, if the options for text that can be registered as metadata notes are limited in advance (for example, if only text selected by the user from multiple predetermined texts in the data registration request in step S701 can be registered as a note), the similar data selection unit 122 may calculate similarity regarding the notes based on whether the text of the notes matches, or may obtain a predetermined similarity for each combination of text of the notes. Note that, for example, the similar data selection unit 122 calculates similarity regarding search tags using a method similar to the method for calculating similarity regarding notes.

[0107] Next, the similar data selection unit 122 executes a hierarchical similarity measurement process (S803). For example, in step S803, the similar data selection unit 122 refers to the parent data ID and child data ID of each piece of metadata whose position has been registered, and identifies a set of registered data that constitutes a hierarchical relationship. The similar data selection unit 122 calculates the similarity between the registered data and the identified set by comparing the registered data with the identified set.

[0108] 9A and 9B are explanatory diagrams showing an example of the hierarchical similarity measurement process in step S803. First, the example of FIG. 9A will be described. It is assumed that the data to be registered is image data A. Furthermore, it is assumed that the set of registered data to be subjected to similarity judgment (a set of registered data in a hierarchical relationship) consists of image data B, which is parent data, and text data C and image data D, which are child data of image data B.

[0109] The similar data selection unit 122 calculates the similarity between image data A, which is the data to be registered, and each piece of data included in the set of registered data. The method of calculating each similarity may be, for example, the same as the vector similarity measurement process in step S801 or the metadata similarity measurement process in step S802. However, for example, the similarity between types of data to which the vector similarity measurement process in step S801 cannot be applied may be calculated by the metadata similarity measurement process in step S802.

[0110] The similar data selection unit 122 aggregates the calculated similarities to calculate the similarity between the image data A, which is the data to be registered, and the set. Specifically, for example, the similar data selection unit 122 may calculate the average value of the similarities between the image data A, which is the data to be registered, and each piece of data included in the set of registered data as the similarity between the image data A, which is the data to be registered, and the set. Furthermore, the similar data selection unit 122 may use a weighted average value, in which a predetermined weight is assigned, instead of the average value. It is desirable that the predetermined weight be a larger value for registered data that is located higher (ancestor) in the hierarchical relationship of the set.

[0111] In addition, the similar data selection unit 122 may calculate the similarity between each piece of data included in the set of registered data in a hierarchical relationship, from the upper side (ancestor side) up to a predetermined number of generations, and the image data A, which is the data to be registered (i.e., the similarity with lower-level data in the set of registered data in a hierarchical relationship may be ignored).

[0112] The example of Fig. 9B will be described. The set of data to be registered and registered data in a hierarchical relationship in the example of Fig. 9B is assumed to be the same as the example of Fig. 9A. The similar data selection unit 122 aggregates image data B, text data C, and image data D included in the set to generate image data E, which is new data that represents the set.

[0113] Specifically, for example, the similar data selection unit 122 generates metadata for image data E by calculating a predetermined statistical quantity (for example, a mode or an average value) for each item of metadata of image data B, text data C, and image data D. Furthermore, for example, the similar data selection unit 122 generates average data of image data that is the most frequent type of data included in the set (which may be the same type of data included in the set as the data to be registered) as image data E. Note that the similar data selection unit 122 may use only data from the upper side (ancestor side) up to a predetermined number of generations out of the data included in a set of registered data in a hierarchical relationship to generate new data that represents the set and metadata for the new data.

[0114] The similar data selection unit 122 calculates the similarity between image data A, which is data to be registered, and image data E, which is new data that represents the set, as the similarity between the data to be registered and the set. The method of calculating the similarity may be the same as the vector similarity measurement process in step S801 or the metadata similarity measurement process in step S802, for example. However, for example, the similarity between types of data to which the vector similarity measurement process in step S801 cannot be applied may be calculated by the metadata similarity measurement process in step S802.

[0115] The similar data selection unit 122 selects registered data similar to the data to be registered in the hierarchical similarity measurement process based on the calculated similarity. Specifically, for example, the similar data selection unit 122 may select only registered data included in a set with the highest similarity as the similar data, or may select registered data included in a predetermined number of sets in descending order of similarity as the similar data.

[0116] Returning to the description of Fig. 8, the similar data selection unit 122 executes a similarity integration process (S804). Specifically, for example, the similar data selection unit 122 identifies registered data (for example, data B) that has been determined to be similar to the data to be registered (for example, data A) in any of steps S801 to S803. The similar data selection unit 122 identifies registered data that is similar to data B but has not been determined to be similar to data A (for example, data C).

[0117] The similar data selection unit 122 selects data C as data similar to data A (data to be registered) in the similarity integration process. The similar data selection unit 122 can measure the similarity by the similarity integration process even for data for which the similarity relationship cannot be directly measured in steps S801 to S803.

[0118] 8, all of the processes from step S801 to step S804 are executed, but only a part of the processes from step S801 to step S803 may be executed, or the process of step S804 may be omitted. Also, which of the processes from step S801 to step S803 is executed, and whether or not the process of step S804 is executed may be specified by, for example, the user of the data management device 100.

[0119] In step S711, the coordinate setting unit 123 may, for example, add the average value of the positions of the data selected as similar data in step S801 to the position of the data to be registered, or may add a weighted average value (for example, the weight is determined according to the similarity) of the positions of the data selected as similar data in step S801 to the position of the data to be registered, or may add all the positions of the data selected as similar data in step S801 to the position of the data to be registered. The coordinate setting unit 123 determines positions to be added to the data to be registered in the same manner for the data selected as similar data in step S802, the data selected as similar data in step S803, and the data selected as similar data in step S804, and adds the determined positions to the data to be registered.

[0120] 10 is a flowchart showing an example of data deletion processing. The input acquisition unit 125 receives a data deletion request for registered data (S1001). The input acquisition unit 125 may receive the data deletion request via input to the input device 105, or may receive the data deletion request from the participant terminal 200 or the worker terminal 300. The data deletion request includes the data ID of the data to be deleted.

[0121] The DB deletion unit 111 deletes the data to be deleted from various DBs (S1002). Specifically, for example, the DB deletion unit 111 deletes the metadata of the data to be deleted indicated by the data deletion request from the metadata DB 151, deletes the data to be deleted from the raw data DB 152, and deletes the thumbnail corresponding to the data to be deleted from the thumbnail DB 153 (S1002).

[0122] The DB deletion unit 111 deletes the hierarchical relationship related to the data to be deleted (S1003) and ends the data deletion process. Specifically, for example, the DB deletion unit 111 deletes the data ID indicated by the data deletion request from the parent data ID and child data ID of each metadata included in the metadata DB 151.

[0123] For example, information indicating the data ID of the data that was the subject of the analysis process may be stored in the metadata of the data created as a result of the analysis process in step S705, and in this case, the DB deletion unit 111 may execute the processes of steps S1002 and S1003, with the data created from the data to be deleted by the analysis process also being subject to deletion.

[0124] 11 is a flowchart showing an example of data editing processing. The input acquisition unit 125 receives a data editing request for registered data (S1101). The input acquisition unit 125 may receive the data editing request via input to the input device 105, or may receive the data editing request from the participant terminal 200 or the worker terminal 300. The data editing request includes the data ID of the data to be edited, and further includes an updated value of any item of the metadata of the data to be edited, and / or the updated data (raw data).

[0125] The DB editing unit 112 determines whether the data editing request includes a request to edit a hierarchical relationship (editing a parent data ID and / or a child data ID) (S1102). If the DB editing unit 112 determines that the data editing request includes a request to edit a hierarchical relationship (S1102: YES), it edits the hierarchical relationship of the metadata of the data to be edited (i.e., parent data ID and / or child data ID) in accordance with the request to edit the hierarchical relationship (S1103).

[0126] In step S1103, the DB editing unit 112 determines whether a new parent data ID has been added to the data to be edited (S1104). If the DB editing unit 112 determines that a new parent data ID has been added to the data to be edited (S1104: YES), the DB editing unit 112 adds all positions of the newly added parent data to the position information of the metadata of the data to be edited (S1105), and proceeds to step S1106.

[0127] If the DB editing unit 112 determines that the data editing request does not include an editing request for a hierarchical relationship (S1102: NO), the process proceeds to step S1106. If the DB editing unit 112 determines that a new parent data ID has not been added to the data to be edited (S1104: NO), the process proceeds to step S1106.

[0128] The DB editing unit 112 determines whether the data editing request includes a position editing request (S1106). If the DB editing unit 112 determines that the data editing request includes a position editing request (S1106: YES), it edits the position of the data to be edited in accordance with the position editing request (S1107), and proceeds to step S1108.

[0129] In order to edit the position of the data to be edited to be the same as the position of the parent data, the data editing request must include both a request for editing the position and a request for editing the hierarchical relationship to delete the parent data ID of the parent data from the metadata of the data to be edited. In other words, the input acquisition unit 125 does not accept a data editing request that includes a request for editing the position of the data to be edited to be the same as the position of the parent data, but does not include a request for editing the hierarchical relationship to delete the parent data ID of the parent data from the metadata of the data to be edited.

[0130] If the DB editing unit 112 determines that the data editing request does not include a position editing request (S1106: NO), the DB editing unit 112 proceeds to step S1108. The DB editing unit 112 determines whether child data exists in the data to be edited (i.e., whether a child data ID is included in the metadata) (S1108).

[0131] If the DB editing unit 112 determines that child data exists in the data to be edited (S1108: YES), it edits the position of the child data (S1109) and proceeds to step S1110. Specifically, for example, when the DB editing unit 112 adds a new position to the data to be edited, it also adds the new position to the child data. Furthermore, for example, when the DB editing unit 112 edits a position included in the data to be edited that is also included in the child data, it also performs the same edit on the position of the child data.

[0132] If the DB editing unit 112 determines that child data does not exist in the data to be edited (S1108: NO), the DB editing unit 112 proceeds to step S1110. In accordance with the data editing request, the DB editing unit 112 edits data (metadata and / or raw data) other than the hierarchical relationship and position of the data to be edited (S1110), and ends the data editing process.

[0133] By the above-mentioned data editing process, the DB editing unit 112 can appropriately edit the position of the data to be edited indicated by the data editing request and / or the data in a hierarchical relationship with the edited data when editing a hierarchical relationship or when editing the position of data included in the hierarchical relationship.

[0134] In addition, if the data editing request includes a request to edit raw data, the DB editing unit 112 may automatically change metadata items, change thumbnails, perform analysis processing, or assign hierarchical relationships to the data resulting from the analysis processing according to the edited raw data.

[0135] In addition, in the above-mentioned steps S801 to S804, if data in a similar relationship are associated and registered, the DB editing unit 112 may re-perform the similar data selection process on data similar to the registered data whose position has been edited in the data editing process, and re-assign the position.

[0136] 12 is a diagram showing an example of the screen configuration of a query input screen for inputting a query to search registered data. Information for displaying the query input screen 1200 is generated, for example, by the DB search unit 114. The query input screen 1200 is displayed, for example, on the output device 106, the output device of the participant terminal 200, and / or the output device of the worker terminal 300.

[0137] The query input screen 1200 includes, for example, a search keyword input area 1210, a search condition setting area 1220, and a 3D model display area 1230. The DB search unit 114 searches the registered information 150 according to the search keyword input in the search keyword input area 1210 and / or the search condition set in the search condition setting area 1220. The search keyword input area 1210 is an area for accepting free input of a search keyword.

[0138] A 3D model is displayed in the 3D model display area 1230. When the query input screen 1200 is displayed on the output device of the participant terminal 200, the virtual space in which the participant of that participant terminal 200 is present is displayed in the 3D model display area 1230 in a field of view specified by that participant. When the query input screen 1200 is displayed on the output device 106 or the output device of the worker terminal 300, for example, 3D model data selected by the user of the output device 106 or the worker terminal 300 is displayed in a position and field of view specified by that user.

[0139] The search condition setting area 1220 includes, for example, a search type setting area 1221, a data type setting area 1222, a data name setting area 1223, a date setting area 1224, a username setting area 1225, a memo setting area 1226, a site name setting area 1227, a location setting area 1228, and a display number setting area 1229.

[0140] The search type setting area 1221 is an area for setting a search range using, for example, a search keyword. When "full text" is selected in the search type setting area 1221, for example, data containing the search keyword in any item of metadata is searched for. When "content" is selected in the search type setting area 1221, for example, data containing the search keyword in content (for example, text in document data or text data) is searched for. When "search tag" is selected in the search type setting area 1221, for example, data containing the search keyword in a search tag in metadata is searched for.

[0141] Note that a radio button for selecting any item included in the metadata (excluding items for which search conditions can be set in the data type setting area 1222, data name setting area 1223, date setting area 1224, user name setting area 1225, memo setting area 1226, site name setting area 1227, or location setting area 1228) may be displayed in the search type setting area 1221. In this case, when the item is selected, data containing the search keyword in the corresponding item of the metadata is searched for.

[0142] Furthermore, multiple items (excluding "full text") may be selectable at the same time in search type setting area 1221. In this case, when the multiple items are selected, data that includes the search keyword in all of the multiple items of metadata may be searched for, or data that includes the search keyword in at least one of the multiple items of metadata may be searched for.

[0143] The data type setting area 1222 is an area for setting the type of data to be searched. The data name setting area 1223 is an area for setting search conditions related to the data name included in the metadata. The date setting area 1224 is an area for setting search conditions related to the date of the data included in the metadata. Data for which the registration date and time of the metadata falls within the date range set in the date setting area 1224 may be searched for, or data for which the edit date and time of the metadata falls within the date range set in the date setting area 1224 may be searched for. Furthermore, in the date setting area 1224, not only the date but also the time may be set.

[0144] The user name setting area 1225 is an area for setting search conditions for the user name included in the metadata. The memo setting area 1226 is an area for setting search conditions for the memo included in the metadata. The site name setting area 1227 is an area for setting search conditions for the site name included in the metadata.

[0145] The position setting area 1228 is an area for setting search conditions related to positions included in metadata. In the example of Fig. 12, the center and radius can be set in the position setting area 1228, and data whose metadata position is included within the sphere defined by the set center and radius is searched for. Note that the position setting area 1228 may also be able to set a position in an area of ​​a shape other than a sphere.

[0146] Furthermore, instead of or in addition to the position setting area 1228, the user may specify the position, shape, and size of area 1231 displayed in the 3D model display area 1230, thereby making it possible to search for data whose metadata position is included in area 1231. This allows a participant who is currently in the virtual space to search for data located within area 1231 included in the participant's field of view while remaining in the virtual space.

[0147] 13 is a diagram showing an example of the screen configuration of a search result display screen. The search result display screen 1300 is a screen that displays data hit by a search using a query entered on the query input screen 1200, and is displayed, for example, on the output device that was displaying the query input screen 1200. Information for displaying the search result display screen 1300 is generated, for example, by the DB search unit 114.

[0148] The search result display screen 1300 includes, for example, a 3D model display area 1310, a search result display area 1320, a button 1330, and a button 1340. A 3D model of an area including a position corresponding to data included in the search results is displayed in the 3D model display area 1310. In addition, thumbnails corresponding to data included in the search results are displayed superimposed on the 3D model displayed in the 3D model display area 1310.

[0149] The thumbnails superimposed on the 3D model are displayed at the positions included in the metadata of the data corresponding to the thumbnails. Also, thumbnails corresponding to data whose metadata includes multiple positions may be displayed at each of the multiple positions of the 3D model.

[0150] The search result display area 1320 displays search results based on the search keywords and search conditions specified on the query input screen 1200. In the example of FIG. 13, thumbnails corresponding to the data, user names, registration dates and times, and edit dates and times are displayed in the search result display area 1320 as search results, but any items included in the metadata may also be displayed. Items extracted from the metadata, such as the user name, registration dates and times, and edit dates and times, may also be displayed superimposed on the 3D model in the 3D model display area 1310. In addition, in the search result display area 1320, data may be sorted and displayed based on predetermined items, for example.

[0151] In order to reduce the load of the search process and the search result display process, thumbnails are displayed on the search result display screen 1300 instead of the raw data of the search results, but the raw data of the search results may be displayed instead of or in addition to the thumbnails.

[0152] Furthermore, hierarchically related data may be displayed in the 3D model display area 1310 and / or the search result display area 1320 so that the user can recognize the data. In the example of FIG. 13 , data in a hierarchical relationship have the same type of thumbnail border ("AAA.jpg" and "CCC.text" are hierarchically related, so the thumbnail borders are shown with dotted lines, while "BBB.wav" is not hierarchically related to these data, so the thumbnail border is shown with a solid line instead of dotted lines). This allows the user to easily recognize, for example, data that should be in a hierarchical relationship but does not have a defined hierarchical relationship, or data that should not be in a hierarchical relationship but does have a defined hierarchical relationship. Furthermore, the 3D model display area 1310 and / or the search result display area 1320 may display data in a hierarchical relationship so that the user can recognize which data each data corresponds to as a parent and / or a child of.

[0153] When the data shown in the search result display area 1320 or the thumbnail displayed in the 3D model display area 1310 is selected and then button 1430 is selected, the screen transitions to a data details display screen that displays details of the selected data. When button 1440 is selected, the screen returns to the query input screen 1200, or simply the search result display screen 1300 is terminated.

[0154] Fig. 14 is a diagram showing an example of the screen configuration of a data details display screen. Fig. 14 shows an example in which "AAA.jpg" is selected as data to be displayed in detail in the search result display area 1320 of Fig. 13, and then button 1330 is selected. The data details display screen 1400 is displayed, for example, on the output device that was displaying the search result display screen 1300. Information for displaying the data details display screen 1400 is generated, for example, by the DB display unit 113.

[0155] The data details display screen 1400 includes, for example, a 3D model display area 1410, a data details display area 1420, a button 1430, a button 1440, and a button 1450. The 3D model display area 1410 displays a display similar to that of the search result display area 1320, except that of the thumbnails displayed in the 3D model display area 1310 in Fig. 13, only thumbnails of the data to be displayed in detail are displayed.

[0156] In addition to the thumbnails of the data to be displayed in detail, the 3D model display area 1410 may also display thumbnails of data that are in a hierarchical relationship with the data to be displayed in detail and / or thumbnails of data that are close to the data to be displayed in detail (for example, within a predetermined distance).

[0157] The data details display area 1420 displays more items than those displayed in the search result display area 1320 for the metadata of the data to be displayed in detail. Specifically, for example, the data details display area 1420 displays the data ID, user name, registration date and time, edit date and time, location, search tag, site name, parent data ID, and child data ID, but any item included in the metadata may also be displayed. Furthermore, the data details display area 1420 may display the data itself (raw data) to be displayed in detail.

[0158] When button 1430 is selected, a data deletion request for deleting the data to be displayed in detail is sent to the data management device 100, and the data deletion process of Fig. 10 is performed. When button 1440 is selected, a screen (not shown) for editing various items of metadata and raw data of the data to be displayed in detail is displayed, and a data editing process is sent to the data management device 100 in accordance with the editing content entered on the screen, and the data editing process of Fig. 11 is performed. When button 1440 is selected, the screen returns to the query input screen 1200 or the search result display screen 1300, or the data detail display screen 1400 is simply terminated.

[0159] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0160] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0161] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0162] 100 Data management device, 101 CPU, 102 memory, 103 auxiliary storage device, 104 communication device, 105 input device, 106 output device, 111 DB deletion unit, 112 DB editing unit, 113 DB display unit, 114 DB search unit, 121 coordinate estimation unit, 122 similar data selection unit, 123 coordinate setting unit, 125 input acquisition unit, 131 analysis unit, 132 thumbnail creation unit, 135 feature calculation unit, 151 metadata DB, 152 raw data DB, 153 thumbnail DB

Claims

1. An information processing device, a processor and a memory, the memory holds registered data and information indicating a position corresponding to the registered data; The processor: Accept a data registration request that includes the data to be registered, acquiring information indicating registered data that has a hierarchical relationship with the registration target data and that shares at least a portion of the position; Identifying registered data that is similar to the registration target data based on a predetermined algorithm; assigning a position corresponding to the data to be registered based on a position corresponding to the registered data that is in the hierarchical relationship with the data to be registered and a position corresponding to the registered data that is in the similarity relationship with the data to be registered; The information processing device stores the registration target data, the assigned position, and information indicating registered data that is in the hierarchical relationship with the registration target data in the memory.

2. 2. The information processing device according to claim 1, the hierarchical relationship includes a parent-child relationship between data; The information processing apparatus includes, in the position to be assigned to the data to be registered, all positions corresponding to registered data that is parent data in the parent-child relationship of the data to be registered.

3. 3. The information processing device according to claim 2, The processor acquires information indicating that registered data relating to the same object as the registration target data is in the parent-child relationship as information indicating registered data in the hierarchical relationship with the registration target data.

4. 3. The information processing device according to claim 2, The processor acquires information indicating that the data indicating the device that acquires the registration target data is the parent data of the registration target data in the parent-child relationship as information indicating registered data that is in the hierarchical relationship with the registration target data.

5. 3. The information processing device according to claim 2, The processor: generating new registration target data from the registration target data by performing an analysis process on the registration target data based on a predetermined algorithm; determining the new registration target data as child data in the parent-child relationship of the registration target data; All positions assigned to the registration target data are included in the positions assigned to the new registration target data, The information processing device stores in the memory the new registration target data, a position assigned to the new registration target data, and information indicating the parent-child relationship between the registration target data and the new registration target data.

6. 3. The information processing device according to claim 2, the memory holds information indicating the parent-child relationship between the registered data; The processor: receiving an editing request to edit the position of the registered data to be edited; Identifying child data in the parent-child relationship of the registered data to be edited from the registered data; an information processing device that edits the positions of the registered data to be edited and the identified child data in accordance with the editing request.

7. 3. The information processing device according to claim 2, the memory holds information indicating the parent-child relationship between the registered data; The processor: receiving an editing request to add information indicating the parent-child relationship that the first registered data is parent data in the parent-child relationship of the second registered data; Editing the information indicating the parent-child relationship in accordance with the editing request; An information processing device that adds all positions corresponding to the first registered data to positions corresponding to the second registered data.

8. 2. The information processing device according to claim 1, the memory holds information indicating a combination of the registered data in the hierarchical relationship; In the combination, at least some positions corresponding to the registered data are shared; The processor: Identifying the combination that has the similarity relationship with the registration target data based on the predetermined algorithm; The information processing device assigns a position corresponding to the data to be registered based on a position corresponding to each of the registered data included in the specified combination.

9. 9. The information processing device according to claim 8, The processor: For each combination of the registered data in the hierarchical relationship, calculate a similarity between the registration target data and each registered data included in the combination; The information processing device identifies the combination that has the similarity relationship with the registration target data based on the calculated similarity.

10. 9. The information processing device according to claim 8, The processor: For each combination of the registered data in the hierarchical relationship, data included in the combination is aggregated to generate new data; Calculating the similarity between the registration target data and the new data; The information processing device identifies the combination that has the similarity relationship with the registration target data based on the calculated similarity.

11. 2. The information processing device according to claim 1, the memory holds information indicating a combination of the registered data in the hierarchical relationship; The processor: receiving a query to search the registered data; execute a search process to search for registered data indicated by the query; an information processing device that generates and outputs information for displaying information indicating registered data included in the search results of the search processing and registered data that is in the hierarchical relationship among the registered data included in the search results.

12. The information processing device according to claim 11, the memory holds a 3D model of an area including locations corresponding to the registered data; The processor: an information processing device that generates and outputs information for displaying the 3D model and information for displaying information indicating registered data included in the search results at a position on the 3D model corresponding to the registered data.

13. An information processing method by an information processing device, the information processing device has a processor and a memory, the memory holds registered data and information indicating a position corresponding to the registered data; The information processing method includes: the processor receives a data registration request including the registration target data; the processor acquires information indicating registered data that has a hierarchical relationship with the registration target data and that shares at least a portion of a position with the registration target data; The processor identifies registered data that is similar to the registration target data based on a predetermined algorithm; the processor assigns a position corresponding to the registration target data based on a position corresponding to registered data that is in the hierarchical relationship with the registration target data and a position corresponding to registered data that is in the similarity relationship with the registration target data; The information processing method, wherein the processor stores in the memory the data to be registered, the assigned position, and information indicating registered data that is in the hierarchical relationship with the data to be registered.

Images