Data analysis device and data analysis method
The data analysis device improves analysis accuracy by grouping and tagging data with spatial information and applying predetermined rules, addressing the challenge of varying measurement environments.
Patent Information
- Application Number
- JP2023082073
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-18
- Publication Date
- 2024-11-28
AI Technical Summary
Existing data analysis technologies face challenges in obtaining accurate measurement data due to varying measurement environments, leading to reduced analysis accuracy.
A data analysis device that includes a processor and memory, which stores and analyzes data with added spatial information, groups similar data based on predetermined conditions, and applies tags to improve analysis accuracy using predetermined rules.
Enhances the accuracy of data analysis by grouping and tagging data based on spatial information and complementary data types, improving the precision of analysis results.
Smart Images

Figure 2024165682000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a data analysis device and a data analysis method. [Background technology]
[0002] Background of this technical field is JP 2017-167889 A (Patent Document 1). This publication states that "the data management device includes a knowledge graph data generation unit that generates knowledge graph data in which spatial data related to the space in a building in which maintenance equipment is installed, equipment data related to the maintenance equipment, and measurement data measuring the operating status of the maintenance equipment are associated according to ontology conversion rules, a search information input unit that inputs a search keyword including a character string representing an incident, a data search unit that searches for knowledge graph data based on the search keyword, and a search result output unit that associates and outputs spatial data, equipment data, and measurement data that match the search keyword based on the search result of the data search unit" (see abstract). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2017-167889 A Summary of the Invention [Problem to be solved by the invention]
[0004] The technology described in Patent Document 1 generates knowledge graph data that associates spatial data, equipment data, and measurement data, but depending on the measurement environment, etc., accurate measurement data may not be obtained, and the analysis accuracy of the measurement data may decrease. Therefore, one aspect of the present invention improves the analysis accuracy of the data to be analyzed. [Means for solving the problem]
[0005] In order to solve the above problem, one aspect of the present invention employs the following configuration: A data analysis device includes a processor and a memory, the memory holds a plurality of analysis target data, analysis results corresponding to each of the plurality of analysis target data, and spatial information indicating a position and / or an object related to each of the plurality of analysis target data, and the processor generates groups of analysis target data having similar spatial information based on a predetermined first condition, aligns the analysis results within each group based on a predetermined rule, assigns a tag indicating the aligned analysis result to the analysis target data included in the group, and stores knowledge information indicating the plurality of analysis target data, the tag, and the spatial information in the memory. Effect of the Invention
[0006] According to one aspect of the present invention, it is possible to improve the analysis accuracy of analysis target data.
[0007] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]
[0008] [Figure 1] 1 is a block diagram illustrating an example of a functional configuration of a data analysis device according to a first embodiment. [Diagram 2] 1 is a block diagram illustrating an example of a hardware configuration of a data analysis device according to a first embodiment. [Diagram 3] FIG. 4 is a diagram illustrating an example of a data configuration of sensor data in the first embodiment. [Figure 4] FIG. 11 is a diagram illustrating an example of a data configuration of spatial information adding method data in the first embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of a data configuration of positioning data in the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a data configuration of an analysis model DB (DataBase) in the first embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of a data configuration of a knowledge DB in the first embodiment. [Figure 8] 1 is a flowchart showing an example of a data analysis process in the first embodiment. [Figure 9] FIG. 11 is a block diagram illustrating an example of a functional configuration of a data analysis device according to a second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a screen configuration of a search screen in the second embodiment. [Figure 11] FIG. 13 is a diagram showing an example of a screen configuration of a search result display screen in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment, the same components are generally designated by the same reference numerals, and repeated explanations are omitted. Note that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention.
[0010] In this embodiment, an example will be described in which the data analysis device 100 analyzes data relating to the environment of a facility such as a large-scale factory or plant, data relating to the equipment of the facility, and data relating to the status or behavior of people in the facility, but the data to be analyzed by the data analysis device 100 is not limited to these. EXAMPLES
[0011] 1 is a block diagram showing an example of a functional configuration of a data analysis device. The data analysis device 100 includes, for example, a spatial information adding unit 102 and a data analysis unit 103, which are both functional units. The data analysis device 100 holds various information including sensor data 101, spatial information adding method data 104, positioning data 105, an analysis model DB (DataBase) 106, and a knowledge DB 107.
[0012] The sensor data 101 includes data measured by a sensor. The sensor includes a sensor that measures values related to the environment of a facility, a sensor that measures values related to equipment of the facility, and a sensor that measures values related to the state or behavior of people in the facility (e.g., workers who use the equipment or workers who maintain the equipment). Specifically, the sensor includes a camera, a microphone, a smartphone, a smart watch, a wearable sensor, an electroencephalograph, eye tracking glasses, a thermometer, a pressure gauge, an ammeter, and the like.
[0013] The types of data stored in the sensor data 101 include, for example, images, videos, voice, audio, motion, acceleration, gyro, geomagnetism, heart rate, pulse rate, electrical skin resistance, sweat rate, body temperature, brain waves, point of gaze, air temperature, humidity, pressure (voltage, oil pressure, air pressure, etc.), and current.
[0014] Furthermore, the sensor data 101 may store data other than data measured by a sensor, so long as the data is information about the environment of a facility, information about equipment in the facility, and information about the state or behavior of people in the facility. Specifically, the sensor data 101 may include, for example, a work procedure manual, an equipment manual, a work report, and a memo (all of which are documents). The data measured by a sensor and data other than data measured by a sensor, such as a work procedure manual, included in the sensor data 101, are both examples of data to be analyzed.
[0015] The spatial information adding unit 102 adds spatial information to the sensor data 101 with reference to the spatial information adding method data 104 and the positioning data 105. The spatial information includes location information (which may be two-dimensional planar coordinate values, or three-dimensional coordinate values including height, or may be the name of a room or section in a facility, or may be a mathematical formula (e.g., an inequality) indicating an area of a room or section) where the data included in the sensor data 101 was collected, and information on things related to the data (hereinafter also referred to as related things). Both the things that are the measurement target of the data and the things described in the data are examples of the related things of the data.
[0016] For example, an image included in the sensor data 101 may be assigned spatial information such as location information of a point (latitude X, longitude Y) (i.e., indicating that the image was acquired at (latitude X, longitude Y)), or may be assigned information of a related object such as transformer A (i.e., indicating that the image is an image of transformer A).
[0017] In addition, as the location information in spatial information, for example, if outdoors, location information determined by GPS (Global Positioning System) can be used, and if indoors, location information determined by PDR (Pedestrial Dead Reckoning), SLAM (Simultaneous Localization and Mapping), infrared beacons, surveillance cameras, RFID, etc., or location information determined by a combination of these can be used.
[0018] As information on related objects in spatial information, for example, the name or ID of the related object may be specified by manual user input, or an object automatically identified by image recognition technology such as generic object detection may be specified.
[0019] The spatial information adding method data 104 indicates a method by which the spatial information adding unit 102 adds spatial information to the sensor data 101. In addition, for example, the spatial information adding method data 104 indicates a positioning device that measures a position to be added to the sensor data 101, a model that detects a related object to be added to the sensor data 101, or the like. The positioning data 105 holds positioning information by a positioning device that measures a position as spatial information to be added to the sensor data 101.
[0020] The data analysis unit 103 analyzes the sensor data 101 using the analysis model indicated by the analysis model DB 106 , updates the analysis result based on the spatial information, and stores the updated analysis result in the knowledge DB 107 .
[0021] The analysis model DB 106 holds an analysis model that outputs an analysis result when data included in the sensor data 101 is input. The knowledge DB 107 holds the analysis result by the data analysis unit 103.
[0022] 2 is a block diagram showing an example of a hardware configuration of data analysis device 100. Data analysis device 100 is configured by a computer having, for example, a CPU (Central Processing Unit) 110, a memory 120, an auxiliary storage device 130, an input device 140, an output device 150, and a communication device 160.
[0023] CPU 110 includes a processor and executes programs stored in memory 120. Memory 120 includes ROM (Read Only Memory), which is a non-volatile storage element, and RAM (Random Access Memory), which is a volatile storage element. ROM stores immutable programs (e.g., BIOS (Basic Input / Output System)) and the like. RAM is a high-speed, volatile storage element such as DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by CPU 110 and data used when the programs are executed.
[0024] The auxiliary storage device 130 is a large-capacity non-volatile storage device such as a magnetic storage device (HDD (Hard Disk Drive)) or a flash memory (SSD (Solid State Drive)), and stores programs executed by the CPU 110 and data used when the programs are executed. That is, the programs are read from the auxiliary storage device 130, loaded into the memory 120, and executed by the CPU 110.
[0025] The input device 140 is a device that receives input from an operator, such as a keyboard, a mouse, a VR (Virtual Reality) controller, a game controller, etc. The output device 150 is a device that outputs the execution result of a program in a format that can be viewed by the operator, such as a display device or a printer.
[0026] The communication device 160 is a network interface device that controls communication with other devices in accordance with a predetermined protocol, and includes a serial interface such as a Universal Serial Bus (USB).
[0027] A part or all of the programs executed by CPU 110 may be provided to data analysis device 100 from a removable medium (CD-ROM, flash memory, etc.) which is a non-transitory storage medium, or from an external computer equipped with a non-transitory storage device via a network, and may be stored in non-volatile auxiliary storage device 130 which is a non-transitory storage medium. For this reason, data analysis device 100 may have an interface for reading data from removable media.
[0028] Data analysis device 100 is a computer system configured on one physical computer or on multiple logically or physically configured computers, and may operate in separate threads on the same computer, or may operate on a virtual computer constructed on multiple physical computer resources.
[0029] CPU 110 includes each functional unit included in data analysis device 100. For example, CPU 110 functions as spatial information adding unit 102 by operating according to a spatial information adding program loaded in memory 120, and functions as data analysis unit 103 by operating according to a data analysis program loaded in memory 120. The relationship between the programs and the functional units is similar in other embodiments described later.
[0030] Some or all of the functions of the functional units included in the CPU 110 may be realized by hardware such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA).
[0031] Various pieces of information held by data analysis device 100 are stored, for example, in auxiliary storage device 130. Note that some or all of the information stored in auxiliary storage device 130 may be stored in memory 120, or may be stored in a database connected to data analysis device 100.
[0032] In this embodiment, the information used by the data analysis device 100 does not depend on the data structure and may be expressed in any data structure. In this embodiment, the information is expressed in a table format, but the information can be stored in any data structure appropriately selected from, for example, a list, a database, or a queue.
[0033] 3 is a diagram showing an example of the data configuration of the sensor data 101. The sensor data 101 includes, for example, a sensor ID column 1011, a data type column 1012, a data column 1013, and a time column 1014.
[0034] The sensor ID column 1011 holds a sensor ID for identifying a sensor. The data type column 1012 holds information indicating the type of data measured by the sensor. The data column 1013 holds data measured by the sensor. The time column 1014 holds the time when the data was measured by the sensor. The time column 1014 may hold a time period.
[0035] As described above, data other than data measured by a sensor may be stored in the sensor data 101. For example, a record of the data type "document" in the sensor data 101 in Fig. 3 is not measured by a sensor, but is registered document data such as a work procedure manual.
[0036] A sensor ID is also assigned to data that is not measured by a sensor, such as document data. As a time corresponding to data that is not measured by a sensor, such as document data, the time at which the data is registered may be registered in the sensor data 101, or a time period indicating the validity period of the data (for example, the validity period of a work procedure manual) may be manually registered.
[0037] 4 is a diagram showing an example of the data configuration of the spatial information adding method data 104. The spatial information adding method data 104 holds, for example, a sensor ID column 1041, a position column 1042, and a related object column 1043.
[0038] The sensor ID column 1041 holds a sensor ID. The position column 1042 indicates a method of assigning a position as spatial information and a remark on the method of assigning the position. The method of assigning a position includes manual (i.e., input by a user via the input device 140) and the above-mentioned GPS and PDR. The remark in the position column 1042 stores, for example, a positioning device ID (information for identifying a positioning device) in the case where a position is assigned by a method other than manual, such as GPS or PDR. When the method of assigning a position is manual, the remark in the position column 1042 may be a null value.
[0039] The related thing column 1043 indicates a method of assigning a related thing as spatial information and a remark about the method of assigning the related thing. The method of assigning the related thing includes manual (i.e., input by the user via the input device 140) and the above-mentioned object detection, etc. The remark in the related thing column 1043 stores, for example, an object detection model in the case where a position is assigned by a method other than manual, such as object detection. When the method of assigning the related thing is manual, the remark in the related thing column 1043 may be a null value. Note that in the spatial information assignment method data 104, at least one of the assignment methods in the position column 1042 and the related thing column 1043 may be registered for each sensor ID.
[0040] 5 is a diagram showing an example of the data configuration of the positioning data 105. The positioning data 105 holds, for example, a positioning device ID 1051, a position column 1052, and a time column 1053. The positioning device ID column 1051 holds a positioning device ID. The position column 1052 holds information indicating a position measured by the positioning device. The time column 1053 holds information indicating a time when the position was measured by the positioning device.
[0041] 6 is a diagram showing an example of the data configuration of the analytical model DB 106. In the analytical model DB 106, analytical models corresponding to data types and analytical models corresponding to data types and spatial information (positions and / or related matters) are defined. The analytical model DB 106 includes, for example, a model ID column 1061, a data type column 1062, a position column 1063, a related matter column 1064, and an analytical model column 1065.
[0042] The model ID column 1061 holds a model ID for identifying an analytical model. The data type column 1062 holds information indicating a data type. The position column 1063 holds information indicating a position. The related object column 1064 holds information indicating a related object.
[0043] The analytical model column 1065 holds a trained analytical model. Specifically, for example, the analytical model column 1065 stores a method for extracting a feature amount as an explanatory variable from the data in the corresponding data type column 1062, and an analytical model that outputs information indicating an analysis result when the feature amount is input as an explanatory variable. The trained analytical model stored in the analytical model DB 106 is, for example, a model trained in advance by machine learning using training data of the corresponding data type.
[0044] 7 is a diagram showing an example of the data configuration of the knowledge DB 107. The knowledge DB 107 includes, for example, a sensor ID column 1071, a data type column 1072, a data column 1073, a tag column 1074, a feature amount column 1075, a position column 1076, a related object column 1077, and a time column 1078.
[0045] The sensor ID column 1071 holds a sensor ID. The data type column 1072 holds information indicating a data type. The data column 1073 holds the sensor ID and data of the data type.
[0046] The tag field 1074 holds tags based on the analysis results output from the analysis model. The name of an object shown in an image is an example of a tag to be added to image data, a keyword included in voice data is an example of a tag to be added to voice data, and a document type is an example of a tag to be added to document data. Note that tags do not need to be highly readable, and may be, for example, features of a high-dimensional vector obtained by feature extraction processing.
[0047] The feature amount column 1075 holds feature amounts extracted by the data analysis unit 103. The position column 1076 holds information indicating a position added by the spatial information adding unit 102. The related object column 1077 holds information indicating a related object added by the spatial information adding unit 102. The time column 1078 holds information indicating a time corresponding to the data in the data column 1073.
[0048] Fig. 8 is a flowchart showing an example of the data analysis process. For example, before the process of Fig. 8 starts, information in the sensor ID column 1011 and the data type column 1012 of the sensor data 101, the spatial information adding method data 104, and the positioning device ID column 1051 of the positioning data 105 are registered in advance.
[0049] The spatial information adding unit 102 acquires a sensor ID, data, and time (e.g., a timestamp of a sensor value) transmitted from a sensor connected to the data analysis device 100 or an external device connected to the data analysis device 100, or input to the input device 140, and stores the acquired data and time in the sensor data 101 (S801). Note that the spatial information adding unit 102 may store in the time column 1014 the time at which it acquires data from the sensor, the external device, or the input device (e.g., the time at which sensor information is uploaded).
[0050] The spatial information adding unit 102 refers to the spatial information adding method data 104 and the positioning data 105 to add spatial information to the sensor data 101 (S802). Specifically, for example, the spatial information adding unit 102 acquires a sensor ID for each record of the sensor data 101, identifies an adding method and notes for the position and related things corresponding to the sensor ID from the spatial information adding method data 104, and adds information on the position and / or related things to the record according to the identified adding method and notes.
[0051] In the examples of Figures 3 and 4, the spatial information assignment unit 102 assigns information on the position and related matters input "manually" via the input device 140 to a record in the sensor data 101 having a sensor ID of "001".
[0052] Also, for example, for a record with a sensor ID of "002" in the sensor data 101, the spatial information adding unit 102 acquires and adds information on a position measured by a positioning device with a positioning device ID of "AAA" from the positioning data 105. In this case, it is preferable that the spatial information adding unit 102 adds information on a position that is closest to the value in the time column 1053 of the time column 1014 with the sensor ID of "002" among the information on the position measured by the positioning device with the positioning device ID of "AAA" indicated in the positioning data 105.
[0053] Also, for example, for a record with a sensor ID of "003" in the sensor data 101, the spatial information assignment unit 102 assigns position information that is "manually" input via the input device 140, and assigns information on related objects (for example, the name of the detected related object) output by "object detection" using the data of the record and "object recognition model X."
[0054] The data analysis unit 103 selects an analysis model corresponding to the data type and the assigned space type from the analysis model DB 106 for each record of the sensor data 101 to which the spatial information was assigned in step S102 (S103). Note that, in the example of the analysis model DB 106 in Fig. 6, an analysis model corresponding to a combination of the data type and the space type is defined, but an analysis model corresponding to only the data type may be defined. In this case, the data analysis unit 103 selects an analysis model corresponding to the data type from the analysis model DB 106 for each record of the sensor data 101 to which the spatial information was assigned in step S102.
[0055] The data analysis unit 103 extracts features as explanatory variables indicated by the analysis model selected in step S103 from the data indicated by each record of the sensor data 101 to which spatial information was added in step S102, and inputs the extracted features into the analysis model to obtain analysis results (S104).
[0056] The data analysis unit 103 groups the sensor data 101 having similar spatial information assigned in step S102 based on a predetermined condition (S105). Specifically, for example, the data analysis unit 103 groups records having similar assigned positions in the sensor data 101 (for example, records having assigned position coordinates within a circle of a predetermined radius, or records having assigned positions indicating the same room or section), or groups records having assigned related matters that are the same or similar (for example, which related matters are similar is determined in advance).
[0057] The data analysis unit 103 may group only records in which the times in the sensor data 101 are close based on a predetermined condition (for example, records whose time difference is within a predetermined value) based on the spatial information. In this way, the data analysis unit 103 can group data more accurately by utilizing the time information, and can thereby improve the accuracy of tagging, which will be described later. Furthermore, records that are not grouped in step S105 are treated as a group consisting of only a single record in step S106, which will be described later.
[0058] The data analysis unit 103 generates a tag for each group created in step S105 based on the analysis result obtained in step S104, and assigns the tag to each record included in the group (S106). Specifically, for example, the data analysis unit 103 assigns the analysis result obtained in step S104 as a tag to a group consisting of only a single record.
[0059] The data analysis unit 103 aligns the analysis results in a group including a plurality of records based on a predetermined rule, and assigns a tag indicating the aligned analysis result. An example of the predetermined rule for assigning a tag will be described below. Note that the rule for assigning a tag is predefined in accordance with the data type of the records included in the group, the combination of the data type of the records included in the group and the analysis model used to analyze the records, the combination of the data type of the records included in the group and spatial information, etc. Note that for a group for which no corresponding rule exists, the data analysis unit 103 assigns the analysis result acquired in step S104 as a tag to each of the plurality of records included in the group.
[0060] For example, there is a rule that data analysis unit 103 assigns tags to a group including a plurality of records based on a predetermined statistic for the analysis results corresponding to each of the plurality of records acquired in step S104.
[0061] An example of the first rule based on a predetermined statistical quantity will be described. For example, assume that the first group contains three records, and A is the analysis result obtained for the first record of the three records, B is the analysis result obtained for the second record, and A is the analysis result obtained for the third record. In this case, A, which is the analysis result that appears most frequently in the first group, is assigned as a tag to each of the three records.
[0062] An example of the second rule based on a predetermined statistical quantity will be described. For example, assume that the second group includes three records, the data types of the three records are all images (hereinafter, also referred to as image 1, image 2, and image 3, respectively), and all three images are input to an object recognition model, which is an analysis model. Also assume that an image is input to the object recognition model, and the probability that an object included in the image is object A, the probability that it is object B, and the probability that it is object C are output as analysis results.
[0063] Furthermore, the analysis results for image 1 by the object recognition model are output as follows: probability that the object contained in image 1 is object A, 0.4; probability that it is object B, 0.5; and probability that it is object C, and the analysis results for image 2 by the object recognition model are output as follows: probability that the object contained in image 2 is object A, 0.6; probability that it is object B, 0.3; and probability that it is object C, and the analysis results for image 3 by the object recognition model are output as follows: probability that the object contained in image 3 is object A, 0.5; probability that it is object B, 0.1; and probability that it is object C,
[0064] In this case, the data analysis unit 103 averages the probabilities indicated by the analysis results for image 1, image 2, and image 3, for example. That is, the data analysis unit 103 obtains an average probability of 0.5 for the object being object A, an average probability of 0.4 for the object being object B, and an average probability of 0.1 for the object being object C. The data analysis unit 103 assigns object B, which has the highest average probability, as a tag for each of the three records in the second group. The data analysis unit 103 may also assign an average probability of 0.5 for the object being object A, an average probability of 0.4 for the object being object B, and an average probability of 0.1 for the object being object C, as tags for each of the three records in the second group.
[0065] In the second example, the object contained in image 1 is most likely to be object B, but when information from images 2 and 3, which are images of the same or a nearby space as image 1, is also taken into account, the analysis results can be corrected to indicate that the object contained in image 1 is object A, and a tag can then be assigned.
[0066] An example of the third rule will be described. For example, it is assumed that the third group includes two records, and the data type of the first record of the two records is an image and the related object is device A, and the data type of the second record is audio and the related object is device A. Furthermore, it is assumed that the image in the first record is input to an object recognition model, which is an analysis model, and the probability that an abnormality occurs in device A and the probability that an abnormality does not occur are output as analysis results. It is also assumed that the audio in the second record is input to an abnormal sound detection model, which is an analysis model, and the probability that an abnormality occurs in device A and the probability that an abnormality does not occur are output as analysis results.
[0067] In this case, the data analysis unit 103 assigns, for example, the analysis result corresponding to the higher of the average probability that an abnormality has occurred in device A and the average probability that an abnormality has not occurred in the analysis results of the first record and the second record (i.e., either an abnormality has occurred or no abnormality has occurred) as a tag to the records of the third group.
[0068] For example, if the space corresponding to the third group becomes dark or the camera capturing the image has a blind spot, the analysis accuracy for the image decreases but the analysis accuracy for the audio does not decrease, and if the amount of sound output from sources other than device A in the space increases, the analysis accuracy for the audio decreases but the analysis accuracy for the image does not decrease. Thus, in the third example, by using different types of data such as images and audio (i.e., multimodal data), tags can be assigned more accurately based on complementary information.
[0069] In addition to the rules based on predetermined statistics, there is a rule that the data analysis unit 103 assigns a tag to a group containing multiple records after prioritizing the analysis result corresponding to any one of the multiple records obtained in step S104 and aligning the analysis results.
[0070] A fourth rule, which is an example of such a rule, will be described. It is assumed that the fourth group includes two records, and the data type of the first record of the two records is (worker) acceleration, and the data type of the second record is document (work procedure manual). It is further assumed that the acceleration in the first record is input to a behavior recognition model, which is an analysis model, and the probability that the work being performed by the worker is work A, the probability that it is work B, and the probability that it is work C are output as analysis results. It is also assumed that the document in the second record is input to a document analysis model, which is an analysis model, and the work (the name of the work) described in the document is output as analysis results.
[0071] Furthermore, as the analysis results for the acceleration of the first record, the probability that the work being performed by the worker is work A is 0.4, the probability that it is work B is 0.5, and the probability that it is work C is 0.1 are output, and as the analysis results for the document of the second record, work A and work C are output.
[0072] In this case, the analysis result for the acceleration of the first record indicates a high probability that task B is being performed, but the data analysis unit 103 prioritizes the analysis result for the document of the second record and excludes task B from the analysis result for the first record. That is, for example, the data analysis unit 103 compares the probability that the task being performed by the worker is task A with the probability that it is task C, and assigns task A, which has a higher probability, as a tag to the first record. Furthermore, the data analysis unit 103 may assign task C, which was included in the analysis result, as a tag to the second record, in addition to task A.
[0073] In this manner, also in the fourth example, by using different types of data such as acceleration and documents (i.e., multimodal data), tags can be assigned more accurately based on complementary information.
[0074] Next, the data analysis unit 103 registers various pieces of information in the knowledge DB 107 (S107), and ends the data analysis process of FIG.
[0075] In this embodiment, an example has been described in which the data analysis device 100 automatically assigns tags to each group in step S106. However, for example, the data analysis device 100 may assign tags by displaying the grouped sensor data 101 to which spatial information has been assigned and the analysis results corresponding to the records of each group on the output device 150, and having the user input the tag into the input device 140 after checking the display.
[0076] In this embodiment, an example has been described in which the sensor data 101 and the positioning data 105 are assigned with time stamps, but these data may not be assigned with time stamps. In this case, the time stamps are not taken into consideration when grouping the data in step S105.
[0077] In addition, the data analysis unit 103 may re-learn the network parameters in the neural network using contrastive learning or the like so that, among the data analyzed using the same neural network (an example of an analysis model), the similarity between the features extracted from the data of records classified into the same group in step S105 is high, and the similarity between the features extracted from the data of records classified into different groups is low.
[0078] The data analysis unit 103 may generate a group ID for identifying the group generated in step S105, and store the group ID in the knowledge DB 107. This allows the user to easily understand which data has been grouped by the spatial information.
[0079] As described above, the data analysis device 100 of this embodiment can improve the accuracy of tagging by adding spatial information to the sensor data 101, analyzing each piece of data included in the sensor data 101, grouping the data included in the sensor data 101 based on the spatial information, and adding a tag indicating the analysis result for the data to each group. The data analysis device 100 can further improve the accuracy of tagging by further using time information in grouping the data. Moreover, the data analysis device 100 can more accurately add tags based on complementary information by grouping multimodal data and adding tags to the data. EXAMPLES
[0080] Hereinafter, the description of the contents common to the first embodiment will be omitted as appropriate, and the differences from the first embodiment will be mainly described.
[0081] 9 is a block diagram showing an example of a functional configuration of the data analysis device 100. The data analysis device 100 of this embodiment further includes a search unit 201, a screen data generation unit 202, an area calculation unit 204, and an intention estimation unit 205, all of which are functional units. In addition, the various information held by the data analysis device 100 further includes 3D model data 203.
[0082] The search unit 201 searches the knowledge DB 107 according to information input to the input device 140. The screen data generation unit 202 generates screen data for a search screen, a search result display screen, and a 3D model, and displays them on the output device 150.
[0083] The 3D model data 203 is a 3D model that represents the interior of the facility from which data is collected. If the facility from which data is collected is a factory, a CAD (Computer Aided Design) model of the factory is an example of the 3D model data 203. If the facility from which data is collected is a construction site, a 3D model created by performing point cloud measurement of the site is an example of the 3D model data 203.
[0084] The 3D model data 203 may be any of various types of data, such as CAD data converted into 3D CG (Computer Graphics), point cloud data, photogrammetry, etc. Instead of the 3D model data 203, 2D data, such as an image captured by a 360-degree camera, may be used.
[0085] The 3D model data 203 indicates spatial information consistent with the spatial information assigned by the spatial information assigning unit 102. Specifically, for example, the 3D model data 203 has the same coordinate system as the position of the positioning data 105 and a position that can be input by the user, and includes names and IDs of related things that can be automatically assigned by the spatial information assigning unit 102 and related things that can be input by the user. This allows the screen data generating unit 202 to display where the data assigned with spatial information is located on the 3D model, which related thing it is linked to, and so on.
[0086] The area calculation unit 204 calculates a display range of the 3D model data 203 to be displayed on the output device 150 according to an instruction from the user input to the input device 140, and outputs the calculation result to the screen data generation unit 202. The area calculation unit 204 may also calculate the display range by performing voice recognition and gesture recognition based on voice input or gesture input to the input device 140. The area calculation unit 204 may also estimate the motion of the user's head and the position of the gaze point, and calculate the display range based on the estimated information.
[0087] When a search condition using an abstract expression is specified, the intention estimation unit 205 performs a search that reflects the user's intention estimated from the expression.
[0088] 10 is a diagram showing an example of a screen configuration of a search screen. A search screen 1000 includes, for example, a search keyword input area 1010 and a search condition setting area 1020. When a search request is received by inputting a search keyword in the search keyword input area 1010 and / or setting a search condition in the search condition setting area 1020, the search unit 201 searches the knowledge DB 107 according to the search keyword and / or the search condition.
[0089] The search condition setting area 1020 includes, for example, a search type setting area 1021 , a time setting area 1022 , a data type setting area 1023 , a location setting area 1024 , a related object setting area 1025 , and a free input area 1026 .
[0090] The search type setting area 1021 is an area for setting a search range by, for example, a search keyword. When "content" is selected in the search type setting area 1021, data including the search keyword in the content (for example, a sentence in a document file, a file name of each file, etc.) is searched for in the knowledge DB 107.
[0091] When "tag" is selected in the search type setting area 1021, for example, data whose tags contain the search keyword is searched for in the knowledge DB 107. When "full text" is selected in the search type setting area 1021, for example, data whose contents and / or tags contain the search keyword is searched for in the knowledge DB 107.
[0092] The time setting area 1022 is an area for setting search conditions related to the time of data. The data type setting area 1023 is an area for setting search conditions related to the data type.
[0093] The position setting area 1024 is an area for setting search conditions related to the position of data. In the example of Fig. 10, a center and a radius can be set in the position setting area 1024, and data indicating a position within a circle determined by the set center and radius can be searched for in the knowledge DB 107. Note that, in the position setting area 1024, a position may be set in an area of a shape other than a circle, and the name of a room or a section may also be set.
[0094] The related thing setting area 1025 is an area for setting search conditions related to related things of data. In the example of Fig. 10, the related thing setting area 1025 displays boxes in which related things included in the knowledge DB 107 can be selected.
[0095] In the above example, search conditions for spatial information including position and related objects are set in the position setting area 1024 and the related object setting area 1025. However, for example, the area calculation unit 204 may calculate a 3D model of the area specified by the user, the screen data generation unit 202 may display the 3D model of the area on the output device 150, and the user may select a position or related object on the 3D model displayed on the output device 150 by input, thereby setting search conditions for spatial information.
[0096] The free input area 1026 is an area for the user to freely input text indicating a search condition. The intention estimation unit 205 performs a search that reflects the user's intention in the text input in the free input area 1026. For example, when the text "image data taken under a cooling device" is input in the free input area 1026, the intention estimation unit 205 searches the knowledge DB 107 for image data whose (x, y) coordinates are within a predetermined distance from the (x, y) coordinates of the cooling device and whose z coordinates are smaller than the z coordinate of the cooling device.
[0097] Also, for example, when the text “report created after task A” is entered in the free input field 1026, the intention estimation unit 205 searches the knowledge DB 107 for only document data that is later in time than the data corresponding to the tag “task A.”
[0098] The intention estimation unit 205 may estimate the intention from the text based on rules created in advance, or based on a language processing algorithm such as question answering.
[0099] Furthermore, when a search result by the intention estimation unit 205 is selected by the user on a search result display screen described later, the search result may be regarded as the data desired by the user (correct data), and information regarding the pair of the text (query) entered in the free input field 1026 and the correct data may be accumulated, and data that should be returned for a specific query may be learned using machine learning techniques.
[0100] Furthermore, the intention estimation unit 205 may output a question asking about the intention when it is determined that the intention input in the free input area 1026 is unclear. For example, when the text "image data near the cooling device" is input in the free input area 1026, the intention estimation unit 205 may output a question asking about the intention such as "Does "near the cooling device" mean within a 5 m radius of the cooling device?"
[0101] 11 is a diagram showing an example of the screen configuration of a search result display screen. The search result display screen 1100 includes, for example, a 3D model display area 1110 and a search result display area 1120. The 3D model display area 1110 displays a 3D model (extracted from the 3D model data 203) of an area including a position and related objects corresponding to the data included in the search result. In addition, the position and related objects corresponding to the data included in the search result are also displayed on the 3D model displayed in the 3D model display area 1110 (the positions and related objects are indicated by the numbers "1" and "2" in the 3D model display area 1110 in FIG. 11).
[0102] Search result display area 1120 displays search results based on the search keywords and search conditions specified on search screen 1000. In the example of Fig. 11, data, data type, time, tag, location, and related things are displayed as search results in search result display area 1120, but any item included in knowledge DB 1107 may be displayed. Also, for example, a group ID may be displayed in search result display area 1120.
[0103] Additionally, items extracted from the knowledge DB 1107, such as data, data type, time, tag, location, and related things, may also be displayed superimposed on the 3D model in the 3D model display area 1110.
[0104] When data displayed in the search result display area 1120 is selected, the data is displayed.
[0105] Furthermore, in the search result display area 1120, for example, the data may be arranged in chronological order, or the data may be sorted by data type and then arranged in chronological order. For example, the search unit 201 may arrange the image data obtained as the search results in chronological order and apply an image analysis method such as change detection or change captioning, thereby detecting or describing major changes that have occurred at a point specified as a search condition and presenting them to the user.
[0106] Furthermore, for example, data may be sorted by related matters and then arranged in chronological order in the search result display area 1120. For example, the search unit 201 may sort document data obtained as a search result by related matters, arrange them in chronological order, and apply a document analysis method such as timeline summarization, thereby making it possible to present to the user changes that have occurred in related matters specified as search conditions.
[0107] In addition to the search results based on the search keywords and search conditions specified on the search screen 1000, the search unit 201 may also display on the search result display screen 1100 data included in the search results and data that was determined in step S105 to belong to the same group and / or has the same tag.
[0108] As a result, for example, if a user wants to check whether an accident has occurred in a cooling device using image data, but the image is taken in a dark space and the user has difficulty understanding the accident situation by looking at the image data, the voice data belonging to the same group as the image data is also displayed, allowing the user to understand the accident situation from the voice data, and so on, so that information that is useful to the user and that the user is not aware of can be presented. In particular, by displaying data that is different from the data type specified by the user and that is determined to belong to the same group and / or has the same tag on the search result display screen 1100, complementary multimodal data can be presented to the user.
[0109] Furthermore, for example, the search unit 201 may display, in addition to the search results based on the search keywords and search conditions specified on the search screen 1000, data that would be included in the search results if the position range set in the position setting area 1024 is expanded by a predetermined distance, or data that would be included in the search results if the time range set in the time setting area 1022 is expanded by a predetermined time (for example, the start time is advanced by a predetermined time and / or the end time is delayed by a predetermined time) on the search result display screen 1100. This makes it possible to present information that is useful to the user but that the user has not noticed.
[0110] Note that a plurality of data analysis devices 100 may be connected via a network such as the Internet, and in this case, the plurality of data analysis devices 100 may share and display search results with each other. Specifically, for example, the screen data generating unit 202 of each of the plurality of data analysis devices 100 displays, on its own search result display screen 1100, search results by other data analysis devices 100 that are displaying search results in which the area of the 3D model displayed in the 3D model display area 1110 is within a predetermined distance (the spatial information (area including the position and / related object) of the data displayed in the search result display area 1120 is close based on a predetermined condition).
[0111] This allows each user of the multiple data analysis devices 100 to share what information (tags, related things, etc.) of a nearby area on the 3D model other users are focusing on in that area. Furthermore, if discussions by chat, phone calls, etc. can be held between the users of the multiple data analysis devices 100, the users can share search results and learn about the information that each user places importance on, which can facilitate consensus building in discussions.
[0112] The present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those having all the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.
[0113] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.
[0114] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]
[0115] 100 Data analysis device, 101 Sensor data, 102 Spatial information assignment unit, 103 Data, 104 Spatial information assignment method data, 105 Positioning data, 106 Analysis model DB, 107 Knowledge DB, 110 CPU, 120 Memory, 130 Auxiliary storage device, 140 Input device, 150 Output device, 160 Communication device, 201 Search unit, 202 Screen data generation unit, 203 3D model data, 204 Area calculation unit, 205 Intention estimation unit, 1000 Search screen, 1100 Search result display screen
Claims
1. 1. A data analysis device, comprising: A processor and a memory, The memory includes: Multiple analysis target data, Analysis results corresponding to each of the plurality of analysis target data; and spatial information indicating a position and / or an object associated with each of the plurality of analysis target data; The processor, generating a group of analysis target data having similar spatial information based on a predetermined first condition; for each group, aligning the analysis results within the group based on a predetermined rule, and assigning a tag indicating the aligned analysis result to the analysis target data included in the group; The data analysis device stores knowledge information indicating the plurality of analysis target data, the tag, and the spatial information in the memory.
2. The data analysis device according to claim 1 , The processor, For a first group that is included in the generated group and is made up of analysis target data of the same data type, The data analysis apparatus aligns the analysis results within the first group based on a predetermined statistical quantity for the analysis result corresponding to the analysis target data.
3. The data analysis device according to claim 2, The first group is made up of analysis target data whose data type is an image, The analysis results corresponding to the image indicate the probability that each object is included in the image, The processor, Calculating an average of the probability that each object is included in the images included in the first group as the predetermined statistical quantity; determining an object included in the image included in the first group based on the calculated average value; A data analysis device that assigns a tag indicating the determined object to the analysis target data included in the first group.
4. The data analysis device according to claim 1 , The processor, For a second group included in the generated group and including a plurality of data types of analysis target data, a data analysis device that prioritizes analysis results corresponding to analysis target data of a predetermined data type included in the plurality of data types and aligns the analysis results within the second group.
5. The data analysis device according to claim 4, the second group includes analysis target data whose data type is a document and analysis target data whose data type is acceleration; the predetermined data type is a document, The analysis results corresponding to the acceleration indicate the probability that each task was performed by the worker whose acceleration was measured, The analysis results corresponding to the document indicate the work described in the document, The processor, A data analysis device that assigns a tag to the analysis target data included in the second group, the tag indicating the task described in the document that has the highest probability in the analysis result corresponding to the acceleration, for the second group.
6. The data analysis device according to claim 1 , the memory holds time information corresponding to each of the plurality of analysis target data; The processor generates groups of analysis target data in which the spatial information is similar based on the predetermined first condition and the time information is close based on a predetermined second condition.
7. The data analysis device according to claim 1 , connected to an input device and a display device; The processor, receiving a search request for searching for analysis target data included in the knowledge information via an input to the input device; The data analysis device displays, on the display device, analysis target data that matches the search request and analysis target data that belongs to the same group as the analysis target data as a search result.
8. The data analysis device according to claim 7, maintaining a 3D model of the location including the positions and / or objects represented by the spatial information; The processor, calculating an area including a position and / or an object corresponding to the analysis target data included in the search result by referring to the 3D model; Displaying a 3D model of the calculated area on the display device; A data analysis device that displays the search results on a 3D model of the calculated area.
9. The data analysis device according to claim 7, connected to another data analysis device, The processor, A data analysis device that displays on the display device search results determined based on a predetermined third condition to be close to an area including a position and / or object related to the analysis target data included in the search results by the other data analysis device, the area including a position and / or object related to the analysis target data included in the search results by the processor.
10. A data analysis method using a data analysis device, The data analysis device includes a processor and a memory. The memory includes: Multiple analysis target data, Analysis results corresponding to each of the plurality of analysis target data; and spatial information indicating a position and / or an object associated with each of the plurality of analysis target data; The data analysis method includes: The processor generates a group of analysis target data having similar spatial information based on a predetermined first condition, the processor aligns the analysis results within each group based on a predetermined rule, and assigns a tag indicating the aligned analysis result to the analysis target data included in the group; The data analysis method, wherein the processor stores knowledge information indicating the plurality of analysis target data, the tag, and the spatial information in the memory.
Citation Information
Patent Citations
Data management device, data management system, and data management method
JP2017167889A
