Information processing device, information processing method, and information processing program
The information processing device addresses the challenge of associating elements across different data types by generating and linking element information, allowing for easy confirmation and management of related elements within diverse data sets.
Patent Information
- Application Number
- JP2021127611
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-08-03
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-08-03
AI Technical Summary
Conventional systems fail to associate elements across multiple types of target data effectively, making it difficult to provide a group of related elements in an easily confirmable manner.
An information processing device that includes an acquisition section to gather target data, a generation section to create element information for each element in the data, and an association section to link similar or related element information across different types of data, enabling easy confirmation of related elements.
Facilitates easy verification of related elements across multiple types of data, enhancing the ability to manage and utilize interconnected information from diverse data sources.
Smart Images

Figure 0007581148000001 
Figure 0007581148000002 
Figure 0007581148000003
Abstract
Description
[Technical field]
[0001] An embodiment of the present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]
[0002] Systems for managing various types of target data are known. For example, a system is known that displays various target data such as drawing data and image data stored in a storage unit on a display unit. Also, for example, a system is disclosed that displays image data selected by an operation instruction by a user in a superimposed manner on the displayed drawing data at a specific position designated by the user (see, for example, Patent Document 1).
[0003] However, in conventional technology, there is no association between elements contained in multiple target data of different types, making it difficult to provide an easily identifiable group of related elements contained in multiple target data. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2005-310044 A Summary of the Invention [Problem to be solved by the invention]
[0005] The present invention has been made in consideration of the above, and has an object to provide a group of related elements contained in multiple target data in an easily identifiable manner. [Means for solving the problem]
[0006] According to an embodiment, an information processing apparatus includes an acquisition unit, a generation unit, and an association unit. Target data relating to equipment, A plurality of data sets having different data formats, data types, data structures, and data contents. The above The target data is acquired. The generating unit generates a target data according to the type of the target data. , image processing, drawing analysis processing, and document analysis processing By carrying out the analysis process, the target data The equipment specified in For each element, the element in the target data About Information and includes information representing the contents of at least one of the following items: a storage location of the target data to which the element belongs, a name of the target data to which the element belongs, a type of the target data to which the element belongs, a position of the element in the target data, a feature amount of the element, identification information of the element, a type of the element, an estimated position of the element in real space, a creation date and time of the target data to which the element belongs, and a reference destination related to the element. Generate element information. The association unit: For each of the element information, other element information in which the information expressing the content of one of the at least one items included in the element information is the same or similar is selected as the related element information. The data are stored in association with each other in a storage unit. [Brief description of the drawings]
[0007] [Figure 1] FIG. 1 is a schematic diagram of an information processing system. [Figure 2A] FIG. [Figure 2B] FIG. [Figure 2C] FIG. [Figure 2D] FIG. [Figure 2E] FIG. [Figure 2F] FIG. [Diagram 3] FIG. 4 is a diagram showing a data structure of element information. [Figure 4] FIG. 4 is a schematic diagram showing the data structure of a management DB. [Diagram 5] FIG. [Figure 6] FIG. [Figure 7] FIG. [Figure 8] FIG. 13 is a schematic diagram showing a display screen for anomaly detection results. [Figure 9] FIG. 4 is a schematic diagram showing a data structure of incident information. [Figure 10] FIG. 4 is a schematic diagram showing the data structure of a management DB. [Figure 11] 11 is a flowchart showing the flow of an association process. [Figure 12] 10 is a flowchart showing the flow of a display process on a display screen. [Figure 13] 11 is a flowchart showing the flow of an anomaly detection process. [Figure 14]FIG. 2 is a hardware configuration diagram of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] Hereinafter, embodiments of an information processing device, an information processing method, and an information processing program will be described in detail with reference to the accompanying drawings.
[0009] FIG. 1 is a schematic diagram showing an example of an information processing system 1 according to the present embodiment.
[0010] The information processing system 1 includes an information processing device 10, an input unit 12, a display unit 14, and a communication unit 16. The information processing device 10, the input unit 12, the display unit 14, and the communication unit 16 are communicatively connected via a bus 18 or the like.
[0011] The input unit 12 receives various operations from a user and is, for example, a keyboard, a pointing device such as a mouse, a microphone, and the like.
[0012] The display unit 14 displays various types of information. The display unit 14 is a display. The input unit 12 and the display unit 14 may be integrated into a touch panel.
[0013] The communication unit 16 is a communication interface that communicates with other information processing devices other than the information processing device 10 via a network or the like.
[0014] The information processing device 10 includes a control unit 20 and a storage unit 22. The control unit 20 and the storage unit 22 are communicatively connected via a bus 18 or the like. The information processing device 10 may include the control unit 20, the storage unit 22, and at least one of the input unit 12, the display unit 14, and the communication unit 16. The control unit 20 and the storage unit 22 may be configured separately and communicatively connected via a network or wirelessly.
[0015] The storage unit 22 stores various data. The storage unit 22 is, for example, a semiconductor memory element such as a RAM or a flash memory, a hard disk, an optical disk, etc. Note that the storage unit 22 may be a storage device provided outside the information processing device 10.
[0016] In this embodiment, the storage unit 22 stores target data 30 and a management DB 40.
[0017] The target data 30 is data that is to be subjected to processing, which will be described later, in the information processing device 10. The storage unit 22 stores a plurality of target data 30 of different types.
[0018] A plurality of target data 30 of different types means that the target data 30 differs in at least one of the data format, data type, data structure, and data content.
[0019] A data format is a method of recording data, and may also be referred to as a file format. For example, when the target data 30 is image data, the data format may include JPG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), PNG (Portable Network Graphics), BMP (Microsoft Windows (registered trademark) Bitmap Image), etc. When the target data 30 is a document file, the data format may include document format, text format, etc. When the target data 30 is a video file, the data format may include mp4 (MPEG-4), avc (Advanced Video Coding), wmv (Windows (registered trademark) Media Video), etc. When the target data 30 is a CAD drawing, the data format may include dgw, dxf (Drawing Exchange Format), SVG (Scalable Vector Graphics), etc.
[0020] The data type corresponds to a label of each group of data classified according to a common property based on a predetermined classification criterion. The data type is, for example, a document, an image, a video, an audio, etc. The data type may be further classified. For example, an image, which is an example of a data type, may be further classified into design drawings, photographed images, etc.
[0021] The data structure refers to the format in which data is stored. For example, the data structure may be structured data such as statistical data, or unstructured data such as images. Note that the data structure may be further classified.
[0022] The data content means the content indicated by the data. For example, when the target data 30 is image data, even if the image data is of the same target, if at least one of the shooting angle of view, shooting direction, shooting date and time, etc. is different, the data content is different. Also, for example, when the target data 30 is data representing the inspection result, if at least one of the object such as the equipment to be inspected, the inspection conditions, etc. is different, the data content is different.
[0023] 2A to 2F are diagrams showing an example of target data 30. In this embodiment, a case where the information processing device 10 uses a plurality of types of target data 30 related to inspection will be described as an example. Also, a case where the inspection target is equipment installed in a facility will be described as an example. Note that the target data 30 to be processed by the information processing device 10 is not limited to the target data 30 related to inspection.
[0024] 2A is a schematic diagram showing an example of the data configuration of the inspection date and time list 30A. The inspection date and time list 30A is an example of the target data 30. The inspection date and time list 30A is, for example, document data in which the inspection date and time are associated with the inspection content. The inspection content includes, for example, monthly inspection and annual inspection.
[0025] 2B is a schematic diagram showing an example of the data configuration of the inspection work instruction 30B. The inspection work instruction 30B is an example of the target data 30. The inspection work instruction 30B is, for example, a table that associates an inspection item with the installation location of the equipment that is the inspection item and the inspection content.
[0026] 2C is a schematic diagram showing an example of a drawing 30C of the facility 1. The drawing 30C of the facility 1 is an example of the target data 30. The drawing 30C of the facility 1 is an example of drawing data of the facility 1 including parts such as part A, part B, and part C.
[0027] 2D is a schematic diagram showing an example of the data configuration of an inspection report 30D. The inspection report 30D is an example of target data 30. The inspection report 30D is an example of document data representing a report that associates inspection items with inspection results.
[0028] 2E is a schematic diagram showing an example of a facility floor plan 30E. The facility floor plan 30E is an example of target data 30. The facility floor plan 30E is an example of floor plan data of a facility in which equipment 1 is installed. The facility floor plan 30E shows, for example, the positions of each of equipment A to equipment D within the facility.
[0029] 2F is a schematic diagram showing an example of an inspection image 30F. The inspection image 30F is an example of the target data 30. In this embodiment, an example is described in which the inspection image 30F is an image captured during the inspection work of each of the facilities A to D. In detail, in this embodiment, an example is described in which an inspection image 30FA of the facility A, an inspection image 30FB of the facility B, an inspection image 30FC of the facility C, and an inspection image 30FD of the facility D are used as the inspection image 30F.
[0030] 2A to 2F, in this embodiment, a plurality of types of target data 30 are stored in the storage unit 22. The target data 30 may be stored in the storage unit 22 by the control unit 20 described later, or may be stored in the storage unit 22 in advance.
[0031] Returning to Figure 1, we will continue with the explanation.
[0032] The control unit 20 executes various information processes in the information processing device 10. The control unit 20 includes an acquisition unit 24A, a generation unit 24B, an association unit 24C, a reception unit 24D, a display control unit 24E, an anomaly detection unit 24F, and a storage control unit 24G.
[0033] The acquisition unit 24A, the generation unit 24B, the association unit 24C, the reception unit 24D, the display control unit 24E, the anomaly detection unit 24F, and the storage control unit 24G are realized by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) execute a program, that is, by software. Each of the above units may be realized by a processor such as a dedicated IC, that is, by hardware. Each of the above units may be realized by using both software and hardware. When multiple processors are used, each processor may realize one of the units, or two or more of the units.
[0034] The control unit 20 may be configured as a first control unit 20A including an acquisition unit 24A, a generation unit 24B, and an association unit 24C. The control unit 20 may be configured as a second control unit 20B including an acquisition unit 24A, a generation unit 24B, an association unit 24C, a reception unit 24D, and a display control unit 24E. The control unit 20 may be configured as a third control unit 20C including an acquisition unit 24A, a generation unit 24B, an association unit 24C, a reception unit 24D, a display control unit 24E, and an anomaly detection unit 24F. The control unit 20 may be configured as a fourth control unit 20D including an acquisition unit 24A, a generation unit 24B, an association unit 24C, a reception unit 24D, a display control unit 24E, an anomaly detection unit 24F, and a storage control unit 24G.
[0035] In this embodiment, an example will be described in which the control unit 20 is configured as a fourth control unit 20D that includes an acquisition unit 24A, a generation unit 24B, an association unit 24C, a reception unit 24D, a display control unit 24E, an anomaly detection unit 24F, and a memory control unit 24G.
[0036] The acquiring unit 24A acquires a plurality of target data 30 of different types. The acquiring unit 24A may acquire the target data 30 from an external information processing device via the communication unit 16. The acquiring unit 24A may also acquire the plurality of target data 30 from the storage unit 22. The acquiring unit 24A may also acquire the target data 30 every time new target data 30 is stored in the storage unit 22. In this embodiment, a form in which the acquiring unit 24A acquires the plurality of target data 30 from the storage unit 22 will be described as an example.
[0037] The generating unit 24B generates element information representing each element included in the plurality of target data 30 acquired by the acquiring unit 24A.
[0038] The generating unit 24B identifies the target data 30 for which element information has not been generated, among the plurality of target data 30 acquired by the acquiring unit 24A. Then, the generating unit 24B generates element information of elements contained in each of the identified one or more target data 30.
[0039] An element refers to each of the objects that can be classified and identified and are included in the object data 30. The elements are, for example, facilities, parts, objects, items, character strings, inspection items, and the like included in the object data 30.
[0040] As described above, in the present embodiment, the case where the information processing device 10 uses a plurality of types of target data 30 related to the inspection of each of a plurality of pieces of equipment will be described as an example. Also, in the present embodiment, a form in which an equipment is included in an element will be described as an example. Also, an object to which an equipment, which is an example of an element, belongs is a facility that includes the equipment.
[0041] The generating unit 24B identifies elements included in the target data 30 for each piece of target data 30, and generates element information for each identified element.
[0042] The element information is information that represents an element, and is information that can identify an element in the target data 30.
[0043] 3 is information showing an example of the data configuration of the element information 42. The element information 42 includes information on at least one of the following items: the target data 30 to which the element belongs, the name of the target data 30 to which the element belongs, the type of the target data 30 to which the element belongs, the position of the element in the target data 30, the feature amount of the element, the identification information of the element, the type of the element, the estimated position of the element in real space, the date and time related to the element, and the reference information indicating the reference destination related to the element. Note that the element information 42 is not limited to information on these items as long as it is information indicating an element in the target data 30.
[0044] The target data 30 to which the element belongs is information that indicates the storage location of the target data 30 to which the element belongs. The name of the target data 30 is the file name of the target data 30, or the like.
[0045] The type of target data 30 is information indicating the type of target data 30. The type of target data 30 is information indicating the data format, data type, data structure, or data content of the target data 30, such as an image of an inspection location, a CAD (Computer Aided Design) drawing of an inspection facility, an inspection work instruction sheet, an inspection report, an inspection list, a drawing image of an inspection facility, map data of an inspection facility, etc.
[0046] The position of an element in the target data 30 is information that can identify the position of an element in the target data 30. For example, when the target data 30 is table data, the position of an element in the target data 30 is information that represents the position of a cell in which the element is registered. When the target data 30 is image data, the position of an element in the target data 30 is information that is represented by a pixel position, a group of pixel positions, position coordinates, or a group of position coordinates in the image data. When the target data 30 is vector data such as CAD data, the position of an element in the target data 30 is information that can identify one or a group of objects included in the CAD data. When the target data 30 is document data, the position of an element in the target data 30 is information that represents the position in the document data in bytes, information that represents the row number and column number, or information that represents the section number. When the target data 30 is document data tagged with XML (Extensible Markup Language) or the like, the position of an element in the target data 30 may be information that represents a tag. Furthermore, when the target data 30 is data including tables or figures, the positions of elements in the target data 30 may be information representing the captions of these tables or figures.
[0047] The feature amount of an element is represented by, for example, a feature vector of the element. A publicly known feature amount may be used as the feature amount. The generating unit 24B may use a feature amount according to the type of the target data 30, the purpose of the processing, etc., as the feature amount of the element.
[0048] The identification information of an element is information that uniquely identifies the element. For example, assume that the element is equipment. In this case, the identification information of the element may be an equipment ID (Identity Document). The equipment ID is, for example, identification information of the equipment to be inspected that is installed in the facility to be inspected, or information that can uniquely identify a part of the equipment. The equipment ID may be assigned in advance to each piece of equipment. Furthermore, the equipment ID may be, for example, a serial number of the equipment.
[0049] The type of element is information that indicates the type of element. For example, assume that the element is equipment. In this case, the type of element is, for example, an equipment type ID. The equipment type ID may be any information that can identify the type of equipment, such as the model of the equipment.
[0050] The estimated position of an element in real space is information that represents an estimated position of the element in real space. For example, the estimated position of an element in real space is information that represents an estimated position of the element in an inspection facility.
[0051] The date and time related to an element is information that indicates the date and time related to the element, such as the creation date and time of the target data 30 that includes the element, or date and time information included in the element.
[0052] The reference information is information that indicates a reference destination related to an element. The reference information is sometimes called a hyperlink.
[0053] The generating unit 24B performs an analysis process according to the type of the target data 30, thereby generating element information 42 for each element included in the target data 30. The analysis process is, for example, image processing, drawing analysis processing, document analysis processing, etc., but it is sufficient that at least one of these analysis processes is performed, and further, it is not limited to these analysis processes.
[0054] For example, assume that the target data 30 is image data obtained by photographing an inspection location.
[0055] In this case, the generating unit 24B identifies each of the objects included in the target data 30, which is image data, as each of the elements included in the target data 30.
[0056] The generating unit 24B identifies elements included in the target data 30 using, for example, a technique for recognizing an object in an image as a rectangle or an object region extraction technique for estimating the region of each object in an image. Examples of techniques for recognizing an object in an image as a rectangle include those disclosed in “Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, CY, & Berg, AC (2016, October). Ssd: Singleshot multibox detector. In European conference on computer vision (pp. 21-37). Springer, Cham.” and the like. Examples of object region extraction techniques include those disclosed in “S. Ito and S. Kubota, “Point Proposal based Instance Segmentation with Rectangular Masks for Robot Picking Task”, ACCV2020”, and the like.
[0057] The generating unit 24B generates element information 42 for each element identified from the target data 30.
[0058] For example, the generating unit 24B identifies the storage location of the target data 30 to which the element belongs, the name of the target data 30, and the type of the target data 30.
[0059] Furthermore, the generating unit 24B specifies the pixel position of the element in the target data 30, and thereby specifies the position of the element in the target data 30.
[0060] Furthermore, the generating unit 24B calculates feature amounts of elements included in the target data 30. For example, the generating unit 24B calculates feature amounts of elements using HOG (Histograms of Oriented Gradients) features, deep learning features, or the like. For the deep learning features, a convolutional neural network (CNN) that performs class classification and is trained using a large amount of images such as ImageNet is used. For example, the generating unit 24B uses an intermediate output just before the final layer that performs class classification by the CNN as the deep learning features.
[0061] Furthermore, the generating unit 24B specifies the identification information and type of the element included in the target data 30. For example, a case where the element is equipment will be described. In this case, the generating unit 24B recognizes the area of the equipment included in the image data. The area of the equipment is, for example, a barcode, a QR (Quick Response) (registered trademark) code, a character string, etc., that appears in the image. The generating unit 24B then analyzes the recognized area of the equipment using one or more known image recognition methods, etc., to specify the identification information and the type of the element. For example, the generating unit 24B specifies the serial number of the equipment as an equipment ID, which is an example of the identification information of the element. Furthermore, the generating unit 24B specifies the model of the equipment, the model number of the equipment, etc., as the type of the element.
[0062] Furthermore, the generating unit 24B specifies the estimated position in real space of the element included in the target data 30. For example, the generating unit 24B specifies the shooting position and shooting direction of the image data by analyzing the entire image data or the area of the element included in the image data using one or more known image recognition methods, etc. Then, the generating unit 24B further specifies the shooting position and shooting direction of the element using one or more known image recognition methods, etc., to specify the estimated position of the element in real space.
[0063] The generating unit 24B may use the camera position at which the target data 30 was captured as the capturing position. In this case, the generating unit 24B may use the camera position represented by GPS (Global Positioning System) information included in the image data as the capturing position. The generating unit 24B may also identify the capturing position by a comparison process between the target data 30 and other image data whose capturing positions have been identified. For example, a method disclosed in "R. Nakashima and A. Seki, "SIR-Net:Scene Independent End-to-End Trainable Visual Relocalizer," 3DV, 2019" or the like may be used as a method for identifying the capturing position by a comparison process.
[0064] Furthermore, the generating unit 24B identifies the date and time related to the element included in the target data 30. For example, the generating unit 24B may identify the creation date and time of the target data 30 to which the element belongs, the date and time of the element itself, etc. as the date and time related to the element. For the creation date and time of the target data 30, for example, the photographing date and time of the image data which is the target data 30 may be used. For the date and time of the element itself, for example, a character string representing the date and time, or information representing the date and time such as the inspection date, which is included in an area representing the element in the target data 30, may be used.
[0065] Furthermore, the generating unit 24B may specify the update date and time of the target data 30 to which the element belongs as the date and time related to the element. Furthermore, the generating unit 24B may further extract the update date and update content of the target data 30 to which the element belongs, and include them in the element information 42. Furthermore, the target data 30 may include location information indicating the creation location of the target data 30. In this case, the generating unit 24B may further extract the location information, and include it in the element information 42.
[0066] Furthermore, when a hyperlink is set to an element included in the target data 30, the generating unit 24B identifies the set hyperlink as reference information.
[0067] Then, the generating unit 24B generates, as element information 42, information on each of the items identified and extracted by these processes.
[0068] It is also assumed that the target data 30 is image data obtained by scanning printed sheets such as wiring diagrams, drawings, and documents, or image data obtained by imaging CAD drawings.
[0069] In this case, the generating unit 24B may convert the image data into document data such as a table or text, and then perform the same process as when the target data 30 described later is document data, to generate the element information 42. The generating unit 24B may also convert the image data into vector data. When the image data is converted into vector data, the generating unit 24B may generate the element information 42 by performing the same process as when the target data 30 described later is CAD data. Note that, for example, line detection using a Hough transform, symbol detection, character string recognition processing such as OCR (Optical Character Recognition), or the like may be used for conversion into document data or vector data.
[0070] It is also assumed that the target data 30 is document data.
[0071] In this case, when the target data 30, which is document data, includes a table, the generation unit 24B identifies the caption of the table and each cell or row as one element. When the target data 30, which is document data, includes a figure, the generation unit 24B may identify an element by performing the same process on the figure as when the target data 30 is image data. The generation unit 24B may also identify the caption and image of the figure included in the document data, which is the target data 30, as one element. When the data format of the target data 30, which is document data, is text format, the generation unit 24B may also identify a character string representing a preset facility, etc., as an element.
[0072] Then, the generation unit 24B may generate the element information 42 for the identified element in the same manner as described above.
[0073] It is also assumed that the target data 30 is drawing data represented by vector data such as CAD data.
[0074] In vector data such as CAD data, layer information, colors, and the like are added to line segments and objects included in the CAD data. For example, equipment, which is an example of an element included in the CAD data, is expressed as a specific figure composed of multiple line segments and objects. For example, the generation unit 24B stores in advance template images representing each of the equipment and wiring connectors and members that constitute part of the equipment. Then, the generation unit 24B identifies each of the equipment as an element by using a known template matching process using the target data 30, which is CAD data, and the template image.
[0075] Then, the generation unit 24B may generate the element information 42 for the identified element in the same manner as described above.
[0076] The generating unit 24B may further extract, as the element information 42 of the element, information on other facilities drawn near the specified element in the target data 30, information on character strings drawn at a position connected to the specified element by a flag line, etc. The generating unit 24B may further extract, as the element information 42 of the element, connection relationships between the specified element and other facilities or parts of facilities connected by objects such as lines or polylines, and specific figures such as wiring connectors.
[0077] Furthermore, for an item that is difficult to identify from the target data 30 among the multiple items included in the element information 42, the generating unit 24B may use information input by a user's operation instruction on the input unit 12.
[0078] For example, there are cases where it is difficult for the generation unit 24B to analyze the estimated position of the element in real space from the target data 30. In this case, the generation unit 24B may register the estimated position input by the user's operation instruction on the input unit 12 in the element information 42 as information representing the estimated position of the element in real space.
[0079] The generating unit 24B may perform the element information 42 generation process for all the target data 30 stored in the storage unit 22 at once. In this case, the acquiring unit 24A may acquire all the target data 30 stored in the storage unit 22. In addition, the generating unit 24B may perform the element information 42 generation process for the target data 30 every time the acquiring unit 24A acquires new target data 30 for which element information 42 has not been generated. In addition, the generating unit 24B may perform the element information 42 generation process for the target data 30 designated by the user as a processing target. In this case, the acquiring unit 24A may acquire the target data 30 selected by the user's operation of the input unit 12 from among the target data 30 stored in the storage unit 22.
[0080] The generating unit 24B assigns an element information ID to the generated element information 42 and stores the element information 42 in the storage unit 22. The element information ID is identification information that uniquely identifies the element information 42.
[0081] Continuing the explanation by returning to Fig. 1, the associating unit 24C stores the multiple pieces of element information 42 in the storage unit 22 in association with each other.
[0082] For example, the associating unit 24C assigns, to each of the multiple element information 42, the element information ID of the other related element information 42 as the related element information ID. Note that the associating unit 24C may classify the multiple element information 42 into groups of related element information 42 and store the classified groups in different storage locations in the storage unit 22.
[0083] In this embodiment, the associating unit 24C registers in the management DB 40 an element information ID, element information 42, and a related element information ID in association with each other.
[0084] 4 is a schematic diagram showing an example of the data configuration of the management DB 40. The management DB 40 is a database that associates element information IDs, element information 42, and related element information IDs. The data format of the management DB 40 may be a table or the like, and is not limited to a database.
[0085] The associating unit 24C registers each piece of element information 42 included in the management DB 40 in which the element information ID and the element information 42 are associated and registered by the generating unit 24B, in association with the associated element information ID. Through this registration process, the associating unit 24C associates the multiple pieces of element information 42 with each other and stores them in the storage unit 22. In detail, one of the multiple items included in the element information 42 is associated with the same or similar element information and stored in the storage unit 22. For example, the associating unit 24C associates element information 42 of identical or similar elements, or element information 42 of elements in the same or similar positions, and stores them in the storage unit 22.
[0086] For example, the associating unit 24C executes a comparison process for comparing each of the multiple element information 42 stored in the storage unit 22 with each of the multiple other element information 42, and identifies the element information 42 of other related elements.
[0087] In detail, for example, the associating unit 24C identifies other element information 42 in which the similarity of at least one item included in the element information 42 is equal to or greater than a threshold value as the related element information 42. Then, the element information ID of the identified other element information 42 is associated with the other element information 42 as a related element information ID and registered in the management DB 40.
[0088] For example, the associating unit 24C searches for other element information 42 that has the same "estimated position of the element in real space" included in the element information 42. The estimated positions being the same means that the difference between the estimated positions is within a predetermined range. The associating unit 24C associates the element information ID of the element information 42 with the element information ID of the other element information 42 that has the same "estimated position of the element in real space" included in the element information 42 as a related element information ID and registers it in the management DB 40.
[0089] The associating unit 24C also searches for other element information 42 that has the same "position of element in target data 30" included in the element information 42. The element positions in the target data 30 being the same means that the difference in the element positions is within a predetermined range. The associating unit 24C associates the element information ID of the element information 42 with the element information ID of the other element information 42 that has the same "position of element in target data 30" included in the element information 42 as the related element information ID, and registers the associated element information ID in the management DB 40.
[0090] When the types of the target data 30 to which the elements to be compared belong are different, the associating unit 24C may convert them into the same type and then compare the positions of the elements in the target data 30. For example, consider a case where element information 42 of an element included in the target data 30, which is image data, is compared with element information 42 of an element included in the target data 30, which is drawing data. In this case, the associating unit 24C may convert the drawing data into image data and then compare the positions of the included elements.
[0091] Furthermore, the associating unit 24C may compare the similarity between the positional relationship of a plurality of elements included in the target data 30, which is image data, and the positional relationship of a plurality of elements included in the target data 30, which is drawing data. Then, the associating unit 24C may associate the element information 42 of the element with the highest similarity as the related element information 42.
[0092] Furthermore, the associating unit 24C searches for other element information 42 having the same or similar "element feature amount" included in the element information 42. For example, the associating unit 24C calculates the similarity of the "element feature amount" between the plurality of element information 42. For example, cosine similarity or the like may be used to calculate the similarity of the feature amount. The associating unit 24C identifies the other element information 42 having the calculated similarity equal to or greater than a threshold as the other related element information 42. Then, the associating unit 24C associates the element information ID of the element information 42 with the element information ID of the other element information 42 having the same or similar "element feature amount" included in the element information 42 as the related element information ID and registers the element information ID in the management DB 40. In this case, the associating unit 24C can associate the plurality of element information 42 having similar feature amounts.
[0093] Furthermore, the associating unit 24C searches for other element information 42 having the same or similar "element identification information" and / or "element type" included in the element information 42. The associating unit 24C associates the element information ID of the element information 42 with the element information ID of the other element information 42 having the same or similar "element identification information" and / or "element type" included in the element information 42 as a related element information ID and registers the element information ID in the management DB 40.
[0094] Furthermore, the associating unit 24C may associate the element information ID of another element information 42, which has the same or similar “date and time related to the element” included in the element information 42, as a related element information ID and register it in the management DB 40. “The date and time related to the element” being similar means that the difference between the dates and times related to the element is within a predetermined period.
[0095] The associating unit 24C also searches for other element information 42 having the same or similar at least one of the “target data 30 to which the element belongs”, the “name of the target data 30”, and the “type of the target data 30” included in the element information 42. The associating unit 24C associates the element information ID of the element information 42 with the element information ID of the other element information 42 having the same or similar at least one of the “target data 30 to which the element belongs”, the “name of the target data 30”, and the “type of the target data 30” included in the element information 42 as a related element information ID and registers the element information ID in the management DB 40.
[0096] Note that there may be a plurality of other related element information 42 for one element information 42. In this case, the associating unit 24C may store a plurality of related element information IDs in association with one element information ID.
[0097] In the manner described above, the associating unit 24C stores the multiple pieces of element information 42 in the management DB 40 of the storage unit 22 in association with each piece of related element information 42.
[0098] Therefore, the multiple pieces of element information 42 are associated with each other.
[0099] For example, a description will be given assuming a situation in which element information 42 included in each of the multiple target data 30 shown in Figures 2A to 2F is associated with each other. As described above, Figures 2A to 2F are an example of multiple types of target data 30 related to the inspection of each of multiple pieces of equipment.
[0100] Let us assume that the associating unit 24C associates a plurality of pieces of element information 42 contained in these multiple types of target data 30.
[0101] Specifically, for example, it is assumed that the generation unit 24B generates each row of the inspection date and time list 30A from the inspection date and time list 30A as element information 42 of the element. The element information 42 of the inspection date and time list 30A includes, for example, an inspection date and time, which is a "date and time related to the element," and inspection content indicating a monthly inspection or an annual inspection.
[0102] In this case, the associating unit 24C associates, among the multiple pieces of element information 42 included in the inspection date and time list 30A, element information 42 whose inspection content indicates a monthly inspection with element information 42 that indicates a monthly inspection in the inspection work instruction 30B. Also, the associating unit 24C associates, among the multiple pieces of element information 42 included in the inspection date and time list 30A, element information 42 whose inspection content indicates an annual inspection with element information 42 that indicates an annual inspection in the inspection work instruction 30B.
[0103] In addition, the association unit 24C associates each of the multiple element information 42 included in the inspection date and time list 30A with the element information 42 included in the inspection report 30D, which has the same or similar inspection date and time as the "date and time related to the element". For example, the element information 42 in which the inspection date and time as the "date and time related to the element" included in the inspection date and time list 30A is "February 1, 2021" is associated with the element information 42 in which the inspection result as the "date and time related to the element" included in the inspection report 30D is "February 1, 2021". Similarly, the element information 42 in which the inspection date and time as the "date and time related to the element" included in the inspection date and time list 30A is "March 1, 2021" is associated with the element information 42 in which the inspection result as the "date and time related to the element" included in the inspection report 30D is "March 1, 2021".
[0104] Also, for example, it is assumed that the generation unit 24B generates each line of the inspection work instruction 30B as element information 42 of the element from the inspection work instruction 30B.
[0105] In this case, the association unit 24C associates, among the multiple element information 42 included in the inspection work instruction 30B, element information 42 whose inspection item, which is the “element identification information,” represents equipment A, with element information 42 included in the inspection report 30D whose “element identification information” represents equipment A.
[0106] Furthermore, the associating unit 24C associates element information 42 whose "element identification information" included in the inspection work instruction 30B represents facility A with element information 42 whose "element identification information" included in the facility floor plan 30E represents facility A. Furthermore, the associating unit 24C associates element information 42 whose "element identification information" included in the inspection work instruction 30B represents facility A with element information 42 whose "element identification information" included in the inspection image 30F represents facility A.
[0107] Similarly, the associating unit 24C also associates the element information 42 included in the multiple types of target data 30 with each other for the facilities B to C, for each piece of element information 42 that represents the same facility.
[0108] Furthermore, the multiple pieces of element information 42 representing each of the facilities A to C included in the inspection work instruction sheet 30B all have the same "element type" of facility 1. For this reason, the associating unit 24C associates the multiple pieces of element information 42 representing each of the facilities A to C as element information 42 representing the same "element type."
[0109] Furthermore, the associating unit 24C associates the element information 42 including the character string "A part" included in the inspection report 30D with the element information 42 including the same character string "A part" included in the drawing 30C of the facility 1.
[0110] In addition, the associating unit 24C associates element information 42 with at least one of the items included in the element information 42, namely, the target data 30 to which the element belongs, the name of the target data 30, the type of target data 30, the position of the element in the target data 30, the feature value of the element, the identification information of the element, the type of element, the estimated position of the element in real space, the date and time related to the element, and the reference information, which are the same or similar.
[0111] For this reason, for example, element information 42 of each of the elements of equipment A to equipment D included in the facility floor plan 30E is associated with element information 42 of each of the elements of equipment A to equipment D included in another type of target data 30 such as the inspection image 30F. Also, for multiple pieces of element information 42 included in the same type of target data 30, element information 42 in which at least one of the items included in the element information 42 is identical or similar is associated. In other words, the associating unit 24C can associate multiple pieces of element information 42 included in each of multiple pieces of target data 30 of different types for each related element.
[0112] In addition, the association unit 24C can associate element information 42 of elements that have the same or similar estimated positions in real space among multiple element information 42 belonging to different target data 30 by comparing and associating the “estimated position of the element in real space,” which is an item included in the element information 42.
[0113] Returning to Figure 1, we will continue with the explanation.
[0114] The accepting unit 24D accepts the selection of the element information 42. The display control unit 24E controls the display of various images on the display unit 14.
[0115] For example, the display control unit 24E displays on the display unit 14 a selection screen for accepting the selection of the element information 42.
[0116] 5 is a schematic diagram showing an example of the selection screen 52. The display control unit 24E displays the selection screen 52 including a list of the element information 42 generated by the generation unit 24B on the display unit 14. For example, the display control unit 24E extracts any of the items included in each of the multiple pieces of element information 42 registered in the management DB 40, and displays the extracted items on the selection screen 52.
[0117] 5 shows an example of the inspection dates and times and inspection details included in the inspection date and time list 30A displayed on the selection screen 52. The user operates the input unit 12 while referring to the selection screen 52 to select element information 42 of the elements to be displayed. For example, the user inputs a range of inspection dates and times. The reception unit 24D receives the range of inspection dates and times from the input unit 12, and thereby receives a selection of element information 42 that includes the received range of inspection dates and times.
[0118] Continuing the description by returning to Fig. 1, the display control unit 24E displays on the display unit 14 a display screen including the element information 42 for which the selection has been accepted and other element information 42 associated with the selected element information 42.
[0119] 6 is a schematic diagram showing an example of a display screen 54. The display screen 54 is an example of a display screen 50 including related element information 42.
[0120] In this case, the display control unit 24E reads the related element information ID associated with the element information 42 whose selection has been accepted from the management DB 40. Then, the display control unit 24E reads the element information 42 corresponding to the read element information ID from the management DB 40. The display control unit 24E generates a display screen 50 including the element information 42 whose selection has been accepted and the element information 42 associated with the element information 42, and displays it on the display unit 14.
[0121] For example, assume that the user specifies desired element information 42 in the inspection date and time list 30A. The display control unit 24E displays a display screen 50 including the specified element information 42 and other element information 42 associated with the specified element information 42.
[0122] Specifically, for example, a case is assumed in which the user operates the input unit 12 while viewing the selection screen 52 shown in FIG. 5 to specify a range of inspection dates and times and to operate the search button. The display control unit 24E extracts from the management DB 40 a list of element information 42 including "dates and times related to elements" within the range of the specified inspection dates and times received via the reception unit 24D, and displays the list on the selection screen 52. Furthermore, it is assumed that the user selects the element information 42 of "2021 / 04 / 01 monthly inspection" from among the element information 42 displayed on the selection screen 52. In this case, the display control unit 24E accepts the selection of the element information 42 of "2021 / 04 / 01 monthly inspection".
[0123] The display control unit 24E generates a display screen 54 including the element information 42 for which the selection has been accepted and other element information 42 associated with the element information 42, and displays the display screen 54 on the display unit 14.
[0124] The other element information 42 related to the element information 42 whose selection has been accepted includes, in the management DB 40, element information 42 identified by a related element information ID that is related to the element information 42 whose selection has been accepted, and element information 42 identified by a related element information ID that is further related to the element information 42. Also, element information 42 identified by a related element information ID that is further related to the element information 42 may be included.
[0125] Therefore, for example, a display screen 54 shown in FIG.
[0126] As shown in FIG. 6, the display screen 54 includes element information 42BA, element information 42BB, element information 42BC, and element information 42BD in the inspection work instruction 30B, including the “monthly inspection” that is the inspection content included in the element information 42 of “2021 / 04 / 01 monthly inspection”. The element information 42BA is the element information 42 of equipment A included in the inspection work instruction 30B. The element information 42BB is the element information 42 of equipment B included in the inspection work instruction 30B. The element information 42BC is the element information 42 of equipment C included in the inspection work instruction 30B. The element information 42BD is the element information 42 of equipment C included in the inspection work instruction 30B.
[0127] In addition, the display control unit 24E displays element information 42 that is further associated with other element information 42 that is associated with the element information 42 of "2021 / 04 / 01 monthly inspection" on the display screen 54. For example, the display control unit 24E displays element information 42EA of facility A, element information 42EB of facility B, element information 42EC of facility C, and element information 42ED of facility D in the facility floor plan 30E on the display screen 54.
[0128] Furthermore, the display control unit 24E displays element information 42FA of facility A, element information 42FB of facility B, element information 42FC of facility C, and element information 42FD of facility D, which are included in the inspection image 30F, on the display screen 54. Furthermore, the display control unit 24E displays element information 42DA of facility A, element information 42DB of facility B, element information 42DC of facility C, and element information 42DD of facility D, which are included in the inspection report 30D, on the display screen 54.
[0129] In this manner, the display control unit 24E displays on the display unit 14 a display screen 54 including the element information 42 for which selection has been accepted and other element information 42 associated with the element information 42 in the management DB 40.
[0130] Therefore, by visually checking the display screen 54, the user can easily check a group of related elements included in the multiple target data 30. Furthermore, the user can easily check not only the selected element information 42 but also a list of other element information 42 related to the selected element information 42. Furthermore, the user can easily check, on a single display screen 54, a list of element information 42 related to the selected element information 42 that is included in multiple target data 30 of different types.
[0131] The display control unit 24E may superimpose and display information representing the recommended inspection route on the facility floor plan 30E included in the display screen 54. For example, the display control unit 24E may superimpose and display a line image 55 representing the recommended inspection route as shown in FIG. 6. The information representing the recommended inspection route may be stored in advance in the storage unit 22. The display control unit 24E may also generate information representing the recommended inspection route and display it on the display screen 54.
[0132] Furthermore, there are cases where reference information is included in the element information 42. In this case, the display control unit 24E may display on the display unit 14 the information represented by the reference information.
[0133] FIG. 7 is a schematic diagram showing an example of the display screen 56. The display screen 56 is an example of the display screen 50. For example, assume a scene in which an area of facility A, which is element information 42EA to which a hyperlink is added in a facility floor plan 30E, is operated by a user's operation instruction on the input unit 12. In this case, the display control unit 24E displays the reference information represented by the reference information included in the element information 42EA on the display unit 14. FIG. 7 shows an example in which the reference information represented by the hyperlink is a past inspection image 30FA of facility A and a past inspection result 30DA of facility A.
[0134] In addition, when the user specifies other element information 42 (e.g., element information 42EB) to which a hyperlink has been added, the display control unit 24E displays, on the display screen 56, the reference information represented by the reference information contained in the element information 42EB.
[0135] Therefore, by visually checking the display screen 54, the user can easily check a group of related elements contained in the multiple target data 30, and the reference information represented by the hyperlink.
[0136] In addition, when inspecting an infrastructure facility, for example, the user can easily check other related element information 42 and reference information by specifying element information 42 of a position or element on the inspection route. Therefore, the user can easily check and collect information before and after the inspection by visually checking the display screen 50.
[0137] Returning to Figure 1, we will continue with the explanation.
[0138] Next, the anomaly detection unit 24F will be described. The anomaly detection unit 24F detects an anomaly contained in the target data 30.
[0139] For example, for multiple pieces of target data 30 of the same type but with different creation dates and times, the anomaly detection unit 24F compares elements at the same position in the target data 30, and detects elements whose difference is equal to or greater than a predetermined threshold as an anomaly.
[0140] Specifically, for example, assume that the target data 30 is image data. In this case, the anomaly detection unit 24F identifies elements, such as the facility to be inspected, contained in the captured image data. Then, the anomaly detection unit 24F reads image data, which is other target data 30 with different shooting dates and times and is associated with element information 42 of the identified elements, from the storage unit 22. The anomaly detection unit 24F performs a registration process on the image data with the identified elements and the other read image data so that elements, such as the position of the inspection target, are at the same position in the image. The anomaly detection unit 24F detects an element for which the difference between the aligned image data is equal to or greater than a predetermined threshold as an anomaly.
[0141] Also, for example, assume that the anomaly detection unit 24F detects a general anomaly occurring in a facility or equipment from the target data 30, which is image data. The general anomaly is, for example, a crack or rust occurring in the facility or equipment. In this case, the anomaly detection unit 24F may detect the anomaly contained in the target data 30 by using a trained model.
[0142] For example, a trained model that detects the presence or absence of cracks or rust and the location of cracks or rust from a large number of images of cracks or rust may be trained in advance using a deep neural network or the like, and used to detect anomalies. A trained model that classifies input images into images that include cracks or rust and images that do not include cracks or rust may also be used. A trained model that outputs the presence or absence of cracks or rust and the degree of deterioration may also be used. A trained model that outputs the type of anomaly such as cracks or rust may also be used. A trained model that outputs the presence or absence of cracks or rust and the location or area of the anomaly in pixel units may also be used.
[0143] Alternatively, a trained model may be used that inputs training images that have been instructed only on the presence or absence of cracks or rust, and outputs an anomaly score for each pixel or pixel block. In this case, the trained model may be trained in advance so that the maximum anomaly score to be output matches the presence or absence of anomalies contained in the training images to be input.
[0144] By using a trained model that outputs an anomaly score in pixel or pixel block units from image data, the anomaly detection unit 24F can detect anomalies in pixel or pixel block units.
[0145] The anomaly detection unit 24F also performs a registration process on the target data 30, which is a plurality of image data captured at different times, in the same manner as described above, so that elements such as the position of the inspection target are at the same position in the image. The anomaly detection unit 24F may then compare deep learning features between the registered image data, and determine that an area in which the difference between deep learning features at corresponding positions is equal to or greater than a predetermined threshold is an anomaly. For the deep learning features, for example, a method disclosed in "Sub-Image Anomaly Detection with Deep Pyramid Correspondences" (https: / / arxiv.org / abs / 2005.02357) or the like may be used.
[0146] In addition, the anomaly detection unit 24F may detect anomalies using a trained model trained using only normal image data that does not contain anomalies. For example, the method for detecting anomalies may be described in "Paul Bergmann, Michael Fauser, David Sattlegger, Carsten Steger, Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings, CVPR2020".
[0147] When an abnormality is detected, the display control unit 24E displays, on the display unit 14, an abnormality detection result including the detected abnormality.
[0148] Fig. 8 is a schematic diagram showing an example of an anomaly detection result display screen 58. Fig. 8 shows an example in which an anomaly is detected in a facility floor plan 30E.
[0149] The display control unit 24E displays a display screen 58 including a facility floor plan 30E, which is the target data 30 including an anomaly, and an image E showing the anomaly detected in the facility floor plan 30E on the display unit 14. The display control unit 24E also reads element information 42 including the detected anomaly from the storage unit 22 and displays it on the display unit 14. FIG. 8 shows an example of a display screen 58 including element information 42FA including the detected anomaly. The display control unit 24E may further display other element information 42 associated with the element information 42FA including the detected anomaly. FIG. 8 shows an example of a form in which element information 42F representing a past inspection image of the same location, element information 42D representing a past inspection report, and element information 42C representing a similar case image are displayed, which are associated with the element information 42FA including the detected anomaly.
[0150] By displaying an anomaly detection result including the detected anomaly on the display unit 14, the display control unit 24E can provide the anomaly contained in the target data 30 in an easily identifiable manner.
[0151] Furthermore, the display control unit 24E displays a display screen 58 including element information 42FA including the detected anomaly and other element information 42 associated with the element information 42FA including the detected anomaly on the display unit 14. For this reason, the display control unit 24E can provide the user with a list of various element information 42 such as maps, drawings, and past images related to the anomaly in an easily verifiable manner. Furthermore, the display control unit 24E can provide the user with a history of the anomaly such as aging deterioration in an easily verifiable manner.
[0152] In addition, a hyperlink may be included in at least one of the element information 42FA including the detected anomaly and the other element information 42 associated with the element information 42FA including the detected anomaly. In this case, the display control unit 24E may further display on the display screen 58 the information of the reference destination represented by the hyperlink.
[0153] Therefore, by visually checking the display screen 58, the user can easily check for anomalies without visually checking the anomalies in the real space. Therefore, the information processing device 10 of this embodiment can reduce the cost of inspection by the user. Furthermore, the display control unit 24E can simultaneously present past data of the location where the anomaly was detected, related drawings, documents, and the like on the display screen 58. Therefore, the user can easily check the location of the anomaly and the importance of the anomaly. Furthermore, by allowing the user to easily check the location of the anomaly and the importance of the anomaly, the cost required for dealing with the anomaly, such as making a repair plan or replacing parts, can be reduced.
[0154] Continuing the description by returning to Fig. 1, the storage control unit 24G associates and stores anomaly information indicating an anomaly with element information 42 including an anomaly for which a selection of anomaly recognition has been received from the user, among the anomalies included in the displayed anomaly detection results.
[0155] Assume a scene in which the display screen 58 of the anomaly detection result shown in FIG. 8 is displayed on the display unit 14. The user operates the input unit 12 while viewing the display screen 58 to input whether or not to recognize the displayed anomaly as an anomaly. The user also operates the input unit 12 to input information on the recognition result of the displayed anomaly. For example, the user operates the input unit 12 to input information on the recognition result of the anomaly, such as whether or not to recognize the anomaly as an anomaly and a response after the anomaly occurs. For example, the user may respond to the detected anomaly by replacing parts. In this case, the user may operate the input unit 12 to input the content of the anomaly response as information representing the response after the anomaly occurs. When the user inputs the content of the anomaly response, the display control unit 24E may display an input form for inputting the content of the anomaly response on the display unit 14. The display control unit 24E may also output audio guidance, etc., for inputting the content of the anomaly response from a speaker. The user may input the content of the anomaly response by operating the input unit 12 or by voice input, etc.
[0156] When the information relating to the result of the abnormality determination received from the input unit 12 includes information indicating that the abnormality has been determined, the storage control unit 24G stores the abnormality information in association with the element information 42 including the abnormality.
[0157] 9 is a schematic diagram showing an example of the data configuration of the incident information 44. For example, the incident information 44 includes an incident information ID, an incident occurrence location, an incident occurrence time, an incident degree, a response after the incident occurrence, and a related element information ID.
[0158] The memory control unit 24G generates an incident information ID that uniquely identifies the incident information 44. The memory control unit 24G also specifies information that is included in the information related to the incident determination result received from the input unit 12 and indicates a response after the occurrence of the incident.
[0159] Furthermore, the memory control unit 24G specifies a preset degree of abnormality. Note that the memory control unit 24G may specify a degree of abnormality detected by the abnormality detection unit 24F. Furthermore, the memory control unit 24G may specify a degree of abnormality input by the user operating the input unit 12.
[0160] Furthermore, the memory control unit 24G identifies the time when the abnormality occurred by estimating the time when the abnormality occurred. For example, when the period during which the abnormality occurred can be estimated from the past inspection image 30F, the memory control unit 24G identifies the estimated period during which the abnormality occurred as the time when the abnormality occurred.
[0161] The storage control unit 24G also specifies the position of the detected anomaly in the target data 30 as the anomaly occurrence position. The storage control unit 24G also specifies the element information ID of the element information 42 including the detected anomaly as the related element information ID. The storage control unit 24G may specify any element information ID of the multiple element information 42 displayed on the display screen 58 of the anomaly detection result as the related element information ID. That is, the storage control unit 24G may specify the element information ID of the element information 42 referenced by the anomaly detection unit 24F and the display control unit 24E when the anomaly is detected as the related element information ID. The storage control unit 24G may also specify the element information ID of the element information 42 included in the target data 30 created by the user responding to the anomaly as the related element information ID.
[0162] Then, the storage control unit 24G generates incident information 44 including the generated and specified incident information ID, the incident occurrence location, the incident occurrence time, the incident degree, the response after the incident occurrence, and related element information IDs.
[0163] Furthermore, the storage control unit 24G assigns the abnormality information ID of the abnormality information 44 to element information 42 identified by the related element information ID included in the abnormality information 44. In this case, the storage control unit 24G may store the management DB 40 in the storage unit 22 as a management DB 41 in which the management DB 40 is further associated with and registered with the abnormality information ID.
[0164] 10 is a schematic diagram showing an example of the data configuration of the management DB 41. The management DB 41 is a database in which an incident information ID is further associated with the management DB 40. In other words, the management DB 41 is a database in which an element information ID, element information 42, related element information IDs, and incident information IDs are associated with each other.
[0165] By storing a management DB 41 that associates element information 42 with an abnormality information ID in the memory unit 22, the memory control unit 24G can store abnormality information 44 in association with element information 42 that includes an abnormality included in the abnormality detection result and for which the user has chosen to recognize the abnormality.
[0166] In this way, anomaly detection section 24F detects an anomaly contained in target data 30, and storage control section 24G stores anomaly information 44 in association with element information 42 including an anomaly for which a selection of anomaly recognition has been received from the user.
[0167] Therefore, the information processing device 10 of the present embodiment can provide the user with information that enables a more accurate decision on the occurrence of an abnormality. For example, by checking the abnormality information 44, the user can respond to the abnormality by more accurately inspecting, replacing parts, and creating a repair plan.
[0168] Furthermore, the storage control unit 24G stores the abnormality information 44 in association with the element information 42 including the abnormality for which the selection of abnormality recognition has been accepted from the user. Therefore, for example, when a new abnormality occurs in a related element, the user can easily confirm the location and degree of the abnormality by referring to the abnormality information 44. Therefore, the information processing device 10 of this embodiment can reduce the costs required for repair planning, part replacement, and the like.
[0169] Furthermore, the storage control unit 24G may perform a deletion process of deleting from the storage unit 22, element information 42 that satisfies a predetermined condition, among a plurality of element information 42 stored in the storage unit 22.
[0170] The predetermined condition may be determined in advance. The predetermined condition may be, for example, element information 42 whose creation date and time is a predetermined period or more before the current time, element information 42 of low importance, etc. The element information 42 of low importance may be, for example, element information 42 to be deleted that is input by a user's operation instruction of the input unit 12. Moreover, the element information 42 of low importance may be, for example, element information 42 that includes content of low importance determined in advance. The predetermined condition may be appropriately changeable in response to a user's operation instruction of the input unit 12, etc.
[0171] By performing a deletion process in which the storage control unit 24G deletes element information 42 that satisfies a predetermined condition from the storage unit 22, it is possible to shorten the search time for element information 42 and ensure free space in the storage unit 22.
[0172] Next, an example of the flow of information processing executed by the information processing device 10 of the present embodiment will be described.
[0173] FIG. 11 is a flowchart showing an example of the flow of the association process executed by the information processing device 10 of this embodiment.
[0174] The acquiring unit 24A acquires a plurality of target data 30 from the storage unit 22 (step S100).
[0175] The generating unit 24B generates, for each element included in each of the plurality of target data 30 acquired in step S100, element information 42 representing the element (step S102).
[0176] The associating unit 24C associates the multiple pieces of element information 42 generated in step S102 with each related piece of element information 42 and stores them in the storage unit 22 (step S104). In this embodiment, the associating unit 24C associates the element information ID, the element information 42, and the related element information ID and registers them in the management DB 40. Then, this routine ends.
[0177] FIG. 12 is a flowchart showing an example of the flow of display processing on the display screen 50 executed by the information processing device 10 of this embodiment.
[0178] The display control unit 24E displays, on the display unit 14, a selection screen 52 for receiving a selection of element information 42 (Step S200). For example, the display control unit 24E displays, on the display unit 14, the selection screen 52 shown in FIG.
[0179] The receiving unit 24D receives the selection of the element information 42 (Step S202).
[0180] The display control unit 24E generates a display screen 54 including the element information 42 selected in step S202 and other element information 42 associated with the element information 42, and displays the display screen 54 on the display unit 14 (step S204). Therefore, for example, the display screen 54 shown in Fig. 6 is displayed on the display unit 14. Then, this routine ends.
[0181] FIG. 13 is a flowchart showing an example of the flow of an anomaly detection process executed by the information processing device 10.
[0182] The anomaly detection unit 24F detects an anomaly contained in the target data 30 (Step S300). If the anomaly detection unit 24F does not detect an anomaly (Step S302: No), this routine ends. If the anomaly detection unit 24F detects an anomaly (Step S302: Yes), the routine proceeds to Step S304.
[0183] In step S304, the display control unit 24E displays the anomaly detection result, including the anomaly detected in step S300, on the display unit 14 (step S304). For example, the display unit 14 displays a display screen 58 of the anomaly detection result shown in FIG.
[0184] The storage control unit 24G determines whether or not a selection has been received as to whether or not to recognize the abnormality displayed in step S304 as an abnormality (step S306). If a selection indicating that the abnormality should not be recognized as an abnormality has been received (step S306: No), this routine ends. If a selection indicating that the abnormality should be recognized as an abnormality has been received (step S306: Yes), the routine proceeds to step S308.
[0185] In step S308, the storage control unit 24G stores the incident information 44 in association with the element information 42 including the incident recognized as an incident received in step S306 (step S308), and then ends this routine.
[0186] As described above, the information processing device 10 of the present embodiment includes an acquisition unit 24A, a generation unit 24B, and an association unit 24C. The acquisition unit 24A acquires a plurality of target data 30 of different types. The generation unit 24B generates element information 42 representing each element included in the plurality of target data 30. The association unit 24C stores the plurality of element information 42 in the storage unit 22 in association with each related element information 42.
[0187] In the conventional technology, there was no association between elements contained in multiple target data 30 of different types, making it difficult to provide an easily identifiable group of related elements contained in multiple target data 30.
[0188] On the other hand, the information processing device 10 of this embodiment stores, in the storage unit 22, element information 42 of a plurality of elements included in each of a plurality of different types of target data 30 in association with each related element information 42. The information processing device 10 of this embodiment can provide an easily verifiable group of element information 42 of a specific element and other element information 42 related to the element information 42 of the specific element included in each of a plurality of different types of target data 30.
[0189] Therefore, the information processing device 10 of this embodiment can provide a group of related elements contained in a plurality of target data 30 in an easily identifiable manner.
[0190] For example, assume that multiple types of target data 30 related to inspections are used as the multiple target data 30. In the conventional technology, when inspecting an infrastructure facility, a large amount of CAD data related to the facility, document data such as an inspection management table and work instructions, and past inspection data were managed as unassociated, separate data. For this reason, in the conventional technology, it was not easy for a user to check related drawings, document data, and past data during inspection. On the other hand, in the information processing device 10 of the present embodiment, element information 42 of multiple elements contained in each of multiple target data 30 of different types is associated with each related element information 42 and stored in the storage unit 22. For this reason, the information processing device 10 of the present embodiment can provide a group of elements related to a specific inspection element from multiple types of target data 30 such as documents, drawings, maps, and inspection results related to infrastructure facilities that were managed separately, in a manner that allows easy confirmation.
[0191] Next, an example of the hardware configuration of the information processing device 10 of the above embodiment will be described.
[0192] FIG. 14 is a diagram illustrating an example of a hardware configuration of the information processing device 10 of the present embodiment.
[0193] The information processing device 10 of this embodiment has a hardware configuration utilizing a conventional computer, in which a CPU (Central Processing Unit) 86, a ROM (Read Only Memory) 88, a RAM (Random Access Memory) 90, an I / F 92, etc. are interconnected via a bus 94.
[0194] The CPU 86 is a calculation device that controls the information processing device 10 of this embodiment. The ROM 88 stores programs and the like that realize information processing by the CPU 86. The RAM 90 stores data necessary for various processes by the CPU 86. The I / F 82 is an interface that is connected to the input unit 12, the display unit 14, the communication unit 16, etc., and is used to transmit and receive data.
[0195] In the information processing device 10 of this embodiment, the CPU 86 reads out a program from the ROM 88 onto the RAM 90 and executes it, thereby realizing each of the above functions on the computer.
[0196] The programs for executing the above-mentioned processes executed by the information processing device 10 of the present embodiment may be stored in a HDD (hard disk drive). Also, the programs for executing the above-mentioned processes executed by the information processing device 10 of the present embodiment may be provided by being pre-installed in the ROM 88.
[0197] The program for executing the above-mentioned processing executed by the information processing device 10 of the present embodiment may be stored in a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD) in an installable or executable format file and provided as a computer program product. The program for executing the above-mentioned information processing executed by the information processing device 10 of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. The program for executing the above-mentioned information processing executed by the information processing device 10 of the present embodiment may be provided or distributed via a network such as the Internet.
[0198] Although the embodiment has been described above, the embodiment is presented as an example and is not intended to limit the scope of the invention. This new embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]
[0199] 10. Information processing device 22 Memory section 24A Acquisition Department 24B Generation part 24C Association section 24D Reception 24E Display control unit 24F Anomaly Detection Department 24G Memory control unit 30 Target Data 42 Element Information 44 Abnormality Report
Claims
1. An acquisition unit that acquires a plurality of target data related to equipment, the target data being different in at least one of data format, data type, data structure, and data content; a generation unit that generates, for each of the equipment elements identified from a plurality of the target data, element information, the element information including information representing the contents of at least one of the following items: a storage location of the target data to which the element belongs, a name of the target data to which the element belongs, a type of the target data to which the element belongs, a position of the element in the target data, a feature amount of the element, identification information of the element, a type of the element, an estimated position of the element in real space, a creation date and time of the target data to which the element belongs, and a reference destination for the element; an association unit that associates, for each of the element information, other element information having the same or similar information representing one of the items among the information representing the contents of the at least one item included in the element information, as the related element information and stores the information in a storage unit; An information processing device comprising:
2. The generation unit generates the element information based on at least one processing result of image processing, drawing analysis processing, and document analysis processing for the target data. The information processing device according to claim 1 .
3. The associating unit associates, for each of the element information, other element information having a similarity of information representing the content of the at least one item included in the element information that is equal to or greater than a threshold value as the related element information and stores the associated element information in the storage unit.
3. The information processing device according to claim 1 or 2.
4. A reception unit that receives a selection of the element information; a display control unit that displays a display screen including the element information selected and other element information associated with the element information on a display unit; The information processing device according to claim 1 , further comprising:
5. An anomaly detection unit that detects an anomaly contained in the target data by using a trained model for detecting the anomaly contained in the target data, a difference between the target data and other target data acquired at different times, or a difference between the target data and the target data that does not include an anomaly; The display control unit is displaying an anomaly detection result, which is an image showing the detected anomaly, on the display unit; The information processing device according to claim 4.
6. a storage control unit that associates and stores, among the anomalies included in the displayed anomaly detection result, anomaly information indicating the anomaly, with the element information including the anomaly for which a selection of anomaly recognition has been received from a user; The information processing device according to claim 5 .
7. The storage control unit, deleting, from the storage unit, the element information that satisfies a predetermined condition among the plurality of element information; The information processing device according to claim 6.
8. A computer comprising: A step of acquiring a plurality of target data related to equipment, the target data being different in at least one type of data format, data type, data structure, and data content; a step of generating element information for each of the equipment elements identified from the plurality of target data by performing at least one of analysis processes including image processing, drawing analysis processing, and document analysis processing according to the type of the target data, the element information including information about the element in the target data, the information representing the contents of at least one of the following items: a storage location of the target data to which the element belongs, a name of the target data to which the element belongs, a type of the target data to which the element belongs, a position of the element in the target data, a feature amount of the element, identification information of the element, a type of the element, an estimated position of the element in real space, a creation date and time of the target data to which the element belongs, and a reference destination for the element; storing, for each of the element information, other element information having the same or similar information representing the content of one of the items among the information representing the content of the at least one item included in the element information, associating the other element information as related element information in a storage unit; An information processing method for performing the above.
9. A step of acquiring a plurality of target data related to equipment, the target data being different in at least one of data format, data type, data structure, and data content; a step of generating element information for each of the equipment elements identified from the plurality of target data by performing at least one of analysis processes including image processing, drawing analysis processing, and document analysis processing according to the type of the target data, the element information including information about the element in the target data, the information representing the contents of at least one of the following items: a storage location of the target data to which the element belongs, a name of the target data to which the element belongs, a type of the target data to which the element belongs, a position of the element in the target data, a feature amount of the element, identification information of the element, a type of the element, an estimated position of the element in real space, a creation date and time of the target data to which the element belongs, and a reference destination for the element; storing, for each of the element information, other element information having the same or similar information representing the content of one of the items among the information representing the content of the at least one item included in the element information, associating the other element information as related element information in a storage unit; An information processing program for causing a computer to execute the above.
Citation Information
Patent Citations
Management device for document information
JP2002108893A
Information sharing device and program for information sharing process
JP2005038006A
Metadata extraction device
JP2005235099A
Apparatus, method and program for data processing
JP2005310044A
Information presentation method and system
JP2017107509A
