Management device, display system, display method, and program

Abstract

To provide a management apparatus with which a user can easily confirm procedures related to a facility which needs to be inspected, even when the facility or the like does not come into sight of a user.SOLUTION: A management apparatus includes: an acquisition unit which acquires three-dimensional data including a facility to be monitored; a specifying unit which specifies a plurality of components constituting the facility to be monitored; a display control unit which displays the facility to be monitored so as to be rotated in a facility data display region of a display unit, and displays a plurality of procedures to be executed by a worker related to the facility to be monitored in a procedure display region of the display unit; and an input unit which receives a selection of at least one component displayed in the facility data display region or at least one procedure displayed in the procedure display region. The display control unit displays a procedure associated with the selected at least one component or a component associated with the selected at least one procedure so as to be highlighted.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a management device, a display system, a display method, and a program.

Background Art

[0002] In order to efficiently perform work on facilities such as substations, workers may perform work while checking a procedure manual displayed on a portable terminal device. Patent Document 1 describes a configuration of a work support system that superimposes a manual or the like on real-world facilities visible through an AR (Augmented Reality) display.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the work support system described in Patent Document 1 displays only a manual related to facilities within the range visible to the user wearing the AR display. Therefore, when there are a large number of facilities in a wide area, there are also a large number of facilities that do not fall within the user's field of view. In this case, since the manual related to the facilities that do not fall within the user's field of view cannot be confirmed in advance, there is a problem that the work efficiency of the facilities by the worker deteriorates.

[0005] An object of the present disclosure is to provide a management device, a display system, a display method, and a program that can easily confirm procedures related to facilities that are not within the operator's field of view and that require work.

Means for Solving the Problems

[0006] The management device according to the first aspect of the present disclosure includes an acquisition unit that acquires three-dimensional data including a facility to be monitored, a specification unit that specifies a plurality of components constituting the facility to be monitored, a display control unit that rotatably displays the facility to be monitored in a facility data display area of a display unit and displays a plurality of procedures to be executed by an operator related to the facility to be monitored in a procedure display area of the display unit, and an input unit that accepts selection of at least one component displayed in the facility data display area or at least one procedure displayed in the procedure display area. The display control unit highlights and displays the procedure associated with the selected at least one component or the component associated with the selected at least one procedure.

[0007] The display system according to the second aspect of the present disclosure includes a display device that displays a facility data display area including a facility to be monitored and a procedure display area including a plurality of procedures to be executed by an operator related to the facility to be monitored, an acquisition unit that acquires three-dimensional data including the facility to be monitored, a specification unit that specifies a plurality of components constituting the facility to be monitored, a display control unit that rotatably displays the facility to be monitored in the facility data display area of the display device and displays a plurality of procedures to be executed by an operator related to the facility to be monitored in the procedure display area of the display device, and an input unit that accepts selection of at least one component displayed in the facility data display area or at least one procedure displayed in the procedure display area. The display control unit highlights and displays the procedure associated with the selected at least one component or the component associated with the selected at least one procedure, and includes a management device.

[0008] The display method according to the third aspect of the present disclosure acquires three-dimensional data including a facility to be monitored, Identify a plurality of components that make up the equipment to be monitored, rotatably display the equipment to be monitored in the equipment data display area of the display unit, display a plurality of procedures that an operator should execute for the equipment to be monitored in the procedure display area of the display unit, accept selection of at least one component displayed in the equipment to be monitored data display area or at least one procedure displayed in the procedure display area, and highlight and display the procedure associated with the selected at least one component or the component associated with the selected at least one procedure.

[0009] The program according to the fourth aspect of the present disclosure acquires three-dimensional data including equipment to be monitored, Identify a plurality of components that make up the equipment to be monitored, rotatably display the equipment to be monitored in the equipment data display area of the display unit, display a plurality of procedures that an operator should execute for the equipment to be monitored in the procedure display area of the display unit, accept selection of at least one component displayed in the equipment to be monitored data display area or at least one procedure displayed in the procedure display area, and cause the computer to highlight and display the procedure associated with the selected at least one component or the component associated with the selected at least one procedure.

Advantages of the Invention

[0010] According to the present disclosure, it is possible to provide a management device, a display system, a display method, and a program that can easily confirm procedures related to equipment that is not within the operator's field of view and requires work.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0012] (Embodiment 1) Hereinafter, a configuration example of the management device 10 will be described with reference to FIG. 1. The management device 10 may be a computer device that operates by a processor executing a program stored in a memory. The management device 10 may be, for example, a server device.

[0013] The management device 10 has an acquisition unit 11, a specification unit 12, a display control unit 13, and an input unit 14. The acquisition unit 11, the specification unit 12, the display control unit 13, and the input unit 14 may be software or modules in which processing is executed by a processor executing a program stored in a memory. Alternatively, the acquisition unit 11, the specification unit 12, the display control unit 13, and the input unit 14 may be hardware such as a circuit or a chip.

[0014] The acquisition unit 11 acquires 3D data including the equipment to be monitored. The equipment to be monitored may be equipment for which an operator performs monitoring, inspection, or maintenance work regularly or at any timing. The maintenance work includes work such as repair of the equipment to be monitored and replacement of parts. The equipment to be monitored may be, for example, a substation or equipment such as a substation, or other equipment that requires monitoring or management.

[0015] The 3D data may be, for example, point cloud data which is a set of points specified using 3D coordinates. The 3D data may be generated, for example, using a device that measures the distance to an object. The device that measures the distance to an object may be, for example, a LiDAR (Light Detection And Ranging) device. The LiDAR device measures the distance to an object using the ToF (Time of Flight) method. Further, the LiDAR device generates points indicating the shape of the object based on the measurement results. The set of points indicating the shape of the object becomes the point cloud data. The points indicating the shape of the object may be specified using 3D coordinates in a predetermined space. In other words, the points indicating the shape of the object may be shown using 3D coordinates in a predefined coordinate system.

[0016] Alternatively, the 3D data may be generated based on a plurality of captured images of the equipment to be monitored. For example, a device that generates 3D data may generate 3D data of the equipment to be monitored from a plurality of captured images using SfM (Structure from Motion).

[0017] Alternatively, the 3D data may be generated based on drawing data or the like showing the structure of the equipment to be monitored. For example, the generation of 3D data based on drawing data or the like showing the structure of the equipment to be monitored may be executed by an application or the like.

[0018] The acquisition unit 11 may receive 3D data from other devices via a network. Alternatively, the acquisition unit 11 may acquire 3D data from other devices via a wired cable or a wireless communication line. Alternatively, the acquisition unit 11 may generate 3D data as a LiDAR device.

[0019] The specifying unit 12 specifies a plurality of components that make up the facility to be monitored. The component may be a part of the article or device that makes up the facility to be monitored. For example, when the facility to be monitored is a substation, the component may be a cable, a insulator, etc. Or the component may be a transformer, a circuit breaker, etc.

[0020] The specifying unit 12 may specify a plurality of components that make up the facility to be monitored, for example, using a learning model that takes the 3D data of the facility to be monitored as input and outputs the components that make up the facility to be monitored. The learning model may be an AI (Artificial Intelligence) model, a machine learning model, or the like. The learning model may be a trained model that has been trained in advance using 3D data and 3D data or image data related to the components. The learning model may be generated in the management device 10, or may be generated in a device different from the management device 10.

[0021] Alternatively, the specifying unit 12 may divide the facility to be monitored, which is 3D data, into a plurality of components using semantic segmentation. In semantic segmentation, for example, each pixel constituting an image showing 3D data may be labeled. Or in semantic segmentation, each point constituting a point cloud that is 3D data may be labeled. The label may be, for example, information indicating the name of the component. Or the label may be identification information of the component such as an ID (Identification or Identifier).

[0022] The display control unit 13 rotatably displays the equipment to be monitored in the equipment data display area 21 of the display unit 20. Further, the display control unit 13 displays a plurality of procedures to be executed by the operator on the equipment to be monitored in the procedure display area 22 of the display unit 20.

[0023] The display unit 20 may be, for example, a display that displays image data. The display unit 20 may be used integrally with the management device 10. Alternatively, the display unit 20 may be a device different from the management device 10, for example, a display device. In this case, the management device 10 and the display device may be connected via a cable or the like, or may be connected via a network.

[0024] As shown in FIG. 2, the equipment data display area 21 is a partial area of the screen constituting the display unit 20. The procedure display area 22 is also a partial area of the screen constituting the display unit 20. The equipment data display area 21 and the procedure display area 22 may be areas of the same size, or may be areas of different sizes. Further, the equipment data display area 21 and the procedure display area 22 may be displayed on the left and right sides of the display unit 20 as shown in FIG. 2, or may be displayed on the upper and lower sides of the display unit 20.

[0025] The rotatable display of the equipment to be monitored means that the shape of the equipment to be monitored, which is three-dimensional data, is displayed when viewed from viewpoints in various directions or at various angles in the equipment data display area 21. In the equipment data display area 21, the equipment to be monitored may be rotated about at least one of the X-axis, Y-axis, and Z-axis representing three-dimensional coordinates. Alternatively, in the equipment data display area 21, the equipment to be monitored may be rotated about an arbitrary rotation axis set in the three-dimensional space represented by the three-dimensional coordinates.

[0026] The plurality of procedures that an operator should execute in the equipment to be monitored may be procedures indicating, for example, monitoring operations, inspection operations, maintenance operations, etc. of the equipment to be monitored. In the procedure display area 22, data explaining the procedures may be displayed. The data explaining the procedures may include, for example, at least one of text data, image data, and video data. The operator may cause the desired procedure to be displayed in the procedure display area 22 by scrolling the procedures displayed in the procedure display area 22. The plurality of procedures may include procedures for each different type of part, or may include procedures for each part of the same type but different in version, size, etc.

[0027] The equipment data display area 21 and the procedure display area 22 may be simultaneously displayed on the display unit 20. Or, the equipment data display area 21 and the procedure display area 22 may be sequentially displayed on the display unit 20. For example, first, the equipment data display area 21 may be displayed on the display unit 20, and then the procedure display area 22 may be displayed. The equipment data display area 21 and the procedure display area 22 may be of the same size, or one may be larger than the other. The areas of the equipment data display area 21 and the procedure display area 22 may be variable. Or, the equipment data display area 21 and the procedure display area 22 may display data of the same resolution, or each may display data of different resolutions.

[0028] The input unit 14 accepts the selection of at least one part displayed in the equipment data display area 21 or at least one procedure displayed in the procedure display area 22. The input unit 14 may accept the selection of a part or a procedure, for example, by a touch operation on the touch panel by the operator. Or, the input unit 14 may accept a part or a procedure by a button operation by the operator. Or, the input unit 14 may accept the selection of a part or a procedure by voice, line of sight, etc.

[0029] The display control unit 13 highlights and displays the procedure associated with at least one selected component, or the component associated with at least one selected procedure. For example, the display control unit 13 may display the color of the procedure associated with at least one selected component using a color different from that of other procedures. Alternatively, the display control unit 13 may display the procedure associated with at least one selected component larger than other procedures. Alternatively, the display control unit 13 may display only the procedure associated with at least one selected component in the procedure display area 22. Alternatively, the display control unit 13 may blink the procedure associated with at least one selected component. Alternatively, the display control unit 13 may display the procedure display area 22 for displaying the procedure associated with at least one selected component larger than the facility data display area 21. Alternatively, the display control unit 13 may increase the resolution of the data displayed in the procedure display area 22 for displaying the procedure associated with at least one selected component higher than the resolution of the data displayed in the facility data display area 21.

[0030] Alternatively, the display control unit 13 may display the color of the component associated with at least one selected procedure using a color different from that of other components. Alternatively, the display control unit 13 may display the component associated with at least one selected procedure larger than other components. Alternatively, the display control unit 13 may display only the component associated with at least one selected procedure in the facility data display area 21. Alternatively, the display control unit 13 may blink the component associated with at least one selected procedure. Alternatively, the display control unit 13 may display the facility data display area 21 for displaying the component associated with at least one selected procedure larger than the procedure display area 22. Alternatively, the display control unit 13 may increase the resolution of the data displayed in the facility data display area 21 for displaying the component associated with at least one selected procedure higher than the resolution of the data displayed in the procedure display area 22.

[0031] Next, the flow of the display process executed in the management device 10 will be described with reference to FIG. 3. First, the acquisition unit 11 acquires three-dimensional data including the facility to be monitored (S11). Next, the identification unit 12 identifies a plurality of components that make up the facility to be monitored (S12). Next, the display control unit 13 rotatably displays the facility to be monitored in the facility data display area of the display unit 20, and displays a plurality of procedures that an operator should execute regarding the facility to be monitored in the procedure display area of the display unit 20 (S13). Next, the input unit 14 accepts the selection of at least one component displayed in the facility data display area or at least one procedure displayed in the procedure display area (S14). Next, the display control unit 13 highlights and displays the procedure associated with at least one component, or the component associated with the selected at least one procedure (S15).

[0032] As described above, the management device 10 rotatably displays the facility to be monitored in the facility data display area 21 of the display unit 20. Thereby, the operator can confirm the components of the facility to be monitored in a place that is not within the field of view in the facility data display area 21. Further, the management device 10 can accept the selection of the components of the facility to be monitored in a place that is not within the operator's field of view. As a result, the management device 10 can highlight and display the procedure associated with the components of the facility to be monitored in a place that is not within the operator's field of view.

[0033] Also, the management device 10 may display the components of the facility to be monitored in a place that is not within the operator's field of view by accepting the selection of a procedure. Thereby, the operator can recognize the component associated with the selected procedure.

[0034] (Embodiment 2) Next, a configuration example of the management device 30 will be described with reference to FIG. 4. In FIG. 4, an example in which the management device 10 and the display unit 20 in FIG. 1 are integrally configured will be described. Further, the management device 30 has a configuration in which a management unit 31 is added to the management device 10. In the following description, detailed descriptions of the same processes, operations, functions, etc. as those in FIG. 1 will be omitted.

[0035] The specific unit 12 divides the point cloud data indicating the equipment to be monitored received from the acquisition unit 11 into a plurality of components that make up the equipment to be monitored using a learning model or semantic segmentation. Further, the specific unit 12 assigns identification information to each component based on the shape, size, color of each component, the position of the component in the equipment to be monitored, etc. For example, the specific unit 12 assigns identification information to each component to the point cloud data with a label indicating the type of component by semantic segmentation. Specifically, the specific unit 12 assigns identification information for identifying which part of the equipment to be monitored the cable is installed to the point cloud data with a label for the cable. In other words, the specific unit 12 may further assign identification information at the product unit level, such as ID_1, ID_2, etc., to the point cloud data with a label for the cable. The product unit may also be referred to as the article unit.

[0036] The specific unit 12 may assign identification information for each component to the point cloud data of the labeled component using a learning model that takes as input the shape, size, color, position of each component in the equipment to be monitored, etc., and outputs a component with identification information assigned thereto. The learning model may be a trained model that has been pre-trained using the shape, size, color, position of each component in the equipment to be monitored, and the identification information.

[0037] Here, the management information managed by the management unit 31 will be described with reference to FIGS. 5 and 6. FIG. 5 shows management information in which component names and IDs are associated. The component name may be, for example, the name at the product unit level. The ID may be identification information assigned to each component at the product unit level. FIG. 5 shows that ID_1 is associated with cable_1 and ID_2 is associated with cable_2. Further, FIG. 5 shows that ID_3 is associated with gasket_1 and ID_4 is associated with gasket_2.

[0038] Figure 6 shows management information in which IDs are associated with work instructions. Assume that the work instructions are defined for each component. Work instructions 1 to 4 shown in Figure 6 indicate that the work instructions have different contents. Figure 7 shows that work instruction 1 is associated with ID_1 and work instruction 2 is associated with ID_2. Furthermore, Figure 6 shows that work instruction 3 is associated with ID_3 and work instruction 4 is associated with ID_4. Alternatively, the same work instruction may be associated with IDs ID_1 and ID_2, which are components of the same type.

[0039] As shown in FIGS. 5 and 6, the management unit 31 may separately manage management information associating component names with IDs and management information associating IDs with work instructions, or may manage one piece of management information associating component names, IDs, and work instructions. Furthermore, the management unit 31 may manage data indicating work instructions.

[0040] For example, the management unit 31 may manage the management information using a database. For example, the management unit 31 may record or store the management information in a memory or the like within the management device 30. "Manage" may be replaced with "record", "store", or "hold", etc.

[0041] Returning to FIG. 4, the display control unit 13 displays the monitored facility including the component with an ID in the facility data display area 21. The input unit 14 accepts a selection of a component with an ID. For example, an operator may select a component by tapping the facility data display area 21 displayed on a tablet-type terminal. Alternatively, the operator may select a component by operating a VR controller. Alternatively, the operator may select a component using a mouse as an input device.

[0042] The input unit 14 outputs the ID assigned to the selected component to the display control unit 13. The display control unit 13 specifies the procedure manual associated with the received ID using the management information managed by the management unit 31. The display control unit 13 displays the specified procedure manual in the procedure display area 22. The display control unit 13 may display the specified procedure manual separately from other procedure manuals. For example, the display control unit 13 may display the specified procedure manual in a highlighted manner.

[0043] Also, before receiving the selection of a component, the input unit 14 may receive the input of instruction information for rotating the display of the facility to be monitored. For example, the input unit 14 may receive information indicating the amount of rotation along the X-axis, Y-axis, and Z-axis.

[0044] Here, the information displayed in the facility data display area 21 and the procedure display area 22 will be described with reference to FIG. 7. In the facility data display area 21 of FIG. 7, point cloud data representing the facility to be monitored and the X-axis, Y-axis, and Z-axis are shown. The operator may, for example, select the arrows indicated by dotted lines on the X-axis, Y-axis, and Z-axis to rotate the facility to be monitored. For example, the operator may trace the arrows indicated by dotted lines on the X-axis, Y-axis, and Z-axis on the screen to determine the amount of rotation about each axis. The facility to be monitored displayed in the facility data display area 21 is rotated and displayed according to the amount of rotation of each axis input by the operator. Alternatively, the input unit 14 may receive from the operator the input of information specifying the rotation axis from among the X-axis, Y-axis, and Z-axis and information indicating the amount of rotation about the specified rotation axis. An area for inputting the rotation axis and the amount of rotation may be provided at an arbitrary position within the display unit 20.

[0045] The procedure display area 22 of FIG. 7 contains a plurality of procedure manuals.

[0046] Next, the display of the procedure manual associated with the selected component will be described with reference to FIG. 8. In FIG. 8, the arrow 25 in the facility data display area 21 indicates the component selected by the operator. Specifically, it is assumed that the cable indicated at the tip of the arrow 25 is selected by the operator.

[0047] When the cable indicated at the tip of the arrow 25 displayed in the facility data display area 21 is selected, the procedure manual associated with the ID of the selected cable is displayed in the procedure display area 22. Only the procedure manual associated with the ID of the selected cable is displayed in the procedure display area 22. That is, by displaying only the procedure manual associated with the ID of the selected cable in the procedure display area 22, the procedure manual associated with the ID of the selected cable is emphasized. The procedure manual displayed in the procedure display area 22 is not limited to the display in FIG. 8. For example, a plurality of procedure manuals may be displayed in the procedure display area 22, and the procedure manual associated with the ID of the selected cable may be emphasized and displayed so as to be distinguishable from other procedure manuals in terms of color or size.

[0048] Alternatively, as shown in FIG. 9, the procedure manual associated with the ID of the selected cable may be displayed in an enlarged manner. In other words, when a component is selected in the facility data display area 21, the facility data display area 21 may be reduced and the procedure display area 22 may be enlarged.

[0049] Next, the flow of the display process in the management device 30 will be described with reference to FIG. 10. First, the acquisition unit 11 acquires point cloud data representing the facility to be monitored (S21). When a LiDAR device is used as the acquisition unit 11, acquiring the point cloud data may mean generating the point cloud data.

[0050] Next, the specifying unit 12 divides the point cloud data into a plurality of components that make up the facility to be monitored (S22). The specifying unit 12 may divide, for example, the point cloud data indicating the facility to be monitored into a plurality of components that make up the facility to be monitored using a learning model or semantic segmentation. Dividing into a plurality of components may mean specifying the type of component for each point included in the point cloud data.

[0051] Next, the specifying unit 12 assigns identification information to the divided components (S23). The specifying unit 12, for example, further assigns identification information to each component for the point cloud data to which labels indicating the type of component are assigned by semantic segmentation. Each component may be, for example, rephrased as for each product of that component.

[0052] Next, the display control unit 13 rotatably displays the point cloud data indicating the facility to be monitored in the facility data display area 21. Further, the display control unit 13 displays a procedure manual related to the facility to be monitored in the procedure display area 22 (S24). The X-axis, Y-axis, and Z-axis may further be displayed in the facility data display area 21. The operator may input information indicating the amount of rotation of the X-axis, Y-axis, and Z-axis displayed in the facility data display area 21. In response to the information input by the operator, the point cloud data indicating the facility to be monitored is rotated and displayed in the facility data display area 21.

[0053] Next, the input unit 14 accepts the selection of at least one component displayed in the facility data display area 21 (S25). Identification information is assigned to each component. For example, the operator may select the component displayed in the facility data display area 21 by a tap operation or the like. The input unit 14 receives the operation information by the operator as input information.

[0054] The display control unit 13 identifies the procedure manual associated with the selected component using the management information managed by the management unit 31 (S26). The display control unit 13 may identify the procedure manual associated with the identification information assigned to the selected component from the database managed by the management unit 31. Further, the display control unit 13 may receive the data of the identified procedure manual from the management unit 31.

[0055] Next, the display control unit 13 highlights and displays the identified procedure manual in the procedure display area 22 of the display unit 20 (S27). Highlighting and displaying may mean, for example, displaying the identified procedure manual larger than other procedure manuals, or displaying it using a different color. Or, highlighting and displaying may mean displaying only the identified procedure manual.

[0056] As described above, the management device 30 can rotatably display the point cloud data of the equipment to be monitored in the equipment data display area 21. Thereby, the operator can select, in the equipment data display area 21, components of the equipment to be monitored that cannot be visually recognized from the current position.

[0057] Furthermore, the management device 30 highlights and displays the procedure manual associated with the component selected in the equipment data display area 21. Thereby, the operator can easily grasp the procedure to be performed.

[0058] Also, in the second embodiment, mainly an example is described in which the procedure associated with at least one component selected in the equipment data display area 21 is highlighted and displayed in the procedure display area 22, but it is not limited to this. For example, the component associated with at least one procedure selected in the procedure display area 22 may be highlighted and displayed in the equipment data display area 21.

[0059] FIG. 11 is a block diagram showing a configuration example of the management device 10 and the management device 30 (hereinafter referred to as the management device 10 etc.). Referring to FIG. 11, the management device 10 etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) compliant with the IEEE 802.3 series. IEEE represents the Institute of Electrical and Electronics Engineers.

[0060] The processor 1202 reads and executes software (computer program) from the memory 1203 to perform the processing of the RU device 10 etc. described in the above embodiments using the flowchart. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include a plurality of processors.

[0061] The memory 1203 is composed of a combination of a volatile memory and a non-volatile memory. The memory 1203 may include storage located away from the processor 1202. In this case, the processor 1202 may access the memory 1203 via an I / O (Input / Output) interface (not shown).

[0062] In the example of FIG. 11, the memory 1203 is used to store a group of software modules. The processor 1202 can perform the processing of the management device 10 etc. described in the above embodiments by reading and executing these groups of software modules from the memory 1203.

[0063] As described with reference to FIG. 11, each of the processors included in the management device 10 etc. executes one or a plurality of programs including a group of instructions for causing a computer to perform the algorithms described with the drawings.

[0064] In the above example, when the program is loaded into a computer, it includes a set of instructions (or software code) for causing the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disc, or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0065] As described above, the present disclosure has been described with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. And each embodiment can be combined with other embodiments as appropriate.

[0066] The drawings are merely illustrative for explaining one or more embodiments. Each drawing may be associated with not only one specific embodiment but also one or more other embodiments. As those skilled in the art can understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments that are not explicitly illustrated or described. Not all of the features or steps shown in any one drawing for explaining exemplary embodiments are necessarily essential, and some features or steps may be omitted. The order of the steps described in any drawing may be changed as appropriate.

[0067] Some or all of the above embodiments may be described as follows, but are not limited thereto. (Appendix 1) An acquisition unit that acquires three-dimensional data including the equipment to be monitored; An identification unit that identifies a plurality of components constituting the equipment to be monitored; A display control unit that rotatably displays the equipment to be monitored in the equipment data display area of the display unit and displays a plurality of procedures to be executed by an operator regarding the equipment to be monitored in the procedure display area of the display unit; An input unit that receives a selection of at least one component displayed in the equipment data display area or at least one procedure displayed in the procedure display area, and includes: The display control unit is A management device that highlights and displays the procedure associated with the selected at least one component or the component associated with the selected at least one procedure. (Appendix 2) The input unit is The management device according to Appendix 1, which receives a selection of the viewing direction of the equipment to be monitored in order to rotate the equipment to be monitored displayed in the equipment data display area. (Appendix 3) The input unit is The management device according to appended note 1 or 2, which receives an input of a scrolling operation of a plurality of the procedures in order to select one procedure to be displayed in the procedure display area. (Appended note 4) The display control unit The management device according to any one of appended notes 1 to 3, which displays the facility data display area and the procedure display area on one screen. (Appended note 5) The display control unit The management device according to any one of appended notes 1 to 4, which displays the procedure display area for displaying the procedure associated with the at least one selected part larger than the facility data display area, or displays the facility data display area for displaying the part associated with the at least one selected procedure larger than the procedure display area. (Appended note 6) The management device according to any one of appended notes 1 to 5, further comprising a management unit that manages by associating the identification information of the part with the procedure. (Appended note 7) The specific part Identifies the identification information of each of the parts The display control unit The management device according to appended note 6, which extracts the procedure managed in association with the identification information of the part based on the identification information of the at least one selected part. (Appended note 8) A display device that displays a facility data display area including the facility to be monitored and a procedure display area including a plurality of procedures to be executed by an operator regarding the facility to be monitored An acquisition unit that acquires three-dimensional data including the equipment to be monitored, a specifying unit that specifies a plurality of components constituting the equipment to be monitored, a display control unit that rotatably displays the equipment to be monitored in the equipment data display area of the display device and displays a plurality of procedures to be executed by an operator regarding the equipment to be monitored in the procedure display area of the display device, and an input unit that accepts selection of at least one component displayed in the equipment data display area or at least one procedure displayed in the procedure display area, wherein the display control unit highlights and displays the procedure associated with the selected at least one component or the component associated with the selected at least one procedure, a management device, a display system comprising. (Appendix 9) Acquire three-dimensional data including the equipment to be monitored, Specify a plurality of components constituting the equipment to be monitored, Rotatably display the equipment to be monitored in the equipment data display area of the display unit, Display a plurality of procedures to be executed by an operator regarding the equipment to be monitored in the procedure display area of the display unit, Accept selection of at least one component displayed in the equipment data display area of the equipment to be monitored or at least one procedure displayed in the procedure display area, A display method of highlighting and displaying the procedure associated with the selected at least one component or the component associated with the selected at least one procedure. (Appendix 10) Acquire three-dimensional data including the equipment to be monitored, Specify a plurality of components constituting the equipment to be monitored, Rotatably display the equipment to be monitored in the equipment data display area of the display unit, Display a plurality of procedures to be executed by an operator regarding the equipment to be monitored in the procedure display area of the display unit, Accept selection of at least one component displayed in the equipment data display area or at least one procedure displayed in the procedure display area, A program that causes a computer to emphasize and display the procedure associated with the selected at least one component or the component associated with the selected at least one procedure.

[0068] Some or all of the elements (e.g., configurations and functions) described in Appendices 2 to 7 that are subordinate to Appendix 1 may be subordinate to Appendices 8 to 10 in the same subordinate relationship as Appendices 2 to 7. Some or all of the elements described in any appendix may be applied to various hardware, software, recording means for recording software, systems, and methods.

Explanation of Reference Signs

[0069] 10 Management device 11 Acquisition unit 12 Identification unit 13 Display control unit 14 Input unit 20 Display unit 21 Facility data display area 22 Procedure display area 30 Management device 31 Management section

Claims

1. An acquisition unit that acquires point cloud data as three-dimensional data including a facility to be monitored; A specifying unit that specifies the plurality of components by assigning labels indicating the names of the plurality of components constituting the facility to be monitored to each point constituting the point cloud data using semantic segmentation; A display control unit that rotatably displays the facility to be monitored in a facility data display area of a display unit and displays a plurality of procedures to be executed by an operator regarding the facility to be monitored in a procedure display area of the display unit; An input unit that accepts selection of at least one component displayed in the facility data display area or at least one procedure displayed in the procedure display area, and The display control unit, Highlights and displays the procedure associated with the selected at least one component or the component associated with the selected at least one procedure, The input unit, In order to rotate the facility to be monitored displayed in the facility data display area, by an operator operating an arrow shown on the rotation axis of the facility to be monitored, it is possible to select the rotation axis of the facility to be monitored and adjust the amount of rotation, A management device.

2. The input unit, Accepts input of a scroll operation of the plurality of procedures to select one procedure displayed in the procedure display area. The management device according to claim 1.

3. The display control unit, Displays the facility data display area and the procedure display area on one screen. The management device according to claim 1 or 2.

4. The display control unit, Displays the procedure display area for displaying the procedure associated with the selected at least one component larger than the facility data display area, or displays the facility data display area for displaying the component associated with the selected at least one procedure larger than the procedure display area. The management device according to claim 1 or 2.

5. The management device according to claim 1 or 2, further comprising a management unit that manages the identification information of the component and the procedure in association with each other.

6. The specifying unit, Specifies the identification information of each of the components, The display control unit, Extracts the procedure managed in association with the identification information of the component based on the identification information of the selected at least one component. The management device according to claim 5.

7. A display device that displays an equipment data display area including the equipment to be monitored and a procedure display area including a plurality of procedures to be executed by an operator regarding the equipment to be monitored. An acquisition unit that acquires point cloud data as three-dimensional data including the equipment to be monitored, a specification unit that specifies the plurality of components by assigning labels indicating the names of the plurality of components constituting the equipment to be monitored to each point constituting the point cloud data using semantic segmentation, a display control unit that rotatably displays the equipment to be monitored in the equipment data display area of the display device and displays a plurality of procedures to be executed by an operator regarding the equipment to be monitored in the procedure display area of the display device, and an input unit that accepts selection of at least one component displayed in the equipment data display area or at least one procedure displayed in the procedure display area. The display control unit highlights and displays the procedure associated with the selected at least one component or the component associated with the selected at least one procedure. The input unit enables selection of the rotation axis and adjustment of the rotation amount of the equipment to be monitored by an operator operating an arrow shown on the rotation axis of the equipment to be monitored in order to rotate the equipment to be monitored displayed in the equipment data display area. A management device comprising a display system.

8. Acquire point cloud data as three-dimensional data including the equipment to be monitored. Specify the plurality of components by assigning labels indicating the names of the plurality of components constituting the equipment to be monitored to each point constituting the point cloud data using semantic segmentation. Rotatably display the equipment to be monitored in the equipment data display area of the display unit. Display a plurality of procedures to be executed by an operator regarding the equipment to be monitored in the procedure display area of the display unit. By an operator operating an arrow shown on the rotation axis of the equipment to be monitored, it is possible to select the rotation axis and adjust the rotation amount of the equipment to be monitored in order to rotate the equipment to be monitored displayed in the equipment data display area. Accept selection of at least one component displayed in the equipment data display area or at least one procedure displayed in the procedure display area. A display method for highlighting and displaying the procedure associated with the selected at least one component or the component associated with the selected at least one procedure. **Claim 9** Obtain point cloud data as three-dimensional data including the equipment to be monitored, Identify the plurality of components by assigning labels indicating the names of the plurality of components constituting the equipment to be monitored to each point constituting the point cloud data using semantic segmentation, Rotatably display the equipment to be monitored in the equipment data display area of the display unit, Display a plurality of procedures to be executed by an operator regarding the equipment to be monitored in the procedure display area of the display unit, In order to rotate the equipment to be monitored displayed in the equipment data display area, an operator operates an arrow shown on the rotation axis of the equipment to be monitored, enabling selection of the rotation axis of the equipment to be monitored and adjustment of the rotation amount, Accept selection of at least one component displayed in the equipment data display area or at least one procedure displayed in the procedure display area, A program for causing a computer to highlight and display the procedure associated with the selected at least one component or the component associated with the selected at least one procedure.

Images