Facility inspection method and facility inspection system
The head-mounted display system addresses the challenge of guiding unskilled workers by superimposing three-dimensional point cloud data with inspection tags, enhancing the accuracy and efficiency of equipment inspections.
Patent Information
- Application Number
- JP2023047638
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-03-24
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-03-24
AI Technical Summary
Existing equipment inspection methods struggle to allow unskilled workers to accurately identify inspection points and maintain efficiency when inspecting multiple locations, as they lack detailed guidance and real-time alignment with the physical environment.
A head-mounted display system that superimposes three-dimensional point cloud data of the equipment onto the real space, displaying inspection tags in a predetermined order, allowing workers to align the data with the environment and perform inspections efficiently, with integrated management of inspection results and real-time feedback on abnormalities.
Enables accurate and efficient equipment inspections by unskilled workers by providing clear inspection points and reducing transcription errors, while improving work efficiency and ensuring comprehensive coverage of inspection tasks.
Smart Images

Figure 0007581406000001 
Figure 0007581406000002 
Figure 0007581406000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a technique for an equipment inspection method and an equipment inspection system for inspecting equipment. [Background technology]
[0002] Conventionally, techniques relating to equipment inspection methods for inspecting equipment have been publicly known, as described in Patent Document 1, for example.
[0003] Patent Document 1 describes a technology for an inspection method that uses an RFID tag attached to an inspection target device, a portable information terminal capable of contactlessly transmitting and receiving data between the RFID tag and the RFID tag, and a management PC capable of storing inspection data from the portable information terminal. In the technology described in Patent Document 1, when the RFID data is read by the portable information terminal, an inspection value input screen corresponding to the inspection target device to which the RFID tag is attached is displayed on the portable information terminal. A worker performing an inspection can input the inspection value of the inspection target device that he or she reads visually from the inspection value input screen. By configuring in this way, the technology described in Patent Document 1 can prevent the occurrence of input errors such as inputting the inspection value into the input screen of another device when writing the inspection value.
[0004] However, in the technology described in Patent Document 1, since an RFID tag is attached to the equipment to be inspected, it is possible to grasp the equipment to be inspected, but it is not possible to grasp more detailed inspection locations (such as which part of the equipment should be inspected). In addition, when there are many equipment to be inspected in the same facility, it is necessary to grasp in advance where the equipment to be inspected is installed in the facility. Therefore, in the technology described in Patent Document 1, there is no problem when an inspection is performed by a relatively skilled worker who understands the inspection locations in advance, but when an inspection is performed by a worker who is not familiar with inspection work, there is a risk of mistakes in the inspection locations and a decrease in the efficiency of the inspection work. Therefore, there is a demand for a technology of an equipment inspection method and an equipment inspection system that allows even an unskilled worker to easily recognize the inspection locations and perform inspection more appropriately. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2010-61262 A Summary of the Invention [Problem to be solved by the invention]
[0006] One aspect of the present disclosure has been made in consideration of the above-described situation, and the problem it aims to solve is to provide an equipment inspection method and an equipment inspection system that are capable of performing equipment inspection appropriately even when there are multiple inspection target locations scattered around. [Means for solving the problem]
[0007] The problem to be solved by one embodiment of the present disclosure has been described above. Next, the means for solving this problem will be described.
[0008] That is, the equipment inspection method according to one embodiment of the present disclosure is Multiple Three-dimensional point cloud data of the equipment to be inspected is displayed on a display unit of a head-mounted display worn by a wearer, and the wearer superimposes the three-dimensional point cloud data on the real space visually recognized by the wearer through the display unit. In this state, the head mounted display fixes the three-dimensional point cloud data displayed on the display unit with respect to the real space. A matching process in which the head mounted display matches the three-dimensional point cloud data with a preset The multiple inspection target facilities A tag display step of displaying tags indicating inspection locations on the display unit so that the order of inspection can be identified, and an inspection step of the wearer inspecting the inspection locations in accordance with the tags, In the matching step, after the real space and the three-dimensional point cloud data are aligned, the display of the three-dimensional point cloud data on the display unit is erased; In the tag display step, only the tag that is to be inspected earliest in a predetermined order among the tags set at uninspected inspection locations is displayed.
[0009] In addition, in one aspect of the present disclosure, in the inspection step, The wearer The tag ofchoice vinegar and a management ledger of the inspection location corresponding to the selected tag is displayed on the display unit. The wearer Inspection results in the management ledger of input vinegar This is what we have in mind.
[0010] In one aspect of the present disclosure, the inspection result input in the inspection process has an abnormality. It was determined that case, The head mounted display, Notification to prompt the user to give priority to the inspection of another inspection point associated with the inspection point having an abnormality of line cormorant The method further includes a notification step.
[0012] In one aspect of the present disclosure, in the tag display step, is a point The display indicates whether the inspection has been completed or not.
[0014] In one aspect of the present disclosure, in the matching step, an operation image for moving the three-dimensional point cloud data is displayed on the display unit, Decoration When the wearer operates the operation image, the three-dimensional point cloud data is moved to a position corresponding to the real space.
[0015] In addition, the equipment inspection system according to one embodiment of the present disclosure Multiple A storage unit that stores three-dimensional point cloud data of the equipment to be inspected and the inspection points of the three-dimensional point cloud data, and a head-mounted display that is provided with the three-dimensional point cloud data and the inspection points are superimposed on the real space. In this state, the three-dimensional point cloud data and the inspection point are fixed to the real space. Display and after the real space and the three-dimensional point cloud data are aligned, the display of the three-dimensional point cloud data is erased. and a display unit, the display unit displaying a preset three-dimensional point cloud data. The multiple inspection target facilities Tags indicating inspection locations can be displayed so that the inspection order can be identified, and when the tags are displayed so that the inspection order can be identified, only the tag set at uninspected inspection locations that has the earliest predetermined inspection order can be displayed. Effect of the Invention
[0016] According to one aspect of the present disclosure, equipment inspection can be performed appropriately even when there are many inspection points scattered around. [Brief description of the drawings]
[0017] [Figure 1] 1 is a schematic diagram showing the overall configuration of an equipment inspection system according to one embodiment of the present invention; [Diagram 2] FIG. 1 is a block diagram showing the overall configuration of an equipment inspection system. [Diagram 3] 1 is a flowchart showing an example of an equipment inspection method. [Figure 4] FIG. 13 is a diagram showing three-dimensional point cloud data displayed on a display unit in a field of view in real space. [Diagram 5] FIG. 13 is a diagram showing a state in which a solid figure for moving three-dimensional point cloud data is displayed. [Figure 6] FIG. 13 is a diagram showing how three-dimensional point cloud data is moved using a solid figure. [Figure 7] A diagram showing the management ledger displayed when a tag is selected. [Figure 8] A diagram showing how a tag indicating the next inspection item is displayed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0018] First, the overall configuration of an equipment inspection system 1 according to one embodiment of the present invention will be described with reference to FIGS.
[0019] The equipment inspection system 1 is for inspecting various types of equipment. There are no particular limitations on the equipment to be inspected by the equipment inspection system 1, and any equipment (various types of infrastructure equipment, factory equipment, etc.) can be the subject of inspection. In this embodiment, as an example, it is assumed that an inspection of a pumping station A will be performed. The equipment inspection system 1 mainly comprises a head mounted display 10 and a management server 20.
[0020] A head mounted display 10 (hereinafter, simply referred to as HMD 10) is worn on the head of an inspector and provides mixed reality to the inspector. The HMD 10 is a glasses-type wearable device. The HMD 10 mainly includes a display unit 11, a sensor unit 12, a communication unit 13, etc.
[0021] The display unit 11 is disposed in front of the inspector's field of vision and provides the inspector with an appropriate image. The display unit 11 is configured, for example, with a transparent liquid crystal display. The inspector can visually recognize the image displayed on the display unit 11 superimposed on the real space seen through the display unit 11.
[0022] The sensor unit 12 is capable of detecting various kinds of information. The sensor unit 12 is composed of at least one sensor capable of detecting various kinds of information. The type of sensor used is arbitrarily selected according to the application of the HMD 10. Examples of sensors used in the sensor unit 12 include a depth sensor, an acceleration sensor, a gyro sensor, and various cameras. The sensor unit 12 can detect the position, orientation, body (hand, etc.) movement, and three-dimensional shape of the surroundings (real space) of the worker wearing the HMD 10. The HMD 10 can detect the movement of the worker's hand using the sensor unit 12 and perform an operation according to the movement. Furthermore, by using the sensor unit 12, the HMD 10 can recognize the surrounding space. This allows the HMD 10 to move the image displayed on the display unit 11, for example, by following the change in viewpoint or movement of the person wearing the HMD 10.
[0023] The communication unit 13 communicates with external devices by an appropriate communication method (for example, the Internet, a mobile communication network, etc.). In this embodiment, the communication unit 13 communicates with a communication unit 25 of the management server 20, which will be described later, and can transmit and receive various information.
[0024] A user of the HMD 10 can turn the power on / off, adjust the volume, etc., by operating various switches provided on the HMD 10. In addition, the user of the HMD 10 can cause various icons and operation screens to be displayed on the display unit 11 in response to a predetermined operation, and can perform various operations related to the HMD 10 by operating the icons, etc.
[0025] Although detailed description will be omitted, the HMD 10 also includes a processor such as a CPU required for various processes, and storage devices such as a RAM and a ROM, as appropriate.
[0026] The management server 20 manages various types of information related to inspection. Any server, such as a physical server or a virtual server, can be used as the management server 20. The management server 20 can be installed, for example, in a management office B where an inspection work manager works. The management server 20 mainly includes a three-dimensional point cloud database 21, a management ledger database 22, a tag database 23, an inspection sequence database 24, a communication unit 25, and the like.
[0027] The three-dimensional point cloud database 21 stores three-dimensional point cloud data of equipment to be inspected (inspection target equipment). The three-dimensional point cloud data of the inspection target equipment is measured in advance and stored in the three-dimensional point cloud database 21. At this time, the three-dimensional point cloud database 21 stores information for identifying the inspection target equipment (e.g., number, equipment name, equipment address, etc.) linked to the three-dimensional point cloud data of the inspection target equipment.
[0028] For example, the three-dimensional point cloud database 21 stores three-dimensional point cloud data of the entire building of pumping station A, which is the equipment to be inspected in this embodiment. The three-dimensional point cloud data includes three-dimensional point cloud data of the building of pumping station A and various equipment (pump P, engine, control panel, etc.) installed in the building. The three-dimensional point cloud data can be measured using an appropriate measuring device (three-dimensional laser scanner, etc.). FIG. 4 shows three-dimensional point cloud data P' of pump P installed in pumping station A as an example.
[0029] The management ledger database 22 stores information (management ledger) related to the inspection contents of the equipment to be inspected. The management ledger database 22 stores information for identifying an inspection item (e.g., a number, a name of the equipment to be inspected, a name of the device to be inspected, a name of the inspection item, etc.) linked to information related to the contents of the inspection item (standard values, measured values, units of measured values, etc.). Fig. 7 shows, as an example, a management ledger D related to a specific inspection item (in the illustrated example, "discharge pressure") called up on the display unit 11 of the HMD 10.
[0030] The tag database 23 stores tags (tag data) set for inspection locations on the three-dimensional point cloud data stored in the three-dimensional point cloud database 21. Here, the tag is information about the inspection location. In this embodiment, the tag includes the name of the inspection item, information (coordinates, etc.) indicating the inspection location on the three-dimensional point cloud data, and information (URL, etc.) for calling up the management ledger corresponding to the inspection location. FIG. 4 shows, as an example, tags T1 and T2 displayed on the display unit 11 (more specifically, on the three-dimensional point cloud data P' of the pump P called up on the display unit 11). As shown in FIG. 4, the tags T1 and T2 called up on the display unit 11 include, for example, the name of the inspection item and are displayed to point out the inspection location.
[0031] The inspection sequence database 24 stores information regarding the order in which a plurality of inspection items (inspection points) should be inspected. The inspection sequence database 24 stores a predetermined inspection order for each piece of equipment to be inspected.
[0032] The communication unit 25 communicates with external devices by an appropriate communication method (for example, the Internet, a mobile communication network, etc.) In this embodiment, the communication unit 25 communicates with the communication unit 13 of the HMD 10 and can transmit and receive various information.
[0033] Although detailed description will be omitted, the management server 20 also includes a processor such as a CPU required for various processes, storage devices such as RAM and ROM, etc. The management server 20 is also provided with a display device such as an LCD monitor, an input device such as a keyboard, etc., which are used by a manager who manages the inspection work, for example.
[0034] Next, an equipment inspection method using the above-mentioned equipment inspection system 1 will be described.
[0035] As a preliminary preparation for the inspection, three-dimensional point cloud data of the entire building, including the exterior and interior of pumping station A, which is the equipment to be inspected in this embodiment, is measured and stored in a three-dimensional point cloud database 21 of the management server 20. In addition, tags are set to the inspection points of the three-dimensional point cloud data, and information about the tags is stored in a tag database 23 of the management server 20.
[0036] 3, an operator performing an inspection operates the HMD 10 to read into the HMD 10 the three-dimensional point cloud data stored in the three-dimensional point cloud database 21 of the management server 20 and the tags stored in the tag database 23. At this time, the operator operates the HMD 10 to select the equipment to be inspected this time (pumping station A) from among data on a plurality of equipment to be inspected stored in the management server 20. The management server 20 transmits the three-dimensional point cloud data of the selected equipment, etc. to the HMD 10. After the process of step S101 is performed, the process proceeds to step S102.
[0037] In step S102, the operator operates the HMD 10 to display the three-dimensional point cloud data and tags on the display unit 11. Fig. 4 shows, as an example, an example in which the three-dimensional point cloud data P' of the pump P and the tags T1 and T2 set at the inspection points on the three-dimensional point cloud data P' are displayed on the display unit 11. Note that in this embodiment, for convenience, only the tags T1 and T2 are shown, but in reality, in addition to these two tags, tags are displayed at all the inspection points of the pumping station A that is the subject of the current inspection.
[0038] As shown in Fig. 4, by displaying the three-dimensional point cloud data P' on the transparent display unit 11, the operator can simultaneously view the pump P in real space seen through the display unit 11 and the three-dimensional point cloud data P'. The three-dimensional point cloud data and tags displayed on the display unit 11 can be optionally switched to not being displayed. After the process of step S102 is performed, the process proceeds to step S103.
[0039] In step S103, the operator operates the HMD 10 to perform alignment (matching) between the real space and the three-dimensional point cloud data. For example, when the three-dimensional point cloud data is displayed on the display unit 11 in step S102, it is assumed that the position of the pump P in the real space and the three-dimensional point cloud data P' of the pump P are misaligned as shown in Fig. 4. Therefore, in step S103, an operation to align the positions of the two is performed.
[0040] Specifically, the operator operates the HMD 10 to display a three-dimensional figure M as shown in Fig. 5 on the display unit 11. The three-dimensional figure M is composed of an arrow figure M1 indicating three mutually orthogonal directions (X direction, Y direction, and Z direction) and a spherical figure M2 having a scale.
[0041] The solid figure M is for adjusting the position of the three-dimensional point cloud data displayed on the display unit 11. The worker can grab the solid figure M on the display unit 11 with his / her hand and move it (changing the position in the X-direction, Y-direction, and Z-direction, and rotating it). When the solid figure M is moved, the three-dimensional point cloud data also moves so as to follow the solid figure M, as shown in FIG. 6. The worker moves the solid figure M so that the three-dimensional point cloud data matches the real space. For example, the worker can match the three-dimensional point cloud data with the real space by using the pump P as a landmark and moving the solid figure M so that the position of the pump P matches the position of the three-dimensional point cloud data P' of the pump P.
[0042] Although not shown, in addition to the solid figure M, an operation menu (operation panel, numerical input section, icon, etc.) for moving the three-dimensional point cloud data can also be displayed on the display section 11. For example, precise alignment can be performed by making it possible to input the amount of movement of the three-dimensional point cloud data (displacement amount in X, Y, Z directions, and rotation amount) numerically using the numerical input section. Also, for example, the size of the three-dimensional point cloud data can be adjusted using an operation panel that can change the scale of the three-dimensional point cloud data.
[0043] Also, for example, by using the operation panel, it is possible to instantly move to any location on the 3D point cloud data (in other words, to instantly display any location on the 3D point cloud data). This allows efficient alignment even for 3D point cloud data of relatively large-scale facilities.
[0044] After aligning the real space with the three-dimensional point cloud data in step S103, the worker performs a predetermined operation to erase (change to hidden) the three-dimensional point cloud data on the display unit 11. As a result, an image (see FIG. 7) in which tags are attached to the inspection points in the real space is provided on the display unit 11.
[0045] When the worker wearing the HMD 10 moves, the HMD 10 moves the three-dimensional point cloud data internally following the worker's movement. Therefore, there is no deviation between the real space and the position of the tag indicating the inspection point. In this way, the HMD 10 provides a mixed reality that combines the real space and the three-dimensional point cloud data. After the process of step S103 is performed, the process proceeds to step S104.
[0046] In step S104, the worker starts the inspection work by operating the HMD 10. For example, the worker inputs an instruction to start the inspection work to the HMD 10 using an operation menu displayed on the display unit 11 of the HMD 10. After the process of step S104 is performed, the process proceeds to step S105.
[0047] In step S105, the HMD 10 displays tags on the display unit 11 according to the inspection order stored in the inspection order database 24 of the management server 20. Specifically, the HMD 10 refers to the inspection order stored in the inspection order database 24, and displays only the tag corresponding to the earliest inspection item on the display unit 11, and does not display tags corresponding to the other inspection items. Fig. 7 shows an example in which, of two inspection items for a pump P, "discharge pressure" and "vibration," only the tag T1 corresponding to "discharge pressure," which has the earliest inspection order, is displayed.
[0048] Since the inspection points are displayed by tags in this way, the worker can easily grasp the inspection points. With this function, not only experienced workers but also workers who are not yet familiar with inspection work can smoothly perform inspection. In addition, in this embodiment, since only the tag with the earliest inspection order is displayed, the worker can easily grasp the inspection order (the inspection point to be inspected next). In addition to the tag, it is also possible to display an arrow or the like indicating the position of the tag on the display unit 11. This allows the worker to easily find and move to the tag even if the tag is displayed relatively far away (the position of the tag is difficult to grasp), and improves work efficiency. After the process of step S105 is performed, the process proceeds to step S106.
[0049] In step S106, the worker inspects the inspection location and inputs the measurement value using the HMD 10. Specifically, the worker inspects the inspection location indicated by the tag displayed on the display unit 11 in step S105. For example, in the example shown in FIG. 7, the worker measures the "discharge pressure" of the pump P using the instrument indicated by the tag T1. After that, the worker selects the tag T1 using the HMD 10 (for example, by touch operation on the mixed reality). The HMD 10 communicates with the management server 20, calls up the management ledger D corresponding to the inspection location indicated by the selected tag T1, and displays it on the display unit 11. The HMD 10 also displays the keyboard K used for inputting numerical values and the like on the display unit 11 together with the management ledger D. The worker operates the keyboard K displayed on the display unit 11 to input the measurement value into the management ledger D. The measurement value input into the management ledger D is transmitted to the management server 20 and stored in the management ledger database 22. The measurement values input to the management ledger D may be stored in the HMD 10 itself. When the operator has completed (confirmed) input of the measurement values, he / she uses the keyboard K to input that effect.
[0050] If there is any abnormality in the input of the measured value in step S106, it is also possible to notify the fact (by displaying an error, etc.). For example, the measured value may deviate from the standard value by a predetermined threshold value or more, or the inspection order may be incorrect. After the measured value is input in step S106, the HMD 10 or the management server 20 determines whether there is an abnormality, and if an abnormality is found, the operator can be notified of the abnormality by displaying an error on the display unit 11 of the HMD 10, etc. Also, by displaying an error on the liquid crystal monitor of the management server 20 as well as the HMD 10, the administrator can be notified of the abnormality at the same time. The method of notification is not particularly limited, and it is also possible to notify the operator by voice, etc. After the process of step S106 is performed, the process proceeds to step S107.
[0051] In step S107, the HMD 10 judges whether or not all the inspections of the equipment to be inspected this time have been completed. If there are any inspection points that have not been inspected, the process proceeds to step S105. In this manner, the processes of steps S105 and S106 are repeated until the inspections of all the inspection points scheduled in advance have been completed.
[0052] For example, in step S105 following step S107, the HMD 10 displays a tag corresponding to the next inspection item on the display unit 11. As a result, for example, as shown in Fig. 8, tag T2 indicating the next inspection item (inspection location) is displayed, and tag T1 for which inspection has been completed is hidden. In the subsequent step S106, the worker inspects the inspection location indicated by tag T2 and inputs the measurement value.
[0053] In this manner, the processes of steps S105 and S106 are repeated until inspection of all of the previously scheduled inspection points is completed. When inspection of all of the previously scheduled inspection points is completed, the process proceeds from step S107 to step S108.
[0054] In step S108, the worker ends the inspection work by operating the HMD 10. For example, the worker inputs an instruction to end the inspection work to the HMD 10 using an operation menu displayed on the display unit 11 of the HMD 10. This causes the HMD 10 to end the processing related to the inspection work (for example, communication with the management server 20, display of tags, etc.).
[0055] If an inspection has been omitted at the time when the instruction to end the inspection work is input in step S108, it is also possible to notify the operator of the omission (by displaying an error, etc.). For example, after the instruction to end the inspection work is input in step S108, the HMD 10 or the management server 20 determines whether an inspection has been omitted, and if an inspection has been omitted, an error display is displayed on the display unit 11 of the HMD 10 to notify the operator of the omission. If an inspection has been omitted, the process returns to step S105, for example, and the HMD 10 can guide the operator to the omitted inspection location.
[0056] In this manner, in this embodiment, the inspection points can be easily grasped by overlaying the three-dimensional point cloud data on the real space using the HMD 10. This allows not only skilled workers, but also workers who are not familiar with inspection work to easily perform inspections. In addition, since the measurement values can be directly input to the management server 20 (management ledger) using the HMD 10, it is no longer necessary to transcribe the measurement values written on paper, improving work efficiency and preventing transcription errors.
[0057] As described above, the equipment inspection method according to this embodiment includes the following steps: a matching process (step S103) in which three-dimensional point cloud data of the inspection target equipment acquired in advance is displayed on a display unit 11 of an HMD 10 (head mounted display) and the three-dimensional point cloud data is superimposed on a real space visually recognized through the display unit 11; A tag display step (step S105) in which a tag indicating an inspection point in the three-dimensional point cloud data is displayed on the display unit 11; An inspection process (step S106) in which the inspection points are inspected according to the tags; It includes. With this configuration, equipment inspection can be performed effectively even when there are many inspection points scattered around. In other words, since the inspection points can be identified by the tags, not only experienced workers but also workers who are inexperienced can smoothly perform the inspection work.
[0058] In addition, the equipment inspection method according to the present embodiment includes: In the inspection process (step S106), When the tag is selected, the management ledger D of the inspection location corresponding to the selected tag is displayed on the display unit 11, and the inspection results are entered into the management ledger D. With this configuration, the inspection results (measured values) can be easily recorded. Particularly in this embodiment, the measured values input by the worker using the HMD 10 are directly input to the management ledger database 22 of the management server 20, eliminating the need for transcription work, improving work efficiency and preventing transcription errors.
[0059] In addition, the equipment inspection method according to the present embodiment includes: In the matching process (step S103), After the real space and the three-dimensional point cloud data are aligned, the display of the three-dimensional point cloud data on the display unit 11 is erased. This configuration makes it easier to recognize the real space. Also, by not displaying the three-dimensional point cloud data, the processing load on the HMD 10 can be reduced.
[0060] In addition, the equipment inspection method according to the present embodiment includes: In the tag display step (step S105), The tag is displayed so as to enable identification of the inspection order and / or whether or not the inspection has been completed. By configuring in this way, it is possible to easily grasp the order of inspections and whether or not they have been completed.
[0061] In addition, the equipment inspection method according to the present embodiment includes: In the tag display step (step S105), The tags can be switched between visible and invisible according to the order of inspection. With this configuration, it is possible to reduce the number of tags displayed on the display unit 11, and to improve the visibility of the real space.
[0062] In addition, the equipment inspection method according to the present embodiment includes: In the matching process (step S103), A three-dimensional figure M (operation image) for moving the three-dimensional point cloud data is displayed on the display unit 11, and the wearer of the HMD 10 (operator) operates the three-dimensional figure M to move the three-dimensional point cloud data to a position corresponding to the real space. With this configuration, the operator can align the positions of the real space and the three-dimensional point cloud data while checking their positions, thereby enabling efficient alignment.
[0063] In addition, the equipment inspection system 1 according to this embodiment is A storage unit (a three-dimensional point cloud database 21 and a tag database 23) that stores previously acquired three-dimensional point cloud data of an inspection target facility and inspection points of the three-dimensional point cloud data; A display unit 11 provided in an HMD 10 (head mounted display) and capable of displaying the three-dimensional point cloud data and the inspection points in a superimposed manner on a real space; It is equipped with the following. With this configuration, the equipment can be inspected conveniently. That is, since the inspection points displayed on the display unit 11 can be grasped, not only skilled workers but also workers who are unfamiliar with inspection work can smoothly perform the inspection work.
[0064] Although one embodiment of the present invention has been described above, the present invention is not limited to the above configuration, and various modifications are possible within the scope of the invention described in the claims. Modifications of the above embodiment will be described below.
[0065] In the above embodiment, an example was shown in which the worker was allowed to identify the inspection order by switching between displaying and not displaying tags and displaying only the tag of the inspection point to be inspected next, but the present invention is not limited to this, and the method of identifying the inspection order can be changed as desired. For example, it is also possible to display the inspection order in an appropriate manner (e.g., numbers, color coding, etc.) on the tag displayed at each inspection point.
[0066] In the above embodiment, an example is shown in which the order of inspection is identified by tags, but it is also possible to configure the system so that other information can be identified using tags. For example, it is also possible to configure the system so that it can identify whether the inspection has been completed (whether it has been inspected or not) by providing a difference in display (e.g., a difference in color or a difference in display message) between the tags of inspection points that have been inspected and the tags of inspection points that have not yet been inspected.
[0067] In the above embodiment, a predetermined inspection order is stored in the inspection order database 24, and inspection is performed according to this inspection order. However, the inspection order stored in the inspection order database 24 can be determined by any method. For example, it is also possible to configure the inspection order of a series of inspections performed by an operator wearing the HMD 10 to be stored in the inspection order database 24 via the HMD 10, and to guide the operator to perform the inspection in this inspection order from the next inspection onwards. With such a configuration, for example, an expert first performs inspection in a correct (efficient) inspection order, and the order is used from the next inspection onwards, so that even an operator who is not familiar with inspection work can perform the inspection work efficiently. It is also possible to configure the inspection order database 24 to store a plurality of patterns of inspection orders, and to allow the operator to select a desired order.
[0068] Although the above embodiment has been described with one HMD 10 as an example, it is also possible to use multiple HMDs 10 to perform inspection work by multiple workers. For example, by storing the inspection order of each HMD 10 (each worker) in the inspection order database 24 when multiple HMDs 10 are used, it is possible to guide multiple workers to an appropriate inspection order. In this case, by storing the inspection order so that the inspection points of the multiple workers do not overlap with each other and there are no omissions, it is possible to perform the inspection work efficiently.
[0069] In addition, if the measured value input in step S106 in the above embodiment is abnormal (the measured value deviates from the standard value by a predetermined threshold value or more), it is also possible to instruct the operator to give priority to inspecting other inspection points that are correlated with the inspection point. Here, the inspection points being correlated means that the measured values of multiple inspection points are mutually related, and if an abnormality occurs in one, it is highly likely that an abnormality will occur in the other ones as well. For example, the management server 20 associates and stores mutually correlated inspection points in advance, and if an abnormality occurs in the measured value in any of the inspection points, it issues an instruction to prompt the operator to use the HMD 10 to give priority to inspecting other correlated inspection points. This makes it possible to quickly confirm that an abnormality has occurred in a specific piece of equipment.
[0070] Furthermore, if an abnormality is found in the measured values, the location causing the abnormality can be estimated and reported by the HMD 10 or the management server 20. For example, the cause of the abnormality at each inspection point can be estimated by using a cause investigation flow chart or the like prepared by the manufacturer of each piece of equipment. Therefore, if an abnormality is found in the measured values, the cause can be quickly estimated using the cause investigation flow chart or the like, and the abnormality can be quickly resolved.
[0071] Also, if an abnormality is found in the measured values, the fact may be notified at the next inspection onward. For example, the inspection points where the measured values were abnormal are stored in the management server 20, and at the next inspection onward, a message indicating the abnormality is displayed on the tag of the inspection point. By looking at the display on the tag, the worker can recognize that the inspection point is an important inspection point.
[0072] In the above embodiment, an example was shown in which a worker wearing the HMD 10 performs an inspection work using mixed reality, but the state of the inspection work may be simultaneously confirmed on the management server 20. For example, by displaying the mixed reality provided by the HMD 10 in real time on the liquid crystal monitor of the management server 20, the state of the inspection work can be confirmed by related parties other than the worker (such as a manager). In addition, by utilizing communication between the HMD 10 and the management server 20, the worker and the manager can also perform the inspection work in consultation with each other.
[0073] In the above embodiment, an example of performing inspection work using the equipment inspection system 1 has been described, but the method of using the equipment inspection system 1 is not limited to this. For example, it can be used to share various information between a person wearing the HMD 10 and a person operating the management server 20. For example, by sharing a three-dimensional virtual space (metaverse) configured using the three-dimensional point cloud data of the equipment to be inspected stored in the three-dimensional point cloud database 21 between a person wearing the HMD 10 and a person operating the management server 20, it is possible to hold a meeting regarding the inspection of the equipment to be inspected, training on inspection methods, etc. This allows the user to intuitively grasp on-site information using the stereoscopic three-dimensional point cloud data without actually going to the site.
[0074] In the above embodiment, an example has been shown in which the equipment inspection system 1 includes the HMD 10 and the management server 20, but the present invention is not limited to this, and the configuration of the equipment inspection system 1 can be changed as desired. For example, it is also possible to incorporate the functions of the management server 20 into the HMD 10, and to configure the equipment inspection system 1 using the HMD 10 alone. It is also possible to configure the management server 20 with multiple servers, and to share the functions of the management server 20 among the multiple servers.
[0075] The equipment inspection method (see FIG. 3) illustrated in the above embodiment is merely an example, and the contents thereof can be changed as desired. For example, the order of steps (processes) in the above embodiment can be changed, omitted, or added as desired. [Explanation of symbols]
[0076] 1. Equipment Inspection System 10 Head-mounted display 11 Display section 12 Sensor section 20 Management Server 21 3D point cloud database 22 Management ledger database 23 Tag Database 24 Inspection sequence database
Claims
1. a matching process in which three-dimensional point cloud data of a plurality of pieces of equipment to be inspected, which have been acquired in advance, is displayed on a display unit of a head-mounted display worn by a wearer, and the head-mounted display fixes the three-dimensional point cloud data displayed on the display unit to the real space in a state in which the wearer overlays the three-dimensional point cloud data on the real space visually recognized by the wearer through the display unit; a tag display process in which the head mounted display displays tags indicating inspection points of the plurality of inspection target facilities that are set in advance in the three-dimensional point cloud data on the display unit in a manner that allows the order of inspection to be identified; an inspection step in which the wearer inspects the inspection point according to the tag; Including, In the matching step, After the real space and the three-dimensional point cloud data are aligned, the display of the three-dimensional point cloud data on the display unit is erased, In the tag display step, Among the tags set at uninspected inspection locations, only the tag with the earliest predetermined inspection order is displayed. Equipment inspection methods.
2. In the inspection process, When the wearer selects the tag, a management ledger of the inspection location corresponding to the selected tag is displayed on the display unit, and the wearer inputs the inspection result into the management ledger. The facility inspection method according to claim 1.
3. The method further includes a notification step of, when it is determined that there is an abnormality in the inspection result input in the inspection step, making a notification by the head mounted display to prompt the driver to perform inspection of the other inspection points associated with the inspection point having the abnormality with priority. The facility inspection method according to claim 2.
4. In the tag display step, The tag is displayed so as to be able to identify whether the inspection has been completed or not. The facility inspection method according to claim 1.
5. In the matching step, an operation image for moving the three-dimensional point cloud data is displayed on the display unit, and the wearer operates the operation image to move the three-dimensional point cloud data to a position corresponding to the real space; The facility inspection method according to claim 1.
6. A storage unit that stores three-dimensional point cloud data of a plurality of inspection target facilities acquired in advance and inspection points of the three-dimensional point cloud data; a display unit that is provided in the head mounted display, that displays the three-dimensional point cloud data and the inspection location in a state where the three-dimensional point cloud data and the inspection location are superimposed on the real space, and that erases the display of the three-dimensional point cloud data after the real space and the three-dimensional point cloud data are aligned; Equipped with The display unit is Tags indicating the inspection locations of the plurality of pieces of equipment to be inspected that are set in advance in the three-dimensional point cloud data can be displayed in a manner that allows the order of inspection to be identified, and when the tags are displayed in a manner that allows the order of inspection to be identified, only the tag that is set to the uninspected inspection locations and has the earliest predetermined inspection order can be displayed. Equipment inspection system.
Citation Information
Patent Citations
Trouble support method for electric power supply facility
JP2010015206A
Inspection method using RFID
JP2010061262A
Image processing system, visual axis measuring system, image processing method, visual axis measuring method and program
JP2016143321A
Safety management system, and safety management method
JP2016192206A
Maintenance activity support system and maintenance activity support method
JP2020080147A
