Inspection support device, inspection support system, and inspection support method
The inspection support device addresses the challenge of variable positional relationships by creating a single board image for remote inspection, reducing on-site workload and input errors through automated data generation and storage.
Patent Information
- Application Number
- JP2023086852
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-12-06
AI Technical Summary
Existing technologies for remotely reading meter values assume a fixed positional relationship between instruments and cameras, requiring on-site inspection when this relationship changes, increasing workload and potential for input errors.
An inspection support device that creates a single board image from multiple camera shots, analyzes it to acquire inspection values, and reduces manual input by automatically generating inspection data.
Reduces the workload and minimizes input errors by allowing remote inspection of multiple objects, enabling efficient data generation and storage.
Smart Images

Figure 2024169965000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an inspection support device, an inspection support system, and an inspection support method that support the reading of information such as the indicated values of meters and the states of switches provided on a panel to be inspected. [Background technology]
[0002] One example of a technology for assisting in the task of reading a meter reading value is a technology for reading the meter reading value by analyzing a captured image of the meter (for example, Patent Document 1).
[0003] Patent Document 1 discloses a technology in which image processing is performed on images taken with a camera of the indicator needles of the instruments to be read, such as thermometers, pressure gauges, and flow meters, and the obtained image data is analyzed to obtain the indicated values of each instrument. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2022-007681 A Summary of the Invention [Problem to be solved by the invention]
[0005] The technology described in Patent Document 1 assumes that the indicated values of instruments are read remotely, and the number and positions of the instruments from which the indicated values are read do not change. In other words, the acquisition process of indicated values is performed assuming that the relationship (e.g., the positional relationship) between each of the instruments, which are the subjects of the image, and the camera is fixed. For this reason, the technology described in Patent Document 1 has a problem in that it cannot be used under conditions in which the number, type, and position of the instruments to be worked on, and the relationship between the instruments and the camera, can change, such as when a worker goes to the site where the instruments are installed to read the indicated values.
[0006] The present disclosure has been made in consideration of the above, and aims to provide an inspection support device that can reduce the burden of inspection work performed at a site where an object of work is installed. [Means for solving the problem]
[0007] In order to solve the above-mentioned problems and achieve the objectives, the present disclosure provides an inspection support device that supports on-site inspection work of multiple inspection objects installed on a panel, and is characterized by comprising an imaging unit that takes photographs of an area that includes at least a portion of the panel to create multiple images, a panel image creation unit that creates a panel image, which is a single image that includes the entire panel, based on the multiple images created by the imaging unit, and an inspection value acquisition unit that analyzes the panel image to acquire inspection values for the multiple inspection objects. Effect of the Invention
[0008] According to the present disclosure, it is possible to realize an inspection support device that can reduce the burden of inspection work performed at a site where an object of work is installed. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a diagram showing an example of a usage form of an inspection support device according to an embodiment; [Diagram 2] FIG. 1 is a diagram showing an example of an object of inspection work supported by an inspection support device; [Diagram 3] FIG. 1 is a diagram for explaining an example of work support by an inspection support device. [Figure 4] FIG. 1 is a diagram showing a configuration example of an inspection support device and an inspection result storage device of an inspection support system. [Diagram 5] FIG. 13 is a diagram showing an example of board information stored in a board information storage unit of an inspection result storage device. [Figure 6] A diagram showing an example of hardware for implementing an inspection support device. [Figure 7] A flowchart showing an example of an operation of the inspection support device to support an inspection work. [Figure 8] FIG. 1 is a diagram showing an example of the operation of an inspection support device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] An inspection support device, an inspection support system, and an inspection support method according to embodiments of the present disclosure will be described in detail below with reference to the drawings.
[0011] Embodiment FIG. 1 is a diagram showing an example of a usage form of the inspection support device 1 according to the embodiment. As shown in FIG. 1, the inspection support device 1 according to the embodiment is a portable device, and supports the work of a worker 201 who inspects an object. Specifically, the inspection support device 1 photographs objects such as meters, switches, and lamps installed at the site, which are not shown in FIG. 1, to obtain images, and analyzes the images to obtain inspection values of the objects. The inspection value corresponds to the indicated value when the object is a meter, and corresponds to the state of the object when the object is a switch or lamp. Before performing image analysis to obtain the inspection value, the inspection support device 1 performs image processing, etc. as necessary. In this embodiment, it is assumed that a plurality of objects are installed in a state where they are concentrated in a certain range.
[0012] FIG. 2 is a diagram showing an example of an object of inspection work supported by the inspection support device 1. The object of inspection work supported by the inspection support device 1 (hereinafter, sometimes referred to as an inspection object) is, for example, a meter, a lamp, a switch, etc. installed on a panel 301 as shown in FIG. 2. When inspecting the object shown in FIG. 2, a worker 201 using the inspection support device 1 photographs the inspection object arranged on the panel 301 by using a video shooting function of the inspection support device 1. In the video shooting, a moving image of the entire panel 301 is created by changing the shooting range while shooting a part of the panel 301. For example, all the inspection objects are photographed along the route shown in FIG. 3, that is, the route indicated by the dashed arrow. FIG. 3 is a diagram for explaining an example of work support by the inspection support device 1. Note that the worker 201 may use a still image shooting function instead of the video shooting function of the inspection support device 1, and may photograph the entire panel 301 in multiple times by repeatedly shooting still images while changing the shooting range. At this time, multiple inspection objects may be included in one still image.
[0013] Returning to the explanation of FIG. 1, after the inspection support device 1 acquires the inspection values of the inspection object, it creates inspection data based on the inspection values and registers it in the inspection result storage device 2 provided in the cloud. In addition to the inspection data, related information of the inspection object is registered in the inspection result storage device 2. Although details will be described later, the related information indicates the position of the inspection object on the panel 301, etc., and is used in the image analysis process performed by the inspection support device 1 to acquire the inspection values of the inspection object. The inspection support device 1 and the inspection result storage device 2 constitute an inspection support system. Note that the inspection result storage device 2 may be provided somewhere other than the cloud.
[0014] The inspection result storage device 2 provides the inspection results to the user 202. The user 202 is, for example, a manager of a building, a plant, or the like in which an inspection target is installed. The inspection result storage device 2 may process the inspection data stored therein to generate and provide inspection results in a format desired by the user 202. The inspection results in the format desired by the user 202 may be created by the inspection support device 1.
[0015] According to the inspection support device 1, the inspection value is acquired by analyzing the image of the object of the inspection work, so the only work required for the worker 201 is to take a picture of the object, which reduces the workload. Also, input errors that are a problem when the inspection value is input manually can be prevented.
[0016] 4 is a diagram showing a configuration example of the inspection support device 1 and the inspection result storage device 2 of the inspection support system 100. The inspection support system 100 shown in FIG. 4 supports the inspection of multiple objects installed on one panel 301, as exemplified in FIG.
[0017] The inspection support device 1 comprises an imaging unit 11, a board image creation unit 12, an inspection value acquisition unit 13, an inspection data creation unit 14, a display unit 15 and a communication unit 16.
[0018] The photographing unit 11 photographs the object to be inspected and creates a moving image. The board image creation unit 12 creates a board image, which is an image of the board on which the object to be inspected is provided, based on the moving image created by the photographing unit 11. The inspection value acquisition unit 13 acquires the inspection value of the object to be inspected from the board image created by the board image creation unit 12. Examples of the inspection value are the indicated value of a meter, the state of a switch, the state of a lamp, etc. The inspection data creation unit 14 creates inspection data in a predetermined format based on the inspection value acquired by the inspection value acquisition unit 13. For example, the inspection data creation unit 14 creates form data by inserting the inspection value into an electronic form in a format corresponding to the board on which the object to be inspected is provided.
[0019] The inspection result storage device 2 includes a board information storage unit 21, an inspection result storage unit 22, and a communication unit .
[0020] The board information storage unit 21 stores information on a board on which an inspection object is provided. The board information includes information on each of a plurality of inspection objects provided on the board. Hereinafter, the information on the board stored in the board information storage unit 21 is referred to as board information. The board information corresponds to the related information described in FIG. 1, and is used when the inspection support device 1 analyzes the board image to identify the inspection object. The board information is, for example, information having a configuration shown in FIG. 5. FIG. 5 is a diagram showing an example of board information stored in the board information storage unit 21 of the inspection result storage device 2. As shown in FIG. 5, the board information includes a board name, a position, and an equipment name. The "position" is information indicating the position of the inspection object on the board, and the "equipment name" is the name of the inspection object, and also serves as information indicating the type of the inspection object. The board information is information indicating the type of each inspection object and its position on the board. Note that the configuration of the board information is not limited to that shown in FIG. 5. The board information may be configured to include information that can identify the inspection object included in the board image. For example, the board information may be a combination of an image of the entire board and the order and type of the inspection objects. The board information storage unit 21 holds multiple types of board information, and when the inspection support device 1 supports an inspection operation, it provides board information corresponding to the board to be inspected to the inspection support device 1. The board information to be provided to the inspection support device 1 is transmitted from the communication unit 23 to the inspection support device 1.
[0021] The inspection result storage unit 22 stores the inspection data created by the inspection support device 1, and provides the stored inspection data to the user 202 in response to a request from the user 202.
[0022] Next, a description will be given of a hardware configuration of the inspection support device 1. Fig. 6 is a diagram showing an example of hardware that realizes the inspection support device 1. The inspection support device 1 can be realized by a processor 101, a storage device 102, a display device 103, a communication device 104, and an imaging device 105 shown in Fig. 6.
[0023] The processor 101 is a CPU (Central Processing Unit, also called a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, or DSP (Digital Signal Processor)), a system LSI (Large Scale Integration), etc. The storage device 102 is a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (registered trademark) (Electrically Erasable Programmable Read Only Memory), a hard disk drive, etc. The display device 103 is a liquid crystal monitor, a display, etc. The communication device 104 is a wireless communication module, a wireless LAN (Local Area Network) adapter, etc. The imaging device 105 is a CCD (Charge Coupled Device) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, etc.
[0024] The photographing unit 11 of the inspection support device 1 is realized by the imaging device 105. The display unit 15 of the inspection support device 1 is realized by the display device 103. The communication unit 16 of the inspection support device 1 is realized by the communication device 104.
[0025] Furthermore, the board image creation unit 12, the inspection value acquisition unit 13, and the inspection data creation unit 14 of the inspection support device 1 are realized by the processor 101 executing programs for operating as these units. The programs for operating as the board image creation unit 12, the inspection value acquisition unit 13, and the inspection data creation unit 14 are stored in advance in the storage device 102. The processor 101 operates as the board image creation unit 12, the inspection value acquisition unit 13, and the inspection data creation unit 14 by reading out the programs from the storage device 102 and executing them.
[0026] The programs stored in the memory device 102 for operating as the board image creation unit 12, the inspection value acquisition unit 13 and the inspection data creation unit 14 may be provided to the user, for example, in a form written on a storage medium such as a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM, or may be provided to the user via a network.
[0027] 6 may be hardware constituting a tablet terminal. That is, the inspection support device 1 may be realized by a tablet terminal and a program executed on the tablet terminal.
[0028] Next, a description will be given of an operation of the inspection support device 1 to support the inspection work. Fig. 7 is a flowchart showing an example of an operation of the inspection support device 1 to support the inspection work.
[0029] In the operation of supporting the inspection work by the inspection support device 1, first, the photographing unit 11 takes a video of the board on which the inspection object is placed, and acquires board video data (step S11). At this time, the worker 201 performing the inspection takes a video while changing the range photographed by the photographing unit 11, as in the example shown in Fig. 3, and photographs the entire range of the board on which the inspection object is placed. The photographing unit 11 acquires board video data, which is video data including the entire range of the board on which the inspection object is placed.
[0030] Next, the board image creation unit 12 creates a board image, which is an image of the entire board, based on the board video data (step S12). Specifically, the board image creation unit 12 extracts necessary frame images from the frame images included in the board video data, synthesizes the extracted frame images to create one image including the entire board, and uses this as the board image. For example, the board image creation unit 12 extracts frame images every few seconds from the board video data, extracts features, and combines the images to create a board image. The board image creation unit 12 may further perform a process of cutting out an area including the entire board from the combined image, in other words, a process of deleting unnecessary areas that do not show the board, and use the obtained image as the board image. This board image creation process in step S12 is started when the worker 201 using the inspection support device 1 performs a specified operation. For example, the board image creation unit 12 starts creating a board image when an operation is performed to end the board video data acquisition process in step S11. The board image creation unit 12 may start creating a board image when an operation is performed to end the board video data acquisition process in step S11 and further when an operation is performed to instruct the start of creation of a board video.
[0031] In the above step S11, cases may occur where the frame images required to create a board image are not included in the board video data, such as when part of the board is not photographed or part of the object to be inspected is not photographed, due to an operational error by the worker 201. For this reason, when the board image creation unit 12 cannot create a board image, the inspection support device 1, for example, displays on the display unit 15 a command to redo the video capture of the board, and then returns to step S11 to capture the video capture of the board again.
[0032] When the creation of the board image by the board image creation unit 12 is completed, the inspection value acquisition unit 13 then analyzes the board image to acquire the inspection value (step S13). Specifically, the inspection value acquisition unit 13 refers to the board information stored in the board information storage unit 21 of the inspection result storage device 2 to identify the inspection object included in the board image, and acquires the inspection value by analyzing the image of the identified inspection object. The inspection value acquisition unit 13 acquires the board information to be used from the board information storage unit 21 of the inspection result storage device 2 in advance. Which board information to acquire from the board information stored in the board information storage unit 21 is determined by, for example, the worker 201. When the board identification information (for example, the board name, identification number, symbol, etc.) is written on the board, the inspection value acquisition unit 13 may extract the board identification information from the board image, and notify the board information storage unit 21 of the extracted identification information to acquire the corresponding board information. When a single board can be identified from the shape and arrangement of each inspection object on the board, the inspection value acquisition unit 13 may determine the board information to be acquired based on the shape and arrangement of each inspection object.
[0033] Next, the inspection data creation unit 14 creates inspection data based on the inspection values acquired by the inspection value acquisition unit 13 (step S14). For example, the inspection data creation unit 14 creates a list of inspection values as the inspection data. The inspection data includes at least the identification information (for example, an identification number) of the object to be inspected and the corresponding inspection value. The inspection data creation unit 14 may create inspection data in a form. The inspection data creation unit 14 may also create inspection data that further includes the board image created by the board image creation unit 12 in step S12 as an evidence image. By including the evidence image in the inspection data, it becomes possible for a user such as the worker 201 to check the evidence image and determine whether or not the inspection value included in the inspection data is correct. The format of the inspection data created by the inspection data creation unit 14 may be selected by the worker 201 from a plurality of candidates.
[0034] When the inspection data creation unit 14 finishes creating the inspection data, it registers the inspection data in the inspection result storage unit 22 of the inspection result storage device 2 (step S15).
[0035] The operations in steps S13 to S15 may be executed automatically following the operation of creating the board image in step S12, or may be executed according to instructions from the worker 201.
[0036] An example of a case where each operation of steps S13 to S15 is performed according to an instruction from the operator 201 will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the operation of the inspection support device 1.
[0037] When the creation of the board image by the board image creation unit 12 is completed, the inspection support device 1 displays a display screen 51 of the created board image on the display unit 15, as shown in Fig. 8(a). When the operator 201 presses a result display button 61 in this state, the inspection value acquisition unit 13 acquires the inspection values from the board image, and displays an inspection value display screen 52 on the display unit 15, as shown in Fig. 8(b). Furthermore, when the operator 201 presses a registration button 62, the inspection data creation unit 14 creates inspection data. The inspection data creation unit 14 transmits the inspection data to the inspection result storage device 2 via the communication unit 16.
[0038] It is possible that the inspection value acquisition unit 13 cannot acquire the inspection value or the acquired inspection value is incorrect due to reasons such as insufficient quality of the panel image. For this reason, the inspection support device 1 may be configured to accept an operation to correct the displayed inspection value on the inspection value display screen 52 shown in Fig. 8(b). This makes it possible to prevent inspection data from being created based on an incorrect inspection value.
[0039] As described above, the inspection support device 1 of the inspection support system 100 according to this embodiment takes an image of a board on which an inspection object is installed to create video data, and further creates a board image including the entire board based on the video data of the board, and analyzes the board image to obtain the inspection values of the inspection objects. Furthermore, the inspection support device 1 creates inspection data in a prescribed format based on the inspection values of each inspection object, and registers it in the inspection result storage unit 22 of the inspection result storage device 2. The inspection support system 100 can reduce the inspection work load at the site where the inspection objects are installed.
[0040] In this embodiment, the inspection support device 1 is moved while the photographing unit 11 of the inspection support device 1 photographs a part of the board on which the inspection object is installed, and board video data capable of creating a board image including the entire board is acquired. However, the area including the entire board may be photographed first. The inspection support device 1 analyzes the video obtained by photographing the entire board, and if there is no frame image that meets the specified quality among the frame images constituting the video, that is, if there is no frame image including the entire board with a quality that allows the acquisition of an inspection value, the display unit 15 may display a message indicating that re-photographing is required, and the photographing as shown in FIG. 3 may be performed. At this time, a photographing guide may be displayed on the display unit 15. For example, an arrow line indicating the route to be photographed may be displayed on the display unit 15. The photographing method, points to note when photographing, etc. may be communicated to the worker 201 by text or voice. If there is a frame image that is of a quality that allows the acquisition of an inspection value among the frame images constituting the video obtained by photographing the entire board by the photographing unit 11, the board image creation unit 12 selects one of the frame images with a quality that allows the acquisition of an inspection value and outputs it as a board image.
[0041] In addition, before starting the board video data acquisition process of step S11 above, the inspection support device 1 may accept from the worker 201 the selection of the board on which the inspection target is installed, and display a shooting guide for the selected board on the display unit 15.
[0042] In addition, when the inspection support device 1 determines the board information to be used in the process of analyzing the board image and acquiring the inspection value based on the shape and arrangement of each inspection object, the inspection support device 1 may narrow down the candidates using the position information. For example, the inspection support device 1 stores the position information of each board present at the site in advance, and narrows down the candidates for board information based on the position of the own device and the position information of each board when deciding the board information to be used. That is, the inspection support device 1 sets the board information of boards present within a certain range centered on the own device as candidates, and selects one of the candidates for board information based on the shape and arrangement of the inspection object. In this case, the inspection support device 1 acquires the board information narrowed down based on the position information from the inspection result storage device 2, for example, before starting the board video data acquisition process of the above step S11.
[0043] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies. Parts of the configurations may be omitted or modified without departing from the spirit of the invention.
[0044] Various aspects of the present disclosure are summarized below as appendices.
[0045] (Appendix 1) An inspection support device that supports on-site inspection work of a plurality of inspection objects installed on a panel, an image capturing unit that captures an image of an area including at least a portion of the board and creates a plurality of images; A board image creation unit that creates a board image, which is a single image including the entire board, based on the multiple images created by the photographing unit; an inspection value acquisition unit that analyzes the board image and acquires inspection values of the plurality of inspection objects; An inspection support device comprising: (Appendix 2) The photographing unit creates a plurality of the images by photographing a moving image. 2. An inspection support device according to claim 1. (Appendix 3) The board image creation unit extracts one or more images from the plurality of images created by the photographing unit, and creates the board image based on the extracted images. 3. An inspection support device according to claim 1 or 2. (Appendix 4) The board image creation unit extracts a plurality of images each including a different portion of the board, and creates the board image by combining the extracted images. 4. An inspection support device according to claim 3. (Appendix 5) When an image that includes the entire board and satisfies a predetermined quality is present among the multiple images created by the photographing unit, the board image creation unit extracts the image that satisfies the predetermined quality and sets it as the board image. 4. An inspection support device according to claim 3. (Appendix 6) The inspection value acquisition unit identifies the inspection object included in the board image based on board information including information on the position of the inspection object on the board and information on the type of the inspection object, and analyzes an image of the identified inspection object to acquire the inspection value. 6. An inspection support device according to any one of claims 1 to 5. (Appendix 7) an inspection data creation unit that creates inspection data by inserting the inspection values acquired by the inspection value acquisition unit into an electronic document in a format corresponding to the board; 7. An inspection support device according to any one of claims 1 to 6, comprising: (Appendix 8) A shooting guide is displayed on a display unit when the shooting unit shoots the board. 8. An inspection support device according to any one of claims 1 to 7. (Appendix 9) An inspection support device according to any one of appendix 1 to 8; an inspection result storage device that stores the inspection results of the inspection object by the inspection support device and provides the stored inspection results to the user upon receiving a request from the user; An inspection support system comprising: (Appendix 10) An inspection support method implemented by an inspection support device that supports on-site inspection of a plurality of inspection objects installed on a panel, comprising: A photographing step of photographing an area including at least a part of the board to create a plurality of images; a board image creation step of creating a board image, which is a single image including the entire board, based on the multiple images created in the photographing step; an inspection value acquisition step of acquiring inspection values of the plurality of inspection objects by analyzing the board image; An inspection support method comprising: [Explanation of symbols]
[0046] 1 inspection support device, 2 inspection result storage device, 11 photography unit, 12 panel image creation unit, 13 inspection value acquisition unit, 14 inspection data creation unit, 15 display unit, 16, 23 communication unit, 21 panel information storage unit, 22 inspection result storage unit, 51 panel image display screen, 52 inspection value display screen, 61 result display button, 62 registration button, 100 inspection support system, 201 worker, 202 user, 301 panel.
Claims
1. An inspection support device that supports on-site inspection work of a plurality of inspection objects installed on a panel, an image capturing unit that captures an image of an area including at least a portion of the board and creates a plurality of images; a board image creation unit that creates a board image, which is a single image including the entire board, based on the multiple images created by the photographing unit; an inspection value acquisition unit that analyzes the board image and acquires inspection values of the plurality of inspection objects; An inspection support device comprising:
2. The photographing unit creates a plurality of the images by photographing a moving image.
2. The inspection support device according to claim 1 .
3. The board image creation unit extracts one or more images from the plurality of images created by the photographing unit, and creates the board image based on the extracted images.
2. The inspection support device according to claim 1 .
4. The board image creation unit extracts a plurality of images each including a different portion of the board, and creates the board image by combining the extracted images.
4. The inspection support device according to claim 3.
5. When an image that includes the entire board and satisfies a predetermined quality is present among the multiple images created by the photographing unit, the board image creation unit extracts the image that satisfies the predetermined quality and sets it as the board image.
4. The inspection support device according to claim 3.
6. The inspection value acquisition unit identifies the inspection object included in the board image based on board information including information on the position of the inspection object on the board and information on the type of the inspection object, and analyzes an image of the identified inspection object to acquire the inspection value.
2. The inspection support device according to claim 1 .
7. an inspection data creation unit that creates inspection data by inserting the inspection values acquired by the inspection value acquisition unit into an electronic document in a format corresponding to the board; 2. The inspection support device according to claim 1, further comprising:
8. A shooting guide is displayed on a display unit when the shooting unit shoots the board.
8. An inspection support device according to claim 1,
9. The inspection support device according to claim 1 ; an inspection result storage device that stores the inspection results of the inspection object by the inspection support device and provides the stored inspection results to the user upon receiving a request from the user; An inspection support system comprising:
10. An inspection support method implemented by an inspection support device that supports on-site inspection of a plurality of inspection objects installed on a panel, comprising: A photographing step of photographing an area including at least a part of the board to create a plurality of images; a board image creation step of creating a board image, which is a single image including the entire board, based on the multiple images created in the photographing step; an inspection value acquisition step of acquiring inspection values of the plurality of inspection objects by analyzing the board image; An inspection support method comprising:
Citation Information
Patent Citations
Indicated value reading system, method, and program
JP2022007681A
