Field work support system

The field work support system using smart glasses and an operation terminal addresses the challenge of creating document data for field work by reducing worker workload through automated imaging and document creation, enhancing efficiency in maintenance and troubleshooting.

JP2024170925APending Publication Date: 2024-12-11NOHMI BOSAI LTD

Patent Information

Application Number
JP2023087694
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-29
Publication Date
2024-12-11

AI Technical Summary

Technical Problem

Existing systems fail to efficiently create document data related to field work, including imaging circumstantial evidence, without increasing the workload of field workers, particularly in on-site maintenance and troubleshooting for disaster prevention systems.

Method used

A field work support system utilizing smart glasses and an operation terminal that captures and processes imaging data, integrates with a server to create document data, and supports hands-free operation for field workers.

Benefits of technology

The system enables the creation of document data related to field work, including imaging circumstantial evidence, while reducing the workload of field workers, and facilitates sharing and updating of electronic manuals and product acceptance inspections.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024170925000001_ABST
    Figure 2024170925000001_ABST
Patent Text Reader

Abstract

To obtain a field work support system that can create document data related to field work, including a task of imaging circumstantial evidences, while preventing a workload of a field worker from increasing.SOLUTION: A field work support system (100) comprises smart glasses (1) worn by a field worker, and a controller (2) for receiving operations from the field worker and controlling an action of the smart glasses based on the operations. The smart glasses (1) are provided with an imaging unit (15) for imaging an orientation in which the field worker is facing under a state in which the smart glasses are worn by the field worker, and the controller (2) acquires imaged data imaged by the imaging unit (15), and creates document data including the imaged data for transmitting the data to the outside.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a field work support system that supports workers who perform field work such as maintenance and troubleshooting for a disaster prevention system. [Background technology]

[0002] When carrying out on-site work such as maintenance and troubleshooting for disaster prevention systems, on-site workers create documents such as work reports for clients, and in-house defect response cases, work cases, and troubleshooting cases upon completion of their work.

[0003] Here, a device that automatically creates a report on a disaster prevention system is disclosed (for example, see Patent Document 1). The disaster prevention display device according to Patent Document 1 has a configuration that tally up events detected by devices such as fire detectors and gas leak detectors collected by a disaster prevention receiver, and displays the tally up results. In addition, the disaster prevention display device according to Patent Document 1 has a configuration that prints out the tally up results within a predetermined period in a report format. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2004-118776 A Summary of the Invention [Problem to be solved by the invention]

[0005] Depending on the content of on-site work such as maintenance and troubleshooting, it may be necessary to document the details of the work. In this case, in addition to the actual on-site work, on-site workers may also be tasked with taking pictures of the situation as evidence according to the progress of the work.

[0006] The disaster prevention display device described in the above Patent Document 1 is one that compiles events detected by fire detectors, gas leak detectors, etc., and compiles them into a report, but is different from one that compiles the details of on-site work, including the task of capturing images of circumstantial evidence, as described above. For this reason, the technology in Patent Document 1 cannot be applied to creating documents related to on-site work.

[0007] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a field work support system that can create document data regarding field work, including the task of capturing images of circumstantial evidence, while minimizing an increase in the workload of field workers. [Means for solving the problem]

[0008] The field work support system according to the present disclosure includes smart glasses worn by a field worker, and a control unit that receives operations from the field worker and controls the operation of the smart glasses based on the operations, and the smart glasses are provided with an imaging unit that captures an image of the direction in which the field worker is facing when the field worker is wearing the smart glasses. The control unit acquires imaging data captured by the imaging unit, creates document data including the imaging data, and transmits the document data to the outside. Effect of the Invention

[0009] According to the present disclosure, it is possible to create document data regarding on-site work, including the task of capturing images of circumstantial evidence, while suppressing an increase in the workload of on-site workers. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating a configuration example of a field work support system according to a first embodiment of the present disclosure. [Diagram 2] 2 is a block diagram showing an example of the configuration of the smart glasses and the operation terminal shown in FIG. 1. [Diagram 3] 2 is a sequence diagram showing a process in the field work support system shown in FIG. 1. [Figure 4] 3 is a flowchart showing a document data creating operation performed by the operation terminal shown in FIG. 2. [Diagram 5] 1. FIG. 4 is a sequence diagram showing a product acceptance inspection process in the field operation support system shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the on-site work support system of the present disclosure will be described with reference to the drawings. The field work support system of the present disclosure has a technical feature of using smart glasses, a type of wearable device, to support field work such as maintenance of disaster prevention systems and troubleshooting, and in particular to reduce the burden of creating document data related to field work, including the task of capturing images of circumstantial evidence.

[0012] Embodiment 1 FIG. 1 is a diagram illustrating an example of a configuration of a field work support system according to a first embodiment of the present disclosure.

[0013] The field work support system 100 includes smart glasses 1, an operation terminal 2 serving as a control unit, and a server 3.

[0014] The smart glasses 1 are glasses-type wearable devices that are worn by field workers. The smart glasses 1 are provided with a small display 51, a camera 52, and a light 53.

[0015] The small display 51 is provided on the smart glasses 1 so that the display surface is located near the eyes of the field worker and faces the field worker.

[0016] The camera 52 is provided so that its lens surface faces the front side of the smart glasses 1, and captures an image in the direction in which the site worker is facing.

[0017] The lighting 53 is provided to illuminate the front side of the smart glasses 1, and illuminates the direction in which the field worker is facing.

[0018] In addition, the smart glasses 1 can transmit and receive data to and from the operation terminal 2 via short-range wireless communication, and operate upon receiving commands from the operation terminal 2.

[0019] The operation terminal 2 is a terminal operated by a field worker, and controls the operation of the smart glasses 1 according to the operation. The operation terminal 2 displays an image on the small display 51 of the smart glasses 1 via short-range wireless communication. The operation terminal 2 also issues an image capture command to the camera 52 of the smart glasses 1 and acquires image data.

[0020] The operation terminal 2 is provided with operation keys 61. The operation keys 61 are made up of up, down, left and right cross keys and an enter key located in the center of the cross keys.

[0021] The operation terminal 2 is wirelessly connected to a wide area communication network 200 provided by a communication carrier, and transmits and receives data to and from the server 3 .

[0022] The server 3 is a cloud server that transmits and receives data to and from the operation terminal 2 via the wide area communication network 200. The server 3 stores electronic manuals related to the devices that constitute the disaster prevention system.

[0023] Furthermore, format data is stored in the server 3. The format data is a template for document data related to on-site work, and is data in DOCX format in which only frame lines and fixed phrases are written. Document data includes, for example, work reports for customers, in-house defect response cases, work cases, and failure cases, and format data corresponding to each of these document data is prepared in advance and stored in the server 3.

[0024] FIG. 2 is a block diagram showing an example of the configuration of the smart glasses 1 and the operation terminal 2 shown in FIG.

[0025] In FIG. 2, the smart glasses 1 include a display unit 11, a processor 12, a memory unit 13, a short-range wireless communication unit 14, an imaging unit 15, and a microphone 16.

[0026] The display unit 11 includes the above-mentioned small display 51, and is provided so that the field worker can view it when the field worker wears the smart glasses 1. The display unit 11 is capable of displaying content such as characters, symbols, figures, still images, and videos.

[0027] The arithmetic processing unit 12 includes a central processing unit and a main memory device, and executes software stored in the memory unit 13. In this way, the arithmetic processing unit 12 performs overall control over the hardware inside the smart glasses 1.

[0028] The storage unit 13 includes an auxiliary storage device, and stores software that is calculated and executed by the calculation processing unit 12 .

[0029] The short-range wireless communication unit 14 transmits and receives data via short-range wireless communication with the operation terminal 2. As a preliminary step to transmitting and receiving data, the short-range wireless communication unit 14 performs a process for sharing unique information with the operation terminal 2. Then, the short-range wireless communication unit 14 performs short-range wireless communication with the operation terminal 2 as a counterpart, while confirming each other using the unique information.

[0030] The imaging unit 15 includes the above-mentioned camera 52, and captures an image in the direction in which the on-site worker is facing when the on-site worker wears the smart glasses 1. In particular, the imaging unit 15 in the present embodiment 1 is utilized for obtaining imaging data required for creating document data, in addition to distributing the on-site situation, work scenes, and the like to skilled workers in remote locations.

[0031] The microphone 16 is a device for picking up sounds occurring in the surrounding area, and in particular, picks up the voices of workers at the site.

[0032] The operation terminal 2 is a small mobile computer on which an operating system is installed, and includes an operation unit 21, a calculation processing unit 22, a storage unit 23, a short-range wireless communication unit 24, and a wide-area line communication unit 25.

[0033] The operation unit 21 includes the above-mentioned operation keys 61, and receives operations from a site worker.

[0034] The wide area line communication unit 25 performs data communication with the server 3 via the wide area communication network 200. As long as the configuration allows communication with the server 3, a communication configuration involving a wireless LAN (Local Area Network) may be used.

[0035] The arithmetic processing unit 22, the memory unit 23, and the short-range wireless communication unit 24 have the same functions as the arithmetic processing unit 12, the memory unit 13, and the short-range wireless communication unit 14 provided in the smart glasses 1.

[0036] In the first embodiment, a dedicated device for operating the smart glasses 1 is used as the operation terminal 2, but a smartphone with operation software installed thereon may be used instead.

[0037] FIG. 3 is a sequence diagram showing the processing in the site work support system 100 shown in FIG.

[0038] In step S101, the operation terminal 2 acquires information on the work target device, the system model, and the work content selected by the site worker.

[0039] The on-site worker operates the operation unit 21 of the operation terminal 2, specifically the cross key and the decision key, while visually checking the screen for selecting the model and the work content displayed on the smart glasses 1. In this way, the on-site worker selects the model and the work content. Note that examples of the work content that can be selected here include "signal converter replacement," "server malfunction," "regular inspection of fire detectors," and "response to false alarms from receivers."

[0040] In step S102, the operation terminal 2 transmits the acquired information on the model and the work content to the server 3.

[0041] In step S103, the server 3 identifies a corresponding electronic manual from among the stored electronic manuals based on the received information on the model and work content. Also, in step S103, the server 3 identifies a corresponding format data from among the stored format data based on the received information on the model and work content.

[0042] In step S104, the server 3 transmits the identified electronic manual and format data to the operation terminal 2 as working data.

[0043] In step S105, the operation terminal 2 extracts one page as a display page from the electronic manual transmitted from the server 3. Then, in step S106, the operation terminal 2 transmits data of the one page to the smart glasses 1.

[0044] In step S107, the smart glasses 1 display the received data for one page on the display unit 11.

[0045] By simply moving their eyes and changing the direction of their line of sight, field workers can compare the displayed electronic manual with the device they are working on, enabling them to work hands-free.

[0046] The pages are switched by operating the operation terminal 2. That is, every time the left or right button of the cross key of the operation unit 21 is operated, steps S105 to S107 are executed to switch to the next page or the previous page.

[0047] The electronic manual also describes a method for what to capture depending on the progress when creating document data. The field worker presses the decision button of the operation unit 21 according to the description in the electronic manual. As a result, the operation terminal 2 receives an instruction to capture images from the field worker in step S108.

[0048] In step S109, the operation terminal 2 transmits an imaging command to the smart glasses 1. In step S110, the smart glasses 1 capture images in response to the imaging command, and transmit the captured image data to the operation terminal 2 in step S111.

[0049] In step S112, the operation terminal 2 stores the captured image data. Here, the operation terminal 2 may inquire of the field worker as to whether to store the captured image data or to take a new image. In this case, the field worker can decide whether to store the captured image data or to take a new image by operating the up and down buttons of the cross key of the operation unit 21.

[0050] Steps S108 to S112 are carried out every time the site worker presses the decision button on the operation unit 21. This enables the operation terminal 2 to obtain imaging data in accordance with the progress of the work.

[0051] When the work is completed, in step S113, the operation terminal 2 creates document data by inserting each acquired imaging data into the format data received from the server 3. The method of creating this document data will be described later.

[0052] In step S114, the operation terminal 2 transmits the created document data to the server 3. Note that the operation terminal 2 may transmit unprocessed captured image data to the server 3 as evidence data indicating each situation.

[0053] In step S115, the server 3 stores the received document data as archive data.

[0054] 3, the operation terminal 2 acquires the electronic manual and format data from the server 3, and finally returns the created document data to the server 3. However, the on-site work support system according to the first embodiment can also be realized as a minimum configuration including the smart glasses 1 and the operation terminal 2 without communicating with the server 3.

[0055] When such a minimum configuration is adopted, a site worker can create document data by following the procedure below. Step 1: Electronic manuals and format data relating to the equipment and system to be worked on are already stored in the memory unit 23 of the operation terminal 2, and the field worker operates the operation terminal 2 to obtain the necessary electronic manuals and format data.

[0056] Step 2: The field worker inserts the image data captured by the imaging unit 15 of the smart glasses 1 into the format data according to the progress of the work to create document data.

[0057] Step 3: Instead of sending the document data into which the captured image data has been inserted to the server 3, the document data is sent to an external printer, and printing is performed. By carrying out such a series of tasks, it is possible to realize a field work support system that can create document data regarding field work, including the task of capturing images of circumstantial evidence, while minimizing an increase in the workload of field workers.

[0058] Fig. 4 is a flowchart showing the document data creation operation by the operation terminal 2 shown in Fig. 2. Using Fig. 2, a supplementary explanation will be given of the creation operation in the case where it is necessary to write other information in addition to inserting the imaging data into the document data. Here, it is assumed that the operation terminal 2 creates a DOCX format file as the document data.

[0059] In step S201, the operation terminal 2 unpacks the DOCX format data. A DOCX file is structured by compressing a group of XML text files and a group of image data in ZIP format. In step S201, the operation terminal 2 unpacks the compressed DOCX file so that various data can be directly edited.

[0060] In step S202, the operation terminal 2 stores the captured image data in the expanded format data. In the expanded format data, a media folder is formed as a folder for storing images. The operation terminal 2 changes the file name of the captured image data to a name such as "image[N].png" ([N]=1, 2, 3,...), and stores the captured image data in the media folder. Note that it is assumed that the position in the document where each captured image data is inserted is defined in advance in the format data.

[0061] In step S203, the operation terminal 2 writes information that can be acquired within the system into the expanded format data. Information that can be acquired within the system includes time information, information for identifying the field worker, etc. The operation terminal 2 writes this information into document.xml included in the expanded format data.

[0062] In step S204, the operation terminal 2 writes other necessary information into the expanded format data. Here, the customer name, the work location, and any special notes that the on-site worker wants to keep as a record are written.

[0063] In step S204, the operation terminal 2 collects the voice of the field worker with the microphone 16 and converts it into text data using a voice recognition technique. Then, the operation terminal 2 writes the converted text data into document.xml.

[0064] In step S205, the operation terminal 2 compresses the format data in which the imaging data has been stored and the necessary text data has been written, into a ZIP format, thereby enabling the operation terminal 2 to obtain a file in a DOCX format.

[0065] The document data created in this manner is transmitted to the server 3 in step S114 in Fig. 3. Alternatively, it can be printed out via an external printer.

[0066] So far, an example of providing an electronic manual and creating document data when maintaining a disaster prevention system and responding to failures has been described. In addition, the on-site work support system 100 of the first embodiment can be used for product acceptance inspection. An example of this case will be described below.

[0067] Fig. 5 is a sequence diagram showing the product acceptance inspection process in the on-site work support system 100 shown in Fig. 1. In the following description, it is assumed that individual information for identifying the product to be brought in is stored in advance in the server 3. It is also assumed that the actual product to be brought in has a barcode indicating the individual information attached thereto.

[0068] The site worker presses the enter button on the operation unit 21 while focusing the imaging unit 15 on the barcode attached to the carried-in product. As a result, the operation terminal 2 receives an instruction to capture an image from the site worker in step S301.

[0069] In step S302, the operation terminal 2 outputs an imaging command to the smart glasses 1. In step S303, the smart glasses 1 capture images in response to the imaging command, and in step S304, transmit the captured image data to the operation terminal 2.

[0070] In step S305, the operation terminal 2 acquires individual information from the barcode shown in the captured image data, and in step S306, transmits the acquired individual information to the server 3.

[0071] In step S307, the server 3 checks whether the received individual information matches any of the individual information stored in advance, and then in step S308, the server 3 transmits the check result to the operation terminal 2.

[0072] In step S309, the operation terminal 2 acquires image data indicating acceptance permission or image data indicating non-acceptance based on the collation result from the storage unit 23. Then, in step S310, the operation terminal 2 transmits a display command to the smart glasses 1 together with the acquired image data.

[0073] In step S311, the smart glasses 1 display image data indicating acceptance permission or image data indicating non-acceptance, and notify the site worker.

[0074] Steps S301 to S311 are repeated for each carried-in product, allowing the on-site staff to perform an acceptance inspection of each product.

[0075] 5, the configuration has been described in which individual information for identifying a product to be carried in is stored in advance in the server 3, and a collation result performed by the server 3 is returned to the operation terminal 2. However, the on-site work support system according to the first embodiment can also realize the product acceptance inspection process as a minimum configuration including the smart glasses 1 and the operation terminal 2 without communicating with the server 3.

[0076] When such a minimum configuration is adopted, a site worker can carry out the product receiving inspection process by following the procedure below. Step 1: A site worker uses the imaging unit 15 to read the barcode attached to the brought-in product.

[0077] Step 2: Individual information for identifying the product to be brought in has already been stored in the memory unit 23 of the operation terminal 2, and the on-site worker confirms the matching process executed on the operation terminal 2 as a display on the smart glasses 1. By carrying out such a series of operations, it is possible to realize an on-site operation support system that can quickly carry out product receiving inspection processing on-site while suppressing an increase in the workload of on-site workers.

[0078] In the first embodiment, the smart glasses 1 and the operation terminal 2 are separate from each other, but the functions of the operation terminal 2 may be integrated into the smart glasses 1. In this case, the smart glasses 1 may be provided with an operation unit for operation, or may be provided with a function for performing various operations in response to gestures such as eye movements and blinking.

[0079] In the first embodiment, the operation terminal 2 uses the format data to create document data that conforms to the template of the format data. In contrast, the operation terminal 2 may create free-format document data by simply pasting captured image data in order, without using format data. The operation terminal 2 may also create document data from scratch by writing frame lines, fixed phrases, and the like each time, without using format data.

[0080] In the first embodiment, the field worker takes an image according to the procedure described in the electronic manual, but the present invention is not limited to this. The field worker may take an image at any timing.

[0081] In the first embodiment, the document data is created in DOCX format, but the present invention is not limited to this format. For example, the document data may be created in PDF format. Furthermore, the document data may be created using a method other than the flow chart of FIG. 4, for example, an API (Application Programming Interface) provided by a vendor.

[0082] Also, in the example of FIG. 5, a button operation is required to read the barcode, but a method may be used in which the barcode is automatically read as soon as it is recognized in the image.

[0083] The features of such on-site work support system 100 can be summarized as follows: It has the following configuration and can achieve the following effects.

[0084] The on-site work support system 100 includes smart glasses 1 worn by a on-site worker, and an operation terminal 2 that receives operations from the on-site worker and controls the operation of the smart glasses 1 based on the operations. In addition, a configuration can be adopted that further includes a server 3 that can communicate with the operation terminal 2.

[0085] The smart glasses 1 are also provided with an imaging unit 15 that captures an image of the direction in which the field worker is facing when the field worker is wearing the smart glasses 1.

[0086] Furthermore, the operation terminal 2 acquires the imaging data captured by the imaging unit 15, creates document data including the imaging data, and prints it out on an external printer or transmits it to the server 3. Furthermore, the server 3 accumulates the document data transmitted from the operation terminal 2.

[0087] This configuration makes it possible to create document data related to on-site work, including the task of capturing images of circumstantial evidence, while minimizing the increase in the workload of on-site workers. In addition, by storing document data in server 3, various types of information such as defect response cases, work cases, and failure cases can be shared among related parties within the same company or department.

[0088] The smart glasses 1 are also provided with a display unit 11 that can be viewed by a field worker while wearing the smart glasses 1. The server 3 stores an electronic manual for a target device on which the field worker is to work. The operation terminal 2 receives the electronic manual from the server 3 and causes the display unit 11 to display the electronic manual.

[0089] With this configuration, the on-site worker can carry out on-site work while comparing the electronic manual with the target equipment by simply moving his / her eyes to change the direction of his / her line of sight. Also, by storing the electronic manual in the server 3 and managing it centrally, when updating the electronic manual to the latest version, it is possible to do so by simply updating the data in the server 3.

[0090] In addition, the server 3 stores in advance individual information for identifying a product to be delivered to the site. The imaging unit 15 captures an image of a barcode attached to the product delivered to the site. The operation terminal 2 then acquires the individual information from the barcode captured by the imaging unit 15 and transmits the individual information to the server 3.

[0091] The server 3 verifies whether the transmitted individual information matches individual information stored in advance, and returns the collation result to the operation terminal 2. The operation terminal 2 causes the display unit 11 to display the collation result.

[0092] In addition, if individual information for identifying the product to be brought into the site is stored in advance in the operation terminal 2, it is possible for the operation terminal 2 itself to perform the matching instead of the server 3.

[0093] With this configuration, the on-site worker can easily perform the receiving inspection of the product, thereby reducing the workload. [Explanation of symbols]

[0094] 1 smart glasses, 2 operation terminal (control unit), 3 server, 11 display unit, 15 imaging unit, 100 on-site work support system.

Claims

1. Smart glasses worn by field workers, A control unit that receives an operation from the field worker and controls the operation of the smart glasses based on the operation; Equipped with The smart glasses are provided with an imaging unit that captures an image of a direction in which the field worker is facing while wearing the smart glasses, The control unit acquires imaging data captured by the imaging unit, creates document data including the imaging data, and transmits the document data to an external device. On-site work support system.

2. Further comprising a server capable of communicating with the control unit, The server stores the document data transmitted from the control unit. The on-site work support system according to claim 1 .

3. The smart glasses are provided with a display unit that can be viewed by the field worker while the field worker is wearing the smart glasses, The server stores an electronic manual for a target device on which the field worker operates; The control unit receives the electronic manual from the server and causes the display unit to display the electronic manual. The on-site work support system according to claim 2 .

4. The smart glasses are provided with a display unit that can be viewed by the field worker while the field worker is wearing the smart glasses, The control unit pre-stores individual information for identifying the product to be delivered to the site, The imaging unit captures an image of a barcode attached to a product delivered to the site, the control unit acquires individual information from the barcode captured by the imaging unit, verifies whether the individual information matches individual information stored in advance, and causes the display unit to display a result of the comparison. The on-site work support system according to claim 1 .

5. The smart glasses are provided with a display unit that can be viewed by the field worker while the field worker is wearing the smart glasses, The server stores in advance individual information for identifying the product to be delivered to the site, The imaging unit captures an image of a barcode attached to a product delivered to the site, The control unit acquires individual information from the barcode captured by the imaging unit and transmits the individual information to the server; The server checks whether the transmitted individual information matches pre-stored individual information and returns the check result to the control unit. The control unit causes the display unit to display the collation result. The on-site work support system according to claim 2 .

6. The server stores in advance individual information for identifying the product to be delivered to the site, The imaging unit captures an image of a barcode attached to a product delivered to the site, The control unit acquires individual information from the barcode captured by the imaging unit and transmits the individual information to the server; The server checks whether the transmitted individual information matches pre-stored individual information and returns the check result to the control unit. The control unit causes the display unit to display the collation result. The on-site work support system according to claim 3 .

Citation Information

Patent Citations

  • Disaster prevention display device

    JP2004118776A

Cited By

  • Work support system, work support method, and work support program

    JP2026145532A