Wiring work assistance system, wiring work assistance method, and program

Abstract

To help increase the efficiency of wiring work taking wiring information into account.SOLUTION: A wiring work assistance system 1 comprises: imaging means for capturing a real scenery at the site of wiring work and outputting captured data; storage means for graphical data of the object of wiring work and wiring information representing attribute information including identification information of a wiring provided to the object; conversion means for converting the graphical data of the object and the wiring information into image data for displaying on a display element; and synthesizing means for receiving the captured data outputted from the imaging means and image data from the conversion means and displaying the image of the object in the real scenery at the working work site represented by the captured image and the image of the object represented by the image data converted from the graphical data by the conversion means one on top of another, as well as synthesizing the captured data and the image data and outputting the resulting data to the display element so as to display the wiring information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a wiring work assistance system, a wiring work assistance method, and a program. [Background technology]

[0002] Technologies have been proposed to support work efficiency improvement using XR (Cross Reality) technologies such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality). For example, Patent Document 1 discloses a manufacturing support system that supports assembly work by superimposing a virtual image of a part on a real scene captured by a camera device in smart glasses. Furthermore, Patent Document 2 discloses a maintenance support system that, when replacing a faulty part, identifies the position of the faulty part in the image by analyzing an image sent from AR glasses used by a maintenance worker and supports the replacement work by displaying the identified part transparently on the AR glasses. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 049999 [Patent Document 2] Japanese Patent Publication No. 2020-170249 Summary of the Invention [Problem to be solved by the invention]

[0004] The techniques of Patent Documents 1 and 2 support the worker by displaying the components to be worked on on the worker's mobile device. However, for example, in wiring work, when there are many wires and many types of wires, the wiring information becomes complicated, and simply displaying the wires used in the work on the worker's mobile device can easily lead to work errors and reduce work efficiency. For this reason, there is a need for support for improving the efficiency of wiring work by taking wiring information into consideration.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a wiring work assistance system, a wiring work assistance method, and a program that support improving the efficiency of wiring work by taking wiring information into consideration. [Means for solving the problem]

[0006] In order to achieve the above object, the wiring work support system according to the present disclosure includes an imaging means for imaging an actual scene at a wiring work site and outputting the imaging data, a storage means for storing graphic data of an object of the wiring work and wiring information representing attribute information including identification information of the wiring to be installed on the object, a conversion means for converting the graphic data of the object and the wiring information into image data to be displayed on a display element, and a synthesis means for receiving the imaging data output from the imaging means and the image data from the conversion means, and synthesizing the imaging data and image data to output to the display element so as to superimpose and display an image of the object in the actual scene at the wiring work site represented by the imaging data and an image of the object represented by the image data of the graphic data converted by the conversion means, and displaying the wiring information. [Effects of the Invention]

[0007] According to the present disclosure, an image of an object in an actual scene of a wiring work site represented by imaging data and an image of the object represented by image data of graphic data converted by a conversion means are superimposed and displayed, and the imaging data and image data are synthesized and output to a display element so as to display wiring information. Therefore, it is possible to support the improvement of efficiency of wiring work taking into account wiring information. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a configuration of a wiring work assistance system according to an embodiment. [Figure 2] A block diagram showing the functional configuration of smart glasses according to an embodiment. [Figure 3] FIG. 1 is a block diagram showing a functional configuration of an information management device according to an embodiment. [Figure 4]FIG. 1 is a block diagram showing a functional configuration of a wiring work assistance device according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a wiring list storage unit according to an embodiment. [Figure 6] FIG. 10 is a diagram showing an example of a wiring work DB storage unit according to an embodiment; [Figure 7] FIG. 10 is a diagram illustrating an example of a synthesis table storage unit according to an embodiment. [Figure 8] FIG. 1 is a diagram illustrating an example of a hardware configuration of a wiring work assistance device according to an embodiment. [Figure 9] 1 is a flowchart showing an XR model creation process according to an embodiment. [Figure 10] 1 is a flowchart showing a wiring work support process according to an embodiment; [Figure 11] FIG. 10 is a diagram illustrating an example of a display displayed on the glass display unit. [Figure 12] 1 is a flowchart showing an error display process according to an embodiment; [Figure 13] A diagram to explain an example of highlighting a kitting box DETAILED DESCRIPTION OF THE INVENTION

[0009] (Embodiment) Hereinafter, a wiring work assistance system, a wiring work assistance method, and a program according to embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are designated by the same reference numerals.

[0010] A wiring work assistance system according to an embodiment of the present disclosure uses XR technology to display assistance information representing information for assisting the wiring work on an object existing in real space on a mobile device used by a worker, thereby assisting the wiring work. As shown in FIG. 1 , the wiring work assistance system 1 according to the present embodiment includes a wiring work assistance device 100 that assists the wiring work of a worker HM, an information management device 200 that manages various information necessary to support the wiring work, smart glasses 300 worn on the head of the worker HM, and a terminal device 400 used by the worker HM, a supervisor, or the like of the wiring work. The terminal device 400 includes a display unit 401 that displays various information in response to operations by the worker HM, the supervisor, or the like. Note that in this embodiment, information projected onto the smart glasses 300 using XR technology, including not only CAD data including the shape of the product to be wired but also additional information such as text and dialogue, is referred to as an “XR model.” In this embodiment, the XR model is created, for example, using commercially available XR development software.

[0011] The wiring work assistance device 100, the information management device 200, the smart glasses 300, and the terminal device 400 are connected to each other so as to be able to communicate with each other via a network NW. The wiring work assistance device 100, the information management device 200, and the terminal device 400 are each configured by, for example, a computer. The network NW may be wired, wireless, a WAN (Wide Area Network), a LAN (Local Area Network), an intranet, or an extranet.

[0012] The smart glasses 300 use XR technology to superimpose support information onto objects existing in the real space around the smart glasses 300 and display it on the glasses display unit 301. In this embodiment, the smart glasses 300 are see-through smart glasses 300. The configuration of the smart glasses 300 will be described below with reference to FIG. 2.

[0013] 2, the smart glasses 300 include a third storage unit 330 that stores information necessary to support wiring work, a glasses display unit 301 that displays support information superimposed on objects existing in real space, an imaging unit 302 that captures an image of the actual scene at the wiring work site, an input unit 303 that inputs an input signal to the control unit 305, a communication unit 304 that transmits and receives various data, and the control unit 305 that executes various processes of the smart glasses 300. The wiring work site is an example of a work location according to the present disclosure.

[0014] The third storage unit 330 stores support information 331 representing information necessary for supporting the wiring work. The support information includes graphic data of an object of the wiring work and wiring information representing attribute information including identification information of the wiring to be provided on the object.

[0015] The glasses display unit 301 includes a half-mirror lens 301a and a projection device 301b that projects the XR model onto the lens 301a. The lens 301a is disposed in front of the left and right eyes of the worker HM. Note that the lens 301a may be disposed in front of only one of the left and right eyes of the worker HM. Note that the glasses display unit 301 is an example of a display element according to the present disclosure.

[0016] The imaging unit 302 captures an image of the actual scene at the wiring work site and outputs image information of the captured image representing the captured image to the control unit 305. In this embodiment, the imaging unit 302 is provided at a position where it can capture an image of the field of view of the worker HM, and captures an image of the actual scene included in the field of view of the worker HM. The imaging unit 302 is a camera and includes a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, etc. The imaging unit 302 is an example of an imaging means according to the present disclosure. Furthermore, the image information of the captured image representing the actual scene is an example of imaging data according to the present disclosure.

[0017] The input unit 303 is a device that inputs an input signal to the control unit 305. The input unit 303 includes, for example, an operation button provided on the smart glasses 300, a wireless mouse wirelessly connected to the smart glasses 300, and a wireless keyboard. The worker HM operates the input unit 303 to input, for example, information indicating the completion of a task. Note that the worker HM may also input, for example, information indicating the completion of a task to the terminal device 400.

[0018] The communication unit 304 performs communication in accordance with communication standards such as Ethernet (registered trademark) and USB (Universal Serial Bus), and transmits and receives data under the control of the control unit 305.

[0019] In this embodiment, the control unit 305 has the role of an image recognition function for recognizing images and a control function for controlling the operation of the smart glasses 300.

[0020] Although not shown, the smart glasses 300 are equipped with an information processing terminal for processing information. Most of the information processing of the smart glasses 300 is performed by this information processing terminal.

[0021] Next, the configuration of the information management device 200 will be described with reference to FIG.

[0022] The information management device 200 includes a registration reception unit 201 that receives registration of wiring design information representing information including design information of the product to be wired and wiring information representing the type of wiring, assembly order of the wiring, etc.; a wiring design specification information reception unit 202 that receives wiring design specification information; a wiring design information sending unit 203 that sends the wiring design information to the wiring work support device 100; and a second memory unit 230 that stores wiring design information 231.

[0023] The registration reception unit 201 receives registration of wiring design information 231 and stores the wiring design information 231 in the second storage unit 230. In this embodiment, the design information included in the wiring design information includes CAD data representing the product to be wired and drawing data on which the CAD data was created. The CAD data representing the product to be wired includes the wiring to be provided by the wiring work. The wiring information includes a wiring instruction table representing information such as the type of wiring and the assembly order of the wiring. The wiring design information is stored, for example, in a folder for each product ID representing the product's identification number.

[0024] The wiring design specification information receiving unit 202 receives wiring design specification information that specifies new wiring design information. In order to support the wiring work of the target product, the wiring design information sending unit 203 sends the wiring design information 231 stored in the second storage unit 230 to the wiring work support device 100.

[0025] Next, the configuration of the wiring work assisting device 100 will be described with reference to FIG.

[0026] The wiring work assistance device 100 includes a first memory unit 120 that stores data necessary for assisting wiring work, an information acquisition unit 101 that acquires information including wiring design information, an XR creation reception unit 102 that creates an XR model and projects it on the smart glasses 300, a wiring list construction unit 103 that constructs and displays a wiring list, a database construction unit 104 that constructs a database of wiring work, a database synthesis unit 105 that links a unique ID of a wiring model included in the CAD data with a wiring ID that represents an identification number of the wiring, a software setting reception unit 106 that accepts software settings, and an error display unit 107 that warns of errors.

[0027] The first storage unit 120 includes a wiring design information storage unit 121 that stores wiring design information; a wiring list storage unit 122 that stores a wiring list indicating information such as the type and length of wiring; a wiring work DB storage unit 123 that stores a wiring work database that associates information such as wiring IDs, work flags, worker names, work dates and times, and wiring model unique IDs; a combination table storage unit 124 that stores a combination table that associates wiring IDs with unique IDs of wiring models included in CAD data; and a software setting information storage unit 125 that stores software setting information including a drawing number accepted by the software setting acceptance unit 106 and a storage destination of a wiring instruction table that serves as the source data for the wiring list. The work flag indicates the progress of the wiring work. For example, the work flag may indicate "Completed," indicating that the work is complete, "Incomplete," indicating that the work is incomplete, or "Pending," indicating that the work is pending. The first storage unit 120 also stores commercially available XR development software. Furthermore, the first storage unit 120 stores programs executed by the information acquisition unit 101, the XR creation reception unit 102, the wiring list construction unit 103, the database construction unit 104, the database synthesis unit 105, the software setting reception unit 106, the error display unit 107, etc. The first storage unit 120 is an example of a storage means according to the present disclosure.

[0028] The wiring design information storage unit 121 stores the wiring design information 231 sent from the information management device 200. In this embodiment, the wiring design information includes wiring information representing a wiring instruction table and design information representing CAD data of a product to be wired. The wiring instruction table includes information such as the type of wiring, the assembly order of the wiring, and the length of the wiring. The design information and the wiring information are combined and stored, for example, in a folder for each product ID.

[0029] The wiring list storage unit 122 stores a wiring list created based on a wiring instruction table. As shown in Fig. 5, the wiring list storage unit 122 stores information indicating a wiring assembly order, a wiring ID corresponding to each assembly order of the wiring work, a work flag, a wiring type, and a wiring length. The wiring list storage unit 122 saves the wiring list in a folder for each product ID. Furthermore, work flag information is saved in the wiring list storage unit 122 according to an input operation by the worker.

[0030] The wiring work DB storage unit 123 shown in Fig. 4 stores a database of wiring work. As shown in Fig. 6, the wiring work DB storage unit 123 stores information indicating a wiring ID, a work flag, a worker name, a work date and time, a wiring model unique ID, etc., in association with each other. The work date and time indicates the date and time when the worker HM started the work. The specific construction of the wiring work database will be described later.

[0031] The synthesis table storage unit 124 shown in Fig. 4 stores information indicating a wiring ID and information indicating a wiring model unique ID in association with each other, as exemplified in Fig. 7. Note that the information indicating the wiring model unique ID corresponding to the wiring ID is included in the design information.

[0032] 4 stores software setting information input to the software setting reception unit 106 by the operation of the worker HM. The software setting information includes the specified drawing number, the destination to save the wiring instruction table, and optional information such as the color and transparency when highlighting wiring or connected devices.

[0033] 4 acquires wiring design information 231 stored in the second storage unit 230 of the information management device 200 via the network NW. The information acquisition unit 101 stores the acquired wiring design information 231 in the wiring design information storage unit 121. In addition, the information acquisition unit 101 acquires various types of information transmitted from the smart glasses 300 or the terminal device 400.

[0034] The XR creation receiving unit 102 receives an instruction to create an XR model from the worker HM, and creates the XR model using the XR development software stored in the first storage unit. In this embodiment, the worker HM selects a folder for the product ID of the wiring target from the wiring design information stored in the wiring design information storage unit 121. When the worker HM clicks the "Execute" icon, the XR creation receiving unit 102 inputs the CAD data stored in the folder for the product ID selected by the worker HM into the XR development software. This creates an XR model representing the selected CAD data. Note that the XR creation receiving unit 102 also creates an XR model of a wiring list created based on a wiring instruction table stored in a folder indicating the same product ID as the CAD data, in the same way as the XR model of the CAD data. The XR creation receiving unit 102 converts the graphic data and wiring information of the target object into image data to be displayed on the glasses display unit 301, in other words, into an XR model.

[0035] The XR creation receiving unit 102 also superimposes and displays an image of an object in the actual scene of the wiring work site represented by the imaging data output from the imaging unit 302 of the smart glasses 300 and an image of the object represented by the image data of the converted graphic data of the object, and also plays a role in synthesizing the imaging data and image data to display wiring information and outputting the synthesized data to the smart glasses 300. The imaging data and image data may be synthesized using signal processing technology, or may be synthesized in a form visible to the human eye. The XR creation receiving unit 102 is an example of a conversion means and a synthesis means according to the present disclosure. The XR model is an example of image data according to the present disclosure.

[0036] The wiring list creation unit 103 associates information such as the wiring ID, the type of wiring, and the wiring assembly order written in the wiring instruction table stored in the wiring design information storage unit 121, writes it as a wiring list as shown in Fig. 5, and stores it in the wiring list storage unit 122. Furthermore, the wiring list creation unit 103 stores information indicating a work flag in the wiring list storage unit 122 in accordance with an input operation by the worker HM.

[0037] The database construction unit 104 shown in Fig. 4 constructs wiring work information representing information about wiring work as a wiring work database, as exemplified in Fig. 6. For example, when starting work, a worker HM logs in to the terminal device 400 and inputs the wiring ID of the worker HM to perform the work. Information about the worker HM's login ID, information about the login date and time, and information about the input wiring ID are sent to the wiring work assistance device 100, and the information acquisition unit 101 acquires this information. Based on the information about the login ID, information about the login date and time, and information about the wiring ID acquired by the information acquisition unit 101, the database construction unit 104 adds the wiring ID and the worker name corresponding to the login ID to the wiring work database.

[0038] The database construction unit 104 also adds the login date and time as the work date and time to the wiring work database. The database construction unit 104 also adds a wiring model unique ID corresponding to the wiring ID from the composite table stored in the composite table storage unit 124 to the wiring work database. Note that work flags indicating work statuses such as "completed," "incomplete," and "suspended" are added by an operation of the worker HM. Specifically, the worker HM inputs the wiring ID and information of "completed" to, for example, the terminal device 400, and the information is sent to the wiring work assistance device 100, and the information acquisition unit 101 acquires the wiring ID and the information of "completed." Based on the wiring ID and information of "completed" acquired by the information acquisition unit 101, the wiring list construction unit 103 adds "completed" to the work flag corresponding to the wiring ID in the wiring list stored in the wiring list storage unit 122. The database construction unit 104 adds "completed" to the work flag corresponding to the wiring ID in the wiring work database stored in the wiring work DB storage unit 123. The database creation unit 104 is an example of a database creation means according to the present disclosure. The wiring list creation unit 103 and the database creation unit 104 are an example of an update means according to the present disclosure.

[0039] 7, the database synthesis unit 105 shown in Fig. 4 associates the wiring ID with the wiring model unique ID and adds it to the synthesis table stored in the synthesis table storage unit 124. Specifically, the database synthesis unit 105 associates the wiring ID written in the wiring instruction table stored in the wiring design information storage unit 121 with the wiring model unique ID included in the CAD data and adds it to the synthesis table.

[0040] 4 stores information received through operations by the worker HM in the software setting information storage unit 125. Operations by the worker HM include, for example, designating a drawing number, designating a destination for saving a wiring instruction table, and setting options such as highlight color and transparency of wiring or connected devices.

[0041] When an error occurs, the error display unit 107 issues a warning to the glass display unit 301 of the smart glasses 300. For example, when the control unit 305 of the smart glasses 300 recognizes that the data capacity, number of polygons, etc. of the XR model that can be displayed by hardware or software has been exceeded, the control unit 305 sends error information indicating the error to the wiring work assistance device 100 via the communication unit 304. The information acquisition unit 101 acquires the error information. Based on the error information acquired by the information acquisition unit 101, the error display unit 107 sends an instruction to the smart glasses 300 to display an error warning. The control unit 305 of the smart glasses 300 causes the glass display unit 301 to display the error warning.

[0042] Next, an example of the hardware configuration of the wiring work assistance device 100 will be described with reference to Fig. 8. The wiring work assistance device 100 in Fig. 8 is realized by a computer such as a personal computer or a microcontroller. The hardware configurations of the information management device 200 and the smart glasses 300 are the same as those of the wiring work assistance device 100.

[0043] The wiring work support device 100 comprises a processor 1001 that executes an operation program that is a program for the operation of the wiring work support device 100, a memory 1002 that serves as the main storage area, an interface 1003 that realizes the communication function of the wiring work support device 100, and a secondary storage device 1004 that stores the operation program for executing processing, all of which are connected to each other via a bus 1000.

[0044] The processor 1001 is, for example, a CPU (Central Processing Unit). The processor 1001 loads an operation program stored in a secondary storage device 1004 into a memory 1002 and executes the program, thereby realizing each function of the wiring work assisting device 100.

[0045] The memory 1002 is a main storage device configured, for example, by a RAM (Random Access Memory). The memory 1002 stores the operating program that the processor 1001 reads from the secondary storage device 1004. The memory 1002 also functions as a work memory when the processor 1001 executes the operating program.

[0046] The interface 1003 is an I / O (Input / Output) interface such as a serial port, a USB (Universal Serial Bus) port, a network interface, etc. The interface 1003 realizes the communication function of the wiring work assisting device 100.

[0047] The secondary storage device 1004 is, for example, a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The secondary storage device 1004 stores the operating program executed by the processor 1001, wiring design information, a wiring list, a wiring work database, a synthesis table, and software setting information.

[0048] An XR model creation process representing a process for creating an XR model by the wiring work assisting device 100 will be described. As a premise, a designer of a product to be wired creates CAD data of the product to be wired using a 3D CAD. When the stage of supporting the wiring work arrives, the designer uses his / her own terminal device 400 to store wiring design information 231, including the CAD data of the product to be wired, drawing data on which the CAD data was created, and a wiring instruction table, together with identification information of the product to be wired, in the second storage unit 230 of the information management device 200. Note that the identification information of the product to be wired may be automatically assigned by the registration reception unit 201 when stored in the second storage unit 230. Alternatively, the wiring design information 231 may be stored by a manager of the work facility.

[0049] An administrator, a worker HM, or the like starts the wiring work assistance device 100 and identifies the product ID of a product to be wired. Information on the product ID identified by the administrator, the worker HM, or the like is sent to the information management device 200. A wiring design specification information receiving unit 202 of the information management device 200 receives the sent product ID information. Based on the product ID information received by the wiring design specification information receiving unit 202, a wiring design information sending unit 203 sends wiring design information 231 in a folder of the corresponding product ID stored in the second storage unit 230 to the wiring work assistance device 100. The wiring work assistance device 100 starts an XR model creation process illustrated in FIG. 9 .

[0050] First, the information acquisition unit 101 of the wiring work assisting device 100 acquires the wiring design information 231 from the information management device 200 and stores it in the wiring design information storage unit 121 (step S11).

[0051] The wiring list creation unit 103 creates a wiring list based on the wiring instruction table included in the wiring design information stored in the wiring design information storage unit 121 (step S12). Specifically, the wiring list creation unit 103 creates a wiring list as shown in FIG. 5 from information indicating the wiring assembly order, wiring ID, wiring type, wiring length, etc., shown in the wiring instruction table. Note that in FIG. 5, information indicating "completed" is stored in the work flag corresponding to the wiring ID "Cable-1", but this has not been stored at this point. The wiring list creation unit 103 saves the created wiring list in the wiring list storage unit 122. Note that information indicating the work flag is saved in the wiring list storage unit 122 in accordance with an input operation by the worker HM after the start of the wiring work.

[0052] The database synthesis unit 105 associates the wiring ID with the wiring model unique ID included in the design information and adds it to the synthesis table as shown in Fig. 7 (step S13). Note that the CAD data of the wiring target product, the drawing data on which the CAD data was created, information indicating the wiring model unique ID corresponding to the wiring ID, etc. are stored in the wiring design information storage unit 121 as design information.

[0053] The XR creation receiving unit 102 determines whether or not there is an instruction to create an XR model through an operation input by the worker HM (step S14). If it is determined that there is an instruction to create an XR model (step S14: Yes), the XR creation receiving unit 102 creates an XR model using the XR development software stored in the first storage unit (step S15). The created XR model is sent to the smart glasses 300, and the sent XR model is stored in the third storage unit 330 as support information 331.

[0054] In this embodiment, the worker HM selects a folder for the product ID of the wiring target from the wiring design information stored in the wiring design information storage unit 121. When the worker HM clicks the "Execute" icon, the XR creation receiving unit 102 inputs the CAD data stored in the folder for the product ID selected by the worker HM into the XR development software. This creates an XR model including the CAD data stored in the folder for the selected product ID. As illustrated in FIG. 11, the XR creation receiving unit 102 creates an XR model including CAD data of the product PD1 to be wired. Note that FIG. 11 is an example of an XR model projected on the smart glasses 300, in which the XR model is superimposed on the product PD1. Furthermore, the XR creation receiving unit 102 saves the created XR model in the folder for the product PD1 in the wiring design information storage unit 121.

[0055] Furthermore, the XR creation receiving unit 102 creates an XR model representing information on the wiring list stored in the folder for the selected product ID from the wiring list stored in the wiring list storage unit 122. As illustrated in FIG. 11 , the XR creation receiving unit 102 creates an XR model including information on the wiring list stored in the wiring list storage unit 122, virtual icons indicating work flag information such as “completed,” “pending,” and “incomplete,” and icons for selecting and operating the wiring ID, line type, etc. corresponding to each assembly order of the wiring displayed in the wiring list, such as icons indicating “previous” and “next.” The icon indicating “incomplete” is selected by the worker HM, for example, when canceling completion. Specifically, the worker HM selects the icon indicating “incomplete” by operating a wireless mouse wirelessly connected to the smart glasses 300.

[0056] 11, the wiring model CAD123 corresponding to the wiring assembly order "No. 1," the wiring ID "Cable-1," the task flag "Complete," the type "Line Type A," and the length "100 mm," which are indicated by the dashed line selected by the worker HM, is displayed in an emphasized manner. The wireless mouse connected to the smart glasses 300 is an example of an input receiving unit according to the present disclosure. In the example of FIG. 11, the wiring model CAD123 is displayed in an emphasized manner by adding a pattern to the wiring model CAD123. The XR creation receiving unit 102 converts the graphic data of the product PD1 including the wiring model CAD123 into an XR model so that the wiring model CAD123, which indicates the wiring graphic, is emphasized more than the other parts. Therefore, as illustrated in FIG. 11, the wiring model CAD123 is displayed in an emphasized manner. Methods for emphasizing the wiring include not only adding a pattern to the wiring, but also thickening the wiring, changing the color representing the wiring, highlighting the wiring, and painting a marker on the wiring.

[0057] If it is determined that there is no creation instruction (step S14: No), the XR creation receiving unit 102 repeats the processing of step S14 until there is a creation instruction from the worker HM.

[0058] Next, the operation of the wiring work support process, which represents the process of supporting the wiring work by the wiring work support device 100, will be described. As a premise, when the worker HM starts wiring work, information including the name of the worker HM and the wiring ID of the work to be performed is input to the terminal device 400, and this information is transmitted to the wiring work support device 100 connected via the network NW. This starts the wiring work support process shown in Fig. 10.

[0059] The information acquisition unit 101 acquires information including the name of the worker HM and the wiring ID of the worker HM performing the work (step S21). The database construction unit 104 adds the name of the worker HM and the wiring ID to the wiring work database stored in the wiring work DB storage unit 123 (step S22). The database construction unit 104 adds a wiring model unique ID corresponding to the wiring ID to the wiring work database based on the combination table stored in the combination table storage unit 124. The database construction unit 104 also adds the date and time of input to the terminal device 400 by the worker HM as the work date and time to the wiring work database. The database construction unit 104 constructs a wiring work database for each product.

[0060] The software setting reception unit 106 determines whether software setting information has been input by the operation of the worker HM (step S23). Specifically, the worker HM clicks, with a mouse, an icon for setting software provided in the software setting reception unit 106, and inputs, for example, the color and transparency for highlighting wiring, connected devices, etc. If it is determined that software setting information has been input (step S23: Yes), the software setting reception unit 106 stores the input and set software setting information in the software setting information storage unit 125 (step S24). The input and set software setting information is sent to the smart glasses 300, and the sent software setting information is stored in the third storage unit 330 as support information 331.

[0061] If it is determined that no software setting has been input (step S23: No), the process proceeds to step S25.

[0062] The XR creation receiving unit 102 aligns the product PD1 to be wired with the XR model (step S25). A typical example of alignment is displaying a target marker. For example, a manager, worker HM, or the like attaches a target marker printed on paper to a flat surface, and the CAD data is projected at a position and angle relative to the target marker. In response to an instruction from the XR creation receiving unit 102, the control unit 305 of the smart glasses 300 displays the XR model included in the support information 331 stored in the third storage unit 330 on the glasses display unit 301 (step S26). Specific processes in steps S25 and S26 are described below.

[0063] In this embodiment, a manager, worker HM, or the like affixes target markers to flat areas of the actual product PD1. Information indicating the target markers is stored in the wiring design information storage unit 121. The control unit 305 of the smart glasses 300 sends image information representing the captured image output by the imaging unit 302 to the wiring work assistance device 100 in real time via the communication unit 304. When the XR creation receiving unit 102 recognizes a target marker in the sent image information that matches a target marker stored in the wiring design information storage unit 121, it sends an instruction to the smart glasses 300 to display an XR model. The control unit 305 of the smart glasses 300 displays the XR model included in the assistance information 331 stored in the third storage unit 330 on the glasses display unit 301, as illustrated in FIG. 11 .

[0064] As illustrated in FIG. 11, when the worker HM selects the wiring assembly order "No. 1" and the wiring ID "Cable-1" surrounded by the dashed line, the wiring model CAD123 corresponding to the corresponding work location is highlighted in the product PD1.

[0065] The wiring list creation unit 103 and the database creation unit 104 determine whether or not input information for a work flag has been received (step S27). If it is determined that input information for a work flag has been received (step S27: Yes), the wiring list creation unit 103 and the database creation unit 104 add the input information for the work flag to the wiring list stored in the wiring list storage unit 122 and the wiring work database stored in the wiring work DB storage unit 123 (step S28). For example, when the worker HM selects the flag icon for "Complete" from among the virtual flag icons for "Complete," "Suspended," and "Incomplete" that indicate work flag information shown in FIG. 11, the control unit 305 sends information for the selected work flag for "Complete" and information for the corresponding wiring ID to the wiring work assistance device 100 via the communication unit 304. The wiring list creation unit 103 and the database creation unit 104 add "Complete" to the work flag corresponding to the wiring ID included in the received information in the wiring list and the wiring work database, respectively. At this point, information about the work flag corresponding to the wiring ID included in the transmitted information is stored in the wiring list storage unit 122. As shown in Fig. 5, for example, information indicating "completed" is stored in the work flag corresponding to the wiring ID "Cable-1."

[0066] If it is determined that there is no input information for the work flag (step S27: No), the wiring list creator 103 and the database creator 104 repeat the process of step S27.

[0067] The wiring list creation unit 103 determines whether the input information of the work flag added in step S28 is “completed” information (step S29). If it is determined that the information is “completed” information (step S29: Yes), the wiring list creation unit 103 determines whether all work flags corresponding to the wiring IDs in the wiring list are “completed” (step S30). If it is determined that all work flags corresponding to the wiring IDs in the wiring list are “completed” (step S30: Yes), the information acquisition unit 101 determines whether there is input information to continue wiring work for the same type of product (step S31). The information acquisition unit 101 determines whether there is input information indicating that wiring work for the same type of product will continue, for example, from the terminal device 400. If it is determined that there is no input information to continue wiring work for the same type of product (step S31: No), the wiring work support process ends. For example, if the wiring work for the first product has been completed but the wiring work for the second product of the same type will not be performed, the wiring work support process ends.

[0068] If it is determined in step S29 that the information is not "completed" (step S29: No), the process returns to step S27. If it is determined in step S30 that all of the task flags corresponding to the wiring IDs in the wiring list are not "completed" (step S30: No), the process returns to step S27.

[0069] In step S31, if it is determined that there is input information to continue wiring work on the same type of product (step S31: Yes), the process returns to step S21.

[0070] Furthermore, in parallel with the wiring work support process, the error display unit 107 performs the error display process shown in Fig. 12. The error display process refers to a process in which, for example, when the data capacity, number of polygons, etc. of the XR model that can be displayed by hardware or software is exceeded, the error display unit 107 instructs the smart glasses 300 to display an error warning, and the glass display unit 301 displays the error warning.

[0071] The error display unit 107 determines whether an error has occurred (step S51). If it is determined that an error has occurred (step S51: Yes), the error display unit 107 sends an instruction to display an error warning to the smart glasses 300. The control unit 305 of the smart glasses 300 displays the error warning on the glass display unit 301 (step S52).

[0072] For example, when the control unit 305 of the smart glasses 300 recognizes that the data capacity, number of polygons, etc. of the XR model that can be displayed by hardware or software has been exceeded, the control unit 305 sends error information indicating the error to the wiring work assistance device 100 via the communication unit 304. The information acquisition unit 101 acquires the error information. Based on the error information acquired by the information acquisition unit 101, the error display unit 107 sends an instruction to display an error warning on the smart glasses 300. When the control unit 305 sends an instruction to display the error warning to the glass display unit 301, the glass display unit 301 displays the error warning. For example, the glass display unit 301 displays information such as, "The data capacity of the XR model exceeds the hardware specifications." Information indicating the content of the error warning is stored in the first memory unit 120 of the wiring work assistance device 100 and the third memory unit 330 of the smart glasses 300.

[0073] If it is determined that no error has occurred (step S51: No), the error display unit 107 repeats the process of step S51.

[0074] As described above, the wiring work support system 1 according to the embodiment displays an XR model that represents support information including an image representing the product PD1 and wiring information that represents attribute information including the identification number and type of wiring. When displaying the XR model, the XR model representing the product PD1 is superimposed on an image of the product PD1 in the actual scene captured by the imaging unit 302 of the smart glasses 300. Therefore, it is possible to support the improvement of the efficiency of wiring work taking into account the wiring information.

[0075] (Variation) In the embodiment, in step S26 of the wiring work support process, the control unit 305 of the smart glasses 300 displays the XR model included in the support information 331 stored in the third storage unit 330 on the glasses display unit 301 as illustrated in FIG. 11 . However, the present disclosure is not limited to this. For example, when the worker HM performs wiring work corresponding to the assembly order "No. 1," wiring ID "Cable-1," type "wire type A," and length "100 mm" illustrated in FIG. 11 , the control unit 305 may display on the glasses display unit 301 an image highlighting the kitting box containing the wiring of "wire type A" that the worker HM will be wiring on. As illustrated in FIG. 13 , wires WA to WH corresponding to wire types A to H are contained in kitting boxes KB1 to KB8, respectively. For example, a manager, worker HM, or the like may attach unique QR codes (registered trademark) containing information on each wire type to kitting boxes KB1 to KB8. When the control unit 305 recognizes a QR code attached to a kitting box KB1 containing wiring WA of "line type A," which indicates the type of wiring to be worked on, in an image captured by the imaging unit 302, the control unit 305 displays a highlight image HL that highlights the kitting box KB1. This prevents the worker HM from picking up the wrong wiring. The XR creation receiving unit 102 identifies the type of wiring to be worked on based on the wiring work database stored in the wiring work DB storage unit 123 and the wiring list stored in the wiring list storage unit 122, and sends the identified wiring type to the smart glasses 300. When the control unit 305 in the smart glasses 300 recognizes a QR code representing the type of wiring sent by the XR creation receiving unit 102 in an image captured by the imaging unit 302, the control unit 305 identifies the kitting box to which the recognized QR code is attached as the kitting box containing the type of wiring to be worked on. The XR creation receiving unit 102 synthesizes image data so that an image highlighting the kitting box identified by the control unit 305 is displayed. The QR code is an example of container association information according to the present disclosure. The XR creation reception unit 102 is an example of a wiring type identification means according to the present disclosure. The control unit 305 is an example of a container identification means and a container image data identification means according to the present disclosure. The line types A to H are an example of type information according to the present disclosure.Each wire is placed in a kitting box corresponding to the type of wire, but the wires may be placed in a bag, container, position, location, etc., without being limited to kitting boxes.

[0076] In the embodiment, the control unit 305 in the smart glasses 300 displays the XR model on the glasses display unit 301 in accordance with the instruction information sent from the wiring work assistance device 100 to the smart glasses 300, but the present disclosure is not limited to this. The XR creation receiving unit 102 in the wiring work assistance device 100 may directly send an instruction to the glasses display unit 301 in the smart glasses 300 to display the XR model, and the glasses display unit 301 may display the XR model in accordance with the instruction.

[0077] In the embodiment, the XR model was created using commercially available XR development software, but this is not limited to this, and independently developed XR software may also be used, or any type of XR software may be used as long as it achieves the effects of the present disclosure.

[0078] In the embodiment, the worker HM inputs the work flag information from the virtual icon of the wiring list in the XR display, but the present disclosure is not limited to this. The worker HM may input the work flag information from the terminal device 400, and the input work flag information may be sent to the wiring work assistance device 100.

[0079] Furthermore, when it comes to the stage of assisting the wiring work, the designer stores the wiring design information 231, including CAD data of the product to be wired, in the second storage unit 230 of the information management device 200 from his / her own terminal device 400. However, the present disclosure is not limited to this. The designer may store the wiring design information 231, including CAD data of the product PD1, representing a configuration in which parts unnecessary for assisting the wiring work, aligning the product PD1 with the XR model, such as screws, have been removed, in the second storage unit 230. Therefore, the XR model of the product PD1, representing a configuration in which unnecessary parts have been removed, is displayed on the glasses display unit 301. By not displaying unnecessary information, improved responsiveness during XR projection and improved workability can be expected.

[0080] In the embodiment, an error warning is displayed when the data capacity, number of polygons, etc. of the XR model that can be displayed by hardware or software is exceeded, but the present disclosure is not limited to this, and any error warning may be displayed. For example, an error warning may be displayed when the control unit 305 of the smart glasses 300 loses sight of a target marker for XR alignment. In this case, the glasses display unit 301 displays an error warning such as "Target marker lost."

[0081] In the embodiment, the XR creation receiving unit 102 aligned the wiring target product PD1 and the XR model by displaying a target marker. However, any other method may be used to align the wiring target product PD1 and the XR model as long as the advantageous effects of the present disclosure are achieved. For example, the XR creation receiving unit 102 may align the wiring target product PD1 and the XR model by using an edge recognition method. In the edge recognition method, the edges of the product PD1 as the XR model and the edges of the product PD1 in real space are recognized based on information about the captured image captured by the imaging unit 302, and the XR model of the product PD1 is superimposed on the product PD1 in real space.

[0082] Furthermore, the control unit 305 of the smart glasses 300 may record a work video captured by the imaging unit 302. By viewing the work video, an inexperienced worker HM is expected to improve his / her work skills. Furthermore, the control unit 305 may stream a video of the wiring work of the inexperienced worker HM in real time. This allows the experienced worker HM to view the real-time video and send instructions, including audio, to the inexperienced worker HM.

[0083] Furthermore, the database construction unit 104 may add a work end date and time to the wiring work database in addition to the work start date and time. Thus, the database construction unit 104 may calculate the work time by the difference between the end date and time and the start date and time, and add the calculated work time. Furthermore, the database construction unit 104 may calculate the average work time for each combination of a wiring ID and a wiring model unique ID. This allows an inexperienced worker HM to be assigned to wiring work that does not require much work time, in other words, a wiring work that has a small average work time.

[0084] Furthermore, when multiple people perform wiring work in parallel, an XR model may be displayed on the glass display unit 301 of the smart glasses 300 of each of the multiple people. Note that the information of the wiring list displayed on the glass display unit 301 of the smart glasses 300 of the multiple people is synchronized. For example, assume that any one of the multiple people inputs work completion flag information for a certain wiring ID from a flag icon on XR or the terminal device 400. In this case, in the wiring list displayed on the glass display unit 301 of all the multiple people, the work flag for the corresponding wiring ID is displayed as "completed." This makes it possible to avoid unnecessary work such as rewiring a completed wire.

[0085] In the embodiment, it is assumed that the CAD data includes a wiring model, but the present disclosure is not limited to this. A virtual wiring may be generated even if the CAD data does not include a wiring model. The XR creation receiving unit 102 aligns the XR model with a product PD1, which includes only a housing and devices before the wiring work. An example in which a virtual wiring is generated from device A to device B will be described. First, when generating a virtual wiring from device A, the worker HM operates a wireless mouse wirelessly connected to the smart glasses 300 on the screen viewed on the display unit 301 of the smart glasses 300 to input designation information specifying via point 1 in space. As a result, the XR creation receiving unit 102 generates an XR model of the virtual wiring drawn from device A to via point 1. The worker HM specifies the next via point 2. As a result, the XR creation receiving unit 102 generates an XR model of the virtual wiring provided between via point 1 and via point 2. The XR creation receiving unit 102 ultimately generates an XR model of the virtual wiring to device B. Since the XR model of the virtual wiring provides an estimate of the length of each wire, the length of the wire used per product PD1 can be determined by type, making it possible to purchase and arrange the correct amount of wire. The worker HM may input information such as the diameter, minimum bend radius, and coating color of the virtual wiring through the software setting reception unit 106. The wireless mouse wirelessly connected to the smart glasses 300 is an example of an operation input information reception unit according to the present disclosure. The designation information is an example of operation input information according to the present disclosure. The XR creation reception unit 102 is an example of a drawing unit according to the present disclosure.

[0086] In the embodiment, the various types of information in the first storage unit 120 are provided in the wiring work assistance device 100, but may be provided outside the wiring work assistance device 100. For example, the various types of information in the first storage unit 120 may be provided in a cloud-type server outside the wiring work assistance device 100.

[0087] In the embodiment, the support information is displayed on the glass display unit 301 of the smart glasses 300, but the support information may be displayed on any type of portable device used by the worker HM as long as the advantageous effects of the present disclosure are achieved. For example, the support information may be displayed on a tablet terminal used by the worker HM.

[0088] Although the preferred embodiments have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope of the claims.

[0089] Various aspects of the present disclosure are summarized below as appendices.

[0090] (Appendix 1) an imaging means for imaging an actual scene at a wiring work site and outputting imaging data; a storage means for storing graphic data of an object of the wiring work and wiring information representing attribute information including identification information of the wiring to be provided on the object; a conversion means for converting the graphic data of the object and the wiring information into image data to be displayed on a display element; a combining means for receiving the imaging data output from the imaging means and the image data from the conversion means, and combining the imaging data and the image data to a display element so as to display an image of the object in the actual scene of the wiring work site represented by the imaging data and an image of the object represented by the image data of the graphic data converted by the conversion means in a superimposed manner, and to display the wiring information; Equipped with Wiring work support system. (Appendix 2) further comprising an input receiving means for receiving identification information in accordance with an input operation by an operator; the graphic data of the object includes graphic data of wiring; the synthesizing means converts the graphic data into image data so as to emphasize the wiring graphic corresponding to the identification information received by the input receiving means in the graphic data more than other parts. 2. A wiring work support system according to claim 1. (Appendix 3) The wiring information includes information on a wiring assembly sequence that indicates an assembly sequence of the wiring work. 10. A wiring work support system according to claim 1 or 2. (Appendix 4) the generating means further generates a flag icon of a work flag indicating a work status of the wiring work, the storage means stores, for each of the identification information, a work flag indicating a progress status of the wiring work for the wiring identified by the identification information; further comprising an update means for updating the task flag stored in the storage means in accordance with the information indicating the progress of the task received by the input receiving means; The conversion means converts the work flag into image data, the combining means combines image data of the work flag with image data of the wiring information so that the identification information and the corresponding work flag are displayed in association with each other on a display element. 3. A wiring work support system as described in appendix 2. (Appendix 5) the wiring information includes the identification information and type information indicating the type of wiring identified by the identification information, the storage means includes container correspondence information for identifying a container that stores the wiring for each type of the wiring, a wiring type specifying means for specifying the type of wiring to be worked on, a container specifying means for specifying, from container correspondence information, a container on which the wiring of the type specified by the wiring type specifying means is placed, and a container image data specifying means for specifying, from image data captured by the imaging means, image data of the container specified by the container specifying means, The combining means combines the image data so that an image emphasizing the container identified by the container image data identifying means is displayed. A wiring work support system according to any one of appendices 1 to 4. (Appendix 6) an operation input information receiving means for receiving operation input information in accordance with an operation by an operator; drawing means for drawing a wiring on the image of the object displayed on the display element in accordance with the operation input information received by the operation input information receiving means; Further comprising: 6. A wiring work support system according to any one of appendices 1 to 5. (Appendix 7) An image of the wiring work location is captured to generate image data, generating image data of a diagram representing an object of wiring work and attribute information of wiring provided in the image of the object; displaying the imaging data and the image data so that the image of the object included in the captured work place and a drawing of the object are superimposed on each other; Wiring work support method. (Appendix 8) On the computer, An image of the wiring work location is captured to generate image data, generating image data of a diagram representing an object of wiring work and attribute information of wiring provided in the image of the object; displaying the imaging data and the image data so that the image of the object included in the captured work place and a drawing of the object are superimposed on each other; A program that executes a process. [Explanation of symbols]

[0091] 1 Wiring work support system, 100 Wiring work support device, 101 Information acquisition unit, 102 XR creation reception unit, 103 Wiring list construction unit, 104 Database construction unit, 105 Database synthesis unit, 106 Software setting reception unit, 107 Error display unit, 120 First memory unit, 121 Wiring design information storage unit, 122 Wiring list storage unit, 123 Wiring work DB storage unit, 124 Synthesis table storage unit, 125 Software setting information storage unit, 200 Information management device, 201 Registration reception unit, 202 Wiring design specific information reception unit, 203 Wiring design information sending unit, 230 Second memory unit, 231 Wiring design information, 300 Smart glasses, 301 Glass display unit, 301a Lens, 301b Projection device, 302 Imaging unit, 303 Input unit, 304 Communication unit, 305 Control unit, 330 Third memory unit, 331 support information, 400 terminal device, 401 display unit, 1000 bus, 1001 processor, 1002 memory, 1003 interface, 1004 secondary memory device, CAD123 wiring model, WA,WB,WC,WD,WE,WF,WG,WH wiring, HL highlight image, HM worker, KB1,KB2,KB3,KB4,KB5,KB6,KB7,KB8 kitting box.

Claims

1. an imaging means for imaging an actual scene at a wiring work site and outputting imaging data; a storage means for storing graphic data of an object of the wiring work and wiring information representing attribute information including identification information of the wiring to be provided on the object; a conversion means for converting the graphic data of the object and the wiring information into image data to be displayed on a display element; a combining means for receiving the imaging data output from the imaging means and the image data from the conversion means, and combining the imaging data and the image data to a display element so as to display an image of the object in the actual scene of the wiring work site represented by the imaging data and an image of the object represented by the image data of the graphic data converted by the conversion means in a superimposed manner, and to display the wiring information; Equipped with Wiring work support system.

2. further comprising an input receiving means for receiving identification information in accordance with an input operation by an operator; the graphic data of the object includes graphic data of wiring; the synthesizing means converts the graphic data into image data so as to emphasize the wiring graphic corresponding to the identification information received by the input receiving means in the graphic data more than other parts. The wiring work support system according to claim 1 .

3. The wiring information includes information on a wiring assembly sequence that indicates an assembly sequence of the wiring work. The wiring work support system according to claim 1 .

4. the storage means stores, for each of the identification information, a work flag indicating a progress status of the wiring work for the wiring identified by the identification information; further comprising an update means for updating the task flag stored in the storage means in accordance with the information indicating the progress of the task received by the input receiving means; The conversion means converts the work flag into image data, the combining means combines image data of the work flag with image data of the wiring information so that the identification information and the corresponding work flag are displayed in association with each other on a display element. The wiring work support system according to claim 2 .

5. the wiring information includes the identification information and type information indicating the type of wiring identified by the identification information, the storage means includes container correspondence information for identifying a container that stores the wiring for each type of the wiring, a wiring type specifying means for specifying the type of wiring to be worked on, a container specifying means for specifying, from container correspondence information, a container on which the wiring of the type specified by the wiring type specifying means is placed, and a container image data specifying means for specifying, from image data captured by the imaging means, image data of the container specified by the container specifying means, The combining means combines the image data so that an image emphasizing the container identified by the container image data identifying means is displayed. The wiring work support system according to claim 1 .

6. an operation input information receiving means for receiving operation input information in accordance with an operation by an operator; drawing means for drawing a wiring on the image of the object displayed on the display element in accordance with the operation input information received by the operation input information receiving means; Further comprising: The wiring work support system according to claim 1 .

7. An image of the wiring work location is captured to generate image data, generating image data of a diagram representing an object of wiring work and attribute information of wiring provided in the image of the object; displaying the imaging data and the image data so that the image of the object included in the captured work place and a drawing of the object are superimposed on each other; Wiring work support method.

8. On the computer, An image of the wiring work location is captured to generate image data, generating image data of a diagram representing an object of wiring work and attribute information of wiring provided in the image of the object; displaying the imaging data and the image data so that the image of the object included in the captured work place and a drawing of the object are superimposed on each other; A program that executes a process.

Images