Auxiliary device and method

Abstract

This invention provides an auxiliary device for efficiently verifying whether the terminals on the destination of a cable are correct. [Solution] An auxiliary device 10 used in the process of forming a test circuit by connecting a bundled cable 32 extending from a tester 30 for performing tests on the target device to the terminals of the target device 20, comprising an imaging unit, a portable terminal 10A including a display unit for displaying visual data acquired by the imaging unit, and a main body 10B including a guide generation unit, wherein the guide generation unit generates guidance information for connecting the cable to the terminals based on connection information between the terminals 22 and the cable for forming the test circuit recorded in the manual of the target device and / or the tester, and displays the guidance information so as to overlap with first visual data including the terminals displayed on the display unit.

Description

Technical Field

[0001] The present disclosure relates to an auxiliary device and a method.

Background Art

[0002] For maintenance of electrical equipment used in solar power generation facilities and the like, a test circuit may be formed by connecting a cable of a tester to a terminal included in the electrical equipment, and various evaluation tests may be performed. At this time, it is necessary to connect correct cables to a plurality of terminals according to the test circuit, respectively. This work requires skilled techniques, and in order to assist even inexperienced workers to be able to perform this work, for example, the auxiliary technique described in Patent Document 1 has been proposed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional auxiliary techniques have mainly focused on the function of checking whether the terminals to which the cables are to be connected are appropriate, either on behalf of the worker or together with the worker (double-checking), and have not contributed to an improvement in work efficiency. Generally, a tester is accompanied by a manual that describes the correspondence between terminals and cables and the work procedure according to the test circuit to be formed. However, the arrangement and location of terminals may differ for each target device, and workers with insufficient experience need to connect a large number of cables while comparing the manual with the terminals on site, which requires a lot of time for the work. In view of the current shortage of science and engineering personnel and the shortage of successors to skills, there is a need for a work efficiency improvement technique that does not rely on personal skills. The present disclosure aims to solve at least one of the above conventional techniques.

Means for Solving the Problems

[0005] The first auxiliary device of this disclosure is an auxiliary device used in the work of forming a test circuit by connecting a cable extending from a tester for performing a test on the target device to the terminals of the target device, and comprises an imaging unit, a display unit for displaying visual data acquired by the imaging unit, and a guide generation unit, wherein the guide generation unit generates guidance information for connecting the cable to the terminals based on connection information between the terminals and the cable for forming the test circuit recorded in the manual of the target device and / or the tester, and displays the guidance information so as to overlap with the first visual data including the terminals displayed on the display unit. [Effects of the Invention]

[0006] According to this disclosure, at least one of the above-mentioned prior art problems can be solved. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram illustrating the on-site operation of the auxiliary equipment. [Figure 2] This is a functional block diagram of the auxiliary device. [Figure 3] This diagram shows an example of connection information stored in a database. [Figure 4] This diagram illustrates an example of guidance information displayed overlaid on visual data shown in real time on the display of a mobile device. [Figure 5] This diagram shows guidance information for cases where the destination terminal is not included in the visual data. [Figure 6] This diagram illustrates another example of guidance information displayed overlaid on visual data shown in real time on the display of a mobile device. [Figure 7] This diagram illustrates another example of guidance information displayed overlaid on visual data shown in real time on the display of a mobile device. [Figure 8] This is a flowchart illustrating the procedure for connecting cables using auxiliary devices. [Figure 9]This diagram shows an example of an interface screen displayed on the display unit of a mobile device. [Modes for carrying out the invention]

[0008] The first auxiliary device of this disclosure is an auxiliary device used in the work of forming a test circuit by connecting a cable extending from a tester for performing a test on the target device to the terminals of the target device, and comprises an imaging unit, a display unit for displaying visual data acquired by the imaging unit, and a guide generation unit, wherein the guide generation unit generates guidance information for connecting the cable to the terminals based on connection information between the terminals and the cable for forming the test circuit recorded in the manual of the target device and / or the tester, and displays the guidance information so as to overlap with the first visual data including the terminals displayed on the display unit.

[0009] One of the features of the first auxiliary device is that it includes a guide generation unit, which generates guidance information for connecting cables to terminals and displays this guidance information superimposed on visual data including terminals acquired by the imaging unit. Furthermore, this guidance information is generated based on the manuals for the target equipment and / or the test equipment.

[0010] Using the first auxiliary device, workers can smoothly perform cable connection work by imaging the terminals of the target equipment and referring to guidance information overlaid on the visual data displayed on the display unit. Since this guidance information is generated based on the manuals for the target equipment and / or the test equipment, workers do not need to consult the manuals on-site, and can perform the work smoothly and accurately even if they do not have sufficient work experience.

[0011] The second auxiliary device of this disclosure is an auxiliary device that, in the first auxiliary device, includes a cable determination unit, the cable determination unit generates connection correctness information based on second visual data including the terminal and the cable connected to the terminal, and the connection information.

[0012] The cable determination unit of the second auxiliary device generates information on the correctness of cable connections based on visual data (second visual data) including the connected cables. Previously, the correctness of cable connections was verified by workers. The correctness information generated by the auxiliary device can be used to improve the accuracy of work and prevent accidents.

[0013] A third auxiliary device of this disclosure is an auxiliary device in which, in the second auxiliary device, the connection information includes a work procedure for forming the test circuit, and the guide generation unit generates new guide information based on the correct / incorrect information and displays it so as to overlap the first visual data and / or the second visual data.

[0014] The new guidance information generated by the guide generation unit of the third auxiliary device is based on the correct / incorrect information generated by the cable determination unit. By referring to the guidance information based on the correct / incorrect information, which is displayed overlaid on the visual data including the connected cables, the worker can proceed with the work while constantly checking the accuracy of their work, enabling them to perform the work more smoothly and accurately.

[0015] A fourth auxiliary device of this disclosure is an auxiliary device in which, in the third auxiliary device, if the correct / incorrect information includes that the cable is connected to the wrong terminal, the new guidance information includes information indicating the terminal to which the cable should be connected, or information indicating that the cable should be disconnected.

[0016] If the cables are connected in the wrong order or the test is carried out with the cables still connected to the wrong terminals, accidents such as malfunctions of the target device or the tester, or ignition may occur. The guide generation unit of the fourth auxiliary device guides the correct connection order, the correct connection destination, and the disconnection of the cable when there is an error in the work procedure, for example, an error in the connection order of the cables or the connection destination terminal. The operator can take appropriate measures to prevent malfunctions and accidents based on this guidance information. The content of this new guidance information can be determined based on the work procedure. For example, even if the cable connected at the most recent timing is correct as the cable to be connected, but the connection destination is incorrect, the guidance information may include information guiding the correct connection destination. On the other hand, if the cable connected at the most recent timing is not yet the cable to be connected, that is, if there is a cable that should have been connected earlier than this cable, the guidance information may include information guiding the disconnection of the cable. Determining the guidance information based on the cable connection information contributes to more accurate work execution.

[0017] The fifth auxiliary device of the present disclosure is an auxiliary device in any one of the first to fourth auxiliary devices, wherein the guide generation unit displays an interface screen for receiving selection of the target device and / or the test circuit on the display unit based on a pre-stored list to prompt selection by the user, and generates the guidance information based on the connection information stored for each of the target device and / or the test circuit selected by the user.

[0018] The guide generation unit of the fifth auxiliary device generates an interface screen for receiving a selection of a target device and / or a test circuit from a user (operator), and causes the display unit to display the generated screen. This selection display is based on a pre-stored list, and connection information is pre-stored in association with each item in the list. When the operator's selection is received, the connection information corresponding to the selected target device and / or test circuit is read, and guide information is generated based on this connection information. With such a configuration, the calculation cost of the guide generation unit is reduced, and the processing efficiency of the auxiliary device is improved. The hardware requirements for the auxiliary device are relaxed, and smooth operation is possible even with limited specifications. This is particularly preferable when the auxiliary device is a portable terminal used on-site.

[0019] The sixth auxiliary device of the present disclosure is the second auxiliary device, wherein the cable determination unit includes a cable classification model learned by machine learning using training data composed of images of the labeled cables, and the cables extending from the tester are at least partially or entirely colored in different colors.

[0020] In a plurality of cables, since the whole or part of each cable is colored in a different color, the identification of the cables by the cable classification model becomes more accurate. Further, by assuming cables colored in different colors, feature extraction is made more efficient, so that the calculation cost of the cable classification model is reduced. As a result, the processing efficiency of the auxiliary device is improved. The hardware requirements for the auxiliary device are relaxed, and smooth operation is possible even with limited specifications. This is particularly preferable when the auxiliary device is a portable terminal used on-site.

[0021] The seventh auxiliary device of the present disclosure is the first auxiliary device, further including a tracking unit, wherein the tracking unit generates position information of the terminals and / or the cables in the visual data which is video, and the guide generation unit displays the guide information on the display unit so as to overlap the video based on the position information.

[0022] Thanks to the tracking unit's functionality, guidance information is overlaid on real-time visual data. Since workers can perform their tasks while only viewing the display unit, the auxiliary device can accommodate various form factors, such as wearable devices, depending on its application and location.

[0023] The eighth auxiliary device of the present disclosure is an auxiliary device in the third auxiliary device in which the guide generation unit generates a procedure list based on the connection information and displays it on the display unit, and if the correct / incorrect information includes that the cable is connected to the correct terminal, it adds information to the procedure list indicating that the corresponding procedure in the procedure list has been completed.

[0024] The procedure list displayed by the guide generation unit of the eighth auxiliary device contributes to the smooth operation of the worker. Furthermore, by adding check marks to completed procedures, the worker can check the next task and the overall progress of the work, contributing to even more accurate and smooth operation.

[0025] The first method of this disclosure is a method for performing the task of forming a test circuit by connecting a cable extending from a tester for performing a test on a target device to a terminal of the target device using a first auxiliary device, the method comprising acquiring visual data including the terminal with the imaging unit and connecting the cable to the terminal based on the guidance information displayed on the display unit so as to overlap with the visual data displayed on the display unit.

[0026] According to the method of this disclosure, because the first auxiliary device is used, workers do not need to refer to the manual on-site, and can perform the work smoothly and accurately even if they do not have sufficient work experience.

[0027] The embodiments will be described in detail below with reference to the attached drawings. The specification provides examples of specific materials and methods for realizing the technical idea of ​​the disclosure. The technical idea of ​​this disclosure is not limited to the following specific examples. The technical idea of ​​this disclosure can be modified in various ways within the scope of the matters described in the claims. In particular, it should be noted that the drawings are schematic and may differ from reality.

[0028] Figure 1 is an explanatory diagram of the on-site operation of the auxiliary device of Example 1. The auxiliary device 10 comprises a mobile terminal 10A and a main unit 10B. The mobile terminal 10A and the main unit 10B are connected via a network. In this example, the main unit 10B is installed in a location separate from the mobile terminal 10A and is connected via the internet, enabling the transmission and reception of data, etc.

[0029] The target equipment 20 is the equipment to be tested by the test device 30. In this example, the target equipment 20 is a ground fault directional relay included in a solar power generation facility. However, the target equipment 20 is not limited to the above. It can be any electrical equipment to be tested for performance verification, characteristic measurement, maintenance, calibration, etc., by connecting the test device to form a test circuit. Examples of such electrical equipment include relays other than ground fault directional relays, circuit breakers, transformers, uninterruptible power supplies, inverters, synchronous generators, load switches, battery systems, power conditioners, phase condensers, distributed power supplies, capacitor banks, reactors, insulation monitoring devices, and power meters. For any target equipment, a test circuit is formed by connecting the cable of the test device and the test is performed. The auxiliary device of the embodiment can be used to assist in the test circuit formation work for such target equipment.

[0030] The target device 20 has multiple terminals 22. When the target device 20 is in operation, the necessary operation cables (not shown) are connected to each terminal 22. On the other hand, when forming a test circuit, a test bundle cable 32 is connected together with the operation cables, or in place of the operation cables. The bundle cable 32 is a collection of multiple cables, each with a different function. The bundle cable 32 has multiple clips 32A. Each clip 32A is placed at the end of a cable 32C having a different function. Each clip 32A is connected to its corresponding terminal 22. A cable label 32B is attached near each clip 32A. The cable labels 32B are used to identify each cable 32C included in the bundle cable 32.

[0031] Meanwhile, each terminal 22 is also assigned a terminal label 24. The relationship between the terminals 22 to be connected when forming a specific test circuit and the cable 32C is described or recorded in the manual for the equipment 20 as the correspondence between the cable label 32B and the terminal label 24. By referring to this manual, the operator can correctly connect the two and form the test circuit.

[0032] Tester 30 is a typically portable device used to test the target equipment 20. In this example, tester 30 is a phase characteristic tester. A phase characteristic tester is a device that applies a predetermined electrical signal to a ground fault directional relay and checks whether the various functions of the ground fault directional relay are working correctly. Note that the tester is not limited to the above, and any device that is connected to the target equipment by cable and used for electrical testing of the target equipment is acceptable. Examples of such testers include relay testers, insulation resistance meters, earth resistance meters, withstand voltage testers, circuit injectors, transformer ratio testers, partial discharge detectors, power quality analyzers, battery testers, and synchronous testers. Any of these testers are connected to the target equipment by cable, form a test circuit, and are used to perform the test.

[0033] The tester 30 is equipped with switches 30A, knobs 30B (or sliders), and a display 30C for performing various tests. The tester 30 also includes a terminal 30E for bundling and connecting multiple cables. A bundled cable 32, which is made up of multiple cables 32C bundled together, is connected to the terminal 30E. The other end of the bundled cable 32 is composed of multiple cables 32C, and a clip 32A is placed at the end (termination) and connected to the terminal 22.

[0034] Each cable 32C is fitted with a cable label 32B, allowing workers to distinguish between the cables 32C by their labels. In this example, each cable consists of a conductor covered with an insulating layer and a protective layer, with the outermost protective layer being color-coded. The color-coded cables allow for more accurate identification by the cable identification unit 50, as will be described later. On the other hand, it is not necessary for the entire cable to be color-coded; the same effect can be achieved if only a part of it is color-coded. For example, using different colors for the cable labels 32B also constitutes color-coding the cables. Alternatively, the clips at the ends, or their covering layers, can also be color-coded.

[0035] The manuals for the target equipment 20 and / or the test equipment 30 contain descriptions or records of methods for performing various tests. In this specification, this is referred to as connection information. Connection information may include test circuit diagrams, correspondences between terminals 22 and cables 32C for forming test circuits, and work procedures. Work procedures are a list of operations (work items) to be performed in the order in which they should be performed. Work items may include, for example, the operation of disconnecting an operating cable from a specific terminal 22, and the operation of connecting a specific cable 32C to a specific terminal 22. Skilled workers can connect terminals 22 and cables 32C in the appropriate correspondences using the appropriate procedures, depending on the type of test. On the other hand, inexperienced workers often refer to the manual on-site when forming test circuits, which has been one of the factors contributing to the prolonged work time.

[0036] The tester 30 is connected to form a test circuit for performing a phase characteristic test on the target equipment 20. The type of test can be varied depending on the target equipment and purpose. Examples of such tests include electrical characteristic tests such as insulation resistance tests, withstand voltage tests, ground resistance tests, and transformer ratio tests; protective device-related tests such as operating current / voltage tests, time characteristic tests, trip-interlock tests, and phase characteristic tests; signal / communication-related tests such as communication tests and protocol tests; and battery-related tests such as capacity tests, internal resistance tests, and discharge tests.

[0037] The mobile terminal 10A of the auxiliary device 10 is a smartphone, tablet, or the like. The mobile terminal 10A is equipped with a processor, memory, camera, and touch-sensitive display, and also has a communication function with the main unit 10B. On the other hand, the main unit 10B is a computer equipped with a processor and memory. In this example, the auxiliary device 10 is equipped with a mobile terminal 10A and the main unit 10B. However, the auxiliary device is not limited to this form and may consist of a single portable device that has the functions of both the mobile terminal 10A and the main unit 10B.

[0038] Next, the functions of each part of the auxiliary device 10 will be described in detail. Figure 2 is a functional block diagram of the auxiliary device 10. The mobile terminal 10A comprises a control unit 12A, a display unit 12B, an imaging unit 12C, and a communication unit 12E.

[0039] The control unit 12A controls the entire mobile terminal 10A. The functions of the control unit 12A are implemented by a processor or the like. The mobile terminal 10A may also include a storage unit (not shown). The storage unit stores programs and necessary data. The functions of the storage unit are implemented by memory or the like.

[0040] The display unit 12B is a touch-sensitive display that displays visual data obtained by the imaging unit 12C, as well as guidance information and interface screens generated by the guide generation unit 44, which will be described later. The display method and touch technology of the display are not particularly limited and known methods can be used in combination as appropriate.

[0041] Examples of display formats include liquid crystal displays, organic EL displays, and micro-LED displays. Examples of touch technologies include capacitive, resistive, infrared, and optical touch technologies. By employing a touch-sensitive display, the display unit 12B also functions as an input device that accepts input from the operator.

[0042] The imaging unit 12C is a device for acquiring visual data such as the terminals 22 of the target device 20, and in this example, it is a camera. The imaging unit 12C may be something other than a camera, as long as it can acquire visual data including the terminals 22 of the target device 20, but it is preferable to have an image sensor. In addition to the image sensor, it may also be further equipped with a depth sensor.

[0043] The visual data acquired by the imaging unit 12C may be a still image or a video. If the visual data is a video, the operator's understanding of the guidance information when it is overlaid on the image is more likely to be improved.

[0044] The communication unit 12E is a function implemented by hardware and software that handles network communication between the mobile terminal 10A and the main unit 10B. The hardware includes network interfaces such as a "Wi-Fi" module, a cellular communication module, and a "Bluetooth" module. Furthermore, it includes driver programs and software such as a communication protocol stack that correspond to this hardware. The communication unit 12E is responsible for real-time data transmission and reception between the main unit and the communication unit.

[0045] The main unit 10B is a computer equipped with a processor and memory, and further comprises software to realize each function. First, the control unit 58 is implemented by the processor and has the function of comprehensively controlling the operation of the entire auxiliary device 10. The storage unit 56 is a part for storing various information, data, and programs necessary for the operation of the auxiliary device 10, and holds relevant data used, for example, for generating connection information and guidance information. In addition, the communication unit 54 has the function of sending and receiving data with the target device 20, the test device 30, the mobile terminal 10A, and / or external devices, enabling the acquisition and transmission of information necessary for the operation of the auxiliary device 10.

[0046] The visual data acquisition unit 40 is a function realized by the processor executing a program stored in memory, and it receives visual data acquired by the imaging unit 12C and stores it in the storage unit 56. This communication is performed by the communication unit 54, and the function of the communication unit 54 is the same as that of the communication unit 12E of the mobile terminal 10A.

[0047] The terminal detection unit 42 is a function realized by the processor executing a program stored in memory, and detects the terminals 22 of the target device 20 included in the acquired visual data. The method for detecting the terminals 22 from the visual data is not particularly limited, but one method is to detect the terminals 22 from the visual data using a known object detection algorithm.

[0048] In one configuration, the terminal detection unit 42 includes a terminal classification model for detecting and identifying terminals included in visual data. The basic structure of the terminal classification model can be a combination of "YOLO" and "Faster R-CNN," etc. When using the above model as the basic structure, machine learning is performed in advance using an image dataset of labeled terminals 22.

[0049] Furthermore, the terminal detection unit 42 can also detect terminals 22 using edge detection algorithms such as "Canny" in addition to object detection algorithms. If the visual data is video, the terminals 22 in each image can be detected by dividing the image into frames and using the method described above.

[0050] The terminal detection unit 42 may further detect terminal labels 24 and identify terminals by reading the information written on the terminal labels 24. Reading the terminal labels 24 can be achieved by cutting out the area containing the terminal labels 24 from the area of ​​terminal 22 cut out by the object detection algorithm, and reading the characters in that area using an OCR library such as "Tessaeract". The method for cutting out the area containing the terminal labels 24 from the area of ​​terminal 22 is not particularly limited, but the same object detection algorithm as for detecting terminal 22 can be used.

[0051] The terminal detection unit 42 may also refer to a pre-stored list for each type of target device 20 to check if the string read from the terminal label 24 matches a specific string. In other words, the OCR detection accuracy may be supplemented by the pre-stored list. Generally, the terminal labels 24 provided by the target device 20 are described or recorded in the manual for the target device 20. The above list is prepared in advance based on the manual for the target device 20 and stored in the storage unit 56. By comparing the string read from the terminal label 24 with the list, more accurate reading of the terminal label 24 becomes possible. If the read string does not exist in the list, the system may be configured to consider it a reading failure and attempt to read the terminal label 24 again.

[0052] The procedure for training the terminal classification model of the terminal detection unit 42 involves first collecting visual data of terminals of various shapes and structures as training data. The collected visual data includes the type, color, label, and positional relationship of the terminals. Furthermore, a comprehensive dataset is constructed, including data taken when terminals are connected, disconnected, and under different background environments. This ensures a diverse range of training data to improve the discrimination accuracy of the terminal detection unit 42.

[0053] The terminal detection unit 42 is trained using the above dataset to learn the characteristics of terminals from the input visual data. In this training, features such as the shape, color, dimensions, and arrangement pattern of the terminals are extracted, and a classification model is generated. Specifically, supervised learning is applied, and training is performed by associating the correct label for each terminal with the visual data. After training, the terminal detection unit 42 becomes capable of detecting and identifying terminals with high accuracy from unknown visual data.

[0054] The tracking unit 46 is a function implemented by the processor executing a program stored in memory, and tracks the position of the terminal 22 identified in the acquired visual data. The tracking unit 46 works in conjunction with the terminal detection unit 42. As a specific example, when the image acquired by the imaging unit 12C is divided into frames, the object detection algorithm of the terminal detection unit 42 extracts the area of ​​the terminal 22 from one of the frames. The tracking unit 46 tracks the movement of the detected terminal 22. Once tracking by the tracking unit 46 has started, detection of the terminal 22 by the terminal detection unit 42 is not necessary unless tracking fails or the terminal 22 becomes invisible in the visual data.

[0055] Furthermore, it is preferable that the tracking unit 46 is configured to track the movement of the cable 32C detected by the cable detection unit 48, which will be described later, using a similar procedure. The tracking unit 46 generates position information for the terminal 22 and the cable 32C in the visual data (if the visual data is video, it generates position information for each frame, or for each frame at predetermined intervals), and passes it to the guide generation unit 44. This makes it easier for the guide generation unit 44 to display guidance information so as to overlap with the visual data.

[0056] For tracking, known tracking algorithms such as "Optical Flow," "Kalman Filter," and "Deep SORT" can be used. Note that tracking may fail if the mobile terminal 10A is moved quickly; in this case, the system may be configured to perform object detection by the terminal detection unit 42 at regular intervals, regardless of the success or failure of tracking. By providing both the terminal detection unit 42 and the tracking unit 46, real-time overlay display of the guidance display becomes possible.

[0057] The guide generation unit 44 is a function realized by the processor executing a program stored in memory. Based on the connection information recorded in the manual for the target equipment 20 and / or the tester 30, it generates guidance information in accordance with the work procedure and displays the guidance information on the display unit 12B so as to overlap with the visual data including the terminals 22 displayed on the display unit 12B.

[0058] The manuals for the target equipment 20 and / or the test equipment 30 contain connection information for terminals 22 and cable 32C for forming the test circuit. Preferably, the connection information is pre-organized and stored in a database for each type of target equipment 20, test equipment 30, and test circuit. However, a similar effect can be obtained even if the information is organized by each type of target equipment 20 and test circuit. This is because although the specifications of the test equipment 30 often differ depending on the manufacturer, the work procedure is often the same even when using different test equipment 30s, once the test circuit is identified.

[0059] Figure 3 is a diagram showing an example of connection information stored in the database. The connection information 60 includes the correspondence between terminal 22 and cable 32C, as well as work procedures. The work procedures include tasks such as "disconnecting the P1 and P2 terminals of the relay" (procedure 60A) and "disconnecting the a and c terminals of the relay" (procedure 60B), which involve removing the operation cable connected to terminal 22, and tasks such as "connecting the P1 cable to terminal P1 of the relay" (procedure 60C) and "connecting the P2 cable to terminal P2 of the relay" (procedure 60D), which involve connecting cable 32C to terminal 22. The connection information 60 may be automatically generated based on the manual for the target equipment 20 and / or the tester 30, or it may be generated manually (i.e., by an operator). The connection information 60 is generated in advance, compiled into a database, and stored in the storage unit 56.

[0060] The "guidance information" generated by the guide generation unit 44, which follows the work procedure, is generated based on the connection information. Figure 4 shows an example of guidance information displayed overlaid on the visual data 66 displayed in real time on the display unit 12B of the mobile terminal 10A. Guidance information 64A first includes information guiding the connection of a specific cable 32C to a specific terminal 22. This guidance information 64A includes a cursor image 62A that points to the specific terminal 22, and text information 62B that displays the type of cable 32C to be connected to this terminal 22. Guidance information 64A may remain displayed on the display unit 12B until the connection status, described later, can be confirmed.

[0061] The cursor image 62A may be displayed only when the visual data 66 includes the terminal 22 to be connected. Figure 5 shows the guidance information 64B when the visual data 66 does not include the terminal 22 to be connected. The guidance information 64B includes text information 62B and a cursor image 62C. In this case, since the visual data 66 does not include the terminal 22 to be connected, the cursor image 62C indicates off-screen. That is, it indicates that the mobile terminal 10A should be moved to the left.

[0062] Furthermore, the guidance information may include information to inform the user if cable 32C is connected to the wrong terminal 22. Figure 6 shows another example of guidance information displayed overlaid on visual data 66 displayed in real time on the display unit 12B of the mobile terminal 10A. The guidance information 64C includes a frame image 62E that points to the incorrectly connected cable 32C, a cursor image 62F that points to the terminal 22 to which cable 32C should be connected, and text information 62G that instructs the user to remove (disconnect) or move the connected cable 32C.

[0063] Furthermore, the guidance information may also include work procedures (procedure lists) and information indicating that a predetermined procedure in the procedure list has been completed. Figure 7 is a diagram showing another example of guidance information displayed overlaid on visual data 66 displayed in real time on the display unit 12B of the mobile terminal 10A. The guidance information 64E includes a procedure list 62H generated by the guide generation unit 44 and information 62J representing already completed procedures, consisting of checks and strikethroughs. The procedure list 62H may be generated based on connection information. The information 62J representing already completed procedures may be generated based on correct / incorrect information generated by the cable determination unit 50, which will be described later.

[0064] Preferably, the images and text information included in the guidance information are generated in advance, either automatically or manually, and are stored in the storage unit 56, categorized according to the work procedure (procedure list), the state of the visual data (whether the target terminals and cables are included in the image, etc.), and the correctness of the work (correct or incorrect), and linked to each respective condition.

[0065] Returning to Figure 2, the cable detection unit 48 is a function realized by the processor executing a program stored in memory, and it detects the cable 32C of the tester 30 included in the acquired visual data. The method for detecting the cable 32C from the visual data is not particularly limited, but the same algorithm as that used in the terminal detection unit 42 can be employed. In this case, instead of the terminal label 24, discrimination may be performed using the cable label 32B, or discrimination may be performed based on the color of the cable 32C. Discrimination by color is particularly preferable due to its high accuracy.

[0066] Furthermore, the identification of cables 32C by character recognition and color recognition by the cable detection unit 48 can be performed using the same method as that used in the terminal detection unit 42. That is, the cable detection unit 48 is equipped with a cable classification model for detecting and identifying cables included in visual data, and by inputting visual data, it detects and identifies cables 32C included in the visual data.

[0067] The procedure for training the cable classification model of the cable detection unit 48 involves first collecting visual data of cables 32C (which may include clips 32A and cable labels 32B, collectively referred to as "cables, etc.") of various shapes and structures as training data. The collected visual data includes the type, color, label, and positional relationship of the cables, etc. Furthermore, a comprehensive dataset is constructed, including data taken when the cables 32C are connected, when they are not connected, and under different background environments. This ensures a diverse range of training data to improve the discrimination accuracy of the cable detection unit 48.

[0068] The cable detection unit 48 is trained using the above dataset to learn the characteristics of cables and the like from the input visual data. In this training, features such as the shape, color, dimensions, and arrangement pattern of cables and the like are extracted, and a classification model is generated. Specifically, supervised learning is applied, and training is performed by associating the correct labels for cables and the like with the visual data. After training, the cable detection unit 48 becomes capable of detecting and identifying cables 32C with high accuracy from unknown visual data.

[0069] The cable detection unit 50 is a function realized by the processor executing a program stored in memory. It detects the connection of cable 32C to terminal 22 and, based on the detection results from terminal detection unit 42 and cable detection unit 48, determines whether the correspondence between the connected terminal 22 and cable 32C is correct or incorrect.

[0070] The cable determination unit 50 first detects that the cable 32C is connected to the terminal 22 (connection status) based on the visual data acquired by the visual data acquisition unit 40. In this specification, for convenience, the visual data after the terminal 22 to be guided and the cable 32C have been connected is sometimes referred to as the second visual data, and the visual data when the terminal 22 to be guided and the cable 32C are not connected is sometimes referred to as the first visual data.

[0071] The method by which the cable determination unit 50 detects the connection status is not particularly limited, but a method (finite difference method) can be employed in which the cable 32C is in an unconnected state is compared with the visual data at the time of determination (which may be the "second visual data") to detect the change. In the finite difference method, the cable determination unit 50 first preprocesses the acquired visual data. The preprocessing method is not particularly limited, but examples include resizing the visual data, color conversion, noise reduction, edge detection, region of interest (ROI) extraction, binarization, and contour detection. Template matching or object detection models can be used for ROI extraction. In addition, "YOLO" or "OpenCV" can be used for contour detection.

[0072] Next, if the relative position of the detected cable 32C and terminal 22 is within a predetermined range, the cable determination unit 50 determines that the cable 32C and terminal 22 are connected, and then performs a correct / incorrect determination. The method of correct / incorrect determination is not particularly limited, but examples include identification by characters on the terminal label 24 and cable label 32B (character recognition) and identification by color. Note that if the terminal and cable 32C in the visual data have already been detected, identified, and tracked by the terminal detection unit 42 and cable detection unit 48, respectively, there is no need to perform further detection and identification.

[0073] By comparing the correspondence between the connected terminal 22 and cable 32C with the connection information, the cable determination unit 50 generates correct / incorrect information for cable 32C connected to terminal 22. The cable determination unit 50 generates discrimination information regarding whether the connection is correct or incorrect and sends it to the guide generation unit 44, etc.

[0074] Next, the procedure for connecting cables using the auxiliary device 10 will be described. Figure 8 is a flowchart showing the procedure for connecting cables using the auxiliary device.

[0075] First, in step S10, the guide generation unit 44 displays an interface screen on the display unit 12B of the mobile terminal 10A for accepting the selection of the target equipment 20, the test device 30, and the test circuit based on a pre-stored list. The type of target equipment 20 and the test circuit (the type of test to be performed) can be determined from visual data, but by accepting the selection of at least the target equipment 20 or the test circuit from the user (operator) and utilizing that information, processing can be sped up, and processing can be performed even on a computer (tablet) with low processing power.

[0076] Figure 9 shows an example of an interface screen displayed on the display unit 12B of the mobile terminal 10A. The interface screen 70 includes a first drop-down list 71 for selecting the target equipment 20, a second drop-down list 72 for selecting the test equipment 30, a third drop-down list 73 for selecting the test circuit, a cancel button 74, and a confirm button 75. The contents of the lower-level drop-down lists change in conjunction with the selection of the higher-level drop-down lists. That is, the list displayed in the lower-level drop-down lists changes in conjunction with the selection of the higher-level drop-down list displayed based on a certain list. Specifically, the selectable contents of the test equipment 30 in the second drop-down list 72 and the selectable contents of the test circuit in the third drop-down list 73 change in conjunction with the selection of the first drop-down list 71. Also, the selectable contents of the third drop-down list 73 change in conjunction with the selection of the test equipment 30 in the second drop-down list 72. The images and lists necessary for these displays are stored in advance in the storage unit 56.

[0077] The guide generation unit 44 displays the interface screen 70 on the display unit 12B to prompt the user (operator) to make a selection. Based on the interface screen 70, the operator selects the target equipment 20, the test device 30, and the test circuit, touches the confirmation button 75, and transmits the selection. Based on the selection, the guide generation unit 44 reads the corresponding connection information stored in the storage unit 56. As described above, the connection information is information generated based on the manuals for the target equipment 20 and / or the test device 30, etc. In other words, once the target equipment 20, the test device 30, and the test circuit, etc., are identified, the connection information can also be easily identified. The storage unit 56 stores the connection information linked to this information. The selected connection information is used in subsequent processing. The confirmation button 75 on the interface screen 70 may also be used as a signal to start the acquisition of visual data.

[0078] Next, in step S11, visual data including the terminal 22 of the target device 20 is acquired. Specifically, visual data is acquired when the operator points the imaging unit 12C of the mobile terminal 10A at the terminal 22 of the target device. The visual data acquisition unit 40 of the main unit 10B acquires this visual data via the communication unit 12E of the mobile terminal 10A and temporarily stores it in the storage unit 56.

[0079] Next, in step S12, the terminal detection unit 42 detects the terminal 22 included in the visual data. The method for detecting the terminal 22 has already been described.

[0080] Next, in step S13, the control unit 58 refers to the connection information and determines whether all work procedures for forming the target test circuit have been completed. If all work procedures are completed as a result of this determination (step S13: YES), the processing flow ends. On the other hand, if there are incomplete work procedures (step S13: NO), the guide generation unit 44 generates guidance information corresponding to the incomplete work procedure that should be processed first, based on the connection information stored in the storage unit 56 and the detected terminals 22 (step S14). This guidance information is generated as image data, text data, or a combination of these.

[0081] Next, in step S15, the tracking unit 46 identifies the terminal 22 (hereinafter referred to as the target terminal) that is the target of the current work procedure and starts tracking. The "work procedure" here includes, for example, the work of disconnecting an already connected operational cable from the target terminal or the work of connecting a specific cable of the test equipment to the target terminal. The tracking unit 46 tracks the target terminal in real time using visual data transmitted from the mobile terminal 10A. If the target terminal temporarily disappears from the visual data (moves off-screen), it works in cooperation with the terminal detection unit 42 to re-detect it and continue tracking. This makes it possible to accurately grasp the position and status of the target terminal. Then, the guide generation unit 44 displays guidance information based on the position information generated by the tracking unit 46, overlaid on the visual data displayed in real time on the display unit 12B.

[0082] Next, in step S16, the cable detection unit 48 detects the connection of cable 32C to the target terminal. The cable detection unit 48 identifies the cable 32C included in the visual data based on the cable label 32B and the color of cable 32C. This identified cable 32C is then tracked by the tracking unit 46.

[0083] Furthermore, in step S17, the cable determination unit 50 determines whether cable 32C is connected to the target terminal. For example, the difference method is used to determine whether cable 32C and terminal 22 are connected. If it is determined in this determination that cable 32C and terminal 22 are not connected (step S17: NO), the process from step S16 onwards is repeated. On the other hand, if it is determined that cable 32C and terminal 22 are connected (step S17: YES), then the correctness of the connected cable 32C is determined (step S18).

[0084] The correctness of the connected cable 32C is determined by comparing the correspondence between the terminal 22 detected and tracked by the terminal detection unit 42, the cable detection unit 48, and the tracking unit 46 with the connection information. If the comparison determines that the correct cable 32C is connected to the target terminal (step S18: YES), the operation on the target terminal is completed, and information indicating that the current work procedure is complete is stored in the storage unit 56. The process then returns to step S13. On the other hand, if it is determined that the correct cable is not connected (step S18: NO), the guide generation unit 44 generates new guidance information indicating that there is an error in the operation performed based on the current work procedure and the correct operating method, and displays it on the display unit 12B (step S19). The process then returns to step S15.

[0085] Following the above procedure, all necessary guidance information will be appropriately displayed on the display of the mobile terminal 10A until all work procedures included in the connection information are completed. This allows the worker to perform cable connections reliably and efficiently while referring to visual guidance.

[0086] Steps S16 to S18 describe the case where the work (operation) is to connect cable 32C to terminal 22. If the work is to disconnect an already connected operational cable from the target terminal, steps S16 to S18 can be changed according to the work. Also, if the guidance information includes a procedure list, in step S19, when the completion information for the current work procedure is generated, the guide generation unit 44 may be configured to update the guidance information and display information indicating that the predetermined procedure has been completed. [Explanation of symbols]

[0087] 10 Auxiliary device, 10A Mobile terminal, 10B Main unit, 20 Target equipment, 22 Terminal, 24 Terminal label, 30 Tester, 32B Cable label, 32C Cable, 40 Visual data acquisition unit, 42 Terminal detection unit, 44 Guide generation unit, 46 Tracking unit, 48 Cable detection unit, 50 Cable determination unit

Claims

1. An auxiliary device used in the process of forming a test circuit by connecting a cable extending from a tester for performing tests on the target device to the terminals of the target device, It comprises an imaging unit, a display unit for displaying visual data acquired by the imaging unit, and a guide generation unit. The guide generation unit is an auxiliary device that generates guidance information for connecting the cable to the terminals based on connection information between the terminals and the cable for forming the test circuit, recorded in the manual for the target equipment and / or the tester, and displays the guidance information so as to overlap with first visual data including the terminals displayed on the display unit.

2. Equipped with a cable detection unit, The auxiliary device according to claim 1, wherein the cable determination unit generates connection correctness information based on second visual data including the terminal and the cable connected to the terminal, and the connection information.

3. The connection information includes the procedure for forming the test circuit. The auxiliary device according to claim 2, wherein the guide generation unit generates new guidance information based on the correct / incorrect information and displays it so as to overlap the first visual data and / or the second visual data.

4. If the correct / incorrect information includes that the cable is connected to the wrong terminal, the new guidance information includes information indicating the terminal to which the cable should be connected, or information indicating that the cable should be disconnected, as described in claim 3.

5. The auxiliary device according to any one of claims 1 to 4, wherein the guide generation unit displays an interface screen on the display unit for accepting the selection of the target equipment and / or the test circuit based on a pre-stored list to prompt the user to make a selection, and generates the guidance information based on the connection information stored for each target equipment and / or the test circuit selected by the user.

6. The cable determination unit includes a cable classification model trained using training data composed of images of the labeled cables. The auxiliary device according to claim 2, wherein the cable extending from the tester is color-coded in all or part in different colors.

7. Equipped with a tracking unit, The tracking unit generates positional information of the terminal and / or the cable in the video data, which is video data. The auxiliary device according to claim 1, wherein the guide generation unit displays the guidance information on the display unit so as to overlap with the image based on the position information.

8. The guide generation unit generates a procedure list based on the connection information and displays it on the display unit. The auxiliary device according to claim 3, wherein if the correct / incorrect information includes that the cable is connected to the correct terminal, it adds information to the procedure list indicating that the corresponding procedure in the procedure list has been completed.

9. A method for performing the task of forming a test circuit by connecting a cable extending from a tester for performing a test on the target equipment to the terminals of the target equipment using the auxiliary device described in claim 1, The imaging unit acquires visual data including the aforementioned terminals, A method comprising connecting the cable to the terminal based on the guidance information displayed so as to overlap with the visual data displayed on the display unit.

Images