COMMUNICATION DEVICE, MAINTENANCE SUPPORT SYSTEM, MAINTENANCE SUPPORT DEVICE, AND MAINTENANCE SUPPORT METHOD

Abstract

To perform AR display of a communication device without providing an element for image recognition in the communication device.SOLUTION: A communication device includes a rectangular parallelepiped casing and a plurality of communication ports arranged on one side of the casing, the one side including a plurality of ventilation holes arranged in a row, and the arrangement pattern of the plurality of ventilation holes at the plurality of positions on the one side is a pattern assigned to the model of the communication device.SELECTED DRAWING: Figure 3A

Description

[Technical Field]

[0001] The present disclosure relates to a communication device, a maintenance support system, a maintenance support device, and a maintenance support method. [Background technology]

[0002] Patent Document 1 discloses a method for using augmented reality (AR) to assist in the insertion and removal of connector plug terminals into connector adapter terminals in a connector connection board of an optical fiber wiring module. In this method, position detection markers are installed on a connector connection board in which a large number of connector adapter terminals are arranged in a matrix, the connector connection board is imaged by a camera, and a wiring work assistance device receives the image and recognizes the image of the position detection marker in the image. The wiring work assistance device sets the coordinates of the center position of the recognized position detection marker in the image as an origin reference. The wiring work assistance device generates image patterns of a coordinate grid and a target grid, and offsets the generated image patterns from the origin reference based on a drawing position offset amount included in pre-stored definition information, superimposing them on the image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-247700 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method disclosed in Patent Document 1 requires that a position detection marker for image recognition be provided on the connector board. [Means for solving the problem]

[0005] A communication device according to one aspect of the present disclosure comprises a rectangular housing and a plurality of communication ports arranged on one side of the housing, the one side including a plurality of ventilation holes arranged in a row, and the arrangement pattern of the plurality of ventilation holes at multiple positions on the one side is a pattern assigned to the model of the communication device.

[0006] A communication device according to another aspect of the present disclosure is a communication device capable of mounting a communication module, comprising a rectangular housing, one side of the housing including an opening through which a module panel that is part of the communication module mounted to the communication device is exposed, the opening having a shape and size that exposes a plurality of first air vents arranged in a row at a plurality of positions on the module panel in a pattern assigned to the model of the communication module, the one side including a plurality of second air vents arranged in a row, the plurality of second air vents being arranged in a pattern according to their positions on the one side.

[0007] A maintenance support system according to one aspect of the present disclosure is a maintenance support system that supports the maintenance of a communication device having a plurality of communication ports and a plurality of air vents on one side of a housing, and includes: a maintenance support terminal including a camera and a display; a recognition unit that recognizes the model of the communication device based on the arrangement pattern of the plurality of air vents in an image of the side captured by the camera; a position measurement unit that measures the position in real space of the side of the communication device whose model has been recognized by the recognition unit; a virtualization unit that determines a representative position of the side based on the position of the side measured by the position measurement unit and places a virtual surface, which is a virtual three-dimensional model of the side, at a position corresponding to the representative position in a virtual space corresponding to the real space; and a display control unit that superimposes the virtual surface placed in the virtual space by the virtualization unit and the image of the side captured by the camera on the display.

[0008] A maintenance support device according to one aspect of the present disclosure is a maintenance support device that supports the maintenance of a communication device having a plurality of communication ports and a plurality of air vents on one side of a housing, and includes: a recognition unit that recognizes the model of the communication device based on the arrangement pattern of the plurality of air vents in an image of the side captured by a camera; a virtualization unit that determines a representative position of the side based on the position in real space of the side of the communication device whose model has been recognized by the recognition unit, and places a virtual surface, which is a virtual three-dimensional model of the side, at a position corresponding to the representative position in a virtual space corresponding to the real space; and a display control unit that generates an augmented reality image in which the virtual surface placed in the virtual space by the virtualization unit is superimposed on the image of the side captured by the camera.

[0009] A maintenance support method according to one aspect of the present disclosure is a maintenance support method for supporting the maintenance of a communication device having a plurality of communication ports and a plurality of air vents on one side of a housing, and includes the steps of: recognizing the model of the communication device based on the arrangement pattern of the plurality of air vents in an image of the side captured by a camera; determining a representative position of the side based on the position in real space of the side of the communication device whose model has been recognized; and placing a virtual surface, which is a virtual three-dimensional model of the side, at a position corresponding to the representative position in a virtual space corresponding to the real space; and generating an augmented reality image by superimposing the virtual surface placed in the virtual space and the image of the side captured by the camera.

[0010] The present disclosure can be realized not only as a communication device having the above-described characteristic configuration, a maintenance support system for supporting maintenance of the communication device, a maintenance support device, and a maintenance support method, but also as a computer program for causing a computer to execute the characteristic steps included in the above-described maintenance support method, as a maintenance management terminal included in the maintenance support system, or as a part of the maintenance support device as a semiconductor integrated circuit. [Effects of the Invention]

[0011] According to the present disclosure, there is no need to provide a marker dedicated to image recognition in a communication device. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of a configuration of a maintenance support system according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a maintenance support terminal according to the embodiment. [Figure 3A] FIG. 2 is a diagram illustrating an example of a panel of a communication device according to an embodiment. [Figure 3B] FIG. 3B shows a virtual panel corresponding to the panel of FIG. 3A. [Figure 3C] 3B is a diagram showing an AR image in which the panel shown in FIG. 3A and the virtual panel shown in FIG. 3B are superimposed. [Figure 4A] FIG. 10 is a diagram showing an example of an arrangement pattern of upper-level ventilation openings. [Figure 4B] 10A and 10B are diagrams showing other examples of the arrangement pattern of the upper vent holes. [Figure 5A] FIG. 10 is a diagram showing a first example of an arrangement pattern of lower vent holes. [Figure 5B] FIG. 10 is a diagram showing a second example of an arrangement pattern of lower vent holes. [Figure 5C] FIG. 10 is a diagram showing a third example of an arrangement pattern of lower vent holes. [Figure 6] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server according to the embodiment. [Figure 7] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a maintenance management terminal according to the embodiment. [Figure 8] FIG. 2 is a functional block diagram illustrating an example of functions of the maintenance support system according to the embodiment. [Figure 9] 1A and 1B are diagrams for explaining partial elements of a panel of a communication device according to an embodiment. [Figure 10] 10A and 10B are diagrams for explaining measurement of the position of a partial element by a position measurement unit. [Figure 11]10A and 10B are diagrams for explaining the arrangement of virtual panels in a virtual space by a virtualization unit according to the embodiment. [Figure 12] 10A and 10B are diagrams for explaining the relationship between the position of a panel in real space and the position of a virtual panel in virtual space. [Figure 13A] FIG. 2 is a diagram illustrating a first example of the positional relationship of partial elements in a communication device. [Figure 13B] FIG. 10 is a diagram illustrating a second example of the positional relationship of partial elements in a communication device. [Figure 13C] FIG. 10 is a diagram illustrating a third example of the positional relationship of partial elements in a communication device. [Figure 14] FIG. 10 is a diagram showing an example of an AR image displayed on a display. [Figure 15] FIG. 10 is a diagram showing an example of an AR image when an abnormality occurs in a communication port. [Figure 16] FIG. 10 is a diagram showing an example of an AR image when some communication ports are selected. [Figure 17] FIG. 10 is a diagram showing an example of an AR image in which a communication port is specified and work information is displayed. [Figure 18] FIG. 10 is a diagram showing an example of an AR image in which port information is displayed. [Figure 19A] FIG. 10 is a diagram showing an example of an AR image displaying information on the first step of dealing with an abnormality. [Figure 19B] FIG. 10 is a diagram showing an example of an AR image displaying information on a second stage of dealing with an abnormality. [Figure 19C] FIG. 10 is a diagram showing an example of an AR image displaying information on a third stage of dealing with an abnormality. [Figure 19D] FIG. 10 is a diagram showing an example of an AR image in which information on the completion of dealing with an abnormality is displayed. [Figure 20] 10 is a flowchart illustrating an example of an AR display process by the maintenance support terminal according to the embodiment. [Figure 21] 10 is a flowchart illustrating an example of a port abnormality process performed by the maintenance support terminal according to the embodiment. [Figure 22] 10 is a flowchart illustrating an example of a selection condition display process by the maintenance support terminal according to the embodiment. [Figure 23] 10 is a flowchart illustrating an example of a port information display process by the maintenance support terminal according to the embodiment. [Figure 24] 10 is a flowchart illustrating an example of a tutorial process by the maintenance support terminal according to the embodiment. [Figure 25] FIG. 10 is a sequence diagram illustrating an example of a work instruction process performed by the maintenance support system according to the embodiment. [Figure 26] 10A and 10B are diagrams illustrating modified examples of the panel of the communication device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] <Summary of Embodiments of the Present Disclosure> The following provides an outline of embodiments of the present disclosure.

[0014] (1) A communication device according to this embodiment includes a rectangular parallelepiped housing and a plurality of communication ports arranged on one side of the housing. The one side includes a plurality of vents arranged in a row, and the arrangement pattern of the plurality of vents at a plurality of positions on the one side is a pattern assigned to the model of the communication device. This allows the pattern of the vents in an image of the one side of the communication device to be used for image recognition, eliminating the need to provide elements such as markers or product name displays for image recognition on the one side. Furthermore, even if elements such as markers or product name displays are provided on the one side of the communication device, if the elements are hidden by cables or the like, the model of the communication device can be identified by image recognition of the pattern of the vents.

[0015] (2) The plurality of ventilation openings may include a plurality of first ventilation openings arranged in a row and a plurality of second ventilation openings arranged in a row, the plurality of first ventilation openings being arranged at a plurality of positions on the one surface in a pattern assigned to the model, and the plurality of second ventilation openings being arranged in a pattern according to the positions on the one surface. By performing image recognition on the plurality of first ventilation openings and the plurality of second ventilation openings, the model of the communication device and its position on the panel can be identified.

[0016] (3) The plurality of second vents may be arranged in a first pattern on the left side of the panel, in a second pattern on the center of the panel, and in a third pattern on the right side of the panel. By performing image recognition on the second vents on the left side of the panel, the second vents on the center of the panel, and the second vents on the right side of the panel, it is possible to identify whether the recognized area is the left side, the center, or the right side of the panel.

[0017] (4) The one surface may include a partial element used for image recognition, and the arrangement pattern of the plurality of vents included in the partial element may be a pattern assigned to the model, thereby enabling the model of the communication device to be identified by image recognition of the partial element.

[0018] (5) A communication device according to this embodiment is a communication device capable of mounting a communication module, comprising a rectangular parallelepiped housing, one side of the housing including an opening through which a module panel, which is part of the communication module mounted on the communication device, is exposed, the opening having a shape and size that exposes a plurality of second vents arranged in a row at a plurality of positions on the module panel in a pattern assigned to the model of the communication module, the one side including a plurality of second vents arranged in a row, the plurality of second vents being arranged in a pattern corresponding to the positions on the one side. This allows the model of the communication device and its position on the panel to be identified by image recognition of the first vents and second vents in an image of the one side of the communication device. Because the patterns of the first vents and second vents can be used for image recognition, there is no need to provide elements such as markers or product name displays for image recognition on the one side. Furthermore, even if an element such as a marker or product name display is provided on one side of the communication device, if the element is hidden by a cable or the like, the model of the communication module can be identified by image recognition of the pattern of the first air vent.

[0019] (6) The one surface may include a first opening and a second opening, wherein the first opening exposes a plurality of vent holes arranged in a row in a pattern assigned to a model of the first communication module at a plurality of positions on a module panel of the first communication module when a first communication module is attached to the communication device, and the second opening exposes a plurality of vent holes arranged in a row in a pattern assigned to a model of the second communication module at a plurality of positions on a module panel of the second communication module when a second communication module is attached to the communication device. This makes it possible to individually identify the model of the first communication module and the model of the second communication module.

[0020] (7) A maintenance support system according to this embodiment is a maintenance support system that supports the maintenance of a communication device having a housing with multiple communication ports and multiple ventilation holes on one side thereof, and includes: a maintenance support terminal including a camera and a display; a recognition unit that recognizes the model of the communication device based on the arrangement pattern of the multiple ventilation holes in an image of the face captured by the camera; a position measurement unit that measures the position in real space of the face of the communication device whose model has been recognized by the recognition unit; a virtualization unit that determines a representative position of the face based on the position of the face measured by the position measurement unit and places a virtual surface, which is a virtual three-dimensional model of the face, at a position corresponding to the representative position in a virtual space corresponding to the real space; and a display control unit that superimposes the virtual surface placed in the virtual space by the virtualization unit and the image of the face captured by the camera on the display. Because the pattern of the ventilation holes in the image of the face of the communication device is used for image recognition, there is no need to provide elements such as markers for image recognition or product name displays on the face. Furthermore, even if elements such as markers or product name displays are provided on one side of the communication device, if the elements are hidden by cables or the like, the model of the communication device can be identified by image recognition of the ventilation hole pattern.

[0021] (8) A maintenance support device according to this embodiment supports the maintenance of a communication device having a housing with multiple communication ports and multiple vents on one side thereof. The maintenance support device includes: a recognition unit that recognizes the model of the communication device based on the arrangement pattern of the multiple vents in an image of the face captured by a camera; a virtualization unit that determines a representative position of the face based on the position in real space of the face of the communication device whose model is recognized by the recognition unit, and places a virtual surface, which is a virtual three-dimensional model of the face, at a position corresponding to the representative position in a virtual space corresponding to the real space; and a display control unit that generates an augmented reality image in which the virtual surface placed in the virtual space by the virtualization unit is superimposed on the image of the face captured by the camera. Because the pattern of the vents in the image of the face of the communication device is used for image recognition, there is no need to provide elements such as markers or product name displays for image recognition on the face. Furthermore, even if elements such as markers or product name displays are provided on the face of the communication device, if the elements are hidden by cables or the like, the model of the communication device can be identified by image recognition of the pattern of the vents.

[0022] (9) A maintenance support method according to this embodiment supports the maintenance of a communication device having a housing with multiple communication ports and multiple vents on one side thereof. The method includes the steps of: recognizing the model of the communication device based on the arrangement pattern of the multiple vents in an image of the face captured by a camera; determining a representative position of the face based on the position in real space of the face of the communication device for which the model has been recognized; and placing a virtual surface, which is a virtual three-dimensional model of the face, at a position corresponding to the representative position in a virtual space corresponding to the real space; and generating an augmented reality image by superimposing the virtual surface placed in the virtual space on the image of the face captured by the camera. Because the pattern of the vents in the image of the face of the communication device is used for image recognition, there is no need to provide elements such as markers or product name displays for image recognition on the face. Furthermore, even if elements such as markers or product name displays are provided on the face of the communication device, if the elements are hidden by cables or the like, the model of the communication device can be identified by image recognition of the pattern of the vents.

[0023] <Details of the embodiment of the present disclosure> DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. At least some of the following preferred embodiments may be combined in any desired manner.

[0024] [1. Maintenance support system] 1 is a diagram illustrating an example of the configuration of a maintenance support system according to an embodiment. The maintenance support system 10 supports a user (maintenance worker) in maintaining communication devices 200A, 200B, 200C, 200D, 200E, 200F, 200G, and 200H. The maintenance support system 10 includes a maintenance support terminal 100, a server 300, and a maintenance management terminal 400.

[0025] Communication devices 200A, 200B, 200C, 200D, 200E, 200F, 200G, and 200H are installed in communication center 20, which is a facility of a telecommunications carrier. Communication devices 200A, 200B, 200C, and 200D are mounted in rack 250A, and communication devices 200E, 200F, 200G, and 200H are mounted in rack 250B. In the following description, communication devices 200A, 200B, 200C, 200D, 200E, 200F, 200G, and 200H may be collectively referred to as "communication devices 200," and racks 250A and 250B may be collectively referred to as "rack 250."

[0026] The communication device 200 includes multiple communication ports. The communication ports are provided on the front surface of the housing of the communication device 200. The front surface of the housing of the communication device 200 is composed of a single panel. The communication device 200 is, for example, an optical communication device, and an optical transceiver can be inserted (mounted) into each communication port.

[0027] Various models of communication devices 200 are mounted on the rack 250. For example, the communication devices 200A and 200B are a first model of communication devices, and the communication devices 200C and 200D are a second model of communication devices. The configuration of the front panel of the communication device differs depending on the model.

[0028] The maintenance support terminal 100 is an example of a maintenance support device. The maintenance support terminal 100 is used by a user in the communication center 20 for maintenance work on the communication device 200. The maintenance support terminal 100 is a portable terminal having a communication function, such as a tablet, smart glasses, or a smartphone.

[0029] The server 300 and the maintenance management terminal 400 are installed in, for example, a management center 30 which is a facility of a management company that manages the communication device 200. The management center 30 is installed in a remote location from the communication center 20, for example.

[0030] The server 300 and the maintenance management terminal 400 are communicably connected to the maintenance support terminal 100 via a network 500. The server 300 provides the maintenance support terminal 100 with information and functions used in maintenance work on the communication device 200. The maintenance management terminal 400 is used by an administrator. The administrator uses the maintenance management terminal 400 to support users in performing maintenance work on the communication device 200.

[0031] [2.Configuration of maintenance support terminal] 2 is a block diagram showing an example of the hardware configuration of a maintenance support terminal according to this embodiment. The maintenance support terminal 100 includes a processor 101, a non-volatile memory 102, a volatile memory 103, an input device 104, a display 105, a graphics processing unit 106, a communication interface (I / F) 107, a camera 108, a position measurement unit 109, a gyro sensor 112, and an acceleration sensor 113.

[0032] The volatile memory 103 is, for example, a semiconductor memory such as an SRAM (Static Random Access Memory) or a DRAM (Dynamic Random Access Memory). The non-volatile memory 102 is, for example, a flash memory, a hard disk, a ROM (Read Only Memory), etc. The non-volatile memory 102 stores a maintenance support application (APP) 120 and data used for executing the maintenance support APP 120. The non-volatile memory 102 stores a trained model 600 generated by machine learning. The trained model 600 is used by the maintenance support APP 120. The non-volatile memory 102 stores data of virtual panels 700A, 700B, 700C, ..., which are three-dimensional virtual models of panels of the communication device 200. The virtual panels 700A, 700B, 700C, ... are examples of virtual surfaces. The virtual panels 700A, 700B, 700C, ... are used by the maintenance support APP 120. Nonvolatile memory 102 stores virtual panels 700A, 700B, and 700C for each model of communication device 200. In the following description, virtual panels 700A, 700B, and 700C may be collectively referred to as "virtual panel 700."

[0033] The panel of the communication device 200 and the virtual panel 700 will be described. 3A is a diagram showing an example of a panel of the communication device according to the embodiment. The panel 210 includes an external port 211, a communication port 212, and air vents 221 and 222.

[0034] The external port 211 is provided at the left end of the panel 210. The external port 211 is a communication port for connecting to the network 500. For example, the external port 211 is used for communication with an external maintenance device such as the maintenance support terminal 100 connected to the network 500.

[0035] Optical transceivers can be attached / detached to the communication ports 212. A plurality of communication ports 212 are arranged on two levels, upper and lower, on the panel 210. The communication ports 212 include two types of communication ports: a communication port compatible with 1 Gbps optical transceivers and a communication port compatible with 10 Gbps optical transceivers.

[0036] A plurality of ventilation holes 221, 222 are provided on the upper part of the panel 210, above the communication port 212. The ventilation holes 221, 222 are intake holes that introduce air into the housing of the communication device 200, or exhaust holes that exhaust air from the housing of the communication device 200.

[0037] FIG. 3B is a diagram showing a virtual panel corresponding to the panel in FIG. 3A. Virtual panel 700 is composed of multiple part models 701, 702, and 703. Part model 701 is a 3D virtual model of a portion of panel 210 that includes external port 211 at the left edge. Part model 702 is a 3D virtual model of a portion of panel 210 that includes communication port 212. Part model 703 is a 3D model that defines the outer edge of panel 210. Part models 701 and 702 are placed in part model 703 at positions where corresponding portions are placed in panel 210. In other words, part model 701 is placed at the left edge of part model 703. Part model 702 is placed in part model 703 at a position that corresponds to the position of communication port 212 in panel 210.

[0038] The above-described virtual panel 700 is placed in a virtual three-dimensional space (virtual space) set in the maintenance support terminal 100 in correspondence with the real space. As a result, as will be described later, an image of the panel 210 captured by the camera 108 and the virtual panel 700 are displayed superimposed on each other. FIG. 3C is a diagram showing an AR image in which the panel shown in FIG. 3A and the virtual panel shown in FIG. 3B are superimposed on each other. In the AR image, the position of the outline of the panel 210 matches the position of the part model 703. The position of the portion of the panel 210 including the external port 211 at the left end matches the position of the part model 701. Furthermore, the positions of the multiple communication ports 212 and the positions of the multiple part models 702 match each other.

[0039] The virtual panel 700 is transparent, so that the panel 210 appears through the virtual panel 700 in the AR image.

[0040] 3A , the ventilation holes 221, 222 are provided in two separate rows, an upper row and an lower row. The upper ventilation hole 221 is used to identify the model of the communication device 200. In other words, the ventilation hole 221 also serves as an identification code for the model of the communication device 200. The lower ventilation hole 222 is used to identify a position within the panel 210. In other words, the ventilation hole 222 also serves as an identification code for a position within the panel 210.

[0041] For example, multiple ventilation openings 221 are lined up in a row, and multiple ventilation openings 222 are lined up in a row. Hereinafter, the vertical length of one ventilation opening 221, 222 will be referred to as the "height," and the horizontal length will be referred to as the "width." The ventilation openings 221, 222 are provided on the panel 210 so as to satisfy certain requirements. The requirements include that the width of the ventilation openings 221, 222 is within a range defined by predetermined upper and lower limits, that a certain number or more of ventilation openings 221, 222 are provided per section of a certain length in the horizontal direction, and that the shape of the ventilation openings 221, 222 is selected from a plurality of predetermined shapes.

[0042] In the following description, the shape of the ventilation openings 221, 222 is assumed to be rectangular. However, the shape of the ventilation openings 221, 222 may be other than rectangular, and may be, for example, circular, triangular, or star-shaped.

[0043] 4A is a diagram showing an example of an arrangement pattern of the upper vents, and FIG. 4B is a diagram showing another example of an arrangement pattern of the upper vents. An arrangement pattern of the air vents 221 is determined for each model of the communication device 200. For example, the arrangement pattern of the air vents 221A shown in FIG. 4A is a pattern specific to the models of the communication devices 200A and 200B, and the arrangement pattern of the air vents 221B shown in FIG. 4B is a pattern specific to the models of the communication devices 200C and 200D. Therefore, the maintenance support terminal 100 can identify the model of the communication device 200 by image recognition of the arrangement pattern of the air vents 221.

[0044] The ventilation opening 221 is selected from two shapes: a square opening and a rectangular opening. A square opening is a square opening and has a width equal to its height. A rectangular opening has a width greater than its height. The height of a square opening and a rectangular opening is the same. The width of a rectangular opening is equal to the width of two square openings. When two square openings are placed side by side, a certain gap is provided between adjacent square openings. The width of a rectangular opening is equal to the sum of the widths of the two square openings and the gap between them. Furthermore, in addition to square openings and rectangular openings, closed square openings are used in the arrangement pattern of ventilation openings 221. A closed square opening is a closed area (with no holes) that has the same shape and size as a square opening. The arrangement pattern of the air vents 221 is defined by the combination and arrangement order of square openings, rectangular openings, and square closed openings in a section (hereinafter referred to as a "unit section") having a width equivalent to three square openings in the left-right direction, and the same pattern is repeated for each unit section. The number of square closed openings in one unit section is either 1 or 0. The arrangement pattern of the air vents 221 in one unit section is assigned to the model of the communication device 200. In other words, the arrangement pattern of the air vents 221 in one unit section is the model code of the communication device 200. In this case, eight different model codes can be defined.

[0045] A blank section of a certain length is provided between adjacent unit sections. The blank section has a width greater than the distance between adjacent openings (square openings or rectangular openings) in the unit section. By providing the blank section, the unit section can be recognized by image recognition.

[0046] The arrangement pattern of the vents 221A shown in FIG. 4A includes two square openings and one square closed opening in the unit section. The unit section is defined by three regions, a "first region," a "second region," and a "third region," arranged from left to right. The arrangement pattern of the vents 221A is a pattern in which square openings are arranged in each of the first and second regions, and a square closed opening is arranged in the third region. The arrangement pattern of the vents 221B shown in FIG. 4B is a pattern in which square openings are arranged in the first region, and rectangular openings are arranged in the second and third regions. Since the same arrangement pattern of the vents 221 is repeated for each unit section, the same pattern of the vents 221 is arranged at multiple positions on the panel 210. Therefore, even if the camera 108 captures only a portion of the panel 210, the model of the communication device 200 can be identified based on the pattern of the vents 221 included in the captured portion.

[0047] FIG. 5A is a diagram showing a first example of an arrangement pattern of lower-level vents, FIG. 5B is a diagram showing a second example of an arrangement pattern of lower-level vents, and FIG. 5C is a diagram showing a third example of an arrangement pattern of lower-level vents. Lower-level vents 222 are used to identify their positions (left, center, right) on panel 210. Arrangement patterns of vents 222 are defined for each model of communication device 200, respectively, for the left, center, and right portions of panel 210. For example, the arrangement pattern of vents 222A shown in FIG. 5A is a pattern specific to the left portion of panel 210, the arrangement pattern of vents 222B shown in FIG. 5B is a pattern specific to the center portion of panel 210, and the arrangement pattern of vents 222C shown in FIG. 5C is a pattern specific to the right portion of panel 210. Therefore, the maintenance support terminal 100 can identify the position (left part, center part, right part) of the recognized part on the panel 210 by performing image recognition of the arrangement pattern of the air vents 222.

[0048] The arrangement pattern of the ventilation openings 222A shown in FIG. 5A is a pattern in which square openings are arranged in each of the first and second regions and square closed openings are arranged in the third region. The arrangement pattern of the ventilation openings 222B shown in FIG. 5B is a pattern in which square openings are arranged in the first region and rectangular openings are arranged in the second and third regions. The arrangement pattern of the ventilation openings 222C shown in FIG. 5C is a pattern in which square openings are arranged in the first region, square closed openings in the second region, and square openings in the third region. Note that, although the unit section in the above example is a section corresponding to three square openings, it may also be a section corresponding to four or more square openings. The unit section may be of a size that allows for defining model codes at least equal to the number of models of the communication device 200. As another example, the model of the communication device 200 may be identified by combining the pattern of the ventilation openings 222 with features other than the pattern of the ventilation openings 222, such as the height of the communication device 200 or the color of the panel 210.

[0049] 2, the maintenance support terminal 100 is configured to include a computer, and some or all of the functions of the maintenance support terminal 100 are realized by the processor 101 executing a maintenance support APP 120, which is a computer program stored in a storage device of the computer. The maintenance support APP 120 can be stored in a recording medium such as a flash memory, a ROM, or a CD-ROM. The processor 101 executes the maintenance support APP 120, allowing the maintenance support terminal 100 to perform an AR display of the communication device 200.

[0050] The processor 101 is, for example, a CPU (Central Processing Unit). However, the processor 101 is not limited to a CPU. The processor 101 may also be a GPU (Graphics Processing Unit). The processor 101 may also be, for example, an ASIC (Application Specific Integrated Circuit) or a programmable logic device such as a gate array or FPGA (Field Programmable Gate Array). In this case, the ASIC or programmable logic device is configured to be able to execute processing similar to that of the maintenance support APP 120.

[0051] The graphics processing unit 106 is connected to the display 105 and controls the display on the display 105. The graphics processing unit 106 includes, for example, a GPU and a VRAM (Video RAM), stores data to be displayed on the display 105 in the VRAM, periodically reads one frame of video data from the VRAM, and generates a video signal. The generated video signal is output to the display 105, and the video is displayed on the display 105. The function of the graphics processing unit 106 may be included in the processor 101. A portion of the area of ​​the volatile memory 103 may be used as the VRAM.

[0052] The display 105 includes, for example, a liquid crystal monitor or an OEL (organic electroluminescence) monitor. The display 105 can display text or graphic information. The input device 104 includes, for example, a capacitive or pressure-sensitive touchpad overlaid on the display 105. The input device 104 may also be a keyboard and a pointing device such as a mouse. The input device 104 is used to input information to the maintenance support terminal 100.

[0053] The communication I / F 107 is, for example, a wireless communication interface. In a specific example, the communication I / F 107 is a wireless LAN interface, a Bluetooth interface, or a wireless communication interface compliant with 5G (fifth generation mobile communication system) or 4G (fourth generation mobile communication system). The communication I / F 107 can be connected to the network 500 wirelessly. The maintenance support terminal 100 can communicate with the server 300 and the maintenance management terminal 400 via the communication I / F 107.

[0054] The camera 108 includes an image sensor and a lens, for example, configured using a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 108 generates captured images and outputs the images. For example, the camera 108 can output a moving image by continuously generating still images at a predetermined frame rate and outputting the still images at the same frame rate. Hereinafter, the image output from the camera 108 will also be referred to as a "camera image."

[0055] The position measurement unit 109 measures the position of an object outside the maintenance support terminal 100, i.e., the coordinates in three-dimensional space (real space). For example, the position measurement unit 109 measures the position of the object by a ray casting method. The position measurement unit 109 includes a laser irradiation unit 110 and a light receiving unit 111. The position measurement unit 109 measures the position of the object by irradiating a laser from the laser irradiation unit 110 toward the object and receiving the laser light (reflected light) reflected from the object with the light receiving unit 111.

[0056] The gyro sensor 112 detects the angular velocity of the maintenance support terminal 100. The acceleration sensor 113 detects the acceleration of the maintenance support terminal 100. The maintenance support terminal 100 can calculate the position and orientation (azimuth angle) of the maintenance support terminal 100 from the detection values ​​of the gyro sensor 112 and the acceleration sensor 113.

[0057] [3. Server configuration] 6 is a block diagram showing an example of the hardware configuration of a server according to this embodiment. The server 300 includes a processor 301, a non-volatile memory 302, a volatile memory 303, and a communication I / F 304.

[0058] The volatile memory 303 is, for example, a semiconductor memory such as an SRAM or a DRAM. The non-volatile memory 302 is, for example, a flash memory, a hard disk, a ROM, or the like. The non-volatile memory 302 stores a server program 305 and data used to execute the server program 305. The non-volatile memory 302 further includes a communication device database (DB) 306. The communication device DB 306 stores information for supporting maintenance of the communication device 200. The server 300 is configured with a computer, and some or all of the functions of the server 300 are realized by the processor 301 executing a server program 305, which is a computer program stored in a storage device of the computer. The server program 305 can be stored in a recording medium such as a flash memory, a ROM, or a CD-ROM. By the processor 301 executing the server program 305, the server 300 can provide the maintenance support terminal 100 with information and functions for supporting maintenance of the communication device 200.

[0059] The processor 301 is, for example, a CPU. However, the processor 301 is not limited to a CPU. The processor 301 may be a GPU. The processor 301 may be, for example, an ASIC, or a programmable logic device such as a gate array or FPGA. In this case, the ASIC or programmable logic device is configured to be able to execute the same processing as the server program 305.

[0060] The communication I / F 304 is a communication interface such as an Ethernet interface, a wireless LAN interface, or a Bluetooth interface. The communication I / F 304 is connected to the network 500. The server 300 can communicate with the maintenance support terminal 100 and the maintenance management terminal 400 via the communication I / F 304.

[0061] The communication device 200 is connected to a network 500. The server 300 can communicate with the communication device 200 via the communication I / F 304.

[0062] [4. Configuration of the maintenance management terminal] 7 is a block diagram showing an example of the hardware configuration of a maintenance management terminal according to this embodiment. The maintenance management terminal 400 includes a processor 401, a nonvolatile memory 402, a volatile memory 403, an input device 404, a display 405, a graphics processing unit 406, and a communication I / F 407.

[0063] The volatile memory 403 is, for example, a semiconductor memory such as an SRAM or a DRAM. The non-volatile memory 402 is, for example, a flash memory, a hard disk, a ROM, or the like. The non-volatile memory 402 stores the maintenance management APP 410 and data used to execute the maintenance management APP 410. The non-volatile memory 402 stores data for the virtual panels 700A, 700B, 700C, etc. The data for the virtual panels 710A, 710B, 710C, etc. is used by the maintenance management APP 410. The virtual panels 710A, 710B, 710C, etc. are three-dimensional virtual models of the panels of the communication device 200 in an opaque color. The virtual panels 710A, 710B, 710C, etc. contain the same data as the virtual panels 700A, 700B, 700C, etc. except that they are opaque. In the following description, the virtual panels 710A, 710B, and 710C may be collectively referred to as "virtual panel 710."

[0064] The maintenance management terminal 400 is configured with a computer, and some or all of the functions of the maintenance management terminal 400 are realized by a maintenance management APP 410, which is a computer program stored in a storage device of the computer, being executed by a processor 401. The maintenance management APP 410 can be stored in a recording medium such as a flash memory, a ROM, or a CD-ROM. By the processor 401 executing the maintenance management APP 410, the maintenance management terminal 400 can display a three-dimensional model of the panel 210 of the communication device 200.

[0065] The processor 401 is, for example, a CPU. However, the processor 401 is not limited to a CPU. The processor 401 may be a GPU. The processor 401 may be, for example, an ASIC, or a programmable logic device such as a gate array or FPGA. In this case, the ASIC or programmable logic device is configured to be able to execute the same processing as the maintenance management APP 410.

[0066] The graphics processing unit 406 is connected to the display 405 and controls the display on the display 405. The functions of the graphics processing unit 406 may be included in the processor 401. A portion of the area of ​​the volatile memory 403 may be used as a VRAM.

[0067] The display 405 includes, for example, a liquid crystal panel or an OEL panel. The display 405 can display text or graphic information. The input device 404 includes, for example, a keyboard and a pointing device such as a mouse. The input device 404 may be a capacitive or pressure-sensitive touchpad overlaid on the display 405. The input device 404 is used to input information to the maintenance management terminal 400.

[0068] The communication I / F 404 is a communication interface such as an Ethernet interface, a wireless LAN interface, or a Bluetooth interface. The communication I / F 404 is connected to the network 500. The maintenance management terminal 400 can communicate with the maintenance support terminal 100 and the server 300 via the communication I / F 404.

[0069] [5. Functions of the maintenance support system] 8 is a functional block diagram showing an example of functions of the maintenance support system according to the embodiment. When the processor 101 of the maintenance support terminal 100 executes the maintenance support APP 120, the maintenance support terminal 100 functions as a recognition unit 121, a virtualization unit 122, a display control unit 123, a notification acceptance unit 124, and a receiving unit 125. The recognition unit 121 and the virtualization unit 122 are realized by the processor 101. The display control unit 123 is realized by the processor 101 and the graphics processing unit 106. The notification acceptance unit 124 and the receiving unit 125 are realized by the processor 101 and the communication I / F 107. When the processor 401 of the maintenance management terminal 400 executes the maintenance management APP 410, the maintenance management terminal 400 functions as a display control unit 411 (second display control unit) and a communication unit 412. The display control unit 411 is realized by the processor 401 and the graphics processing unit 406. The communication unit 412 is realized by the processor 401 and the communication I / F 407 .

[0070] When a user performs maintenance on the communication device 200, the user captures an image of the panel 210 of the communication device 200 using the camera 108 of the maintenance support terminal 100. The camera image output from the camera 108 is provided to the recognition unit 121. The recognition unit 121 recognizes partial elements of the panel 210 in the image of the panel 210 captured by the camera 108 using the trained model 600. The partial elements are elements that include functional units of the communication device 200. An example of a functional unit is the communication port 212, another example is the external port 211, and yet another example is the air vents 221 and 222.

[0071] Fig. 9 is a diagram illustrating partial elements of a panel of a communication device according to this embodiment. In the example of Fig. 9, the leftmost region of panel 210 is partial element 230A, the leftmost region of the center of panel 210 is partial element 230B, the rightmost region of the center of panel 210 is partial element 230C, and the rightmost region of panel 210 is partial element 230D.

[0072] Partial element 230A includes, as functional units, an external port 211 and vents 221 and 222. Partial elements 230B, 230C, and 230D each include, as functional units, a communication port 212 and vents 221 and 222. The trained model 600 recognizes partial elements 230A, 230B, 230C, and 230D based on the shape of the images of partial elements 230A, 230B, 230C, and 230D. In the following description, partial elements 230A, 230B, 230C, and 230D may be collectively referred to as "partial elements 230."

[0073] An example of the trained model 600 will be described below. The trained model 600 is configured by a neural network. The trained model 600 includes an input layer, an intermediate layer, and an output layer. Input data of the trained model 600 is a camera image including a partial element 230. The trained model 600 recognizes the image of the partial element 230 in the camera image and identifies the position of the image of the recognized partial element 230 in the camera image. The trained model 600 identifies the model of the communication device 200 from the pattern of the air vent 221 included in the recognized partial element 230. Output data of the trained model 600 is the identified model of the communication device 200 and position information in the image of the recognized partial element 230, i.e., two-dimensional coordinate values ​​of the screen, which is the display screen.

[0074] The construction of the trained model 600 will be described. The trained model 600 is generated by machine learning using, as input data, an image of the panel 210 in which the partial element 230 is identified. The machine learning is executed, for example, by a learning device different from the maintenance support terminal 100. Specifically, an image including the partial element 230 and the model name of the communication device 200 are input to the learning device as training data. The training data is created by semantic annotation that identifies the partial element 230 in each of images captured of the communication device 200 at various angles. The machine learning is performed repeatedly using multiple training data. When a camera image including the partial element 230 is input, the trained model 600 constructed by machine learning outputs position information of the partial element 230 in the image and model information of the communication device 200.

[0075] A reference point 240 is predefined for the partial element 230. For example, when an image of the partial element 230A shown in FIG. 9 is provided to the trained model 600, the trained model 600 outputs position information (two-dimensional coordinate values ​​on the screen) of the reference point 240A of the partial element 230A as position information of the partial element 230A in the camera image. When an image of the partial element 230B is provided to the trained model 600, the trained model 600 outputs position information of the reference point 240B of the partial element 230B in the camera image. When an image of the partial element 230C is provided to the trained model 600, the trained model 600 outputs position information of the reference point 240B of the partial element 230C in the camera image. When an image of the partial element 230D is provided to the trained model 600, the trained model 600 outputs position information of the reference point 240C of the partial element 230D in the camera image.

[0076] Returning to FIG. 8 , the position measurement unit 109 measures the position in real space of the partial element 230 recognized by the trained model 600. FIG. 10 is a diagram for explaining measurement of the position of a partial element by the position measurement unit. In the example shown in FIG. 10 , a partial element 230A of a panel 210 is image-recognized. The laser irradiation direction is identified from the position (two-dimensional coordinate value on the screen) of a reference point 240A of the partial element 230A in a camera image displayed on the display 105. The laser irradiation unit 110 of the position measurement unit 109 irradiates a laser in the identified direction (direction toward the reference point 240A). The laser is reflected at the reference point 240A, and the reflected light is received by the light receiving unit 111 of the position measurement unit 109. The position measurement unit 109 measures the position of the reference point 240A of the partial element 230A in real space (three-dimensional coordinate system xyz) by irradiating the laser and receiving the reflected light.

[0077] Returning to Fig. 8, the trained model 600 is provided with camera images of at least two partial elements 230 in one communication device 200. The position measurement unit 109 measures the positions of the reference points 240 of the at least two partial elements 230. For example, in Fig. 9, the trained model 600 recognizes any one of the combination of partial elements 230A and 230D, the combination of partial elements 230A and 230B, and the combination of partial elements 230C and 230D.

[0078] The virtualization unit 122 determines a representative position of the panel 210 based on the position of the partial element 230 measured by the position measurement unit 109. An example of the representative position of the panel 210 is the center point of the panel 210. FIG. 11 is a diagram for explaining the arrangement of a virtual panel in a virtual space by the virtualization unit according to this embodiment. FIG. 11 is a plan view of the panel 210 as viewed from above. For example, when partial elements 230A and 230D are recognized by the trained model 600, the position measurement unit 109 measures the position (10, 50, -20) of the reference point 240A and the position (80, 47, -15) of 240D in the real space xyz. The virtualization unit 122 determines the midpoint (45, 48.5, -17.5) of the reference points 240A and 240D in the real space xyz as the representative position 210P.

[0079] The virtualization unit 122 selects the virtual panel 700 that matches the model of the communication device 200 recognized by the trained model 600 from among the multiple virtual panels 700A, 700B, and 700C.

[0080] The virtualization unit 122 places the selected virtual panel 700 at a position corresponding to the representative position 210P in a virtual space x'y'z' that corresponds to the real space xyz. The virtual space x'y'z' has virtual coordinate axes x', y', z' that coincide with the coordinate axes x, y, z of the real space xyz. In the example of FIG. 11 , the virtualization unit 122 places the representative position 210P of the virtual panel 700 at coordinates (45, 48.5, -17.5) in the virtual space x'y'z', in accordance with the representative position 210P (45, 48.5, -17.5) of the panel 210 in the real space xyz.

[0081] More specifically, the virtualization unit 122 places the virtual panel 700 in the virtual space x'y'z' at an angle corresponding to the angle of the panel 210 in the real space xyz, based on the positions of each of the multiple partial elements 230 in the real space xyz. In the example of Fig. 11, the virtualization unit 122 places the reference point 240A in the virtual panel 700 at coordinates (10, 50, -20) in the virtual space x'y'z' to match the position (10, 50, -20) of the reference point 240A of the partial element 230A in the real space xyz, and places the reference point 240A in the virtual panel 700 at coordinates (80, 47, -15) in the virtual space x'y'z' to match the position (80, 47, -15) of the reference point 240D of the partial element 230D in the real space xyz.

[0082] Here, the angle of panel 210 refers to an angle in the horizontal direction (for example, an angle with respect to the x-axis). FIG. 12 is a diagram illustrating the relationship between the position of a panel in real space and the position of a virtual panel in virtual space. In this embodiment, it is assumed that communication device 200 is installed so that panel 210 is vertical. That is, panel 210 is always vertical and is not tilted. Therefore, by placing virtual panel 700 in virtual space x'y'z' so that the position of reference point 240 of partial element 230 in real space xyz coincides with the position of reference point 240 of virtual panel 700 in virtual space x'y'z' as described above, it is possible to align the angle of panel 210 in real space xyz with the angle in virtual space x'y'z'.

[0083] Returning to FIG. 8 , the virtualization unit 122 determines whether the multiple partial elements 230 recognized by the trained model 600 are partial elements 230 of the same communication device, based on the positions of the multiple partial elements 230 measured by the position measurement unit 109. In a specific example, the virtualization unit 122 determines whether the positions of the multiple partial elements 230 measured by the position measurement unit 109 meet a determination condition, which is a condition for the positional relationship of the multiple partial elements 230. If the positions of the multiple partial elements 230 measured by the position measurement unit 109 meet the determination condition, the virtualization unit 122 determines that the multiple partial elements 230 recognized by the trained model 600 are partial elements 230 of the same communication device. If the positions of the multiple partial elements 230 measured by the position measurement unit 109 do not meet the determination condition, the virtualization unit 122 determines that the multiple partial elements 230 recognized by the trained model 600 are not partial elements 230 of the same communication device.

[0084] 13A is a diagram showing a first example of the positional relationship of partial elements in a communication device. In this embodiment, it is assumed that the communication device 200 is a rectangular parallelepiped and that the top and bottom surfaces of the communication device 200 are arranged horizontally in the rack 250. In FIG. 13A, the heights (z coordinates) of the partial elements 230A_1 and 230D_1 included in the panel 210_1 of the communication device 200_1 are the same. The height of the partial element 230A_1 included in the panel 210_1 of the communication device 200_1 and the height of the partial element 230D_2 included in the panel 210_2 of the communication device 200_2 are different from each other. When partial elements 230A_1 and 230D_1 are recognized by the trained model 600, the difference in z coordinate between reference point 240A_1 of partial element 230A_1 and reference point 240D_1 of partial element 230D_1 is within the allowable range, and therefore, the virtualization unit 122 determines that partial elements 230A_1 and 230D_1 are partial elements of the same communication device 200_1. When partial elements 230A_1 and 230D_2 are recognized by the trained model 600, the difference in z coordinate between reference point 240A_1 of partial element 230A_1 and reference point 240D_2 of partial element 230D_2 exceeds the allowable range, and therefore, the virtualization unit 122 determines that partial elements 230A_1 and 230D_2 are not partial elements of the same communication device.

[0085] FIG. 13B is a diagram showing a second example of the positional relationship of partial elements in a communication device. In the example shown in FIG. 13B, two communication devices 200_3 and 200_4 are arranged facing the same direction with their positions aligned in the horizontal direction (x direction) and offset in the depth direction (y direction). In the example of FIG. 13B, for ease of explanation, the longitudinal direction of panels 210_3 and 210_4 is defined as the x direction, and the depth direction of communication devices 200_3 and 200_4 is defined as the y direction. The depth direction positions (y coordinates) of partial elements 230A_3 and 230D_3 included in panel 210_3 of communication device 200_3 are the same. The depth direction positions of partial elements 230A_4 and 230D_4 included in panel 210_4 of communication device 200_4 are the same. The position in the depth direction of the partial element 230A_3 included in the panel 210_3 of the communication device 200_3 is different from the position in the depth direction of the partial element 230D_4 included in the panel 210_4 of the communication device 200_4. For example, when a user successively captures images of the partial elements 230A_3 and 230D_3 of the panel 210_3 of one communication device 200_3 with the camera 108, the partial elements 230A_3 and 230D_3 are recognized by the trained model 600. In this case, the difference in the x-coordinate between the reference point 240A_3 of the partial element 230A_3 and the reference point 240D_3 of the partial element 230D_3 matches the length of the panel 210_3, and the difference in the y-coordinate between the reference point 240A_3 and the reference point 240D_3 is within the allowable range. Therefore, the virtualization unit 122 determines that the partial elements 230A_3 and 230D_3 are partial elements of the same communication device 200_3. It is conceivable that a user captures a video using the camera 108 while changing the position or orientation of the maintenance support terminal 100. A partial element recognized in a certain screen of the video is accumulated in three-dimensional space. In other words, even if the field of view of the camera 108 changes after the partial element is captured and the partial element is no longer included in the screen, the partial element is not deleted from the three-dimensional space.For example, if a user captures an image of the partial element 230A_3 of the panel 210_3 of the communication device 200_3 with the camera 108, then moves to the front side of the communication device 200_4 and points the camera 108 toward the panel 210_4 of the communication device 200_4 to capture an image of the partial element 230D_4, the partial elements 230A_3 and 230D_4 are captured. In this case, the trained model 600 recognizes the partial elements 230A_3 and 230D_4. The difference in the x-coordinate between the reference point 240A_3 of the partial element 230A_3 and the reference point 240D_4 of the partial element 230D_4 matches the length of the panel 210_3, but since the difference in the y-coordinate between the reference point 240A_3 and the reference point 240D_4 exceeds the allowable range, the virtualization unit 122 determines that the partial elements 230A_3 and 230D_4 are not partial elements of the same communication device.

[0086] FIG. 13C is a diagram showing a third example of the positional relationship of partial elements in a communication device. In the example shown in FIG. 13D, two communication devices 200_5 and 200_6 are arranged facing in opposite directions, aligned in the horizontal direction (x direction) and shifted in the depth direction (y direction). In the example of FIG. 13C, for ease of explanation, the longitudinal direction of panels 210_5 and 210_6 is defined as the x direction, and the depth direction of communication devices 200_5 and 200_6 is defined as the y direction. The depth direction positions (y coordinates) of partial elements 230A_5 and 230D_5 included in panel 210_5 of communication device 200_5 are the same. The depth direction positions of partial elements 230A_6 and 230D_6 included in panel 210_6 of communication device 200_6 are the same. The depth direction position of a partial element 230A_5 included in a panel 210_5 of a communication device 200_5 is different from the depth direction position of a partial element 230D_6 included in a panel 210_6 of a communication device 200_6. When the partial elements 230A_5 and 230D_5 are recognized by the trained model 600, the difference in the x coordinate between the reference point 240A_5 of the partial element 230A_5 and the reference point 240D_5 of the partial element 230D_5 matches the length of the panel 210_5, and the difference in the y coordinate between the reference point 240A_5 and the reference point 240D_5 is within an allowable range, and therefore the virtualization unit 122 determines that the partial elements 230A_5 and 230D_5 are partial elements of the same communication device 200_5. When partial elements 230A_5 and 230D_6 are recognized by the trained model 600, the difference in the x-coordinate between reference point 240A_5 of partial element 230A_5 and reference point 240D_6 of partial element 230D_6 is smaller than the allowable range. Therefore, even if the difference in the y-coordinate between reference point 240A_5 and reference point 240D_6 is within the allowable range, the virtualization unit 122 determines that partial elements 230A_5 and 230D_6 are not partial elements of the same communication device.

[0087] As described above, if it is determined that the two recognized partial elements 230 are not partial elements of the same communication device, the virtualization unit 122 does not perform the placement of the virtual panel 700 in the virtual space x'y'z'.

[0088] Returning to FIG. 8 , the display control unit 123 causes the display 105 to superimpose the virtual panel 700 placed in the virtual space x'y'z' by the virtualization unit 122 on the image of the panel 210 captured by the camera 108. That is, the virtual panel 700 is superimposed only on the panel 210 that appears (is image-recognized) in the image captured by the camera 108. More specifically, the display control unit 123 superimposes the virtual panel 700 placed in the virtual space x'y'z' by the virtualization unit 122 on the image of the panel 210 captured by the camera 108, as if it were captured by a camera in the virtual space x'y'z'. FIG. 14 is a diagram showing an example of an AR image displayed on the display. For simplicity's sake, FIG. 14 omits the optical fiber cable extending from the optical transceiver 280 inserted into the communication port 212 from its midpoint. The display 105 displays an AR image 900 in which the virtual panel 700 is superimposed on a camera image 910. The position of the virtual panel 700 in the virtual space x'y'z' is matched to the position of the panel 210 in the real space xyz, so in the AR image 900, the virtual panel 700 is superimposed on the position of the panel 210 in the camera image 910. However, because the virtual panel 700 is transparent, the user cannot recognize that the virtual panel 700 is superimposed on the camera image 910.

[0089] Returning to FIG. 8 , the notification receiving unit 124 can receive notification of the state of the communication port 212 in the communication device 200. The notification receiving unit 124 receives notification of an abnormal communication port 212 in the communication device 200. In a specific example, when an abnormality occurs in the communication port 212, the communication device 200 detects the abnormality. The communication device 200 notifies the server 300 of the abnormality in the communication port 212. The notification information of the abnormality state includes the port number of the communication port 212 in which the abnormality occurred and the type of the abnormality. The server 300 records the notified abnormal state in the communication device DB 306 (see FIG. 6 ). The communication device DB 306 stores identification information (e.g., serial number) of the communication device 200, the port number of the communication port in which the abnormality occurred, and the type of the abnormality, in association with each other. When the server 300 receives notification of the abnormality in the communication port 212 from the communication device 200, the server 300 notifies the maintenance support terminal 100 of the abnormality in the communication port 212. The notification receiving unit 124 receives a notification of an abnormality in the communication port 212 from the server 300 .

[0090] The notification from the server 300 of the abnormality in the communication port 212 may be a push-type notification or a pull-type notification. In other words, the server 300 may notify the maintenance support terminal 100 of the abnormality in the communication port 212 without the maintenance support terminal 100 making an inquiry to the server 300, or the server 300 may notify the maintenance support terminal 100 of the abnormality in the communication port 212 when an inquiry is made from the maintenance support terminal 100.

[0091] A normal color and an abnormal color are set for the component model 702. The normal color is the color used when the communication port 212 corresponding to the component model 702 is normal. The abnormal color is the color used when the communication port 212 corresponding to the component model 702 is abnormal. When the notification receiving unit 124 receives a notification of an abnormal communication port 212, the display control unit 123 switches the color of the component model 702 corresponding to the communication port 212 from the normal color to the abnormal color.

[0092] FIG. 15 is a diagram showing an example of an AR image when an abnormality occurs in a communication port. In this embodiment, when a communication port 212 is normal, the part model 702 corresponding to the communication port 212 is transparent. Therefore, the normal color is transparent. When an abnormality in the communication port 212 is notified from the server 300, the color of the part model 702a corresponding to the communication port 212 where the abnormality occurred switches to an abnormal color, which is an opaque color. The abnormal color is, for example, red. In the AR image 900 on the display 105, the abnormal communication port 212 is displayed in red, and the normal communication port 212 is not displayed in red. Therefore, the user can recognize the status of the corresponding communication port 212 from the color of the part model 702 on the virtual panel 700.

[0093] 8, a non-selected color and a selected color are set for the component model 702. The non-selected color is the color when the communication port 212 corresponding to the component model 702 is not selected. The selected color is the color when the communication port 212 corresponding to the component model 702 is selected. When the display control unit 123 receives selection conditions for selecting a communication port 212, it switches the color of the component model 702 corresponding to the communication port 212 that matches the selection conditions from the non-selected color to the selected color.

[0094] FIG. 16 is a diagram showing an example of an AR image when some communication ports are selected. In the example of FIG. 16, the selection condition is "the communication port 212 must be compatible with a 1 Gbps optical transceiver 280." The selection condition is input by performing a predetermined operation on the maintenance support terminal 100. The user can input an instruction to display the selection condition input section to the input device 104 of the maintenance support terminal 100. When this instruction is input to the input device 104 of the maintenance support terminal 100, the display control unit 123 superimposes and displays the selection condition input section 920 on the AR image 900 on the display 105. The selection condition input section 920 has two check boxes for selecting 1 Gbps and 10 Gbps. The user can input the selection condition by selecting one of the check boxes.

[0095] The display control unit 123 selects a communication port 212 that matches the selection conditions from among the multiple communication ports 212 of the communication device 200. For example, the maintenance support terminal 100 can acquire information on each communication port 212 from the communication device DB 306 of the server 300, and use the acquired information to select a communication port 212 that matches the selection conditions.

[0096] In this embodiment, when a communication port 212 is not selected, the component model 702 corresponding to that communication port 212 is transparent. Therefore, the non-selected color is transparent. When selection conditions for the communication port 212 are input, the color of the component model 702b corresponding to the communication port 212 that matches the selection conditions is switched to an opaque selection color. The selection color is, for example, yellow. In the example of FIG. 16 , the component model 702b corresponding to the communication port 212 corresponding to the 1 Gbps optical transceiver 280 is displayed in the selection color. In the AR image 900 on the display 105, the communication port 212 that matches the selection conditions is displayed in yellow, and the communication port 212 that does not match the selection conditions is not displayed in yellow. Therefore, the user can recognize the communication port 212 that matches the selection conditions by the color of the component model 702 on the virtual panel 700.

[0097] Returning to FIG. 8 , the maintenance management terminal 400 can accept input of designation of the communication port 212 of the communication device 200 and work information indicating work to be performed on the designated communication port 212. In a specific example, the display control unit 411 of the maintenance management terminal 400 causes the display 405 to display a virtual panel 710 corresponding to the panel 210 of the communication device 200. The maintenance management terminal 400 accepts the designation of the communication port 212 when the input device 404 accepts input designating the communication port 212 of the virtual panel 710. That is, the administrator can select a component model corresponding to the communication port 212 by tapping, clicking, or other operations on the virtual panel 710 displayed on the display 405. When a component model is selected, the maintenance management terminal 400 accepts the designation of the communication port 212 corresponding to the selected component model.

[0098] The manager can use the input device 404 to input work information into the maintenance management terminal 400. The work information is text information including the work content.

[0099] The communication unit 412 of the maintenance management terminal 400 transmits the input work information and designation information indicating the designated communication port 212 to the server 300. In a specific example, the designation information includes the port number of the designated communication port 212 or information for identifying the selected part model. The server 300 transfers the received designation information and work information to the maintenance support terminal 100. The receiving unit 125 of the maintenance support terminal 100 receives the designation information and work information transmitted from the maintenance management terminal 400 and transferred by the server 300.

[0100] The part model 702 is set to a non-designated color and a designated color. The non-designated color is the color when the communication port 212 corresponding to the part model 702 is not designated. The designated color is the color when the communication port 212 corresponding to the part model 702 is designated. The display control unit 123 switches the color of the part model corresponding to the communication port 212 designated in the designation information received by the receiving unit 125 from the non-designated color to the designated color. Furthermore, the display control unit 123 causes the display 105 to display the work information received by the receiving unit 125.

[0101] FIG. 17 is a diagram showing an example of an AR image in which a communication port is specified and work information is displayed. In this embodiment, if a communication port 212 is not specified, the part model 702 corresponding to that communication port 212 is transparent. Therefore, the non-specified color is transparent. If some communication ports 212 are specified, the color of the part model 702c corresponding to the specified communication port 212 is switched to an opaque specified color. The specified color is, for example, blue. In the AR image 900 on the display 105, the specified communication port 212 is displayed in blue, and the unspecified communication ports 212 are not displayed in blue. Therefore, the user can recognize the specified communication port 212 by the color of the part model 702 on the virtual panel 700.

[0102] The display control unit 123 superimposes a work instruction section 930 including work information on the AR image 900 on the display 105. In the example shown in Fig. 17, the work information including the input date and time of the work information, the affiliation and name of the manager who issued the work instruction, and the work content is displayed in the work instruction section 930. The user can recognize the content of the work to be performed for the specified communication port 212.

[0103] Returning to FIG. 8 , when the input device 104 receives an input specifying a component model 702 in the virtual panel 700, the display control unit 123 causes the display 105 to display attribute information, status information, and communication volume information (hereinafter, information including attribute information, status information, and communication volume information will be referred to as "port information") related to the communication port 212 corresponding to the component model 702. The attribute information is information indicating the attribute of the optical transceiver 280 attached to the communication port 212 corresponding to the specified component model 702. The status information is information indicating the status of the communication port 212. The communication volume information is information indicating the communication volume of the communication port 212. Thus, by specifying a component model 702 in the virtual panel 700, the user can cause the display 105 to display the attribute information, status information, and communication volume information of the communication port 212 corresponding to the component model 702.

[0104] For example, the display control unit 123 may switch the color of the part model 702 corresponding to the communication port 212 specified by the user to a specified color. The specified color of the part model 702 specified by the user may be the same as or different from the specified color of the part model 702c specified by the administrator.

[0105] Fig. 18 is a diagram showing an example of an AR image in which port information is displayed. In the example of Fig. 18, port information of the component model 702a corresponding to the communication port 212 in which an abnormality has occurred in the example of Fig. 15 is displayed.

[0106] The display control unit 123 superimposes a port information display unit 940 including port information on the AR image 900 on the display 105. In the example shown in FIG. 18 , the attribute information includes the vendor name, model number, serial number, type (compatible communication standard), transmission power, reception power, bias current, and temperature of the optical transceiver 280. The status information includes whether the transmission power is normal or abnormal, whether the reception power is normal or abnormal, whether the bias current is normal or abnormal, and whether the temperature is normal or abnormal. The communication volume information includes the number of received octets, the number of received unicast packets, the number of received broadcast packets, the number of received multicast packets, the number of received packets discarded, the number of reception errors, the number of transmitted octets, the number of transmitted unicast packets, the number of transmitted broadcast packets, the number of transmitted multicast packets, the number of transmitted packets discarded, and the number of transmission errors.

[0107] The port information may include a graph showing, in time series, numerical information of the communication port 212. For example, the display control unit 123 may display the transmission power, the reception power, the bias current, or the temperature in a time series graph.

[0108] 18, an abnormal temperature has occurred in the optical transceiver 280 inserted into the communication port 212 corresponding to the component model 702a. "ALARM," indicating an abnormality, is displayed in the port information display section 940 in accordance with the temperature in the status information. This allows the user to recognize that an abnormal temperature has occurred in the optical transceiver 280 inserted into the communication port 212 displayed in red.

[0109] 8, when the notification receiving unit 124 receives a notification of an abnormal state of the communication port 212, the display control unit 123 causes the display 105 to display first response information indicating a first-stage method of dealing with the abnormality. When the state of the communication port 212 changes as a result of the first-stage method of dealing with the abnormality and the notification receiving unit 124 receives a notification of the state of the communication port 212 after the change, the display control unit 123 switches the first response information displayed on the display 105 to second response information indicating a second-stage method of dealing with the abnormality. This allows the user to proceed with the work of dealing with the abnormality of the communication port 212 according to the AR display on the display 105.

[0110] 18, a button 940a for calling up a troubleshooting tutorial is provided to the right of "ALARM" for the temperature in the status information of the port information display section 940. The user can select button 940a using the input device 104. When button 940a is selected, a tutorial is displayed that shows step-by-step methods for dealing with abnormalities.

[0111] 19A to 19D are diagrams showing examples of AR images in which information on how to deal with an abnormality is displayed. When button 940a in FIG. 18 is selected, the display control unit 123 queries the communication device DB 306 of the server 300 and acquires first action information indicating a first-stage action method for dealing with the abnormality in the communication port 212. As shown in FIG. 19A, the display control unit 123 displays a tutorial section 950A including the acquired first action information superimposed on the AR image 900 on the display 105. The tutorial section 950A includes text information as the first action information: "The transceiver needs to be replaced. Please remove the transceiver in port #3."

[0112] The tutorial section 950A enables the user to recognize that the first step in recovering from the abnormality is to remove the optical transceiver 280 from the communication port 212 with port number 3, which is displayed in red in the AR image 900. The user removes the optical transceiver 280 from the communication port 212 with port number 3 in accordance with the first corrective action information.

[0113] 8 , when the optical transceiver 280 is removed from the communication port 212 corresponding to port number 3, the communication device 200 detects the removal of the optical transceiver 280 from the communication port 212. The communication device 200 notifies the server 300 of status information indicating that the optical transceiver 280 has been removed from the communication port 212 corresponding to port number 3. The server 300 records the notified status information in the communication device DB 306 and notifies the maintenance support terminal 100 of the status information. Furthermore, the server 300 obtains second handling information indicating a second-stage handling method from the communication device DB 306 and transmits the second handling information to the maintenance support terminal 100.

[0114] When the notification receiving unit 124 receives the status information and the second handling information of the communication device 200, the display control unit 123 switches the tutorial section 950A of the AR image 900 to a tutorial section 950B including the acquired second handling information, as shown in Fig. 19B. The tutorial section 950B includes text information as the second handling information: "It has been confirmed that the transceiver in port #3 has been removed. Please insert a transceiver into port #3."

[0115] The tutorial section 950B enables the user to recognize that the second-stage solution to recover from the abnormality is to insert a new optical transceiver 280 into the communication port 212 of port number 3. The user inserts the new optical transceiver 280 into the communication port 212 of port number 3 in accordance with the second solution information.

[0116] 8 , when the optical transceiver 280 is inserted into the communication port 212 with port number 3, the communication device 200 detects the insertion of the optical transceiver 280 into the communication port 212. The communication device 200 starts initial configuration of the inserted optical transceiver 280 and notifies the server 300 of status information indicating that the optical transceiver 280 has been inserted into the communication port 212 with port number 3. The server 300 records the notified status information in the communication device DB 306 and notifies the maintenance support terminal 100 of the status information. Furthermore, the server 300 obtains third action information indicating a third-stage action method from the communication device DB 306 and transmits the third action information to the maintenance support terminal 100.

[0117] When the notification receiving unit 124 receives the status information of the communication device 200 and the third handling information, the display control unit 123 switches the tutorial section 950B of the AR image 900 to a tutorial section 950C including the acquired third handling information, as shown in Fig. 19C. The tutorial section 950C includes text information as the third handling information, such as "Insertion of a transceiver into port #3 has been confirmed. Initial settings of the transceiver are being performed. Please wait a moment."

[0118] The tutorial section 950C allows the user to recognize that the third stage of action to recover from the abnormality is to wait until the initial setup of the optical transceiver 280 inserted into the communication port 212 with port number 3 is completed. The user waits in accordance with the third action information.

[0119] 8 , when the initial setting of the optical transceiver 280 inserted into the communication port 212 of port number 3 is completed, the communication device 200 detects that the initial setting has been completed successfully. Furthermore, if the abnormality of the communication port 212 of port number 3 has been resolved, the communication device 200 detects that the communication port 212 is normal. The communication device 200 notifies the server 300 of status information indicating that the initial setting of the optical transceiver 280 has been completed for the communication port 212 of port number 3 and that the communication port is normal. The server 300 records the notified status information in the communication device DB 306 and notifies the maintenance support terminal 100 of the status information. Furthermore, the server 300 obtains completion information from the communication device DB 306 indicating that the response method has been completed, and transmits the information to the maintenance support terminal 100.

[0120] When the notification receiving unit 124 receives the status information and termination information of the communication device 200, the display control unit 123 switches the tutorial section 950C of the AR image 900 to a tutorial section 950D that includes the acquired termination information, as shown in FIG. 19D. The tutorial section 950D includes text information as termination information, such as "Initial settings of the transceiver have been completed successfully. Recovery from the abnormal state has been confirmed." The tutorial section 950D allows the user to recognize that recovery from the abnormality has been completed.

[0121] [6. Operation of the maintenance support system] The following describes the operation of the maintenance support system 10 according to the embodiment. The processor 101 of the maintenance support terminal 100 executes the maintenance support APP 120 to perform the following AR display processing, port abnormality processing, selection condition display processing, port information display processing, tutorial processing, and work instruction display processing.

[0122] FIG. 20 is a flowchart illustrating an example of an AR display process by the maintenance support terminal according to the embodiment.

[0123] The panel 210 of the communication device 200 is imaged by the camera 108 of the maintenance support terminal 100. The processor 101 captures the camera image 910 output from the camera 108 (step S101).

[0124] The processor 101 inputs the camera image 910 into the trained model 600. The trained model 600 recognizes the partial element 230 and recognizes the model of the communication device 200 from the pattern of the air vent 221 (step S102).

[0125] The processor 101 causes the position measurement unit 109 to measure the positions of the reference points 240 in the real space xyz for the two partial elements 230 (step S103).

[0126] Processor 101 determines representative position 210P of panel 210 from the positions of two reference points 240 (step S104).

[0127] Processor 101 selects virtual panel 700 corresponding to the recognized model and places the selected virtual panel 700 in virtual space x'y'z' (step S105). At this time, processor 101 places virtual panel 700 in virtual space x'y'z by aligning representative position 210P (center point) of virtual panel 700 corresponding to representative position 210P of panel 210 with representative position 210P in real space xyz. Furthermore, processor 101 places virtual panel 700 in virtual space x'y'z' so that the position in real space xyz of reference point 240 of partial element 230 coincides with the position in virtual space x'y'z' corresponding to reference point 240 of virtual panel 700, and matches the angle of panel 210 in real space xyz with the angle between virtual space x'y'z'.

[0128] The processor 101 generates an AR image 900 by superimposing the virtual panel 700 on the camera image 910, and displays the AR image 900 on the display 105 (step S106). The processor 101 repeats the above AR display process at every predetermined sampling period.

[0129] The following port abnormality processing, selection condition display processing, port information display processing, tutorial processing, and work instruction display processing are executed while the AR image 900 is displayed on the display 105.

[0130] FIG. 21 is a flowchart illustrating an example of port abnormality processing by the maintenance support terminal according to the embodiment.

[0131] When the communication device 200 detects an abnormality in the communication port 212, it notifies the server 300 of the abnormality in the communication port 212. The server 300 transfers the abnormality notification of the communication port 212 to the maintenance support terminal 100. The maintenance support terminal 100 receives the abnormality notification of the communication port 212 of the communication device 200 (step S201).

[0132] The processor 101 changes the color of the component model 702 corresponding to the communication port 212 in which the abnormality has occurred to an abnormal color (step S2020). This completes the port abnormality processing.

[0133] FIG. 22 is a flowchart illustrating an example of a selection condition display process by the maintenance support terminal according to the embodiment.

[0134] The user inputs a display instruction for the selection condition input unit 920 to the input device 104. As a result, the selection condition input unit 920 is displayed superimposed on the AR image 900. The user inputs the selection conditions for the communication port 212 by selecting a check box in the selection condition input unit 920. The processor 101 accepts the input of the selection conditions (step S301).

[0135] The processor 101 acquires information on each communication port 212 from, for example, the communication device DB 306 of the server 300, and selects a communication port 212 that meets a selection condition from among the multiple communication ports 212 of the communication device 200 (step S302).

[0136] The processor 101 switches the color of the part model 702 corresponding to the selected communication port 212 to the selected color (step S303). This completes the selection condition display process.

[0137] FIG. 23 is a flowchart showing an example of a port information display process by the maintenance support terminal according to the embodiment.

[0138] The user specifies the part model 702 (communication port 212) in the virtual panel 700 by tapping or clicking on it in the AR image 900. The processor 101 accepts the specification of the part model 702 (step S401).

[0139] The processor 101 acquires port information of the communication port 212 corresponding to the specified component model 702, for example, by making an inquiry to the communication device DB 306 of the server 300 (step S402).

[0140] The processor 101 processes the acquired port information (step S403). For example, the processor 101 creates a graph from time-series numerical information of the transmission power, the reception power, the bias current, or the temperature.

[0141] The processor 101 displays the port information display portion 940 including the port information by superimposing it on the AR image 900 (step S404). This completes the port information display process.

[0142] FIG. 24 is a flowchart illustrating an example of a tutorial process by the maintenance support terminal according to the embodiment.

[0143] When the color of the component model 702 corresponding to the communication port where an abnormality has occurred is displayed in an abnormal color in the AR image 900 by the above-described abnormal port processing and the port information display section 940 is displayed by the above-described port information display processing, the user can instruct the display of a tutorial by selecting button 940a (see FIG. 18). The processor 101 accepts a tutorial display instruction from the user (step S501).

[0144] The processor 101 acquires countermeasure information that indicates, in a step-by-step manner, how to deal with the abnormality in the communication port 212 (step S502). Next, the processor 101 superimposes and displays a tutorial portion including the acquired countermeasure information on the AR image 900 (step S503).

[0145] The user deals with the abnormality in accordance with the handling information. As a result, the state of the communication port 212 changes, and the communication device 200 detects this state change. When the communication device 200 detects the change in the state of the communication port 212, it notifies the server 300 of the state of the communication port 212 after the change. The server 300 transfers the state notification of the communication port 212 to the maintenance support terminal 100.

[0146] The processor 101 determines whether or not a status notification of the communication port 212 of the communication device 200 has been received (step S504). If the maintenance support terminal 100 has not received a status notification (NO in step S504), the processor 101 returns to step S504.

[0147] If the maintenance support terminal 100 receives the status notification (YES in step S504), the processor 101 checks the notified status and determines whether the abnormality in the communication port 212 has been resolved (step S505). If the abnormality in the communication port 212 has not been resolved (NO in step S505), the processor 101 returns to step S502 and acquires the next stage of handling information. As a result, the displayed handling information is switched to the next stage.

[0148] If the abnormality of the communication port 212 is resolved (YES in step S505), the processor 101 switches the color of the component model 702 corresponding to the communication port 212 whose abnormality has been resolved to a normal color (step S506), and stops displaying the handling information (step S507). This completes the tutorial process.

[0149] 25 is a sequence diagram showing an example of a work instruction process by the maintenance support system according to the embodiment. In the work instruction process, the maintenance management terminal 400 executes a work instruction notification process, and the maintenance support terminal 100 executes a work instruction display process.

[0150] In the work instruction notification process, the processor 401 of the maintenance management terminal 400 displays the virtual panel 710 on the display 405 in response to an instruction input by the manager (step S601).

[0151] The administrator specifies a part model corresponding to the communication port 212 that is the target of the work instruction on the virtual panel 710. The processor 401 accepts the specification of the part model (step S602).

[0152] The manager uses the input device 404 to input work information, which is text information indicating the content of the work to be instructed, into the maintenance management terminal 400. The processor 401 accepts the input of the work information (step S603).

[0153] The maintenance management terminal 400 transmits the designation information indicating the communication port 212 corresponding to the designated part model and the input work information to the server 300 (step S604). The server 300 receives the designation information and the work information, and transfers the received designation information and work information to the maintenance support terminal 100 (step S605).

[0154] The maintenance support terminal 100 receives the specification information and the work information. In the work instruction display process, the processor 101 of the maintenance support terminal 100 changes the color of the part model 702 corresponding to the communication port 212 specified in the specification information to the specified color (step S606). The processor 101 displays the work instruction section 930 including the received work information superimposed on the AR image 900 (step S607).

[0155] [7. Variations] FIG. 26 is a diagram illustrating a modified example of a panel of a communication device according to an embodiment. A communication device 200a according to this modified example allows for the attachment and detachment of communication modules. A panel 210a of the communication device 200a is provided with openings 213A, 213B, and 213C. Panels 270A, 270B, and 270C of the attached communication modules 260A, 260B, and 260C are exposed from the openings 213A, 213B, and 213C. In other words, a portion of the panel 210a is the panel 270A, 270B, and 270C of the communication modules 260A, 260B, and 260C attached to the communication device 200a. A plurality of communication ports 212 are provided on each of the panels 270A, 270B, and 270C of the communication modules 260A, 260B, and 260C.

[0156] Panel 270A of communication module 260A is provided with ventilation opening 221aA. Panel 270B of communication module 260B is provided with ventilation opening 221aB. Panel 270C of communication module 260C is provided with ventilation opening 221aC. Ventilation openings 221aA, 221aB, and 221aC are provided above communication port 212, i.e., at the upper ends of panels 270A, 270B, and 270C, respectively.

[0157] The ventilation holes 221aA, 221aB, and 221aC are used to identify the models of the communication modules 260A, 260B, and 260C. In other words, the ventilation holes 221aA, 221aB, and 221aC also serve as identification codes for the models of the communication modules 260A, 260B, and 260C. The ventilation holes 221aA provided in the panel 230A of the communication module 260A are arranged in a pattern unique to the communication module 260A. The ventilation holes 221aB provided in the panel 230B of the communication module 260B are arranged in a pattern unique to the communication module 260B. The ventilation holes 221aC provided in the panel 230C of the communication module 260C are arranged in a pattern unique to the communication module 260C.

[0158] As a result, in the communication device 200a to which the communication modules 260A, 260B, and 260C can be attached, the models of the communication modules 260A, 260B, and 260C can be identified by image recognition of the ventilation openings 221aA, 221aB, and 221aC.

[0159] In the panel 210a of the communication device 200a, a plurality of ventilation holes 222a are provided above the openings 213A, 213B, and 213C. That is, the plurality of ventilation holes 222a are provided in a portion of the panel 210a that is different from the portions of the panels 230A, 230B, and 230C of the communication modules. The ventilation holes 222a are used to identify positions within the panel 210a. That is, the ventilation holes 222a also serve as identification codes for positions within the panel 210a. The arrangement pattern and uses of the ventilation holes 222a are the same as the arrangement pattern and uses of the ventilation holes 222 described in the embodiment.

[0160] In the above-described embodiment, the maintenance support terminal 100 functions as the recognition unit 121, the virtualization unit 122, and the display control unit 123, arranges the virtual panel 700 in the virtual space x'y'z', and generates the AR image 900. However, the present invention is not limited to this. The server 300 may arrange the virtual panel 700 in the virtual space x'y'z', generate the AR image 900, and transmit data for displaying the AR image 900 to the maintenance support terminal 100. In other words, the server 300 may function as the recognition unit 121, the virtualization unit 122, and the display control unit 123.

[0161] In the above-described embodiment, the position of the partial element 230 is measured by the ray casting method, but the present invention is not limited to this. For example, the position of the partial element 230 may be measured by capturing an image of the partial element 230 with a compound eye camera.

[0162] In the above-described embodiment, a moving image is captured by the camera 108, and the AR display process is sequentially repeated to display the AR image 900 in which the virtual panel 700 is superimposed on the camera image 910 as a moving image, but this is not limiting. The virtual panel 700 may be superimposed on a still image captured by the camera 108 to display the AR image of the still image.

[0163] In the above-described embodiment, the partial element 230 of the panel 210 is captured by the camera 108, and the partial element 230 included in the camera image 910 is recognized by the trained model 600. However, this is not limiting. The entire panel 210 may be captured by the camera 108, and the entire panel 210 included in the camera image 910 may be recognized by the trained model 600. In this case, for example, a specific position of the recognized panel 210 (e.g., the top left and top right corners of the panel 210) may be measured, and a representative position may be determined based on the measured specific position.

[0164] [8. Supplementary Notes] The embodiments disclosed herein are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims rather than the above-described embodiments, and includes meanings equivalent to the claims and all modifications within the scope thereof. [Explanation of symbols]

[0165] 10 Maintenance Support System 20. Communications Center 30 Management Center 100 Maintenance support terminal (maintenance support device) 101 processors 102 Non-volatile memory 103 Volatile Memory 104 Input Device 105 Display 106 Graphics Processing Unit 107 Communication Interface (Communication I / F) 108 Camera 109 Position measurement section 110 Laser irradiation unit 111 Light receiving section 112 Gyro Sensor 113 Acceleration Sensor 120 Maintenance Support Application (Maintenance Support APP) 121 Recognition part 122 Virtualization Department 123 Display control unit 124 Notification Reception Department 125 Receiver 200,200A,200B,200C,200D,200E,200F,200G,200H,200_1,200_2,200_3,200_4,200_5,200_6,200a Communication equipment 210, 210_1, 210_2, 210_3, 210_4, 210_5, 210_6, 210a Panel (one side) 210P representative position 211 external port 212 communication port 213A, 213B, 213C opening (1st opening, 2nd opening) 221, 221A, 221B, 221aA, 221aB, 221aC Vent (first vent) 222, 222A, 222B, 222C, 222a Vent (second vent) 230,230A,230B,230C,230D,230A_1,230D_1,230D_2,230A_3,230D_3,230A_4,230D_4,230A_5,230D_5,230A_6,230D_6 Partial element 240,240A,240B,240C,240D,240A_1,240D_1,240D_2,240A_3,240D_3,240D_4,240A_5,240D_5,240D_6 Reference point 250, 250A, 250B rack 260A, 260B, 260C Communication modules (first communication module, second communication module) 270A, 270B, 270C Panel 280 Optical Transceiver 300 servers 301 processor 302 Non-volatile memory 303 Volatile Memory 304 Communication Interface (Communication I / F) 305 Server Program 306 Communication Equipment Database (Communication Equipment DB) 400 Maintenance management terminal 401 processor 402 Non-volatile memory 403 Volatile Memory 404 Input Device 405 Display 406 Graphics Processing Unit 407 Communication Interface (Communication I / F) 410 Maintenance management APP 411 Display control unit 412 Communications Department 500 Network 600 pre-trained models 700, 700A, 700B, 700C Virtual Panel (Virtual Surface) 701,702,702a,702b,702c,703 Parts Model 710, 710A, 710B, 710C Virtual Panel 900 AR images 910 camera images 920 Selection condition input section 930 Work instruction department 940 Port information display section 940a Button 950A, 950B, 950C, 950D Tutorial Section

Claims

1. A rectangular parallelepiped housing; a plurality of communication ports arranged on one surface of the housing; Equipped with the one surface includes a plurality of vents arranged in a row; an arrangement pattern of the plurality of vents at the plurality of positions on the one surface is a pattern assigned to a model of a communication device; Communication equipment.

2. the plurality of vents include a plurality of first vents arranged in a row and a plurality of second vents arranged in a row; the plurality of first vents are arranged at a plurality of positions on the one surface in a pattern assigned to the model; the plurality of second vents are arranged in a pattern according to their positions on the one surface; The communication device according to claim 1 .

3. the plurality of second vent holes are arranged in a first pattern on a left side of the one face, in a second pattern on a central part of the one face, and in a third pattern on a right side of the one face; The communication device according to claim 2 .

4. the one surface includes a partial element used for image recognition; The arrangement pattern of the plurality of air vents included in the partial element is a pattern assigned to the model. The communication device according to any one of claims 1 to 3.

5. A communication device to which a communication module can be attached, Rectangular casing, Equipped with one surface of the housing includes an opening through which a module panel that is a part of the communication module attached to the communication device is exposed; the opening has a shape and a size that exposes a plurality of first vent holes arranged in a row in a pattern assigned to the model of the communication module at a plurality of positions on the module panel; the one surface includes a plurality of second vent holes arranged in a row; the plurality of second vents are arranged in a pattern according to their positions on the one surface; Communication equipment.

6. The one surface includes a first opening and a second opening, the first opening exposes a plurality of first vent holes arranged in a row in a pattern assigned to a model of the first communication module at a plurality of positions on a module panel of the first communication module when the first communication module is attached to the communication device; When a second communication module is attached to the communication device, the second opening exposes a plurality of first vent holes arranged in a row in a pattern assigned to a model of the second communication module at a plurality of positions on a module panel of the second communication module. The communication device according to claim 5 .

7. A maintenance support system that supports maintenance of a communication device having a housing with a plurality of communication ports and a plurality of vent holes on one surface thereof, a maintenance support terminal including a camera and a display; a recognition unit that recognizes a model of the communication device based on an arrangement pattern of the plurality of air vents in the image of the one surface captured by the camera; a position measurement unit that measures a position in real space of the one surface of the communication device whose model has been recognized by the recognition unit; a virtualization unit that determines a representative position of the one surface based on the position of the one surface measured by the position measurement unit, and places a virtual surface that is a virtual three-dimensional model of the one surface at a position corresponding to the representative position in a virtual space that corresponds to the real space; a display control unit that causes the virtual surface arranged in the virtual space by the virtualization unit and the image of the surface captured by the camera to be superimposed on the display; Equipped with Maintenance support system.

8. A maintenance support device that supports maintenance of a communication device having a housing with a plurality of communication ports and a plurality of vent holes on one surface thereof, a recognition unit that recognizes a model of the communication device based on an arrangement pattern of the plurality of air vents in the image of the one surface captured by a camera; a virtualization unit that determines a representative position of the one surface based on a position in real space of the one surface of the communication device whose model has been recognized by the recognition unit, and places a virtual surface that is a virtual three-dimensional model of the one surface at a position corresponding to the representative position in a virtual space that corresponds to the real space; a display control unit that generates an augmented reality image by superimposing the virtual surface arranged in the virtual space by the virtualization unit and the image of the surface captured by the camera; Equipped with Maintenance support equipment.

9. A maintenance support method for supporting maintenance of a communication device having a plurality of communication ports and a plurality of ventilation holes on one surface of a housing, comprising: a step of a maintenance support device recognizing a model of the communication device based on an arrangement pattern of the plurality of air vents in an image of the one surface captured by a camera; a step in which the maintenance support device determines a representative position of the one surface based on a position in real space of the one surface of the communication device whose model has been recognized, and places a virtual surface that is a virtual three-dimensional model of the one surface at a position corresponding to the representative position in a virtual space that corresponds to the real space; a step in which the maintenance support device generates an augmented reality image by superimposing the virtual surface arranged in the virtual space and the image of the surface captured by the camera; Including, Maintenance support method.

Images