Information processing device, information processing system, program, and recording medium

Abstract

To indicate display information corresponding to a photographed portion of a device only by photographing the portion of the device.SOLUTION: A photographing unit 31 takes an image of a target device 50 for which operation guidance display or repair guidance display is performed. An information storage unit 43 is a database in which display information to be indicated for a portion such as a display panel or an internal portion of the target device 50 is stored. An information analysis unit 41 receives information such as image data on the target device 50 transmitted from a transmission unit 32 of a portable terminal device 10, and identifies a portion of the target device 50 photographed by the photographing unit 31 or included in an image being photographed. A display information acquisition unit 42 acquires display information to be indicated for the portion of the target device 50 identified by the information analysis unit 41, from the information storage unit 43. A display unit 37 indicates display information acquired by the display information acquisition unit 42 and transmitted via a transfer unit 44 and a reception unit 33 on the basis of control of a display control unit 36.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system, a program, and a recording medium. [Background technology]

[0002] Various types of devices are used as automatic analyzers that analyze the types and amounts of components contained in biological samples such as blood and urine collected from the human body. Multiple types of automatic analyzers may be used in one facility. In particular, when analyzing multiple measurement items in one facility, multiple automatic analyzers made by different manufacturers are often used in combination.

[0003] Therefore, a user who wishes to use a device must check the model of the device and follow the instructions in the instruction manual for that model when operating the device or performing simple maintenance work.

[0004] In addition, there may be many versions of software programs installed to control the operation of an automated analyzer, and differences between new and old versions of the software programs may result in changes to, for example, the design of the layout of the operation screen, and changes to the number, font, and arrangement of operation buttons.

[0005] Furthermore, not only users of the automatic analyzer, but also customer service personnel (hereinafter referred to as service personnel) from the manufacturer who repair the automatic analyzer typically use the above-mentioned instruction manual, as well as a service manual specifically for service personnel which they use when providing repair services.

[0006] The instruction manuals and service manuals described above are often provided as paper-based booklets and also as electronic media connected to a terminal device. For this reason, a system has been proposed in which code information such as a QR code (registered trademark) is displayed on the target device, and the code information is read by a mobile terminal device to obtain instruction manual information and support information for the target device from a management server (see, for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2008-83966 A [Patent Document 2] Patent Publication No. 2022-15470 Summary of the Invention [Problem to be solved by the invention]

[0008] However, a user who is unfamiliar with operating the device may not be able to understand what operations to perform to achieve a certain operation goal just by looking at the instruction manual. Also, a serviceman who is unfamiliar with the structure of the device may not know which part of the service manual to look at and how to perform the repair, even if he or she removes the device's casing to repair the inside.

[0009] An object of the present disclosure is to provide an information processing device, an information processing system, a program, and a recording medium that are capable of displaying display information corresponding to a part of a device that has been photographed simply by photographing the part of the device. [Means for solving the problem]

[0010] An information processing device according to an aspect of the present disclosure includes: a storage unit that stores display information to be displayed for a portion of a device; An image capturing unit that captures an image of the device; an identification unit that identifies a part of a device included in an image that has been photographed or is being photographed by the photographing unit; an acquisition unit that acquires, from the storage unit, display information to be displayed for the part of the device identified by the identification unit; and a display unit that displays the display information acquired by the acquisition unit. Effect of the Invention

[0011] According to the present disclosure, it is possible to obtain an effect that, by simply photographing a part of a device, it is possible to display display information corresponding to the photographed part of the device. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating a system configuration of an information processing system according to an embodiment of the present disclosure. [Diagram 2] 1 is a block diagram showing a hardware configuration of a mobile terminal device 10 in an information processing system according to an embodiment of the present disclosure. [Diagram 3] 2 is a block diagram showing a hardware configuration of a server device 20 in the information processing system according to an embodiment of the present disclosure. FIG. [Figure 4] 2 is a block diagram showing the functional configuration of a mobile terminal device 10 and a server device 20. FIG. [Diagram 5] 10 is a flowchart for explaining an operation when a user photographs a target device 50 with a mobile terminal device 10 and performs operation guidance display. [Figure 6] 13 is a diagram showing an example of an operation screen when the operation information receiving unit 35 receives input of information on the model name and destination of the target device 50 from the user. [Figure 7] 13 is a diagram showing an example of an operation screen when the operation information receiving unit 35 receives, from the user, an input of an operation purpose of an operation that the user is about to perform. FIG. [Figure 8] 13 is a diagram for explaining how an image of a display panel 51 of a target device 50 is captured by an image capturing unit 31. FIG. [Figure 9]13A and 13B are diagrams showing examples of operation guidance displayed on a display unit 37. [Figure 10] 13 is a diagram showing an example of an operation guidance display displayed on the display unit 37 after a transition of the display screen. FIG. [Figure 11] 10 is a flowchart for explaining an operation when a service technician photographs a target device 50 using a mobile terminal device 10 and displays repair guidance. [Figure 12] 13 is a diagram showing an example of an operation screen when an operation information receiving unit 35 receives input of information on a part to be repaired from a user. FIG. [Figure 13] 13 is a diagram showing an example of an operation screen when an operation information receiving unit 35 receives input of user information from a user. FIG. [Figure 14] 13 is a diagram for explaining how an image of an internal part of a target device 50 is captured by a capturing unit 31. FIG. [Figure 15] FIG. 13 is a diagram showing a state in which text information indicating the names of each part of the internal part is superimposed and displayed on an image of the internal part of a photographed target device 50. [Figure 16] FIG. 13 is a diagram showing how image information indicating a repair procedure is displayed as repair guidance information. [Figure 17] 2 is a block diagram showing a functional configuration of a mobile terminal device 10A having a configuration incorporating the functions of a server device 20. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Next, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0014] An example of an embodiment of the technology of the present disclosure will be described below with reference to the drawings. Note that components and processes that perform the same operation, action, and function may be given the same reference numerals throughout all the drawings, and duplicated descriptions may be omitted as appropriate. Each drawing is merely a schematic illustration to the extent that the technology of the present disclosure can be fully understood. Therefore, the technology of the present disclosure is not limited to only the illustrated examples. In addition, in this embodiment, descriptions of configurations that are not directly related to the present disclosure or well-known configurations may be omitted.

[0015] FIG. 1 is a diagram showing a system configuration of an information processing system according to an embodiment of the present disclosure.

[0016] The information processing system of this embodiment is a system for displaying information such as operation guidance and repair guidance for target devices 50, such as automatic analyzers, such as diabetes testing devices and urine testing devices, installed in a certain facility.

[0017] This information processing system includes a mobile terminal device 10 such as a smartphone or a tablet terminal device, and a server device 20.

[0018] The server device 20 stores operation guidance information and instruction manual information showing how to operate the target device 50, repair guidance information showing how to repair a malfunctioning part of the target device 50, and service manual information.

[0019] The mobile terminal device 10 and the server device 20 are connected to each other via a wireless LAN terminal 30 and the Internet 40 .

[0020] Next, the hardware configuration of the above-described mobile terminal device 10 is shown in a block diagram of FIG.

[0021] 2, the mobile terminal device 10 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a user interface (UI) device 14, a storage 15, a camera 16, and a communication interface (IF) 17. Each component is connected to each other via a control bus 18 so as to be able to communicate with each other.

[0022] The ROM 12 records various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 15 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and records various programs including an operating system, and various data.

[0023] The CPU 11 (an example of a processor) is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 15, and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various arithmetic processing according to the program stored in the ROM 12 or the storage 15.

[0024] Next, the hardware configuration of the above-described server device 20 is shown in a block diagram of FIG.

[0025] 3, the server device 20 includes a CPU 21, a ROM 22, a RAM 23, a UI device 24, a storage 25, and a communication interface (IF) 26. Each component is connected via a control bus 27 so as to be able to communicate with each other.

[0026] The ROM 22 records various programs and various data. The RAM 23 temporarily stores programs or data as a working area. The storage 25 is configured with an HDD or SSD, and records various programs including an operating system, and various data.

[0027] The CPU 21 is a central processing unit that executes various programs and controls each part. That is, the CPU 21 reads a program from the ROM 22 or the storage 25, and executes the program using the RAM 23 as a working area. The CPU 21 controls each of the above components and performs various arithmetic processing according to the program stored in the ROM 22 or the storage 25.

[0028] The functional configuration of the mobile terminal device 10 and the server device 20 configured as above is shown in a block diagram of FIG.

[0029] As shown in FIG. 4, the mobile terminal device 10 includes an image capturing unit 31, a transmitting unit 32, a receiving unit 33, an image information acquiring unit 34, an operation information accepting unit 35, a display control unit 36, and a display unit 37.

[0030] By installing an application program (hereinafter, abbreviated as an app) in the mobile terminal device 10, an image information acquisition unit 34, an operation information acceptance unit 35, and a display control unit 36 ​​are realized.

[0031] As shown in FIG. 4, the server device 20 includes an information analysis unit 41, a display information acquisition unit 42, an information storage unit 43, and a transfer unit 44.

[0032] The photographing unit 31 photographs an image of the target device 50 on which the operation guidance display or repair guidance display is to be performed.

[0033] The image information acquisition unit 34 acquires image data of the target device 50 that has been photographed or is currently being photographed by the photographing unit 31.

[0034] The operation information receiving unit 35 receives, from the user, input of information related to the target device 50. Specifically, the operation information receiving unit 35 receives input of at least one of model name information of the target device 50, destination information of the target device 50, unit name information of the internal part of the target device 50 photographed by the photographing unit 31, information related to the user's operation purpose, user attribute information, and user identification information.

[0035] The transmission unit 32 transmits, to the server device 20, image data of the target device 50 photographed by the photographing unit 31 and acquired by the image information acquisition unit 34, and input information from the user accepted by the operation information acceptance unit 35.

[0036] The receiving unit 33 receives the display information transferred from the server device 20 which is an external device, specifically, the display information transferred from the transfer unit 44 of the server device 20 .

[0037] The display control unit 36 ​​causes the display unit 37 to display the display information received by the receiving unit 33 .

[0038] The display unit 37, under the control of the display control unit 36, displays the display information acquired by the display information acquisition unit 42 and transmitted via the transfer unit 44 and the receiving unit 33. In other words, the display unit 37 displays the display information received by the receiving unit 33.

[0039] Here, the display unit 37 superimposes and displays the display information acquired by the display information acquisition unit 42 at a position corresponding to the position of the target device 50 in the image captured by the imaging unit 31 .

[0040] For example, when the image capturing unit 31 captures an image of the display panel of the target device 50, the display unit 37 displays the operation guidance information by superimposing it on the display panel in the captured image.

[0041] Furthermore, for example, when the internal parts of the target device 50 are photographed by the photographing unit 31, the display unit 37 displays repair guidance information showing a repair method for repairing the faulty part and information such as the names of each part of the internal parts, superimposed on the parts of the internal parts in the photographed image.

[0042] The information analysis unit 41 functions as an identification unit that receives information such as image data of the target device 50 transmitted from the transmission unit 32 of the mobile terminal device 10, and identifies the part of the target device 50 contained in the image captured by the photographing unit 31.

[0043] The information analysis unit 41 has prepared in advance an image analysis model in which correlations between a plurality of images relating to the target device 50 and the parts of the target device 50 corresponding to the plurality of images are constructed in advance.

[0044] In addition, this image analysis model may be a trained learning model generated by performing machine learning using training data composed of multiple images of the target device 50 and parts of the target device 50 corresponding to the multiple images.

[0045] Then, the information analysis unit 41 uses the image analysis model constructed as described above to identify the part of the target device 50 included in the image captured by the imaging unit 31.

[0046] Here, when performing machine learning of the learning model for displaying operation guidance, learning is performed using learning data as teacher data, which consists of an image of the display panel of the target device 50 that was photographed, information such as the model name and destination of the target device 50, version information of the software program installed in the target device 50, and display information to be superimposed and displayed on the display panel.

[0047] In addition, when performing machine learning of the learning model for displaying repair guidance, learning is performed using learning data consisting of photographed images of the internal parts of the target device 50, information such as the model name and destination of the target device 50, and display information regarding the internal parts to be displayed as teacher data.

[0048] The information storage unit 43 is a database in which display information to be displayed on the display panel of the target device 50, internal parts, and other parts is stored.

[0049] Here, when operation guidance is displayed, the display information stored in the information storage unit 43 is operation guidance information for showing the user how to operate the target device 50 in order to achieve a certain operation purpose, or instruction manual information for the target device 50.

[0050] In addition, when repair guidance is displayed, the display information stored in the information storage unit 43 includes at least one of service manual information for repairing an internal part of the target device 50, information regarding the part name of the internal part, or repair guidance information showing a method for repairing the internal part.

[0051] The display information acquisition unit 42 acquires, from the information storage unit 43, display information to be displayed for the part of the target device 50 identified by the information analysis unit 41.

[0052] For example, when the part of the target device 50 specified by the information analysis unit 41 is a display panel of the target device 50, the display information acquisition unit 42 acquires operation guidance information corresponding to the display content of the display panel as display information from the information storage unit 43.

[0053] Also, for example, when the part of the target device 50 identified by the information analysis part 41 is an internal part of the target device 50, the display information acquisition part 42 acquires information about the internal part from the information storage part 43 as display information.

[0054] In addition, in some cases, various information related to the target device 50, such as information on the model name of the target device 50 and destination information, is input to the operation information receiving unit 35 of the mobile terminal device 10 and transmitted to the server device 20 via the transmission unit 32. In such cases, the display information acquisition unit 42 refers to the information related to the target device 50 received by the operation information receiving unit 35, and acquires display information corresponding to the part of the target device 50 specified by the information analysis unit 41 from the information storage unit 43.

[0055] The transfer unit 44 transfers the display information acquired by the display information acquisition unit 42 to the mobile terminal device 10 .

[0056] There may be many versions of the software program installed to control the operation of the target device 50. The difference between the new and old versions of this software program may change the layout of the operation screen displayed in the display panel, and the number, font, arrangement, etc. of the operation buttons may change.

[0057] Therefore, the information analysis unit 41 may determine the version of the software program installed in the target device 50 based on the display content displayed on the display panel.

[0058] In this case, the display information acquisition unit 42 acquires, as display information, from the information storage unit 43, operation guidance information corresponding to the version of the software program determined by the information analysis unit 41.

[0059] Next, the operation of the information processing system according to the embodiment of the present disclosure will be described in detail with reference to the drawings.

[0060] [Operation Guidance Display] First, the operation when the user photographs the target device 50 with the mobile terminal device 10 and displays operation guidance will be described with reference to the flowchart of FIG.

[0061] First, the operation information receiving unit 35 of the mobile terminal device 10 receives an input of model information of the target device 50 for which operation guidance is to be displayed (step S101).

[0062] For example, the operation information receiving unit 35 receives input of information on the model name and destination of the target device 50 from the user through an operation screen as shown in FIG.

[0063] Next, the operation information accepting section 35 of the mobile terminal device 10 accepts an input of an operation purpose that the user intends to perform on the target device 50 (step S102).

[0064] For example, as shown in FIG. 7, the operation information receiving unit 35 receives an input of an operation purpose of an operation that the user is about to perform from the user by having the user select an operation purpose that the user is about to perform from among a plurality of operation purposes.

[0065] Next, the photographing unit 31 photographs the target device 50 (step S103). Since the purpose here is to display operation guidance, the photographing unit 31 photographs the display panel 51 of the target device 50 as shown in Fig. 8. Note that the photographing by the photographing unit 31 does not have to be a still image, and the photographing unit 31 may also photograph a moving image of the display panel 51.

[0066] Then, the image data of the display panel 51 photographed by the photographing unit 31 and the information such as the device information and the operation purpose received by the operation information receiving unit 35 are transmitted to the server device 20 by the transmitting unit 32. Then, in the server device 20, the information analyzing unit 41 analyzes the transmitted image data using a trained learning model, and identifies the version of the software installed in the target device 50 from the image of the display panel 51 in the image data, identifies the currently displayed operation screen, and so on (step S104). Then, the information analyzing unit 41 determines whether there is display information to be displayed on the current operation screen (step S105).

[0067] Then, if it is determined that there is display information to be displayed on the current operation screen, the display information acquisition unit 42 acquires the display information to be displayed from the information storage unit 43, and transfers the acquired display information to the mobile terminal device 10 via the transfer unit 44 (step S106).

[0068] If it is determined that there is no display information to be displayed on the current operation screen, the display information acquisition unit 42 transfers display information indicating an error to the mobile terminal device 10 via the transfer unit 44 (step S107).

[0069] Then, in the mobile terminal device 10, the display information transferred from the server device 20 is received by the receiving unit 33, and the received display information is displayed on the display unit 37 by the display control unit 36 ​​(step S108).

[0070] An example of the operation guidance displayed on the display unit 37 in this way is shown in Fig. 9. Referring to Fig. 9, it can be seen that in the mobile terminal device 10, operation guidance for identifying the operation button to be operated to achieve the operation purpose is superimposed and displayed on the captured image of the display panel 51 of the target device 50.

[0071] Then, when it is detected that the display screen of display panel 51 has changed as a result of the user performing an operation following this operation guidance (step S109), the processes of steps S103 to S108 are repeated.

[0072] As a result, an example of the operation guidance displayed on the display unit 37 of the mobile terminal device 10 is shown in Fig. 10. Referring to Fig. 10, it can be seen that the operation guidance for identifying the operation button that the user should operate next is superimposed on the display screen after the transition.

[0073] By repeating such display switching, operation guidance is provided to achieve the operation purpose input by the user.

[0074] [Display repair guidance] Next, the operation of the service technician when he photographs the target device 50 with the mobile terminal device 10 and displays repair guidance will be described with reference to the flow chart of FIG.

[0075] First, the operation information receiving unit 35 of the mobile terminal device 10 receives an input of model information of the target device 50 for which repair guidance is to be displayed, via an operation screen such as that shown in FIG. 6 (step S201).

[0076] Next, the operation information receiving unit 35 of the mobile terminal device 10 receives input of information on the part to be repaired and user information (step S202).

[0077] For example, the operation information receiving unit 35 receives input of information on a part to be repaired from a user through an operation screen as shown in Fig. 12. The operation information receiving unit 35 also receives input of information such as a user ID (user identification information) and whether the user is a general user or a serviceman as attribute information of the user who will carry out the repair through an operation screen as shown in Fig. 13.

[0078] By inputting the user's attribute information in this manner, it becomes possible to acquire and display display information according to the user's attributes. Specifically, if the user is a general user, display information showing simple repair methods acceptable to general users is displayed, and if the user is a professional user such as a serviceman, a service manual showing detailed repair methods is displayed.

[0079] Next, the photographing unit 31 photographs the target device 50 (step S203). Since the purpose here is to display repair guidance, the photographing unit 31 photographs the internal parts of the target device 50 with the housing cover removed, as shown in Fig. 14. Note that the photographing by the photographing unit 31 does not have to be a still image, and the photographing unit 31 may also photograph a moving image of the display panel 51.

[0080] Then, the image data of the display panel 51 photographed by the photographing unit 31 and the information such as the device information and the operation purpose received by the operation information receiving unit 35 are transmitted to the server device 20 by the transmitting unit 32. Then, in the server device 20, the information analyzing unit 41 analyzes the transmitted image data using the trained learning model, and identifies the location of the photographed internal part (step S204). Then, the information analyzing unit 41 determines whether or not there is display information related to the identified internal part (step S205). Here, the display information related to the internal part is service manual information for repairing the internal part of the target device 50, information related to the part names of the internal part, and repair guidance information for showing a method for repairing the internal part.

[0081] Then, when it is determined that there is display information to be displayed, the display information acquisition unit 42 acquires the display information to be displayed from the information storage unit 43, and transfers the acquired display information to the mobile terminal device 10 by the transfer unit 44 (step S206).

[0082] If it is determined that there is no display information to be displayed, the display information acquisition unit 42 transfers display information indicating an error to the mobile terminal device 10 via the transfer unit 44 (step S207).

[0083] Then, in the mobile terminal device 10, the display information transferred from the server device 20 is received by the receiving unit 33, and the received display information is displayed on the display unit 37 by the display control unit 36 ​​(step S208).

[0084] Then, when the user carries out repairs according to this repair guidance, and some screen operation such as an instruction to transition to the next screen or input of repair completion is detected (step S209), the processing from steps S203 to S208 is repeated.

[0085] Examples of repair guidance displayed on the display unit 37 in this manner are shown in FIGS.

[0086] In the repair guidance display example shown in FIG. 15, text information indicating the names of each part of the internal part is superimposed on the image of the internal part of the photographed target device 50.

[0087] 15, an operation button 60 is displayed with the words "Display service manual." When the user presses this operation button 60, the service manual of the photographed target device 50 is displayed. When the service manual is displayed, the page of the service manual related to the photographed internal part may be displayed in an open state.

[0088] Furthermore, when a portion of the internal parts of the target device 50 is photographed in an enlarged manner by the photographing unit 31, the corresponding page of the service manual relating to the photographed target part may be automatically displayed, or repair guidance showing how to repair the target part may be automatically displayed.

[0089] For example, when an image of the "table base" portion is captured with the housing cover removed, the information analysis unit 41 recognizes that the learning model is near the table base based on the surrounding mechanical structure, components, and positional relationship with the surroundings, etc. As a result, the display information acquisition unit 42 acquires display information of the page of the service manual related to the table base, and the acquired display information of the page of the service manual related to the table base is displayed on the display unit 37.

[0090] 16, image information showing the repair procedure for executing the "method of replacing the XXX unit" is displayed as repair guidance information. The user can achieve the intended repair by sequentially executing repair operations on the target device 50 while viewing this repair guidance information.

[0091] [Configuration realized by mobile terminal device only] In the embodiment described above, operation guidance information and repair guidance information are stored in the server device 20, and the mobile terminal device 10 transmits a photographed image of the target device 50 to the server device 20 and displays the display information obtained from the server device 20.

[0092] However, the functions of the server device 20 may be incorporated into the mobile terminal device so that the above-mentioned functions can be realized by the mobile terminal device alone. The functional configuration of a mobile terminal device 10A having such a configuration is shown in FIG.

[0093] As shown in FIG. 17, the mobile terminal device 10A includes an imaging unit 31, an operation information receiving unit 35, a display control unit 36, an information analysis unit 41, a display information acquisition unit 42, an information storage unit 43, and a display unit 37.

[0094] By installing an application in the mobile terminal device 10A, an operation information receiving unit 35, a display control unit 36, an information analysis unit 41, a display information acquisition unit 42, and an information storage unit 43 are realized.

[0095] The information analysis unit 41 identifies a part of the target device 50 included in the image captured by the imaging unit 31. The display information acquisition unit 42 references the operation information accepted by the operation information acceptance unit 35 and acquires, from the information storage unit 43, display information to be displayed on the part of the target device 50 identified by the information analysis unit 41. Then, the display control unit 36 ​​causes the display information acquired by the display information acquisition unit 42 to be displayed on the display unit 37. The display unit 37 displays the display information acquired by the display information acquisition unit 42 under the control of the display control unit 36.

[0096] The display unit 37 may superimpose and display the display information acquired by the display information acquisition unit 42 at a position corresponding to the position of the target device 50 in the image being captured by the capture unit 31, instead of the still image captured by the capture unit 31. For example, a video of the target device 50 may be captured using VR (Virtual Reality) goggles instead of the mobile terminal device 10, and the display information acquired by the display information acquisition unit 42 may be superimposed and displayed at a position corresponding to the position of the target device 50 in the video being captured.

[0097] [Variations] In this embodiment, the case where the target device is an automatic analyzer such as a diabetes testing device or a urine testing device and information such as operation guidance and repair guidance is displayed has been described. However, the present disclosure can be similarly applied to cases where the target device is a device other than an automatic analyzer.

[0098] In addition, in the present embodiment, a case has been described in which display information such as operation guidance information or repair guidance information for a target device is displayed using a mobile terminal device such as a smartphone or a tablet terminal device, but the present disclosure is not limited to such a case. The present disclosure can be similarly applied to a case in which display information such as operation guidance information or repair guidance information for a target device is displayed using various information processing devices equipped with a shooting function such as a notebook computer or a video camera device.

[0099] [Note] Preferred embodiments of the present disclosure will be described below.

[0100] [Appendix 1] a storage unit in which display information to be displayed for a part of the device is stored; An image capturing unit that captures an image of the device; an identification unit that identifies a part of a device included in an image that has been photographed or is being photographed by the photographing unit; an acquisition unit that acquires, from the storage unit, display information to be displayed for the part of the device identified by the identification unit; a display unit that displays the display information acquired by the acquisition unit; An information processing device comprising:

[0101] [Appendix 2] When the part of the device identified by the identification unit is a display panel of the device, the acquisition unit acquires, from the storage unit, operation guidance information corresponding to a display content of the display panel as the display information. 2. An information processing device according to claim 1.

[0102] [Appendix 3] The identification unit determines a version of the software program installed in the device based on the display content displayed on the display panel; the acquiring unit acquiring, from the storage unit, operation guidance information corresponding to the version of the software program determined by the identifying unit, as the display information; 3. An information processing device according to claim 2.

[0103] [Appendix 4] when the part of the device identified by the identification unit is an internal part of the device, the acquisition unit acquires information about the internal part as the display information from the storage unit. 2. An information processing device according to claim 1.

[0104] [Appendix 5] The display information includes at least one of service manual information for repairing the internal portion, information on the part name of the internal portion, and repair guidance information for showing a method for repairing the internal portion. 5. The information processing device according to claim 4.

[0105] [Appendix 6] The information processing device according to any one of claims 1 to 5, wherein the display unit superimposes and displays the display information acquired by the acquisition unit at a position corresponding to the position of the device in an image captured or being captured by the imaging unit.

[0106] [Appendix 7] The information processing device according to any one of Supplementary Note 1 to Supplementary Note 6, wherein the identification unit identifies the part of the equipment contained in the image captured by the photographing unit or being captured by the photographing unit using an image analysis model in which a correlation between a plurality of images relating to the equipment and the parts of the equipment corresponding to the plurality of images is pre-constructed.

[0107] [Appendix 8] The information processing device described in Appendix 7, wherein the image analysis model is a trained learning model generated by performing machine learning using training data composed of a plurality of images related to the equipment and parts of the equipment corresponding to the plurality of images.

[0108] [Appendix 9] a receiving unit that receives an input of information about the device from a user, the acquisition unit refers to information about the device accepted by the acceptance unit, and acquires, from the storage unit, display information corresponding to the part of the device identified by the identification unit. 9. An information processing device according to any one of claims 1 to 8.

[0109] [Appendix 10] The information processing device described in Appendix 9, wherein the reception unit receives input of at least one of model name information of the device, destination information of the device, unit name information of an internal part of the device that has been photographed or is being photographed by the photographing unit, information regarding the user's operation purpose, user attribute information, or user identification information.

[0110] [Appendix 11] A terminal device including a photographing unit that photographs an image of a device, a transmitting unit that transmits an image that has been photographed or is being photographed by the photographing unit, a receiving unit that receives display information from an external device, and a display unit that displays the display information received by the receiving unit; a server device including: a storage unit in which display information to be displayed for a part of an equipment is stored; an identification unit that identifies the part of the equipment included in an image transmitted by the transmission unit; an acquisition unit that acquires from the storage unit display information to be displayed for the part of the equipment identified by the identification unit; and a transfer unit that transfers the display information acquired by the acquisition unit to the terminal device; An information processing system having the above configuration.

[0111] [Appendix 12] An image capturing step of capturing an image of the device; a step of identifying a part of the device included in the image captured or being captured in the capturing step; an acquisition step of acquiring display information to be displayed for the part of the device identified in the identification step from a storage unit in which display information to be displayed for the part of the device is stored; a display step of displaying the display information acquired in the acquisition step; A program for causing a computer to execute the following.

[0112] [Appendix 13] An image capturing step of capturing an image of the device; a step of identifying a part of the device included in the image captured or being captured in the capturing step; an acquisition step of acquiring display information to be displayed for the part of the device identified in the identification step from a storage unit in which display information to be displayed for the part of the device is stored; a display step of displaying the display information acquired in the acquisition step; A non-transitory recording medium on which a program for causing a computer to execute the above is recorded. [Explanation of symbols]

[0113] 10, 10A Portable terminal device 11 CPU 12 ROM 13 RAM 14 User Interface (UI) Devices 15. Storage 16 Cameras 17 Communication Interface (IF) 18 Control Bus 20 Server device 21 CPU 22 ROM 23 RAM 24 User Interface (UI) Devices 25 Storage 26 Communication Interface (IF) 27 Control Bus 30 Wireless LAN Terminal 31 Photography Department 32 Transmitter 33 Receiving section 34 Image information acquisition unit 35 Operation information reception section 36 Display control section 37 Display section 40 Internet 41 Information Analysis Department 42 Display information acquisition section 43 Information storage section 44 Transfer Unit 50 Target Devices 51 Display Panel 60 Operation Buttons

Claims

1. a storage unit in which display information to be displayed for a part of the device is stored; An image capturing unit that captures an image of the device; an identification unit that identifies a part of a device included in an image that has been photographed or is being photographed by the photographing unit; an acquisition unit that acquires, from the storage unit, display information to be displayed for the part of the device identified by the identification unit; a display unit that displays the display information acquired by the acquisition unit; An information processing device comprising:

2. When the part of the device identified by the identification unit is a display panel of the device, the acquisition unit acquires, from the storage unit, operation guidance information corresponding to a display content of the display panel as the display information.

2. The information processing device according to claim 1.

3. The identification unit determines a version of the software program installed in the device based on the display content displayed on the display panel; the acquiring unit acquiring, from the storage unit, operation guidance information corresponding to the version of the software program determined by the identifying unit, as the display information; 3. The information processing device according to claim 2.

4. when the part of the device identified by the identification unit is an internal part of the device, the acquisition unit acquires information about the internal part from the storage unit as the display information.

2. The information processing device according to claim 1.

5. The display information includes at least one of service manual information for repairing the internal portion, information on the part name of the internal portion, and repair guidance information for showing a method for repairing the internal portion.

5. The information processing device according to claim 4.

6. The information processing device according to claim 1 , wherein the display unit superimposes and displays the display information acquired by the acquisition unit at a position corresponding to a position of the device in an image captured or being captured by the imaging unit.

7. 2. The information processing device of claim 1, wherein the identification unit identifies the part of the equipment contained in the image captured by the photographing unit or being captured by the photographing unit using an image analysis model in which a correlation between multiple images of the equipment and the parts of the equipment corresponding to the multiple images is pre-constructed.

8. The information processing device according to claim 7 , wherein the image analysis model is a trained learning model generated by performing machine learning using training data composed of a plurality of images relating to the equipment and parts of the equipment corresponding to the plurality of images.

9. a receiving unit that receives an input of information about the device from a user, the acquisition unit refers to information about the device accepted by the acceptance unit, and acquires, from the storage unit, display information corresponding to the part of the device identified by the identification unit.

2. The information processing device according to claim 1.

10. 10. The information processing device of claim 9, wherein the reception unit receives input of at least one of model name information of the device, destination information of the device, unit name information of an internal part of the device photographed or being photographed by the photographing unit, information regarding a user's purpose of operation, user attribute information, or user identification information.

11. A terminal device including a photographing unit that photographs an image of a device, a transmitting unit that transmits an image that has been photographed or is being photographed by the photographing unit, a receiving unit that receives display information from an external device, and a display unit that displays the display information received by the receiving unit; a server device including: a storage unit in which display information to be displayed for a part of an equipment is stored; an identification unit that identifies the part of the equipment included in an image transmitted by the transmission unit; an acquisition unit that acquires from the storage unit display information to be displayed for the part of the equipment identified by the identification unit; and a transfer unit that transfers the display information acquired by the acquisition unit to the terminal device; An information processing system having the above configuration.

12. An image capturing step of capturing an image of the device; a step of identifying a part of the device included in the image captured or being captured in the capturing step; an acquisition step of acquiring display information to be displayed for the part of the device identified in the identification step from a storage unit in which display information to be displayed for the part of the device is stored; a display step of displaying the display information acquired in the acquisition step; A program for causing a computer to execute the following.

13. An image capturing step of capturing an image of the device; a step of identifying a part of the device included in the image captured or being captured in the capturing step; an acquisition step of acquiring display information to be displayed for the part of the device identified in the identification step from a storage unit in which display information to be displayed for the part of the device is stored; a display step of displaying the display information acquired in the acquisition step; A non-transitory recording medium on which a program for causing a computer to execute the above is recorded.

Images