Information processing device, information processing system, and program product

Abstract

The invention provides an information processing apparatus, an information processing system, and a program product. An information processing apparatus includes: a storage unit that stores display information to be displayed on a portion of a device; an imaging unit that captures an image of the device; a specifying unit that specifies a part of the device included in the image captured or captured by the imaging unit; an acquisition unit that acquires, from the storage unit, display information to be displayed for the location of the device specified by the specification unit; and a display unit that displays the display information acquired by the acquisition unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing system, and a program product. Background Art

[0002] Various types of equipment are used as automatic analyzers for analyzing the types and amounts of components contained in biological samples collected from the human body, such as blood and urine. In addition, multiple types of automatic analyzers can be used in one facility. In particular, when analyzing multiple measurement items in one facility, multiple automatic analyzers from different manufacturers are often used in combination.

[0003] Therefore, a user who intends to use a device needs to inquire about the model of the device and operate the device or perform simple maintenance work according to the contents of the instruction manual of the device.

[0004] In addition, there may be many versions of software programs installed to control the operation of the automatic analyzer. In addition, sometimes due to differences between new and old versions of the software program, for example, the design of the operation screen layout is changed, and the number of operation buttons, character fonts, and configurations are changed.

[0005] Furthermore, the above-mentioned instruction manuals are usually used not only by users of automatic analyzers but also by customer service personnel of manufacturers (hereinafter referred to as service personnel) who repair automatic analyzers, and they usually carry a service manual dedicated to service personnel for use during maintenance services.

[0006] Instructions for use and service manuals as described above are often paper booklets and are provided as electronic media connected to a terminal device. Therefore, for example, in Japanese Patent Publication No. 2008-83966 and Japanese Patent Publication No. 2022-15470, a system is proposed that displays code information such as a QR code (registered trademark) on a target device and reads the code information through a portable terminal device, thereby acquiring instruction manual information and support information of the target device from a management server.

[0007] However, for users who are not familiar with the operation of the equipment, they do not know what operations to perform to achieve a certain operation purpose just by reading the instruction manual. In addition, maintenance personnel who are not familiar with the structure of the equipment do not know which part of the maintenance manual to read or what maintenance method to take even if they remove the equipment casing for internal maintenance. Summary of the invention

[0008] An object of the present disclosure is to provide an information processing apparatus, an information processing system, a program, and a recording medium capable of displaying display information corresponding to a photographed portion of a device only by photographing the portion of the device.

[0009] According to one aspect of the present disclosure, an information processing device comprises:

[0010] a storage unit storing display information for displaying a part of the device;

[0011] a photographing unit that photographs an image of the device;

[0012] a determination unit that determines a location of a device included in the image captured or being captured by the capturing unit;

[0013] an acquisition unit that acquires, from the storage unit, the display information to be displayed for the location of the device specified by the specifying unit; and

[0014] A display unit displays the display information acquired by the acquisition unit.

[0015] According to the present disclosure, it is possible to obtain an effect of displaying display information corresponding to the photographed part of the device only by photographing the part of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Exemplary embodiments will be described in detail based on the following drawings.

[0017] Figure 1 is a diagram showing a system configuration of an information processing system according to one embodiment of the present disclosure.

[0018] Figure 2 1 is a block diagram showing a hardware configuration of the portable terminal device 10 in the information processing system according to one embodiment of the present disclosure.

[0019] Figure 3 : is a block diagram showing the hardware structure of the server device 20 in the information processing system according to one embodiment of the present disclosure.

[0020] Figure 4 It is a block diagram showing the functional configuration of the portable terminal device 10 and the server device 20 .

[0021] Figure 5 This is a flowchart for explaining the operation when the user captures an image of the target device 50 with the portable terminal device 10 and an operation guidance display is performed.

[0022] Figure 6 1 is a diagram showing an example of an operation screen when the operation information accepting unit 35 accepts input of information on the model name of the target device 50 and the destination from the user.

[0023] Figure 7 1 is a diagram showing an example of an operation screen when the operation information accepting unit 35 accepts input of an operation purpose to be performed by the user from the user.

[0024] Figure 8 This is a diagram for explaining a state in which the image capturing unit 31 captures an image of the display panel 51 of the target device 50 .

[0025] Fig. 9 2 is a diagram showing an example of an operation guidance display displayed on the display unit 37 .

[0026] Fig.10 2 is a diagram showing an example of an operation guidance display displayed on the display unit 37 after the display screen is shifted.

[0027] Fig.11 This is a flowchart for explaining the operation when the service personnel uses the portable terminal device 10 to photograph the target device 50 to display maintenance guidance.

[0028] Fig.12 1 is a diagram showing an example of an operation screen when the operation information accepting unit 35 accepts input of information on a maintenance target portion from a user.

[0029] Fig.13 1 is a diagram showing an example of an operation screen when the operation information accepting unit 35 accepts input of user information from the user.

[0030] Fig.14 This is a diagram for explaining a state in which the imaging unit 31 captures an image of the internal portion of the target device 50 .

[0031] Fig.15 1 is a diagram showing a state in which character information indicating the name of each component of the internal part is superimposed and displayed on the image of the internal part of the target device 50 captured.

[0032] Fig.16 FIG. 1 is a diagram showing a state in which image information indicating a maintenance procedure is displayed as maintenance guidance information.

[0033] Fig.17 This is a block diagram showing the functional configuration of the portable terminal device 10A incorporating the functional configuration of the server device 20 . DETAILED DESCRIPTION

[0034] Next, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0035] Below, an example of an implementation of the technology disclosed in the present invention will be described based on the accompanying drawings. In addition, throughout the accompanying drawings, structural elements and processes having the same actions, effects, and functions may be represented by the same figure marks, and repeated descriptions may be appropriately omitted. Each of the accompanying drawings is only schematically shown so that the technology disclosed in the present invention is fully understood. Therefore, the technology disclosed in the present invention is not limited to the examples shown. In addition, in this embodiment, the description of structures that are not directly related to the present invention or known structures may be omitted.

[0036] Figure 1 is a diagram showing a system configuration of an information processing system according to an embodiment of the present disclosure.

[0037] The information processing system of the present embodiment is a system for displaying information such as operation guidance and maintenance guidance by using an automatic analyzer such as a diabetes test device or a urine test device installed in a certain facility as a target device 50 .

[0038] The information processing system is composed of a portable terminal device 10 such as a smartphone (hereinafter simply referred to as a smartphone) or a tablet terminal device, and a server device 20 .

[0039] The server device 20 stores operation guidance information and instruction manual information indicating the operation method of the target device 50 , and repair guidance information and service manual information indicating the method for repairing a faulty part of the target device 50 .

[0040] In addition, the portable terminal device 10 and the server device 20 are connected to each other via the wireless LAN terminal 30 and the Internet 40 .

[0041] Next, in Figure 2 The hardware structure of the portable terminal device 10 is shown in the block diagram.

[0042] like Figure 2 As shown, the portable terminal device 10 includes a central processing unit (CPU) 11, a read only memory (ROM) 12, a random access memory (RAM) 13, a user interface (UI) device 14, a memory 15, a camera 16, and a communication interface (IF) 17. Each component is connected via a control bus 18 so as to be communicable.

[0043] The ROM 12 records various programs and various data. The RAM 13 temporarily stores programs or data as a work area. The storage 15 is composed of a hard disk drive (HDD) or a solid state drive (SSD), and records various programs including an operating system and various data.

[0044] The CPU 11 (an example of a processor) is a central processing unit that executes various programs and controls various parts. That is, the CPU 11 reads a program from the ROM 12 or the memory 15 and executes the program using the RAM 13 as a work area. The CPU 11 controls the above-mentioned structures and performs various calculations according to the program stored in the ROM 12 or the memory 15.

[0045] Next, in Figure 3 The hardware structure of the server device 20 is shown in the block diagram.

[0046] like Figure 3As shown, the server device 20 includes a CPU 21, a ROM 22, a RAM 23, a UI device 24, a storage 25, and a communication interface (IF) 26. The components are connected via a control bus 27 so as to be communicable.

[0047] The ROM 22 records various programs and various data. The RAM 23 temporarily stores programs and data as a work area. The storage 25 is composed of a HDD or a SSD, and records various programs including an operating system and various data.

[0048] The CPU 21 is a central processing unit that executes various programs or controls various parts. That is, the CPU 21 reads the program from the ROM 22 or the memory 25 and executes the program using the RAM 23 as a work area. The CPU 21 controls the above-mentioned structures and performs various calculations according to the program stored in the ROM 22 or the memory 25.

[0049] exist Figure 4 The functional structure of the portable terminal device 10 and the server device 20 constructed as above is shown in the block diagram.

[0050] like Figure 4 As shown, the portable terminal device 10 includes an imaging unit 31 , a transmitting unit 32 , a receiving unit 33 , an image information acquiring unit 34 , an operation information accepting unit 35 , a display control unit 36 ​​, and a display unit 37 .

[0051] Furthermore, by installing an application program (hereinafter referred to as an application) in the portable terminal device 10 , the image information acquisition unit 34 , the operation information reception unit 35 , and the display control unit 36 ​​are realized.

[0052] In addition, if Figure 4 As shown, the server device 20 includes an information analysis unit 41 , a display information acquisition unit 42 , an information storage unit 43 , and a transmission unit 44 .

[0053] The imaging unit 31 captures an image of the target device 50 that performs the operation guidance display or the maintenance guidance display.

[0054] The image information acquisition unit 34 acquires image data of the target device 50 captured or being captured by the imaging unit 31 .

[0055] The operation information receiving unit 35 receives input of information about the target device 50 from the user. Specifically, the operation information receiving unit 35 receives input of at least one of the model name information of the target device 50, the destination information of the target device 50, the unit name information of the internal part of the target device 50 photographed by the photographing unit 31, the information about the user's operation purpose, the user's attribute information, or the user identification information.

[0056] The transmission unit 32 transmits the image data of the target device 50 captured by the imaging unit 31 and acquired by the image information acquisition unit 34 and the input information from the user accepted by the operation information acceptance unit 35 to the server device 20 .

[0057] The receiving unit 33 receives display information transmitted from the server device 20 as an external device, specifically, the display information transmitted from the transmitting unit 44 of the server device 20 .

[0058] The display control unit 36 ​​causes the display unit 37 to display the display information received by the receiving unit 33 .

[0059] The display unit 37 displays the display information acquired by the display information acquisition unit 42 and transmitted via the transmission unit 44 and the reception unit 33 under the control of the display control unit 36. That is, the display unit 37 displays the display information received by the reception unit 33.

[0060] Here, the display unit 37 displays the display information acquired by the display information acquisition unit 42 by superimposing it on a position corresponding to the position of the target device 50 in the image captured by the imaging unit 31 .

[0061] For example, when the display panel of the target device 50 is captured by the capturing unit 31 , the display unit 37 superimposes and displays the operation guidance information on the display panel in the captured image.

[0062] In addition, for example, when the internal parts of the target device 50 are photographed by the imaging unit 31, the display unit 37 superimposes maintenance guidance information indicating the maintenance method for repairing the faulty part and the names of the various components of the internal part on the components of the internal part in the photographed image and displays them.

[0063] The information analysis unit 41 functions as a specifying unit that receives information such as image data of the target device 50 transmitted from the transmission unit 32 of the portable terminal device 10 and specifies a part of the target device 50 included in the image captured by the imaging unit 31 .

[0064] An image analysis model is prepared in advance in the information analysis unit 41 . The image analysis model constructs in advance a correlation relationship between a plurality of images related to the target device 50 and the parts of the target device 50 corresponding to the plurality of images.

[0065] In addition, the image analysis model may be a learning model generated by machine learning using teacher data consisting of a plurality of images related to the target device 50 and parts of the target device 50 corresponding to the plurality of images.

[0066] Furthermore, the information analysis unit 41 specifies the location of the target device 50 included in the image captured by the imaging unit 31 using the image analysis model constructed as described above.

[0067] Here, when machine learning of the row learning model is performed when an operation guidance display is performed, learning is performed using learning data consisting of an image of the display panel of the target device 50 captured, information such as the model name or destination of the target device 50, version information of the software program installed in the target device 50, and display information to be superimposed on the display panel for display as teacher data.

[0068] In addition, when machine learning of the learning model is performed when a maintenance guidance display is performed, learning data consisting of images of the internal parts of the target device 50 taken, information such as the model name or arrival place of the target device 50, and display information related to the internal parts to be displayed is used as teacher data for learning.

[0069] The information storage unit 43 is a database that stores display information displayed on a portion such as a display panel or an internal portion of the target device 50 .

[0070] Here, in the case of performing operation guidance display, the display information stored in the information storage unit 43 is operation guidance information for showing the user how to operate the target device 50 to achieve a certain operation purpose or instruction manual information of the target device 50 .

[0071] In addition, when performing a maintenance guidance display, the display information stored in the information storage unit 43 includes at least any one of service manual information for repairing internal parts of the target device 50, information related to the component name of the internal part, or maintenance guidance information for indicating a method for repairing the internal part.

[0072] The display information acquisition unit 42 acquires, from the information storage unit 43 , display information to be displayed for the location of the target device 50 identified by the information analysis unit 41 .

[0073] For example, when the part of the target device 50 determined by the information analysis unit 41 is the display panel of the target device 50 , the display information acquisition unit 42 acquires operation guidance information corresponding to the display content of the display panel from the information storage unit 43 as display information.

[0074] Furthermore, for example, when the location of the target device 50 identified by the information analysis unit 41 is an internal location of the target device 50 , the display information acquisition unit 42 acquires information related to the internal location from the information storage unit 43 as display information.

[0075] In addition, sometimes, various information related to the target device 50, such as the model name of the target device 50 and the destination information, is input into the operation information receiving unit 35 of the portable terminal device 10, and is transmitted to the server device 20 via the transmitting unit 32. In this case, the display information acquiring unit 42 refers to the information related to the target device 50 received by the operation information receiving unit 35, and acquires the display information corresponding to the part of the target device 50 determined by the information analyzing unit 41 from the information storage unit 43.

[0076] The transmission unit 44 transmits the display information acquired by the display information acquisition unit 42 to the portable terminal device 10 .

[0077] In addition, there may be many versions of the software program installed to control the operation of the target device 50. In addition, sometimes due to the difference between the new version and the old version of the software program, the layout of the operation screen displayed on the display panel changes, and the number, character font, and configuration of the operation buttons change.

[0078] Therefore, the information analysis section 41 can determine the version of the software program installed in the target device 50 based on the display content displayed on the display panel.

[0079] In this case, the display information acquisition unit 42 acquires, from the information storage unit 43 , the operation guidance information corresponding to the version of the software program determined by the information analysis unit 41 as the display information.

[0080] Next, the action of the information processing system according to one embodiment of the present disclosure will be described in detail with reference to the drawings.

[0081] [Operation guidance display]

[0082] First, refer to Figure 5 The flowchart is used to explain the operation when the user captures the target device 50 through the portable terminal device 10 to display the operation guidance.

[0083] Initially, the operation information accepting unit 35 of the portable terminal device 10 accepts input of model information of the target device 50 for which operation guidance is to be displayed (step S101 ).

[0084] For example, the operation information receiving unit 35 receives Figure 6 The illustrated operation screen receives input of the model name of the target device 50 or information on the destination from the user.

[0085] Next, the operation information accepting unit 35 of the portable terminal device 10 accepts the user's input of the purpose of operation to be performed on the target device 50 (step S102 ).

[0086] For example, Figure 7As shown, the operation information accepting unit 35 accepts input of the operation purpose that the user wants to perform from the user by causing the user to select the operation purpose that the user wants to perform from a plurality of operation purposes.

[0087] Next, the image capturing unit 31 captures the target device 50 (step S103). Figure 8 As shown, the display panel 51 of the target device 50 is photographed by the photographing unit 31. In addition, the photographing performed by the photographing unit 31 may not be the photographing of a static image, but the photographing unit 31 may be the photographing of a dynamic image of the display panel 51.

[0088] Then, the image data of the display panel 51 captured by the image capturing unit 31 and the information such as the device information or the operation purpose received by the operation information receiving unit 35 are transmitted to the server device 20 by the transmitting unit 32. Then, in the server device 20, the information analyzing unit 41 analyzes the transmitted image data using the learned learning model, and determines the version of the software installed in the target device 50 and the currently displayed operation screen from the image of the display panel 51 in the image data (step S104). Then, in the information analyzing unit 41, it is determined whether there is display information to be displayed for the current operation screen (step S105).

[0089] Then, when it is determined that the current operation screen has display information to be displayed, the display information acquisition unit 42 acquires the display information to be displayed from the information storage unit 43, and transmits the acquired display information to the portable terminal device 10 via the transmission unit 44 (step S106).

[0090] When determining that there is no display information to be displayed on the current operation screen, the display information acquisition unit 42 transmits display information indicating an error to the portable terminal device 10 via the transmission unit 44 (step S107 ).

[0091] Then, in the portable terminal device 10, the display information transmitted from the server device 20 is received by the receiving unit 33, and the received display information is displayed on the display unit 37 by the display control unit 36 ​​(step S108).

[0092] As described above, the operation guidance display displayed on the display unit 37 is shown in FIG. Fig. 9 . refer to Fig. 9 , in the portable terminal device 10 , an operation guide for determining an operation button that should be operated to achieve an operation purpose is superimposed and displayed on an image of the display panel 51 of the target device 50 .

[0093] Then, when it is detected that the display screen of the display panel 51 has been changed due to the user's operation according to the operation guidance (step S109 ), the processing of steps S103 to S108 is repeated.

[0094] As a result, Fig.10 FIG. 4 shows an example of an operation guidance display displayed on the display unit 37 of the portable terminal device 10. Fig.10 It can be seen that an operation guide for determining the operation button that the user should operate next is superimposed and displayed on the display screen after the transfer.

[0095] By repeating such switching of the display, operation guidance for achieving the operation purpose input by the user is performed.

[0096] [Maintenance guidance display]

[0097] Next, we will refer to Fig.11 The flowchart is used to explain the actions when the service personnel uses the portable terminal device 10 to photograph the target equipment 50 to display the maintenance guidance.

[0098] Initially, the operation information receiving unit 35 of the portable terminal device 10 uses Figure 6 The operation screen shown in FIG. 2 accepts input of model information of the target device 50 for which maintenance guidance is to be displayed (step S201).

[0099] Next, the operation information accepting unit 35 of the portable terminal device 10 accepts input of information of a maintenance target portion to be maintained and user information (step S202).

[0100] For example, the operation information receiving unit 35 receives Fig.12 The operation screen shown in FIG. 1 receives input of information of a maintenance target part to be maintained from the user. Fig.13 The operation screen shown receives input of information such as a user ID (user identification information) and whether the user is a general user or a service person from the user as attribute information of the user who performs maintenance.

[0101] By inputting the user's attribute information in this way, display information corresponding to the user's attribute can be obtained and displayed. Specifically, when the user is a general user, the display information can be switched to display simple maintenance methods allowed by general users, and when the user is a professional user such as a service personnel, the display information can be switched to display a service manual showing detailed maintenance methods.

[0102] Next, the image capturing unit 31 captures the target device 50 (step S203). Fig.14As shown, the internal part of the target device 50 with the housing cover removed is photographed by the photographing unit 31. In addition, the photographing performed by the photographing unit 31 may not be a static image, but may be a dynamic image of the row display panel 51.

[0103] Then, the image data of the display panel 51 captured by the capturing unit 31 and the information such as the device information or the operation purpose received by the operation information receiving unit 35 are sent to the server device 20 by the sending unit 32. Then, in the server device 20, the information analysis unit 41 uses the learned learning model to analyze the sent image data and determine the position of the captured internal part (step S204). Then, the information analysis unit 41 determines whether there is display information related to the determined internal part (step S205). Here, the display information related to the internal part is the service manual information for repairing the internal part of the target device 50, the information related to the component name of the internal part, and the maintenance guide information for indicating the method for repairing the internal part.

[0104] Then, when it is determined that there is display information to be displayed, the display information acquisition unit 42 acquires the display information to be displayed from the information storage unit 43 and transmits the acquired display information to the portable terminal device 10 via the transmission unit 44 (step S206 ).

[0105] When determining that the display information to be displayed does not exist, the display information acquisition unit 42 transmits the display information indicating an error to the portable terminal device 10 via the transmission unit 44 (step S207 ).

[0106] Then, in the portable terminal device 10, the display information transmitted from the server device 20 is received by the receiving unit 33, and the received display information is displayed on the display unit 37 by the display control unit 36 ​​(step S208).

[0107] Then, when the user performs maintenance according to the maintenance guidance and a certain screen operation such as an instruction to move to the next screen or maintenance completion input is detected (step S209), the processing of steps S203 to S208 is repeated.

[0108] As above, in Fig.15 , Fig.16 2 shows an example of a maintenance guidance display displayed on the display unit 37.

[0109] exist Fig.15 In the maintenance guidance display example shown, character information indicating the names of the components of the internal portion is superimposed and displayed on the captured image of the internal portion of the target device 50 .

[0110] In addition, Fig.15In the example of the display screen shown, an operation button 60 showing "Display service manual" is displayed. When the user presses the operation button 60, the service manual of the photographed target device 50 is displayed. In addition, when the service manual is displayed, the page related to the photographed internal part among the pages of the service manual can be displayed in a state where it is opened.

[0111] Furthermore, when a portion of the internal part of the target device 50 is enlarged and photographed by the imaging unit 31, the corresponding page of the service manual related to the photographed target can be automatically displayed, or a maintenance guide indicating a maintenance method related to the target position can be automatically displayed.

[0112] For example, when an image of the "workbench base" portion is captured with the housing cover removed, the information analysis unit 41 recognizes that the learning model is near the workbench base based on the surrounding mechanical structure, structural components, positional relationship with the surroundings, etc. As a result, the display information acquisition unit 42 acquires display information of a page of the service manual related to the workbench base, and displays the acquired display information of the page of the service manual related to the workbench base on the display unit 37.

[0113] In addition, Fig.16 In the example of the display screen shown, image information showing the maintenance process for performing "replacement method of unit 000" is displayed as maintenance guidance information. When the user sequentially performs maintenance operations on the target device 50 while viewing the maintenance guidance information, the expected maintenance content can be achieved.

[0114] [Structure implemented only by portable terminal device]

[0115] In the above embodiment, the operation guidance information or the maintenance guidance information is stored in the server device 20 , and the portable terminal device 10 transmits an image of the target device 50 to the server device 20 and displays the display information obtained from the server device 20 .

[0116] However, the functions of the server device 20 may be incorporated into the portable terminal device, and the above functions may be realized by the portable terminal device alone. Fig.17 shown.

[0117] like Fig.17 As shown, the portable terminal device 10A includes an imaging unit 31 , an operation information receiving unit 35 , a display control unit 36 ​​, an information analyzing unit 41 , a display information acquiring unit 42 , an information storage unit 43 , and a display unit 37 .

[0118] Furthermore, by installing the application program in the portable terminal device 10A, the operation information accepting unit 35 , the display control unit 36 ​​, the information analyzing unit 41 , the display information acquiring unit 42 , and the information storage unit 43 are realized.

[0119] The information analysis unit 41 specifies the part of the target device 50 included in the image captured by the image capturing unit 31. The display information acquisition unit 42 refers to the operation information received by the operation information receiving unit 35, and acquires from the information storage unit 43 the display information to be displayed for the part of the target device 50 specified by the information analysis unit 41. Then, the display control unit 36 ​​causes the display unit 37 to display the display information acquired by the display information acquisition unit 42. The display unit 37 displays the display information acquired by the display information acquisition unit 42 based on the control of the display control unit 36.

[0120] Furthermore, the display unit 37 may superimpose and display the display information acquired by the display information acquisition unit 42 at a position corresponding to the position of the target device 50 in the image being captured by the imaging unit 31, instead of displaying a static image captured by the imaging unit 31. For example, a dynamic image of the target device 50 may be captured using VR (virtual reality) goggles instead of the portable terminal device 10, and the display information acquired by the display information acquisition unit 42 may be superimposed and displayed at a position corresponding to the position of the target device 50 in the dynamic image being captured.

[0121] [Modifications]

[0122] In addition, in this embodiment, the case where an automatic analyzer such as a diabetes test device or a urine test device is used as a target device to display information such as operation guidance and maintenance guidance is described, but the present disclosure is also applicable to the case where a device other than an automatic analyzer is used as a target device.

[0123] In addition, in the present embodiment, a case where display information such as operation guidance information or maintenance guidance information for a target device is displayed using a portable terminal device such as a smart phone or a tablet terminal device is described, but the present disclosure is not limited to such a case. The present disclosure can also be applied to a case where display information such as operation guidance information or maintenance guidance information for a target device is displayed using various information processing devices having a shooting function such as a notebook computer or a video camera device.

[0124] [Note]

[0125] Hereinafter, preferred embodiments of the present disclosure will be described.

[0126] [Note 1]

[0127] An information processing device comprising:

[0128] a storage unit storing display information displayed on a part of the device;

[0129] a photographing unit that photographs an image of the device;

[0130] a determination unit that determines a location of a device included in the image captured or being captured by the capturing unit;

[0131] an acquisition unit that acquires, from the storage unit, display information displayed for the location of the device specified by the specifying unit; and

[0132] A display unit displays the display information acquired by the acquisition unit.

[0133] [Note 2]

[0134] The information processing device according to Supplement 1, wherein:

[0135] When the part of the device specified by the specifying unit is a display panel of the device, the acquiring unit acquires operation guidance information corresponding to display content of the display panel from the storage unit as the display information.

[0136] [Note 3]

[0137] The information processing device according to Supplement 2, wherein:

[0138] The determination unit determines the version of the software program installed in the device based on the display content displayed on the display panel,

[0139] The acquisition unit acquires, from the storage unit, operation guidance information corresponding to the version of the software program determined by the identification unit as the display information.

[0140] [Note 4]

[0141] The information processing device according to Supplement 1, wherein:

[0142] When the part of the device specified by the specifying unit is an internal part of the device, the acquiring unit acquires information related to the internal part from the storage unit as the display information.

[0143] [Note 5]

[0144] The information processing device according to Supplement 4, wherein:

[0145] The display information includes at least any one of service manual information for repairing the internal part, information related to a component name of the internal part, or repair guide information for indicating a method for repairing the internal part.

[0146] [Note 6]

[0147] The information processing device according to any one of Annexes 1 to 5, wherein:

[0148] The display unit displays the display information captured by the imaging unit in a superimposed manner at a position corresponding to the position of the device in the image captured by the imaging unit or in the process of being captured.

[0149] [Note 7]

[0150] The information processing device according to any one of Annexes 1 to 6, wherein:

[0151] The determination unit uses an image analysis model to determine the location of the device contained in the image captured by or being captured by the capturing unit, wherein the image analysis model pre-constructs a correlation between a plurality of images related to the device and the locations of the device corresponding to the plurality of images.

[0152] [Note 8]

[0153] The information processing device according to Supplement 7, wherein:

[0154] The image analysis model is a learning model generated by machine learning using teacher data, wherein the teacher data is composed of a plurality of images related to the device and parts of the device corresponding to the plurality of images.

[0155] [Note 9]

[0156] The information processing device according to any one of Annexes 1 to 8, wherein:

[0157] The information processing device further includes a receiving unit that receives input of information related to the device from a user.

[0158] The acquisition unit refers to the information related to the device accepted by the acceptance unit, and acquires, from the storage unit, display information corresponding to the location of the device specified by the specification unit.

[0159] [Note 10]

[0160] The information processing device according to Supplement 9, wherein:

[0161] The receiving unit receives input of at least one of the model name information of the device, the destination information of the device, the unit name information of the internal part of the device photographed by the shooting unit or being photographed, information related to the user's operation purpose, and the user's attribute information or user identification information.

[0162] [Note 11]

[0163] An information processing system comprising:

[0164] A terminal device comprising: a photographing unit that photographs an image of a device; a transmitting unit that transmits an image photographed or being photographed by the photographing unit; a receiving unit that receives display information from an external device; and a display unit that displays the display information received by the receiving unit; and

[0165] A server device comprises: a storage unit that stores display information displayed on a part of a device; a determination unit that determines the part of the device included in an image sent by the sending unit; an acquisition unit that acquires, from the storage unit, display information that should be displayed on the part of the device determined by the determination unit; and a transmission unit that transmits the display information acquired by the acquisition unit to the terminal device.

[0166] [Note 12]

[0167] A program that causes a computer to execute the following steps:

[0168] A photographing step, photographing an image of the device;

[0169] a determination step of determining a location of the device contained in the image captured or being captured in the capturing step;

[0170] an acquisition step of acquiring, from a storage unit storing display information to be displayed on a device location, display information to be displayed on the device location determined in the determination step; and

[0171] The display step displays the display information acquired in the acquisition step.

[0172] [Note 13]

[0173] A non-transitory recording medium causes a computer to execute the following steps:

[0174] A photographing step, photographing an image of the device;

[0175] a determination step of determining a location of the device contained in the image captured or being captured in the capturing step;

[0176] an acquisition step of acquiring, from a storage unit storing display information to be displayed on a device location, display information to be displayed on the device location determined in the determination step; and

[0177] The display step displays the display information acquired in the acquisition step.

Claims

1. An information processing device, comprising: a storage unit storing display information displayed on a part of the device; a photographing unit that photographs an image of the device; a determination unit that determines a location of a device included in the image captured or being captured by the capturing unit; an acquisition unit that acquires, from the storage unit, display information to be displayed for the location of the device specified by the specifying unit; as well as A display unit displays the display information acquired by the acquisition unit.

2. The information processing device according to claim 1, wherein: When the part of the device specified by the specifying unit is a display panel of the device, the acquiring unit acquires operation guidance information corresponding to display content of the display panel from the storage unit as the display information.

3. The information processing device according to claim 2, wherein: The determination unit determines the version of the software program installed in the device based on the display content displayed on the display panel, The acquisition unit acquires, from the storage unit, operation guidance information corresponding to the version of the software program determined by the identification unit as the display information.

4. The information processing device according to claim 1, wherein: When the part of the device specified by the specifying unit is an internal part of the device, the acquiring unit acquires information related to the internal part from the storage unit as the display information.

5. The information processing device according to claim 4, wherein: The display information includes at least any one of service manual information for repairing the internal part, information related to a component name of the internal part, or repair guide information for indicating a method for repairing the internal part.

6. The information processing device according to any one of claims 1 to 5, wherein: The display unit displays the display information captured by the imaging unit in a superimposed manner at a position corresponding to the position of the device in the image captured by the imaging unit or in the process of being captured.

7. The information processing device according to claim 1, wherein: The determination unit uses an image analysis model to determine the location of the device contained in the image captured by or being captured by the capturing unit, wherein the image analysis model pre-constructs a correlation between a plurality of images related to the device and the locations of the device corresponding to the plurality of images.

8. The information processing device according to claim 7, wherein: The image analysis model is a learning model generated by machine learning using teacher data, wherein the teacher data is composed of a plurality of images related to the device and parts of the device corresponding to the plurality of images.

9. The information processing device according to claim 1, wherein: The information processing device further includes a receiving unit that receives input of information related to the device from a user. The acquisition unit refers to the information related to the device accepted by the acceptance unit, and acquires, from the storage unit, display information corresponding to the location of the device specified by the specification unit.

10. The information processing device according to claim 9, wherein: The receiving unit receives input of at least one of the model name information of the device, the destination information of the device, the unit name information of the internal part of the device photographed by the shooting unit or being photographed, information related to the user's operation purpose, and the user's attribute information or user identification information.

11. An information processing system comprising a terminal device and a server device, The terminal device comprises: a photographing unit that photographs an image of a device; a transmitting unit that transmits an image photographed or being photographed by the photographing unit; a receiving unit that receives display information from an external device; and a display unit that displays the display information received by the receiving unit; and The server device includes: a storage unit that stores display information displayed on a part of a device; a determination unit that determines the part of a device included in an image sent by the sending unit; an acquisition unit that acquires, from the storage unit, display information that should be displayed on the part of the device determined by the determination unit; and a transmission unit that transmits the display information acquired by the acquisition unit to the terminal device.

12. A program product causing a computer to execute the following steps: A photographing step, photographing an image of the device; a determination step of determining a location of the device contained in the image captured or being captured in the capturing step; an acquisition step of acquiring, from a storage unit storing display information to be displayed on a device location, display information to be displayed on the device location determined in the determination step; and The display step displays the display information acquired in the acquisition step.

Images