Systems and methods for enriching system schematics using troubleshooting and repair information from the case handling

Abstract

A system (10) for use in medical device servicing includes a database (30, 31) storing medical device schematics (32) and annotations (33) to the medical device schematics; a search engine (14) configured to receive information (34) related to servicing of a medical device (12) and search the annotations to the medical device schematics in the database using a search query based on the received information to retrieve one or more medical device schematics in the database; and a navigation device (18) including a display (24) and at least one input device (22), the navigation device operatively connected to receive the one or more medical device schematics from the search engine and configured to display the one or more medical device schematics annotated with the annotations to the one or more medical device schematics.

Description

2023P00708WGSYSTEMS AND METHODS FOR ENRICHING SYSTEM SCHEMATICS USING TROUBLESHOOTING AND REPAIR INFORMATION FROM THE CASE HANDLINGFIELD

[0001] The following relates generally to the medical device maintenance arts, medical imaging device maintenance arts, medical device schematic arts, and related arts.BACKGROUND

[0002] As an Original Equipment Manufacturer (OEM), healthcare equipment vendors endeavor to provide timely after sales service to customers. As these devices are used for medical diagnosis, treatment, follow-up and the like in a hospital, medical clinic, or other medical facility, it is desirable to provide on time maintenance service with minimum down time. Vendors may have a serviceability and maintenance department to address issues or complaints related to medical equipment. In such serviceability and maintenance teams, there can be Remote Service Engineers (RSEs) responsible for receiving reports of issues with medical devices from the customer side and diagnosing and resolving (if possible) the problem remotely. The vendor may also employ Field Service Engineers (FSEs) who visit the site when an issue is not successfully diagnosed or resolved remotely. The FSE physically checks the equipment and resolves the issue.

[0003] Often, service engineers (SEs, such as RSEs or FSEs) need to consult service manuals while diagnosing and fixing the problem. These service manuals contain very complex schematic diagrams, which can often be hierarchical in nature) with lots of components, subcomponents, and connections (e.g., electrical, mechanical, etc.) among them. SEs can have difficulty in understanding and mapping the current issue to those complex schematics. This can lead to misdiagnosis of the issue, unsuccessful repair attempts, escalation to senior engineers, and / or repeated visits, all of which leads to longer downtime of the medical equipment causing halting of hospital operations, among others.

[0004] The following discloses certain improvements to overcome these problems and others.SUMMARY

[0005] In one aspect, a system for use in medical device servicing includes a database storing medical device schematics and annotations to the medical device schematics; a search2023PF00708 2 engine configured to receive information related to servicing of a medical device and search the annotations to the medical device schematics in the database using a search query based on the received information to retrieve one or more medical device schematics in the database; and a navigation device including a display and at least one input device, the navigation device operatively connected to receive the one or more medical device schematics from the search engine and configured to display the one or more medical device schematics annotated with the annotations to the one or more medical device schematics.

[0006] In another aspect, a non-transitory computer readable medium stores a database storing a plurality of medical device schematics for medical devices and annotations to the medical device schematics; and instructions executable by at least one electronic processor to perform a medical device service support method including receiving information describing servicing performed on a medical device; identifying one or more medical device schematics by searching the annotations based on the received information; and displaying the one or more medical device schematics annotated with the annotations to the medical device schematics.

[0007] In another aspect, a medical device servicing method includes receiving information related to servicing of a medical device; searching annotations the medical device schematics using a search query based on the received information to retrieve one or more medical device schematics from a database; receiving a new annotation to the one or more medical device schematics; and adding the new annotation to the database.

[0008] One advantage resides in providing a system that can intelligently read service manuals, and help SEs navigate through the diagrams interactively to diagnose the problem and then identify the right solution for a medical device undergoing servicing.

[0009] Another advantage resides in enhancing semantics of existing schematics using information collected from service engineers during the case handling process for a medical device undergoing servicing.

[0010] Another advantage resides in enriching system schematics by linking them hierarchically to information from service cases collected by service engineers.

[0011] Another advantage resides in enriching system schematics to allow locating more precisely parts of system schematics that are relevant for new service cases and are useful for engineers who may not have sufficient experience with a system or an issue.2023PF00708 3

[0012] A given embodiment may provide none, one, two, more, or all of the foregoing advantages, and / or may provide other advantages as will become apparent to one of ordinary skill in the art upon reading and understanding the present disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The disclosure may take form in various components and arrangements of components, and in various steps and arrangements of steps. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the disclosure.

[0014] FIGURE 1 diagrammatically illustrates a system for use in medical device servicing in accordance with the present disclosure.

[0015] FIGURE 2 diagrammatically illustrates an embodiment of a medical device maintenance guidance method using the apparatus of FIGURE 1.

[0016] FIGURE 3 diagrammatically illustrates an embodiment of modules of the medical device guidance apparatus of FIGURE 1.DETAILED DESCRIPTION

[0017] Schematics of medical devices are heavily utilized during service calls. As used herein, the term “schematic” (and variants thereof) refers to conventional schematics (e.g., electrical schematics, component layout schematics, technical drawings, etc.) as well as other image content contained in service manuals and other such resources for the servicing of medical devices, such as fault-finding trees, diagnostic flowcharts, or the like. In some examples, a preprocessing step is performed for extracting these schematics from the manual or like resource. For example, the preprocessing may include extraction of schematics from an electronic service manual or the like. The schematics may be in PDF format, although more generally schematics may be in other formats such as docx or various CAD drawing formats, bitmap image formats such as JPEG or PNG, a vector image format such as SVG, various combinations thereof, and / or so forth.

[0018] It will be appreciated that the number of schematics stored in a vendor database of a commercial medical device vendor is quite large, e.g., 100,000 schematics or more, with the number varying based on how they are counted. For example, there may be hierarchical schematics (overall system->sub-system->component levels, for instance), and multiple views may be provided for a given component or (sub)-system (e.g., a technical drawing may have front,2023PF00708 4 side, top, and perspective views, various sectional views, et cetera). Furthermore, since medical devices’ lifetime can be on the order of years or even decades, a vendor maintains drawings for numerous legacy products that are no longer being manufactured. In one nonlimiting illustrative example, the number of schematics may be 600,000 or higher.

[0019] Referring to the appropriate schematic or set of schematics can be very useful for resolving a service call. However, the schematics are not readily amenable to searching. Even if the images are processed by optical character recognition (OCR), the amount of text on drawings may be limited. It is not practically possible to manually annotate the schematics, both due to the massive size of the database and because the human annotator would need to be able to consider a very wide range of service call contexts, with a given schematic potentially being used very differently in different call contexts. Such contexts may also change over time. For example, the diagnosis and / or repair of a component of a medical device may be different depending on the particular make or model of medical device in which it is installed - hence, when a new make or model of medical device comes out for which the diagnosis or repair of the component is different, these differences could not be included in a priori-generated annotations of a schematic of the component. Moreover, resolution of a given service call may entail referring to a sequence of schematics in a specific order, for instance referring to a hierarchical set of schematics of a medical device in a sequence defined by following a certain branch of a fault-finding tree (which itself may be schematic). Resolution of a given service call may also entail referring to different sets of schematics based on the most likely fault, and / or may entail referring to a dynamic sequence of schematics (e.g., using a branching selection process based on fault finding techniques and initial information determined by triage of the service call).

[0020] To resolve such problems, it is disclosed herein to provide a system that leverages the service call reporting system as a platform for efficient annotation of schematics in the context of resolution of actual service calls. In one nonlimiting illustrative approach, a bootstrapper starts the process by identifying a set of potentially useful schematics based on information such as the malfunction area being addressed by the service call. The bootstrapper may leverage metadata about the schematics (e.g., source information) and may also leverage any already-generated schematic annotations.

[0021] A semantic enricher operates in conjunction with the service call reporting system to add annotations to schematics that are actually utilized during the service call. Such enrichment2023PF00708 5 annotations can, in some embodiments, be generated automatically or semi-automatically, for example by detecting reference to a schematic (or a component shown in a schematic) in the service call report being entered and adding corresponding report text to the schematic as an annotation. The enrichment annotations can also be generated semi-automatically by relying on the service engineer to select a schematic (or a part of a schematic) and cut / paste relevant report content on that schematic (or part thereof) as an annotation. For example, the use of standard language from service manuals helps with validation of the annotations. Annotations should not become part of the knowledge base unless (and until) they are validated. The use of the preapproved text and marking the location of the text can assist in the subsequent validation processes. The enricher also may generate links between schematics, This could be done automatically based on the order in which the service engineer displays the schematics, or semi-automatically based on the service engineer adding hyperlinks between schematics, which may be labeled as to their relationship (e.g., parent / child, or large view- zoomed view, or a union link indicating two schematics should be referenced together, etc.).

[0022] The output of the semantic enricher is a set of annotations that are associated to one or more schematics and / or parts of schematics. As each service call may in principle add more annotations to schematics, over time a large database of annotations is collected, which are directly related to usage of the schematics in actual service calls. This is done without performing OCR on the schematics, and with very limited (or in some cases no) additional workload being imposed on the service engineers. The annotations are also automatically contextualized based on the service call context, and can evolve over time (e.g., if the component at issue is installed in a new model medical device that creates new issues, then annotations relating to those issues with the new model medical device are created in association with that particular context).

[0023] In the usage phase, when a user (e.g., service engineer) brings up a particular schematic the annotations for that schematic are retrieved. These may be presented to the user as marginal notes (in one approach, displaying only an introductory portion of a given annotation to save space), or as a semi-transparent overlay on the schematic, or in a separate window shown on the display, or so forth. The user can in some embodiments adjust the presentation (e.g., turn annotations display on / off or change the transparency). The user can select a particular annotation to see the full text (if only a portion is initially displayed), can follow annotated links between2023PF00708 6 schematics, and / or so forth. The user can also add additional annotations via the semantic enricher, thus further expanding the annotations database with each service call.

[0024] In a search scenario, when the user performs a search on the archived service reports database the search query can also be applied against the annotations database and any schematics with annotations matching the query are then included in the search results. This beneficially allows for searching of the schematics without performing OCR to extract text content (if any) of the schematics, and provides more semantically rich searching of the schematics than would be provided by OCR-generated text. In another scenario, the annotations can be used to generate additional search terms. In yet another scenario, annotations which include log events (for instance, an annotation: “Error [xxx] is generated by a failure of component [yyy] shown in region A-9 of Schematic [zzz]” can be linked to instances of log events shown in a log viewer (e.g., if a medical device log for a medical device under service includes log event “Error [xxx]” then this can be annotated with a link to region A-9 of Schematic [zzz]). To do so, the linking of annotations to log events can be performed by outputting one or more predicted search terms generated using a process (e.g., a trained machine learning) configured to predict, with one or more search terms, annotation pairs including a historical list of search terms and corresponding annotations. The search terms are based on an input annotation.

[0025] With reference to FIGURE 1, an illustrative system or apparatus 10 for guidance of maintenance of a medical device 12 is shown. The medical device 12, for example, can comprise an illustrative medical imaging device 12 (also referred to as a medical device, an imaging device, imaging scanner, and variants thereof) which can be a magnetic resonance imaging (MRI) scanner, a computed tomography (CT) scanner, a positron emission tomography (PET) scanner, a gamma camera for performing single-photon emission computed tomography (SPECT), an interventional radiology (IR) device, an X-ray device, an image-guided therapy (IGT) device, an ultrasound (US) device, or so forth. Although described herein as an imaging device, the medical device 12 can also be any other suitable medical device that is used with patients to perform medical functions such as diagnosis and / or treatment, such as a patient monitor, a radiation therapy device, a mechanical ventilator, and so forth.

[0026] A navigation device comprises an electronic processing device 18, such as a workstation computer, or more generally a computer, a smart device (e.g., a cellular telephone (“cell phone”), a smart tablet, and so forth), is operable by a service engineer (SE). The electronic2023PF00708 7 processing device 18 may also include a server computer or a plurality of server computers, e.g., interconnected to form a server cluster, cloud computing resource, or so forth, to perform more complex computational tasks. The electronic processing device 18 includes typical components, such as an electronic processor 20 (e.g., a microprocessor), at least one user input device (e.g., a mouse, a keyboard, a trackball, and / or the like) 22, and a display device 24 (e.g., an LCD display, plasma display, cathode ray tube display, and / or so forth). In some embodiments, the display device 24 can be a separate component from the electronic processing device 18, or may include two or more display devices.

[0027] The electronic processor 20 is operatively connected with one or more non- transitory storage media 26. The non-transitory storage media 26 may, by way of non-limiting illustrative example, include one or more of a magnetic disk, RAID, or other magnetic storage medium; a solid-state drive, flash drive, electronically erasable read-only memory (EEROM) or other electronic memory; an optical disk or other optical storage; various combinations thereof; or so forth; and may be for example a network storage, an internal hard drive of the workstation 18, various combinations thereof, or so forth. It is to be understood that any reference to a non- transitory medium or media 26 herein is to be broadly construed as encompassing a single medium or multiple media of the same or different types. Likewise, the electronic processor 20 may be embodied as a single electronic processor or as two or more electronic processors. The non- transitory storage media 26 stores instructions executable by the at least one electronic processor 20. The instructions include instructions to generate a visualization of a graphical user interface (GUI) 28 for display on the display device 24.

[0028] The navigation device 18 is also in communication with one or more databases 30,31 (shown in FIGURE 1 as a server computer). In some embodiments, a first database 30 stores medical device schematics 32, for example, for the medical device 12, while a second database 31 stores enrichment content for 33 the medical device schematics 32. The medical device schematics32 can comprise technical drawings, electrical schematics, component layout schematics, diagnostic flow charts, or any other suitable format.

[0029] It will be appreciated that the number of medical device schematics 32 in the first database 30 may be large, e.g., tens of thousands of cases, hundreds of thousands of cases, or more, and a search can in many cases return a few tens of thousands of specifications 32 (or more). For example, different OEM specifications may be stored for different models or configurations of the2023PF00708 8 same medical device, different OEM specifications may be stored for different regulatory jurisdictions, and / or so forth. Stored schematics may also relate to components of medical devices, with some such components being installed (or installable) in two or more different makes and / or models of medical device. For example, an X-ray detector array might be installed (or installable) in two or more different makes and / or models of CT scanner. It will be appreciated that the OEM specifications 32 may be stored in a variety of representations, such as PDF, docx, various CAD drawing formats, bitmap image formats such as JPEG or PNG, vector image formats such as SVG, various combinations thereof, and / or so forth. The medical device schematics 32 can include, at a minimum, 100,000 medical device schematics 32, and as much as 600,000 medical device schematics 32, or more. Advantageously, in some nonlimiting illustrative embodiments the first database 30 and the second database 31 do not store textual content extracted from the medical device schematics by OCR.

[0030] The second database 31 stores the annotations 33 to the medical device schematics 32. The annotations 33 can take various forms, such as textual annotations, links, and so forth, with annotations optionally tagged to particular sections or parts of a schematic. In some embodiments, the annotations 33 to the medical device schematics 32 include a link 35 (shown in FIGURE 1 as a dashed arrow in a box) annotated to a first medical device schematic 32 linking the first medical device schematic 32 to a second medical device schematic 32. For example, the annotations 33 to the medical device schematics 32 include a sequence of directed links 35 annotated to respective medical device schematics 32 defining an ordered sequence of the medical device schematics 32.

[0031] The first database 30 and the second database 31 are accessible by or implemented in a search engine 14 comprising a server computer 14. The server computer 14 also has includes or has access to medical device maintenance information 34 about the maintenance performed on the medical device 12 (and, typically, about maintenance performed on other medical devices in the hospital, radiology laboratory, or so forth). In some contexts, this access may extend beyond a single hospital or laboratory. For example, the databases 30 and 32 in some contemplated embodiments are maintained by the medical device vendor and accessible by all customers who have purchased, leased, or otherwise have medical devices from that vendor, and the annotations 33 in the second database 31 may be from all such customers. The medical device maintenance information 34 may, for example, include or be derived from a purchase record of components2023PF00708 9 purchased for installation in the medical device 12 (for example, the purchase record may constitute or be extracted from a parts ordering system, parts inventory, or the like), a service record of the medical device 12, and / or the like. The search engine 14 is configured to receive the information 34 related to servicing of a medical device 12 and search the annotations 33 to the medical device schematics 32 in the first database 30 using a search query based on the received information 34 to retrieve one or more medical device schematics 32 in the first database 30.

[0032] The apparatus 10 is configured as described above to perform a medical device servicing assistance method or process 100 including providing a schematics server with enrichment content (i.e., annotations) compiled from service calls over time as disclosed herein. The non-transitory storage medium 26 stores instructions which are readable and executable by the at least one electronic processor 20 to perform disclosed operations including performing the medical device servicing method or process 100. In some examples, the method 100 may be performed at least in part by cloud processing.

[0033] With reference to FIGURE 2, and with continuing reference to FIGURE 1, an illustrative embodiment of an instance of the medical device servicing assistance method 100 is diagrammatically shown as a flowchart. At an operation 102, the search engine 14 is configured to receive the information 34 related to servicing of a medical device 12 or performed on the medical device 12.

[0034] At an operation 104, one or more medical device schematics 32 are identified by searching the annotations based on the received information. To do so, the search engine 14 is configured to search the annotations 33 in the second database 31 to the medical device schematics 32 in the first database 30 using a search query based on the received information 34 to retrieve one or more medical device schematics 32 in the first database 30.

[0035] At an optional operation 105, the navigation device 18 is configured to receive a new annotation 33 to the one or more medical device schematics 32. To do so, a user operates the user input device(s) 22 (e.g., with keystrokes, mouse clicks, etc.) to provide the new annotation 33 to the one or more medical device schematics 32. The search engine 14 then saves the new annotation 33 in the second database 31.

[0036] To perform the new annotation operation 105, a service report 36 (which can be stored in a service report database 36 implemented in the search engine 14) is retrieved, or otherwise received, at the navigation device 18 via the user input device(s) 22, and is displayed on2023PF00708 10 the display device 24. For example, reference to at least a portion of the medical device schematic 32 in report text of the retrieved service report 36 prepared during the servicing is detected, and the corresponding report text is added to the schematic medical device schematic 32 as a new annotation 33 to the medical device schematic 32. In some embodiments, the new annotation is not added to the medical device schematic 32 unless and until it is first validated. The validation process may be automated, or may employ a manual moderation of new annotations 33. In some nonlimiting illustrative examples of automated validation approaches, the new annotation 33 may be validated if it matches pre-approved text (which, for example, can be stored as a list of preapproved annotation words / phrases). To account for slight variations in wording or spelling, an automated validation may identify the closes words / phrases to the user-input annotation 33 and propose them to the user for selection. The automated validation may employ other approaches, such as applying spell-checking (or the like) to the annotation 33, again proposing corrections to the user. In embodiments employing manual validation, the annotation received at operation 105 may be displayed for the servicing of the medical device 12 currently undergoing servicing, but may not be added to the second database 31 which stores the enrichment content for the medical device schematic 32 unless and until it is approved by a human reviewer. In another example, the new annotation 33 is received by selection of the new annotation 33 from the service report 36 via the at least one user input device 22. To do so, an input indicative of a selection of at least a portion of the schematic 32 used during the servicing is received via the user input device(s) 22, and corresponding report text from the service report 36 to the schematic 32 as a new annotation 33. In another embodiment, log data 37 generated by the medical device 12 receiving servicing is received at the navigation device 18, and the new annotation 33 is received by selection of a portion of the received log data 37 via the at least one user input device 22. To do so, the linking of annotations to log events in the log data 37 can be performed by outputting one or more predicted search terms generated using a process (e.g. a trained machine learning) configured to predict, with one or more search terms, annotation pairs including a historical list of search terms and corresponding annotations. The search terms are based on an input annotation.

[0037] At an operation 106, the navigation device 18 receives the one or more medical device schematics from the search engine and configured to display the one or more medical device schematics annotated with the annotations to the one or more medical device schematics. This can be performed in a variety of manners. In one embodiment, the one or more medical2023PF00708 11 device schematics 32 with the annotations 33 to the one or more medical device schematics are displayed by overlaying text and / or a graphical representation of at least one annotation on the display 24 of the corresponding medical device schematic 32. In another embodiment, the corresponding annotations 33 are displayed in a margin of the displayed medical device schematic 32. In another embodiment, the navigation device 18 displays a first medical device schematic 32 annotated with the link 35, and receives a selection of the link 35 via the at least one user input device 22 causing the navigation device 18 to display a second medical device schematic 32. In another embodiment, the navigation device 18 displays the respective medical device schematics 32 annotated with the directed links 35, and receives a selection of the directed link 35 annotated to a currently displayed medical device schematic 32 of the ordered sequence of the medical device schematics 32 causing the navigation device 18 to display the next medical device schematic 32 in the ordered sequence of the medical device schematics 32. These are merely illustrative examples and should not be construed as limiting.

[0038] As also indicated in FIGURE 2, the new annotation received at the operation 105 may also constitute an additional input to the operation 104, which may update the selection of schematics for display in the operation 106 based on the new annotation. This facilitates dynamic updating of the schematics displayed at the operation 106. For example, the new annotation 33 received at the operation 105 can be added as a new search term and the search engine 14 is invoked with the new search term to dynamically update the identified schematics 32. In similar fashion, the operation 102 may receive updated information based on triage of the servicing of the medical device 12 currently undergoing servicing, and the search engine 14 is invoked with this new information to dynamically update the identified schematics 32. For example, the triaging may include running one or more diagnostic tests on the medical device 12 currently undergoing servicing, and the results of these diagnostic tests (or information derived from these results) may constitute new / additional information that is used to dynamically update the identified schematics.EXAMPLE

[0039] The following describes the system 10 and the method 100 in more detail. The system 10 is configured to enrich system schematics by linking them hierarchically to information from service cases collected by service engineers throughout the entire company. The information from service cases allows locating more precisely parts of system schematics that are relevant for2023PF00708 12 new service cases and are useful for engineers who may not have sufficient experience with a system or an issue.

[0040] FIGURE 3 shows another example of the system 10. As shown in FIGURE 3, the system 10 includes a bootstrapping module 40 implemented in the search engine 14. The bootstrapping module 40 is configured to receive a description of a customer issue as collected by the remote service engineer at the call center, and classifies the received description into a malfunction area. The malfunction area is then used to find possibly relevant system schematics32 based on any already available annotations 33 associated to the schematics 32. Older system schematics 32 can have richer metadata associated to it, due to more annotations 33 related to the older schematics 32 accumulating over time as the older schematic is references by numerous service calls.

[0041] A schematics enricher 44 adds the annotations 33 to the relevant schematic(s) 32 to create the schematic(s) enriched with annotations 42. When a service engineer identifies the correct schematics 32 while handling a service case, he / she views the identified schematic using a schematics navigator 46 (i.e., the GUI 28) implemented at the electronic processing device 18 by displaying the enriched schematic with the enrichment annotations on the GUI 28 shown on the display 24. Optionally, the service engineer (further) annotates the schematic with relevant diagnostics and repair information including the problem description, the malfunction area, any relevant log events, the service actions, and the parts consumed. These comprise new annotations33 that are added to the annotations database 31 by the schematic enricher module 44. The new annotations may be generated manually, automatically, or semi-automatically. In some embodiments, the new annotations 33 may be required to be validated before being added to the annotations database 31. For example, the use of standard language from service manuals or the schematics 32 helps with a validation process of the schematics 32 (e.g., the new annotation 33 may be validated if it matches a standard annotation word or phrase in a standard annotation language database). The new annotations 33 should be stored in annotations database 31, but in some embodiments not until they are validated. Only those annotations 33 that have been validated should be processed for future use. Records are maintained regarding the validation process and optionally stored in the annotations database 31. In some embodiments, the unvalidated data is made available to a select number of service engineers, such as specialists, that have the training and understanding on use of such unvalidated data. The use of the pre-approved text in the2023PF00708 13 schematics 32 and marking the location of the text can assist in the subsequent validation processes. In some embodiments, the annotations 33 stored in annotations database 31 can be dynamically updated (i.e., in real-time) responsive to receipt of new or additional log data 37 and / or actions taken by service engineer.

[0042] In some embodiments, the schematics navigator 46 allows the service engineer to select on the display (i.e., screen) 24 parts of the schematics 32 and the corresponding service information. While doing so, the engineer can also provide a link 35 to multiple schematics 32 to the same service information. The service engineer can also link schematics 32 with each other and specify the type of relationship such as hierarchical (parent / child, zoom in / out), sequential (preceding / following in the troubleshooting or repair procedure), union (need to combine the schematics together for troubleshooting in case of a dependency, such as signals exiting one component and entering another). These operations by the service engineer define new annotations for the schematic(s). The enriched schematics 42 are now searchable by text because of the added annotations 33 to the schematics 32.

[0043] The schematic navigator module 46 also allows the service engineer to search through the enriched schematics 42 focusing on solving the service case. The search can be purely text-based or based on conditions on the various annotations 33 that have been added to the schematics 32. The search is now capable of retrieving more relevant schematics 32 but also the parts of schematics 32 that are relevant as well as the relationship among them. In another example, the results of the search are schematics 32 and parts of them with overlaid information (such as error messages, troubleshooting actions, etc.) and links 35 among schematics 32.

[0044] Service engineers can, in some embodiments, click on parts of the enriched schematic 42 when displayed on the display 24, which triggers search for cases related to that part of the diagram as well previously “associated semantics”. Clicking on other parts can be used to add keywords for the search (AND / OR) or exclude results from the search. The enriched schematics 42 can also in some embodiments be embedded in diagnostic tools such as log viewers so that the semantic information (i.e., annotations) that is associated to the schematics can be linked to, for example, log events, thereby creating further new annotations that are stored in the annotations database 31 for use by service engineers in subsequent service cases. The search results also in some embodiments influence the colors of the parts of the diagram and generate hyperlinks 35 from the current diagram to other diagrams to click through elements of diagrams2023PF00708 14 to see more details. The (optionally hierarchical) relationships between schematics extracted from technical documents can be learned by using NLP or established by user navigation. The enriched schematics 42, which include relevant information for diagnostics and repair, can help service engineers find the service solution more effectively. For example, service engineers can examine and reuse the solution from the historical service cases that are associated to the current schematic 32.

[0045] The disclosure has been described with reference to the preferred embodiments. Modifications and alterations may occur to others upon reading and understanding the preceding detailed description. It is intended that the exemplary embodiment be constructed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

2023PF00708 15CLAIMS:

1. A system (10) for use in medical device servicing, the system comprising: a database (30, 31) storing medical device schematics (32) and annotations (33) to the medical device schematics; a search engine (14) configured to receive information (34) related to servicing of a medical device (12) and search the annotations to the medical device schematics in the database using a search query based on the received information to retrieve one or more medical device schematics in the database; and a navigation device (18) including a display (24) and at least one input device (22), the navigation device operatively connected to receive the one or more medical device schematics from the search engine and configured to display the one or more medical device schematics annotated with the annotations to the one or more medical device schematics.

2. The system (10) of claim 1, wherein the navigation device (18) is further configured to: receive a new annotation (33) to the one or more medical device schematics (32) via the at least one input device (22) and add the new annotation to the database (30, 31).

3. The system (10) of claim 2, wherein the navigation device (18) is further configured to: receive, via the at least one user input device (22), a service report (36) on a medical device and display the service report on the display (24); wherein the new annotation (33) is received by selection of the new annotation from the service report via the at least one user input device.

4. The system (10) of claim 2, wherein the navigation device (18) is further configured to: receive log data (37) generated by a medical device (12) receiving servicing; wherein the new annotation (33) is received by selection of a portion of the received log data via the at least one user input device (22).2023PF00708 165. The system (10) of claim 4, wherein the navigation device (18) is further configured to: generate one or more search terms using the received log data (37); and generate the new annotation (33) using the generated one or more search terms.

6. The system (10) of any one of claims 1-5, wherein the navigation device (18) is configured to: display the one or more medical device schematics (32) with the annotations (33) to the one or more medical device schematics by overlaying text and / or a graphical representation of at least one annotation on the display (24) of the corresponding medical device schematic.

7. The system (10) of any one of claims 1-5, wherein: the annotations (33) to the medical device schematics (32) include a link (35) annotated to a first medical device schematic linking the first medical device schematic to a second medical device schematic, and the navigation device (18) is configured to display the first medical device schematic annotated with the link and to receive a selection of the link via the at least one user input device (22) causing the navigation device to display the second medical device schematic.

8. The system (10) of any one of claims 1-6, wherein: the annotations (33) to the medical device schematics (32) include a sequence of directed links (35) annotated to respective medical device schematics defining an ordered sequence of the medical device schematics; and the navigation device (18) is configured to display the respective medical device schematics annotated with the directed links and to receive a selection of the directed link annotated to a currently displayed medical device schematic of the ordered sequence of the medical device schematics causing the navigation device to display the next medical device schematic in the ordered sequence of the medical device schematics.

9. The system (10) of any one of claims 1-8, wherein the medical device schematics (32) stored in the database (30, 31) include technical drawings, electrical schematics, component layout schematics, and diagnostic flow charts.2023PF00708 1710. The system (10) of any one of claims 1-9, wherein the navigation device (18) is configured to: validate new annotations (33) using the medical device schematics (32); and store the new annotations in the database (31).

11. A non-transitory computer readable medium (14, 26) storing: a database (30, 31) storing a plurality of medical device schematics (32) for medical devices and annotations (33) to the medical device schematics; and instructions executable by at least one electronic processor (20) to perform a medical device service support method (100) including: receiving information (34) describing servicing performed on a medical device (12); identifying one or more medical device schematics by searching the annotations based on the received information; and displaying the one or more medical device schematics annotated with the annotations to the medical device schematics.

12. The non-transitory computer readable medium (14, 26) of claim 11, wherein the method (100) further includes: detecting reference to at least a portion of the medical device schematic (32) in report text of a service report (36) prepared during the servicing; and adding the corresponding report text to the schematic as a new annotation (33) to the medical device schematic.

13. The non-transitory computer readable medium (14, 26) of claim 11 , wherein the adding includes: receiving, via at least one user input device (22), an input indicative of a selection of at least a portion of the medical device schematic (32) used during the servicing; and adding corresponding report text from a service report (36) to the schematic as an annotation (33).2023PF00708 1814. The non- transitory computer readable medium (14, 26) of any one of claims 11-13, wherein: the annotations (33) to the medical device schematics (32) include a link (35) annotated to a first medical device schematic linking the first medical device schematic to a second medical device schematic, and the method (100) further includes displaying the first medical device schematic annotated with the link and to receive a selection of the link via the at least one user input device (22) causing the navigation device to display the second medical device schematic.

15. The non- transitory computer readable medium (14, 26) of any one of claims 11-14, wherein: the annotations (33) to the medical device schematics (32) include a sequence of directed links (35) annotated to respective medical device schematics defining an ordered sequence of the medical device schematics; and the method (100) further includes displaying the respective medical device schematics annotated with the directed links and to receive a selection of the directed link annotated to a currently displayed medical device schematic of the ordered sequence of the medical device schematics causing the navigation device to display the next medical device schematic in the ordered sequence of the medical device schematics.

16. The non- transitory computer readable medium (14, 26) of any one of claims 11-14, wherein: the annotations (33) to the medical device schematics (32) include a sequence of directed links (35) annotated to respective medical device schematics defining an ordered sequence of the medical device schematics; and the navigation device (18) is configured to display the respective medical device schematics annotated with the directed links and to receive a selection of the directed link annotated to a currently displayed medical device schematic of the ordered sequence of the medical device schematics causing the navigation device to display the next medical device schematic in the ordered sequence of the medical device schematics.2023PF00708 1917. The non-transitory computer readable medium (26) of any one of claims 11-16, wherein the displaying includes: displaying the corresponding annotation (33) in a margin of the displayed schematic (32).

18. The non-transitory computer readable medium (26) of any one of claims 11-17, wherein the method (100) further includes: generate one or more search terms using log data (37); and generate the new annotation (33) using the generated one or more search terms.

19. The non-transitory computer readable medium (26) of any one of claims 11-18, wherein the method (100) further includes: validating new annotations (33) using the medical device schematics (32); and storing the new annotations in the database (31).

20. A medical device servicing method (100), comprising: receiving information (34) related to servicing of a medical device (12); searching annotations (33) the medical device schematics (32) using a search query based on the received information to retrieve one or more medical device schematics from a database (30, 31); receiving a new annotation to the one or more medical device schematics; and adding the new annotation to the database.

Images