Intelligent detection and maintenance system and method for imaging modalities

Abstract

This invention discloses a method, apparatus, system, and article of manufacture for providing intelligent discovery of image modality maintenance. The exemplary method includes identifying an imaging device based on an image of the imaging device. The method further includes determining at least one of a brand, model, or modality of the imaging device based on the identification. The method also includes identifying a group corresponding to the imaging device. The method further includes storing error information corresponding to the imaging device and the group of imaging devices, and updating the model corresponding to the group based on the error information. The method also includes deploying the model to a technician's device.

Description

Technical Field

[0001] This disclosure relates generally to imaging systems, and more specifically, to an image modality intelligent discovery and maintenance system and method. Background Technology

[0002] Manufacturers of large machines (e.g., imaging machines in healthcare, turbines in energy, and engines in transportation) deploy such machines to users / customers for field use. Due to the complexity of such machines, some manufacturers provide repair and / or maintenance services through teams of technicians to service the machines during scheduled maintenance and / or when they fail and / or shut down. When a user has a problem with the deployed machine, the user (e.g., via call, email, etc.) contacts the manufacturer describing the problem (e.g., providing symptoms), and a technician is then dispatched to repair the machine. Additionally, manufacturers and / or customers may schedule maintenance calls at set intervals to verify that the machine is functioning correctly. Summary of the Invention

[0003] Some examples provide a computer-readable medium including instructions that, when executed, cause a machine to provide intelligent scheduling for image modality maintenance. Exemplary computer-readable medium includes instructions that cause a machine to identify an imaging device based on at least one of contextual information or an imaging device identifier. Exemplary computer-readable medium includes instructions that cause a machine to determine at least one of a brand, model, or modality of an imaging device. Exemplary computer-readable medium includes instructions that cause a machine to store error information corresponding to a problem associated with a group of imaging devices, and to update the model corresponding to the group based on the error information, when a group of imaging devices corresponding to at least one of a brand, model, or modality exists. Exemplary computer-readable medium includes instructions that cause a machine to deploy a model to a technician's device. Exemplary computer-readable medium includes instructions that cause a machine to flag a problem and transmit a warning corresponding to the flag when the number of times a problem has been identified in a group exceeds a threshold.

[0004] Some examples provide an apparatus for providing intelligent scheduling of image modality maintenance. The example apparatus includes a machine identifier to identify an imaging device based on contextual information of the imaging device, determine at least one of the brand, model, or modality of the imaging device based on the identification, generate a new group corresponding to the brand, model, or modality of the imaging device when no group corresponding to the at least one of the brand, model, or modality does not exist, and store error information corresponding to a problem with the imaging device in relation to that group. The apparatus also includes a model generator to generate a digital twin corresponding to at least one of the brand, model, modality, or error information of the imaging device. The apparatus also includes an interface for deploying the model to a technician's device. The apparatus also includes a machine identifier to flag a problem when the number of times a problem has been identified in the group exceeds a threshold. The apparatus also includes an interface for transmitting a warning corresponding to the flag.

[0005] Some examples provide a method for providing intelligent scheduling of image modality maintenance. This exemplary method includes identifying an imaging device based on images from that device. The method also includes determining at least one of the brand, model, or modality of the imaging device based on the identification. The method further includes identifying a group corresponding to the imaging device. The method also includes storing error information corresponding to the imaging device and the group of imaging devices, and updating the model corresponding to the group based on the error information. The method also includes deploying the model to the technician's equipment. Attached Figure Description

[0006] Figure 1A and / or Figure 1B An exemplary environment is shown, including a machine recovery system for serving the entire machine domain.

[0007] Figure 2 yes Figure 1A and / or Figure 1B A block diagram of an exemplary implementation of a machine recovery system.

[0008] Figure 3 This is an example representation of a digital twin.

[0009] Figures 4A to 4C An exemplary device is depicted to aid in the diagnosis and repair of equipment.

[0010] Figure 5 This is a representation of an exemplary deep learning neural network model.

[0011] Figure 6 This indicates that it can be executed to achieve [the desired result]. Figure 1A and / or Figure 1B The machine recovery system processes data from... Figure 1A and / or Figure 1BA flowchart of machine-readable instructions for a service request from a machine.

[0012] Figure 7 This indicates that it can be executed to achieve [the desired result]. Figure 1A and / or Figure 1B A flowchart of machine-readable instructions executed by the machine recovery system to process intelligent symptoms.

[0013] Figure 8 This indicates that it can be executed to achieve [the desired result]. Figure 1A and / or Figure 1B A flowchart of machine-readable instructions used by the machine recovery system to execute the intelligent maintenance package.

[0014] Figure 9 This indicates that it can be executed to achieve [the desired result]. Figure 1A and / or Figure 1B The flowchart shows the machine-readable instructions executed by the machine recovery system for intelligent scheduling.

[0015] Figure 10 This indicates that it can be executed to achieve [the desired result]. Figure 1A and / or Figure 1B The flowchart shows the machine recovery system executing machine-readable instructions for intelligent discovery.

[0016] Figure 11 It is constructed to execute Figures 6 to 10 Instructions to be implemented Figure 1A and / or Figure 1B A block diagram of an exemplary processing platform for a machine recovery system.

[0017] The accompanying drawings are not to scale. Generally, the same reference numerals will be used in all the drawings and the accompanying written description to refer to the same or similar parts. Detailed Implementation

[0018] Machine clusters, including but not limited to imaging systems, turbines, and engines, are increasingly being deployed over large geographic areas. In the medical field, imaging systems of various modalities, such as magnetic resonance imaging (MRI), computed tomography (CT), nuclear imaging, and ultrasound, are increasingly being deployed in hospitals, clinics, and medical research institutions to perform medical imaging on subjects. Engines deployed in locomotives or aircraft need to operate under diverse environmental conditions. In power generation systems, wind turbines or hydro turbines are installed to extract energy from natural resources. Manufacturers owning machines within machine clusters aim to maximize machine utilization with minimal downtime. However, system failures and outages can disrupt workflow processes involving the machines and reduce their utilization.

[0019] Most manufacturers strive to provide effective periodic maintenance routines and response or call-to-repair services. While preventative maintenance procedures are generally well-developed, machines can sometimes develop problems requiring periodic diagnosis and repair. Typically, such problems are identified by the relevant authority within the manufacturer that manages the machine. The identified problem is submitted as a service request in one or more formats, such as, but not limited to, a text description via a webpage and a voice call via a service hotline. As used herein, the term "service request" refers to a description of a problem, malfunction, or issue associated with a machine, such as an imaging system. The problem, malfunction, or issue may be observed, for example, by a technician or user during routine maintenance checks or during machine use. Service requests can be descriptions provided by the user via a text or audio message through a user interface and can be automatically stored in a database.

[0020] Traditionally, service for machines in a machine group, such as imaging systems, may require component replacement or on-site inspections by a field engineer. Such on-site inspections can be expensive and time-consuming for both the customer and the system manufacturer or repair shop that typically schedules them. Remote diagnostics and repair are often used to expedite system repairs and eliminate or minimize the need for such on-site inspections. However, existing remote diagnostics and repair still require interrupting the use of the imaging system and contacting a repair shop. Additionally, when using remote diagnostics to identify faults, manual intervention may be required to submit a service request, initiate service request processing, and identify the need for an on-site inspection. Traditionally, experts need to manually scan large amounts of data related to the service request to make and / or recommend service options based on the request. Manually processing service requests is inefficient and negatively impacts response time. The goal is to reduce manual overhead when processing service requests without compromising accuracy and response time.

[0021] The examples disclosed herein provide systems for maintaining machines that solve conventional technical problems. The examples disclosed herein efficiently process service requests from machines belonging to a group of machines, such as healthcare imaging systems, with little or no inherent workflow latency. The examples disclosed herein include (A) systems that translate customer-defined symptoms of a machine into actual problems for the machine (e.g., intelligent symptoms), (B) systems that develop maintenance packages including target information and related information that can be used to service the machine to provide to resources (e.g., technicians, automation programs, etc.) (e.g., intelligent maintenance packages), (C) systems that determine which technicians are best suited to serve the request based on customer needs (e.g., intelligent scheduling), (D) systems that identify machine groups corresponding to identified machines to use and / or develop artificial intelligence models (e.g., deep or machine learning models / digital twins) corresponding to those machines (e.g., intelligent discovery), and (E) systems that use technician feedback on service requests to update and fine-tune intelligent symptoms, intelligent maintenance packages, intelligent scheduling, and intelligent discovery processes.

[0022] Figure 1A This is a schematic diagram of an environment 100 used to serve machines from across the machine domain. Environment 100 includes a machine recovery system 102, machines 104, a service center 106, and technician equipment 108. Although environment 100 includes a single machine 104, it is described in combination with multiple machines located in the same and / or different locations.

[0023] Figure 1AMachine recovery system 102 responds to service requests from machine 104 by performing various processes and building knowledge based on previous service requests. Initially, when a problem occurs, machine recovery system 102 receives service requests from service center 106 and / or directly from machine 104. For example, a user of machine 104 may call and / or machine 104 may otherwise communicate with service center 106 to identify the machine's problem and / or describe the symptoms of machine 104. In another example, a user may interact with the user interface of machine 104 or the user interface of a computer to request service and / or identify the symptoms of machine 104. Machine recovery system 102 may perform intelligent symptom processes, intelligent maintenance package processes, intelligent scheduling processes, intelligent discovery processes, and / or update data based on service requests from machine 104 and other machines in the domain. Intelligent symptom processes translate identified symptoms into machine problems. For example, a user of machine 104 may not be able to determine the actual problem that machine 104 is experiencing, but the user may identify symptoms that can be translated into one or more problems. The intelligent maintenance package provides technicians with customized, relevant, and actionable information for equipment 108. This customized, relevant, and actionable information corresponds to relevant manual / tutorial information, machine information, necessary tools, replacement parts, artificial intelligence model (e.g., deep or machine learning network model / digital twin, etc.) information, predicted service problems, and / or solutions. The machine recovery system 102 generates the intelligent maintenance package based on database information corresponding to previous services of machine 104 and / or other machines. The intelligent scheduling process determines which resource (e.g., technician, automated repair procedure, etc.) to select to service the request based on various factors (including, for example, distance to location, technician skill level, technician availability, etc.). The intelligent discovery system identifies machine groups corresponding to a specific machine 104 based on its model, modality, brand, and / or contextual information. A group corresponds to a set of similar machines that may experience the same problems. Therefore, the group corresponds to an artificial intelligence model / digital twin that can be used to virtually test solutions during / before service. Additionally, information about the service request from machine 104 can be added to the machine's model / digital twin to update the model / digital twin for subsequent service requests from other machines in the group. Furthermore, machine recovery system 102 updates information in various databases (e.g., enterprise systems, problem solution database (PSDB), machine log database, and / or dynamic system health status database) to refine intelligent symptoms, intelligent maintenance packages, intelligent scheduling, and intelligent discovery processes for subsequent service requests from machine 104 and other machines in the global domain. Machine recovery system 102 is further described below in conjunction with... Figure 2 Further description.

[0024] Figure 1AMachine 104 is an imaging device or system that may require periodic (e.g., schedule-based) or non-periodic (e.g., problem-based) service. Alternatively, machine 104 can be any type of machine that may require periodic or non-periodic service. For example, machine 104 can be an imaging system (e.g., magnetic resonance imaging (MRI), computed tomography (CT), X-ray, nuclear imaging, or ultrasound), a turbine (e.g., a wind turbine, a water turbine, etc.), an engine (e.g., a locomotive engine, an aircraft engine, etc.), an information system (e.g., an imaging workstation, an image archiving and communication system, a radiological information system, an image archive, an electronic medical record system, etc.). When machine 104 encounters a problem, the user can transmit a service request via a computer, via telephone (e.g., by calling or sending a text message), and / or machine 104 itself can transmit a service request in the event of an error or unexpected result. Service requests can be sent to service center 106 and / or directly to machine recovery system 102. In some examples, keywords from service requests can be extracted and processed to identify symptoms and / or problems of machine 104. In some examples, machine recovery system 102 can transmit differentiated symptoms (e.g., directly or via service center 106) to the user of machine 104 to filter out potential problems.

[0025] Figure 1A The technician equipment 108 is a computing device (e.g., laptop, smartphone, tablet, headset, etc.) that technicians and / or other resources can use to manage schedules, receive service requests, locate information related to one or more machines to be serviced, identify error codes, symptoms, problems and / or solutions, model machine 104 (e.g., using an artificial intelligence model) to aid in repair, and / or interact with imaging equipment to obtain additional and / or confirmatory data. The following is in conjunction with... Figure 3 and / or Figure 5 A specific implementation of the exemplary artificial intelligence model is further described. Machine recovery system 102 provides a customized maintenance package that includes relevant information that can assist technicians or other resources in providing services to requests. The maintenance package may include one or more diagnostic and / or remediation applications, knowledge base information, tools, scripts, routines, workflows, databases, artificial intelligence models (e.g., digital twins), etc., to, for example, diagnose, remediate, and / or assist resources (e.g., technicians, automated service repair procedures, etc.) and / or machine 104 in resolving problems.

[0026] When the technician device 108 of Figure 1 has access to the digital twin, the technician device 108 (e.g., automatically) and / or the user of the technician device 108 can implement different service technologies on the digital twin (e.g., perform virtual actions, execute virtual instructions, etc.) to identify potential problems that may occur during repair and / or identify unexpected errors that do occur during repair. Additionally, after the resource repairs machine 104, the resource can instruct the technician device 108 to complete a service request. Following such an instruction, the machine recovery system 102 transmits an investigation to the technician device 108 to obtain information corresponding to the service request and / or maintenance package. Alternatively or additionally, once the service is completed, the technician device 108 can automatically output an investigation to the technician and transmit the results to the machine recovery system 102. In some examples, the technician device 108 can interact with machine 104 (e.g., directly or via wireless communication) to collect data before, during, and / or after machine 104 has been serviced. Such data can be transmitted to the machine recovery system 102 (e.g., separately and / or as part of a feedback response). The artificial intelligence model of the machine recovery system 102 can use survey results (e.g., feedback) and / or acquired data to optimize future maintenance packages, scheduling, machine discovery, diagnosis (e.g., intelligent symptom processes), etc., to update the database, thereby providing better recommendations for subsequent service requests and / or identifying future problems with similar machines. (See attached...) Figures 4A to 4C A specific implementation of the exemplary technician device 108 is further described.

[0027] In some examples, technician device 108 may be implemented as a repair device (e.g., for device-to-device repair and / or to facilitate technician repair). For example, technician device 108 may interact with machine 104 (e.g., via a wired or wireless connection) to transmit instructions to machine 104 to perform diagnostics, collect information, and / or repair device 108. For example, technician device 108 may be a server or other network-connected device that pushes instructions to machine 104 (e.g., for remote repair). The instructions may cause machine 104 to load a new version of code, apply code patches, restart the machine, and / or perform other processes to repair machine 108.

[0028] Figure 1B The information and / or procedures that can be performed in the example environment 100 of Figure 1 are shown. Figure 1B include Figure 1A The machine recovery system 102, machine 104, service center 106, and technician equipment 108. Figure 1BIt also includes exemplary database systems 110 and 112, exemplary machine connection process 114, and exemplary service device workflow 116. Exemplary service device workflow 116 includes exemplary processes 118, 122, 124, 126, and 128.

[0029] initial, Figure 1B Service center 106 receives service requests from machine 104 and / or users of machine 104 (e.g., via telephone, computing device, and / or machine 104 itself). The service request may correspond to a problem present in machine 104. In some examples, the service request is received directly by machine recovery system 102. Once received, the service request is analyzed to determine potential solutions. In some examples, service center 102 utilizes database information from enterprise system 110 and / or service system 112 to identify specific data (e.g., corresponding to the initial context) relevant to machine 104. Additionally, machine recovery system 102 develops potential solutions, digital models, and / or relevant information corresponding to the service request based on knowledge corresponding to previous service requests. The initial context and information from prior knowledge may be transmitted as part of an intelligent maintenance package to technician device 108, which is executed by the technician device to provide relevant customized data corresponding to the solutions and / or information required to repair machine 104.

[0030] As described above, technician device 108 obtains a maintenance package corresponding to information that can help technician device 108 and / or utilize the resources of technician device 108 to service machine 104. In some examples, as shown in process 114, technician device 108 may connect to machine 104 to obtain system logs, continuous system health status, and / or other analyses corresponding to machine 104. Such data may be obtained before, during, and / or after service completion. Such data (e.g., machine data or imaging device data) may be transmitted to service center 106 and / or machine recovery system 102 to provide additional information that can be used to generate more accurate solution data, more accurate maintenance package information, and / or better resource allocation (e.g., scheduling) for future service requests. Such data, along with survey data from technician device 108, may be transmitted to service center 106 as part of device workflow 116.

[0031] As shown in the equipment workflow 116, each process 118, 120, 124, 126, 128 corresponds to data that can be used to create and / or improve the service center 106's understanding of future service requests. For example, during the review process 118, information corresponding to machine 104 can be collected, which can be used to categorize and / or diagnose problems with machine 104. Service center 106 can use such data to map the relationships between machine symptoms, error codes, problems, etc., and solutions, maintenance packages, and / or other information needed to repair machine 104. During the categorization process 120, information corresponding to problems, symptoms, etc., such as those corresponding to critical or non-critical errors, can be obtained. Such information can help schedule resources. For example, during the diagnostic process 122, service center 106 can obtain information corresponding to digital twins, previous problem solutions, implemented actions, etc., to optimize the diagnostic process for future service requests. For example, during the overhaul process 124, service center 106 can obtain information corresponding to overhaul techniques to determine which techniques are effective or ineffective for future service requests. For example, during the analysis / repair / dispatch process 126, service center 106 may obtain analysis information to update problem solution information stored in the problem solution database. For example, during the reporting / closure process 128, service center 106 may obtain investigation results information from technician equipment 108. Such information can be used to update the processes of service center 106 for future service requests.

[0032] Figure 2 yes Figure 1A and / or Figure 1B A block diagram illustrating an exemplary implementation of a machine recovery system 102. The machine recovery system 102 includes an intelligent symptom system 202, an intelligent maintenance package system 204, an intelligent scheduling system 206, an intelligent discovery system 208, and a recovery updater system 210. The machine recovery system 102 also includes a device interface 212 for interacting with deployed machines, service center 106, and / or technician equipment (e.g., technician equipment 108), and a database interface 214 for interacting with an enterprise system database 216, a machine log database 218, a problem solution database 220, and a dynamic system health status database 222. The intelligent symptom system 202 includes a symptom processor 224 and a filter 226. The intelligent maintenance package system 204 includes a maintenance package generator 228 and a solution predictor 230. The intelligent scheduling system 206 includes a remote instruction executor 232, a technician selector 234 (e.g., a resource selector), and a weight multiplier 236. The intelligent discovery system 208 includes a machine identifier 240 and a model generator 242. The recovery updater system 210 includes a survey generator 244 and an information updater 246.

[0033] Figure 2The intelligent symptom system 202 translates the symptoms of the malfunctioning machine 104 into error codes and / or identified problems. For example, when the machine malfunctions, the operator of machine 104 may not be able to identify the problem. Instead, the operator may only be able to identify the symptoms corresponding to the malfunctioning machine. Alternatively, machine 104 may generate error codes corresponding to specific parts of the machine (e.g., stored in machine log database 218), but not identify the problem itself. The intelligent symptom system 202 identifies the problems and / or error codes of machine 104 based on the symptoms identified by the user and / or the machine. The intelligent symptom system 202 includes a symptom processor 224 and a filter 226, as further described below.

[0034] Figure 2 The intelligent maintenance package system 204 generates intelligent maintenance packages for technicians and / or resources to aid in machine repair. For example, once a problem is identified, the intelligent maintenance package system 204 provides information on how to resolve the problem, information that can help in resolving the problem, tools needed to resolve the problem, replacement parts that may be needed to resolve the problem, and models that technicians can use to model possible repairs. The maintenance package system 204 generates maintenance packages to provide technicians with information about their equipment for use during / before machine repair. The intelligent maintenance package system 204 includes a maintenance package generator 228 and a solution predictor 230, as further described below.

[0035] Figure 2 The intelligent scheduling system 206 determines how to schedule resources (e.g., technicians, automated repair procedures, etc.) to resolve machine problems. For example, the intelligent scheduling system 206 may consider the skills required to repair the machine, other machines in the clinical workflow that may also require service / inspection, the criticality of the repair, how many technicians or other resources should be available to service the request, the distance to the repair site, whether the solution can be executed remotely, etc., to select the best technician / resource to schedule to service the machine. While the intelligent scheduling system 206 may combine technicians as described as resources, other resources may be included in the intelligent scheduling process. For example, the intelligent scheduling system 206 may determine that automated repair procedures are available as resources to execute instructions on machine 104 to repair the machine using software patches, reboots, etc. Resources may include computing devices such as laptops, tablets, smartphones, other handheld technician devices, etc., which may be connected to machine 104 to facilitate, for example, the diagnosis and / or repair of machine 104 and / or other connected components. The intelligent scheduling system 206 includes a remote instruction executor 232, a technician selector 234, and a weighted multiplier 236, as further described below.

[0036] Figure 2The intelligent discovery system 208 identifies machines based on contextual information (e.g., by comparing information obtained corresponding to machine 104 with information stored in the enterprise system database 216) and / or an identifier, and determines whether the identified machine corresponds to a similar group of machines and / or machines deployed in a similar environment. Intelligent discovery 208 may select and / or generate artificial intelligence models (e.g., digital twins / models) based on parameters similar to the machine group, and use feedback to update the artificial intelligence models based on feedback information for more accurate and effective diagnosis / scheduling / package generation of subsequent machine problems. Additionally, intelligent discovery 208 tracks problems in the group to flag common problems for priority services (e.g., to identify upcoming problems and / or any other overall system statistical analysis). Intelligent discovery 208 includes a machine identifier 240 and a model generator 242, as further described below.

[0037] Figure 2 The updater system 210 updates information stored in the problem solution database 220, the dynamic system health status database 222, the layout and / or display information of the intelligent maintenance package, and / or any other databases based on identified error codes and / or feedback from technicians / resources once a service request is completed. Additionally, the updater system 120 may use an artificial intelligence model based on technician / resource feedback to update various information used by the processes disclosed herein. The machine recovery system 102, for example, uses neural networks to build knowledge corresponding to previous machine services based on those services, enabling improvements to subsequent diagnostics, maintenance packages, models / digital twins, scheduling, etc. The updater system 210 includes a survey generator 244 and an information updater 246, as further described below.

[0038] Figure 2Device interface 212 is a structural component that interfaces with machine 104 and / or service center 106. For example, when one of the deployed machines has a problem, machine 104 can automatically and directly transmit information corresponding to the problem / error to device interface via wired or wireless network communication. Alternatively, machine 104 may include an interface for users to transmit problem and / or error information to service center 106 and / or directly to device interface 212. In some examples, a user of machine 104 may determine that machine 104 is not working properly and call service center 106. In such examples, service center 106 may include a user or processor that inputs information about machine 104 and / or the symptoms of machine 104, which is transmitted to device interface 212. Additionally, device interface 212 transmits maintenance packages to selected technician devices (e.g., technician device 108) to alert technicians / resources to service requests and provide technicians / resources with intelligent maintenance packages corresponding to the information required to service the machine. Device interface 212 can also transmit survey information to technician device 108 to obtain feedback from technicians / resources after the service is completed. Responses are received from technician device 108 via device interface 212.

[0039] Figure 2 Database interface 214 is a structural component that interfaces with databases 216, 218, 220, and 222. In some examples, database interface 214 obtains information from one or more of databases 216, 218, 220, and 222 via wireless network communication. Machine recovery system 102 can perform intelligent symptom management, generate intelligent maintenance packages, initiate intelligent scheduling, and perform intelligent discovery based on information in databases 216, 218, 220, and 222. In some examples, database interface 214 interfaces with one or more databases 216, 218, 220, and 222 to update information in databases 216, 218, 220, and 222 based on identified error codes and / or feedback from technicians and / or resources.

[0040] Figure 2Enterprise system database 216 is one or more databases and / or other data repositories / storages that store data corresponding to system identifiers, customer sites and layouts, contractual guarantees, asset locations, and / or other contextual information. Information in enterprise system database 216 can be initially entered when a customer installs a machine and can be updated periodically or non-periodically based on updates to machine information. Enterprise system database 216 can be locally located or remotely located. For example, a machine may include an enterprise system communicating with database interface 214 via wireless network communication, or machine 104 may periodically or non-periodically transmit enterprise system information to enterprise system database 216. Machine recovery system 102 uses this enterprise system information in conjunction with intelligent discovery processes, intelligent symptom processes, and / or intelligent scheduling processes. Enterprise system database 216 may be a single database for information on all deployed machines, or it may be multiple databases corresponding to customers, locations, groups, types, etc. Machine recovery system 102 uses this contextual information to determine symptom identification from service history data and to describe problems from users. Additionally, the machine recovery system 102 can use the input of specific machines into the enterprise system database 216 for database information specific to a particular customer site / geographic region (e.g., postal code, neighborhood, address, etc.) to determine how best to dispatch technicians / resources.

[0041] Figure 2 Machine log 218 is, for example, one or more databases and / or other data repositories / storages containing machine logs, system health status information, identifiers, error traces, utilization information, and usage information corresponding to the machines stored and deployed. The information in machine log 218 is provided by the deployed machines. Machine log 218 can be located locally or remotely. For example, machine 104 may include machine logs at its location, communicating with database interface 214 via wireless network communication, or machine 104 may periodically or non-periodically transmit machine log information to a remote machine log database. Machine log 218 may be a single database containing information for all deployed machines, or machine log 218 may be multiple databases corresponding to customers, locations, groups, types, etc. Machine recovery system 102 uses such machine log information in conjunction with intelligent discovery processes, intelligent maintenance package processes, and / or intelligent symptom processes. For example, machine recovery system 102 may correlate information historically used by users to describe specific system problems with machine logs to form effective intelligent symptoms to identify machine problems. Additionally, the machine recovery system 102 can identify system problems as idiosyncratic or cluster-level problems by viewing machine log information to identify correlations when a problem is identified.

[0042] Figure 2The problem solution database 220 is a database and / or other data repository, storage, etc., based on survey data from previously completed service requests, corresponding to the relevance of previous problems and solutions. The problem solution database 220 can be located locally or remotely. The problem solution database 220 can be a single database for information on all deployed machines, or it can be multiple databases corresponding to customers, locations, groups, types, etc. The machine recovery system 102 uses this problem solution information in conjunction with intelligent maintenance package processes and / or intelligent scheduling processes. For example, the machine recovery system 102 can identify the required manual sections, replacement parts, and / or tools for a specific problem based on previous survey information to develop an intelligent maintenance package. Additionally, the machine recovery system 102 can utilize problem solution information during the intelligent scheduling process, regardless of whether the problem can be classified as a group problem or an idiosyncratic problem.

[0043] Figure 2 The dynamic system health status database 222 is a database and / or other data repository, storage, etc., that stores data corresponding to the health status of deployed machines (e.g., corresponding to the age of the machine compared to similar machines, identified problems, etc.). The dynamic system health status database 222 can be locally located or remotely located. The dynamic system health status database 222 can be a single database for information on all deployed machines, or it can be multiple databases corresponding to customers, locations, groups, types, etc. The machine recovery system 102 uses this dynamic system health status information in conjunction with intelligent maintenance package processes and / or intelligent symptom processes. For example, the machine recovery system 102 can monitor the dynamic health status of machines while a service request is being viewed to provide better capabilities to influence intelligent logs and intelligent symptoms. By viewing the current health status of the machine, the machine recovery system 102 can make better decisions about actual problems and / or the maintenance packages needed to resolve those problems. For example, the system health status of a machine can be monitored by running a background daemon that analyzes processed data in near real-time and updates the system health status in the dynamic system health status database 222 accordingly.

[0044] Figure 2Symptom processor 224 receives identified symptoms from machine 104 and / or service center 106, and uses known problem information (e.g., data corresponding to problem database 227), machine logs, machine information, and dynamic health status information corresponding to the identified symptoms to identify the problems and / or error codes corresponding to the identified symptoms based on problem solution information. For example, when a user receives symptoms from the machine, symptom processor 224 uses the corresponding machine health status, machine information, and machine logs to identify the error code / problem corresponding to the identified symptoms. To reduce the number of error codes / problems in the group, symptom processor 224 may transmit a prompt to the user to identify whether there are distinguishing symptoms in the machine. Distinguishing symptoms are those that are present in one or more error codes / problems in the group but not in all one or more error codes / problems in the group. For example, symptoms common to all error codes / problems in the group are not distinguishing symptoms. Alternatively or concurrently, symptom processor 224 may interact directly with the device via device interface 212 to perform one or more processes to identify whether there are distinguishing symptoms in the machine.

[0045] In addition to or alternatively, Figure 2 Symptom processor 224 can interact with machine 108 (e.g., directly using device interface 212 and / or indirectly via technician device 108) to obtain output data from machine 104 (e.g., photographs taken by machine 104). In such examples, symptom processor 224 can process images obtained by imaging equipment to identify artifacts (e.g., defects) found in the images. Symptom processor 224 can use the identified defects to identify problems, the source of the problem, and / or additional symptoms. For example, symptom processor 224 can determine, based on the identified defects, insufficient radiation, excessive radiation, a faulty detector, misalignment, etc. In some examples, symptom processor 224 can compare the identified problem, the source of the problem, and / or symptoms associated with the defect with previous problem solution information (e.g., stored in exemplary problem solution database 220, etc.) to determine whether the problem corresponds to a configuration and / or design defect. In such examples, symptom processor 224 can transmit alerts to machine 108, users of machine 108 (e.g., via text, email, etc.), system administrators, etc., to identify design defects. Additionally, the machine identifier 240 can update group information and / or generate alerts to similar machines to identify design flaws.

[0046] Once a prompt indicating the presence of distinguishing symptoms is received in the identification machine, Figure 2Filter 226 can filter a set of problems from problem database 227 into a subset based on the presence or absence of distinguishing symptoms. For example, when the first half of a set of error codes / problems corresponds to a distinguishing symptom and the second half of the set of error codes / problems does not correspond to a distinguishing symptom, filter 226 filters out either the first half or the second half of the set to generate a subset based on whether the response to the prompt identifies the presence of a distinguishing feature in the machine.

[0047] Figure 2 Problem database 227 stores a list of known problems. Additionally, problem database 227 stores the corresponding error codes and / or symptoms for the known problems. In some examples, when a technician / resource identifies a new problem, the technician / resource and / or the technician device 108 itself can input information corresponding to the new problem via the technician device 108, and problem database 227 updates the stored data to include the new problem and its corresponding error code and / or symptoms. Alternatively, machine 104 can transmit new error codes and / or new combinations of error codes that the problem database 227 can store as new problems. Alternatively, the manager / system coordinator can add additional problems and their corresponding error codes and / or symptoms to problem database 227.

[0048] Figure 2The maintenance package generator 228 is a structural component for generating customized maintenance packages for service technicians / resources regarding the requesting machine. The maintenance package generator 228 initially has a template maintenance package that includes blank fields that can be customized based on identified solutions, machine information, etc. For example, when the location / type of the machine 104 to be serviced is known, the maintenance package generator 228 may include a map corresponding to the location of machine 104 and information corresponding to the machine type. Additionally, the maintenance package generator 228 may provide more detailed and / or relevant information based on one or more identified error codes or one or more solutions predicted by the solution predictor 230, as further described below. For example, when the solution predictor 230 determines that the solution corresponds to a solution associated with a specific part of the machine, the maintenance package generator 228 may include manual information related to that specific part of machine 104, replacement parts that the service machine may need, and / or specialized tools that may be needed to service that specific part and / or that specific solution. Specialized tools may be tools that are not typically carried by all technicians (e.g., tools that may be present at a specific location and need to be brought to the service request). Maintenance package generator 228 may determine whether replacement parts are needed to service machine 104 based on feedback from previous service requests from technicians that can be stored in problem solution database 220. In some examples, maintenance package generator 228 may include a digital twin and / or network model (e.g., one or more artificial intelligence models) corresponding to the identified machine. Using the digital twin and / or artificial intelligence model, technicians can virtually test the model and / or digital twin before servicing machine 104 to help ensure that the technician's actions will repair the machine. In some examples, when maintenance package generator 228 determines that one or more solutions are remotely executable, maintenance package generator 228 may include instructions on how to remotely service machine 104 and / or code that can be transmitted to the device to remotely service machine 104 in the maintenance package.

[0049] Figure 2 The solution predictor 230 predicts the structural components of a solution based on identified error codes / problems by combining information from a problem solution database and / or system health status information. For example, the solution predictor 230 may identify potential solutions by: linking error codes / problems to other machines based on previous solutions identified by technicians in prior investigations; applying the error / problem to a digital twin / model and simulating a potential solution; identifying similar error codes / problems in other machines (e.g., based on similar models, similar contextual information, similar group information, similar contextual information, etc.); and / or based on the system health status of the machines. In addition or alternatively, the solution predictor 230 may implement neural networks (e.g., such as...). Figure 5A neural network 500 is used to convert error codes and / or problems into problem solution information and / or system health status information to predict solutions. In such examples, previous service request data can be used to train the neural network. The solution predictor 230 transmits the identified solutions to the maintenance package generator 228 to customize maintenance packages based on the identified solutions.

[0050] Figure 2 The remote instruction executor 232 is a structural component that transmits instructions (e.g., via device interface 212) to machine 104 to attempt to resolve a problem remotely. For example, when a maintenance package includes instructions corresponding to a potential remote solution, the remote instruction executor 232 can automatically or based on instructions from a technician transmit the instructions to the device via device interface 212. The device can then execute the instructions to check whether remote service has repaired the machine. Such remote instructions may include software updates, restart commands, patches, repairs, etc.

[0051] Figure 2 The technician selector 234 selects technicians to provide services for a request based on multiple factors. Technicians and their related information are stored in a technician database 238, as further described below. In some examples, the technician selector 234 may determine whether the service request is critical (e.g., based on the severity of the error code / problem and / or the availability of alternative machines in the hospital). When the service request is critical, the technician selector 234 will select only one or more technicians immediately available to service machine 104 (e.g., based on technician availability and the technician's location relative to the machine). In some examples, when the service request is critical, the technician selector 234 may also consider the tools / replacement parts required for the service request when determining the distance of the technician to the machine. For example, when the service request requires the location of a dedicated tool / replacement part at a specific location, the technician selector 234 considers the distance of the technician to the location of the dedicated tool / replacement part. Alternatively or additionally, the technician selector 234 selects one or more technicians to provide services for the request based on technician weights. The technician weight corresponds to the degree to which the technician matches the service request and is determined using a weight multiplier 236, as further described below. In some examples, the technician selector 234 analyzes the clinical workflow based on information from databases 216, 218, 220, and 222 to see if there are any additional machines that should be serviced, will soon require service (e.g., may be part of a problem), or may have problems. In some examples, the technician selector 234 may implement a neural network (e.g., Figure 5An exemplary neural network 500 can be used to select one or more resources based on the aforementioned input / resource characteristics. In such examples, the neural network may be trained based on resources and / or previous service request data from an exemplary technician database 238.

[0052] Figure 2 The weight multiplier 236 is a structural component that weights the available technicians for a service request. For example, the weight multiplier 236 can set an initial weight for each available technician. The weight multiplier 236 adjusts the initial weights based on various factors. These factors may correspond to, for example, the number of machines for which a technician can service a service request, the technician's skill level, the tools currently available to the technician, replacement parts currently available to the technician (e.g., those identifiable by the technician using device 108), the distance to the service site (taking into account the distance to pick up additional tools if needed), the availability of the technician, etc. The amount of weight adjustment for each factor may be based on user, manufacturer, or customer preferences.

[0053] Figure 2 The technician database 238 is a database that includes up-to-date information corresponding to available technicians and / or available resources, technician schedules, technician skill levels, types of machines that technicians and / or other resources can service, tools and / or replacement parts currently owned by technicians, technician locations, etc. The set of available technicians / resources can be filtered and weighted based on the information stored in the technician database 238. For example, automated resources and / or instruction / software-based resources can be filtered out if a problem cannot be resolved remotely. Technician information can be updated periodically, non-periodically, and / or in real-time or near real-time based on information from technician equipment 108 (e.g., via equipment interface 212).

[0054] Figure 2Machine identifier 240 is a structural component that identifies machines and associates the identified machines with a group of machines. For example, machine identifier 240 may identify machine 104 based on a machine identifier (e.g., an imaging device identifier, etc.) provided by machine 104 and / or its user, or it may identify machine 104 based on contextual information (e.g., by comparing the machine's location, type, database information corresponding to the machine's global scope, etc., with system information stored in enterprise system database 216). In some examples, machine identifier 240 may use image processing techniques to identify machine 104 based on photographs submitted by a user, taken by machine 104, taken by technician equipment 108, etc. In such examples, machine recognizer 240 may compare the characteristics of an image with those of a reference image to identify machine 104 based on a matching reference image and / or identify a determined location based on a photograph of the machine (e.g., by identifying a specific machine 104 based on characteristics of a specific machine 104 identified in an image, by identifying the location of a specific machine 104 based on features of the environment surrounding the machine 104 captured in a photograph, etc.). Once a recognizer is determined, machine recognizer 240 determines the brand, model, modality, and / or other information of the identified machine by comparing the recognizer with machine information stored in a database (e.g., enterprise system database 216, etc.). Additionally, machine recognizer 240 determines whether a corresponding group of machines exists. In some examples, machine recognizer 240 may implement a neural network (e.g., Figure 5An exemplary neural network 500 identifies groups based on the identified machines. In such examples, the neural network can be trained based on machine information corresponding to the machine global domain. A machine group corresponds to a set of machines with similar contextual information. A machine group can correspond to any combination of similar locations, similar models, similar types, similar climates, similar ages, similar characteristics, etc. In some examples, machines can correspond to multiple different groups. Each group can have an artificial intelligence model (e.g., a neural network model, digital twin, and / or other machine / deep learning constructs, etc.) that corresponds to a group for applying techniques (e.g., for testing purposes) and / or identifying which solutions can solve the problem. Additionally, the machine identifier 240 can store and tag common problems between groups. For example, when a machine in a group has a problem, the machine identifier 240 can store the problem: the time the problem occurred, the duration the machine 104 was operational when the problem occurred, the time since the last service, etc. When machine identifier 240 determines that the same problem has occurred more than a threshold number of times in the same group, machine identifier 240 may flag the problem and transmit a warning corresponding to the flag to identify that the problem may occur in other machines in the group. For example, when machine identifier 240 determines that three machines in the group have experienced the same problem after three years of operation, machine identifier 240 may transmit a warning related to any machine in the group that has been operating for approximately three years. Additionally, machine identifier 240 may update group information and / or generate alerts for similar machines based on problems identified by other processes of the exemplary machine recovery system 102 (e.g., design flaws, etc.).

[0055] Figure 2 The model generator 242 is a structural component used to generate artificial intelligence models (e.g., network models / digital twins, other machine / deep learning constructs, etc.) when a device does not correspond to one of the groups (e.g., does not correspond to an already available artificial intelligence model). New groups are created with the new artificial intelligence models, and subsequent machines corresponding to the new groups can be added to the groups. The artificial intelligence model corresponds to machine information (e.g., model, years of use, usage profile, etc.).

[0056] Figure 2The survey generator 244 is a structural component for generating surveys (e.g., tips, tags, etc.) for technicians. This survey is provided to the technician device 108 to obtain a response from the technician after a service request is completed. In some examples, the survey is pre-loaded onto the technician device 108. The survey includes information related to: whether the maintenance package includes all relevant information, which information is irrelevant to the layout of the intelligent maintenance package, whether any replacement parts are needed and, if so, which parts, what information should be included but is not, what tools were used, whether the identified solutions are correct, whether there are any unexpected problems, and / or any other information that could generate more accurate intelligent scheduling, intelligent symptoms, intelligent maintenance packages, and / or intelligent discovery processes.

[0057] Figure 2 Information updater 246 updates information in problem solution database 220 and / or dynamic system health status database 222 based on information from responses to surveys / prompts from technicians. Additionally, information updater 246 can update technician database 238 based on changes in technicians (e.g., location, tools, etc.). Information updater 246 may utilize and / or include artificial intelligence models (e.g., digital twins, neural networks, etc.) to perform machine learning processes to tune and improve intelligent symptom management, intelligent maintenance packages, intelligent scheduling, and intelligent discovery processes.

[0058] Artificial intelligence model

[0059] In some examples, the target equipment or machine to be evaluated / repaired (e.g., imaging equipment, imaging workstations, health status information systems, etc.), resources (e.g., technicians and / or other users, technician equipment, maintenance kits, etc.), and machine groups can be modeled as digital twins and / or processed according to artificial neural networks and / or other machine / deep learning network models (referred to herein as “artificial intelligence models”). Using one or more artificial intelligence models, such as digital twins, neural network models, etc., one or more real-world systems can be modeled, monitored, simulated, and prepared for automated field management.

[0060] Digital Twin Examples

[0061] Digital representation, digital model, digital "twin," or digital "shadow" are digital informatics constructs concerning physical systems, processes, etc. That is, digital information can be realized as a "twin" of a physical device / system / person / process and information associated with and / or embedded within the physical device / system / process. A digital twin is linked to the physical system through its lifecycle. In some examples, a digital twin includes a physical object in real space, a digital twin of that physical object existing in virtual space, and information linking the physical object to its digital twin. The digital twin exists in a virtual space corresponding to the real space and includes links for data flows from the real space to the virtual space and connections for information flows from the virtual space to the real space and virtual subspaces.

[0062] For example, Figure 3 This illustrates machines or equipment, such as imaging equipment, radiology workstations, information systems, etc.; resources, such as technicians, other users, computing devices, maintenance kits, etc.; and / or other items 310 in the real space 315 that provide data 320 to the digital twin 330 in the virtual space 335. The digital twin 330 and / or its virtual space 335 provide information 340 back to the real space 315. The digital twin 330 and / or the virtual space 335 may also provide information to one or more virtual subspaces 350, 352, 354. Figure 3 As shown in the example, virtual space 335 may include one or more virtual subspaces 350, 352, 354 and / or be associated with one or more virtual subspaces, which can be used to model one or more parts of digital twin 330 and / or digital “sub-twin”, thereby modeling subsystems / subparts of overall digital twin 330.

[0063] Sensors connected to a physical object (e.g., device / resource 310) can collect data and relay the collected data 320 to the digital twin 330 (e.g., via one or more device sensors, self-reporting, output from one or more system components such as device interface 212, database interface 214, symptom processor 224, filter 226, maintenance package generator 228, solution predictor 230, remote command executor 232, technician selector 234, weight multiplier 236, technician database 238, machine identifier 240, model generator 242, survey generator 224, information updater 246, database interface 246, and / or more generally, exemplary machine recovery system 102, and / or combinations thereof, etc.). The interaction between the digital twin 330 and the device / resource 310 can help improve symptom detection, problem identification, problem resolution, tool configuration, etc. The accurate numerical description 330 of the device / resource / item 310 that benefits from real-time or substantially real-time (e.g., taking into account data transmission, processing and / or storage latency) allows the system 100 to predict “failures”, accuracy, results, communications, etc. in machine / equipment operation.

[0064] In some examples, sensor data, technical support, acquired images, test results, etc., can be used in augmented reality (AR) applications when a technician inspects, diagnoses, and / or otherwise repairs a machine, equipment, etc. (e.g., machine 104). Using AR, the digital twin 330 follows the machine's response to interactions, such as with a technician, technician equipment 108, and / or machine recovery system 102.

[0065] Therefore, the digital twin 330 is not a generic model, but rather a collection of actual, physically-based models reflecting the equipment / resource 310 (e.g., machine 104, machine recovery system 102, technician equipment, etc.) and its associated specifications, conditions, etc. In some examples, a three-dimensional (3D) model of the equipment / resource 310 forms the digital twin 330 of the equipment / resource 310. The digital twin 330 can be used to view the status of the equipment / resource 310 based on input data 320 dynamically provided from sources (e.g., from the equipment / resource 310, technicians, health status information systems, sensors, etc.).

[0066] In some examples, a digital twin 330 of device / resource 310 can be used for monitoring, diagnosis, and prognosis of device / resource 310. Sensor data can be combined with historical information to use the digital twin 330 to identify, predict, and monitor current and / or potential future problems of device / resource 310. The digital twin 330 can be used to monitor the causes, aggravation, and improvement of problems. The digital twin 330 can be used to simulate and visualize the behavior of device / resource 310 for diagnosis, treatment, monitoring, maintenance, etc.

[0067] Unlike computers, humans do not process information in an orderly, step-by-step manner. Instead, humans attempt to conceptualize problems and understand their context. While humans can view data in reports, tables, etc., they are most effective when they visually examine problems and attempt to discover their solutions. However, information is often lost when humans process information visually, record it in alphanumeric form, and then attempt to visually reconceptualize it, and the problem-solving process becomes extremely inefficient over time.

[0068] However, using a digital twin 330 allows people and / or systems to view and assess visualizations of situations (e.g., equipment / resource 310 and associated operational problems) without having to convert data back and forth. Utilizing a digital twin 330 that shares a common perspective with the actual equipment / resource 310, both physical and virtual information can be viewed dynamically and in real-time (or near real-time, taking into account data processing, transmission, and / or storage latency). Instead of reading reports, technicians can use the digital twin 330 to view and simulate to assess the condition, progression, and potential treatments of the equipment / resource 310. In some examples, features, symptoms, trends, indicators, traits, etc., can be labeled and / or otherwise marked in the digital twin 330 to allow technicians to quickly and easily view specified parameters, values, trends, alerts, etc.

[0069] The digital twin 330 can also be used for comparisons (e.g., with device / resource 310, with “normal,” standard, or reference devices or resources, a set of operating standards / symptoms, best practices, protocol procedures, etc.). In some examples, the digital twin 330 of device / resource 310 can be used to measure and visualize the ideal or “gold standard” value state of that device / resource, the tolerance or standard deviation around that value (e.g., positive and / or negative deviations relative to the gold standard value), the actual value, the trend of the actual value, etc. The difference between the actual value or the trend of the actual value and the gold standard (e.g., exceeding the tolerance) can be visualized as alphanumeric values, color indicators, patterns, etc.

[0070] Furthermore, the digital twin 330 of the machine / equipment 104 to be repaired, and the equipment 108 for facilitating diagnosis / repair by technicians, can facilitate collaboration between systems, technicians, etc. Using the digital twin 330, a conceptualization of the equipment / resource 310 and its status / configuration can be shared for evaluation, modification, discussion, etc. For example, collaborating entities do not need to be located in the same location as the equipment / resource 310, but can still view, interact with, and draw conclusions from the same digital twin 330.

[0071] Therefore, a digital twin 330 can be defined as a set of virtual information constructs that describe (e.g., fully describe) the device resource 310 from a microscopic level (e.g., sources, detectors, locators, processors, storage devices, etc.) to a macroscopic level (e.g., the entire imaging apparatus, image capture subsystem, image analysis subsystem, patient monitoring subsystem, etc.). In some examples, the digital twin 330 can be a reference digital twin (e.g., a digital twin prototype, etc.) and / or a digital twin instance. A reference digital twin represents a prototype or "gold standard" model of device / resource 310 or a particular type / category of device / resource 310, while one or more reference digital twins represent a particular device / resource 310. Thus, the digital twin 130 of an x-ray imaging apparatus can be implemented as a sub-reference digital twin organized according to certain standards or "typical" x-ray imaging apparatus characteristics, where a particular digital twin instance represents, for example, a particular x-ray imaging apparatus and its configuration. In some examples, multiple digital twin instances can be aggregated into a digital twin aggregation (e.g., to represent the accumulation or combination of multiple x-ray machines in a group sharing a common reference digital twin, etc.). For example, digital twin aggregation can be used to identify differences, similarities, trends, etc., between machines represented by specific digital twin instances.

[0072] In some examples, the virtual space 335 in which the digital twin 330 (and / or multiple digital twin instances, etc.) operates is referred to as a digital twin environment. The digital twin environment 335 provides a physically-based, integrated multi-domain application space in which the digital twin 330 operates. For example, the digital twin 330 can be analyzed in the digital twin environment 335 to predict future behavior, symptoms, progression, etc., of the device / resource 310. The digital twin 330 can also be queried or questioned in the digital twin environment 335 to retrieve and / or analyze current information 340, past medical history, etc.

[0073] In some examples, the digital twin environment 335 can be divided into multiple virtual spaces 350 to 354. Each virtual space 350 to 354 can model different digital twin instances and / or components of the digital twin 330, and / or each virtual space 350 to 354 can be used to perform different analyses, simulations, etc., on the same digital twin 330. Using multiple virtual spaces 350 to 354, the digital twin 330 can be tested inexpensively and efficiently in a variety of ways, while maintaining the safety of device / resource operation and patients. For example, technicians can then understand how the device / resource 310 can respond to various adjustments in different scenarios.

[0074] In some examples, the digital twin 330 can also model spaces such as operating rooms, surgical centers, preoperative preparation rooms, and postoperative recovery rooms. By modeling environments such as surgical suites, safer, more reliable, and / or more efficient environments can be created for patients and healthcare professionals (e.g., surgeons, nurses, anesthesiologists, technicians, etc.). For example, the digital twin 330 can be used to assess environmental factors that can affect the operation of machine 104 (e.g., spacing, usage patterns, other equipment, etc.).

[0075] In some examples, devices such as optical head-mounted displays (e.g., Google Glass) can be used in conjunction with augmented reality to provide additional information, repair information, diagnostic suggestions, etc., about the machine 104 being viewed. This device can be used to pull in machine and / or technician equipment details, modeled via a digital twin 330, and verified against manufacturer specifications, customer machine configurations, protocols, personnel preferences, etc.

[0076] like Figure 4A As illustrated in the examples, the optical head-mounted display 400 may include a scanner or other sensor 410 that scans items in its field of view (e.g., barcodes, radio frequency identification (RFID), visual profiles / features, etc.). In some examples, the head-mounted display 400 may be... Figure 1A and / or Figure 1B Technical personnel and equipment 108 and / or can be with Figure 1A and / or Figure 1B The equipment 108 is used in conjunction with the technician's equipment. For example, item identification, photographs, video feeds, etc., can be provided to the digital twin 330 by the scanner 410. For example, the scanner 410 and / or the digital twin 330 can identify and track items within the range of the scanner 410. The digital twin 330 can then model the observed environment and / or objects in the observed environment, for example, based at least in part on the input from the scanner 410.

[0077] Alternatively or otherwise, the digital twin 330 can be utilized via a technician's device 450, such as a smartphone, tablet, laptop, etc. In some examples, the exemplary technician's device 450 may be... Figure 1A and / or Figure 1B Technical personnel and equipment 108 and / or can be with Figure 1A and / or Figure 1B The technical personnel and equipment 108 are used in combination. For example... Figure 4BAs shown, the technician device 450 can receive input from users, machine recovery system 102, other diagnostic equipment, health information systems, etc., and provide output to technicians via a display, and communicate with the machine 104 to be inspected. For example, the technician device 450 can provide technicians with suggestions for diagnosing and / or repairing machine 104. For example, the technician device 450 can also be used to interact with machine 104 to diagnose and / or repair problems with machine 104 and / or related components. Figure 4B As illustrated in the example, the technician device 450 may provide one or more diagnostic and / or repair suggestions 452 via the device's display screen 454. The exemplary technician repair device 450 may also communicate 456 with machine 104 (e.g., wirelessly and / or via a wired connection) to extract diagnostic information, transmit configuration settings, etc.

[0078] Figure 4C Examples are shown of providing advice, communication, prompts, other information, and / or interaction to technicians at home, on the go, and / or on-site via a smartwatch 480. For example, one or more suggestions 482 are provided via the display 484 of the smartwatch 480. In some examples, an exemplary head-mounted display 480 may be... Figure 1A and / or Figure 1B The technicians and equipment 108 and / or can be with Figure 1A and / or Figure 1B The technicians and equipment 108 are used in combination.

[0079] Machine and / or deep learning network models

[0080] Machine learning techniques, whether deep learning networks or other experiential / observational learning systems, can be used, together with and / or separately from digital twins, to model information to analyze and / or predict problems, issues, errors, failures, etc., in a system based on log data and / or other symptoms, predicting the outcome of specific solutions on a specific machine. Deep learning is a subset of machine learning that uses a set of algorithms to model high-level abstractions in data using a deep graph with multiple processing layers, including linear and nonlinear transformations. While many machine learning systems first embed initial features and / or network weights and then modify them through learning and updates by the machine learning network, deep learning networks identify “good” features for analysis by training themselves. When using a multi-layered architecture, machines employing deep learning techniques can process raw data better than machines using conventional machine learning techniques. Using different layers for evaluation or abstraction facilitates data inspection of highly correlated values ​​or discriminative topics.

[0081] Deep learning is a class of machine learning techniques that employ representation learning methods, allowing machines to be given raw data and determine the representations needed for data classification. Deep learning uses a backpropagation algorithm to determine the structure of a dataset by altering the machine's internal parameters (e.g., node weights). Deep learning machines can utilize various multi-layer architectures and algorithms. For example, while machine learning involves identifying features to be used to train a network, deep learning processes raw data to identify features of interest without external identification.

[0082] Deep learning in a neural network environment comprises many interconnected nodes called neurons. Input neurons, activated by external sources, activate other neurons based on connections to them controlled by machine parameters. Neural networks function in a certain way based on their own parameters. Learning improves the machine parameters, and more broadly, improves the connections between neurons in the network, causing the neural network to function in the desired manner.

[0083] Deep learning, utilizing convolutional neural networks (CNNs), uses convolutional filters to segment data in order to locate and identify observable features learned from the data. Each filter or layer in a CNN architecture transforms the input data to increase its selectivity and invariance. This abstraction of the data allows the machine to focus on the features it is attempting to classify and ignore irrelevant background information.

[0084] Alternatively, or in addition to CNNs, deep residual networks can be used. In deep residual networks, the required low-level mappings are explicitly defined as being related to the stacked nonlinear inner layers of the network. Due to the use of feedforward neural networks, deep residual networks can include shortcut connections that skip one or more inner layers to connect nodes. Deep residual networks can be trained end-to-end, for example, using backpropagation via stochastic gradient descent (SGD).

[0085] Deep learning operates on the understanding that many datasets contain high-level features, which in turn contain low-level features. For example, when examining an image of an object, instead of searching for the object itself, it's more efficient to search for edges. Edges form motifs, motifs form parts, and parts form the object being searched for. These levels of features are visible in many different forms of data, such as speech and text.

[0086] The learned observable features include the objects the machine learns during supervised learning and quantifiable regularity. Machines with large sets of data that can be effectively classified are better positioned to distinguish and extract features associated with successful classification of new data.

[0087] Deep learning machines that utilize transfer learning can correctly connect data features to certain classifications confirmed by human experts. Conversely, the same machine can update the parameters used for classification when a human expert points out a classification error. For example, settings and / or other configuration information can be guided by the use of learned settings and / or other configuration information, and the number of changes and / or other possibilities in settings and / or other configuration information can be reduced for a given situation as the system is used more frequently (e.g., repeatedly and / or by multiple users).

[0088] For example, an expert classification dataset can be used to train an exemplary deep learning neural network. This dataset constructs the first parameters of the neural network, and this becomes the supervised learning phase. During the supervised learning phase, it is possible to test whether the neural network has achieved the desired behavior.

[0089] Once the desired neural network behavior has been achieved (e.g., the machine is trained to operate according to a specified threshold), the machine can be deployed for use (e.g., testing the machine with "real" data). During operation, the neural network classification can be affirmed or rejected (e.g., by an expert user, expert system, reference database, etc.) to continue improving the neural network behavior. The exemplary neural network is then in a transfer learning state because the classification parameters that determine the neural network behavior are updated based on the ongoing interactions. In some examples, the neural network may provide direct feedback to another process. In some examples, the data output by the neural network is buffered (e.g., via the cloud) and validated before being provided to another process.

[0090] Deep learning machines using convolutional neural networks (CNNs) can be used for data analysis. Stages of CNN analysis can be used for facial recognition in natural images, computer-aided diagnosis (CAD), object recognition and tracking, symptom assessment, and treatment outcome evaluation, among others.

[0091] Deep learning machines can provide computer-aided detection support to improve tasks such as item identification, relevance assessment, and tracking. For example, supervised deep learning can help reduce susceptibility to misclassification. Deep learning machines can leverage transfer learning to offset the small datasets available during supervised training when interacting with technical personnel. These deep learning machines can improve their protocol dependencies over time through training and transfer learning.

[0092] Figure 5As a representation of an exemplary deep learning neural network model 500, this exemplary deep learning neural network model can be used to implement a digital twin 330, work with the digital twin 330 to provide diagnostic and / or remedial suggestions, and / or replace the operation of the digital twin 330 to generate suggested problems based on input symptoms, provide suggested remediation based on input machine configuration and problem information, etc. The exemplary neural network 500 includes layers 520, 540, 560, and 580. Layers 520 and 540 are connected using neural connections 530. Layers 540 and 560 are connected using neural connections 550. Layers 560 and 580 are connected using neural connections 570. Data flows forward from the input layer 520 to the output layer 580 and reaches the output 590 via inputs 512, 514, and 516. In some examples, the neural network model 500 includes... Figure 2 The machine recovery system 102 is used in or by one or more of its components.

[0093] Layer 520 is the input layer, which is in Figure 5 The example includes multiple nodes 522, 524, and 526. Layers 540 and 560 are hidden layers, and... Figure 5 The example includes nodes 542, 544, 546, 548, 562, 564, 566, and 568. The neural network 500 may include more or fewer hidden layers 540 and 560 than shown. Layer 580 is the output layer, and... Figure 5 The example includes node 582 with output 590. Each input 512 to 516 corresponds to nodes 522 to 526 of input layer 520, and each node 522 to 526 of input layer 520 has a connection 530 to each node 542 to 548 of hidden layer 540. Each node 542 to 548 of hidden layer 540 has a connection 550 to each node 562 to 568 of hidden layer 560. Each node 562 to 568 of hidden layer 560 has a connection 570 to output layer 580. Output layer 580 has an output 590 to provide output from exemplary neural network 500.

[0094] In connections 530, 550, and 570, some exemplary connections 532, 552, and 572 may be assigned increased weights, while other exemplary connections 534, 554, and 574 may be assigned smaller weights in neural network 500. For example, input nodes 522 to 526 are activated by receiving input data via inputs 512 to 516. Nodes 542 to 548 and 562 to 568 of hidden layers 540 and 560 are activated by data flowing forward through network 500 via connections 530 and 550, respectively. After the data processed in hidden layers 540 and 560 is sent via connection 570, node 582 of output layer 580 is activated. When output node 582 of output layer 580 is activated, node 582 outputs an appropriate value based on the processing performed in hidden layers 540 and 560 of neural network 500.

[0095] Although Figure 2 It shows Figure 1A and / or Figure 1B An exemplary specific implementation of the machine recovery system 102, Figure 2 One or more of the elements, processes, and / or devices shown may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Additionally, device interface 212, database interface 214, symptom processor 224, filter 226, maintenance package generator 228, solution predictor 230, remote command executor 232, technician selector 234, weight multiplier 236, technician database 238, machine identifier 240, model generator 242, survey generator 224, information updater 246, database interface 246, and / or more generally... Figure 2 The machine recovery system 102 can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, it may include device interface 212, database interface 214, symptom processor 224, filter 226, maintenance package generator 228, solution predictor 230, remote command executor 232, technician selector 234, weight multiplier 236, technician database 238, machine identifier 240, model generator 242, survey generator 224, information updater 246, database interface 246, and / or more generally... Figure 2Any of the machine recovery systems 102 may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field-programmable logic devices (FPLDs). When any of the device or system claims of this patent is read to cover purely software and / or firmware implementations, at least one of the following—device interface 212, database interface 214, symptom processor 224, filter 226, maintenance package generator 228, solution predictor 230, remote instruction executor 232, technician selector 234, weight multiplier 236, technician database 238, machine identifier 240, model generator 242, survey generator 224, information updater 246, and / or database interface 246—is thereby explicitly defined as including non-transitory computer-readable storage devices or storage disks, such as memory, digital versatile optical discs (DVDs), optical discs (CDs), Blu-ray discs, etc., including software and / or firmware. Furthermore, Figure 2 The machine recovery system 102 may include one or more components, processes and / or devices, as Figure 2 The elements, processes, and devices shown may be supplemented or substituted for, and / or may include one or more of any one or all of the shown elements, processes, and devices. As used herein, the phrase “to communicate” includes its variant forms, covering direct communication and / or indirect communication via one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or constant communication, but additionally includes selective communication at periodic intervals, predetermined intervals, non-periodic intervals, and / or one-off events.

[0096] exist Figures 6 to 10 The diagram illustrates exemplary hardware logic, machine-readable instructions, a state machine for hardware implementation, and / or methods for implementation. Figure 2 A flowchart of any combination of machine recovery system 102. Machine-readable instructions may be an executable program or part of an executable program, for use by a computer processor (such as those combined below). Figure 11 The processor platform 800 discussed herein (processor 1112) executes the program. The program may be embodied in software stored on a non-transitory computer-readable storage medium (such as a CD-ROM, floppy disk, hard disk drive, DVD, Blu-ray disc, or memory associated with processor 1112), but the entire program and / or portions thereof may alternatively be executed by a device other than processor 1112 and / or embodied in firmware or dedicated hardware. Furthermore, although references... Figures 6 to 10The flowchart shown illustrates an exemplary procedure, but many other methods for implementing machine recovery system 102 may be used alternatively. For example, the execution order of the blocks may be changed, and / or some blocks in the described blocks may be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers, logic circuits, etc.) configured to perform the corresponding operation without executing software or firmware.

[0097] As mentioned above, Figures 6 to 10 The exemplary process can be implemented using executable instructions (e.g., computer and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium, such as a hard disk drive, flash memory, read-only memory, optical disk, digital universal optical disk, cache, random access memory, and / or any other storage device or disk for storing information for any duration (e.g., extended storage period, permanent storage, for transient situations, for temporary buffering, and / or for information caching). As used herein, the term non-transitory computer-readable medium is explicitly defined to include any type of computer-readable storage device and / or disk, excluding propagation signals and transmission media.

[0098] "Comprising" and "including" (and all their forms and tenses) are used herein as open-ended terms. Therefore, whenever a claim uses any form of "comprising" or "including" (e.g., comprising, comprising, including, having, etc.) as a preamble or within any kind of claim statement, it should be understood that additional elements, terms, etc., may be present that do not fall outside the scope of the corresponding claim or statement. As used herein, the phrase "at least" is open-ended, as are the terms "comprising" and "including". When used, for example, in the form of A, B, and / or C, the term "and / or" refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, and (7) A and B and C. As used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the execution or implementation of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A and B" is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing the execution or implementation of processes, instructions, actions, activities and / or steps, the phrase “at least one of A or B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0099] Figure 6 Including the indication that it can be generated by Figure 2 A flowchart 600 shows the machine-readable instructions implemented by the structural components of the machine recovery system 102. Although combined with Figure 1 and / or Figure 2 The machine recovery system 102 describes Figure 6 The flowchart is 600, but other types of recovery systems and / or other types of processors can also be used instead.

[0100] At box 602, the intelligent symptom system 202 of the machine recovery system 102 executes the intelligent symptom process. (See the following text for details.) Figure 7 Furthermore, the intelligent symptom system 202 translates the machine's symptoms into problem and / or error codes to serve the machine. At box 604, the intelligent maintenance package system 204 of the machine recovery system 102 executes the intelligent maintenance package process. This is further elaborated below. Figure 8 Furthermore, the intelligent maintenance package system 204 generates an intelligent maintenance package for the technician's equipment 108 to serve the machine 104. At block 606, the intelligent scheduling system 206 of the machine recovery system 102 executes an intelligent scheduling process. This is further elaborated below. Figure 9 Furthermore, the intelligent scheduling system 206 performs intelligent scheduling to select one or more technicians to provide services for the machine.

[0101] At box 608, the intelligent discovery system 208 of the machine recovery system 102 determines whether the intelligent discovery process has been performed within one of the previous processes (e.g., intelligent symptoms, intelligent maintenance packages, and / or intelligent scheduling processes). If the intelligent discovery process was not performed during the initial process, intelligent discovery may be performed in conjunction with a technician serving the request. For example, a technician may enter information into technician device 108 during service to identify the machine. When the intelligent discovery system 208 determines that the intelligent discovery process has been performed (box 608: Yes), the process continues to box 612. When the intelligent discovery system 208 determines that the intelligent discovery process has not been performed (box 608: No), the intelligent discovery system 208 performs the intelligent discovery process (box 610), as described below. Figure 10 Further details are provided.

[0102] At box 612, survey generator 244 determines whether a service request has been completed. For example, when a technician completes a service request, the technician may recognize completion via technician device 108, which transmits a completion trigger via device interface 212. Survey generator 244 determines that the service request has been completed in response to receiving the completion trigger. When survey generator 244 determines that the service request has not been completed (box 612: No), the process returns to box 612 until the service request is completed. When survey generator 244 determines that the service request is completed (box 612: Yes), survey generator 244 transmits the generated survey to technician device 108 via device interface 212 (box 614). Alternatively, a prompt may be preloaded into technician device 108. In such an example, survey generator 244 does not transmit the survey. Instead, survey generator 244 determines that the service request has been completed when feedback from the preloaded prompt is received at machine recovery system 102. At box 616, information updater 246 obtains feedback information based on survey results input by technicians (e.g., via device interface 212). At box 618, information updater 246 updates the problem solution information, intelligent maintenance package format, and / or provided information and / or artificial intelligence models (model / digital twin, neural network, etc.) in problem solution database 220 based on the feedback using an artificial network model performing machine learning. For example, Figure 8 The neural network 800 can be used to implement the digital twin 630, work with the digital twin 630 to provide diagnostic and / or repair recommendations, and / or replace the operation of the digital twin 630 to generate suggested problems based on input symptoms, provide suggested repairs based on input machine configuration and problem information, etc., update data in databases 216, 218, 220, 222, 238 and / or update the digital twin and / or optimize intelligent symptom processes, intelligent maintenance package processes, intelligent scheduling processes and / or intelligent discovery processes for future service requests.

[0103] Figure 7 Includes a flowchart 602 representing machine-readable instructions, which can be generated by... Figure 2 The structural components of the machine recovery system 102 are implemented to perform intelligent symptom processes. Although combined with Figure 1 and / or Figure 2 The machine recovery system 102 describes Figure 7 The flowchart is 602, but other types of recovery systems and / or other types of processors can also be used instead.

[0104] At block 702, the symptom processor 224 receives a service request from the user via device interface 212. For example, the user may call service center 106, which transmits the service request to device interface 212, or the user may enter the service request on a user interface on a computer or on machine 104 itself, which is transmitted directly or indirectly to device interface 212. At block 704, machine identifier 240 determines whether a machine identifier or imaging device identifier and / or context information is available to the machine. For example, when a user interacts with a user interface on machine 104 and machine 104 transmits a service request, machine 104 may include additional information including a machine / imaging device ID and / or context information.

[0105] If the machine identifier 240 determines that the machine / imaging device ID or context information is unavailable (box 704: No), the process continues to box 708. When the machine identifier 240 determines that the machine / imaging device ID or context information is available (box 704: Yes), the machine recovery system 102 executes the intelligent discovery process (box 706), as described below. Figure 10 Further described. At box 708, symptom processor 224 obtains one or more symptoms from the user via device interface 212. The one or more symptoms may be included in a service request. In some examples, the service request may not include any symptoms. At box 710, symptom processor 224 obtains error log (e.g., logs identifying one or more error codes) information from machine log database 218 corresponding to the machine. At box 712, filter 226 filters out issues that do not correspond to the error log and / or symptoms to generate a subset of issues that correspond to the error log and / or symptoms. Filter 226 obtains issues from issue database 227.

[0106] At box 714, symptom processor 224 identifies distinguishing symptoms of remaining problems. For example, symptom processor 224 may identify distinguishing symptoms to categorize problems into first problems corresponding to distinguishing symptoms and second problems not corresponding to distinguishing symptoms. Symptom processor 224 may identify the presence of one or more of the distinguishing features in machine 104, and filter 226 may filter out the first or second problem based on system feedback and / or user response to reduce the number of potential problems that technicians need to inspect / repair for machine 104.

[0107] At box 716, symptom processor 224 selects the distinguishing symptom. Symptom processor 224 may select the most distinguishing symptom (e.g., the symptom corresponding to approximately half of the remaining problems). At box 718, symptom processor 224 transmits a prompt based on the distinguishing symptom via device interface 212 to a machine, user, and / or user device (e.g., a computing device used by the user to transmit data corresponding to the machine). The prompt may be sent directly to machine 104 and / or service center 106 (e.g., which then instructs the machine, user, and / or user device based on the prompt). In some examples, the prompt is preloaded on technician device 108. The prompt requests a response from machine, user, and / or technician device 108 to identify the presence of the selected distinguishing symptom and / or to perform a process to check for the presence of the distinguishing symptom. For example, the prompt may include instructions to be automatically executed by the machine to provide additional feedback corresponding to the additional symptom (e.g., instructions to perform one or more processes and provide the results of one or more processes as feedback). Alternatively, the prompt may identify the distinguishing symptom to a user device (e.g., a computer or computing device) that prompts the user for additional feedback corresponding to the distinguishing symptom. In some examples, technician device 108 may obtain data directly or indirectly from machine 104 and include the obtained data in feedback (e.g., input and / or output data from different parts of machine 104 obtained before, during and / or after service).

[0108] At block 720, symptom processor 224 receives a response from a user (e.g., directly via machine 104 or via service center 106) and / or from a machine (e.g., no input from the user) via device interface 212. This response includes data related to the presence of a distinct symptom at the machine. At block 722, filter 226 filters out questions based on the response. For example, when the response indicates the presence of a distinct symptom at the machine, filter 226 filters out questions that do not correspond to the distinct symptom, and when the response indicates the absence of a distinct symptom at the machine, filter 226 filters out questions that correspond to the distinct symptom.

[0109] At box 724, symptom processor 224 determines whether more than a threshold number of questions remain. This threshold number of questions may be based on, for example, user and / or manufacturer preferences. When symptom processor 224 determines that no more than the threshold number of questions remains (box 724: No), the process returns to... Figure 6Box 604. When symptom processor 224 determines that more than the threshold number of problems remain (box 724: Yes), symptom processor 224 determines whether additional distinguishing symptoms remain (box 726). When symptom processor 224 determines that additional distinguishing symptoms remain (box 726: Yes), the process returns to box 716 for further problem filtering. When symptom processor 224 determines that no additional distinguishing symptoms remain (box 726: No), the process returns to... Figure 6 The box number is 604.

[0110] In addition to or alternatively, Figure 2 Symptom processor 224 can interact with machine 108 (e.g., directly using device interface 212 and / or indirectly via technician device 108) to obtain output data from machine 104 (e.g., photographs taken by machine 104). In such examples, symptom processor 224 can process images obtained by imaging equipment to identify artifacts (e.g., defects) found in the images. Symptom processor 224 can use the identified defects to identify problems, the source of the problem, and / or additional symptoms. For example, symptom processor 224 can determine, based on the identified defects, insufficient radiation, excessive radiation, a faulty detector, misalignment issues, etc. In some examples, symptom processor 224 can compare the identified problem, the source of the problem, and / or symptoms associated with the defect with previous problem solution information (e.g., stored in exemplary problem solution database 220) to determine whether the problem corresponds to a configuration and / or design defect. In such examples, symptom processor 224 can transmit alerts to machine 108, users of machine 108 (e.g., via text, email, etc.), system administrators, etc., to identify design defects. Additionally, the machine identifier 240 can update group information and / or generate alerts to similar machines to identify design flaws.

[0111] Figure 8 Includes a flowchart 604 representing machine-readable instructions, which can be generated by... Figure 2 The structural components of the machine recovery system 102 are implemented to perform the intelligent maintenance package process. Although combined with Figure 1 and / or Figure 2 The machine recovery system 102 describes Figure 8 The flowchart is 604, but other types of recovery systems and / or other types of processors can also be used instead.

[0112] At box 800, maintenance package generator 225 obtains error codes (e.g., error codes from machine log database 218 via database interface 214) and / or compensation issues from symptom processor 224. In some examples, machine log 218 may include machine identifiers and / or contextual information that can be used to perform the smart discovery process. At box 802, machine identifier 240 determines whether smart discovery should be performed. For example, machine identifier 240 may perform the smart process if the smart discovery process has not yet been performed and the error codes provide additional information (e.g., machine identifiers and / or specific contextual information) for the smart discovery process to be performed.

[0113] When machine identifier 240 determines that smart discovery will not be performed (box 802: No), the process continues to box 806. When machine identifier 240 determines that the smart discovery process should be performed (box 802: Yes), machine recovery system 102 performs the smart discovery process (box 804), as described below. Figure 10 Further described. At box 806, maintenance package generator 228 generates a maintenance package based on a template layout. The initial template layout may be based on the preferences of technicians and / or manufacturers and may be customizable. At box 808, maintenance package generator 228 accesses information from databases 216, 220, and 222 via database interface 214. For example, when machine 104 has been identified, maintenance package generator 228 may access (A) contextual information (e.g., customer site and layout information, contract guarantees, asset locations, historical machine information, etc.) from enterprise system database 216, (B) problem solution data corresponding to error codes and / or identified problems from problem solution database 220, and / or (C) system health status information corresponding to machine 104 from dynamic system health status database 222.

[0114] At box 810, maintenance package generator 204 determines whether the model / digital twin corresponding to machine 104 is available. For example, if an intelligent discovery process has been performed, the model / digital twin corresponding to machine 104 has been identified. Alternatively, the model / digital twin can be selected based on error codes, identified problems, and / or accessed information. When maintenance package generator 204 determines that the model / digital twin corresponding to machine 104 is unavailable (box 810: No), the process continues to box 814. When maintenance package generator 204 determines that the model / digital twin corresponding to machine 104 is available (box 810: Yes), solution predictor 230 tests the corresponding model / digital twin based on error codes, identified problems, and / or accessed information to identify potential solutions to the machine problem.

[0115] For example, digital twins of x-ray imaging equipment and / or components of x-ray imaging equipment (e.g., digital twins of x-ray detectors, etc.) can be processed based on symptoms and / or other information. Digital twins model symptoms to identify associated system problems and / or model problems to simulate one or more solutions and associated results to help determine, for example, solutions or “repairs” to be applied to improve the symptoms exhibited by machine 104. Since a digital twin is a physically-based model or replica of the actual machine 104 and its environment reproduced in virtual space, the effects on the digital twin should indicate similar effects on the actual machine 104.

[0116] Alternatively or otherwise, deep learning networks (such as deep learning network 500) can be used to take symptoms, problems, configuration information of machine 104, etc. as input, and correlate these inputs with resource constraints, customer needs, machine limitations, etc., to determine, for example, the proposed solution to solve the problem.

[0117] At box 814, solution predictor 230 predicts a solution based on test and / or problem solution data from problem solution database 220 corresponding to the obtained error codes, identified problems, and / or accessed information from an artificial intelligence model (e.g., model / digital twin, neural network, etc.). At box 816, solution predictor 230 determines whether at least one solution can be predicted. If at least one solution cannot be predicted (box 816: No), the process continues to box 820. If at least one solution can be predicted (box 816: Yes), maintenance package generator 228 adds the predicted solution information to a maintenance package (box 818). For example, maintenance package generator 228 may add a section corresponding to the predicted solution's manual / tutorial.

[0118] At box 820, maintenance package generator 228 determines whether there are tools corresponding to error codes, problems, and / or predicted solutions for specialized tools. For example, each technician may carry or need to carry certain tools, and may be allowed not to carry specialized tools that are either too large to be carried consistently, rarely used, and / or too expensive for the company to purchase for each technician. Therefore, some specialized tools may be located at predetermined locations where technicians can obtain them when needed. Thus, it is preferable to know whether a service request will require a specialized tool so that the technician can carry the tool upon arrival, rather than discovering he / she needs it after arriving at the service location. The correspondence between specialized tools and error codes, problems, and / or predicted solutions may be stored in problem solution database 220.

[0119] If maintenance package generator 228 determines that one or more tools corresponding to the error code, problem, and / or predicted solution are not specialized tools (box 820: No), the process continues to box 824. When maintenance package generator 228 determines that one or more tools corresponding to the error code, problem, and / or predicted solution are specialized tools (box 820: Yes), maintenance package generator 228 adds a specialized tool identifier to the maintenance package (box 822). Thus, the technician knows that he / she may need to obtain the specialized tool for the service request. At box 824, maintenance package generator 228 adds other relevant information and / or portions of machine manuals / tutorials to the maintenance package based on the error code, problem, possible replacement parts, site location information (e.g., a map and / or directions to the site location), and / or predicted solution.

[0120] The final maintenance package is a data structure containing relevant customer data, machine configuration information, executable instructions / code, etc., which can be used by resources to resolve and / or identify problems with machine 104. When a problem can be resolved remotely (e.g., automatically resolved by transmitting instructions to be executed by machine 104), the maintenance package may include software instructions, patches, etc., which can be transmitted to machine 104 for repair and / or to provide additional data corresponding to the problem. Additionally, the maintenance package may include software that, when executed by machine 104, identifies whether the problem and / or other issues have been resolved. When the maintenance package is being transmitted to a technician's equipment 108, it is a data structure containing relevant and customized data that the technician may need to avoid unnecessary downtime, costs, etc. For example, the maintenance package allows the technician to identify replacement parts, special tools, etc., required for requested service before arriving at the location. Furthermore, the maintenance package has identified relevant sections of the service manual, saving the technician time and effort in finding the correct manual information corresponding to the machine and / or problem. Additionally, the maintenance package eliminates the need for new technicians to search for incorrect information when servicing machine 104. Additionally, when the maintenance package includes a digital twin, technicians can avoid further damage to the machine and / or waste time on unsuccessful service techniques by first applying the technology to the digital twin.

[0121] Figure 9 Includes a flowchart 606 representing machine-readable instructions, which can be generated by... Figure 2 The structural components of the machine recovery system 102 are implemented to perform an intelligent scheduling process. Although combined with Figure 1 and / or Figure 2 The machine recovery system 102 describes Figure 9 The flowchart is 606, but other types of recovery systems and / or other types of processors can also be used instead.

[0122] At box 902, remote instruction executor 232 determines whether one or more of the identified solutions correspond to one or more remote solutions (e.g., solutions that can be resolved via them). For example, problem solution database 220 may include information related to previous problems that have been resolved or can be resolved remotely via instructions. Therefore, remote instruction executor 232 can access problem solution database 220 via database interface 214 using the identified solutions to determine whether one or more of the identified solutions can be resolved remotely. When remote instruction executor 232 determines that one or more solutions do not correspond to remote solutions (box 902: No), the process continues to box 908. When remote instruction executor 232 determines that one or more solutions correspond to remote solutions (box 902: Yes), remote instruction executor 232 (e.g., directly) executes the remote solution to machine 104 and / or as part of a maintenance package to a technician (box 904). For example, remote instruction executor 232 may transmit software patches, reboot instructions, and / or other remote instructions to attempt to resolve one or more problems affecting the machine. Alternatively, remote instruction executor 232 may alert a technician to potential remote instruction solutions via a maintenance package. In such examples, technicians attempt to remotely transmit instructions to machine 104 in an attempt to resolve the problem.

[0123] At box 906, remote command executor 232 determines whether the problem with machine 104 has been resolved. For example, remote command executor 232 may determine that the problem with machine 104 has been resolved based on direct network communication with machine 104 and / or based on a completion trigger and / or any other trigger from technician device 108 that identifies the completion of a service request. When remote command executor 232 determines that the problem has been resolved (box 906: Yes), the process returns to... Figure 6 Box 608. If the remote instruction executor 232 determines that the problem has not been resolved (box 906: No), the process continues to box 908.

[0124] At box 908, technician selector 234 determines whether the identified error codes and / or problems correspond to critical errors. Error codes and / or problems may be identified as critical errors based on user / manufacturer / customer preferences. For example, when a customer has two identical machines, any error code from the other machine is not critical as long as one machine is operational. In another example, any error code and / or problem that causes machine 104 to shut down completely may correspond to a critical error. When technician selector 234 determines that the identified error codes and / or problems do not correspond to critical errors (box 908: No), the process continues to box 912. When technician selector 234 determines that the identified error codes and / or problems correspond to critical errors (box 908: Yes), technician selector 234 generates a set of available technicians by filtering technicians in technician database 238 who are not available to service critical errors within a threshold time amount (box 910). For example, technician selector 234 can analyze technician schedules to determine which technicians are immediately available (e.g., based on their schedules and their distance to the service location, etc., they can serve the request within a threshold timeframe). The distance to the service location may include a first distance from the technician's current location to a first location used to obtain specialized parts and / or replacement parts, and a second distance from the first location to the service location.

[0125] At box 912, technician selector 234 examines the clinical workflow corresponding to a machine. A clinical workflow corresponds to a set of machines used to complete a specific task. For example, other machines in the clinical workflow may be evaluated to determine whether to service another machine (e.g., whether it should be checked, whether it is part of a problem, etc.) and / or whether the problem originates from another machine. For example, when the output of the first machine is used in the second machine, where a problem is identified, that problem may result in invalid input. In such an example, even if a problem is identified in the second machine, the actual problem may originate from the first machine. Therefore, it may be desirable to select technicians who can service multiple machines, or multiple technicians to service multiple machines in the clinical workflow, to avoid dispatching different technicians at different times.

[0126] At box 914, technician selector 234 determines whether additional machines at the site location require service (e.g., whether other machines in the building or based on the clinical workflow should be checked or also correspond to errors / problems). When technician selector 234 determines that no additional machines at the site location require service (box 914: No), the process continues to box 918. When technician selector 234 determines that additional machines at the site location require service (box 914: Yes), weight multiplier 236 adjusts the preset weights of available technicians based on the number of additional machines a technician can service (box 916). For example, when technician selector 234 identifies three machines that need / should be serviced, a higher weight is applied to technicians who can service all three machines, and a lower weight is applied to technicians who can service only one machine. When a technician cannot service any machine, a zero weight is applied, thus filtering out that technician unless they are the only available technician.

[0127] At box 918, weight multiplier 236 adjusts the weights of available technicians based on skill level, tools, distance, availability, etc. In some examples, the weight multiplier may utilize a neural network (e.g., Figure 5 The exemplary neural network 500 generates weights based on inputs including skill level, tools, distance, etc. The number of adjustments and / or the strength of the weights may be based on user / manufacturer / customer preferences. For example, when time is a high priority for the customer, distance and availability may receive higher weights than skill level. At box 920, technician selector 234 determines the number of technicians required to servicing the machine at the site location. The number of technicians may be based on contracts with customers, technician availability, the complexity of the machine problem, and / or the number of machines requiring servicing. At box 922, technician selector 234 selects X highest-weighted technicians, where X corresponds to the number determined in box 920. At box 924, technician selector 234 transmits an indication of a service request along with a corresponding maintenance package to the selected technicians via device interface 212.

[0128] Figure 10 Including flowcharts 610, 706, and 802 representing machine-readable instructions, which can be generated by... Figure 2 The structural components of the machine recovery system 102 are implemented to perform the intelligent discovery process. Although combined with Figure 1 and / or Figure 2 The machine recovery system 102 describes Figure 10 The flowcharts 610, 706, and 802 are used, but other types of recovery systems and / or other types of processors can also be used instead.

[0129] At box 1002, machine identifier 240 determines whether a machine ID is available. The machine ID may be provided by a user, the machine, a technician (e.g., when servicing the machine), and / or may be stored in one or more databases in databases 216, 218, 220, and 222 in conjunction with machine 104. When machine identifier 240 determines that a machine ID is unavailable (box 1002: No), machine identifier 240 uses information obtained about machine 104 and compares the obtained information (e.g., contextual information) with information stored in enterprise system database 216 to identify the machine (box 1004). In some examples, the obtained information may include one or more photographs of machine 104. In such examples, machine identifier 240 utilizes image processing techniques to determine different features of machine 104 and / or the location of machine 104, and compares the processed photographs and / or different features with reference data and / or reference location information to identify machine 104. In some examples, machine identifier 420 may utilize neural networks (e.g., Figure 5 The exemplary neural network 500 identifies machines based on contextual information.

[0130] If machine identifier 240 determines that a machine ID is available (box 1002: Yes), then machine identifier 240 identifies machine 104 based on the machine ID (box 1006). At box 1008, machine identifier 240 determines the brand, model, modality, and / or other information of the identified machine. For example, machine identifier 240 may access enterprise system database 216 via database interface 214 to obtain information corresponding to the identified machine. At box 1010, model generator 242 determines whether a corresponding group exists for the brand, model, modality, and / or other information of the identified machine. Machine groups have similar brands, models, modalities, and / or other information. Information from machines in a group can be used to diagnose, predict problems, and / or serve other machines in the group. Additionally, the group corresponds to an artificial intelligence model (e.g., a model, digital twin, and / or neural network) customized based on information obtained from the group. In some examples, a machine may correspond to multiple groups. Group information may be stored in enterprise system database 216.

[0131] If model generator 242 determines that a corresponding machine group does not exist (box 1010: No), model generator 242 selects a template AI model (e.g., model / digital twin, neural network, etc.) based on brand, modality, and / or model (box 1012). At box 1014, model generator 242 generates an AI model (e.g., model / digital twin, neural network, etc.) for machine 104 by modifying the template AI model (e.g., model / digital twin, neural network, etc.) with modality and / or other available information corresponding to the machine. When model generator 242 determines that a corresponding machine group exists (box 1010: Yes), model generator 242 selects an AI model (e.g., model, digital twin, neural network, etc.) for the corresponding group (box 1016). At box 1018, model generator 242 deploys the AI ​​model (e.g., model, digital twin, neural network, etc.) for the corresponding group to the technician's device 108 (e.g., which may be included in a smart package) via device interface 212. Technicians can use artificial intelligence models (e.g., models, digital twins, neural networks, etc.) as a test machine before / during the service of machine 104.

[0132] At box 1020, information updater 246 stores, for example, error information (e.g., error code, identified problem, etc.) of the identified machines corresponding to the group (e.g., stored in enterprise system database 216 via database interface 214). Using this error information, the group can track errors / problems of machines within the group to conduct further analysis and / or predict future errors within the group. At box 1022, information updater 246 determines whether other machines in the group correspond to similar errors / problems. When information updater 246 determines that other machines in the group correspond to similar errors / problems, it marks the error / problem as a common error associated with contextual information (e.g., the time the error occurred, the number of executions prior to the error, the amount of time between the last service and the error, the machine's age, location information, etc.). At box 1026, device interface 212 transmits a warning corresponding to this marking for machines with similar contextual information. For example, if machine 104 experiences an error after five years of operation, device interface 212 can transmit a warning to other machines that have been operating for more than four years. Device interface 212 can transmit warnings to the device, the device's user (e.g., via email, text message, etc.), the customer, the administrator of machine recovery system 102, service center 106, and / or any other relevant party.

[0133] Figure 11 It is constructed to execute Figure 6 , Figure 7 , Figure 8 , Figure 9 and / or Figure 10Instructions to achieve Figure 2 A block diagram of the processor platform 1100 of the machine recovery system 102. The processor platform 1100 can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), an internet device, or any other type of computing device.

[0134] The processor platform 1100 shown in the example includes a processor 1112. The processor 1112 shown in the example is hardware. For example, the processor 1112 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired product family or manufacturer. The hardware processor can be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 1112 implements a device interface 212, a database interface 214, a symptom processor 224, a filter 226, a maintenance package generator 228, a solution predictor 230, a remote instruction executor 232, a technician selector 234, a weight multiplier 236, a technician database 238, a machine identifier 240, a model generator 242, a survey generator 244, an information updater 246, and the database interface 214.

[0135] The processor 1112 of the illustrated example includes local memory 1113 (e.g., cache). The processor 1112 of the illustrated example communicates via bus 1118 with main memory, which includes volatile memory 1114 and non-volatile memory 1116. The volatile memory 1114 may be synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), etc. Dynamic Random Access Memory And / or any other type of random access memory device. The non-volatile memory 1116 may be implemented by flash memory and / or any other desired type of memory device. Access to the main memory 1114, 1116 is controlled by the memory controller.

[0136] The processor platform 1100 shown in the example also includes interface circuitry 1120. Interface circuitry 1120 can be implemented using any type of interface standard, such as an Ethernet interface, Universal Serial Bus (USB), etc. Interface, Near Field Communication (NFC) interface, and / or PCI express interface.

[0137] In the example shown, one or more input devices 1122 are connected to interface circuitry 1120. Input devices 1122 allow users to input data and commands into processor 1112. One or more input devices may be implemented as, for example, an audio sensor, microphone, camera (still camera or video camera), keyboard, buttons, mouse, touchscreen, touchpad, trackball, isopoint, and / or a voice recognition system.

[0138] One or more output devices 1124 are also connected to the interface circuitry 1120 of the illustrated example. The output devices 1124 may be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), cathode ray tube displays (CRTs), in-place switching (IPS) displays, touchscreens, etc.), haptic output devices, printers, and / or speakers. Therefore, the interface circuitry 1120 of the illustrated example typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0139] The interface circuit 1120 of the example shown also includes communication devices such as transmitters, receivers, transceivers, modems, home gateways, wireless access points, and / or network interfaces to facilitate the exchange of data with external machines (e.g., any kind of computing device) via network 1126. Communication can be carried out via, for example, Ethernet connections, Subscriber Line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, line-to-line wireless systems, cellular telephone systems, etc.

[0140] The processor platform 1100 shown in the example also includes one or more mass storage devices 1128 for storing software and / or data. Examples of such mass storage devices 1128 include floppy disk drives, hard disk drives, optical disk drives, Blu-ray disc drives, redundant array of independent disks (RAID) systems, and digital universal optical disc (DVD) drives.

[0141] Figure 6 , Figure 7 , Figure 8 , Figure 9 and / or Figure 10 The machine-executable instructions 1132 may be stored in mass storage device 1128, volatile memory 1114, non-volatile memory 1116 and / or on a removable, non-transitory computer-readable storage medium, such as a CD or DVD.

[0142] Based on the foregoing, it should be understood that exemplary methods, apparatuses, and articles of art have been disclosed that provide novel, technologically advanced imaging modal maintenance, monitoring, and repair. The disclosed methods, apparatuses, and articles of art improve the efficiency of using computing devices by transforming a computing device into a diagnostic and repair tool to receive symptoms and / or configuration information of a machine (e.g., through data transmission, measurement, monitoring, etc.), identify problems with the machine and / or other connected components, model and drive the selection of solutions to the problems, and identify appropriate resources to execute the solutions to improve the symptoms / problems at the machine. Some examples provide simulations through modeling and / or network analysis and generate maintenance packages to be deployed to resolve the problems. Therefore, the disclosed methods, apparatuses, and articles of art relate to one or more improvements in computer functionality.

[0143] For example, results and observed behavior can be modeled to develop realistic, physically-based virtual counterparts of imaging modal devices, repair resources, environments, etc., and / or robust learning network models to generate solutions based on indications of machine problems. Some examples include the severity, immediacy, and scope of the problem, as well as the available tools, technicians, and / or other resources for generating packaged solutions to address problems in the target machine and / or other machines in its group, other devices in its environment, other components on its maintenance path / protocol, etc. Some examples generate repair or maintenance packages to diagnose and / or treat machine problems, rather than sending insufficient resources to resolve the problem, spending hundreds of dollars per day on travel and time, exacerbating the problem and costing more to resolve, resulting in greater machine unavailability. Solutions can be tailored to a specific machine in a specific environment and / or extended to similar machines in a location, a group, etc. While existing methods involve manual technician investigations, some examples provide customized, evolved solutions for specific machines with specific problems. For example, an MR machine used for 12 scans per day can be modeled and processed differently from another MR machine used for 30 scans per day. Machines can be compared to assign the same response and / or deviate from it to develop different responses for machines whose usage patterns differ from other machines in the group. Some examples provide group modeling, site modeling, workflow / protocol modeling, etc., to leverage the environment for the benefits of machine repair and to leverage the benefits of repairing one machine for the benefits of a larger environment.

[0144] While certain exemplary methods, apparatuses, and articles of manufacture have been disclosed herein, the scope of this patent is not limited thereto. Rather, this patent covers all methods, apparatuses, and articles of manufacture that reasonably fall within the scope of the claims of this patent.

Claims

1. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium comprising instructions, which, when executed, cause a machine to at least: The imaging device is identified based on at least one of contextual information or an imaging device identifier. Determine at least one of the brand, model, or modality of the imaging device; Determine whether an imaging device group exists corresponding to the imaging device, and if the imaging device does not belong to any imaging device group: The template AI model is selected based on at least one of the brand, model, or modality of the imaging device. The template artificial intelligence model is modified based on the modality of at least the imaging device; as well as Store error information corresponding to problems with the imaging device, and update the modified template AI model based on the error information; When there is an imaging device group corresponding to at least one of the brand, the model, or the modality, the error information corresponding to the problem of the imaging device and the group is stored, and the artificial intelligence model corresponding to the group is updated based on the error information; The modified template AI model, updated based on the error information, or the AI ​​model itself, is deployed to the technician's device. as well as When the number of times the problem has been identified in the group exceeds the threshold: The question is marked; and The warning corresponding to the flag is transmitted.

2. The computer-readable storage medium of claim 1, wherein the instructions cause the machine to identify the imaging device based on the context information by comparing the context information with data stored in a database, the stored data associating the context information with an imaging device identifier.

3. The computer-readable storage medium of claim 1, wherein when there is no group corresponding to the imaging device for at least one of the brand, the model, or the modality, the instructions cause the machine to generate a digital twin based on the brand, the model, or the modality.

4. The computer-readable storage medium of claim 1, wherein the instructions cause the machine to select a digital twin corresponding to the group.

5. The computer-readable storage medium of claim 4, wherein the instructions cause the machine to predict a solution to the problem by testing a digital twin.

6. The computer-readable storage medium of claim 4, wherein the instructions cause the machine to transmit the digital twin to the device of the technician.

7. The computer-readable storage medium of claim 1, wherein the warning includes the context information.

8. An apparatus comprising: Processor, the processor being used for: Identify the imaging device based on its contextual information. Based on the identification, at least one of the brand, model, or modality of the imaging device is determined; Determine that there is no group of imaging devices corresponding to the imaging device, and select a template artificial intelligence model based on at least one of the brand, model, or modality of the imaging device; The template artificial intelligence model is modified based on the modality of at least the imaging device; Generate a group corresponding to at least one of the brand, model, or modality of the imaging device; Store error information corresponding to the problem of the imaging device and the group; Update the modified template AI model corresponding to at least one of the brand of the imaging device, the model of the imaging device, the modality of the imaging device, or the error message; Deploy the modified template AI model onto the technician's device; The problem is flagged when the number of times the problem has been identified in the group exceeds a threshold. as well as The warning corresponding to the flag is transmitted.

9. The apparatus of claim 8, wherein the processor identifies the imaging device based on the context information by comparing the context information with data stored in a database, the stored data associating the context information with an imaging device identifier.

10. The apparatus of claim 8, wherein the processor predicts a solution to the problem by testing the modified template artificial intelligence model.

11. The apparatus of claim 10, wherein the modified template AI model is generated based on at least one of the brand, the model, or the modality.

12. The apparatus of claim 8, wherein the warning includes the context information.

13. A method, the method comprising: Image recognition based on the imaging device; Based on the identification, at least one of the brand, model, or modality of the imaging device is determined; Determine whether an imaging device group exists corresponding to the imaging device, and when the imaging device does not belong to any imaging device group: The template AI model is selected based on at least one of the brand, model, or modality of the imaging device. The template artificial intelligence model is modified based on the modality of at least the imaging device; Store error information corresponding to problems with the imaging device, and update the modified template AI model based on the error information; The modified template AI model, updated based on the error information, is deployed to the technician's device; as well as When the number of times the problem has been identified in the group exceeds a threshold, the problem is flagged and a warning corresponding to the flag is transmitted.

14. The method of claim 13, further comprising identifying the imaging device based on the context information by comparing the context information with data stored in a database, wherein the stored data associates the context information with an imaging device identifier.

Images