Digital Twin-enabled Machine Diagnostics Based on Acoustic Modeling

Abstract

One approach to digital twin-enabled equipment diagnostics based on acoustic modeling includes receiving real-time audio inputs of an asset from a mobile device, analyzing the real-time audio inputs using one or more acoustic modeling algorithms to establish deviations from a baseline associated with the asset's digital twin, and, in response to determining that the deviations from the baseline are greater than a predetermined threshold, repeatedly prompting a user to move the mobile device until a stopping criterion is met.

Description

[Technical Field]

[0001] The present invention relates generally to the field of data processing, and more particularly to digital twin enabled equipment diagnostics based on acoustic modeling. [Background technology]

[0002] A digital twin is a virtual representation of a physical object or system throughout its lifecycle. It uses real-time data and other sources to enable learning, inference, and dynamic recalibration to improve decision-making. This simply means creating a highly complex virtual model that is an exact counterpart (or twin) of a physical thing. The "thing" could be a car, a tunnel, a bridge, or even a jet engine. Connected sensors on the physical asset collect data that can be mapped to the virtual model. By looking at the digital twin, users can see important information about how the physical thing is operating in the real world.

[0003] Speech recognition is a technology based on both traditional pattern recognition theory and speech signal analysis methods. Typical speech recognition techniques include preliminary data processing, feature extraction, and classification algorithms. Feature vectors are generated as a result of preliminary data processing and linear predictive coding. Speech recognition then classifies these feature vectors. An acoustic fingerprint is a condensed digital summary generated deterministically from the speech signal and can be used to identify a speech sample or quickly locate similar items in a speech database. Summary of the Invention

[0004] Embodiments of the present invention disclose methods, computer program products, and systems for digital twin-enabled equipment diagnostics based on acoustic modeling. In one embodiment, real-time audio input of an asset is received from a mobile device. The real-time audio input is analyzed using one or more acoustic modeling algorithms to establish deviations from a baseline associated with the asset's digital twin. In response to determining that the deviation from the baseline is greater than a predetermined threshold, a user is repeatedly prompted to move the mobile device until stopping criteria are met. [Brief explanation of the drawings]

[0005] [Figure 1] 1 is a functional block diagram illustrating a distributed data processing environment, according to one embodiment of the present invention. [Figure 2] 2 is a flowchart illustrating operational steps of an acoustic diagnostic program for digital twin-enabled equipment diagnosis based on acoustic modeling on a computing device in the distributed data processing environment of FIG. 1 according to one embodiment of the present invention. [Figure 3] 2 is a block diagram of components of a computing device in the distributed data processing environment of FIG. 1 executing an acoustic diagnostic program, according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0006] Today's world is more connected than ever before. The advent of Internet of Things (IoT) sensors and the digitization of well-defined physical assets through digital twins are enabling innovations in the way engineers do their jobs. These inventions could enable techniques for engineers working with machinery to better diagnose malfunctions and other abnormal conditions, thus saving time and money. Every action a machine takes produces sound. What is needed is a way for machine operators to use existing mobile devices and advances in acoustic analysis technology to more accurately diagnose equipment based on changes in the sounds it emits. This invention provides engineers with the best possible data for troubleshooting digital twin-enabled equipment.

[0007] A digital twin is a virtual model of a process, product, or service. This pairing of the virtual and physical worlds allows for the analysis of data and monitoring of systems to prevent problems before they occur, prevent downtime, develop new opportunities, and even plan for the future using simulation. For example, a digital twin of an IoT device provides both the elements and dynamics of how the device will operate and survive throughout its lifecycle.

[0008] Digital twins can integrate IoT, artificial intelligence, machine learning, and software analytics to create living digital simulation models that update and change as their physical counterparts change. Digital twins continually learn and update themselves from multiple sources to represent their near-real-time situation, working state, or location. Digital twins also integrate historical data from past machine use into their digital models.

[0009] The present invention allows users in this field to inspect specific components of a machine using their digital twin counterparts. The invention records sounds from the equipment or asset, analyzes the sound quality in real time, and compares the acoustic signature with the asset's digital twin. If the acoustic quality is not optimal, the invention determines the optimal recording location and prompts the user to relocate the recording device accordingly. When optimal acoustic quality is achieved, the invention detects anomalies in the sounds from the asset based on the digital twin and notifies the user of the part of the machine that is likely causing the malfunction.

[0010] FIG. 1 is a functional block diagram illustrating a distributed data processing environment, generally designated 100, suitable for operation of an acoustic diagnostic program 112 in accordance with at least one embodiment of the present invention. As used herein, the term "distributed" refers to a computer system that includes multiple physically separate devices operating together as a single computer system. FIG. 1 illustrates only one embodiment and is not intended to imply any limitations with respect to the environments in which different embodiments may be implemented. Those skilled in the art will recognize that numerous modifications to the depicted environment may be implemented without departing from the scope of the present invention as defined by the appended claims.

[0011] Distributed data processing environment 100 includes computing device 110 and user device 130, both connected to network 120. Network 120 may be, for example, a telecommunications network, a local area network (LAN), a wide area network (WAN), such as the Internet, or a combination of these three networks, and may include wired, wireless, or fiber optic connections. Network 120 may include one or more wired and / or wireless networks capable of transmitting and receiving data, voice, or video signals, or combinations thereof, including multimedia signals containing voice, data, and video information. In general, network 120 may be any combination of connections and protocols that support communication between computing device 110, user device 130, and other computing devices (not shown) in distributed data processing environment 100.

[0012] Computing device 110 may be a standalone computing device, an administrative server, a web server, a mobile computing device, or any other electronic device or computing system capable of receiving, transmitting, and processing data. In one embodiment, computing device 110 may be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smart phone, or any programmable electronic device capable of communicating with other computing devices (not shown) in distributed data processing environment 100 via network 120. In another embodiment, computing device 110 may represent a server computing system that utilizes multiple computers as a server system, for example, in a cloud computing environment. In yet another embodiment, computing device 110 represents a computing system that utilizes clustered computers and components (e.g., database server computers, application server computers, etc.) that act as a single pool of seamless resources when accessed within distributed data processing environment 100.

[0013] In one embodiment, computing device 110 includes acoustic diagnostic program 112. In one embodiment, acoustic diagnostic program 112 is a program, application, or subprogram of a larger program for digital twin-enabled device diagnosis based on acoustic modeling. In alternative embodiments, acoustic diagnostic program 112 may reside on any other device accessible to computing device 110 via network 120.

[0014] In one embodiment, computing device 110 includes information repository 114. In one embodiment, information repository 114 may be managed by acoustic diagnostic program 112. In an alternative embodiment, the device's operating system, alone or together with acoustic diagnostic program 112, may manage information repository 114. Information repository 114 is a data repository capable of storing, collecting, comparing, and / or combining information. In some embodiments, information repository 114 is located external to computing device 110 and is accessed via a communications network, such as network 120. In some embodiments, information repository 114 is stored on computing device 110. In some embodiments, information repository 114 may be located on another computing device (not shown), provided that computing device 110 has access to information repository 114. Information repository 114 includes, but is not limited to, acoustic data, digital twin data, system data, user data, and other data received by acoustic diagnostic program 112 from one or more sources, as well as data generated by acoustic diagnostic program 112.

[0015] In one embodiment, information repository 114 may further include a digital twin repository, which may be a separate repository from information repository 114, provided that computing device 110 has access to the digital twin repository.

[0016] The information repository 114 may be implemented using any volatile or non-volatile storage medium known in the art for storing information. For example, the information repository 114 may be implemented using a tape library, an optical library, one or more independent hard disk drives, multiple hard disk drives in a redundant array of independent disks (RAID), a solid-state drive (SSD), or random access memory (RAM). Similarly, the information repository 114 may be implemented using any suitable storage architecture known in the art, such as a relational database, a NoSQL database, an object-oriented database, or one or more tables.

[0017] User device 130 may be a smart phone, a standalone computing device, a mobile computing device, or any other electronic device or computing system capable of capturing audio and receiving, transmitting, and processing data. In one embodiment, user device 130 may be a smart phone, a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), or any programmable electronic device capable of capturing audio and communicating over network 120 with other computing devices (not shown) in distributed data processing environment 100.

[0018] 2 is a flowchart of a workflow 200 illustrating the operational steps of acoustic diagnostic program 112 for digital twin-enabled equipment diagnosis based on acoustic modeling, in accordance with at least one embodiment of the present invention. In alternative embodiments, other programs may perform the steps of workflow 200 while operating in acoustic diagnostic program 112.

[0019] In one embodiment, the acoustic diagnostic program 112 is initiated by a user to analyze an asset to assess the health of, for example, a mechanical component. In one embodiment, the acoustic diagnostic program 112 uses known acoustic modeling algorithms to establish how much a current recording deviates from a baseline established from previous recordings. In one embodiment, the acoustic diagnostic program 112 utilizes the asset's digital twin (e.g., schematics, 3D models, and previous sound profiles) to determine whether anomalies exist in the sound recording. In one embodiment, the acoustic diagnostic program 112 determines whether the location of the device recording the sound is optimal for accurately detecting the sound. In one embodiment, if the acoustic diagnostic program 112 determines that the location of the device recording the sound is not optimal for accurately detecting the sound, the acoustic diagnostic program 112 instructs the user on where to reposition the recording device to obtain an optimal sound recording. In one embodiment, the acoustic diagnostic program 112 provides an enhanced sound recording that isolates specific abnormal sounds. In one embodiment, the acoustic diagnostic program 112 stores the sound data in a digital twin data warehouse. In one embodiment, the acoustic diagnostic program 112 displays the results of the analysis.

[0020] It should be appreciated that embodiments of the present invention provide, at a minimum, digital twin-enabled equipment diagnostics based on acoustic modeling. However, FIG. 2 illustrates only one embodiment and is not intended to imply limitations with respect to the environments in which different embodiments may be implemented. Those skilled in the art will appreciate that many modifications to the depicted environment may be implemented without departing from the scope of the present invention as defined by the appended claims.

[0021] The acoustic diagnostic program 112 initiates an audio recording session (step 202). In one embodiment, the acoustic diagnostic program 112 is initiated by a user to analyze an asset, for example, to assess the health of a mechanical component. In one embodiment, the acoustic diagnostic program 112 initiates the audio recording session in response to a request from a user. In another embodiment, the acoustic diagnostic program 112 initiates the audio recording session when a user opens an app on a mobile device, for example, the user device 130 of FIG. 1 .

[0022] The acoustic diagnostic program 112 analyzes the captured audio in real time (step 204). In one embodiment, the acoustic diagnostic program 112 uses known acoustic modeling algorithms to establish how much the current recording deviates from a baseline established from previous recordings. For example, the acoustic diagnostic program 112 uses a fast Fourier transform or a discrete Fourier transform to convert the recording into a series of frequencies contained throughout the recording. In various embodiments, the acoustic diagnostic program 112 can use a sequence-to-sequence autoencoder model or a convolutional long short-term memory autoencoder to establish how much the current recording deviates from the baseline. In one embodiment, the acoustic diagnostic program 112 retrieves the baseline established from previous recordings from a repository of the asset's acoustic data stored in the knowledge repository 114. In some embodiments, deviation from a baseline established from previous recordings can be as simple as detecting a frequency not previously detected in the previous audio recording, for example, a high-pitched grinding noise within an otherwise low-frequency hum.

[0023] The acoustic diagnostic program 112 compares the captured audio to the digital twin (step 206). In one embodiment, the acoustic diagnostic program 112 utilizes the asset's digital twin (e.g., schematics, 3D models, and previous sound profiles) to determine if there are any anomalies in the audio recordings. In one embodiment, the acoustic diagnostic program 112 establishes a baseline associated with the digital twin. For example, the digital twin will know the location of any internal parts, as well as the acoustic "signature" of the part's ideal operating state and predetermined thresholds for when there are anomalies.

[0024] The acoustic diagnostic program 112 determines whether the position is optimal (decision block 208). In one embodiment, the acoustic diagnostic program 112 determines whether the position of the device recording the sound is optimal for accurately detecting the sound. In one embodiment, the acoustic diagnostic program 112 uses a predetermined threshold to determine deviation from the baseline established in the previous step. In one embodiment, the acoustic diagnostic program 112 determines whether the deviation from the baseline is greater than a predetermined threshold, and if the deviation from the baseline is greater than the predetermined threshold, the acoustic diagnostic program 112 repeatedly prompts the user for the optimal position for acquiring the sound. In one embodiment, the threshold is a system default value. In another embodiment, the threshold is established by a user of the acoustic diagnostic program 112.

[0025] In one embodiment, the acoustic diagnosis program 112 utilizes the asset's digital twin (e.g., schematics, 3D models, and previous sound profiles) to determine whether the recording location is optimal based on several factors. In various embodiments, these factors include the required proximity to internal components that are commonly prone to errors, the machine's dimensions, weight, and safety information, anomalies in the audio recording—for example, anomalies that are more prominent in the left channel, meaning the mobile device's microphone is positioned to the right of one that may be causing a malfunction—and a comparison of the received audio clip with previous recordings to match likely problem areas. In one embodiment, if the acoustic diagnosis program 112 determines that the device recording the audio is optimal for accurately detecting sound (the "Yes" branch of decision block 208), the acoustic diagnosis program 112 proceeds to step 212.

[0026] The acoustic diagnosis program 112 instructs the user on the optimal location (step 210). In one embodiment, if the acoustic diagnosis program 112 determines that the location of the device recording the audio is not optimal for accurately detecting sound (the "No" branch of decision block 208), the acoustic diagnosis program 112 instructs the user on where to reposition the recording device to obtain an optimal audio recording. In one embodiment, the acoustic diagnosis program 112 sends movement instructions to the user device to direct the user to the incomplete area to capture additional audio recordings. For example, the acoustic diagnosis program 112 can send arrows to the screen of the user device to instruct the user on the direction the user should move to capture more audio recordings. In another embodiment, the acoustic diagnosis program 112 directs the user to the incomplete area to capture additional audio recordings by sending the user an image of the asset with the recording location highlighted. In yet another embodiment, the acoustic diagnosis program 112 directs the user to the area to capture additional audio recordings using any appropriate notification method as would be known to one of ordinary skill in the art.

[0027] In one embodiment, acoustic diagnostic program 112 establishes stopping criteria. In various embodiments, the stopping criteria may include detecting that characteristics of the real-time audio input reflect a previously identified problem seen in the asset's digital twin, determining that a required proximity to a known error-prone location of the asset has been reached, or detecting that characteristics of the real-time audio reflect a new problem not seen in the asset's digital twin. When acoustic diagnostic program 112 determines that the stopping criteria have been met, acoustic diagnostic program 112 informs the user that the current location is the optimal location for capturing audio.

[0028] The acoustic diagnostic program 112 provides an enhanced audio recording (step 212). In one embodiment, the acoustic diagnostic program 112 provides an enhanced audio recording that isolates specific abnormal sounds. In one embodiment, the acoustic diagnostic program 112 uses other acoustic data from audio recordings at multiple locations and existing acoustic processing methods to create the enhanced audio recording. In one embodiment, the acoustic diagnostic program 112 combines the different recordings taken (audio recordings at multiple locations) to create the best recording for storage and evaluation, e.g., stereo vs. mono.

[0029] The acoustic diagnostic program 112 saves the audio data in a digital twin warehouse (step 214). In one embodiment, the acoustic diagnostic program 112 stores the audio data in the knowledge repository 114. In one embodiment, the acoustic diagnostic program 112 stores the audio data in a digital twin data warehouse that is part of the knowledge repository 114. In one embodiment, the acoustic diagnostic program 112 feeds the audio data into a machine learning model that aids in future diagnostic sessions on the same equipment as the unit under test.

[0030] The acoustic diagnostic program 112 displays the system output (step 216). In one embodiment, the acoustic diagnostic program 112 displays the results of the analysis to the user. In one embodiment, the acoustic diagnostic program 112 displays the results of the analysis on a user device, such as user device 130 of FIG. 1. In one embodiment, the user uses this information to determine whether the asset is malfunctioning or whether further diagnosis is needed to confirm the malfunction. If a new issue is determined to be causing the asset to malfunction, the digital twin is updated to include information about the new issue. The acoustic diagnostic program 112 then completes the cycle.

[0031] Figure 3 is a block diagram illustrating components of a computing device 110 suitable for an acoustic diagnostic program 112 in accordance with at least one embodiment of the present invention. Figure 3 illustrates a computer 300, one or more processors 304 (including one or more computer processors), a communications fabric 302, memory 306 including random access memory (RAM) 316 and cache 318, persistent storage 308, a communications unit 312, an I / O interface 314, a display 322, and external devices 320. It should be understood that Figure 3 is illustrative of only one embodiment and does not imply limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be implemented.

[0032] As shown, computer 300 operates via communications fabric 302, which provides communications between computer processor 304, memory 306, persistent storage 308, communications unit 312, and I / O interface 314. Communications fabric 302 may be implemented with any suitable architecture for communicating data or controlling information between processor 304 (e.g., a microprocessor, communications processor, and network processor), memory 306, external device 320, and any other hardware components in the system. For example, communications fabric 302 may be implemented with one or more buses.

[0033] Memory 306 and persistent storage 308 are computer-readable storage media. In the illustrated embodiment, memory 306 includes RAM 316 and cache 318. Generally, memory 306 may include any suitable computer-readable storage medium, either volatile or non-volatile. Cache 318 is a high-speed memory that enhances the performance of processor 304 by holding recently and nearly recently accessed data from RAM 316.

[0034] The program instructions of the acoustic diagnostic program 112 may be stored in persistent storage 308, or more generally, any computer-readable storage medium, for execution by one or more of the respective computer processors 304 via one or more memories of memory 306. The persistent storage 308 may be a magnetic hard disk drive, a solid-state disk drive, a semiconductor storage device, a read-only memory (ROM), an electronically erasable programmable read-only memory (EEPROM), a flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0035] The media used by persistent storage 308 may also be removable media. For example, a removable hard drive may be used for persistent storage 308. Other examples include optical and magnetic disks, thumb drives, and smart cards inserted into a drive for transfer to another computer-readable storage medium that is also part of persistent storage 308.

[0036] In these examples, communications unit 312 provides for communication with other data processing systems or devices. In these examples, communications unit 312 includes one or more network interface cards. Communications unit 312 may provide for communication through the use of either or both physical and wireless communications links. In the context of some embodiments of the present invention, sources of various input data may be physically remote from computer 300 such that input data may be received, and similarly output may be transmitted, via communications unit 312.

[0037] The I / O interface 314 allows for the input and output of data to and from other devices that may be connected to the computer 300. For example, the I / O interface 314 may provide connection to external devices 320, such as a keyboard, keypad, touch screen, microphone, digital camera, or some other suitable input device or combination thereof. The external devices 320 may further include portable computer-readable storage media, such as thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to implement embodiments of the present invention, such as the acoustic diagnostic program 112, may be stored on such portable computer-readable storage media and loaded into the persistent storage device 308 via the I / O interface 314. The I / O interface 314 also connects to a display 322.

[0038] Display 322 provides a mechanism for displaying data to a user and may be, for example, a computer monitor. Display 322 may also function as a touch screen, such as the display of a tablet computer.

[0039] The programs described herein are identified based on the application in which they are implemented in particular embodiments of the invention. However, it should be understood that any particular program name herein is used merely as a matter of convenience and therefore should not limit the invention to use with any particular application identified and / or implied by such name.

[0040] The present invention may be a system, a method, or a computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0041] The computer-readable storage medium may be any tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or raised structures in grooves having instructions recorded thereon, and any suitable combination thereof. As used herein, a computer-readable storage medium should not be construed as being a transitory signal itself, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through a fiber optic cable), or an electrical signal transmitted over an electrical wire.

[0042] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or can be downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. This network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface within each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to a computer-readable storage medium within the respective computing / processing device for storage.

[0043] Computer-readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages ​​such as Smalltalk®, C++, and conventional procedural programming languages ​​such as the “C” programming language or the like. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or remote server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry to carry out aspects of the present invention.

[0044] Aspects of the present invention are described herein with reference to flowchart and / or block diagram illustrations of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of those flowchart and / or block diagram illustrations, and combinations of blocks in those flowchart and / or block diagram illustrations, can be implemented by computer-readable program instructions.

[0045] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, generate means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium, instructing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium having the instructions stored therein comprises an article of manufacture containing instructions for performing aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0046] These computer-readable program instructions may also be loaded into a computer, other programmable data processing device, or other device to cause a series of operational steps to be performed on the computer, other programmable device, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable device, or other device perform the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0047] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing specified logical functions. In some alternative implementations, the functions shown in the blocks may be performed in an order different from that shown in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or implements a combination of dedicated hardware and computer instructions.

[0048] The description of various embodiments of the present invention has been provided for illustrative purposes and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art that do not depart from the scope and spirit of the present invention. The terms used herein have been selected to best explain the principles, practical applications, or technical improvements of the embodiments over those found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for diagnosing equipment using a digital twin, comprising: creating a digital twin of the equipment that simulates the equipment; and receiving, by one or more computer processors, real-time audio input for said appliance from a mobile device; analyzing, by the one or more computer processors, the real-time audio input using one or more acoustic modeling algorithms to establish deviations from a baseline of a sound profile associated with the digital twin of the device; in response to determining that the deviation from the baseline is greater than a predetermined threshold, prompting a user, by the one or more computer processors, where to reposition the mobile device to obtain optimal audio input until a stopping criterion is met; the stopping criteria include detecting that characteristics of the real-time audio input reflect a previously identified problem seen in the digital twin of the device, determining that a given proximity to a known error-prone location of the device has been reached, or detecting that characteristics of the real-time audio reflect a new problem not seen in the digital twin of the device; the relocation location is determined by a computer processor based on factors including required proximity to error-prone internal components of the device based on the digital twin, anomalies in the real-time audio input, or a comparison of the real-time audio input with the sound profile; analyzing, by one or more processors, the real-time audio input to detect anomalous sounds based on the sound profile associated with the digital twin; and Presenting the results of said analysis 20. A computer-implemented method comprising:

2. capturing, by the one or more computer processors, the real-time audio input of the device; and updating, by the one or more computer processors, a machine learning model of the device that is used for future diagnostic sessions of the device using the real-time voice input. The computer-implemented method of claim 1 further comprising:

3. capturing, by the one or more computer processors, the real-time audio input of the device; and updating, by the one or more computer processors, the digital twin of the device with the real-time voice input. The computer-implemented method of claim 1 further comprising:

4. A computer program for equipment diagnosis, the computer program causing one or more computer processors to carry out the computer-implemented method of any one of claims 1 to 3.

5. A computer system for diagnosing equipment using a digital twin, wherein a digital twin of the equipment is created by simulating the equipment, and the computer system: one or more computer processors; one or more computer-readable storage media; program instructions stored on the one or more computer-readable storage media that are executed by at least one computer processor of the one or more computer processors; wherein the stored program instructions include: instructions for receiving real-time voice input for said appliance from a mobile device; instructions for analyzing the real-time audio input using one or more acoustic modeling algorithms to establish deviations from a baseline of a sound profile associated with the digital twin of the device; responsive to determining that the deviation from the baseline is greater than a predetermined threshold, instructions for prompting a user as to where to reposition the mobile device to obtain optimal audio input until a stopping criterion is met; the stopping criteria include detecting that characteristics of the real-time audio input reflect a previously identified problem seen in the digital twin of the device, determining that a given proximity to a known error-prone location of the device has been reached, or detecting that characteristics of the real-time audio reflect a new problem not seen in the digital twin of the device; the relocation location is determined based on factors including required proximity to error-prone internal components of the device based on the digital twin, anomalies in the real-time audio input, or a comparison of the real-time audio input with the sound profile; instructions for analyzing the real-time audio input to detect anomalous sounds based on the sound profile associated with the digital twin; and instructions for presenting the results of said analysis; 1. A computer system comprising:

6. The program instructions: instructions for capturing the real-time audio input of the device; and instructions for updating a machine learning model of the device using the real-time voice input for use in future diagnostic sessions of the device; 6. The computer system of claim 5, further comprising:

7. The program instructions: instructions for capturing the real-time audio input of the device; and instructions for updating the digital twin of the device using the real-time voice input.

6. The computer system of claim 5, further comprising:

Images