Voice interactive fault diagnosis and prediction method in digital twin environment

By introducing speech models and natural language processing technology in a digital twin environment, combined with machine learning algorithms, voice interaction and fault diagnosis between physical systems and users are realized, real-time and intelligent problems of fault diagnosis in the existing technology are solved, and the problem of real-time and intelligent problems of fault diagnosis in the existing technology is improved. The accuracy of fault prediction and equipment reliability are achieved.

CN118939753AInactive Publication Date: 2024-11-12CHINA YANGTZE POWER

Patent Information

Application Number
CN202410915943.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing fault diagnosis and prediction systems lack real-time and continuity, and require manual intervention, resulting in an expansion of the scope of the fault impact and increasing maintenance difficulties, and lack of intelligence and automation capabilities.

Method used

Introduce a voice model in a digital twin environment, use natural language processing technology to realize voice interaction between the physical system and the user, combine machine learning algorithms for fault monitoring and diagnosis, and provide users with real-time alarms and feedback through the voice interaction module.

Benefits of technology

It improves the interactive efficiency and user experience of the digital twin system, realizes real-time fault diagnosis and prediction, reduces maintenance costs, and improves equipment reliability and fault prediction accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939753A_ABST
    Figure CN118939753A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of fault diagnosis and prediction, in particular to a voice interactive fault diagnosis and prediction method in a digital twin environment, which comprises the following steps of: creating a virtual model by using real-time data of a physical system for simulating and analyzing behaviors of the physical system; a voice interaction module is integrated in the virtual model, and the voice interaction module realizes voice interaction between a physical system and a user through a natural language processing technology; and integrating a fault monitoring and diagnosis module in the virtual model, wherein the fault monitoring and diagnosis module detects abnormal behaviors of the system through a machine learning algorithm. The method comprises the following steps: mapping preprocessed equipment operation data to a virtual model to realize synchronization of physical equipment and the virtual model, and establishing a time sequence model by constructing a trend analysis model; the development trend of the loss degree is reflected through a time sequence model, so that the operation state and the life loss condition of the equipment are predicted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of fault diagnosis and prediction technology, and in particular to a voice interactive fault diagnosis and prediction method in a digital twin environment. Background Art

[0002] The so-called digital twin utilizes data such as physical models, sensor updates and operation history, integrates multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes, completes mapping in virtual space, and thus reflects the entire life cycle of the corresponding physical equipment.

[0003] Existing fault diagnosis and prediction systems usually lack real-time and continuity. They may only be able to diagnose and predict at a specific point in time, but cannot provide continuous monitoring and early warning functions. This may lead to the inability to discover and take measures in time before the fault occurs, increasing the scope of the fault and the difficulty of repair. In addition, they lack intelligent and automated capabilities. They may require manual intervention and judgment and cannot automatically diagnose and predict faults, which not only increases labor costs, but may also introduce interference from human errors and subjective judgments.

[0004] Digital twin technology has significant advantages in improving fault diagnosis and prediction. It creates a virtual copy of the physical entity (digital twin model) to achieve real-time data synchronization and dynamic update, improve the accuracy and speed of fault diagnosis; at the same time, digital twins can simulate various fault conditions, provide a scientific basis for prediction, reduce maintenance costs and improve equipment reliability.

[0005] To this end, we propose a method in the digital twin environment to improve the efficiency and experience of human-computer interaction in the digital twin system. Summary of the invention

[0006] The purpose of the present invention is to provide a technical solution by introducing a speech model into a digital twin environment. By using natural language processing (NLP) technology, the model can deeply analyze the meaning of human language and generate corresponding responses, thereby improving the interaction efficiency and user experience of the digital twin system to solve the problems in the background technology.

[0007] The technical solution of the present invention is: a voice interactive fault diagnosis and prediction method in a digital twin environment, the method comprising the following steps:

[0008] Use the data of the physical system to create a virtual model to simulate and analyze the behavior of the physical system, and transmit data between the virtual model and the physical device through the OPC_UA communication protocol;

[0009] Integrate a voice interaction module in the virtual model, wherein the voice interaction module realizes voice interaction between the physical system and the user through natural language processing technology;

[0010] A fault monitoring and diagnosis module is integrated in the virtual model, and the fault monitoring and diagnosis module is data-connected with the voice interaction module; the fault monitoring and diagnosis module detects abnormal behavior of the physical system through a machine learning algorithm; the voice interaction module receives inquiries from users, and reports the system status to the user in the form of synthesized voice based on the data and status information provided by the fault monitoring and diagnosis module;

[0011] When receiving an alarm command, the voice interaction module sends an alarm to the user through synthesized voice, prompting the user to take action or respond;

[0012] The creating a virtual model using data of the physical system comprises:

[0013] Acquire physical structure characteristic data, performance parameters, operation rules and historical operation data of physical equipment, and pre-process the physical structure characteristic data, performance parameters, operation rules and historical operation data;

[0014] Build a virtual model of the physical equipment based on the physical structure data, performance parameters, and operating rules of the physical equipment;

[0015] The physical structure data includes design drawings and specifications of physical equipment.

[0016] Preferably, the obtaining of physical structure characteristic data, performance parameters, operating rules and historical operating data of the physical device, and preprocessing the physical structure characteristic data, performance parameters, operating rules and historical operating data includes:

[0017] Obtaining the operation data of physical equipment through various sensors deployed on the physical equipment;

[0018] Obtain performance parameters, operating rules and historical operating data of physical equipment from the database;

[0019] Use field control equipment to pre-process and store the operation data of physical equipment acquired by sensors;

[0020] Use wireless network communication equipment to send operating data to a cloud server or cloud platform;

[0021] The equipment operation data is further preprocessed on the cloud server or cloud platform.

[0022] Preferably, the preprocessing of the operation data of the device on the cloud server or cloud platform includes cleaning, normalizing and denoising the operation data of the device to improve the quality of the data.

[0023] Preferably, the operating data includes but is not limited to temperature data, pressure data, vibration data, equipment environment data and operating condition data of the physical equipment.

[0024] Preferably, detecting abnormal system behavior by a machine learning algorithm and diagnosing the cause of the fault includes:

[0025] Map the pre-processed equipment operation data to the virtual model to achieve synchronization between the physical equipment and the virtual model;

[0026] Obtain historical operating data of the equipment from the database, including temperature, pressure, current, voltage, vibration parameters and fault data;

[0027] Construct a prediction model: use the temperature, pressure, current, and voltage of the physical equipment as the input of the model, and the equipment loss as the output of the model; use the regression analysis method to fit a prediction model based on historical data, and predict the equipment loss at a future moment based on the new input data;

[0028] Build a trend analysis model: The data on the change of the degree of loss of physical equipment over time is converted into time series data, and a time series model is established to reflect the development trend of the degree of loss based on the time series model.

[0029] Preferably, the prediction model is: t+1 =(T t α+V t β+I t χ+P t δ)γ+Δυ (1)

[0030] Among them, Y t+1 is the life loss value of the physical device at time t+1, T t 、V t ,I t , P t are the temperature, input voltage, input current and internal pressure of the physical device at time t, γ is the correction parameter, Δυ is the intercept, α, β, χ, γ are the regression coefficients;

[0031] Assuming the life of the equipment is W years, the equipment loss is converted into the annual life value, which is set as w.

[0032] Perform further conversion to obtain the lifespan value in minutes:

[0033]

[0034] The Y t+1 =n t+1 ×w(3);

[0035] Where n is the loss coefficient, 10i≥n t+1 ≥10; i is the safety ratio parameter, i≥3;

[0036] Substituting equations (2) and (3) into equation (1), the prediction model is transformed into:

[0037]

[0038] By t+1 Reflects the degree of equipment loss at the next moment. If 10i≥n t+1 ≥10, it means that the degree of loss of physical equipment is normal;

[0039] If the predicted loss coefficient at the next moment is greater than 10i, it is considered that the current operating state will cause great loss to the equipment at the next moment. At this time, the virtual model sends a voice alarm command to the voice interaction module;

[0040] After receiving the voice alarm command, the voice interaction module generates an alarm voice and the user can receive the alarm information.

[0041] Preferably, the method of realizing voice interaction between the physical system and the user by using natural language processing technology comprises the following steps:

[0042] Receive user voice input acquired by an external sound collection device;

[0043] Convert speech signals into text data through speech recognition models;

[0044] Then, natural language processing technology is used to understand the user’s intent and context;

[0045] Call the corresponding business logic in the system according to the user's voice information to query the status; or retrieve the historical data and current operation data of the physical device from the database and / or virtual model according to the user's voice command;

[0046] The data is generated into text information, which is then converted into voice information and played through an external audio output device for users to listen to and complete voice feedback, thus achieving a complete voice interaction closed loop.

[0047] The historical data and current operation data of the physical device include one or more of temperature, pressure, vibration, device environment, etc.

[0048] Preferably, the data of the wear degree of the physical equipment changing over time is used to form time series data, a time series model is established, and the development trend of the wear degree is reflected according to the time series model, including:

[0049] Obtain multiple equipment loss coefficients and attach a corresponding timestamp to each loss coefficient to form a time series data set of length t: N = {n1,…n t-1 ,n t};

[0050] Select the sliding time window T;

[0051] Get the loss coefficient within a sliding time window and calculate the mean loss coefficient

[0052] Move the time window forward one step, repeat the above steps until the window exceeds the end of the time series data set, and obtain multiple loss coefficient means:

[0053] The loss coefficient means calculated in each sliding window are arranged in chronological order to construct a new time series. The new time series is analyzed through the autocorrelation function to detect the periodic fluctuations and trends of the data.

[0054] The present invention also provides an electronic device, comprising a memory and a processor, and the electronic device executes the voice interactive fault diagnosis and prediction method in a digital twin environment.

[0055] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the voice interactive fault diagnosis and prediction method in a digital twin environment.

[0056] Beneficial effects of the present invention:

[0057] 1. The present invention integrates a voice interaction module and a fault monitoring and diagnosis module in a virtual model; the voice interaction module realizes voice interaction between the physical system and the user through natural language processing technology, allowing the user to operate through voice commands and receive voice feedback; on the other hand, the fault monitoring and diagnosis module uses a machine learning algorithm to detect abnormal behavior in the system and outputs alarm information through voice.

[0058] 2. In the present invention, abnormal behavior of the system is detected by a machine learning algorithm: the pre-processed equipment operation data is mapped to a virtual model to achieve synchronization between the physical equipment and the virtual model; the historical operation data of the equipment is obtained from the database, including temperature, pressure, current, voltage, vibration parameters and fault data; the temperature, pressure, current and voltage of the physical equipment are used as the input of the model, and the equipment loss is used as the output of the model; a prediction model is fitted using a regression analysis method through historical data, and the loss of the equipment at a future moment is predicted based on the new input data. If the loss coefficient exceeds the set threshold range, it can be judged that the existing input is abnormal, resulting in excessive equipment loss at a future moment, and the equipment is very likely to fail. By comparing the input at this moment with the historical input, it is determined which input is abnormal.

[0059] 3. In the present invention, a time series data set with a length of t is formed by attaching a corresponding timestamp to each loss coefficient, and the loss coefficient means calculated by each sliding window are arranged in chronological order to establish a time series. The new time series is analyzed by the autocorrelation function to analyze the periodic fluctuations and trends of the data. The time series model can reflect the development trend of the degree of loss of physical equipment, thereby predicting the operating status and life loss of the equipment, providing strong support for the maintenance and management of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Shown is a flow chart of a voice interactive fault diagnosis and prediction method in a digital twin environment of the present invention. DETAILED DESCRIPTION

[0061] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0062] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0063] refer to Figure 1 The present invention provides a technical solution: a voice interactive fault diagnosis and prediction method in a digital twin environment, the method comprising the following steps:

[0064] Use real-time data from physical systems to create virtual models for simulating and analyzing physical system behavior;

[0065] Integrate a voice interaction module in the virtual model, wherein the voice interaction module realizes voice interaction between the physical system and the user through natural language processing technology;

[0066] A fault monitoring and diagnosis module is integrated into the virtual model. The fault monitoring and diagnosis module detects abnormal system behavior through a machine learning algorithm. The voice interaction module feeds back physical system status information based on the collected user voice information, and feeds back alarm voice information based on the alarm instruction output by the fault monitoring and diagnosis module.

[0067] The creating of a virtual model using real-time data of a physical system comprises:

[0068] Obtain physical structure characteristic data, performance parameters, operation rules and historical operation data of physical equipment;

[0069] Build a virtual model of the physical equipment based on the physical structure data, performance parameters, and operating rules of the physical equipment;

[0070] The physical structure data includes physical equipment design drawings and specifications.

[0071] In some preferred embodiments, operating data of the physical device is acquired through sensors, and these data include but are not limited to temperature data, pressure data, vibration data, device environment data and operating condition data of the physical device.

[0072] These data are collected by various sensors and can provide detailed information about the operating status of physical devices. In order to further process and store these operating data, we use field control devices (field control devices usually include PLC, DCS or other forms of controllers) to pre-process the data obtained by the sensors. These pre-processing processes include data cleaning, conversion and standardization to ensure the quality and consistency of the data. After pre-processing, we store these data locally or on a cloud platform for subsequent analysis and utilization. In order to achieve remote monitoring and data analysis, we send the operating data of the equipment to a cloud server or cloud platform via a wireless network. This wireless transmission method can provide higher flexibility and scalability, allowing us to access and analyze the operating data of the equipment anytime and anywhere. On the cloud server or cloud platform, we further pre-process the operating data of the equipment. These pre-processing operations may include data aggregation, filtering and statistical analysis to extract higher-level information and insights. Through these pre-processing operations, we can better understand and analyze the operating status of the equipment and make corresponding decisions and optimizations accordingly. In order to achieve data transmission between virtual models and physical devices, we use the OPC_UA communication protocol. OPC_UA is an open standard communication protocol used to achieve data exchange and interoperability between devices in industrial automation systems. By using the OPC_UA communication protocol, we can ensure that the data transmission between the virtual model and the physical device is reliable, secure and efficient. These steps together build an efficient and reliable equipment monitoring system, providing strong support for the operation and maintenance of the equipment.

[0073] Pre-process the equipment's operating data on the cloud server or cloud platform, including:

[0074] Data cleaning: Identify and remove data points that are significantly out of the normal range. This can be done by setting thresholds or using statistical methods such as IQR (interquartile range) to identify outliers; there may be missing values ​​in the equipment operation data, which need to be filled or ignored according to the situation. Common processing methods include mean filling, median filling, interpolation using adjacent data points, or directly deleting records with missing values. Ensure that all data follows a unified format and standard, such as consistency in date and time formats, and the way values ​​are represented.

[0075] Normalization: Adjust the data scale to a uniform range, usually [0,1] or [-1,1]. This can be achieved through a variety of methods, such as min-max scaling or Z-score normalization; since device data may contain different units (such as temperature may be Celsius or Fahrenheit), it needs to be converted to a uniform unit for comparison and analysis.

[0076] Noise reduction: Reducing random noise in the data through filtering techniques. Common methods include moving average filters, exponential smoothing, or more complex methods such as Kalman filtering.

[0077] The machine learning algorithm is used to detect abnormal system behavior and diagnose the causes of failures, including:

[0078] Map the pre-processed equipment operation data to the virtual model to achieve synchronization between the physical equipment and the virtual model; obtain the equipment's historical operation data from the database, including temperature, pressure, current, voltage, vibration parameters and fault data;

[0079] Build a prediction model: use the temperature, pressure, current, and voltage of the physical equipment as the input of the model, and use the equipment loss as the output of the model;

[0080] Use regression analysis to fit a prediction model based on historical data, and predict the loss of the equipment at a certain moment in the future based on new input data;

[0081] Build a trend analysis model: The data on the change of the degree of loss of physical equipment over time is converted into time series data, and a time series model is established to reflect the development trend of the degree of loss based on the time series model.

[0082] Among them, the prediction model is: Y t+1 =(T t α+V t β+I t χ+P t δ)γ+Δυ (1)

[0083] Among them, Y t+1 is the life loss value of the physical device at time t+1, T t 、V t ,I t , P t are the temperature, input voltage, input current and internal pressure of the physical device at time t, γ is the correction parameter, Δυ is the intercept, α, β, χ, γ are the regression coefficients;

[0084] Assuming the life of the equipment is W years, the equipment loss is converted into the annual life value, which is set as w.

[0085] Perform further conversion to obtain the lifespan value in minutes:

[0086]

[0087] The Y t+1 =n t+1 ×w(3);

[0088] Where n is the loss coefficient, 10i≥n t+1 ≥10; i is the safety ratio parameter, i≥3;

[0089] Substituting equations (2) and (3) into equation (1), the prediction model is transformed into:

[0090]

[0091] By t+1 Reflects the degree of equipment loss at the next moment. If 10i≥n t+1 ≥10, it means that the degree of loss of physical equipment is normal;

[0092] If the predicted loss coefficient at the next moment is greater than 10i, it is considered that the current operating state will cause great loss to the equipment at the next moment. At this time, the fault monitoring and diagnosis module sends a voice alarm command to the voice interaction module;

[0093] After receiving the voice alarm command, the voice interaction module generates an alarm voice, and the user can receive the alarm information. In addition, each input quantity at the current moment is compared with the corresponding historical input quantity. If the difference between the current input quantity and the corresponding historical input quantity exceeds the preset safety threshold, the variable is judged to be an abnormal variable, where the safety threshold is 20% of the parameter value that ensures the safe operation of the equipment;

[0094] For example, it can be set according to the rated current and rated voltage of the equipment. Taking current as an example, the safety threshold can be set to 20% of the rated current. If the difference between the current in the input quantity at the current moment and the rated current value exceeds 20% of the rated current value, the current value is considered abnormal.

[0095] A time series model is established to reflect the development trend of the loss degree, specifically:

[0096] Assume that the total number of physical devices is n, the length of the time series is t, and the loss coefficient of each device i at time j is n i,j .

[0097] Choose a suitable sliding time window size of T minutes. This window will be used to calculate the moving average of the loss coefficient.

[0098] For each time point j, calculate the average loss coefficient of the device from j to j+T-1. If j+T-1≤t, the calculation formula is:

[0099] is the mean of the sliding window loss coefficient at time j;

[0100] Move the time window forward one step, starting from j+1, and repeat the above process of calculating the mean until the window exceeds the end of the time series.

[0101] When analyzing a device, the value of n is 1, and the calculation formula becomes:

[0102] Arrange the loss coefficient means calculated by each sliding window in chronological order to construct a new time series

[0103] To analyze the newly constructed time series, time series analysis methods such as autocorrelation function (ACF) and partial autocorrelation function (PACF) can be used to detect periodic fluctuations and trends in the data. In addition, statistical software or packages in programming languages ​​(such as Python and R) can be used to assist in the analysis. In this way, the loss trend of the equipment can be obtained, thereby predicting the trend of the speed of shortening the life of the equipment.

[0104] Using natural language processing technology to achieve voice interaction between the physical system and the user includes the following steps:

[0105] 1. Receiving the user's voice input acquired by an external sound collection device, specifically including the following steps:

[0106] Voice collection: Use microphones or other sound sensing devices to capture user voice commands.

[0107] Signal optimization: Preprocess the collected sound signals, such as noise reduction and enhancement, to improve the accuracy of speech recognition.

[0108] 2. Convert speech signals into text data through a speech recognition model. Specifically, the acoustic model is used to analyze the acoustic features in the speech signal and generate a series of possible pronunciation segments. The language model is combined with linguistic knowledge to infer the most likely word sequence, i.e., a sentence in text form, based on the output of the acoustic model.

[0109] 3. Understand the user’s intent and context through natural language processing technology, which includes the following steps:

[0110] Intent recognition: Use classification models to determine the user's specific intentions, such as querying device status, adjusting set values, etc.; Entity recognition: Identify key information in sentences, such as device name, parameter type (temperature, pressure, etc.); Context understanding: Consider information such as conversation history and user background to more accurately interpret user needs.

[0111] 4. Call the corresponding business logic in the system according to the user's voice information, which specifically includes the following steps: Execute operations: Based on the recognized intentions and entities, perform operations such as querying status, starting or stopping the device, etc.; Information retrieval: If the user needs specific data, the system will retrieve the required information from the relevant database or virtual model.

[0112] 5. The acquired data generates text information, which specifically includes the following data: Data formatting: Arrange the temperature data, pressure data, etc. obtained from the query in a format that is easy to understand and read; Information integration: Combine the user's intention and context to generate coherent and accurate feedback text information.

[0113] 6. Convert the generated text information into voice information, which specifically includes the following steps: Text-to-speech (TTS) conversion: Use speech synthesis technology to convert text information into voice output; Adjust the tone and speed: Adjust the tone and speed according to the context of the feedback content to make the voice more natural and friendly.

[0114] 7. Play through the audio output device of the peripheral, specifically: through audio output devices such as speakers or headphones, the user can hear the voice feedback generated by the system.

[0115] The present invention also provides an electronic device, which includes a memory and a processor, and the electronic device executes the voice interactive fault diagnosis and prediction method in a digital twin environment.

[0116] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the voice interactive fault diagnosis and prediction method in a digital twin environment.

[0117] The embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the above-mentioned computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be deformed or modified in any way.

Claims

1. A voice interactive fault diagnosis and prediction method in a digital twin environment, characterized in that: The method comprises the following steps: Use the data of the physical system to create a virtual model to simulate and analyze the behavior of the physical system, and transmit data between the virtual model and the physical device through the OPC_UA communication protocol; Integrate a voice interaction module in the virtual model, wherein the voice interaction module realizes voice interaction between the physical system and the user through natural language processing technology; A fault monitoring and diagnosis module is integrated in the virtual model, and the fault monitoring and diagnosis module is data-connected with the voice interaction module; the fault monitoring and diagnosis module detects abnormal behavior of the physical system through a machine learning algorithm; the voice interaction module receives inquiries from users, and reports the system status to the user in the form of synthesized voice based on the data and status information provided by the fault monitoring and diagnosis module; When receiving an alarm command, the voice interaction module sends an alarm to the user through synthesized voice, prompting the user to take action or respond; The creating a virtual model using data of the physical system comprises: Acquire physical structure characteristic data, performance parameters, operation rules and historical operation data of physical equipment, and pre-process the physical structure characteristic data, performance parameters, operation rules and historical operation data; Build a virtual model of the physical equipment based on the physical structure data, performance parameters, and operating rules of the physical equipment; The physical structure data includes design drawings and specifications of physical equipment.

2. According to the voice interactive fault diagnosis and prediction method in a digital twin environment according to claim 1, it is characterized in that: The obtaining of physical structure characteristic data, performance parameters, operation rules and historical operation data of the physical device, and preprocessing of the physical structure characteristic data, performance parameters, operation rules and historical operation data includes: Obtaining the operation data of physical equipment through various sensors deployed on the physical equipment; Obtain performance parameters, operating rules and historical operating data of physical equipment from the database; Use field control equipment to pre-process and store the operation data of physical equipment acquired by sensors; Use wireless network communication equipment to send operating data to a cloud server or cloud platform; The operation data of the equipment is further preprocessed on the cloud server or cloud platform.

3. The voice interactive fault diagnosis and prediction method in a digital twin environment according to claim 2 is characterized in that: The preprocessing of the operation data of the device on the cloud server or cloud platform includes cleaning, normalizing and denoising the operation data of the device to improve the quality of the data.

4. The voice interactive fault diagnosis and prediction method in a digital twin environment according to claim 3 is characterized in that: The operation data includes, but is not limited to, temperature data, pressure data, vibration data, equipment environment data, and operating condition data of the physical equipment.

5. The voice interactive fault diagnosis and prediction method in a digital twin environment according to claim 1, characterized in that: The detection of abnormal system behavior through machine learning algorithms and diagnosis of fault causes include: Map the pre-processed equipment operation data to the virtual model to achieve synchronization between the physical equipment and the virtual model; Obtain historical operating data of the equipment from the database, including temperature, pressure, current, voltage, vibration parameters and fault data; Construct a prediction model: use the temperature, pressure, current, and voltage of the physical equipment as the input of the model, and the equipment loss as the output of the model; use the regression analysis method to fit a prediction model based on historical data, and predict the equipment loss at a future moment based on the new input data; Build a trend analysis model: The data on the change of the degree of loss of physical equipment over time is converted into time series data, and a time series model is established to reflect the development trend of the degree of loss based on the time series model.

6. The voice interactive fault diagnosis and prediction method in a digital twin environment according to claim 5 is characterized in that: The prediction model is: t+1 =(T t α+V t β+I t χ+P t δ)γ+Δυ(1) Among them, Y t+1 is the life loss value of the physical device at time t+1, T t 、V t ,I t , P t are the temperature, input voltage, input current and internal pressure of the physical device at time t, γ is the correction parameter, Δυ is the intercept, α, β, χ, γ are the regression coefficients; Assuming the life of the equipment is W years, the equipment loss is converted into the annual life value, which is set as w. Perform further conversion to obtain the lifespan value in minutes: The Y t+1 =n t+1 ×w(3); Where n is the loss coefficient, 10i≥n t+1 ≥10; i is the safety ratio parameter, i≥3; Substituting equations (2) and (3) into equation (1), the prediction model is transformed into: By t+1 Reflects the degree of equipment loss at the next moment. If 10i≥n t+1 ≥10, it means that the degree of loss of physical equipment is normal; If the predicted loss coefficient at the next moment is greater than 10i, it is considered that the current operating state will cause great loss to the equipment at the next moment. At this time, the virtual model sends a voice alarm command to the voice interaction module; After receiving the voice alarm command, the voice interaction module generates an alarm voice and the user can receive the alarm information.

7. The voice interactive fault diagnosis and prediction method in a digital twin environment according to claim 1, characterized in that: The method of realizing voice interaction between the physical system and the user through natural language processing technology includes the following steps: Receive user voice input acquired by an external sound collection device; Convert speech signals into text data through speech recognition models; Then, natural language processing technology is used to understand the user’s intent and context; Call the corresponding business logic in the system according to the user's voice information to query the status; or retrieve the historical data and current operation data of the physical device from the database and / or virtual model according to the user's voice command; The data is generated into text information, which is then converted into voice information and played through an external audio output device for users to listen to and complete voice feedback, thus achieving a complete voice interaction closed loop. The historical data and current operation data of the physical device include one or more of temperature, pressure, vibration, device environment, etc.

8. The voice interactive fault diagnosis and prediction method in a digital twin environment according to claim 5, characterized in that: The data of the wear degree of the physical equipment changing over time is used to form time series data, a time series model is established, and the development trend of the wear degree is reflected according to the time series model, including: Obtain multiple equipment loss coefficients and attach a corresponding timestamp to each loss coefficient to form a time series data set of length t: N = {n1,…n t-1 ,n t }; Select the sliding time window T; Get the loss coefficient within a sliding time window and calculate the mean loss coefficient n T1 ; Move the time window forward one step, repeat the above steps until the window exceeds the end of the time series data set, and obtain multiple loss coefficient means: The loss coefficient means calculated in each sliding window are arranged in chronological order to construct a new time series. The new time series is analyzed through the autocorrelation function to detect the periodic fluctuations and trends of the data.

9. An electronic device, comprising a memory and a processor, characterized in that: The electronic device executes a voice interactive fault diagnosis and prediction method in a digital twin environment as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a voice interactive fault diagnosis and prediction method in a digital twin environment as described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Method and system for on-line monitoring steam turbine roter low-cycle fatigue life consumption

    CN101042059A

  • Method and device for controlling household appliances, and server

    CN114488829A

  • Automobile factory control system based on digital twinning

    CN117270482A

  • Power plant power equipment expert diagnosis system and method based on historical data

    CN117290685A

Cited By

  • Environment monitoring method and system based on digital twinborn technology

    CN119147048A

  • Simulation voice line intervention state monitoring system and method based on big data

    CN119520679A

  • Intelligent voice-based intelligent fault diagnosis operation and maintenance system for digital twinning of overcharge equipment

    CN119989206A

  • Charging equipment cloud call fault diagnosis method and device and computer equipment

    CN120748397A