A method for assisting complex signal-to-text conversion in a multi-modal large model

Abstract

The application relates to the technical field of fault diagnosis, and discloses an M-BaseFre clustering method for assisting in converting complex signals in a multi-modal large model into texts, which comprises the following steps: 1) calculating the amplitude spectrum of a component signal; 2) finding all frequency peak values in the amplitude spectrum of the component and reserving corresponding frequency values based on a findpeaks function of MATLAB; 3) calculating the greatest common divisor between all peak frequency values of each component and the greatest common divisor between the components, and recombining component signals with a common fundamental frequency; and 4) calculating the sideband value between all peak frequency values of residual components, and recombining component signals with the same value. The application can provide basic support for signal-to-text description, realize the application of a large language model in fault diagnosis, and prevent major accidents caused by missed diagnosis and misdiagnosis of equipment faults.

Description

Technical Field

[0001] This invention relates to the fields of fault diagnosis and large language models, and in particular to a method for converting complex signals into text in a multimodal large model. Background Technology

[0002] In the widespread application of industrial equipment, mechanical systems, and complex structures, a large amount of complex vibration signals from these devices are collected and analyzed. Traditional fault diagnosis methods mainly rely on expert experience and signal processing techniques, such as Fourier transform and wavelet decomposition. While these methods can extract features from signals to a certain extent, they also have significant limitations. Specifically, traditional methods often rely on experts to manually interpret the features, which is insufficient to meet the demands of intelligent and automated development.

[0003] In recent years, large language models have demonstrated powerful capabilities in natural language processing, text generation, and understanding contextual relationships, making them a potential tool for industrial intelligence. By combining large language models with complex signal feature extraction and text generation, the intelligence level of signal analysis can be improved. However, directly inputting complex signal data into large multimodal models for training presents significant problems. Complex signals often exhibit feature aliasing, with interference noise mixed with useful features, making it difficult for large multimodal models to accurately extract meaningful features when learning from this data. This results in poor model interpretability and affects diagnostic accuracy. Therefore, the powerful processing capabilities of large language models alone cannot directly solve the problem of complex signal analysis, especially in fault diagnosis applications, where ensuring the accuracy and effectiveness of signal feature extraction and text description remains a major challenge.

[0004] Therefore, it is necessary to decompose and preprocess complex signals, breaking them down into single-component signals, and then combining these components with similar physical characteristics to generate an accurate text description, thereby improving the accuracy and effectiveness of the description. This process not only better aligns with the signal diagnosis and analysis process of simulation fault diagnosis experts but also effectively reduces noise interference, making the descriptions generated by the large language model more interpretable and practical.

[0005] Therefore, this invention proposes a method for converting complex signals into text in large multimodal models, to better provide structured signal feature input for large language models. By decomposing and clustering complex signals, signal components with the same physical characteristics can be recombined, resulting in more accurate text descriptions and effectively reducing the influence of interference components. This improves the efficiency of signal analysis and the application effect of large language models, meeting the high-precision and high-efficiency requirements of industrial intelligence for signal analysis. Summary of the Invention

[0006] The technical problem to be solved by the first aspect of the present invention is to provide a method for converting complex signals into text in a multimodal large model. This method can provide basic support for intelligent diagnosis of devices based on large language models, avoid human misdiagnosis and missed diagnosis, and thus prevent major accidents caused by faults.

[0007] To address the aforementioned technical problems, this invention provides a method for converting complex signals into text in a large multimodal model, the method comprising the following steps:

[0008] S1 calculates the amplitude spectrum for all component signals;

[0009] S2 uses MATLAB's built-in findpeaks function to locate the frequency peaks in the amplitude spectrum of all components and retains the corresponding frequency values.

[0010] S3 calculates the greatest common divisor among all peak frequency values ​​of each component and the greatest common divisor among the components, and reassembles the component signals with a common fundamental frequency;

[0011] S4 calculates the sideband values ​​between all peak frequency values ​​of the remaining components and reassembles the component signals with the same value.

[0012] S5 provides a textual description of the recombined signal.

[0013] Preferably, in step S1, the component signal can be obtained by a signal decomposition algorithm, including but not limited to matrix singular value decomposition, matrix singular spectrum decomposition, tensor singular value decomposition, and local mean decomposition.

[0014] More preferably, in step S2, the retained peak frequency value is rounded up, rounded down, or rounded to one decimal place to reduce the error caused by frequency fluctuations and thus improve the accuracy of clustering. MATLAB can use its built-in `ceil`, `floor`, and `round` functions to achieve this.

[0015] Preferably, step S3 includes the following specific steps:

[0016] 41) For each component signal, calculate the greatest common divisor (GCD) among all its frequency peaks. If the amplitude spectrum of the component signal has only one peak, then the frequency of that peak is taken as the GCD of the component. If no valid GCD is found in the component signal, then the component is marked as "ungrouped component".

[0017] 42) For component signals with the greatest common divisor, further iteratively calculate whether there is a common fundamental frequency among the components. If there is, reassemble these component signals with a common fundamental frequency. For component signals without a common fundamental frequency, group them separately; if there are ultimately N component signals and each component signal has an independent fundamental frequency, then divide these N component signals into an independent group.

[0018] More preferably, step S4 includes the following specific steps:

[0019] 51) For the components marked as "not yet grouped" in step 41, calculate the sideband values ​​between all their frequency peaks, that is, calculate whether the difference between adjacent peak frequencies is a fixed value.

[0020] 52) For component signals without sideband values, reassemble them into a set of signals;

[0021] 53) For component signals with sideband values, further determine whether the sideband values ​​between components are the same, and then reassemble the component signals with common sideband values. For component signals without common sideband values, group them separately.

[0022] Preferably, step S5 includes the following specific steps:

[0023] 61) For the signal reconstructed in step 52, directly describe the signal characteristics from the amplitude spectrum and its phase spectrum features, including the fundamental frequency, fundamental frequency amplitude and phase, second harmonic frequency, amplitude and phase, and so on. If the signal only has a fundamental frequency and no harmonics, only the fundamental frequency, fundamental frequency amplitude and phase need to be described;

[0024] 62) For the signal reconstructed in step 53, draw its corresponding envelope spectrum, then find the frequency peaks in the envelope spectrum and describe them in text. If there is a common fundamental frequency among the peaks, the text description includes the fundamental frequency, fundamental frequency amplitude and phase, the second harmonic frequency, amplitude and phase, and so on. If there is no harmonic relationship between the peaks, only the frequency, amplitude and phase of each peak need to be described.

[0025] Through the above technical solution, the method of this invention can accurately and efficiently reassemble and group all component signals, allowing signal components with the same physical meaning to be rationally combined, thereby achieving accurate signal-to-text description. This method, combined with decomposition algorithms, achieves deep analysis and clustering processing of complex signals, effectively eliminating interference noise and feature aliasing problems, making the description results more accurate and physically interpretable. This method not only improves the accuracy of signal feature extraction but also significantly enhances the clarity and consistency of signal-to-text description, thus providing a reliable and high-quality foundation for intelligent fault diagnosis based on large language models.

[0026] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] Figure 1 This is a flowchart of the process of the present invention;

[0029] Figure 2 This is a time-domain diagram of an embodiment of the present invention;

[0030] Figure 3 This is the first grouping result after processing by the method of the present invention in an embodiment of the present invention.

[0031] Figure 4 This is the second grouping result after processing by the method of the present invention in an embodiment of the present invention. Detailed Implementation

[0032] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0033] A method for converting complex signals into text in a large multimodal model, the method comprising the following steps:

[0034] S1 calculates the amplitude spectrum for all component signals;

[0035] like Figure 1 The diagram shown is a flowchart of the process of this invention. Figure 2 The illustrated embodiment performs tensor singular value decomposition to obtain 11 component signals. Fourier transform is then used to obtain the amplitude spectrum of these 11 component signals.

[0036] S2 uses MATLAB's built-in findpeaks function to locate the frequency peaks in the amplitude spectrum of all components and retains the corresponding frequency values.

[0037] The `findpeaks` function in MATLAB is used to locate the frequency peaks in the amplitude spectrum of the 11 component signals, and the frequency values ​​corresponding to the peaks are retained. Rounding up and down the frequency values ​​ensures that all frequency values ​​are accurate to one decimal place, reducing the error caused by frequency fluctuations and thus improving the accuracy of clustering.

[0038] S3 calculates the greatest common divisor among all peak frequency values ​​of each component and the greatest common divisor among the components, and reassembles the component signals with a common fundamental frequency;

[0039] For the 11 component signals in this embodiment, the greatest common divisor (GCD) among all frequency peaks within each component is calculated. Some component signals have only one frequency peak in their amplitude spectrum; this unique frequency value is used as the GCD of these component signals. Component signals for which no valid GCD is found are marked as "ungrouped components." Next, for component signals with GCDs, the existence of a common fundamental frequency among these components is further iteratively calculated; component signals with a common fundamental frequency are recombined. Component signals without a common fundamental frequency are grouped separately.

[0040] S4 calculates the sideband values ​​between all peak frequency values ​​of the remaining components and reassembles the component signals with the same value.

[0041] For components marked "not yet grouped" in the previous step, calculate the sideband values ​​between all their frequency peaks, i.e., calculate whether the difference between adjacent peak frequencies is a fixed value; for component signals without sideband values, regroup them into a group; for component signals with sideband values, further determine whether the sideband values ​​between components are the same, and then regroup the component signals with common sideband values. For component signals without common sideband values, group them separately.

[0042] S5 provides a textual description of the recombined signal.

[0043] For the signal reconstructed in step 52, the signal characteristics are directly described in text based on its amplitude spectrum and phase spectrum features, including the fundamental frequency, fundamental frequency amplitude and phase, second harmonic frequency, amplitude and phase, and so on. If the signal only has a fundamental frequency and no harmonics, only the fundamental frequency, fundamental frequency amplitude and phase need to be described. For the signal reconstructed in step 53, its corresponding envelope spectrum is plotted, and then the frequency peaks in the envelope spectrum are identified and described in text. If there is a common fundamental frequency among the peaks, the text description includes the fundamental frequency and amplitude, the second harmonic frequency and amplitude, and so on. If there is no harmonic relationship between the peaks, only the frequency, amplitude and phase of each peak need to be described.

[0044] In this embodiment, after processing in steps 52 and 53, the 11 component signals are recombined into 2 signals. Figure 3 The signal obtained in step 52, Figure 4 This is the signal after processing in step 53. For Figure 3 The signal in the text is described directly from its amplitude spectrum and phase spectrum characteristics, including a fundamental frequency of 10.98 Hz and a fundamental frequency amplitude of 0.23 m / s. 2 The phase is 0.78 radians, the second harmonic frequency is 22.01 Hz, and the amplitude is 0.146 m / s. 2Its phase is 3.02 radians, its third harmonic frequency is 32.99 Hz, and its amplitude is 0.037 m / s. 2 Phase -1.36 radians; 4th harmonic frequency 43.98 Hz, amplitude 0.21 m / s 2 Phase -1.2 radians. For Figure 4 For the signal in the image, draw its corresponding envelope spectrum, then find the frequency peaks in the envelope spectrum and describe them in text. The text description includes the fundamental frequency of 42.99 Hz and the fundamental amplitude of 0.215 m / s. 2 The second harmonic frequency is 85.98 Hz and the amplitude is 0.089 m / s. 2 The third harmonic frequency is 128.975 Hz and the amplitude is 0.041 m / s. 2 The fourth harmonic frequency is 172.00 Hz and the amplitude is 0.0096 m / s. 2 The fifth harmonic frequency is 214.99 Hz and the amplitude is 0.0081 m / s. 2 The text description of the entire signal is as follows:

[0045]

[0046] In the description of this invention, the reference to terms such as "one embodiment," "some embodiments," "one implementation," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this invention, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0047] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings; however, the present invention is not limited thereto. Within the scope of the inventive concept, various simple modifications can be made to the technical solutions of the present invention, including combinations of various specific technical features in any suitable manner. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately. However, these simple modifications and combinations should also be considered as the content disclosed in the present invention and are all within the protection scope of the present invention.

Claims

1. A method for converting complex signals into text in a large multimodal model, characterized in that, The method includes the following steps: S1 calculates the amplitude spectrum for all component signals; S2 uses MATLAB's built-in findpeaks function to locate the frequency peaks in the amplitude spectrum of all components and retains the corresponding frequency values. S3 calculates the greatest common divisor among all peak frequency values ​​of each component and the greatest common divisor among the components, and reassembles the component signals with a common fundamental frequency; S4 calculates the sideband values ​​between all peak frequency values ​​of the remaining components and reassembles the component signals with the same value; S5 provides a textual description of the recombined signal.

2. The method for converting complex signals to text in an auxiliary multimodal large model according to claim 1, characterized in that, In step S1, the component signals are obtained by signal decomposition algorithms, including matrix singular value decomposition, matrix singular spectrum decomposition, tensor singular value decomposition, and local mean decomposition.

3. The method for converting complex signals to text in an auxiliary multimodal large model according to claim 2, characterized in that, In step S2, the retained peak frequency value is rounded up, rounded down, or rounded to one decimal place to reduce the error caused by frequency fluctuations and thus improve the accuracy of clustering.

4. The method for converting complex signals to text in an auxiliary multimodal large model according to claim 3, characterized in that, Step S3 includes the following specific steps: 41) For each component signal, calculate the greatest common divisor among all its frequency peaks. If the amplitude spectrum of the component signal has only one peak, then the frequency of that peak is taken as the greatest common divisor of the component. If no valid greatest common divisor is found in the component signal, then the component is marked as "ungrouped component". 42) For component signals with the greatest common divisor, further iterate to calculate whether there is a common fundamental frequency among the components. If there is, reassemble these component signals with a common fundamental frequency. For component signals without a common fundamental frequency, group them separately. If there are N component signals in the end and each component signal has an independent fundamental frequency, then divide these N component signals into an independent group.

5. The method for converting complex signals to text in an auxiliary multimodal large model according to claim 4, characterized in that, Step S4 includes the following specific steps: 51) For the components marked as "not yet grouped" in step 41, calculate the sideband values ​​between all their frequency peaks, that is, calculate whether the difference between adjacent peak frequencies is a fixed value. 52) For component signals without sideband values, reassemble them into a set of signals; 53) For component signals with sideband values, further determine whether the sideband values ​​between components are the same, and then reassemble the component signals with common sideband values; for component signals without common sideband values, group them separately.

6. The method for converting complex signals to text in an auxiliary multimodal large model according to claim 5, characterized in that, Step S5 includes the following specific steps: 61) For the signals grouped in step 52, describe the signal characteristics in text from the amplitude spectrum and its phase spectrum characteristics. The content includes the fundamental frequency, fundamental frequency amplitude and phase, and second harmonic frequency, amplitude and phase. If the signal only has a fundamental frequency and no harmonics, it is only necessary to describe the fundamental frequency, fundamental frequency amplitude and phase of the signal; 62) For the signal grouped in step 53, draw its corresponding envelope spectrum, then find the frequency peaks in the envelope spectrum and describe them in text. If there is a common fundamental frequency among the peaks, the text description includes the fundamental frequency, fundamental frequency amplitude and phase, and the second harmonic frequency, amplitude and phase. If there is no harmonic relationship between the peaks, only describe the frequency, amplitude and phase of each peak.

Images