Multi-modal data intelligent chart query method and system based on large model

Abstract

The invention discloses a multi-modal data intelligent chart query method and system based on a large model. The method comprises the steps that fuzzy query information input by a user and real-time monitoring data of a machine tool are acquired; performing intention recognition and entity extraction on the fuzzy query information, and analyzing the fuzzy query information into a structured query condition; establishing a multi-modal data association library based on a large model, inputting the real-time monitoring data of the machine tool into the multi-modal data association library, and storing the real-time monitoring data according to a preset association requirement; and inputting a structured query condition into the established multi-modal data association library, generating an interactive chart according to output information of the multi-modal data association library, and embedding a key conclusion abstract. According to the method, the user intention is understood through a large model, the association chart is dynamically recommended, a traditional fixed menu navigation mode is broken through, and data and chart conjoint analysis intelligent interaction is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial intelligence and numerical control machining, and particularly to a multi-modal data intelligent chart query method and system based on a large model. Background Art

[0002] The data analysis of traditional rolling machines relies on professionals to manually query charts (such as vibration spectra, Campbell diagrams) through special software, and there are the following pain points: Complex operation: It is necessary to be familiar with professional terms and tool interfaces, and it is difficult for non-technical personnel to operate independently; Response delay: Manually screening data is time-consuming and cannot respond to the dynamic working condition requirements in real time; Knowledge island: Data and charts are scattered in different modules, lacking a unified interaction entry.

[0003] Although existing systems support chart generation, they lack the intelligent interaction ability driven by natural language and cannot achieve end-to-end automation of "intention - data - chart". Summary of the Invention

[0004] Therefore, the purpose of the present invention is to provide a multi-modal data intelligent chart query method and system based on a large model, which uses a large model to understand user intentions, dynamically recommends associated charts, breaks through the traditional fixed menu navigation mode, and realizes intelligent interaction for joint analysis of data and charts.

[0005] In order to achieve the above purpose, a multi-modal data intelligent chart query method based on a large model provided by the present invention includes the following steps: S1. Obtain the fuzzy query information input by the user and the real-time monitoring data of the machine tool; S2. Perform intention recognition and entity extraction on the fuzzy query information, and parse it into structured query conditions; S3. Establish a multi-modal data association library based on the large model, input the real-time monitoring data of the machine tool into the multi-modal data association library, and store it according to the preset association requirements; S4. Input the structured query conditions into the established multi-modal data association library, generate an interactive chart according to the output information of the multi-modal data association library, and embed a key conclusion summary.

[0006] Further, in S2, performing intention recognition and entity extraction on the fuzzy query information, and parsing it into structured query conditions includes: Adopt a multi-head attention mechanism to perform intention recognition and entity extraction in the fuzzy query information input by the user; The multi-head attention mechanism includes a shared encoding layer, an intention recognition branch, and an entity extraction branch; Represent the input as a sequence of word embeddings after segmenting the user query text; Perform single-head attention calculation, multi-head concatenation, and linear transformation on the input sequence of word embeddings; Adopt an intent recognition branch and an entity extraction branch to calculate the loss functions respectively; Calculate the total loss, jointly train the relevance of the intent recognition and entity recognition tasks, and finally achieve high-precision structured query condition parsing; form an SQL query statement.

[0007] Furthermore, it also includes formulating a self-learning optimization mechanism: recording users' frequent queries and feedback, continuously optimizing the domain knowledge base and chart generation rules of the large model, and improving the query accuracy.

[0008] Furthermore, the multimodal data association library includes numerically controlled machine tool system data, sensor time-domain graph data, sensor frequency-domain graph data, workpiece quality data, and expert experience, which are associated and mapped according to the requirements of data relevance; The numerically controlled machine tool system data includes: position data, speed data, tool data, load data, and pressure data; The sensor time-domain graph data includes: vibration sensor data, temperature sensor data; The sensor frequency-domain graph data includes: spectrograms and Campbell diagrams formed after frequency-domain conversion of vibration sensor data; The workpiece quality data includes: dimensional accuracy data, surface roughness data, quality assessment results, and quality index data; The expert experience includes: equipment maintenance expert fault troubleshooting and repair experience data, process optimization expert experience data.

[0009] Still further, the multimodal data association library also includes hierarchical storage of all data and establishing an association mapping table for the hierarchically stored data; After hierarchical storage, the multimodal data association library includes: A structured data layer for storing relational data, including: machine tool basic information, process parameters, and production data; A time-series data layer for storing the real-time data of sensors and the frequency-domain data after Fourier transform according to time series; An unstructured data layer for storing expert experience, including fault troubleshooting rules and process optimization strategies formed according to expert experience.

[0010] Optionally, the preset relevance requirements include: Establish a direct relationship table, which includes: machine tool operation status table, sensor data table, workpiece quality table; Establish an indirect data table, which includes a maintenance record table and a process parameter optimization table; Establish foreign key associations between various data tables through the machine tool ID and timestamp; Define indirect influence factors and dynamically calculate the association strength through a statistical model.

[0011] Furthermore, the establishment of foreign key associations between various data tables through the machine tool ID and timestamp includes: Associate the load data with the vibration sensor data through the machine tool ID and timestamp to generate a time-domain graph or frequency-domain graph corresponding to the load data; Pre-define a rule library according to expert experience, establish the association between query conditions and corresponding type charts, and establish the association between chart threshold data and failure causes according to monitoring thresholds; Train a supervised model, input maintenance records, process parameters, and load data into the supervised model, and output the indirect influence weight on the workpiece quality; Define the nodes and edges of failure phenomena and causes, establish causal associations, construct a Bayesian network, and verify the causal strength through historical data.

[0012] The present invention also provides a multi-modal data intelligent chart query system based on a large model for implementing the steps of the above-mentioned multi-modal data intelligent chart query method based on a large model, including: A data acquisition module for acquiring real-time monitoring data of a machine tool; A user input module for acquiring fuzzy query information input by a user; An information extraction module for performing intention recognition and entity extraction on the fuzzy query information and parsing it into structured query conditions; A database establishment module for storing the machine tool numerical control system data, sensor time-domain graph data, sensor frequency-domain graph data, workpiece quality data, and expert experience according to the preset association requirements based on a large model; A chart interaction module for inputting the structured query conditions into the established multi-modal data association library, generating an interactive chart according to the output information of the multi-modal data association library, and embedding a key conclusion summary.

[0013] The present invention also provides an electronic device, including: A memory storing computer program instructions; A processor, when the computer program instructions are executed by the processor, implementing the steps of the above-mentioned multi-modal data intelligent chart query method based on a large model.

[0014] The present invention also provides a computer-readable storage medium for storing instructions, when the stored instructions are run on a computer, causing the computer to execute the steps of the above-mentioned multi-modal data intelligent chart query method based on a large model.

[0015] The multi-modal data intelligent chart query method and system based on large models disclosed in this application use large models to understand user intentions and dynamically recommend associated charts, breaking through the traditional fixed menu navigation mode and realizing intelligent interaction for joint analysis of data and charts.

[0016] This application adjusts the chart detail level according to user needs and the current working conditions, and continuously optimizes the domain knowledge base of the large model and the chart generation rules by recording users' frequent queries and feedback, improving the query accuracy rate.

[0017] This application has shortened the chart query time from the minute level to the second level, reducing the operator training cost by 50%; through multi-chart correlation analysis, the accuracy rate of judging the root cause of anomalies has been increased to over 90%; it has transformed implicit experience into explicit chart rules, supporting new employees to get started quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic flow chart of the multi-modal data intelligent chart query method based on large models of the present invention.

[0019] Figure 2 It is a structural block diagram of the multi-modal data intelligent chart query system based on large models provided by the present invention.

[0020] Figure 3 It is a schematic diagram of chart interaction output after a user's question in Embodiment 1 of the present invention.

[0021] Figure 4 It is a schematic diagram of chart interaction output after a user's question in Embodiment 2 of the present invention.

[0022] Figure 5 It is a schematic diagram of workpiece monitoring chart interaction corresponding to the first abnormal data in Embodiment 2 of the present invention.

[0023] Figure 6 It is a summary display diagram of possible causes and solutions for abnormal data in Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0025] As Figure 1 shown, an embodiment of one aspect of the present invention provides a multi-modal data intelligent chart query method based on large models, including the following steps: S1. Obtain the fuzzy query information input by the user and the real-time monitoring data of the machine tool; it should be noted that the real-time monitoring data of the machine tool in this application is stored in real time, so when performing data analysis later, historical data is output and analyzed for the time range of the user's question. The fuzzy query information input by the user can be classified according to the intention, mainly including: data query category, fault analysis category and optimization prediction category; The data query class outputs data based on the entity queried by the user, such as "trend chart of abnormal fluctuations in rolling pressure in the last two hours"), performs intent recognition and entity extraction, and parses it into structured query conditions: time range, monitoring items, and chart type.

[0026] The fault analysis category retrieves the corresponding time domain graph and spectrum graph based on the user's query question and extracts the abnormal components in the graph. For example, "The surface roughness of the workpiece processed in the past two days has increased. Is there an abnormality in the cutting load of the machine tool?" The structured query conditions are extracted: time range, surface roughness, and cutting load abnormality; the time domain graph and frequency domain graph of the sensor during this period are retrieved, and abnormal signals in the time domain graph or frequency domain graph such as increased or decreased amplitude and the appearance of high-frequency components are extracted for analysis, and adjustment suggestions are given.

[0027] The optimization prediction class retrieves the corresponding time domain graph and spectrum graph according to the user's query question, and extracts the abnormal components in the graph; for example: When the wear of the guide rail and the lead screw is serious, can the wear trend be predicted by analyzing the machine tool data? The structured query conditions extracted are guide rail, lead screw, wear, machine tool data, and predicted wear trend; according to the load data fluctuation monitored by the sensor, the change of the friction resistance and the high-frequency components of the vibration data in the frequency domain graph are reflected, and the relationship between the abnormal changes in friction resistance and vibration and the degree of wear is analyzed, and the future wear trend is predicted based on historical data and current data.

[0028] S2, performing intent recognition and entity extraction on the fuzzy query information, and parsing it into a structured query condition; further, performing intent recognition and entity extraction on the fuzzy query information, and parsing it into a structured query condition; including: A multi-head attention mechanism is used to perform intent recognition and entity extraction in the fuzzy query information input by the user; The multi-head attention mechanism includes a shared encoding layer, an intent recognition branch, and an entity extraction branch; Structured query conditions are formed according to the preset template to form SQL query statements.

[0029] Furthermore, it also includes the development of a self-learning optimization mechanism: recording users' high-frequency queries and user feedback, continuously optimizing the domain knowledge base and graph generation rules of the large model, and improving query accuracy.

[0030] This application uses a deep learning model based on the multi-head attention mechanism. On the basis of the shared encoding layer, an intent recognition branch and an entity extraction branch are constructed, and the structured parsing of user fuzzy query information is realized through joint training. The model framework design consists of three parts, namely the shared encoding layer, the multi-head attention layer, and the dual-task branch.

[0031] The shared encoding layer provides consistent input features for all attention heads, but each attention head projects the shared features from different perspectives through independent linear transformations (query, key, value weight matrices).

[0032] The multi-head attention layer extracts semantic features of different dimensions through the multi-head attention mechanism.

[0033] The dual-task branch includes intent recognition (classification task) and entity extraction (sequence labeling task).

[0034] The input is represented as a sequence of word embeddings obtained by tokenizing the user query text, ; where n is the sequence length and d is the embedding dimension. The multi-head attention splits the embedding into h independent attention heads, and each head focuses on different semantic features.

[0035] Queries (Q), keys (K), and values (V) are generated through linear transformations, , , , where .

[0036] Single-head attention calculation .

[0037] Multi-head concatenation and linear transformation , where is the output weight matrix.

[0038] The intent recognition branch implemented by the dual-task branch is as follows: The embedding representation of the input CLS Token , The classification layer , The loss function cross-entropy loss .

[0039] The entity extraction branch implemented by the dual-task branch is as follows: The embeddings of all input Tokens , The sequence labeling layer , Loss function: CRF negative log-likelihood 。

[0040] Joint training and loss balancing. The total loss is the weighted sum of the two tasks, and the training stability is balanced through dynamic weights. with an initial value During training, it is dynamically adjusted according to the task difficulty. In the shared encoding layer, the two branches are independently trained, and the parameters of the classification / annotation layer are shared. After the model outputs the intent category and entity label, they are mapped to a structured SQL statement according to a preset template. The multi-head attention mechanism is used to enhance the model's ability to capture complex semantics, and the joint training strategy balances the relevance between the intent and entity tasks, ultimately achieving high-precision structured query condition parsing.

[0041] S3. Establish a multi-modal data association library based on a large model, input the real-time monitoring data of the machine tool into the multi-modal data association library, and store it according to the preset relevance requirements; Further, the multi-modal data association library includes the numerical control system data of the machine tool, the time-domain graph data of the sensor, the frequency-domain graph data of the sensor, the workpiece quality data, and the expert experience, which are associated and mapped according to the data relevance requirements; The numerical control system data of the machine tool includes: position data, speed data, tool data, load data, and pressure data; The time-domain graph data of the sensor includes: vibration sensor data, temperature sensor data; The frequency-domain graph data of the sensor includes: the spectrogram and Campbell diagram formed after the frequency-domain conversion of the vibration sensor data; The workpiece quality data includes: dimensional accuracy data, surface roughness data, quality assessment results, and quality index data; The expert experience includes: equipment maintenance expert's fault troubleshooting and repair experience data, process optimization expert's experience data.

[0042] Still further, the multi-modal data association library also includes hierarchical storage of all data, and an association mapping table is established for the hierarchically stored data; After hierarchical storage, the multi-modal data association library includes: The structured data layer, used to store relational data, including: machine tool basic information, process parameters, and production data; The time-series data layer, used to store the real-time data of the sensor and the frequency-domain data after Fourier transform in time series; The unstructured data layer, used to store expert experience, including fault troubleshooting rules and process optimization strategies formed according to expert experience.

[0043] Optionally, the association mapping according to the data relevance requirements includes: Establish a direct relationship table, where the direct relationship table includes: a machine tool operation status table, a sensor data table, and a workpiece quality table; Establish an indirect data table, where the indirect data table includes a maintenance record table and a process parameter optimization table; Establish foreign key associations between each data table through the machine tool ID and timestamp; Define an indirect influence factor and dynamically calculate the association strength through a statistical model. The indirect influence factor is used to measure the indirect relevance between non-directly associated parameters through mediating factors or historical data patterns. Specifically, the parameter relevance it measures includes indirect causal association and statistical dependence association. For indirect causal association, for example, the association between maintenance records (such as lubrication frequency) and workpiece quality. Maintenance records themselves do not directly affect quality, but indirectly act on workpiece surface roughness or dimensional accuracy by affecting machine tool stability (such as reducing vibration or wear). The association between process parameter optimization (such as cutting speed adjustment) and failure rate. Parameter optimization may indirectly reduce the probability of abnormal vibration or temperature rise by reducing tool load fluctuations. For statistical dependence association, for example, the association between load data and sensor frequency domain characteristics (such as high-frequency components in the spectrogram). Load changes may indirectly cause abnormal frequency domain data by triggering machine tool resonance or mechanical stress. The association between historical process parameters and expert experience rules. By analyzing the implicit relationship between historical parameter settings and expert-recommended optimization strategies through a statistical model, the association strength is dynamically adjusted.

[0044] Furthermore, the establishment of foreign key associations between each data table through the machine tool ID and timestamp includes: Associate load data with vibration sensor data through the machine tool ID and timestamp to generate a time domain graph or frequency domain graph corresponding to the load data; According to the pre-defined rule library of expert experience, establish the association between query conditions and corresponding type charts, and establish the association between chart threshold data and failure causes according to the monitoring threshold; Train a supervised model, input maintenance records, process parameters, and load data into the supervised model, and output the indirect influence weight on workpiece quality; Establish a causal association, construct a Bayesian network, and verify the causal strength through historical data. By defining nodes and edges, transform expert experience such as "unstable hydraulic system may cause workpiece quality problems" into a computable probability relationship model to make implicit knowledge explicit.

[0045] S4. Input the structured query conditions into the established multi-modal data association library, generate an interactive chart according to the output information of the multi-modal data association library, and embed a summary of key conclusions.

[0046] Such as Figure 2, the present invention also provides a multi-modal data intelligent chart query system based on a large model for implementing the steps of the above-mentioned multi-modal data intelligent chart query method based on a large model, including: A data acquisition module for acquiring real-time monitoring data of a machine tool; A user input module for acquiring fuzzy query information input by a user; An information extraction module for performing intent recognition and entity extraction on the fuzzy query information and parsing it into structured query conditions; A database establishment module for storing machine tool numerical control system data, sensor time-domain graph data, sensor frequency-domain graph data, workpiece quality data, and expert experience based on a large model according to preset relevance requirements; A chart interaction module for inputting the structured query conditions into the established multi-modal data association library, generating an interactive chart according to the output information of the multi-modal data association library, and embedding a summary of key conclusions.

[0047] The present invention also provides an electronic device, including: A memory storing computer program instructions; A processor, when the computer program instructions are executed by the processor, implementing the steps of the above-mentioned multi-modal data intelligent chart query method based on a large model.

[0048] The present invention also provides a computer-readable storage medium for storing instructions, when the stored instructions run on a computer, causing the computer to execute the steps of the above-mentioned multi-modal data intelligent chart query method based on a large model.

[0049] Embodiment 1 This application classifies the data intent in the fuzzy query information input by the user. If it belongs to the data query category, such as querying production data, when the entity in the user's question contains (production / monitoring) data for a specific time period, such as: production data for the day before yesterday / yesterday / today: As Figure 3 shown, the production data report: the data statistical dimension is within 1 day + 1 machine tool + 1 part number, that is, to count the production quantity of part a produced on machine tool A on the current day; the production quantity is counted according to the total number of processing serial numbers on the current day; the start time is recorded according to the start time of the first workpiece of the part produced by the machine tool on the current day; the end time is recorded according to the start time of the last workpiece of the part produced by the machine tool on the current day; the list is in ascending order according to the start time of a certain machine tool.

[0050] Chart data expectation: If there is no production data on the current day / at a certain time, it is necessary to inform that there is no production. For example, there is no production data for the machine tool on February 6, 2025. When exceeding the data output boundary, give a friendly prompt.

[0051] Example 2 This application also includes the ability to directly query the production curve. For example: The query information is: the production process curve of machining '** part' on the '** machine tool (machine tool ID)' the day before yesterday / yesterday / today at ** process step / (or all process steps).

[0052] The chart interaction display result is as Figure 4 shown. The production process curve of machining workpiece b on machine tool B on February 28, 2025 at process step / (or all process steps) 1 is as follows: (Note: For multiple abnormal parts, the production process curves, problems and solutions are displayed separately) Among them, the production process curve of the part with processing abnormality includes: the production process curve of the 170th workpiece as Figure 5 shown. Figure 5 The rightmost coordinate axes all represent the vibration amount.

[0053] Finally, a conclusion is given. The possible problems and solutions through analysis are as follows: The monitoring items of the same machine tool, the same part number, and the same processing sequence on the same day need to be de-duplicated. The possible reasons and solutions are summarized and shown as Figure 6 shown.

[0054] Example 3 This application can also be classified according to user roles. For example, the production department mainly focuses on the production volume data, while the process department mainly focuses on the workpiece quality data. When the process department asks: the abnormal data of the day before yesterday / yesterday / today.

[0055] Prototype note: As long as the question contains the key content, the corresponding content can be output. The large language model needs to understand the time including the day before yesterday, yesterday, today, etc., and can also understand and replace specific time parameters. For example, the abnormal data on February 6, 2025. Abnormal data report: The data statistical dimension is within 1 day + 1 machine tool + 1 part number + 1 monitoring item, that is, to count the data of each monitoring item that exceeds the upper and lower limits generated by machining part a on machine tool A on the current day; the alarm time is recorded according to the first alarm time generated when a certain monitoring item of the machine tool and part number exceeds the tolerance on the current day; the list order is in ascending order of the alarm time of the same machine tool and the same part.

[0056] Chart data expectation: If there is no abnormal data on the current day / at a certain time, it is necessary to inform that there is no abnormality. For example, there is no abnormal data for the machine tool on February 6, 2025. When exceeding the data output boundary, give a friendly prompt.

[0057] Question: Generate the order Campbell diagram of machining 'Part Name' in 'Processing Step' on 'Machine Tool ID' in the last two weeks. The order Campbell diagram of machining part b in processing step 1 on machine tool B from February 15, 2025 to February 28, 2025: Through analysis, it is found that the amplitude of the 111th machined workpiece is too high at 08:00:40.000 on February 16, 2025. Through analysis, it is found that there is an out-of-tolerance phenomenon in the 33rd order.

[0058] Example 4 When the fuzzy query information input by the user is of the fault analysis type, for example: Recently, we have found that the surface roughness of the machined workpieces has increased. Is there an abnormality in the cutting load of the machine tool? Answer from the large model application: By analyzing the cutting load data in the machine tool numerical control system, we found that the cutting load has fluctuated greatly recently, especially in the high-speed cutting stage.

[0059] Combined with the time-domain diagram of the sensor data, it is observed that the cutting force shows an unstable trend over time.

[0060] The frequency-domain diagram shows that the high-frequency components in the cutting force have increased, which may mean increased tool wear or improper cutting parameter settings.

[0061] It is recommended to check the tool wear condition and optimize the cutting parameters to reduce the surface roughness of the workpiece.

[0062] Another example: The vibration of the machine tool is abnormal during processing. Does it have an impact on the workpiece quality? Answer from the large model application: By analyzing the feed force data in the machine tool numerical control system, we found that the feed force fluctuates significantly during vibration.

[0063] The time-domain diagram of the sensor data shows that the vibration causes a significant increase in the amplitude.

[0064] The frequency-domain diagram further reveals the frequency components of the vibration, which are close to the natural frequency of the machine tool and may cause resonance.

[0065] This vibration has a negative impact on the workpiece quality, such as surface waviness and dimensional accuracy degradation.

[0066] It is recommended to adjust the machine tool parameters, avoid operating near the resonance frequency, and enhance the stability of the machine tool.

[0067] Example 5 When the fuzzy query information input by the user is of the optimization prediction type, for example: Question: We have noticed that the wear of the guide rails and lead screws of a certain machine tool is relatively serious after long-term operation. Can we analyze the machine tool data to predict the wear trend and formulate a preventive maintenance plan? Answer for large model application: By analyzing the load and pressure data in the machine tool numerical control system and the time-domain and frequency-domain diagrams of sensor data, we can monitor the wear conditions of the guide rails and lead screws.

[0068] The fluctuations in the load data can reflect the changes in the frictional resistance of the guide rails and lead screws during movement.

[0069] The time-domain diagram of the sensor data can show the vibration conditions of the guide rails and lead screws, while the frequency-domain diagram can reveal the frequency components of the vibration, which are closely related to the wear degree of the guide rails and lead screws.

[0070] It is recommended to establish a wear prediction model for the guide rails and lead screws to predict the future wear trend based on historical data and current data.

[0071] Formulate a preventive maintenance plan according to the prediction results, such as regular lubrication, adjustment of the clearances of the guide rails and lead screws, replacement of severely worn components, etc.

[0072] Obviously, the above embodiments are only examples given for clear illustration and are not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or variations can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. And the obvious changes or variations derived therefrom are still within the protection scope of the present invention.

Claims

1. A multi-modal data intelligent chart query method based on a large model, characterized in that, It includes the following steps: Obtain the fuzzy query information input by the user and the real-time monitoring data of the machine tool; Perform intent recognition and entity extraction on the fuzzy query information, and parse it into structured query conditions; Establish a multi-modal data association library based on a large model, input the real-time monitoring data of the machine tool into the multi-modal data association library, and store it according to the preset relevance requirements; Input the structured query conditions into the established multi-modal data association library, generate an interactive chart according to the output information of the multi-modal data association library, and embed a summary of key conclusions.

2. The multimodal data intelligent chart query method based on a large model according to claim 1, wherein Perform intent recognition and entity extraction on the fuzzy query information, and parse it into structured query conditions; including: Adopt a multi-head attention mechanism to perform intent recognition and entity extraction in the fuzzy query information input by the user; The multi-head attention mechanism includes a shared encoding layer, an intent recognition branch, and an entity extraction branch; Represent the input as a sequence of word embeddings obtained by tokenizing the user's query text; Perform single-head attention calculation, multi-head concatenation, and linear transformation on the input sequence of word embeddings; Adopt the intent recognition branch and the entity extraction branch to calculate the loss function respectively; Calculate the total loss, jointly train the relevance of the intent recognition and entity recognition tasks, and finally achieve high-precision structured query condition parsing; form an SQL query statement.

3. The multi-modal data intelligent chart query method based on a large model according to claim 2, wherein It also includes formulating a self-learning optimization mechanism: record the user's frequent queries and feedback, continuously optimize the domain knowledge base of the large model and the chart generation rules, and improve the query accuracy.

4. The intelligent chart query method for multimodal data based on large models according to claim 1, characterized in that, The multi-modal data association library is formed by associating and mapping the machine tool numerical control system data, sensor time-domain diagram data, sensor frequency-domain diagram data, workpiece quality data, and expert experience according to the data relevance requirements; The machine tool numerical control system data includes: position data, speed data, tool data, load data, and pressure data; The sensor time-domain diagram data includes: vibration sensor data, temperature sensor data; The sensor frequency-domain diagram data includes: spectrograms and Campbell diagrams formed after frequency-domain conversion of vibration sensor data; The workpiece quality data includes: dimensional accuracy data, surface roughness data, quality evaluation results, and quality index data; The expert experience includes: equipment maintenance expert fault troubleshooting and repair experience data, process optimization expert experience data.

5. The intelligent chart query method for multimodal data based on a large model according to claim 3, wherein The multi-modal data association library also includes hierarchical storage of all data, and establishing an association mapping table for the hierarchically stored data; After hierarchical storage, the multi-modal data association library includes: A structured data layer for storing relational data, including: machine tool basic information, process parameters, and production data; A time-series data layer for storing the real-time data of sensors and the frequency-domain data after Fourier transform in time series; An unstructured data layer for storing expert experience, including fault troubleshooting rules and process optimization strategies formed according to expert experience.

6. The intelligent chart query method for multimodal data based on large models according to claim 3 or 4, characterized in that The preset relevance requirements include: Establish a direct association table, and the direct association table includes: machine tool operation status table, sensor data table, workpiece quality table; Establish foreign key associations between the direct association tables through the machine tool ID and timestamp to achieve direct data linkage; An indirect association mapping table is established, and the indirect association mapping table includes a maintenance record table and a process parameter optimization table; Calculate the association strength between the indirect association mapping tables through the weight association rule, including defining an indirect influence factor and dynamically calculating the association strength through a statistical model.

7. The intelligent multi-modal data intelligent chart query method based on a large model according to claim 6, characterized in that The establishment of a foreign key association between each direct association table through the machine tool ID and the timestamp includes: Associate the load data with the vibration sensor data through the machine tool ID and the timestamp, and generate a time-domain graph or a frequency-domain graph corresponding to the load data; Pre-define a rule library according to expert experience, establish the association between the query condition and the corresponding type of chart, and establish the association between the chart threshold data and the cause of the failure according to the monitoring threshold.

8. The multimodal data intelligent chart query method based on a large model according to claim 6, wherein The calculation of the association strength between the indirect association mapping tables through the weight association rule includes: Train a supervised model, input maintenance records, process parameters, and load data into the supervised model, and output the indirect influence weight on the workpiece quality; Define the nodes and edges of the fault phenomenon and cause, establish a causal association, construct a Bayesian network, and verify the causal strength through historical data.

9. A multi-modal data intelligent chart query system based on a large model, which is used to implement the steps of the multi-modal data intelligent chart query method based on a large model according to any one of the above claims 1-8, characterized in that, Including: A data acquisition module for acquiring real-time monitoring data of the machine tool; A user input module for acquiring fuzzy query information input by the user; An information extraction module for performing intention recognition and entity extraction on the fuzzy query information and parsing it into a structured query condition; A database establishment module stores the machine tool numerical control system data, sensor time-domain graph data, sensor frequency-domain graph data, workpiece quality data, and expert experience according to the data based on the large model according to the preset relevance requirements; A chart interaction module inputs the structured query condition into the established multi-modal data association library, generates an interactive chart according to the output information of the multi-modal data association library, and embeds a key conclusion summary.

10. An electronic device, characterized in that, Including: A memory storing computer program instructions; A processor, when the computer program instructions are executed by the processor, implements the steps of the intelligent multi-modal data intelligent chart query method based on a large model according to any one of claims 1 to 8.

Images