A collaborative control system and method for CNC machine tools based on multimodal fusion

Abstract

The present invention provides a collaborative control system and method for CNC machine tools based on multimodal fusion, belonging to the field of machine tool control technology. The system comprises an acquisition module that acquires multiple modes of the CNC machine tool and collects state information of all modes of the CNC machine tool in real time to determine real-time modal data; a construction module that acquires historical modal data of all modes of the CNC machine tool and constructs an attention mechanism model; an analysis module that determines the modal feature matrix of all modes and the fused feature vector of the CNC machine tool; a control module that determines a collaborative control strategy for the CNC machine tool based on the historical modal data, the modal feature matrix, and the fused feature vector; and an execution module that executes the coordinated control strategy to implement coordinated control of the CNC machine tool based on multimodal fusion. This system can form an efficient collaborative control mechanism, improve the intelligence and precision of machine tool control, enhance machining accuracy and production efficiency, and enhance the adaptability and operational efficiency of CNC machine tools under complex working conditions.

Description

Technical Field

[0001] The present invention relates to the field of machine tool control technology, and in particular to a collaborative control system and method for CNC machine tools based on multimodal fusion. Background Art

[0002] CNC machine tools (CNC) play a vital role in modern manufacturing. With the rise of industrial automation and intelligent manufacturing, traditional CNC machine tools are increasingly turning to multimodal control to improve machining accuracy, efficiency, and adaptability to complex working environments. Early CNC systems primarily relied on simple sensor data (such as position and speed) for control. These systems could only process single-modal data and lacked flexibility and adaptability when faced with complex machining tasks.

[0003] With technological advancements, multimodal data fusion technology has emerged. This technology can simultaneously process multiple data types (such as temperature, pressure, and vibration) from multiple sensors, thereby providing more comprehensive and accurate machine tool status information. However, technical issues such as low data fusion efficiency and insufficient real-time performance still exist.

[0004] Therefore, the present invention provides a collaborative control system and method for CNC machine tools based on multimodal fusion. Summary of the Invention

[0005] The present invention provides a collaborative control system and method for CNC machine tools based on multimodal fusion. This system analyzes real-time modal data to determine a modal feature matrix, then determines a fused feature vector based on the modal feature matrix and a constructed attention mechanism model. A collaborative control strategy is then determined based on historical modal data, the modal feature matrix, and the fused feature vector. This coordinated control strategy is then executed to achieve coordinated control of CNC machine tools based on multimodal fusion. This system can form an efficient collaborative control mechanism, improve the intelligence and precision of machine tool control, enhance machining accuracy and production efficiency, and enhance the adaptability and operational efficiency of CNC machine tools under complex working conditions.

[0006] In one aspect, the present invention provides a collaborative control system for CNC machine tools based on multimodal fusion, comprising:

[0007] Acquisition module: acquires multiple modes of the CNC machine tool, collects the status information of all modes of the CNC machine tool in real time to determine the real-time modal data;

[0008] Construction module: Obtain historical modal data of all modes of CNC machine tools and build an attention mechanism model;

[0009] Analysis module: Determines the modal feature matrix of all modes based on real-time modal data, and determines the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model;

[0010] Control module: Determines the collaborative control strategy of the CNC machine tool based on historical modal data, modal feature matrix, and fused feature vector;

[0011] Execution module: executes the coordinated control strategy to realize the coordinated control of CNC machine tools based on multimodal fusion.

[0012] According to the present invention, a CNC machine tool collaborative control system based on multimodal fusion is provided, wherein the acquisition module includes:

[0013] The first acquisition unit is used to acquire the machine type, structure and parameters of the machine tool, and to acquire the characteristics and processing technology of the part to be processed;

[0014] Mode determination unit: determines multiple modes of the CNC machine tool based on the machine tool type, machine tool structure, machine tool parameters, part characteristics of the part to be processed, and processing technology;

[0015] Modal sensor group unit: determines a modal sensor group for each mode based on the structure of the CNC machine tool, wherein the modal sensor group includes at least one modal sensor;

[0016] Sub-real-time modal data unit: collects state information of each modality based on the modal sensor group of each modality and determines the sub-real-time modal data of each modality;

[0017] Real-time modal data unit: determines the real-time modal data of the CNC machine tool based on the sub-real-time modal data of all modes.

[0018] According to the present invention, a collaborative control system for CNC machine tools based on multimodal fusion is provided, and a building module includes:

[0019] Historical modal data unit: obtains historical modal data of all modes processed by the CNC machine tool multiple times, where the historical modal data includes historical environment vectors, historical task vectors, historical modal vectors of each mode, historical modal weights, historical fusion vectors, and optimal modal matrix;

[0020] Initialization unit: Initializes the model parameters of the attention mechanism model, where the model parameters include at least the initial modal transformation weight matrix, the initial attention weight vector, the initial dimensionality reduction weight matrix, and the initial bias vector;

[0021] Model training unit: Trains the initialized attention mechanism model based on all processed historical modal data, uses the historical modal vectors of all modalities in all processed historical modal data as the model input of the attention mechanism model, and uses the historical modal weights and historical fusion vectors of all modalities in all processed historical modal data as the model output of the attention mechanism model;

[0022] Prediction unit: The attention mechanism model performs forward propagation analysis on the model input to determine the predicted modal weight and predicted fusion vector for each modality processed each time;

[0023] A first comparison unit: performing a first comparison on the historical modal weight and the predicted modal weight of each modality in the historical modal data processed each time;

[0024] Adjustment unit: The attention mechanism model performs backpropagation analysis based on the first comparison result, adjusts the initialization model parameters of the attention mechanism model based on the analysis result, and determines the adjustment parameters, wherein the adjustment parameters at least include adjusting the transformation weight matrix, adjusting the attention weight vector, adjusting the dimensionality reduction weight matrix, and adjusting the bias vector;

[0025] A second comparison unit: performing a second comparison on the historical fusion vector and the predicted fusion vector in the historical modal data processed each time;

[0026] Model evaluation unit: evaluates the attention mechanism model based on the second comparison result.

[0027] According to the present invention, a collaborative control system for CNC machine tools based on multimodal fusion, an analysis module, includes:

[0028] Modal eigenvector unit: preprocesses the sub-real-time modal data of each mode in the real-time modal data, extracts features from the preprocessed sub-real-time modal data of each mode, and determines the modal eigenvector of each mode;

[0029] ;

[0030] in, represents the modal eigenvector of the i-th mode at the current time t, They represent the 1st feature, jth feature, and N1th feature of the i-th mode at the current time t, respectively, where N1 represents the number of features in the modal feature vector;

[0031] Modal characteristic matrix unit: determines the modal characteristic matrix based on the modal eigenvectors of all modes;

[0032] ;

[0033] in, represents the modal characteristic matrix at the current time t, They represent the modal eigenvectors of the i-th mode at the current time t, and N2 represents the number of modes.

[0034] According to the present invention, a collaborative control system for CNC machine tools based on multimodal fusion, the analysis module further includes:

[0035] Fusion feature vector unit: Determines the fusion feature vector of the CNC machine tool based on the modal feature vectors of all modalities, the modal feature matrix, and the adjustment parameters of the attention mechanism model;

[0036] ;

[0037] ;

[0038] ;

[0039] ;

[0040] in, represents the fused feature vector at the current time t, represents the weighted feature vector at the current time t, represents the transformation feature vector at the current time t, B represents the adjustment bias vector, Indicates the concatenation of the weighted feature vector and the transformed feature vector at the current time t. represents the relative weight of the i-th mode at the current time t, Represents the adjusted attention weight vector of the i-th modality in the adjustment parameters of the attention mechanism model, Represents the adjustment of the attention weight vector The transpose of Represents the adjustment transformation weight matrix in the adjustment parameters of the attention mechanism model, represents the activation value of the i-th mode at the current time t, represents the activation parameter, Represents the adjusted dimensionality reduction weight matrix in the adjustment parameters of the attention mechanism model.

[0041] According to the present invention, a collaborative control system for CNC machine tools based on multimodal fusion is provided, wherein the control module includes:

[0042] Feature extraction unit: This unit obtains the real-time environment data, real-time processing tasks, and processing task requirements of the CNC machine tool, performs feature extraction on the real-time environment data of the CNC machine tool, and determines the real-time environment vector. At the same time, it performs feature extraction on the real-time processing tasks and processing task requirements of the CNC machine tool and determines the real-time task vector.

[0043] A first selected modal data unit extracts historical environment vectors from all historical modal data, determines that the historical modal data corresponding to the extracted historical environment vectors whose similarity with the real-time environment vector is greater than a first pre-threshold value are first selected modal data;

[0044] The second modal data selection unit extracts historical task vectors from all historical modal data, determines that the historical modal data corresponding to the extracted historical task vectors whose similarity with the real-time task vector is greater than a second pre-threshold value are the second selected modal data;

[0045] Selecting a modal matrix unit: determining an optimal modal matrix of each historical modal data belonging to both the first modal data and the second modal data as a selected modal matrix;

[0046] Collaborative control strategy unit: Determines the collaborative control strategy based on the modal feature matrix, fused feature vectors and all selected modal matrices.

[0047] According to the present invention, a collaborative control system for a CNC machine tool based on multimodal fusion is provided, and the collaborative control strategy unit includes:

[0048] ;

[0049] ;

[0050] in, represents the error matrix at the current time t,

[0051] Nu represents the number of selected modal matrices, CM represents the fitting change matrix, represents the ath selected modal matrix, represents the best fit matrix, represents the nuclear norm of the fitted variation matrix, represents the regularization parameter, tE represents the predicted end time of the real-time processing task, represents the fusion control vector.

[0052] On the other hand, the present invention also provides a collaborative control method for CNC machine tools based on multimodal fusion, comprising:

[0053] Step 1: Multiple modes of the CNC machine tool, real-time state information of all modes of the CNC machine tool is collected to determine real-time modal data;

[0054] Step 2: Obtain historical modal data of all modes of CNC machine tools and build an attention mechanism model;

[0055] Step 3: Determine the modal feature matrix of all modes based on the real-time modal data, and determine the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model;

[0056] Step 4: Determine the collaborative control strategy of the CNC machine tool based on the historical modal data, modal feature matrix, and fusion feature vector;

[0057] Step 5: Execute the coordinated control strategy to realize the coordinated control of CNC machine tools based on multimodal fusion.

[0058] Compared with the prior art, the present invention has the following advantages:

[0059] By analyzing the real-time modal data to determine the modal feature matrix, the fused feature vector is determined based on the modal feature matrix and the constructed attention mechanism model. A collaborative control strategy is then determined based on the historical modal data, the modal feature matrix, and the fused feature vector. This coordinated control strategy is then executed to achieve coordinated control of CNC machine tools based on multimodal fusion. This can form an efficient collaborative control mechanism, improve the intelligence and precision of machine tool control, enhance machining accuracy and production efficiency, and enhance the adaptability and operational efficiency of CNC machine tools under complex working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 It is a structural diagram of a CNC machine tool collaborative control system based on multimodal fusion provided by an embodiment of the present invention.

[0062] Figure 2 It is a flow chart of a collaborative control method for CNC machine tools based on multimodal fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0064] Example 1:

[0065] The embodiment of the present invention provides a CNC machine tool collaborative control system based on multimodal fusion, such as Figure 1 Shown, including:

[0066] Acquisition module: acquires multiple modes of the CNC machine tool, collects the status information of all modes of the CNC machine tool in real time to determine the real-time modal data;

[0067] Construction module: Obtain historical modal data of all modes of CNC machine tools and build an attention mechanism model;

[0068] Analysis module: Determines the modal feature matrix of all modes based on real-time modal data, and determines the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model;

[0069] Control module: Determines the collaborative control strategy of the CNC machine tool based on historical modal data, modal feature matrix, and fused feature vector;

[0070] Execution module: executes the coordinated control strategy to realize the coordinated control of CNC machine tools based on multimodal fusion.

[0071] In this embodiment, multiple modes of a CNC machine tool (such as vibration, temperature, and pressure) are determined, and state information of these modes is collected. The data collected in real time is used to form real-time modal data, which can reflect the current operating status of the machine tool.

[0072] In this embodiment, a model based on an attention mechanism is constructed and trained using historical modal data. The attention mechanism model adjusts weights, enabling the system to weight different modal states according to their impact, enhancing the influence of important modalities. This helps to more accurately understand the role of historical data and provides high-quality data for determining collaborative control strategies.

[0073] In this embodiment, features of all modalities are extracted based on real-time modal data to generate a modal feature matrix. This is combined with an attention mechanism model to generate a fused feature vector.

[0074] In this embodiment, the control module determines the most appropriate collaborative control strategy by analyzing historical modal data, modal feature matrices, and fused feature vectors. These strategies will adjust the behavior of the machine tool to ensure it consistently performs optimally under different operating conditions.

[0075] In this embodiment, corresponding control operations are performed according to the control strategy to ensure that the operation of the machine tool complies with the optimized collaborative control solution.

[0076] The beneficial effects of the above technical solution include: determining the modal feature matrix by analyzing real-time modal data, determining the fused feature vector based on the modal feature matrix and the constructed attention mechanism model, determining the coordinated control strategy based on historical modal data, the modal feature matrix, and the fused feature vector, and executing the coordinated control strategy to achieve coordinated control of CNC machine tools based on multimodal fusion. This can form an efficient coordinated control mechanism, improve the intelligence and precision of machine tool control, enhance machining accuracy and production efficiency, and improve the adaptability and operating efficiency of CNC machine tools under complex working conditions.

[0077] Example 2:

[0078] The embodiment of the present invention provides a collaborative control system for CNC machine tools based on multimodal fusion, and an acquisition module, including:

[0079] The first acquisition unit is used to acquire the machine type, structure and parameters of the machine tool, and to acquire the characteristics and processing technology of the part to be processed;

[0080] Mode determination unit: determines multiple modes of the CNC machine tool based on the machine tool type, machine tool structure, machine tool parameters, part characteristics of the part to be processed, and processing technology;

[0081] Modal sensor group unit: determines a modal sensor group for each mode based on the structure of the CNC machine tool, wherein the modal sensor group includes at least one modal sensor;

[0082] Sub-real-time modal data unit: collects state information of each modality based on the modal sensor group of each modality and determines the sub-real-time modal data of each modality;

[0083] Real-time modal data unit: determines the real-time modal data of the CNC machine tool based on the sub-real-time modal data of all modes.

[0084] In this embodiment, basic information of the CNC machine tool (such as type, structure, parameters) as well as characteristics of the part to be processed (such as material, shape, etc.) and processing technology (such as cutting method, feed rate, etc.) are obtained.

[0085] In this embodiment, multiple modes of the CNC machine tool are determined based on the machine tool type, structure, parameters, as well as workpiece characteristics and machining process.

[0086] In this embodiment, the multiple modes of a CNC machine tool can be vibration modes: the machine tool generates vibrations during operation, and the magnitude and frequency of the vibrations reflect the machine tool's operational stability and component wear. Excessive vibration can affect machining accuracy and surface quality, and may even cause tool damage and equipment failure. Sound modes: The sounds produced during machining contain a wealth of information, such as the cutting state between the tool and the workpiece, and the operating conditions of the components. Abnormal sounds may indicate tool wear, loose components, or poor lubrication. Temperature modes: Key machine tool components, such as the spindle, motor, and bearings, generate heat during operation. Excessive temperatures can cause component thermal deformation, affecting machining accuracy and accelerating component wear and aging. Force and torque modes: Cutting force and spindle torque reflect the cutting load and machining capacity of the machine tool. By monitoring the force and torque modes, cutting parameters can be optimized, overload can be avoided, and machining efficiency and tool life can be improved. Current and voltage modes: Changes in motor current and voltage reflect the motor's operating status and load. Abnormal current and voltage values ​​may indicate motor failure or electrical system problems; Image mode: Using industrial cameras to capture images of the processing area, you can intuitively observe tool wear, workpiece processing status and clamping conditions, etc., providing a basis for timely adjustment of processing parameters and ensuring processing quality; Position and speed mode: Monitor the position and speed information of each moving part of the machine tool to ensure that the machine tool runs according to the predetermined trajectory and speed, thereby ensuring processing accuracy and production efficiency.

[0087] In this embodiment, the sensor group required for each mode is determined based on the structure of the CNC machine tool and the characteristics of different modes. Each mode may require multiple sensors to accurately collect data. The sensors can monitor different parameters such as vibration, temperature, and force.

[0088] In this embodiment, the state information of each modality (such as vibration, temperature, force, etc.) is collected in real time by the modal sensor group.

[0089] In this embodiment, all the sub-real-time data of the modalities are integrated to finally determine the real-time modal data of the CNC machine tool. This data is a real-time reflection of the current state of the machine tool.

[0090] The beneficial effects of the above technical solution are: obtaining multiple modes of the CNC machine tool, collecting the status information of all modes of the CNC machine tool in real time to determine the real-time modal data, and providing a data basis for determining the modal characteristic matrix.

[0091] Example 3:

[0092] The embodiment of the present invention provides a collaborative control system for CNC machine tools based on multimodal fusion, and the building blocks include:

[0093] Historical modal data unit: obtains historical modal data of all modes processed by the CNC machine tool multiple times, where the historical modal data includes historical environment vectors, historical task vectors, historical modal vectors of each mode, historical modal weights, historical fusion vectors, and optimal modal matrix;

[0094] Initialization unit: Initializes the model parameters of the attention mechanism model, where the model parameters include at least the initial modal transformation weight matrix, the initial attention weight vector, the initial dimensionality reduction weight matrix, and the initial bias vector;

[0095] Model training unit: Trains the initialized attention mechanism model based on all processed historical modal data, uses the historical modal vectors of all modalities in all processed historical modal data as the model input of the attention mechanism model, and uses the historical modal weights and historical fusion vectors of all modalities in all processed historical modal data as the model output of the attention mechanism model;

[0096] Prediction unit: The attention mechanism model performs forward propagation analysis on the model input to determine the predicted modal weight and predicted fusion vector for each modality processed each time;

[0097] A first comparison unit: performing a first comparison on the historical modal weight and the predicted modal weight of each modality in the historical modal data processed each time;

[0098] Adjustment unit: The attention mechanism model performs backpropagation analysis based on the first comparison result, adjusts the initialization model parameters of the attention mechanism model based on the analysis result, and determines the adjustment parameters, wherein the adjustment parameters at least include adjusting the transformation weight matrix, adjusting the attention weight vector, adjusting the dimensionality reduction weight matrix, and adjusting the bias vector;

[0099] A second comparison unit: performing a second comparison on the historical fusion vector and the predicted fusion vector in the historical modal data processed each time;

[0100] Model evaluation unit: evaluates the attention mechanism model based on the second comparison result.

[0101] In this embodiment, all modal history data of multiple processings of the CNC machine tool are collected, and each processing corresponds to a piece of historical modal data.

[0102] In this embodiment, the historical modal data includes multiple aspects: historical environment vector: representing factors in the processing environment, such as temperature, humidity, air pressure, etc.; historical task vector: describing the parameters of each processing task, such as processing time, tool type, etc.; historical modal vector: performance data of each mode (such as vibration, heat, etc.) in historical processing; historical modal weight: the relative importance of each mode to processing accuracy; historical fusion vector: the fusion result of multimodal data, representing the overall processing state; optimal modal matrix: the modal data matrix that can represent the optimal processing state among all historical data.

[0103] In this embodiment, the parameters of the attention mechanism model are initialized. These parameters include: an initial modality transformation weight matrix (used to transform modal data weights); an initial attention weight vector (used to determine the relative importance of each modality); an initial dimensionality reduction matrix (used to reduce high-dimensional modal data to a manageable low-dimensional space); and an initial bias vector (used to adjust the baseline bias of the model output). Initialization of the modality transformation weight matrix, attention weight vector, and feature concatenation weight matrix can be performed using Xavier initialization or Gaussian distribution initialization. The bias vector can be initialized to zero or a small constant value.

[0104] In this embodiment, training is performed based on all historical modal data. This unit takes all modal vectors in the historical modal data as input and outputs historical modal weights and historical fusion vectors to train the attention mechanism model. The goal of training is to enable the model to learn how to determine effective weights and fusion vectors based on the modal data.

[0105] In this embodiment, the trained attention mechanism model is used to perform forward propagation analysis on the modal data processed each time to predict the weight of each modality and the fused feature vector.

[0106] In this embodiment, the modal weights predicted in each process are compared with the historical modal weights, and the comparison results are used to adjust the model parameters.

[0107] In this embodiment, based on the first comparison result of the first comparison unit, back propagation is performed to update the parameters of the training model. The adjusted parameters include adjusting the transformation weight matrix, adjusting the attention weight vector, adjusting the dimensionality reduction weight matrix, and adjusting the bias vector to optimize model performance.

[0108] In this embodiment, the historical fusion vector and the predicted fusion vector of each processing are compared to evaluate the accuracy of the fusion result.

[0109] In this embodiment, based on the results of the second comparison unit, the performance of the attention mechanism model is comprehensively evaluated to determine whether the model parameters need to be further adjusted to improve the prediction accuracy.

[0110] The beneficial effects of the above technical solution are: obtaining historical modal data of all modes of CNC machine tools and constructing an attention mechanism model, which can be used to determine the fusion feature vector of CNC machine tools based on real-time modal data, to determine the coordinated control strategy, and to achieve efficient, precise and intelligent adjustment of CNC machine tools.

[0111] Example 4:

[0112] The embodiment of the present invention provides a collaborative control system for CNC machine tools based on multimodal fusion, and an analysis module, including:

[0113] Modal eigenvector unit: preprocesses the sub-real-time modal data of each mode in the real-time modal data, extracts features from the preprocessed sub-real-time modal data of each mode, and determines the modal eigenvector of each mode;

[0114] ;

[0115] in, represents the modal eigenvector of the i-th mode at the current time t, They represent the 1st feature, jth feature, and N1th feature of the i-th mode at the current time t, respectively, where N1 represents the number of features in the modal feature vector;

[0116] Modal characteristic matrix unit: determines the modal characteristic matrix based on the modal eigenvectors of all modes;

[0117] ;

[0118] in, represents the modal characteristic matrix at the current time t, They represent the modal eigenvectors of the i-th mode at the current time t, and N2 represents the number of modes.

[0119] In this embodiment, the sub-real-time modal data of each modality in the real-time modal data is preprocessed. Preprocessing may include steps such as denoising, smoothing, and normalization to better extract features. The preprocessed data is used for feature extraction, which aims to extract key feature information from the data of each modality and form modal feature vectors. These feature vectors reflect the operating status of each modality.

[0120] In this embodiment, a comprehensive modal feature matrix is ​​constructed based on the modal feature vectors of all modes. The modal feature matrix aggregates the feature vectors of multiple modes and presents the overall working status of the machine tool at the current moment.

[0121] The beneficial effects of the above technical solution are: determining the modal characteristic matrix of all modes based on real-time modal data can provide a comprehensive, accurate, and high-quality data basis for determining the fusion characteristic vector of the CNC machine tool, improve the utilization efficiency of real-time modal data, and enhance the adaptability of the control system under different working conditions.

[0122] Example 5:

[0123] The embodiment of the present invention provides a collaborative control system for CNC machine tools based on multimodal fusion, the analysis module further comprising:

[0124] Fusion feature vector unit: Determines the fusion feature vector of the CNC machine tool based on the modal feature vectors of all modalities, the modal feature matrix, and the adjustment parameters of the attention mechanism model;

[0125] ;

[0126] ;

[0127] ;

[0128] ;

[0129] in, represents the fused feature vector at the current time t, represents the weighted feature vector at the current time t, represents the transformation feature vector at the current time t, B represents the adjustment bias vector, Indicates the concatenation of the weighted feature vector and the transformed feature vector at the current time t. represents the relative weight of the i-th mode at the current time t, Represents the adjusted attention weight vector of the i-th modality in the adjustment parameters of the attention mechanism model, Represents the adjustment of the attention weight vector The transpose of Represents the adjustment transformation weight matrix in the adjustment parameters of the attention mechanism model, represents the activation value of the i-th mode at the current time t, represents the activation parameter, Represents the adjusted dimensionality reduction weight matrix in the adjustment parameters of the attention mechanism model.

[0130] In this embodiment, the attention weight vector is adjusted The dimension is N1, which represents the relative importance of all features in the corresponding modality.

[0131] In this embodiment, the transformation weight matrix is ​​adjusted The dimension is N1×N1, which is used to characterize the feature vector of each mode Perform a linear transformation.

[0132] In this embodiment, the activation parameter Set it to a small parameter, maybe 0.01, to avoid the problem of neurons.

[0133] In this embodiment, the dimensionality reduction weight matrix is ​​adjusted The dimension is k×(N1×N2), which maps the modal feature matrix to a new low-dimensional space.

[0134] In this embodiment, the dimension of the modal feature matrix is ​​N1×N2, and the dimension reduction weight matrix is ​​adjusted The dimension is k×(N1×N2), Results A vector of dimension k×1.

[0135] In this embodiment, The dimension is a vector of N1×1.

[0136] In this embodiment, The dimension is N1 plus The dimension of is k, which is an N1+k dimensional vector.

[0137] The beneficial effects of the above technical solution are as follows: determining the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model can enhance the control system's adaptability to complex working conditions, accurately reflect the real-time status of the CNC machine tool, and improve the accuracy and efficiency of the machining process.

[0138] Example 6:

[0139] The embodiment of the present invention provides a collaborative control system for CNC machine tools based on multimodal fusion, the control module including:

[0140] Feature extraction unit: This unit obtains the real-time environment data, real-time processing tasks, and processing task requirements of the CNC machine tool, performs feature extraction on the real-time environment data of the CNC machine tool, and determines the real-time environment vector. At the same time, it performs feature extraction on the real-time processing tasks and processing task requirements of the CNC machine tool and determines the real-time task vector.

[0141] A first selected modal data unit extracts historical environment vectors from all historical modal data, determines that the historical modal data corresponding to the extracted historical environment vectors whose similarity with the real-time environment vector is greater than a first pre-threshold value are first selected modal data;

[0142] The second modal data selection unit extracts historical task vectors from all historical modal data, determines that the historical modal data corresponding to the extracted historical task vectors whose similarity with the real-time task vector is greater than a second pre-threshold value are the second selected modal data;

[0143] Selecting a modal matrix unit: determining an optimal modal matrix of each historical modal data belonging to both the first modal data and the second modal data as a selected modal matrix;

[0144] Collaborative control strategy unit: Determines the collaborative control strategy based on the modal feature matrix, fused feature vectors and all selected modal matrices.

[0145] In this embodiment, feature extraction is performed by acquiring the real-time environmental data (such as temperature, humidity, pressure, etc.) and real-time processing task data (such as processing speed, processing accuracy requirements, etc.) of the machine tool to generate the following: real-time environmental vector: containing the status data of the environment in which the machine tool is located, which is used to describe the environmental factors that may affect the performance of the machine tool during the processing process; real-time task vector: describes the requirements of the current processing task, such as processing goals, accuracy requirements, workpiece type, etc.

[0146] In this embodiment, a historical environment vector is extracted from the historical modal data and compared with the real-time environment vector for similarity. If the similarity is greater than a first preset threshold, the historical modal data is considered similar to the current environment, and the corresponding historical modal data is selected as the first selected modal data.

[0147] In this embodiment, a historical task vector is extracted from the historical modal data and compared with the real-time task vector for similarity. If the similarity is greater than a second preset threshold, the corresponding historical modal data is selected as the second selected modal data.

[0148] In this embodiment, the similarity between the extracted historical task vector and the real-time task vector can be calculated by cosine similarity, Euclidean distance or Manhattan distance; the similarity between the extracted historical task vector and the real-time task vector can be calculated by cosine similarity, Euclidean distance or Manhattan distance.

[0149] In this embodiment, according to the intersection of the first selected modal data and the second selected modal data, the optimal modal matrix of each historical modal data is determined as the selected modal matrix.

[0150] The beneficial effects of the above technical solution are: determining the collaborative control strategy of CNC machine tools based on historical modal data, modal feature matrix and fusion feature vector, which can provide precise and intelligent coordinated control solutions and improve the adaptability and processing accuracy of CNC machine tools under complex working conditions.

[0151] Example 7:

[0152] An embodiment of the present invention provides a collaborative control system for CNC machine tools based on multimodal fusion, and a collaborative control strategy unit, comprising:

[0153] ;

[0154] ;

[0155] in, represents the error matrix at the current time t,

[0156] Nu represents the number of selected modal matrices, CM represents the fitting change matrix, represents the ath selected modal matrix, represents the best fit matrix, represents the nuclear norm of the fitted variation matrix, represents the regularization parameter, tE represents the predicted end time of the real-time processing task, represents the fusion control vector.

[0157] In this embodiment, the fused control vector is used to adjust the influence of the fused feature vector on the collaborative control strategy.

[0158] In this embodiment, is the calculated optimal fitting matrix, which represents the fitting change matrix that minimizes the objective function.

[0159] In this embodiment, represents the Frobenius norm.

[0160] The beneficial effects of the above technical solution are as follows: based on the modal feature matrix, the fused feature vector and all the selected modal matrices, the collaborative control strategy is determined, which can improve the accuracy of the collaborative control strategy and improve the adaptability and processing accuracy of CNC machine tools under complex working conditions.

[0161] Example 8:

[0162] The embodiment of the present invention provides a collaborative control method for CNC machine tools based on multimodal fusion, such as Figure 2 Shown, including:

[0163] Step 1: Multiple modes of the CNC machine tool, real-time state information of all modes of the CNC machine tool is collected to determine real-time modal data;

[0164] Step 2: Obtain historical modal data of all modes of CNC machine tools and build an attention mechanism model;

[0165] Step 3: Determine the modal feature matrix of all modes based on the real-time modal data, and determine the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model;

[0166] Step 4: Determine the collaborative control strategy of the CNC machine tool based on the historical modal data, modal feature matrix, and fusion feature vector;

[0167] Step 5: Execute the coordinated control strategy to realize the coordinated control of CNC machine tools based on multimodal fusion.

[0168] The beneficial effects of the above technical solution include: determining the modal feature matrix by analyzing real-time modal data, determining the fused feature vector based on the modal feature matrix and the constructed attention mechanism model, determining the coordinated control strategy based on historical modal data, the modal feature matrix, and the fused feature vector, and executing the coordinated control strategy to achieve coordinated control of CNC machine tools based on multimodal fusion. This can form an efficient coordinated control mechanism, improve the intelligence and precision of machine tool control, enhance machining accuracy and production efficiency, and improve the adaptability and operating efficiency of CNC machine tools under complex working conditions.

[0169] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0170] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A collaborative control system for CNC machine tools based on multimodal fusion, characterized in that: include: Acquisition module: acquires multiple modes of the CNC machine tool, collects the status information of all modes of the CNC machine tool in real time to determine the real-time modal data; Construction module: Obtain historical modal data of all modes of CNC machine tools and build an attention mechanism model; Analysis module: Determines the modal feature matrix of all modes based on real-time modal data, and determines the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model; Control module: Determines the collaborative control strategy of the CNC machine tool based on historical modal data, modal feature matrix, and fused feature vector; Execution module: executes the coordinated control strategy to realize the coordinated control of CNC machine tools based on multi-modal fusion; The acquisition module includes: The first acquisition unit is used to acquire the machine type, structure and parameters of the machine tool, and to acquire the characteristics and processing technology of the part to be processed; Mode determination unit: determines multiple modes of the CNC machine tool based on the machine tool type, machine tool structure, machine tool parameters, part characteristics of the part to be processed, and processing technology; Modal sensor group unit: determines a modal sensor group for each mode based on the structure of the CNC machine tool, wherein the modal sensor group includes at least one modal sensor; Sub-real-time modal data unit: collects state information of each modality based on the modal sensor group of each modality and determines the sub-real-time modal data of each modality; Real-time modal data unit: determines the real-time modal data of the CNC machine tool based on the sub-real-time modal data of all modes; The analysis module includes: Modal eigenvector unit: preprocesses the sub-real-time modal data of each mode in the real-time modal data, extracts features from the preprocessed sub-real-time modal data of each mode, and determines the modal eigenvector of each mode; ; in, represents the modal eigenvector of the i-th mode at the current time t, They represent the 1st feature, jth feature, and N1th feature of the i-th mode at the current time t, respectively, where N1 represents the number of features in the modal feature vector; Modal characteristic matrix unit: determines the modal characteristic matrix based on the modal eigenvectors of all modes; ； in, represents the modal characteristic matrix at the current time t, They represent the modal eigenvectors of the i-th mode at the current time t, and N2 represents the number of modes; Fusion feature vector unit: Determines the fusion feature vector of the CNC machine tool based on the modal feature vectors of all modalities, the modal feature matrix, and the adjustment parameters of the attention mechanism model; ； ； ； ； in, represents the fused feature vector at the current time t, represents the weighted feature vector at the current time t, represents the transformation feature vector at the current time t, B represents the adjustment bias vector, Indicates the concatenation of the weighted feature vector and the transformed feature vector at the current time t. represents the relative weight of the i-th mode at the current time t, Represents the adjusted attention weight vector of the i-th modality in the adjustment parameters of the attention mechanism model, Represents the adjustment of the attention weight vector The transpose of Represents the adjustment transformation weight matrix in the adjustment parameters of the attention mechanism model, represents the activation value of the i-th mode at the current time t, represents the activation parameter, Represents the adjusted dimensionality reduction weight matrix in the adjustment parameters of the attention mechanism model.

2. The CNC machine tool collaborative control system based on multimodal fusion according to claim 1 is characterized in that: Building blocks, including: Historical modal data unit: obtains historical modal data of all modes processed by the CNC machine tool multiple times, where the historical modal data includes historical environment vectors, historical task vectors, historical modal vectors of each mode, historical modal weights, historical fusion vectors, and optimal modal matrix; Initialization unit: Initializes the model parameters of the attention mechanism model, where the model parameters include at least the initial modal transformation weight matrix, the initial attention weight vector, the initial dimensionality reduction weight matrix, and the initial bias vector; Model training unit: Trains the initialized attention mechanism model based on all processed historical modal data, uses the historical modal vectors of all modalities in all processed historical modal data as the model input of the attention mechanism model, and uses the historical modal weights and historical fusion vectors of all modalities in all processed historical modal data as the model output of the attention mechanism model; Prediction unit: The attention mechanism model performs forward propagation analysis on the model input to determine the predicted modal weight and predicted fusion vector for each modality processed each time; A first comparison unit: performing a first comparison on the historical modal weight and the predicted modal weight of each modality in the historical modal data processed each time; Adjustment unit: The attention mechanism model performs backpropagation analysis based on the first comparison result, adjusts the initialization model parameters of the attention mechanism model based on the analysis result, and determines the adjustment parameters, wherein the adjustment parameters at least include adjusting the transformation weight matrix, adjusting the attention weight vector, adjusting the dimensionality reduction weight matrix, and adjusting the bias vector; A second comparison unit: performing a second comparison on the historical fusion vector and the predicted fusion vector in the historical modal data processed each time; Model evaluation unit: evaluates the attention mechanism model based on the second comparison result.

3. The CNC machine tool collaborative control system based on multimodal fusion according to claim 1, characterized in that: Control module, including: Feature extraction unit: This unit obtains the real-time environment data, real-time processing tasks, and processing task requirements of the CNC machine tool, performs feature extraction on the real-time environment data of the CNC machine tool, and determines the real-time environment vector. At the same time, it performs feature extraction on the real-time processing tasks and processing task requirements of the CNC machine tool and determines the real-time task vector. A first selected modal data unit extracts historical environment vectors from all historical modal data, determines that the historical modal data corresponding to the extracted historical environment vectors whose similarity with the real-time environment vector is greater than a first pre-threshold value are first selected modal data; The second modal data selection unit extracts historical task vectors from all historical modal data, determines that the historical modal data corresponding to the extracted historical task vectors whose similarity with the real-time task vector is greater than a second pre-threshold value are the second selected modal data; Selecting a modal matrix unit: determining an optimal modal matrix of each historical modal data belonging to both the first modal data and the second modal data as a selected modal matrix; Collaborative control strategy unit: Determines the collaborative control strategy based on the modal feature matrix, fused feature vectors and all selected modal matrices.

4. The CNC machine tool collaborative control system based on multimodal fusion according to claim 3 is characterized in that: Collaborative control strategy unit, including: ； ； in, represents the error matrix at the current time t, Nu represents the number of selected modal matrices, CM represents the fitting change matrix, represents the ath selected modal matrix, represents the best fit matrix, represents the nuclear norm of the fitted variation matrix, represents the regularization parameter, tE represents the predicted end time of the real-time processing task, represents the fusion control vector.

5. A collaborative control method for CNC machine tools based on multimodal fusion, characterized in that: A collaborative control system for CNC machine tools based on multimodal fusion for executing any one of claims 1 to 4, comprising: Step 1: Multiple modes of the CNC machine tool, real-time state information of all modes of the CNC machine tool is collected to determine real-time modal data; Step 2: Obtain historical modal data of all modes of CNC machine tools and build an attention mechanism model; Step 3: Determine the modal feature matrix of all modes based on the real-time modal data, and determine the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model; Step 4: Determine the collaborative control strategy of the CNC machine tool based on the historical modal data, modal feature matrix, and fusion feature vector; Step 5: Execute the coordinated control strategy to realize the coordinated control of CNC machine tools based on multimodal fusion; Among them, the multiple modes of the CNC machine tool, the state information of all modes of the CNC machine tool is collected in real time to determine the real-time modal data, including: The first acquisition unit is used to acquire the machine type, structure and parameters of the machine tool, and to acquire the characteristics and processing technology of the part to be processed; Mode determination unit: determines multiple modes of the CNC machine tool based on the machine tool type, machine tool structure, machine tool parameters, part characteristics of the part to be processed, and processing technology; Modal sensor group unit: determines a modal sensor group for each mode based on the structure of the CNC machine tool, wherein the modal sensor group includes at least one modal sensor; Sub-real-time modal data unit: collects state information of each modality based on the modal sensor group of each modality and determines the sub-real-time modal data of each modality; Real-time modal data unit: determines the real-time modal data of the CNC machine tool based on the sub-real-time modal data of all modes; Among them, the modal feature matrix of all modes is determined based on real-time modal data, and the fusion feature vector of the CNC machine tool is determined based on the modal feature matrix of all modes and the attention mechanism model, including: Modal eigenvector unit: preprocesses the sub-real-time modal data of each mode in the real-time modal data, extracts features from the preprocessed sub-real-time modal data of each mode, and determines the modal eigenvector of each mode; ; in, represents the modal eigenvector of the i-th mode at the current time t, They represent the 1st feature, jth feature, and N1th feature of the i-th mode at the current time t, respectively, where N1 represents the number of features in the modal feature vector; Modal characteristic matrix unit: determines the modal characteristic matrix based on the modal eigenvectors of all modes; ； in, represents the modal characteristic matrix at the current time t, They represent the modal eigenvectors of the i-th mode at the current time t, and N2 represents the number of modes; Fusion feature vector unit: Determines the fusion feature vector of the CNC machine tool based on the modal feature vectors of all modalities, the modal feature matrix, and the adjustment parameters of the attention mechanism model; ； ； ； ； in, represents the fused feature vector at the current time t, represents the weighted feature vector at the current time t, represents the transformation feature vector at the current time t, B represents the adjustment bias vector, Indicates the concatenation of the weighted feature vector and the transformed feature vector at the current time t. represents the relative weight of the i-th mode at the current time t, Represents the adjusted attention weight vector of the i-th modality in the adjustment parameters of the attention mechanism model, Represents the adjustment of the attention weight vector The transpose of Represents the adjustment transformation weight matrix in the adjustment parameters of the attention mechanism model, represents the activation value of the i-th mode at the current time t, represents the activation parameter, Represents the adjusted dimensionality reduction weight matrix in the adjustment parameters of the attention mechanism model.

Images