Numerically-controlled machine tool cooperative control system and method based on multi-modal fusion

Abstract

The invention provides a numerical control machine tool cooperative control system and method based on multi-modal fusion, and belongs to the technical field of machine tool control, and the system comprises a collection module which obtains a plurality of modals of a numerical control machine tool, collects the state information of all modals of the numerical control machine tool in real time, and determines the real-time modal data; the construction module is used for acquiring historical modal data of all modals of the numerical control machine tool and constructing an attention mechanism model; the analysis module is used for determining a modal feature matrix of all modals and determining a fusion feature vector of the numerical control machine tool; the control module is used for determining a cooperative control strategy of the numerical control machine tool based on the historical modal data, the modal feature matrix and the fusion feature vector; and the execution module is used for executing the coordination control strategy and realizing coordination control of the numerical control machine tool based on multi-modal fusion. An efficient cooperative control mechanism can be formed, the intelligence and precision of machine tool control are improved, the machining precision and the production efficiency are improved, and the adaptability and the operation efficiency of the numerical control machine tool under complex working conditions are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine tool control, and particularly to a collaborative control system and method for a numerically controlled machine tool based on multimodal fusion. Background Art

[0002] Numerical control machine tools (CNCs) play a crucial role in modern manufacturing. With the rise of industrial automation and intelligent manufacturing, traditional numerical control machine tools have gradually shifted towards multimodal control to improve machining accuracy, efficiency, and the ability to adapt to complex working environments. Early numerical control systems mainly relied on simple sensor data (such as position, speed, etc.) for control. These systems could only process single-modal data and lacked flexibility and adaptability when faced with complex machining tasks.

[0003] With the progress of technology, multimodal data fusion technology has emerged. This technology can simultaneously process multiple data types from multiple sensors (such as temperature, pressure, vibration, etc.), thereby providing more comprehensive and accurate machine tool status information. However, there are still technical problems such as low data fusion efficiency and insufficient real-time performance.

[0004] Therefore, the present invention provides a collaborative control system and method for a numerically controlled machine tool based on multimodal fusion. Summary of the Invention

[0005] The present invention provides a collaborative control system and method for a numerically controlled machine tool based on multimodal fusion. By analyzing and determining the real-time modal data, a modal feature matrix is determined. According to the modal feature matrix and the constructed attention mechanism model, a fusion feature vector is determined. According to the historical modal data, the modal feature matrix, and the fusion feature vector, a collaborative control strategy is determined, and the coordinated control strategy is executed to achieve the coordinated control of the numerically controlled machine tool based on multimodal fusion. An efficient collaborative control mechanism can be formed, improving the intelligence and accuracy of machine tool control, enhancing machining accuracy and production efficiency, and improving the adaptability and operating efficiency of numerically controlled machine tools under complex working conditions.

[0006] On the one hand, the present invention provides a collaborative control system for a numerically controlled machine tool based on multimodal fusion, including: An acquisition module: acquiring multiple modes of the numerically controlled machine tool and collecting the status information of all modes of the numerically controlled machine tool in real time to determine real-time modal data; A construction module: acquiring the historical modal data of all modes of the numerically controlled machine tool and constructing an attention mechanism model; An analysis module: determining the modal feature matrix of all modes based on the real-time modal data, and determining the fusion feature vector of the numerically controlled machine tool based on the modal feature matrix of all modes and the attention mechanism model; Control module: Determine the collaborative control strategy of the CNC machine tool based on historical modal data, modal feature matrix, and fusion feature vector; Execution module: Execute the coordinated control strategy to achieve the coordinated control of the CNC machine tool based on multi-modal fusion.

[0007] According to a collaborative control system for a CNC machine tool based on multi-modal fusion provided by the present invention, the acquisition module includes: First acquisition unit: Acquire the machine tool type, machine tool structure, and machine tool parameters of the machine tool, and acquire the part characteristics and processing technology of the part to be processed; Modal determination unit: Determine multiple modes of the CNC machine tool based on the machine tool type, machine tool structure, machine tool parameters, part characteristics, and processing technology of the machine tool; Modal sensor group unit: Determine the modal sensor group for each mode based on the CNC machine tool structure, where the modal sensor group includes at least one or more modal sensors; Sub-real-time modal data unit: Acquire the state information of each mode based on the modal sensor group of each mode, and determine the sub-real-time modal data of each mode; Real-time modal data unit: Determine the real-time modal data of the CNC machine tool based on the sub-real-time modal data of all modes.

[0008] According to a collaborative control system for a CNC machine tool based on multi-modal fusion provided by the present invention, the construction module includes: Historical modal data unit: Acquire the historical modal data of all modes for multiple machining operations of the CNC machine tool, where the historical modal data includes historical environment vectors, historical task vectors, historical modal vectors of each mode, historical modal weights, historical fusion vectors, and the best modal matrix; Initialization unit: Initialize the model parameters of the attention mechanism model, where the model parameters at least include the initial modal transformation weight matrix, initial attention weight vector, initial dimensionality reduction weight matrix, and initial bias vector; Model training unit: Train the initialized attention mechanism model based on the historical modal data of all machining operations, use the historical modal vectors of all modes in the modal data of all historical machining operations as the model input of the attention mechanism model, and use the historical modal weights and historical fusion vectors of all modes in the modal data of all machining operations as the model output of the attention mechanism model; Prediction unit: The attention mechanism model performs forward propagation analysis on the model input to determine the predicted modal weights and predicted fusion vectors of each mode for each machining operation; First comparison unit: Perform a first comparison on the historical modal weights and predicted modal weights of each mode in the historical modal data of each machining operation; Adjustment unit: The attention mechanism model performs backpropagation analysis based on the first comparison result, and adjusts the initial model parameters of the attention mechanism model based on the analysis result to determine adjustment parameters, where the adjustment parameters at least include an adjusted transformation weight matrix, an adjusted attention weight vector, an adjusted dimensionality reduction weight matrix, and an adjusted bias vector; Second comparison unit: performs a second comparison on the historical fusion vector and the predicted fusion vector in the historical modal data processed each time; Model evaluation unit: evaluates the attention mechanism model based on the second comparison result.

[0009] According to a multi-modal fusion-based numerical control machine tool collaborative control system provided by the present invention, an analysis module includes: Modal feature vector unit: preprocesses the sub-real-time modal data of each modality in the real-time modal data, extracts features from the preprocessed sub-real-time modal data of each modality, and determines the modal feature vector of each modality; ; Wherein, represents the modal feature vector of the i-th modality at the current time t, respectively represent the first feature, the j-th feature, and the N1-th feature of the i-th modality at the current time t, and N1 represents the number of features in the modal feature vector; Modal feature matrix unit: determines a modal feature matrix based on the modal feature vectors of all modalities; ; Wherein, represents the modal feature matrix at the current time t, respectively represent the modal feature vectors of the i-th modality at the current time t, and N2 represents the number of modalities.

[0010] According to a multi-modal fusion-based numerical control machine tool collaborative control system provided by the present invention, the analysis module further includes: Fusion feature vector unit: determines the fusion feature vector of the numerical control machine tool based on the modal feature vectors of all modalities, the modal feature matrix, and the adjustment parameters of the attention mechanism model; ; ; ; ; Wherein, represents the fusion feature vector at the current time t, represents the weighted feature vector at the current time t, denotes the transformed feature vector at the current moment t, and B denotes the adjustment bias vector. denotes concatenating the weighted feature vector and the transformed feature vector at the current moment t. denotes the relative weight of the i-th modality at the current moment t. denotes the adjusted attention weight vector of the i-th modality in the adjustment parameters of the attention mechanism model. denotes the adjusted attention weight vector transpose. denotes the adjusted transformation weight matrix in the adjustment parameters of the attention mechanism model. denotes the activation value of the i-th modality at the current moment t. denotes the activation parameter. denotes the adjusted dimensionality reduction weight matrix in the adjustment parameters of the attention mechanism model.

[0011] According to a collaborative control system for numerically controlled machine tools based on multi-modal fusion provided by the present invention, the control module includes: Feature extraction unit: Obtain the real-time environmental data, real-time processing tasks, and processing task requirements of the numerically controlled machine tool, extract features from the real-time environmental data of the numerically controlled machine tool to determine the real-time environmental vector, and at the same time, extract features from the real-time processing tasks and processing task requirements of the numerically controlled machine tool to determine the real-time task vector; First selected modality data unit: Extract the historical environmental vectors from all historical modality data, and determine that the historical modality data corresponding to all historical environmental vectors with a similarity greater than the first pre-threshold to the real-time environmental vector is the first selected modality data; Second selected modality data unit: Extract the historical task vectors from all historical modality data, and determine that the historical modality data corresponding to all historical task vectors with a similarity greater than the second pre-threshold to the real-time task vector is the second selected modality data; Selected modality matrix unit: Determine the best modality matrix of each historical modality data that belongs to both the first modality data and the second modality data as the selected modality matrix; Collaborative control strategy unit: Determine the collaborative control strategy based on the modality feature matrix, the fused feature vector, and all the selected modality matrices.

[0012] According to a collaborative control system for numerically controlled machine tools based on multi-modal fusion provided by the present invention, the collaborative control strategy unit includes: ; ; Among them, denotes the error matrix at the current moment t. $N_u$ represents the number of selected modal matrices, and $CM$ represents the fitting change matrix. represents the $a$-th selected modal matrix. represents the optimal fitting matrix. represents the nuclear norm of the fitting change matrix. represents the regularization parameter, and $t_E$ represents the predicted end time of the real-time machining task. represents the fusion control vector.

[0013] On the other hand, the present invention also provides a collaborative control method for a numerically controlled machine tool based on multi-modal fusion, including: Step 1: For multiple modes of the numerically controlled machine tool, collect the state information of all modes of the numerically controlled machine tool in real time to determine real-time modal data. Step 2: Obtain the historical modal data of all modes of the numerically controlled machine tool and construct an attention mechanism model. Step 3: Determine the modal feature matrix of all modes based on the real-time modal data, and determine the fusion feature vector of the numerically controlled machine tool based on the modal feature matrix of all modes and the attention mechanism model. Step 4: Determine the collaborative control strategy of the numerically controlled machine tool based on the historical modal data, the modal feature matrix, and the fusion feature vector. Step 5: Execute the coordinated control strategy to achieve the coordinated control of the numerically controlled machine tool based on multi-modal fusion.

[0014] Compared with the prior art, the beneficial effects of the present application are as follows: Determine the modal feature matrix by analyzing the determined real-time modal data, determine the fusion feature vector according to the modal feature matrix and the constructed attention mechanism model, determine the collaborative control strategy according to the historical modal data, the modal feature matrix, and the fusion feature vector, and execute the coordinated control strategy to achieve the coordinated control of the numerically controlled machine tool based on multi-modal fusion. An efficient collaborative control mechanism can be formed, improving the intelligence and accuracy of machine tool control, enhancing the machining accuracy and production efficiency, and improving the adaptability and operating efficiency of the numerically controlled machine tool under complex working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 is a schematic structural diagram of a collaborative control system for a numerically controlled machine tool based on multi-modal fusion provided by an embodiment of the present invention.

[0017] Figure 2 It is a schematic flow chart of a collaborative control method for numerically controlled machine tools based on multi-modal fusion provided by an embodiment of the present invention. Specific implementation manners

[0018] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] Embodiment 1: An embodiment of the present invention provides a collaborative control system for numerically controlled machine tools based on multi-modal fusion, as Figure 1 shown, including: Acquisition module: acquiring multiple modalities of the numerically controlled machine tool, and determining real-time modal data by collecting the state information of all modalities of the numerically controlled machine tool in real time; Construction module: acquiring the historical modal data of all modalities of the numerically controlled machine tool, and constructing an attention mechanism model; Analysis module: determining the modal feature matrix of all modalities based on the real-time modal data, and determining the fusion feature vector of the numerically controlled machine tool based on the modal feature matrix of all modalities and the attention mechanism model; Control module: determining the collaborative control strategy of the numerically controlled machine tool based on the historical modal data, the modal feature matrix and the fusion feature vector; Execution module: executing the coordinated control strategy to achieve the coordinated control of the numerically controlled machine tool based on multi-modal fusion.

[0020] In this embodiment, multiple modalities of the numerically controlled machine tool (such as vibration, temperature, pressure, etc.) are determined, and the state information of these modalities is collected. The real-time collected data is used to form real-time modal data, which can reflect the current operating state of the machine tool.

[0021] In this embodiment, an attention mechanism model is constructed and trained by using the historical modal data. The attention mechanism model adjusts the weights so that the system can perform weighted processing on different modal states, enhancing the influence of important modalities. This helps to more accurately understand the role of historical data and provides a high-quality data basis for determining the collaborative control strategy.

[0022] In this embodiment, based on the real-time modal data, the features of all modalities are extracted to generate a modal feature matrix. Combining with the attention mechanism model, a fusion feature vector is generated.

[0023] In this embodiment, the control module determines the most suitable cooperative control strategy through the analysis of historical modal data, modal feature matrices, and fusion feature vectors. These strategies will regulate the behavior of the machine tool to ensure that it always exhibits optimal performance under different operating conditions.

[0024] In this embodiment, according to the control strategy, corresponding control operations are executed to ensure that the operation of the machine tool conforms to the optimized cooperative control scheme.

[0025] Beneficial effects of the above technical solution: The modal feature matrix is determined through the analysis of the determined real-time modal data. The fusion feature vector is determined based on the modal feature matrix and the constructed attention mechanism model. The cooperative control strategy is determined according to the historical modal data, modal feature matrix, and fusion feature vector, and the coordinated control strategy is executed to achieve the coordinated control of the CNC machine tool based on multi-modal fusion. An efficient cooperative control mechanism can be formed, improving the intelligence and accuracy of machine tool control, enhancing the machining accuracy and production efficiency, and improving the adaptability and operating efficiency of the CNC machine tool under complex working conditions.

[0026] Embodiment 2: The embodiment of the present invention provides a cooperative control system for a CNC machine tool based on multi-modal fusion. The acquisition module includes: The first acquisition unit: acquires the machine tool type, machine tool structure, and machine tool parameters of the machine tool, and acquires the part characteristics and processing technology of the part to be machined; The modal determination unit: determines multiple modes of the CNC machine tool based on the machine tool type, machine tool structure, machine tool parameters, part characteristics of the part to be machined, and processing technology; The modal sensor group unit: determines the modal sensor group for each mode based on the CNC machine tool structure, where the modal sensor group includes at least one or more modal sensors; The sub-real-time modal data unit: acquires the state information of each mode based on the modal sensor group of each mode, and determines the sub-real-time modal data of each mode; The real-time modal data unit: determines the real-time modal data of the CNC machine tool based on the sub-real-time modal data of all modes.

[0027] In this embodiment, the basic information of the CNC machine tool (such as type, structure, parameters) and the characteristics of the part to be machined (such as material, shape, etc.) and processing technology (such as cutting method, feed rate, etc.) are acquired.

[0028] In this embodiment, multiple modes of the CNC machine tool are determined based on the machine tool type, structure, parameters, as well as the workpiece characteristics and processing technology.

[0029] In this embodiment, the multiple modalities of the numerically controlled machine tool can be respectively the vibration modality: the machine tool generates vibration during operation, and the magnitude and frequency of the vibration reflect the operating stability of the machine tool and the wear condition of the components. Excessive vibration will affect the machining accuracy and surface quality, and may even lead to tool damage and equipment failure; the sound modality: the sound emitted during machining of the machine tool contains rich information, such as the cutting state of the tool and the workpiece, the operating condition of the components, etc. Abnormal sounds may indicate problems such as tool wear, component loosening or poor lubrication; the temperature modality: key components of the machine tool, such as the spindle, motor, bearings, etc., generate heat during operation. Excessive temperature will cause thermal deformation of the components, affect the machining accuracy, and at the same time accelerate the wear and aging of the components; the force and torque modality: the cutting force and spindle torque reflect the cutting load and machining ability of the machine tool. By monitoring the force and torque modalities, the cutting parameters can be optimized, overload operation can be avoided, and the machining efficiency and tool life can be improved; the current and voltage modality: the changes in the current and voltage of the motor reflect the operating state and load condition of the motor. Abnormal current and voltage values may indicate motor failure or electrical system problems; the image modality: using an industrial camera to collect images of the machining area, the wear condition of the tool, the machining state of the workpiece and the clamping condition, etc. can be visually observed, providing a basis for timely adjusting the machining parameters and ensuring the machining quality; the position and speed modality: monitoring the position and speed information of each moving component of the machine tool to ensure that the machine tool operates according to the predetermined trajectory and speed, guaranteeing the machining accuracy and production efficiency.

[0030] In this embodiment, according to the structure of the numerically controlled machine tool and the characteristics of different modalities, the sensor group required for each modality is determined. Each modality may require multiple sensors to accurately collect data, and the sensors can monitor different parameters such as vibration, temperature, and force.

[0031] In this embodiment, the state information (such as vibration, temperature, force, etc.) of each modality is collected in real time through the modality sensor group.

[0032] In this embodiment, the sub-real-time data of all modalities are integrated, and finally the real-time modality data of the numerically controlled machine tool are determined. This data is a real-time reflection of the current state of the machine tool.

[0033] The beneficial effects of the above technical solution: By obtaining multiple modalities of the numerically controlled machine tool and collecting the state information of all modalities of the numerically controlled machine tool in real time to determine the real-time modality data, it can provide a data basis for determining the modality feature matrix.

[0034] Embodiment 3: The embodiment of the present invention provides a collaborative control system for a numerically controlled machine tool based on multi-modal fusion, and a construction module, including: Historical Modal Number Unit: Obtain the historical modal data of all modes for multiple machining operations of a numerically controlled machine tool. The historical modal data includes historical environmental vectors, historical task vectors, historical modal vectors for each mode, historical modal weights, historical fusion vectors, and the optimal modal matrix. Initialization Unit: Initialize the model parameters of the attention mechanism model. The model parameters at least include the initial modal transformation weight matrix, the initial attention weight vector, the initial dimensionality reduction weight matrix, and the initial bias vector. Model Training Unit: Train the initialized attention mechanism model based on the historical modal data of all machining operations. Use the historical modal vectors of all modes in the modal data of all historical machining operations as the model input of the attention mechanism model, and use the historical modal weights and historical fusion vectors of all modes in the historical modal data of all machining operations as the model output of the attention mechanism model. Prediction Unit: The attention mechanism model performs forward propagation analysis on the model input to determine the predicted modal weights and predicted fusion vectors for each mode of each machining operation. First Comparison Unit: Perform a first comparison on the historical modal weights and predicted modal weights of each mode in the historical modal data of each machining operation. Adjustment Unit: The attention mechanism model performs backward propagation analysis based on the first comparison result, and adjusts the initialized model parameters of the attention mechanism model based on the analysis result to determine the adjustment parameters. The adjustment parameters at least include the adjusted transformation weight matrix, the adjusted attention weight vector, the adjusted dimensionality reduction weight matrix, and the adjusted bias vector. Second Comparison Unit: Perform a second comparison on the historical fusion vectors and predicted fusion vectors in the historical modal data of each machining operation. Model Evaluation Unit: Evaluate the attention mechanism model based on the second comparison result.

[0035] In this embodiment, all historical modal data of multiple machining operations of the numerically controlled machine tool are collected, and each machining operation corresponds to a historical modal data.

[0036] In this embodiment, the historical modal data includes multiple aspects: Historical environmental vector: Represents factors in the machining environment, such as temperature, humidity, air pressure, etc.; Historical task vector: Describes the parameters of each machining task, such as machining time, tool type, etc.; Historical modal vector: The performance data of each mode (such as vibration, heat, etc.) in historical machining; Historical modal weight: The relative importance of each mode to machining accuracy; Historical fusion vector: The fusion result of multi-modal data, representing the overall machining state; Optimal modal matrix: The modal data matrix that can represent the best machining state in all historical data.

[0037] In this embodiment, the parameters of the attention mechanism model are initialized. The parameters include: an initial modal transformation weight matrix: the weight for transforming modal data; an initial attention weight vector: determining the relative importance of each modality; an initial dimensionality reduction matrix: reducing high-dimensional modal data to a manageable low-dimensional space; an initial bias vector: adjusting the baseline deviation of the model output. The initialization of the modal transformation weight matrix, the attention weight vector, and the feature concatenation weight matrix can be initialized by Xavier initialization or Gaussian distribution initialization, and the bias vector can be initialized to zero or a small constant value.

[0038] In this embodiment, training is performed based on all historical modal data. This unit takes all modal vectors in the historical modal data as input, and outputs historical modal weights and historical fusion vectors to train the attention mechanism model. The goal of training is to enable the model to learn how to determine effective weights and fusion vectors based on modal data.

[0039] In this embodiment, the trained attention mechanism model is used to perform forward propagation analysis on the modal data processed each time, predicting the weight of each modality and the fused feature vector.

[0040] In this embodiment, the modal weights predicted in each process are compared with the historical modal weights. The comparison result is used to adjust the model parameters.

[0041] In this embodiment, based on the first comparison result of the first comparison unit, the parameters of the training model are updated by backpropagation. The adjusted parameters include adjusting the transformation weight matrix, adjusting the attention weight vector, adjusting the dimensionality reduction weight matrix, and adjusting the bias vector to optimize the model performance.

[0042] In this embodiment, the historical fusion vector and the predicted fusion vector processed each time are compared to evaluate the accuracy of the fusion result.

[0043] In this embodiment, based on the result of the second comparison unit, the performance of the attention mechanism model is comprehensively evaluated to determine whether it is necessary to further adjust the model parameters to improve the prediction accuracy.

[0044] Beneficial effects of the above technical solution: Obtaining the historical modal data of all modalities of the CNC machine tool and constructing an attention mechanism model can be used to determine the fusion feature vector of the CNC machine tool based on real-time modal data, so as to determine the coordinated control strategy and achieve the efficient, precise, and intelligent adjustment of the CNC machine tool.

[0045] Embodiment 4: The embodiment of the present invention provides a collaborative control system for a CNC machine tool based on multi-modal fusion, and the analysis module includes: Modal Feature Vector Unit: Preprocess the sub-real-time modal data of each modality in the real-time modal data, extract features from the preprocessed sub-real-time modal data of each modality, and determine the modal feature vector of each modality; ; wherein, represents the modal feature vector of the i-th modality at the current time t, respectively represent the first feature, the j-th feature, and the N1-th feature of the i-th modality at the current time t, and N1 represents the number of features in the modal feature vector; Modal Feature Matrix Unit: Determine the modal feature matrix based on the modal feature vectors of all modalities; ; wherein, represents the modal feature matrix at the current time t, respectively represent the modal feature vectors of the i-th modality at the current time t, and N2 represents the number of modalities.

[0046] In this embodiment, the sub-real-time modal data of each modality in the real-time modal data is preprocessed. The preprocessing may include steps such as denoising, smoothing, and normalization to better extract features. The preprocessed data is used for feature extraction, aiming to extract key feature information from the data of each modality to form a modal feature vector. These feature vectors reflect the working state of each modality.

[0047] In this embodiment, based on the modal feature vectors of all modalities, a comprehensive modal feature matrix is constructed. The modal feature matrix aggregates the feature vectors of multiple modalities, presenting the overall working state of the machine tool at the current time.

[0048] Beneficial effects of the above technical solution: Determining the modal feature matrices of all modalities based on real-time modal data can provide a comprehensive, accurate, and high-quality data basis for determining the fusion feature vector of the CNC machine tool, improve the utilization efficiency of real-time modal data, and enhance the adaptability of the control system under different working conditions.

[0049] Embodiment 5: The embodiment of the present invention provides a collaborative control system for a CNC machine tool based on multi-modal fusion. The analysis module further includes: Fusion Feature Vector Unit: Determine the fusion feature vector of the CNC machine tool based on the modal feature vectors of all modalities, the modal feature matrix, and the adjustment parameters of the attention mechanism model; ; ; ; ; Among them, represents the fused feature vector at the current moment t, represents the weighted feature vector at the current moment t, represents the transformed feature vector at the current moment t, and B represents the adjustment bias vector, represents concatenating the weighted feature vector and the transformed feature vector at the current moment t, represents the relative weight of the i-th modality at the current moment t, represents the adjusted attention weight vector of the i-th modality in the adjustment parameters of the attention mechanism model, represents the adjusted attention weight vector transpose of, represents the adjusted transformation weight matrix in the adjustment parameters of the attention mechanism model, represents the activation value of the i-th modality at the current moment t, represents the activation parameter, represents the adjusted dimensionality reduction weight matrix in the adjustment parameters of the attention mechanism model.

[0050] In this embodiment, the adjusted attention weight vector has a dimension of N1 and represents the relative importance of all features in the corresponding modality.

[0051] In this embodiment, the adjusted transformation weight matrix has a dimension of N1×N1 and is used to linearly transform the feature vector of each modality.

[0052] In this embodiment, the activation parameter is set to a small parameter, which can be 0.01, to avoid neuron problems.

[0053] In this embodiment, the adjusted dimensionality reduction weight matrix has a dimension of k×(N1×N2) and maps the modality feature matrix to a new low-dimensional space.

[0054] In this embodiment, the dimension of the modality feature matrix is N1×N2, and the adjusted dimensionality reduction weight matrix has a dimension of k×(N1×N2), the result of is a vector with a dimension of k×1.

[0055] In this embodiment, is a vector with a dimension of N1×1.

[0056] In this embodiment, has a dimension of N1 plus The dimension of is k, which is an N1 + k-dimensional vector.

[0057] Beneficial effects of the above technical solution: Based on the modal feature matrices of all modalities and the attention mechanism model, the fusion feature vector of the CNC machine tool is determined, which can improve the adaptability of the control system to complex working conditions, accurately reflect the real-time state of the CNC machine tool, and improve the accuracy and efficiency of the machining process.

[0058] Embodiment 6: The embodiment of the present invention provides a collaborative control system for a CNC machine tool based on multi-modal fusion, and the control module includes: Feature extraction unit: Obtain the real-time environment data, real-time machining tasks, and machining task requirements of the CNC machine tool, extract features from the real-time environment data of the CNC machine tool to determine the real-time environment vector, and at the same time, extract features from the real-time machining tasks and machining task requirements of the CNC machine tool to determine the real-time task vector; First selected modal data unit: Extract the historical environment vectors from all historical modal data, and determine that all historical modal data corresponding to the historical environment vectors whose similarity to the real-time environment vector is greater than the first pre-threshold is the first selected modal data; Second selected modal data unit: Extract the historical task vectors from all historical modal data, and determine that all historical modal data corresponding to the historical task vectors whose similarity to the real-time task vector is greater than the second pre-threshold is the second selected modal data; Selected modal matrix unit: Determine the optimal modal matrix of each historical modal data that belongs to both the first modal data and the second modal data as the selected modal matrix; Collaborative control strategy unit: Determine the collaborative control strategy based on the modal feature matrix, fusion feature vector, and all selected modal matrices.

[0059] In this embodiment, by obtaining the real-time environment data (such as temperature, humidity, pressure, etc.) and real-time machining task data (such as machining speed, machining accuracy requirements, etc.) of the machine tool, feature extraction is performed to respectively generate: Real-time environment vector: It contains the state data of the environment where the machine tool is located and is used to describe the environmental factors that may affect the performance of the machine tool during the machining process; Real-time task vector: It describes the requirements of the current machining task, such as machining target, accuracy requirements, workpiece type, etc.

[0060] In this embodiment, the historical environment vectors are extracted from the historical modal data and compared with the real-time environment vectors for similarity. If the similarity is greater than the first preset threshold, it is considered that the historical modal data is similar to the current environment, and the corresponding historical modal data is selected as the first selected modal data.

[0061] In this embodiment, a historical task vector is extracted from historical modal data and compared with a real-time task vector for similarity. If the similarity is greater than a second preset threshold, the corresponding historical modal data is selected as the second selected modal data.

[0062] In this embodiment, the similarity between the extracted historical task vector and the real-time task vector can be calculated by cosine similarity, Euclidean distance, or Manhattan distance; the similarity between the extracted historical task vector and the real-time task vector can be calculated by cosine similarity, Euclidean distance, or Manhattan distance.

[0063] In this embodiment, according to the intersection of the first selected modal data and the second selected modal data, the optimal modal matrix of each historical modal data is determined as the selected modal matrix.

[0064] The beneficial effects of the above technical solution: Based on historical modal data, modal feature matrices, and fusion feature vectors, a collaborative control strategy for a numerically controlled machine tool is determined, which can provide a precise and intelligent coordination control solution, improving the adaptability and machining accuracy of the numerically controlled machine tool under complex working conditions.

[0065] Embodiment 7: An embodiment of the present invention provides a collaborative control system for a numerically controlled machine tool based on multi-modal fusion. The collaborative control strategy unit includes: ; ; Among them, represents the error matrix at the current time t, Nu represents the number of selected modal matrices, CM represents the fitting change matrix, represents the a-th selected modal matrix, represents the optimal fitting matrix, represents the nuclear norm of the fitting change matrix, represents the regularization parameter, tE represents the predicted end time of the real-time machining task, represents the fusion control vector.

[0066] In this embodiment, the fusion control vector is used to adjust the influence of the fusion feature vector on the collaborative control strategy.

[0067] In this embodiment, is the calculated optimal fitting matrix, representing the fitting change matrix that minimizes the objective function.

[0068] In this embodiment, represents the Frobenius norm.

[0069] Advantages of the above technical solution: Based on the modal feature matrix, the fusion feature vector, and all selected modal matrices, a cooperative control strategy is determined, which can improve the accuracy of the cooperative control strategy and enhance the adaptability and machining accuracy of the CNC machine tool under complex working conditions.

[0070] Example 8: An embodiment of the present invention provides a cooperative control method for a CNC machine tool based on multi-modal fusion, as Figure 2 shown, including: Step 1: For multiple modes of the CNC machine tool, collect the status information of all modes of the CNC machine tool in real time to determine real-time modal data; Step 2: Obtain the historical modal data of all modes of the CNC machine tool and construct an attention mechanism model; Step 3: Determine the modal feature matrix of all modes based on the real-time modal data, and determine the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model; Step 4: Determine the cooperative control strategy of the CNC machine tool based on the historical modal data, the modal feature matrix, and the fusion feature vector; Step 5: Execute the coordinated control strategy to achieve the coordinated control of the CNC machine tool based on multi-modal fusion.

[0071] Advantages of the above technical solution: By analyzing and determining the real-time modal data to determine the modal feature matrix, determining the fusion feature vector according to the modal feature matrix and the constructed attention mechanism model, determining the cooperative control strategy according to the historical modal data, the modal feature matrix, and the fusion feature vector, and executing the coordinated control strategy to achieve the coordinated control of the CNC machine tool based on multi-modal fusion. An efficient cooperative control mechanism can be formed, improving the intelligence and accuracy of machine tool control, enhancing machining accuracy and production efficiency, and improving the adaptability and operating efficiency of the CNC machine tool under complex working conditions.

[0072] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0073] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A collaborative control system for numerically controlled machine tools based on multimodal fusion, characterized in that, Including: Acquisition module: Obtain multiple modes of the CNC machine tool, and collect the status information of all modes of the CNC machine tool in real time to determine the real-time modal data; Construction module: Obtain the historical modal data of all modes of the CNC machine tool, and construct an attention mechanism model; Analysis module: Determine the modal feature matrix of all modes based on the real-time modal data, and determine the fusion feature vector of the CNC machine tool based on the modal feature matrix of all modes and the attention mechanism model; Control module: Determine the cooperative control strategy of the CNC machine tool based on the historical modal data, modal feature matrix, and fusion feature vector; Execution module: Execute the coordinated control strategy to achieve the coordinated control of the CNC machine tool based on multi-modal fusion.

2. The collaborative control system for a numerically controlled machine tool based on multimodal fusion according to claim 1, wherein, The acquisition module includes: First acquisition unit: Obtain the machine tool type, machine tool structure, and machine tool parameters of the machine tool, and obtain the part characteristics and processing technology of the part to be processed; Modal determination unit: Determine multiple modes of the CNC machine tool based on the machine tool type, machine tool structure, machine tool parameters, part characteristics, and processing technology of the machine tool; Modal sensor group unit: Determine the modal sensor group of each mode based on the structure of the CNC machine tool, where the modal sensor group includes at least one or more modal sensors; Sub-real-time modal data unit: Collect the status information of each mode based on the modal sensor group of each mode, and determine the sub-real-time modal data of each mode; Real-time modal data unit: Determine the real-time modal data of the CNC machine tool based on the sub-real-time modal data of all modes.

3. The collaborative control system for a numerically controlled machine tool based on multimodal fusion according to claim 1, characterized in that, The construction module includes: Historical modal data unit: Obtain the historical modal data of all modes of the CNC machine tool during multiple machining operations, where the historical modal data includes historical environment vectors, historical task vectors, historical modal vectors of each mode, historical modal weights, historical fusion vectors, and the best modal matrix; Initialization unit: Initialize the model parameters of the attention mechanism model, where the model parameters at least include an initial modal transformation weight matrix, an initial attention weight vector, an initial dimensionality reduction weight matrix, and an initial bias vector; Model training unit: Train the initialized attention mechanism model based on the historical modal data of all machining operations, use the historical modal vectors of all modes in the modal data of all historical machining operations as the model input of the attention mechanism model, and use the historical modal weights and historical fusion vectors of all modes in the modal data of all machining operations as the model output of the attention mechanism model; Prediction unit: The attention mechanism model performs forward propagation analysis on the model input to determine the predicted modal weight and predicted fusion vector of each mode for each machining operation; First comparison unit: Perform a first comparison on the historical modal weight and predicted modal weight of each mode in the historical modal data of each machining operation; Adjustment unit: The attention mechanism model performs backward propagation analysis based on the first comparison result, and adjusts the initialized model parameters of the attention mechanism model based on the analysis result to determine the adjustment parameters, where the adjustment parameters at least include an adjusted transformation weight matrix, an adjusted attention weight vector, an adjusted dimensionality reduction weight matrix, and an adjusted bias vector; The second comparison unit: perform a second comparison on the historical fusion vector and the predicted fusion vector in the historical modal data of each processing; The model evaluation unit: evaluate the attention mechanism model based on the second comparison result.

4. A collaborative control system for a numerically controlled machine tool based on multimodal fusion according to claim 2, characterized in that, The analysis module includes: The modal feature vector unit: preprocess the sub-real-time modal data of each modality in the real-time modal data, extract features from the preprocessed sub-real-time modal data of each modality, and determine the modal feature vector of each modality; ; Among them, represents the modal feature vector of the i-th mode at the current moment t, respectively represent the first feature, the j-th feature, and the N1-th feature of the i-th mode at the current moment t, where N1 represents the number of features in the modal feature vector; The modal feature matrix unit: determine the modal feature matrix based on the modal feature vectors of all modalities; ； Among them, represents the modal feature matrix at the current moment t, respectively represent the modal feature vectors of the i-th mode at the current moment t, and N2 represents the number of modes.

5. The collaborative control system for a numerically controlled machine tool based on multimodal fusion according to claim 4, wherein The analysis module further includes: The fusion feature vector unit: determine the fusion feature vector of the CNC machine tool based on the modal feature vectors of all modalities, the modal feature matrix, and the adjustment parameters of the attention mechanism model; ； ； ； ； Among them, represents the fused feature vector at the current moment t, represents the weighted feature vector at the current moment t, represents the transformed feature vector at the current moment t, and B represents the adjustment bias vector, represents the concatenation of the weighted feature vector and the transformed feature vector at the current moment t, represents the relative weight of the i-th modality at the current moment t, represents the adjusted attention weight vector of the i-th modality in the adjustment parameters of the attention mechanism model, represents the adjusted attention weight vector transpose of, represents the adjusted transformation weight matrix in the adjustment parameters of the attention mechanism model, represents the activation value of the i-th modality at the current moment t, represents the activation parameter, represents the adjusted dimensionality reduction weight matrix in the adjustment parameters of the attention mechanism model.

6. The collaborative control system for a numerically controlled machine tool based on multi-modal fusion according to claim 5, wherein The control module includes: The feature extraction unit: obtain the real-time environment data, real-time processing task, and processing task requirements of the CNC machine tool, extract features from the real-time environment data of the CNC machine tool to determine the real-time environment vector, and at the same time, extract features from the real-time processing task and processing task requirements of the CNC machine tool to determine the real-time task vector; The first selected modal data unit: extract the historical environment vectors in all historical modal data, and determine that the historical modal data corresponding to all historical environment vectors with a similarity greater than the first pre-threshold to the real-time environment vector is the first selected modal data; The second selected modal data unit: extract the historical task vectors in all historical modal data, and determine that the historical modal data corresponding to all historical task vectors with a similarity greater than the second pre-threshold to the real-time task vector is the second selected modal data; The selected modal matrix unit: determine the optimal modal matrix of each historical modal data that belongs to both the first modal data and the second modal data as the selected modal matrix; The cooperative control strategy unit: determine the cooperative control strategy based on the modal feature matrix, the fusion feature vector, and all the selected modal matrices.

7. A collaborative control system for a numerically controlled machine tool based on multi-modal fusion according to claim 6, characterized in that, The cooperative control strategy unit includes: ； ； Among them, represents the error matrix at the current time t. $N_u$ represents the number of selected modal matrices, and $CM$ represents the fitting change matrix. represents the $a$-th selected modal matrix. represents the optimal fitting matrix. represents the nuclear norm of the fitting change matrix. represents the regularization parameter, and $t_E$ represents the predicted end time of the real-time machining task. represents the fusion control vector.

8. A collaborative control method for numerically controlled machine tools based on multimodal fusion, characterized in that, including: Step 1: Multiple modalities of the CNC machine tool collect the state information of all modalities of the CNC machine tool in real time to determine the real-time modal data; Step 2: Obtain the historical modal data of all modalities of the CNC machine tool and construct an attention mechanism model; Step 3: Determine the modal feature matrix of all modalities based on the real-time modal data, and determine the fusion feature vector of the CNC machine tool based on the modal feature matrices of all modalities and the attention mechanism model; Step 4: Determine the cooperative control strategy of the CNC machine tool based on the historical modal data, the modal feature matrix, and the fusion feature vector; Step 5: Execute the coordinated control strategy to achieve the coordinated control of the CNC machine tool based on multi-modal fusion.

Images