Multi-modal large model-based actuator motion control performance optimization method and device, medium and equipment

Abstract

The invention discloses an actuator motion control performance optimization method and device based on a multi-modal large model, a medium and equipment, and the method comprises the steps: projecting an image coding vector, a text coding vector and a numerical value coding vector to a same semantic space, and generating a representation vector after multi-modal fusion; performing prediction processing on the characterization vector of the multi-modal fusion to obtain probability distribution of motion control parameters, and obtaining motion control recommendation parameters based on the probability distribution of the parameters through a Bayesian optimization proxy function; burning the motion control recommendation parameters to the actuator to execute a motion test, and collecting performance indexes when the actuator executes the motion test. According to the method, multi-modal information perception is achieved, the method is combined with an existing Bayesian proxy function, the current motion control recommendation parameters are obtained through sampling, the corresponding actuators are operated through the motion control recommendation parameters, the process is repeated, and continuous optimization of the motion control parameters is achieved till the optimal parameter combination is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an executor motion control performance optimization method and device based on a multi-modal large model, a medium and equipment. BACKGROUND

[0002] Motion control algorithm is one of the core technologies in the field of industrial automation, which originated in the early 20th century and has become the cornerstone of modern control systems after more than a century of evolution. A representative motion control algorithm is the PID algorithm, which involves three core parameters in the tuning process: proportion (p), adjusting the output according to the size of the current error (the larger the error, the greater the adjustment); integral (I), eliminating long-term cumulative errors (such as small deviations in temperature control); and differential (d), predicting error trends and suppressing oscillations in advance. Because of its simple structure, strong robustness (insensitivity to model errors), and no need for accurate mathematical models, it plays a key role in areas such as automotive cruise control, human-machine balance control, and robotic motion control.

[0003] However, as the complexity of industrial systems increases (such as nonlinear, time-varying, and multi-variable coupling scenarios), the limitations of traditional motion control algorithms gradually emerge, including fixed parameters that are difficult to adapt to dynamic environments (such as load changes and mechanical wear), and difficulty in balancing multiple objectives (precision, speed, and energy consumption) when optimizing multiple objectives. Therefore, motion control algorithms are constantly improving, such as introducing a rule base to dynamically adjust parameters based on errors, and combining model reference adaptive control or neural networks to learn system characteristics and optimize parameters online.

[0004] Existing executor motion control methods have their training nodes and execution nodes located on the host computer and the executor, respectively, and the data collection process requires human intervention, resulting in low automation. On the other hand, the data input in the optimization process of the output control parameters is relatively single, and cannot achieve multi-directional information perception, making it difficult to obtain optimal control parameters. SUMMARY

[0005] In view of the above problems, the present application is proposed to provide an executor motion control performance optimization method, device, medium and equipment based on a multi-modal large model to overcome the above problems or at least partially solve the above problems.

[0006] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0007] According to a first aspect of an embodiment of the present application, an executor motion control performance optimization method based on a multi-modal large model is provided, which comprises: S1, collect the GUI function interface image of the upper computer in communication with the actuator, and collect protocol version information and parameter information of the actuator to obtain basic data, and perform multi-modal information encoding processing according to the basic data to obtain an image encoding vector, a text encoding vector and a numerical encoding vector respectively; S2, projecting the image encoding vector, the text encoding vector and the numerical encoding vector to the same semantic space to generate a multi-modal fusion representation vector; S3, performing prediction processing on the multi-modal fusion representation vector to obtain a probability distribution of the motion control parameter, and then obtaining a motion control recommended parameter based on the parameter probability distribution through a Bayesian optimization agent function; S4, burning the motion control recommended parameter to the actuator to perform motion test, collecting performance indicators when the actuator performs motion test, and repeating steps S1-S3.

[0008] In some embodiments of the application, the multi-modal information encoding processing according to the basic data to obtain an image encoding vector, a text encoding vector and a numerical encoding vector respectively comprises: collecting the GUI function interface image of the upper computer in communication with the actuator, and performing encoding processing on the GUI function interface image based on an image encoder to obtain the image encoding vector; collecting the protocol version information of the actuator, and performing encoding processing on the protocol version information based on a text encoder to obtain the text encoding vector; collecting the parameter information of the actuator, and performing encoding processing on the parameter information based on a hierarchical encoder to obtain the numerical encoding vector.

[0009] In some embodiments of the application, the method comprises projecting the image encoding vector, the text encoding vector and the numerical encoding vector to the same semantic space through a multi-layer perceptron (MLP), wherein: ; ; ; In the formula, the image encoding vector is represented as the transformed output after projection into the semantic space, the text encoding vector is represented as the transformed output after projection into the semantic space, the numerical encoding vector is represented as the transformed output after projection into the semantic space.

[0010] In some embodiments of the application, the multi-modal fusion representation vector is obtained through the following formula: ; ; In the formula, is a cascade of the transformerDecoderBlock structure, L is the number of layers of the stacked transformerDecoderBlock structure, is the multi-modal fusion representation vector.

[0011] According to a second aspect of the embodiments of the present application, a multi-modal large model-based actuator motion control performance optimization device is provided, which comprises: A multi-modal information perception module is configured to collect a GUI function interface image of an upper computer in communication with an actuator, and collect protocol version information and parameter information of the actuator to obtain basic data, and perform multi-modal information encoding processing on the basis of the basic data to obtain an image encoding vector, a text encoding vector and a numerical encoding vector respectively; A probabilistic large model agent module is configured to project the image encoding vector, the text encoding vector and the numerical encoding vector to the same semantic space to generate a multi-modal fusion representation vector; A probabilistic prediction output layer module is configured to perform prediction processing on the multi-modal fusion representation vector to obtain a probability distribution of a motion control parameter, and then obtain a motion control recommended parameter based on the parameter probability distribution through a Bayesian optimization agent function; A motion control performance optimization module is configured to burn the motion control recommended parameter to the actuator to perform a motion test, collect performance indicators when the actuator performs the motion test, and control the multi-modal information perception module, the probabilistic large model agent module and the probabilistic prediction output layer module to restart operation.

[0012] In some embodiments of the present application, the multi-modal information perception module is configured to: Collect a GUI function interface image of an upper computer in communication with an actuator, and perform encoding processing on the GUI function interface image based on an image encoder to obtain the image encoding vector; Collect protocol version information of the actuator, and perform encoding processing on the protocol version information based on a text encoder to obtain the text encoding vector; Collect parameter information of the actuator, and perform encoding processing on the parameter information based on a hierarchical encoder to obtain the numerical encoding vector.

[0013] In some embodiments of the present application, the probabilistic large model agent module is configured to project the image encoding vector, the text encoding vector and the numerical encoding vector to the same semantic space through a multi-layer perceptron (MLP), wherein: wherein, characterized as an image encoding vector transformed output after projection into semantic space, characterized as a text encoding vector transformed output after projection into semantic space, characterized as a numerical encoding vector transformed output after projection into semantic space.

[0014] In some embodiments of the present application, the probability prediction output layer module generates a multi-modal fused representation vector, which is obtained by the following formula: wherein, is a concatenation of , L is the number of stacked transformerDecoderBlock structures, is the multi-modal fused representation vector.

[0015] According to a third aspect of the embodiments of the present application, a computer readable storage medium is provided, the computer readable storage medium stores computer program instructions, the computer program instructions are loaded and executed by a processor to implement the operations performed by the method according to any one of the above aspects.

[0016] According to a fourth aspect of the embodiments of the present application, an electronic device is provided, which includes a processor and a memory, the memory stores computer program instructions executable by the processor, and the processor executes the computer program instructions to implement the instructions of the method according to any one of the above aspects.

[0017] The technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: ​​​​​The embodiment of the application provides a kind of based on multimodal big model's executor motion control performance optimization method, device, medium and equipment, the method is by collecting the GUI function interface image of host computer, the protocol version information and parameter information of executor, and by encoding processing obtains image encoding vector, text encoding vector and numerical encoding vector, realizes multimodal information perception, then aligns each encoding vector obtained to same semantic space as input, outputs the probability distribution of control algorithm parameter, and combines the existing bayesian proxy function with probability prediction big model, obtains the current motion control recommended parameter by sampling, with the motion control recommended parameter of the executor corresponding to the motion control recommended parameter is burned and runs, repeat the process, realize the efficient data closed loop of cross node and the continuous optimization to executor motion control parameter, until reaching optimal parameter combination, realize the automatic flow of interactive information, can adapt to different training algorithm and motion control algorithm, specific preferable expansibility.

[0018] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the content of the specification can be implemented, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0020] Figure 1 The flowchart of the executor motion control performance optimization method based on the multimodal big model provided by the embodiment of the present application is shown in the figure. Figure 2 The principle structure diagram of the executor motion control performance optimization device based on the multimodal big model provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0021] The exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

[0022] Various structural diagrams according to embodiments of the present disclosure are shown in the drawings. These diagrams are not drawn to scale in which certain details are exaggerated for clarity of presentation and may omit certain details pertaining to the nature of the application. The shapes of the various regions, layers shown in the drawings, and their relative sizes and positional relationships are merely exemplary, and in actuality, they may deviate due to manufacturing tolerances or technical limitations, and regions / layers with different shapes, sizes, relative positions can be additionally designed according to actual needs by those skilled in the art.

[0023] In the context of the present disclosure, when a layer / element is referred to as being located "on" another layer / element, the layer / element can be directly located on the other layer / element, or there can be an intervening layer / element between them. In addition, if a layer / element is located "on" another layer / element in one orientation, it can be located "under" the other layer / element when the orientation is reversed. In the context of the present disclosure, similar or identical components can be denoted by the same or similar reference numerals.

[0024] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with specific embodiments. It should be understood that the embodiments of the present disclosure and the specific features in the embodiments are detailed descriptions of the technical solutions of the present disclosure, and not limitations of the technical solutions of the present disclosure. In the case of no conflict, the technical features in the embodiments of the present disclosure and the embodiments can be combined with each other.

[0025] Figure 1 is a flow diagram of an actuator motion control performance optimization method based on a multi-modal large model provided by an embodiment of the present disclosure, as shown in Figure 1 The actuator motion control performance optimization method based on a multi-modal large model includes the following steps: S1, collect the GUI function interface image of the upper computer communicating with the actuator, and collect the protocol version information and parameter information of the actuator to obtain basic data, and perform multi-modal information coding processing on the basic data to obtain an image coding vector, a text coding vector and a numerical coding vector respectively; The multi-modal information coding processing on the basic data to obtain an image coding vector, a text coding vector and a numerical coding vector according to the embodiments of the present disclosure includes: collecting the GUI function interface image of the upper computer communicating with the actuator, performing coding processing on the GUI function interface image based on an image encoder to obtain the image coding vector; collecting the protocol version information of the actuator, performing coding processing on the protocol version information based on a text encoder to obtain the text coding vector; collecting the parameter information of the actuator, performing coding processing on the parameter information based on a hierarchical encoder to obtain the numerical coding vector.

[0026] Specifically, the embodiment of the present application encodes the GUI function interface image based on an image encoder to obtain the image encoding vector, including: flattening the collected GUI function interface image is divided into image blocks with a block size of , H, W and C represent the length, width and color channel number of the image respectively, P is the image block size, and the divided image blocks have a flattened dimension of , the divided image blocks are transformed by the following formula to obtain: ; , wherein is a transformation matrix (D is the dimension of the transformed hidden layer, R represents the real number field, and Flatten is a flattening operation), is a bias vector.

[0027] In order to retain spatial position information, a learnable position encoding is added , to obtain a new transformation output : ; The embodiment of the present application inputs the transformer encoder network structure to obtain the image encoding vector: .

[0028] The embodiment of the present application encodes the protocol version information based on a text encoder, and obtains the text encoding vector by the following formula : ; ; , wherein t is an input text token; is a projection matrix of the text encoder.

[0029] In the embodiment of the present application, the text encoder may be a BiLSTM (bidirectional LSTM network) model or a BERT model.

[0030] The embodiment of the present application encodes the parameter information by a hierarchical encoder to obtain the numerical encoding vector, wherein the hierarchical encoder obtains the numerical encoding vector based on a multi-layer MLP residual network structure , including the following formula: ; ; , wherein​ is a numerical value input for parameter information, is a learnable bias term, is the number of hierarchical coding layers, is a numerical value transformation matrix, and LayerNorm is a layer normalization function, is an activation function, and the numerical coding output vector .

[0031] S2, projecting the image coding vector, the text coding vector and the numerical coding vector into the same semantic space to generate a multi-modal fusion representation vector; The embodiment of the application projects the image coding vector, the text coding vector and the numerical coding vector into the same semantic space through a multi-layer perceptron (MLP), wherein: ; ; ; In the formula, represents the image coding vector transformed output after projection into the semantic space, represents the text coding vector transformed output after projection into the semantic space, represents the numerical coding vector transformed output after projection into the semantic space.

[0032] After obtaining the transformed outputs of the image coding vector, the text coding vector and the numerical coding vector, the embodiment of the application obtains the multi-modal fusion representation vector through the following formula : ; ; In the formula, is a concatenation of , and L is the number of layers of the stacked transformerDecoderBlock structure, is the multi-modal fusion representation vector.

[0033] S3, performing prediction processing on the multi-modal fusion representation vector to obtain a probability distribution of the motion control parameter, and then obtaining a motion control recommendation parameter based on the probability distribution of the parameter through a Bayesian optimization agent function; The embodiment of the application performs prediction processing on the multi-modal fusion representation vector through a probability prediction large model to obtain a probability distribution of the motion control parameter, that is: ; Generate control parameter mean value conforming to Gaussian distribution based on output vector of last layer of transformerDecoderBlock structure and standard deviation network.

[0034] Specifically, the embodiment of the application performs prediction processing on the multi-modal fusion representation vector through a probability prediction output layer, including the following formula: ; ; In the formula, is is the dimension of the motion control parameter, is the transformation vector, , two-dimensional is compressed into one dimension by maximum value and average value respectively.

[0035] The optimization objective of the probability prediction large model mainly includes three parts, is the regression output of the probability large model, is the regular constraint loss of the model parameter, is the contrastive learning loss of the extracted image representation.

[0036] ; Wherein: ; ; ; Wherein represents the set of trainable weight parameters of the probability prediction large model, represents the set of teacher model parameters for external knowledge distillation of the multi-modal part of the probability prediction large model; represents the cosine similarity. loss weight hyperparameters, which can be set artificially.

[0037] Then the application obtains the motion control recommended parameter based on the probability distribution of the parameter through the conventional Bayesian optimization agent function, which is realized through the following formula: ; Wherein, the motion control recommended parameter is the next parameter group recommendation based on the current motion control parameter . is the parameter sample distribution space, respectively represent the parameters a mean and a standard deviation of the multi-modal fusion representation vector.

[0038] S4, burn the motion control recommendation parameter to the actuator to perform a motion test, collect the performance index when the actuator performs the motion test, and repeat steps S1-S3.

[0039] In the embodiment of the application, the performance index , wherein alpha, beta and gamma are hyperparameters, the hyperparameters are set values, and are set according to requirements; representing an overshoot, representing a steady state time, representing a control error, the overshoot, the steady state time and the control error being obtained by the actuator, when the actuator moves to a steady state, the three physical quantities corresponding to the steady state are calculated according to the collected motion state data, and are sent to the upper computer.

[0040] The embodiment of the application obtains the current performance index and the PID value triple by analyzing the GUI function interface image, collects protocol version information T and parameter information d of the actuator, obtains basic data, that is, an initial data set of t pairs.

[0041] The embodiment of the application updates the probability prediction large model based on the training target and the basic data, wherein the probability prediction large model of the tth edition is obtained based on the current data set, to obtain a probability distribution of the motion control parameter , based on the probability distribution of the motion control parameter , the motion control recommendation parameter , that is, the new motion control recommendation parameter is obtained by the Bayesian optimization agent function based on the probability distribution of the parameter; burn the motion control recommendation parameter to the actuator to run and perform a motion test, collect the GUI function interface image carrying the new performance index , and repeat the foregoing steps S1-S3, to realize continuous optimization and iteration of the motion control performance.

[0042] It should be noted that the execution of steps S1-S4 of the embodiment of the present application can be performed by a large model based agent, which is a software program capable of autonomously performing tasks by simulating human decision-making processes to complete specific work. The large model based agent is an intelligent entity with autonomous perception, reasoning, decision-making and execution capabilities, with a large-scale pre-trained model as its core. Its core features include: multi-modal interaction, supporting text, speech, image and other multi-modal input and output; task generalization, achieving cross-scene task migration through the context understanding ability of the large model; dynamic evolution, continuously optimizing strategies and adapting to environmental changes through incremental learning mechanisms. The design of the large model based agent aims to improve work efficiency, reduce human errors, and work uninterruptedly for 24 hours. The large model based agent has good applicability and expandability and can be applied in various fields such as customer service, data analysis, automated testing, etc.

[0043] The method for optimizing the performance of the actuator motion control based on the multi-modal large model, by collecting the GUI function interface image of the upper computer, the protocol version information and the parameter information of the actuator, and through coding processing to obtain image coding vectors, text coding vectors and numerical coding vectors, realizes multi-modal information perception, and then aligns the obtained various coding vectors to the same semantic space as input, outputs the probability distribution of the control algorithm parameters, and combines with the existing Bayesian agent function, obtains the current motion control recommended parameter through sampling, and burns and runs the corresponding actuator with the motion control recommended parameter, repeats the process, uploads the performance index of the actuator to the upper computer, and multi-modal perceives the GUI function interface image of the upper computer and inputs the probability prediction large model, realizes efficient data closed loop across nodes and continuous optimization of the actuator motion control parameters, until the optimal parameter combination is reached, realizes automatic flow of interactive information, and can adapt to different training algorithms and motion control algorithms, and has specific better expandability.

[0044] On the basis of the above embodiment, as an implementation of the method shown in the above Figure 1 The present application provides an embodiment of an actuator motion control performance optimization device based on a multi-modal large model. The device embodiment corresponds to the method embodiment shown in the above Figure 1 The device can be applied to various electronic devices, as shown in the above Figure 2 The actuator motion control performance optimization device based on a multi-modal large model includes: The multi-modal information perception module 100 is configured to collect a GUI function interface image of an upper computer communicating with an actuator, and collect protocol version information and parameter information of the actuator, to obtain basic data, and perform multi-modal information coding processing on the basic data to obtain an image coding vector, a text coding vector and a numerical coding vector respectively. The probabilistic large model agent module 200 is configured to project the image coding vector, the text coding vector and the numerical coding vector to the same semantic space to generate a multi-modal fusion representation vector. The probabilistic prediction output layer module 300 is configured to perform prediction processing on the multi-modal fusion representation vector to obtain a probability distribution of a motion control parameter, and obtain a motion control recommended parameter based on the parameter probability distribution through a Bayesian optimization agent function. The motion control performance optimization module 400 is configured to burn the motion control recommended parameter to the actuator to perform a motion test, collect a performance index when the actuator performs the motion test, and control the multi-modal information perception module, the probabilistic large model agent module and the probabilistic prediction output layer module to restart operation.

[0045] In the embodiment of the present application, the multi-modal information perception module 100 is configured to collect a GUI function interface image of an upper computer communicating with an actuator, and collect protocol version information and parameter information of the actuator, to obtain basic data, and perform multi-modal information coding processing on the basic data to obtain an image coding vector, a text coding vector and a numerical coding vector respectively.

[0046] In the embodiment of the present application, the probabilistic large model agent module 200 is configured to project the image coding vector, the text coding vector and the numerical coding vector to the same semantic space through a multi-layer perceptron (MLP), wherein: ; ; ; In the formula, The image coding vector is represented as The transformed output after projection to the semantic space, The text coding vector is represented as The transformed output after projection to the semantic space, The numerical coding vector is represented as The transformed output after projection to the semantic space.

[0047] In the embodiment of the present application, the probability prediction output layer module 200 generates a multi-modal fused feature vector, which is obtained by the following formula: ; ; is a concatenation of , L is the number of stacked transformerDecoderBlock structures, is the multi-modal fused feature vector.

[0048] The executor motion control performance optimization device based on the multi-modal large model provided in the embodiment of the present application can execute the executor motion control performance optimization method based on the multi-modal large model provided in the above embodiment. The executor motion control performance optimization device based on the multi-modal large model has the corresponding functional steps and beneficial effects of the executor motion control performance optimization method based on the multi-modal large model described in the above embodiment. For details, please refer to the embodiments of the executor motion control performance optimization method based on the multi-modal large model. The embodiments of the present application will not be repeated here.

[0049] The embodiment of the present application further provides an electronic device, which can include a processor and a memory. The processor and the memory can be connected through a bus or other means. The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or combinations of the above chips. The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the executor motion control performance optimization method based on the multi-modal large model in the embodiment of the present application. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the executor motion control performance optimization method based on the multi-modal large model in the above method embodiment.

[0050] ​The memory can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required by at least one function, and the like, and the data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. The one or more modules are stored in the memory and, when executed by the processor, perform the method of optimizing the performance of the motion control of the executor based on the multi-modal large model as described above. The above electronic device can correspond to the relevant description and effects of the corresponding embodiment of the above method. Here, it will not be repeated. Those skilled in the art can understand that all or part of the processes in the above embodiment method can be completed by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above embodiments. The storage medium can be a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memory.

[0051] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to not obscure the understanding of this description.

[0052] Similarly, it is to be understood that the above-described embodiments of the application are merely illustrative of the application and should not be considered restrictive of the scope of the application as this scope is to be determined by the appended claims. Similarly, while the application has been disclosed in the context of the existing technology, the application is not limited to the existing technology.

Claims

1. A method for optimizing actuator motion control performance based on a multimodal large model, characterized in that, The actuator motion control performance optimization method based on a multimodal large model includes: S1. Collect the image of the host computer GUI function interface that communicates with the actuator, and collect the protocol version information and parameter information of the actuator to obtain basic data. Based on the basic data, perform multimodal information encoding processing to obtain image encoding vector, text encoding vector and numerical encoding vector respectively. S2. Project the image encoding vector, text encoding vector, and numerical encoding vector onto the same semantic space to generate a multimodal fusion representation vector; S3. The multimodal fusion representation vector is subjected to prediction processing to obtain the probability distribution of motion control parameters. Then, the recommended motion control parameters are obtained based on the probability distribution of the parameters by Bayesian optimization surrogate function. S4. Burn the recommended motion control parameters into the actuator to perform motion testing, collect the performance indicators of the actuator when performing motion testing, and repeat steps S1-S3.

2. The actuator motion control performance optimization method based on a multimodal large model according to claim 1, characterized in that, The step of performing multimodal information encoding processing based on the basic data to obtain image encoding vectors, text encoding vectors, and numerical encoding vectors includes: The GUI function interface image of the host computer communicating with the actuator is acquired, and the GUI function interface image is encoded based on the image encoder to obtain the image encoding vector; The protocol version information of the executor is collected, and the protocol version information is encoded using a text encoder to obtain the text encoding vector; The parameter information of the actuator is collected, and the parameter information is encoded based on the hierarchical encoder to obtain the numerical encoding vector.

3. The actuator motion control performance optimization method based on a multimodal large model according to claim 1, characterized in that, The method includes: projecting the image encoding vector, text encoding vector, and numerical encoding vector into the same semantic space using a multilayer perceptron (MLP), wherein: ； ； ； In the formula, Represented as image coding vector The transformed output after projection onto the semantic space. Represented as a text encoding vector The transformed output after projection onto the semantic space. Represented as a numerical encoding vector The transformed output after projection onto the semantic space.

4. The actuator motion control performance optimization method based on a multimodal large model according to claim 3, characterized in that, The multimodal fusion representation vector is obtained by the following formula: ； ； In the formula, for The cascading of transformerDecoderBlock structures, where L is the number of layers in the stacked transformerDecoderBlock structure. is the representation vector after multimodal fusion.

5. A device for optimizing actuator motion control performance based on a multimodal large model, characterized in that, The actuator motion control performance optimization device based on a multimodal large model includes: The multimodal information perception module is used to collect images of the GUI function interface of the host computer communicating with the actuator, as well as to collect the protocol version information and parameter information of the actuator to obtain basic data. Based on the basic data, multimodal information encoding processing is performed to obtain image encoding vector, text encoding vector and numerical encoding vector respectively. The high-probability model agent module is used to project the image encoding vector, text encoding vector and numerical encoding vector onto the same semantic space to generate a multimodal fusion representation vector. The probability prediction output layer module is used to perform prediction processing on the multimodal fusion representation vector to obtain the probability distribution of motion control parameters, and then obtain the recommended motion control parameters based on the probability distribution of the parameters through a Bayesian optimization surrogate function. The motion control performance optimization module is used to burn the recommended motion control parameters into the actuator to perform motion testing, collect the performance indicators of the actuator when performing motion testing, and control the multimodal information perception module, the high probability model intelligent agent module, and the probability prediction output layer module to restart and run.

6. The actuator motion control performance optimization device based on a multimodal large model according to claim 5, characterized in that, The multimodal information sensing module is used for: The GUI function interface image of the host computer communicating with the actuator is acquired, and the GUI function interface image is encoded based on the image encoder to obtain the image encoding vector; The protocol version information of the executor is collected, and the protocol version information is encoded using a text encoder to obtain the text encoding vector; The parameter information of the actuator is collected, and the parameter information is encoded based on the hierarchical encoder to obtain the numerical encoding vector.

7. The actuator motion control performance optimization device based on a multimodal large model according to claim 5, characterized in that, The high-probability model agent module is used to: project the image encoding vector, text encoding vector, and numerical encoding vector into the same semantic space using a multilayer perceptron (MLP), wherein: ； ； ； In the formula, Represented as image coding vector The transformed output after projection onto the semantic space. Represented as a text encoding vector The transformed output after projection onto the semantic space. Represented as a numerical encoding vector The transformed output after projection onto the semantic space.

8. The actuator motion control performance optimization device based on a multimodal large model according to claim 7, characterized in that, The probability prediction output layer module generates a multimodal fused representation vector, which is obtained through the following formula: ； ； In the formula, for The cascading of transformerDecoderBlock structures, where L is the number of layers in the stacked transformerDecoderBlock structure. is the representation vector after multimodal fusion.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that are loaded and executed by a processor to perform the operations described in any one of claims 1-4.

10. An electronic device comprising a processor and a memory, characterized in that, The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements the instructions of the method as described in any one of claims 1-4.

Images