Method and apparatus for generating workpiece surface topography based on the generation of multimodal images

Abstract

The present invention discloses a method and apparatus for generating a workpiece surface topography based on multimodal image generation, belonging to the technical field of machining data processing. The method includes: constructing a guidance vector based on multimodal information from historical data of different machining methods; adding noise to a low-dimensional representation of the surface topography grayscale image based on a diffusion model to obtain a noise vector; inputting the guidance vector, time step, and noise vector into a de-diffusion process to reduce noise layer by layer and restore the low-dimensional representation; extracting target multimodal information to construct a target guidance vector; inputting the random noise latent variable and the target guidance vector into the trained diffusion model to obtain the target low-dimensional representation; and obtaining a grayscale image of the target surface topography through a decoder; and performing quality evaluation using an image quality comprehensive evaluation module. The present invention utilizes a diffusion model to achieve accurate mapping from multimodal information to a surface topography image, which is characterized by fast generation and high fidelity, and has great potential for real-time surface topography prediction.

Description

[Technical Field]

[0001] The present invention belongs to the technical field of machining data processing, and more specifically, relates to a method and apparatus for generating surface topography of a workpiece based on the generation of multimodal images. [Background technology]

[0002] In the field of machining, machining technology is the foundation. Various processing parameters, such as the surface roughness of the part and the surface topography error, which are related to the mechanical and optical properties of the part surface, affect the surface quality of the part. Because different processing parameters correspond to different processing topographies, in actual experiments there are many processing parameters and corresponding topography measurement history data. How to use existing data a priori to provide empirical knowledge for subsequent processing parameters is one of the key factors in realizing intelligent manufacturing.

[0003] Prior art techniques typically require the use of expensive inspection equipment to measure the surface topography of machined parts offline or in situ, which requires a continuous machining-inspection-remachining process and multiple iterations to achieve the target machining accuracy. This requires significant manpower and expense, but results in low machining efficiency. Therefore, it is more advantageous to use methods for simulating part surface topography and generating grayscale images of the surface topography. Current surface topography simulation methods fall into two categories: determining the 3D surface topography of a workpiece through analytical models, and, in recent years, generating 3D surface topography using learning techniques. However, methods for obtaining surface topography images based on surface topography simulation typically require complex modeling, extensive computation, and are quite time-consuming.

[0004] The patent document with publication number CN112387995A discloses a method for predicting the surface topography after ultra-precision turning of a free-form surface. By combining the two research directions of tool path planning and surface topography simulation, the surface topography simulation is performed in the area L according to the tool path planned based on the active control of machining accuracy. x ×L y Resolution D x and d y The method divides the surface into m×n grids, calculates the coordinate data of all grid points within the simulation region based on the geometric relationship between each grid point and the cutter contact point in the planned tool path, and uses the calculated coordinate data to reconstruct the curved surface, achieving simulation and modeling of the surface topography by single-point diamond turning of the curved surface, and removing the shape components of the curved surface from the obtained simulation model to predict the machining error. However, the method of determining the surface topography based on analytical modeling used in this invention has problems such as complex modeling, large calculation volume, and time-consuming.

[0005] The patent document with publication number CN116012480A discloses a data-driven method for generating grayscale images of cutting surface topography, in which a generator and a discriminator of a neural network-based generative adversarial network model are used to directly convert the machining signal spectrum map and tool stiffness data into grayscale image data of cutting surface topography. However, the generative adversarial network model used in this invention lacks precision, and the generated data cannot well simulate the distribution of actual data, and the generalization performance is poor. [Overview of the Initiative]

[0006] The objective of the present invention is to provide a method and apparatus for generating the surface topography of a workpiece based on the generation of a multi-modal image, which uses a diffusion model to achieve accurate mapping from the multi-modal information of the workpiece to the surface topography image of the workpiece, has a stronger ability to utilize historical data, and can realize the generation of a grayscale image of the surface topography with high speed and high fidelity, while accurately predicting the current surface topography information through real-time processing information.

[0007] In order to achieve the above objectives of the present invention, the technical solutions provided by the present invention are as follows:

[0008] In a first aspect, an embodiment of the present invention provides a method for generating a surface topography of a workpiece based on generating a multimodal image, the method comprising: Step 1 involves collecting and marking surface topography images and processing signal spectrum diagrams using different processing methods, obtaining multimodal information, and processing the multimodal information to obtain guide vectors. Step 2: compressing a grayscale image corresponding to the surface topography image into a first low-dimensional representation through an encoder and inputting the compressed grayscale image into a diffusion model; adding noise to the first low-dimensional representation layer through a forward diffusion process to obtain a noise vector; reducing noise layer by layer through a reverse diffusion process based on the guide vector, the time step, and the noise vector; and then restoring a second low-dimensional representation to train the diffusion model; Step 3: extracting the target multi-modal information in application to construct a target guide vector, inputting the randomly generated Gaussian noise latent variables, time steps and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and passing the target low-dimensional representation through a decoder to obtain a grayscale image of the target surface topography; The process includes step 4, inputting the grayscale image of the target surface topography into an image quality comprehensive evaluation module to evaluate the fidelity of the grayscale image of the target surface topography.

[0009] The present invention adopts a diffusion model. First, based on the historical data of machining technology, multi-modal information of different machining methods is collected and a guide vector is constructed. During the model training process, the grayscale image corresponding to the surface topography image is compressed into a first low-dimensional representation through an encoder. Noise is added layer by layer to the first low-dimensional representation using the forward diffusion process of the diffusion model to obtain a noise vector. The noise is reduced layer by layer using the inverse diffusion process of the diffusion model and the noise vector is restored to a second low-dimensional representation to train the diffusion model. During actual application, a target guide vector is constructed based on the target multi-modal information. The randomly generated two-dimensional Gaussian noise latent variables, time steps and target guide vector are input into the trained diffusion model to obtain a target low-dimensional representation. A decoder is used to convert the target low-dimensional representation into a grayscale image of the surface topography. Finally, an image quality comprehensive evaluation module is used to evaluate the fidelity of the generated image. The method proposed by the present invention can realize accurate mapping from multi-modal information to surface topography images, make full use of historical data, and realize fast and high-fidelity generation of grayscale images of surface topography, which is also very suitable for predicting surface topography images in real-time machining scenes.

[0010] Furthermore, in step 1, the step of marking the surface topography image and the processed signal spectrum diagram to obtain multimodal information includes: marking the surface topography image to obtain corresponding text information, the text information including a processing method, a feed rate, a workpiece material, a tool geometry, and a vibration between the tool and the workpiece; marking the processed signal spectrum diagram to obtain a corresponding processed spectral signal; The multimodal information includes text information and processed spectral signals.

[0011] Furthermore, in step 1, the step of processing the multimodal information to obtain a guidance vector includes: The method includes converting the text information and the processed spectral signal into a representation format by a text encoder and a spectral signal encoder, respectively, and cascading the converted representation format and using the obtained embedding feature vector as a guiding vector, wherein the text encoder and the spectral signal encoder use a contrastive language image pre-training model (CLIP).

[0012] Furthermore, the encoder and decoder constitute a variational autoencoder.

[0013] Furthermore, in step 2, the despreading process uses a Unet noise estimation network based on a cross-attention mechanism, which is used to generate estimated noise, and the estimated noise is used for noise reduction at each time step.

[0014] Furthermore, in step 4, the image quality comprehensive evaluation module includes a high-dimensional semantic feature extractor, a low-dimensional deformation feature extractor, and a regression model. The grayscale image of the target surface topography is passed through a high-dimensional semantic feature extractor and a low-dimensional deformation feature extractor to extract semantic features and deformation features, respectively. The semantic features and deformation features are then input into a regression model after feature fusion, and a quality score for evaluating the fidelity of the grayscale image of the target surface topography is predicted by logistic regression of the regression model.

[0015] Furthermore, in step 4, the high-dimensional semantic feature extractor includes a pre-trained EfficientNetV2 network, and the low-dimensional deformation feature extractor includes a pre-trained VGG16 network.

[0016] In a second aspect, in order to achieve the above-mentioned object of the present invention, embodiments of the present invention are provided. a guide vector construction unit for collecting and marking surface topography images and processing signal spectrum diagrams according to different processing methods, obtaining multi-modal information, and processing the multi-modal information to obtain a guide vector; a model training unit for compressing a grayscale image corresponding to the surface topography image into a first low-dimensional representation through an encoder and inputting the compressed grayscale image into a diffusion model; adding noise to the first low-dimensional representation layer layer by layer through a forward diffusion process to obtain a noise vector; reducing noise layer by layer through a reverse diffusion process based on the guide vector, the time step, and the noise vector; and restoring a second low-dimensional representation to train the diffusion model; a model application unit for extracting target multi-modal information during application to construct a target guide vector, inputting the randomly generated Gaussian noise latent variables, time steps and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and passing the target low-dimensional representation through a decoder to obtain a grayscale image of the target surface topography; The present invention further provides an apparatus for generating a surface topography of a workpiece based on the generation of a multimodal image, the apparatus including: a quality evaluation unit for inputting the grayscale image of the target surface topography into an image quality comprehensive evaluation module to evaluate the fidelity of the grayscale image of the target surface topography.

[0017] In a third aspect, in order to achieve the above-mentioned object of the present invention, an embodiment of the present invention further provides an equipment for generating a surface topography of a workpiece based on generation of a multimodal image, the equipment including a memory for storing a computer program and a processor, and when the processor executes the computer program, realizes the method for generating a surface topography of a workpiece based on generation of a multimodal image provided by the embodiment of the present invention in the first aspect.

[0018] In a fourth aspect, in order to achieve the above object of the present invention, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed on a computer, realizes the method for generating a surface topography of a workpiece based on generating a multimodal image provided by the embodiment of the present invention in the first aspect.

[0019] The beneficial effects of the present invention are as follows: (1) The method proposed by the present invention realizes accurate mapping from multimodal information to a grayscale image of surface topography. Compared with traditional detection devices and surface topography simulation methods, the method proposed by the present invention significantly reduces data collection time and has stronger capabilities in utilizing historical data. (2) The method proposed by the present invention only involves the training process of the generator, i.e., the training of the forward diffusion process and the reverse diffusion process, and the training process is more stable. Compared with the conventional technology that involves training two parts, the generator and the discriminator, and requires the convergence of two networks, the method proposed by the present invention significantly reduces the image generation time while avoiding problems such as mode collapse and deterioration of generalization performance of the generative adversarial network during the training process. (3) The method proposed by the present invention uses randomly generated noise as the input of the diffusion model in practical applications, and because of its randomness and diversity, the method proposed by the present invention can generate a variety of surface topography images that are high-quality, diverse, and suitable for real situations. (4) The method proposed by the present invention can accurately predict the current surface topography information through real-time processing information, which helps processors to timely obtain surface topography images during the processing process, and helps operators quickly judge the product quality of the processed parts and adjust the processing parameters in time. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a flowchart of a method for generating a surface topography of a workpiece based on generating a multimodal image, provided by an embodiment of the present invention. [Figure 2] Figure 2 is a schematic flowchart of the backdiffusion process based on a cross-attention mechanism provided by an embodiment of the present invention. [Figure 3] Figure 3 is a schematic flowchart of an actual application process based on a diffusion model, provided by an embodiment of the present invention. [Figure 4] Figure 4 is a schematic diagram of an image quality comprehensive evaluation module provided by an embodiment of the present invention. [Figure 5] FIG. 5 is a schematic diagram of an apparatus for generating a surface topography of a workpiece based on generating a multimodal image, provided by an embodiment of the present invention. [Figure 6] FIG. 6 is a schematic structural diagram of a workpiece surface topography generation equipment based on multi-modal image generation provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] To further clarify the object, technical solution, and advantages of the present invention, the invention will be described in more detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are used solely for illustrative purposes and do not limit the scope of protection of the invention.

[0022] The inventive concept of the present invention is as follows: Current methods for determining surface topography based on analytical modeling methods suffer from complex modeling, high computational complexity, and time-consuming processing. Generative adversarial network models lack precision, and the generated data cannot adequately simulate the distribution of actual data, resulting in poor generalization performance. In response to these problems, the present invention proposes a method and apparatus for generating the surface topography of a workpiece based on multimodal image generation. Using a diffusion model, the method builds a mapping model between multimodal information, such as machining signals, machining methods, feed rates, workpiece materials, tool geometry, and vibration between the tool and workpiece, and a grayscale image of the surface topography. Compared with methods that directly use a detection device to sample surface topography and methods that calculate surface topography based on analytical modeling, the present invention significantly reduces the data sampling time and image generation time. Compared with generative adversarial network methods, the training process of this method is more stable and less susceptible to training instability. It can generate high-quality, diverse samples that are adapted to real-world conditions, and is characterized by fast generation and high fidelity, offering great potential for real-time surface topography prediction.

[0023] Diffusion models are a broad mathematical model with widespread applications in probability theory, statistics, and related fields. Specifically, diffusion models describe the evolution or diffusion of a phenomenon from an initial state to another state over a period of time. Diffusion models can be considered a type of latent variable model, which attempts to learn the distribution of noise under a data distribution. They use Markov chains to gradually add noise to the data, learning the posterior probability distribution of the data in the process. Diffusion models can provide an effective solution when dealing with complex data distributions, such as multimodal information in a work.

[0024] 1 is a flowchart of a method for generating a surface topography of a workpiece based on generation of a multimodal image provided by an embodiment of the present invention. As shown in FIG. 1, the embodiment provides a method for generating a surface topography of a workpiece based on generation of a multimodal image, including the following steps:

[0025] S110 collects and marks surface topography images and processing signal spectral diagrams for different processing methods, obtains multimodal information, and processes the multimodal information to obtain guide vectors.

[0026] In this embodiment, taking the cutting process as an example, a 3D scanner is used to collect surface data of the actual machined part during cutting, the surface data including a surface topography image and a processing signal spectrum diagram, the surface topography image is projected into the image space and converted into a grayscale image of the surface topography, and used as the corresponding target data for monitoring the generated data.

[0027] The text information corresponding to the grayscale image of the surface topography is marked, and the text information includes the machining method, feed rate, workpiece material, tool geometry, and vibration between the tool and workpiece. The machining spectrum signal of the machining signal spectrum diagram is marked, and the text information and the machining spectrum signal are used as multimodal information. A text encoder and a spectrum signal encoder based on the contrastive language image pre-training model CLIP are used to convert the text information and the machining spectrum signal into a representation format and cascade it to generate an embedded feature vector E. T The obtained embedding feature vector is mapped into a joint space with the grayscale image of the surface topography, and a semantic relationship is established between the grayscale image of the surface topography and the multi-modal information. The embedding feature vector is used as a guiding vector to provide a condition for generating the grayscale image of the surface topography and guide the de-diffusion process in the noise vector of the noise estimation network of the diffusion model.

[0028] In the training stage, the accuracy of the model is ensured mainly by using historical machining parameters and the surface topography data of the part measured by a topography detector and a 3D scanner, and in the actual application stage, predictions are made based on the actual machining parameters.

[0029] S120: compressing the grayscale image corresponding to the surface topography image into a first low-dimensional representation via an encoder and inputting it into a diffusion model; adding noise to the first low-dimensional representation layer layer by layer through a forward diffusion process to obtain a noise vector; reducing noise layer by layer through a reverse diffusion process based on the guide vector, the time step and the noise vector; and then restoring a second low-dimensional representation to train the diffusion model.

[0030] The pre-training variational autoencoder includes two parts: an encoder and a decoder. The encoder is used to compress the grayscale image of the surface topography in S110 into a first low-dimensional representation Z0, which is then input to a diffusion model. The diffusion model specifically includes a forward diffusion process and an inverse diffusion process. The training phase includes a training forward diffusion process and an optimization inverse diffusion process. The trained inverse diffusion process is used in the actual application phase. In the training phase, based on a randomly generated two-dimensional Gaussian noise ε, the forward diffusion process adds noise to the input first low-dimensional representation Z0 layer by layer. After T time steps, the noise-added noise vector Z T In the de-diffusion process, we use a Unet noise estimation network based on a multi-head Attention (Q, K, V) structure, which is a cross-attention mechanism, and obtains the noise vector Z as shown in Figure 2. T , time step T and embedding feature vector E T is input to the Unet noise estimation network based on the multi-head Attention(Q,K,V) structure, and the estimated noise ε θ and obtain the estimated noise ε θ Compare with randomly generated two-dimensional Gaussian noise ε to estimate noise εθ A loss function between ε and a randomly generated two-dimensional Gaussian noise ε is constructed, and the loss function is expressed as follows:

[0031]

number

[0032]

number

[0033] S130, during application, extract target multi-modal information to construct a target guide vector, input the randomly generated Gaussian noise latent variables, time steps and the target guide vector into the trained diffusion model, obtain a target low-dimensional representation through a reverse diffusion process, and pass the target low-dimensional representation through a decoder to obtain a grayscale image of the target surface topography.

[0034] In practical application, target multimodal information, including text information (cutting process, feed rate, workpiece material, tool geometry, and tool-workpiece vibration) and processing spectrum signals, are extracted. Then, the text information and processing spectrum signals are converted into representation forms via a text encoder and a spectrum signal encoder, respectively, and the resulting embedded representation vector is used as the target guide vector. A randomly generated two-dimensional Gaussian noise signal is converted into a representation vector in the latent space. Specifically, a 64x64 latent variable is obtained by convolving the two-dimensional Gaussian noise signal. The latent variable and target guide vector are then input into a trained diffusion model, and a de-diffusion process is performed to obtain a 64x64 conditional latent variable. After multiple noise removal using an optimized iterative algorithm, a target low-dimensional representation is obtained. The target low-dimensional representation is then transmitted to the decoder part of the pre-trained variational autoencoder. The decoder then restores the implicit target low-dimensional representation into image information, generating a grayscale image of the target surface topography, as shown in Figure 3.

[0035] S140, the grayscale image of the target surface topography is input to the image quality comprehensive evaluation module to evaluate the fidelity of the grayscale image of the target surface topography.

[0036] In this embodiment, a pre-trained variational autoencoder decoder is used to restore the target low-dimensional representation into grayscale images of three different viewpoints of the grayscale image of the target surface topography, and the generated multi-view grayscale images are input into an image quality comprehensive evaluation module to perform feature extraction and distortion evaluation, and evaluate whether the generated grayscale image of the target surface topography meets the requirements of semantic content and fidelity.

[0037] As shown in Figure 4, the image quality comprehensive evaluation module includes a high-dimensional semantic feature extractor and a low-dimensional deformation feature extractor. The high-dimensional semantic feature extractor uses the last four layers of a pre-trained EfficientNetV2 network to extract high-dimensional features, which are used as high-dimensional semantic distortion features, including semantic features such as content information, physical properties, and spatiotemporal relationships between content. The low-dimensional deformation feature extractor uses the first four layers of a pre-trained VGG16 network to extract low-dimensional features, which are used as low-dimensional deformation distortion features, which include compression, noise, blur, overexposure or underexposure, chromatic aberration, sharpness, and blocking artifacts. After feature fusion of the high-dimensional semantic distortion features and the low-dimensional deformation distortion features, they are input into an image distortion quality regression model consisting of three fully connected layers to obtain the corresponding quality score. The aforementioned image distortion quality regression model evaluates the quality of the generated image and uses a combination of subjective and objective evaluation metrics to use high-dimensional features (such as physical features of objects, spatiotemporal relationships between objects, and content information of objects) and low-dimensional features (such as compression, noise, blur, overexposure or underexposure, chromatic aberration, sharpness, and blocking artifacts) as distortion indices. Taking distortion processing of high-quality images obtained from a real scene as an example, the distortion index of high-quality images is set to 0, and various degrees of distortion processing are performed on these high-quality images, scoring them between 0 and 1. Since a score closer to 1 indicates a higher degree of distortion, the grayscale image of surface topography generated by the method of the present invention, after a series of processing steps, will have less distortion and higher fidelity as the quality score obtained by logistic regression of the image distortion quality regression model decreases, and higher quality indicates higher fidelity.

[0038] Based on the same inventive concept, as shown in FIG. 5, an embodiment of the present invention: a guiding vector construction unit 510 for collecting and marking surface topography images and processing signal spectrum diagrams of different processing methods, obtaining multi-modal information, and processing the multi-modal information to obtain a guiding vector; a model training unit 520 for compressing a grayscale image corresponding to the surface topography image into a first low-dimensional representation through an encoder and inputting the compressed grayscale image into a diffusion model, adding noise to the first low-dimensional representation layer by layer through a forward diffusion process to obtain a noise vector, reducing noise by layer through a reverse diffusion process based on the guide vector, the time step and the noise vector, and then restoring a second low-dimensional representation to train the diffusion model; a model application unit 530 for extracting target multi-modal information during application to construct a target guide vector, inputting the randomly generated Gaussian noise latent variables, time steps and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and passing the target low-dimensional representation through a decoder to obtain a grayscale image of the target surface topography; The present invention further provides an apparatus 500 for generating a surface topography of a workpiece based on the generation of a multimodal image, the apparatus including: a quality evaluation unit 540 for inputting the grayscale image of the target surface topography into an image quality comprehensive evaluation module to evaluate the fidelity of the grayscale image of the target surface topography.

[0039] The apparatus for generating a surface topography of a workpiece based on the generation of a multimodal image provided by the embodiments of the present invention basically corresponds to the embodiment of the method, so please refer to the partial description of the embodiment of the method for related details. The above-described embodiment of the apparatus is merely illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. To achieve the objectives of the solution of the present invention, some or all of the modules can be selected according to actual needs. Those skilled in the art can understand and implement it without any creative effort.

[0040] Based on the same inventive concept, as shown in Figure 6, this embodiment further provides a workpiece surface topography generation apparatus based on the generation of multimodal images, the apparatus comprising memory for storing a computer program and a processor, and when the processor executes the computer program, it realizes the above-described method for generating a workpiece surface topography based on the generation of multimodal images.

[0041] The workpiece surface topography generation equipment based on the generation of multimodal images, as proposed in the embodiments of the present invention, may be equipment such as a computer. The embodiment of the equipment may be implemented by software, by hardware, or by a combination of software and hardware. Taking software implementation as an example, it is formed by reading corresponding computer program instructions in non-volatile memory into memory through the processor of any equipment having data processing capabilities and executing them. From a hardware level perspective, Figure 6 is a schematic diagram of the workpiece surface topography generation equipment based on the generation of multimodal images, as provided in the embodiments of the present invention. In addition to the processor, memory, network interface, and non-volatile memory shown in Figure 6, the workpiece surface topography generation equipment based on the generation of multimodal images, as provided in the embodiments of the present invention, may also include other hardware depending on the actual functionality of any equipment with data processing capabilities, which will not be described again.

[0042] Based on a similar inventive concept, this embodiment further provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed on a computer, it performs a method for generating a workpiece surface topography based on the generation of the multimodal image described above.

[0043] The computer-readable storage medium may be an internal storage unit of any equipment equipped with data processing functions, such as a hard disk or memory, as described in any of the embodiments above. The computer-readable storage medium may also be an external storage device of a wind turbine, such as a plug-in hard disk, Smart Media® Card (SMC), SD card, or Flash Card installed in the equipment. Furthermore, the computer-readable storage medium may include both internal storage units and external storage devices of any equipment equipped with data processing functions. The computer-readable storage medium can be used to store the computer program and other programs and data required by any equipment equipped with data processing functions, and can also be used to temporarily store output or output data.

[0044] Furthermore, the apparatus for generating surface topography of a workpiece based on the generation of multimodal images, the equipment for generating surface topography of a workpiece based on the generation of multimodal images, and the computer-readable storage medium provided in the above embodiment all belong to the same concept as the embodiment of the method for generating surface topography of a workpiece based on the generation of multimodal images, and their specific implementation process can be found by referring to the embodiment of the method for generating surface topography of a workpiece based on the generation of multimodal images, but will not be explained again here.

[0045] The above are merely preferred embodiments of the present invention and do not impose any formal limitations on the invention. Although the process of carrying out the present invention has been described in detail above, those skilled in the art can modify the technical solutions recorded in the above examples or make equivalent substitutions for some of the technical features. All modifications, equivalent substitutions, etc., made within the spirit and principles of the present invention are within the scope of protection of the present invention.

Claims

1. 1. A method for generating a surface topography of a workpiece based on generating a multimodal image, comprising: Step 1: collecting and marking surface topography images and processing signal spectrum diagrams according to different processing methods to obtain multi-modal information, and processing the multi-modal information to obtain a guiding vector; Step 2: compressing a grayscale image corresponding to the surface topography image into a first low-dimensional representation through an encoder and inputting the first low-dimensional representation into a diffusion model; adding noise to the first low-dimensional representation layer layer by layer through a forward diffusion process to obtain a noise vector; reducing noise layer by layer through a reverse diffusion process based on the guide vector, the time step, and the noise vector; and then restoring a second low-dimensional representation to train the diffusion model; Step 3: extracting target multi-modal information during application to construct a target guide vector, inputting the randomly generated Gaussian noise latent variables, time steps and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and passing the target low-dimensional representation through a decoder to obtain a grayscale image of the target surface topography; and (4) inputting the grayscale image of the target surface topography into an image quality comprehensive evaluation module to evaluate the fidelity of the grayscale image of the target surface topography; The step of marking the surface topography image and the processed signal spectrum diagram to obtain multimodal information includes: marking the surface topography image to obtain corresponding text information, the text information including a processing method, a feed rate, a workpiece material, a tool geometry, and a vibration between the tool and the workpiece; marking the processed signal spectrum diagram to obtain a corresponding processed spectral signal; 10. The method of claim 9, wherein the multimodal information includes text information and a processed spectral signal.

2. The step of processing the multimodal information to obtain a guidance vector includes: The method for generating a surface topography of a workpiece based on the generation of a multimodal image, as described in claim 1, further comprising the steps of converting the text information and the processed spectral signal into an expression format by a text encoder and a spectral signal encoder, respectively, in a cascade manner, and using the obtained embedding feature vector as a guide vector, wherein the text encoder and the spectral signal encoder use a contrastive language image pre-training model CLIP.

3. The method for generating a surface topography of a workpiece based on generation of a multimodal image according to claim 1 , wherein the encoder and the decoder constitute a variational autoencoder.

4. The method for generating a surface topography of a workpiece based on generation of a multimodal image as described in claim 1, characterized in that the de-diffusion process uses a Unet noise estimation network based on a cross-attention mechanism, the Unet noise estimation network is used to generate estimated noise, and the estimated noise is used to reduce noise at each time step.

5. The image quality comprehensive evaluation module includes a high-dimensional semantic feature extractor, a low-dimensional transformation feature extractor, and a regression model; The method for generating a surface topography of a workpiece based on the generation of a multimodal image, as described in claim 1, characterized in that semantic features and deformation features are extracted from the grayscale image of the target surface topography through a high-dimensional semantic feature extractor and a low-dimensional deformation feature extractor, respectively, and the semantic features and deformation features are input into a regression model after feature fusion, and a quality score for evaluating the fidelity of the grayscale image of the target surface topography is predicted through logistic regression of the regression model.

6. The method for generating a surface topography of a workpiece based on generation of a multimodal image according to claim 5, wherein the high-dimensional semantic feature extractor includes a pre-trained EfficientNetV2 network, and the low-dimensional deformation feature extractor includes a pre-trained VGG16 network.

7. a guide vector construction unit for collecting and marking surface topography images and processing signal spectrum diagrams of different processing methods, obtaining multi-modal information, and processing the multi-modal information to obtain a guide vector; a model training unit for compressing a grayscale image corresponding to the surface topography image into a first low-dimensional representation through an encoder, inputting the first low-dimensional representation into a diffusion model, adding noise to the first low-dimensional representation layer layer by layer through a forward diffusion process to obtain a noise vector, reducing noise layer by layer through a reverse diffusion process based on the guide vector, the time step and the noise vector, and then restoring a second low-dimensional representation to train the diffusion model; a model application unit for extracting target multi-modal information during application to construct a target guide vector, inputting the randomly generated Gaussian noise latent variables, time steps and the target guide vector into a trained diffusion model, obtaining a target low-dimensional representation through a reverse diffusion process, and passing the target low-dimensional representation through a decoder to obtain a grayscale image of the target surface topography; a quality evaluation unit for inputting the grayscale image of the target surface topography into an image quality comprehensive evaluation module to evaluate the fidelity of the grayscale image of the target surface topography; The step of marking the surface topography image and the processed signal spectrum diagram to obtain multimodal information includes: marking the surface topography image to obtain corresponding text information, the text information including a processing method, a feed rate, a workpiece material, a tool geometry, and a vibration between the tool and the workpiece; marking the processed signal spectrum diagram to obtain a corresponding processed spectral signal; An apparatus for generating a surface topography of a workpiece based on the generation of a multimodal image, characterized in that the multimodal information includes text information and processed spectral signals.

8. An apparatus for generating a surface topography of a workpiece based on the generation of a multimodal image, the apparatus comprising: a memory for storing a computer program; and a processor, wherein, when the processor executes the computer program, the apparatus realizes the method for generating a surface topography of a workpiece based on the generation of a multimodal image according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, the computer program realizing, when executed by a computer, a method for generating a surface topography of a workpiece based on generation of a multimodal image according to any one of claims 1 to 6.

Images