Industrial multi-modal data extraction method and system based on large language model

Abstract

The invention discloses an industrial multi-modal data extraction method and system based on a large language model, and relates to the technical field of large language models.The industrial multi-modal data extraction method comprises the steps that industrial multi-modal data are collected, high-dimensional features are extracted through a convolutional neural network, and unified feature vectors are formed through splicing after normalization; determining a first retention ratio search range by using the PCA accumulated variance and the reconstruction error elbow point, and generating an initial retention ratio in combination with the semantic information density; taking the proportion as a center, generating a plurality of candidate retention proportions by adopting Gaussian perturbation, performing semantic compression on each proportion, extracting multi-dimensional features, and calculating a compression score; and mapping the proportion and the score into a screening vector, forming a continuous scoring surface through two-dimensional mapping and interpolation, analyzing a scoring extreme value, screening out an optimal retention proportion, and finally applying the optimal retention proportion to industrial semantic compression. The problems that in existing industrial multi-modal data semantic compression, the retention ratio is too high, so that real-time performance is reduced, and the retention ratio is too low, so that semantics are lost are solved.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and more specifically, to a method and system for extracting industrial multimodal data based on large language models. Background Technology

[0002] With the intelligent upgrading of industrial scenarios, an increasing number of applications are adopting industrial multimodal data extraction methods based on large language models. By unifying the feature representations of multimodal data such as images, signals, and text, the multimodal feature vectors are input into a large language model for semantic compression, generating low-dimensional semantic embeddings for downstream applications such as condition recognition, anomaly detection, and decision control. This type of method significantly improves the expressive power of industrial multimodal data, enabling better fusion of semantic information between different modalities, thereby improving the accuracy and response speed of complex condition recognition. It has become an important tool for industrial intelligent edge computing.

[0003] In industrial multimodal data extraction methods, the semantic compression retention ratio is a core parameter affecting the quality of the final embedding. The retention ratio determines the amount of information retained when mapping from a high-dimensional feature space to a low-dimensional space, impacting both the accuracy of feature representation and the consumption of computational resources. Setting the retention ratio too high results in a larger dimensionality of the generated embedding, consuming more storage space and increasing the computational latency of edge devices; while setting it too low leads to missing semantic information, reducing the ability to distinguish complex working conditions and causing a decline in the performance of downstream tasks. Therefore, reasonably determining the retention ratio is crucial for ensuring the real-time performance and accuracy of the system.

[0004] However, both excessively high and excessively low retention ratios in industrial semantic compression have significant drawbacks, negatively impacting the extraction results of industrial multimodal data. When the retention ratio is too low, useful features in the embedding are severely discarded, resulting in incomplete semantic information and preventing large language models from correctly understanding subtle differences between different operating conditions. When the retention ratio is too high, although semantic information is preserved more completely, the computational burden increases significantly, reducing the system's real-time processing capabilities and potentially causing delayed alarms or even missing critical control opportunities.

[0005] Specifically, when the retention rate of industrial semantic compression is too low, the extracted embedding vectors cannot effectively cover the key information in the original multimodal data. This leads to a decrease in the model's recognition rate for complex, abnormal, or sparse industrial conditions, and a significant increase in false alarm and false negative rates. This is extremely disadvantageous in industrial scenarios that require high-precision monitoring and rapid fault response, and can easily cause production accidents or equipment damage.

[0006] When the retention rate of industrial semantic compression is too high, although the semantic expression of the embedding vector is more comprehensive, edge computing devices need to process data of more dimensions, resulting in a significant increase in computational load and memory consumption. This severely impacts real-time performance, causing time delays in alarms, control commands, or operational adjustments. In high-speed industrial production lines, this delay may lead to missing the optimal control window, causing fluctuations in product quality or increased energy consumption.

[0007] In existing technologies, the selection of the retention ratio typically relies on experience or fixed hyperparameter settings, lacking dynamic adjustment strategies for different data distributions and operating conditions, and thus failing to balance the dual requirements of semantic fidelity and real-time computation. This lack of adaptive control makes industrial multimodal data extraction methods based on large language models prone to computational latency or decreased recognition accuracy, limiting their large-scale application in industrial scenarios with high real-time and reliability requirements.

[0008] To address the above problems, this invention proposes a solution. Summary of the Invention

[0009] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide an industrial multimodal data extraction method and system based on a large language model. By limiting the search range based on PCA and reconstruction error elbows and generating an initial retention ratio in combination with semantic information density, an enumeration evaluation and screening is performed to solve the problems in existing industrial multimodal data semantic compression where an excessively high retention ratio leads to decreased real-time performance and an excessively low retention ratio leads to semantic loss.

[0010] To achieve the above objectives, the present invention provides the following technical solution: The method for extracting industrial multimodal data based on a large language model includes the following steps: collecting industrial multimodal data, extracting high-dimensional features based on a convolutional neural network, and normalizing and concatenating them into industrial multimodal feature vectors in a unified feature space; obtaining the search range of the first retention ratio for industrial semantic compression based on PCA cumulative variance and reconstruction error elbows in parallel constraints; generating the first retention ratio based on the semantic information density of the industrial multimodal data in the coding space of the large language model; generating several second retention ratios within the search range of the first retention ratio centered on the first retention ratio, based on the Gaussian perturbation generation method; compressing the industrial multimodal feature vectors by iterating through the second retention ratios one by one, and extracting multidimensional features from the compression process and structure, performing weighted scoring based on the multidimensional features, and generating compression score values; mapping the second retention ratios to the corresponding compression score values ​​and forming a selection vector; performing two-dimensional mapping and interpolation to complete the several selection vectors into a two-dimensional image, and performing data analysis on the two-dimensional image to select the best second retention ratio as the third retention ratio for semantic compression.

[0011] In a preferred embodiment, the process of acquiring industrial multimodal data, extracting high-dimensional features based on a convolutional neural network, and normalizing and concatenating them into an industrial multimodal feature vector with a unified feature space specifically involves: acquiring multimodal data and dividing and preprocessing image and signal data; extracting multi-branch features from the preprocessed multimodal data based on a preset multi-branch convolutional neural network; generating high-dimensional industrial multimodal feature vectors; applying min-max normalization to the high-dimensional industrial multimodal feature vectors; and concatenating the normalized modal feature vectors in a fixed order to form a unified industrial multimodal feature vector.

[0012] In a preferred embodiment, the step of obtaining the first retention ratio search range for industrial semantic compression based on PCA cumulative variance and reconstruction error elbows through parallel constraints specifically involves: constructing an industrial multimodal feature matrix; generating a discrete candidate dimension set based on the feature dimensions of the industrial multimodal feature matrix, covering a uniform distribution range from low-dimensional compression to high-dimensional near-original features; performing PCA decomposition on the feature matrix to obtain the eigenvalues ​​of each principal component; calculating the cumulative explained variance of each candidate dimension based on the eigenvalues ​​of each principal component; traversing to find the smallest dimension whose cumulative explained variance is greater than a preset first threshold, and obtaining the lower bound ratio by the ratio of the feature dimension to the smallest dimension; reconstructing the industrial multimodal feature matrix for each candidate dimension using a lightweight autoencoder, calculating the reconstruction error, and plotting the curve of reconstruction error versus candidate dimension; using an elbow detection algorithm to find the elbows of the curve, and using the ratio of the elbows to the feature dimensions as the upper bound ratio; and combining the upper bound ratio and the lower bound ratio to obtain the first retention ratio search range.

[0013] In a preferred embodiment, generating a first retention ratio based on the semantic information density of industrial multimodal data in the large language model encoding space specifically involves: inputting the industrial multimodal data into a pre-trained large language model encoder, mapping each data point to a semantic vector to unify the data representation of different modalities; quantifying the information content carried by each semantic vector in the semantic space based on Shannon entropy; averaging the single semantic information content of all industrial multimodal data points to obtain the average semantic entropy of the entire dataset in the large language model encoding space; and mapping the average semantic entropy to an initial retention ratio for industrial semantic compression using a nonlinear mapping method, with the generated initial retention ratio serving as the first retention ratio.

[0014] In a preferred embodiment, the step of generating several second retention ratios within the search range of the first retention ratio using a Gaussian perturbation generation method, centered on the first retention ratio, specifically involves: using the first retention ratio as the center value; using the search range of the first retention ratio as the boundary condition for the Gaussian perturbation; constructing a Gaussian distribution with the first retention ratio as the mean at the center position of the first retention ratio, and setting a standard deviation parameter to control the dispersion of the generated ratios; randomly sampling several values ​​from the probability density function according to the Gaussian distribution to generate a corresponding number of second retention ratios, where the second retention ratios represent possible configurations for one industrial semantic compression; performing boundary verification on the generated second retention ratios, removing values ​​that exceed the search range of the first retention ratio or do not meet reasonable constraints; and outputting the set of second retention ratios that have undergone boundary verification.

[0015] In a preferred embodiment, the step of sequentially traversing the second retention ratio to compress the industrial multimodal feature vector, and extracting multidimensional features from the compression process and structure, specifically involves: using several second retention ratios one by one as compression intensity parameters for industrial semantic compression, and processing the industrial multimodal feature vectors in a unified feature space; for each second retention ratio, inputting the original multimodal feature vector into the encoding module of a large language model for compression to generate a corresponding low-dimensional embedding vector, thus completing the semantic compression process; extracting multidimensional features from the embedding vector generated by each compression, including dimensionality redundancy features, information loss features, distribution sparsity features, and reconstruction consistency features; and combining and weighting the multidimensional features according to preset weights to output a compression score.

[0016] In a preferred embodiment, the step of performing two-dimensional mapping and interpolation to complete the two-dimensional image of several screening vectors specifically involves: arranging several screening vectors in order of a second retention ratio, with each vector containing a compressed score value and multi-dimensional feature quantization result corresponding to that ratio, forming a two-dimensional data matrix, where rows represent different second retention ratios and columns represent feature scores for each dimension; mapping each screening vector onto a two-dimensional plane using the second retention ratio as the horizontal axis and the corresponding compressed score value or multi-dimensional feature comprehensive index as the vertical axis, forming a discrete scatter set; performing interpolation calculation on the discrete scatter set based on bilinear interpolation to generate a continuous two-dimensional score surface, used to fill in the gaps between the original discrete screening vectors; mapping the continuous score surface obtained after interpolation into a color image, where each pixel position corresponds to a specific second retention ratio and compressed score value, and the image color represents the magnitude of the compressed score, thus obtaining a two-dimensional image.

[0017] In a preferred embodiment, data analysis is performed on the two-dimensional image to select the optimal second retention ratio as the third retention ratio for semantic compression. Specifically, the compression score value corresponding to each pixel in the two-dimensional image is scanned to identify all local minima. The identified local minima are spatially clustered, and continuous low-score regions are divided into independent candidate regions based on the compression score value trend of adjacent pixels. Each region corresponds to a set of closely spaced second retention ratios. The pixel compression score values ​​in each candidate region are statistically analyzed to calculate the region's average score, score variance, and minimum score, and to evaluate the overall compression effect and stability of the region. The region with the lowest average score and the smallest score variance is preferentially selected. Within the preferred region, the second retention ratio corresponding to the pixel with the smallest compression score value is selected as the final third retention ratio for actual industrial semantic compression operations.

[0018] This system for extracting industrial multimodal data based on a large language model includes a data acquisition module, a retention ratio search range constraint module, a retention ratio initialization module, an enumeration data acquisition module, a compression evaluation module, and a screening application module. The data acquisition module collects industrial multimodal data, extracts high-dimensional features using a convolutional neural network, and normalizes and concatenates them into industrial multimodal feature vectors in a unified feature space. The retention ratio search range constraint module uses PCA cumulative variance and reconstruction error elbows to obtain the first retention ratio search range for industrial semantic compression in parallel. The retention ratio initialization module generates the first retention ratio based on the semantic information density of the industrial multimodal data in the large language model encoding space. The system comprises the following modules: a proportion module; an enumeration data acquisition module, used to generate several second retention proportions within the search range of the first retention proportion, centered on the first retention proportion, based on the Gaussian perturbation generation method; a compression evaluation module, used to traverse the second retention proportions one by one to compress the industrial multimodal feature vectors, and to extract multidimensional features from the compression process and structure, and to perform weighted scoring based on the multidimensional features to generate a compression score value; mapping the second retention proportions to the corresponding compression score values ​​to form a screening vector; and a screening application module, used to perform two-dimensional mapping and interpolation to complete the several screening vectors into a two-dimensional image, and to perform data analysis on the two-dimensional image to select the best second retention proportion as the third retention proportion for semantic compression.

[0019] The technical effects and advantages of the industrial multimodal data extraction method and system based on a large language model in this invention are as follows: 1. This invention obtains a reasonable search range for the first retention ratio by employing dual constraints based on the cumulative variance of PCA and the reconstruction error elbow. Furthermore, it generates an initial retention ratio based on the semantic information density of industrial multimodal data within the large language model coding space, making the initial search point for the compression ratio closer to the optimal value. This approach avoids the high computational overhead of blindly traversing the entire dimensional space, significantly narrowing the search interval for the compression ratio and improving the convergence speed of subsequent compression and scoring calculations. For edge computing scenarios, this method reduces the number of model tuning iterations, lowers computational resource consumption, and ensures rapid convergence to a suitable retention ratio in real-time industrial scenarios, thereby improving overall decision-making efficiency.

[0020] 2. This invention generates multiple second retention ratios within a search range, centered on a first retention ratio, and calculates scores for each ratio by compressing them sequentially. Combining two-dimensional visualization and spatial clustering analysis, it can automatically identify locally optimal and stable compression ratio regions and select the ratio with the best compression score as the third retention ratio. This scheme balances semantic information fidelity with the compression of the embedding vector dimension, avoiding edge inference delays caused by excessively high retention ratios and preventing semantic loss and misclassification problems caused by excessively low retention ratios. Ultimately, it can reduce feature dimensions while maintaining model recognition accuracy, enabling more timely responses from downstream tasks and ensuring real-time control and safe operation in critical industrial scenarios. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the industrial multimodal data extraction method based on a large language model according to the present invention.

[0022] Figure 2 This is a schematic diagram of the structure of the industrial multimodal data extraction system based on a large language model according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1, Figure 1 This invention presents a method for extracting industrial multimodal data based on a large language model, comprising: S1 collects industrial multimodal data, extracts high-dimensional features based on convolutional neural networks, and normalizes and concatenates them into industrial multimodal feature vectors in a unified feature space.

[0025] In this embodiment, the process of collecting industrial multimodal data, extracting high-dimensional features based on a convolutional neural network, and normalizing and concatenating them into an industrial multimodal feature vector with a unified feature space specifically involves: Multimodal data acquisition, and segmentation and preprocessing of image and signal data; Multi-branch feature extraction is performed on the preprocessed multimodal data based on a pre-defined multi-branch convolutional neural network; generating high-dimensional feature vectors for industrial multimodal data. Minimum-maximum normalization is applied to the high-dimensional feature vectors of industrial multimodal systems. The normalized modal feature vectors are concatenated in a fixed order to form a unified industrial multimodal feature vector.

[0026] The multi-branch structure includes an image branch, a thermal imaging branch, and a vibration signal branch. It should be noted that the image branch uses convolutional layers + batch normalization + activation functions + pooling layers to extract spatial features. The thermal imaging branch uses shallow convolution + a thermal image feature extraction module to extract temperature distribution features. The vibration signal branch uses 1D convolution + pooling + residual connections to extract frequency and time domain features. Furthermore, the high-dimensional feature vectors output from each branch are preserved, forming an independent high-dimensional representation for each modality.

[0027] It should be noted that image data preprocessing involves performing grayscale normalization, uniform scaling, noise filtering, and contrast enhancement on the acquired industrial equipment images and infrared images to generate standardized image input data, thereby ensuring the consistency and stability of the input to the convolutional neural network.

[0028] It should be noted that the preprocessing of signal data involves performing sampling frequency unification, filtering and noise reduction (such as bandpass filtering to remove interference from non-target frequency bands) and normalization on time series signals such as vibration, temperature and pressure. The signal is then divided into sample segments according to a fixed time window to ensure that the length of each segment is consistent, which facilitates subsequent feature extraction.

[0029] In this embodiment, multimodal data acquisition refers to the synchronous acquisition of data output from various sensors and devices in the industrial field, including high-resolution images, infrared images, and video streams acquired by image sensors, as well as various signal data such as pressure, temperature, current, and voltage. This step not only acquires raw information but also considers data temporal consistency, sampling frequency matching, and data alignment between different modalities, providing a reliable foundation for subsequent feature extraction and multimodal fusion. Multimodal acquisition ensures the integrity of different physical quantities and visual information in the industrial system, providing sufficient data support for semantic understanding in a large language model coding space.

[0030] In this embodiment, image and signal data are divided and preprocessed. This refers to classifying and processing the acquired multimodal data according to its type to suit the respective feature extraction methods. For image data, preprocessing includes steps such as denoising, resolution unification, and color or grayscale normalization to ensure that the convolutional neural network can stably extract spatial features. For signal data, preprocessing includes operations such as filtering, normalization, sampling alignment, and feature enhancement to ensure that temporal information and key patterns are accurately preserved. This step ensures the availability and consistency of data from different modalities before they are input into the multi-branch network.

[0031] In this embodiment, a pre-designed multi-branch convolutional neural network is used to extract features from pre-processed multimodal data. This involves inputting different modalities of data into the designed multi-branch convolutional neural network for feature extraction. Each branch network is designed with different convolutional layers, pooling layers, and activation functions based on the characteristics of the data modality to extract modality-specific spatial or temporal patterns while maintaining feature independence between modalities. Through multi-branch design, the spatial texture features of images, the temporal series features of signals, and the potential complementary information of each modality can be fully explored, providing a high-dimensional semantic representation for subsequent feature fusion.

[0032] In this embodiment, generating high-dimensional feature vectors for industrial multimodal applications involves encoding the features of each modality extracted by a multi-branch convolutional neural network into a unified representation using high-dimensional encoding. This feature vector contains key information and latent semantic patterns for each modality, capable of describing the complex states and implicit rules of the industrial environment. The high-dimensional feature vectors not only preserve the unique patterns of each modality but also retain the potential relationships between modalities, providing a rich foundation of raw information for subsequent semantic understanding, compression, and analysis by the large language model.

[0033] In this embodiment, the high-dimensional feature vectors of industrial multimodal applications employ min-max normalization. This involves linearly mapping each dimension of the high-dimensional feature vector according to its minimum and maximum values, unifying its value range to a fixed interval. This normalization operation not only eliminates the differences in the magnitude of features across different modalities but also ensures a more balanced contribution of each dimension in subsequent feature concatenation and industrial semantic compression. This prevents certain features from dominating the entire representation due to excessively large absolute values, while simultaneously enhancing the model's robustness to outliers and noise.

[0034] In this embodiment, the normalized modal feature vectors are concatenated in a fixed order to form a unified industrial multimodal feature vector. This means connecting the processed image modality, signal modality, and other branch feature vectors into a single vector according to a predefined order. This concatenation operation not only preserves the integrity and order information of each modal feature but also ensures that the entire industrial multimodal feature vector can be understood and processed holistically during subsequent semantic compression, scoring, and mapping processes, thereby achieving collaborative representation and efficient fusion of multimodal data. S2, based on PCA cumulative variance and reconstruction error elbow, obtains the first retention ratio search range for industrial semantic compression through parallel constraints.

[0035] In this embodiment, the search range for the first retention ratio of industrial semantic compression, obtained by parallel constraints based on PCA cumulative variance and reconstruction error elbow, is specifically as follows: Construct an industrial multimodal feature matrix, and generate a discrete candidate dimension set based on the feature dimensions of the industrial multimodal feature matrix, covering a uniform distribution range from low-dimensional compression to high-dimensional near-original features; PCA decomposition is performed on the feature matrix to obtain the eigenvalues ​​of each principal component; Calculate the cumulative explained variance of each candidate dimension based on the eigenvalues ​​of each principal component, iterate through the dimensions to find the smallest dimension whose cumulative explained variance is greater than a preset first threshold, and obtain the lower bound ratio by the ratio of the feature dimension to the smallest dimension. The industrial multimodal feature matrix is ​​reconstructed for each candidate dimension using a lightweight autoencoder. The reconstruction error is calculated, and the curve of reconstruction error versus candidate dimension is plotted. The elbow detection algorithm is used to find the elbow of the curve, and the ratio of the elbow to the feature dimension is used as the upper bound ratio. The upper and lower bound ratios are combined to obtain the first retention ratio search range.

[0036] In this embodiment, the industrial multimodal feature matrix refers to a matrix structure that organizes the preprocessed and multi-branch convolutional neural network-extracted modal features according to sample and feature dimensions. Each row of the matrix represents a combination of multimodal features of an industrial sample, and each column corresponds to a feature value of a specific dimension. This matrix not only contains high-dimensional representations of various types of information such as image modalities and signal modalities, but also maintains the relative relationships between samples and features, providing a unified input basis for subsequent principal component analysis and feature compression.

[0037] In this embodiment, the discrete candidate dimension set refers to a set of optional compressed dimension values ​​generated based on the total feature dimensions of the industrial multimodal feature matrix, uniformly distributed from the minimum dimension to close to the original dimension. This set is used for enumeration testing during feature compression to evaluate the amount of semantic information and structural integrity preserved under different compression ratios. The design of the discrete candidate dimension set not only covers the global situation of the compression range but also ensures that the optimal compression interval can be found in subsequent PCA and autoencoder analyses.

[0038] In this embodiment, PCA decomposition and principal component eigenvalues ​​refer to performing principal component analysis on the industrial multimodal feature matrix, mapping the original high-dimensional features onto a set of mutually orthogonal principal components, and obtaining the eigenvalues ​​of each principal component. The principal component eigenvalues ​​reflect the contribution of the corresponding principal component to the variance of the original feature matrix, and can measure the amount of information in each direction. By analyzing the eigenvalue distribution, it is possible to identify how much key information is retained in different dimensions, thus providing a data-driven basis for determining the compression ratio.

[0039] In this embodiment, the cumulative explained variance and lower bound ratio refer to calculating the cumulative variance contribution of the principal components for each candidate dimension, finding the smallest feature dimension that can cover a preset information ratio, and then calculating the lower bound ratio by comparing it with the original dimension. This lower bound ratio reflects the minimum compression ratio allowed while ensuring the preservation of the most basic information. It avoids excessive compression that could result in the loss of key information and provides a theoretical lower limit for subsequent searches of the retention ratio.

[0040] In this embodiment, the lightweight autoencoder reconstruction error and elbow detection refer to using a lightweight autoencoder to compress and reconstruct feature matrices of different candidate dimensions. By comparing the errors of the input feature matrix and the reconstructed matrix, a curve showing the change of error with the candidate dimension is generated. An elbow detection algorithm is then used to find locations where the error descent rate significantly slows down. The ratio of the feature dimension corresponding to this elbow to the original dimension serves as an upper bound, reflecting the limited information gain that further increasing the dimension can bring, thus guiding the determination of the compression upper limit.

[0041] In this embodiment, the first retention ratio search range refers to combining the lower bound ratio obtained from the cumulative explained variance and the upper bound ratio obtained from the reconstruction error elbow point to form a continuous or discrete search interval. Enumerating or optimizing the industrial semantic compression retention ratio within this interval can effectively cover the balance area between sufficient information retention and compression efficiency. This search range ensures reasonable upper and lower bound constraints when generating the initial retention ratio and the derived retention ratio, improving the convergence speed and accuracy of enumeration and optimization.

[0042] S3 generates the first retention ratio based on the semantic information density of industrial multimodal data in the large language model coding space.

[0043] In this embodiment, the generation of the first retention ratio based on the semantic information density of industrial multimodal data in the large language model coding space specifically involves: Industrial multimodal data is input into a pre-trained large language model encoder, which maps each data point to a semantic vector, thus unifying the data representation of different modalities. The amount of information carried by each semantic vector in the semantic space is quantified based on Shannon entropy; The average semantic information content of each individual data point in all industrial multimodal data is taken to obtain the average semantic entropy of the entire dataset in the large language model coding space. By using a nonlinear mapping method, the average semantic entropy is mapped to the initial retention ratio of industrial semantic compression, and the generated initial retention ratio is used as the first retention ratio.

[0044] In this embodiment, the large language model encoder refers to a deep neural network model trained on a large-scale industrial corpus, used to map input industrial multimodal data to a unified high-dimensional semantic vector space. This encoder can understand the semantic relationships between different modalities and transform different types of data, such as image features and signal features, into comparable vector representations in the same semantic space, thereby providing a foundation for semantic compression and feature evaluation of industrial multimodal data.

[0045] In this embodiment, the semantic vector refers to a high-dimensional vector output by the large language model encoder, used to represent the semantic information of a single piece of industrial multimodal data. Each semantic vector contains the semantic features of the data in the encoding space, and the information of different modalities is uniformly represented as the feature values ​​of each dimension of the vector. The design of the semantic vector not only captures the explicit features of a single piece of data, but also implicitly includes the correlation and contextual information between modalities, providing a foundation for subsequent calculation of semantic information.

[0046] In this embodiment, Shannon entropy quantification of semantic information refers to using the concept of entropy in information theory to evaluate the amount of information carried by each semantic vector in the encoding space. By measuring the uncertainty or information distribution density of each dimension of the semantic vector, the effective information contained in a single data point can be quantified. This method can reveal the inherent semantic richness of the data, providing an objective basis for determining the proportion of information that should be retained during compression.

[0047] In this embodiment, the average semantic entropy refers to the statistical averaging of the semantic information content of individual data points across all industrial multimodal data, yielding the overall semantic information density of the entire dataset within the large language model's coding space. The average semantic entropy reflects the semantic complexity and information distribution characteristics of the entire industrial dataset and can be used as a reference standard for generating the initial compression ratio, ensuring the rationality of the first retention ratio in terms of overall information content.

[0048] In this embodiment, the nonlinear mapping method generates an initial retention ratio, which refers to mapping the average semantic entropy to the initial retention ratio space of industrial semantic compression through a nonlinear function. The nonlinear mapping can adjust the retention ratio according to the distribution characteristics of semantic entropy, resulting in a larger retention ratio for datasets with high semantic information and a smaller retention ratio for datasets with low semantic information, thereby achieving a more adaptive and customized compression effect. The generated initial retention ratio serves as the first retention ratio, providing a reasonable starting point for the derivation and enumeration of subsequent retention ratios.

[0049] S4, with the first retention ratio as the center, generates several second retention ratios within the search range of the first retention ratio based on the Gaussian perturbation generation method.

[0050] In this embodiment, the step of generating several second retention ratios within the search range of the first retention ratio based on the Gaussian perturbation generation method, centered on the first retention ratio, specifically involves: The first retention ratio is used as the center value; The search range of the first retention ratio is used as the boundary condition for the Gaussian perturbation; At the center of the first retention ratio, a Gaussian distribution with that value as the mean is constructed, and the standard deviation parameter is set to control the dispersion of the generated ratio; Based on the Gaussian distribution, several values ​​are randomly selected from the probability density function to generate a corresponding number of second retention ratios. The second retention ratios represent the possible configurations of an industrial semantic compression. Perform boundary checks on the generated second retention ratio and remove values ​​that exceed the search range of the first retention ratio or do not meet reasonable constraints; Output the second set of retention ratios after boundary verification.

[0051] In this embodiment, the first retention ratio refers to the initial industrial semantic compression ratio generated in step S3 based on the semantic information density of industrial multimodal data in the large language model coding space. This ratio represents the proportion of semantic information that should be retained when compressing industrial multimodal feature vectors relative to the original features. It is the central reference value for generating the subsequent second retention ratio, and its rationality directly affects the convergence speed and accuracy of the subsequent enumeration search.

[0052] In this embodiment, the first retention ratio search range refers to the feasible industrial semantic compression ratio range obtained in step S2 based on the cumulative variance of PCA and the elbow constraint of reconstruction error. This range defines the upper and lower boundaries of the second retention ratio generation, ensuring a balance between high information retention and low compression, and preventing the generated compression ratio from exceeding the semantic loss limit that the data can tolerate.

[0053] In this embodiment, the Gaussian perturbation generation method refers to a method that uses a first retention ratio as the center and randomly generates a second retention ratio by utilizing the probability characteristics of a Gaussian distribution. Gaussian perturbation can generate multiple retention ratio samples near the center value, so that the exploration of retention ratios covers the high-probability region near the center while retaining a certain degree of randomness, thereby increasing the diversity and robustness of the search.

[0054] In this embodiment, the Gaussian distribution mean and standard deviation refer to using the first retention ratio as the mean and controlling the dispersion of the generated ratio through the standard deviation. The mean ensures that the generated ratio is concentrated near the most reasonable initial value, while the standard deviation reflects the size of the search exploration range. The larger the standard deviation, the more dispersed the distribution of the generated ratio; the smaller the standard deviation, the more concentrated the generated ratio, ensuring the refinement of the enumeration process.

[0055] In this embodiment, boundary verification refers to constraining and checking the generated second retention ratio, eliminating values ​​that exceed the search range of the first retention ratio or do not meet the reasonableness requirements of industrial semantic compression. Boundary verification ensures that the generated second retention ratio is within an acceptable compression ratio range, while maintaining the semantic integrity of industrial multimodal features and avoiding over-compression or ineffective compression.

[0056] In this embodiment, the second set of retained proportions refers to several candidate proportions for industrial semantic compression obtained after Gaussian perturbation and boundary verification. This set serves as the input for subsequent enumeration and scoring. Each proportion corresponds to a potential compression configuration, and the distribution of proportions in the set directly determines the efficiency and accuracy of subsequent multi-dimensional feature scoring and optimal compression proportion selection.

[0057] S5, traverse the second retention ratio one by one to compress the industrial multimodal feature vector, and extract multidimensional features of the compression process and structure, perform weighted scoring based on multidimensional features, and generate compression score value; The second retention ratio is mapped to the corresponding compression score value, and a filtering vector is formed.

[0058] In this embodiment, the step of sequentially traversing the second retention ratio to compress the industrial multimodal feature vector, and performing multidimensional feature extraction on the compression process and structure, specifically involves: Several second retention ratios are used one by one as compression strength parameters for industrial semantic compression to process industrial multimodal feature vectors in a unified feature space. For each second retention ratio, the original multimodal feature vector is input into the encoding module of the large language model for compression to generate the corresponding low-dimensional embedding vector, thus completing the semantic compression process. For each compression-generated embedding vector, multi-dimensional features are extracted, including dimensional redundancy features, information loss features, distribution sparsity features, and reconstruction consistency features. Multidimensional features are combined and weighted according to preset weights to output a compressed score value.

[0059] It should be noted that the following is an example of a feasible arithmetic representation of dimensional redundancy features: ; In the formula, For dimensional redundancy features, For all sample vectors in the j-th dimension after compression, For variance, Let be the vector of all samples in the j-th dimension of the original feature, d be the dimension of the compressed feature, and D be the dimension of the original feature.

[0060] It should be noted that the following are examples of feasible arithmetic representations of information loss degree features: ; In the formula, The value of the original sample i in dimension j. The value of sample i in dimension j after feature compression reconstruction. For information loss degree features, For the sample size, For sample index, Indexed by feature dimensions.

[0061] It should be noted that the following are examples of feasible arithmetic representations of distribution sparsity features: ; In the formula, For the sparsity characteristics of the distribution, For indicator functions, if If it is not zero, it is 1; otherwise, it is 0.

[0062] It should be noted that the following are examples of feasible arithmetic representations of reconstructed consistency features: ; In the formula, To reconstruct the consistency feature, the numerator is the inner product of the original vector and the reconstructed vector, and the denominator is the product of the L2 norms of the original vector and the reconstructed vector. The closer the denominator is to 1, the more complete the semantics are preserved.

[0063] The second retention ratio in this embodiment refers to the candidate industrial semantic compression ratio obtained in step S4 based on the Gaussian perturbation generation method. Each second retention ratio corresponds to a compression intensity of the industrial multimodal feature vector, and is a basic parameter for subsequent compression scoring and screening, used to explore the semantic retention and multidimensional feature performance of the feature vector under different compression intensities.

[0064] In this embodiment, industrial semantic compression refers to the process of using a large language model encoding module to perform low-dimensional mapping on industrial multimodal feature vectors. Through semantic compression, the core semantic information of the original high-dimensional features is preserved in a low-dimensional representation, while reducing redundant and unnecessary feature dimensions, providing operable compressed results for subsequent feature analysis and scoring.

[0065] In this embodiment, the embedding vector refers to the low-dimensional feature representation after compression by the large language model encoding module. The embedding vector uniformly maps the original multimodal features in the semantic space, preserving the semantic relationships and information density between features. This allows subsequent multi-dimensional feature extraction to quantify the compression effect and facilitates comparison of performance differences under different compression ratios.

[0066] In this embodiment, the dimensionality redundancy feature refers to the feature that measures the repetition or highly relevant information present in the compressed embedding vector. Dimensional redundancy reflects the possibility of information duplication due to an excessively low compression ratio, thus indicating potential redundancy issues at a high retention ratio, and is one of the important criteria for compression scoring.

[0067] In this embodiment, the information loss rate feature refers to the feature that measures the amount of original semantic information lost during the compression process. The information loss rate reflects the risk of important semantics being discarded due to excessive compression ratio, helps quantify the information loss caused by low retention ratio, and thus provides an objective reference for scoring.

[0068] In this embodiment, the distribution sparsity feature refers to the feature describing the sparseness of the compressed embedding vector distribution in the semantic space. Distribution sparsity reflects the spatial expansion and clustering of compressed data points in the low-dimensional space, and can reveal the data distribution imbalance that may be caused by a high compression ratio. It plays an important role in the interpretability analysis of industrial semantic compression.

[0069] In this embodiment, the reconstruction consistency feature refers to the feature that measures the degree of consistency between the compressed embedding vector and the original feature vector after the decoding or reconstruction process. Reconstruction consistency reflects the ability to maintain the original semantic structure during compression and is an important indicator for quantifying compression effectiveness and evaluating the merits of different retention ratios.

[0070] In this embodiment, the compression score refers to a comprehensive score calculated by combining multiple features such as dimensional redundancy, information loss, distribution sparsity, and reconstruction consistency according to preset weights. The compression score is used to quantitatively evaluate the industrial semantic compression effect corresponding to each second retention ratio, providing a decision-making basis for subsequent selection of the optimal compression ratio.

[0071] S6. Several selected vectors are mapped in two dimensions and completed by interpolation to form a two-dimensional image. The two-dimensional image is then analyzed to select the optimal second retention ratio as the third retention ratio and perform semantic compression.

[0072] In this embodiment, the step of performing two-dimensional mapping and interpolation to complete the two-dimensional image of several filtering vectors specifically involves: Several selection vectors are arranged in order of the second retention ratio. Each vector contains the compressed score value and multi-dimensional feature quantization result corresponding to the ratio, forming a two-dimensional data matrix, where the rows represent different second retention ratios and the columns represent feature scores of each dimension. Using the second retention ratio as the horizontal axis and the corresponding compressed score or multi-dimensional feature comprehensive index as the vertical axis, each screening vector is mapped onto a two-dimensional plane to form a discrete scatter set. Interpolation calculations are performed on discrete scatter points based on bilinear interpolation to generate a continuous two-dimensional scoring surface, which is used to fill in the gaps between the original discrete screening vectors. The interpolated continuous score surface is mapped to a color image, with each pixel position corresponding to a specific second retention ratio and compressed score value. The image color represents the size of the compressed score, resulting in a two-dimensional image.

[0073] In this embodiment, the step of performing data analysis on the two-dimensional image, selecting the optimal second retention ratio as the third retention ratio, and performing semantic compression specifically involves: The compression score value corresponding to each pixel in the two-dimensional image is scanned to identify all local minima. The identified local minima are spatially clustered, and continuous low-score regions are divided into independent candidate regions based on the compression score trend of adjacent pixels. Each region corresponds to a set of close second retention ratios. Statistical analysis is performed on the pixel compression score values ​​of each candidate region to calculate the region's average score, score variance, and minimum score, thereby evaluating the overall compression effect and stability of the region. Prioritize selecting the region with the lowest average score and the smallest score variance. Within the preferred region, select the second retention ratio corresponding to the pixel with the smallest compression score as the final third retention ratio for actual industrial semantic compression operations.

[0074] In this embodiment, the selection vector refers to the vector formed by combining the compression score value corresponding to each second retention ratio with the multi-dimensional feature quantization result. Each selection vector not only records the compression effect but also reflects the performance of different feature dimensions, enabling comparison of industrial semantic compression performance under different retention ratios and providing basic data for subsequent two-dimensional mapping and scoring surface generation.

[0075] In this embodiment, the two-dimensional data matrix refers to a matrix structure formed by arranging several selection vectors in order of a second retention ratio. Rows represent different second retention ratios, and columns represent multi-dimensional feature scores. The two-dimensional data matrix can intuitively reflect the impact of different retention ratios on features of each dimension, providing an organized data foundation for interpolation and two-dimensional visualization.

[0076] In this embodiment, the discrete scatter set refers to the set of points formed by mapping each selection vector in the two-dimensional data matrix onto a plane, with the second retention ratio as the x-axis and the compressed score or multi-dimensional feature comprehensive index as the y-axis. The discrete scatter set reflects the distribution of the actual measured compression effect and is the key original information for generating continuous scoring surfaces and analyzing local extrema.

[0077] In this embodiment, bilinear interpolation refers to a method of spatial interpolating a discrete set of data points to generate a continuous two-dimensional scoring surface. Through bilinear interpolation, scoring information can be supplemented between the original discrete data, smoothing feature variation trends and ensuring the compression effect remains continuous in two-dimensional space, thus providing a reliable foundation for subsequent image analysis and local region selection.

[0078] In this embodiment, the two-dimensional scoring surface refers to a continuous two-dimensional plane obtained after interpolation, where each position corresponds to a specific second retention ratio and compression score value. The two-dimensional scoring surface can intuitively display the distribution of compression effects under different combinations of retention ratios, providing a visual basis for evaluating compression performance, identifying extreme value regions, and determining the optimal retention ratio.

[0079] In this embodiment, a local minimum point refers to a point in the two-dimensional scoring surface where the compression score is significantly lower than that of surrounding pixels. Local minimum points reflect the relatively optimal state of industrial semantic compression at a specific retention ratio and are an important reference for selecting the best compression ratio.

[0080] In this embodiment, spatial clustering refers to the process of dividing adjacent low-scoring points in a two-dimensional scoring surface into independent candidate regions based on spatial proximity and scoring trends. Spatial clustering can organize discrete low-scoring points into continuous regions, helping to analyze the stability of the compression effect and the overall performance of the regions, thereby avoiding the influence of single-point anomalies on the final retention ratio selection.

[0081] In this embodiment, the candidate region refers to a continuous low-score region obtained through spatial clustering, with each region containing a set of closely adjacent second retention ratios. The candidate region can reflect the overall trend of compression performance within a similar ratio range, facilitating statistical analysis and selection of the most stable compression parameters.

[0082] In this embodiment, the regional average score refers to the average compression score of all pixels within the candidate region, used to measure the overall compression effect of that region. The regional average score can determine whether a region has an overall low score, helping to identify the optimal compression configuration range under various retention ratios.

[0083] In this embodiment, the score variance refers to the dispersion of pixel compression score values ​​within a candidate region, used to evaluate the stability of the compression effect within that region. A smaller score variance indicates that different retention ratios within that region have a more consistent impact on the compression effect, which is beneficial for selecting a more reliable compression ratio in practical applications.

[0084] Example 2, Figure 2 This paper presents an industrial multimodal data extraction system based on a large language model, including a data acquisition module, a retention ratio search range constraint module, a retention ratio initialization module, an enumeration data acquisition module, a compression evaluation module, and a screening application module. The data acquisition module collects industrial multimodal data, extracts high-dimensional features based on a convolutional neural network, and normalizes and concatenates them into industrial multimodal feature vectors in a unified feature space. The retention ratio search range constraint module uses PCA cumulative variance and reconstruction error elbows to obtain the first retention ratio search range for industrial semantic compression in parallel constraints. The retention ratio initialization module generates the first retention ratio initialization module based on the semantic information density of the industrial multimodal data in the large language model encoding space. The system comprises the following modules: a retention ratio module; an enumeration data acquisition module, used to generate several second retention ratios within the search range of the first retention ratio, centered on the first retention ratio, based on the Gaussian perturbation generation method; a compression evaluation module, used to traverse the second retention ratios one by one to compress the industrial multimodal feature vectors, and to extract multidimensional features from the compression process and structure, and to perform weighted scoring based on the multidimensional features to generate a compression score value; mapping the second retention ratios to the corresponding compression score values ​​to form a selection vector; and a selection application module, used to perform two-dimensional mapping and interpolation to complete the several selection vectors into a two-dimensional image, and to perform data analysis on the two-dimensional image to select the best second retention ratio as the third retention ratio for semantic compression.

[0085] In the embodiments provided by this invention, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0086] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0087] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0088] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0089] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0091] In the embodiments provided in this disclosure, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0092] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0093] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for extracting industrial multimodal data based on a large language model, characterized in that, Includes the following steps: Collect industrial multimodal data, extract high-dimensional features based on convolutional neural networks, and normalize and concatenate them into industrial multimodal feature vectors with a unified feature space; Based on the cumulative variance of PCA and the reconstruction error elbow, parallel constraints are used to obtain the first retention ratio search range for industrial semantic compression. Based on the semantic information density of industrial multimodal data in the large language model coding space, a first retention ratio is generated; Centered on the first retention ratio, several second retention ratios are generated within the search range of the first retention ratio using the Gaussian perturbation generation method; The industrial multimodal feature vector is compressed by iterating through the second retention ratio one by one, and multidimensional features are extracted from the compression process and structure. Based on the multidimensional features, a weighted score is calculated to generate a compression score value. The second retention ratio is mapped to the corresponding compression score value, and a filtering vector is formed; Several selected vectors are mapped and interpolated to complete the two-dimensional image. The two-dimensional image is then analyzed to select the optimal second retention ratio as the third retention ratio for semantic compression.

2. The industrial multimodal data extraction method based on a large language model according to claim 1, characterized in that, The process involves collecting industrial multimodal data, extracting high-dimensional features based on convolutional neural networks, and then normalizing and concatenating these features into an industrial multimodal feature vector with a unified feature space. Specifically: Multimodal data acquisition, and segmentation and preprocessing of image and signal data; Multi-branch feature extraction is performed on the preprocessed multimodal data based on a pre-defined multi-branch convolutional neural network; generating high-dimensional feature vectors for industrial multimodal data. Minimum-maximum normalization is applied to the high-dimensional feature vectors of industrial multimodal systems. The normalized modal feature vectors are concatenated in a fixed order to form a unified industrial multimodal feature vector.

3. The method for extracting industrial multimodal data based on a large language model according to claim 2, characterized in that, The search range for the first retention ratio of industrial semantic compression, based on PCA cumulative variance and reconstruction error elbows and parallel constraints, is as follows: Construct an industrial multimodal feature matrix, and generate a discrete candidate dimension set based on the feature dimensions of the industrial multimodal feature matrix, covering a uniform distribution range from low-dimensional compression to high-dimensional near-original features; PCA decomposition is performed on the feature matrix to obtain the eigenvalues ​​of each principal component; Calculate the cumulative explained variance of each candidate dimension based on the eigenvalues ​​of each principal component, iterate through the smallest dimension whose cumulative explained variance is greater than a preset first threshold, and obtain the lower bound ratio by the ratio of the eigendimensional dimension to the smallest dimension. The industrial multimodal feature matrix is ​​reconstructed for each candidate dimension using a lightweight autoencoder. The reconstruction error is calculated, and the curve of reconstruction error versus candidate dimension is plotted. An elbow detection algorithm is used to find the elbow of the curve, and the ratio of the elbow to the feature dimension is used as the upper bound ratio. The upper and lower bound ratios are combined to obtain the first retention ratio search range.

4. The method for extracting industrial multimodal data based on a large language model according to claim 3, characterized in that, The first retention ratio is generated based on the semantic information density of industrial multimodal data in the large language model coding space, specifically as follows: Industrial multimodal data is input into a pre-trained large language model encoder, which maps each data point to a semantic vector, thus unifying the data representation of different modalities. The amount of information carried by each semantic vector in the semantic space is quantified based on Shannon entropy; The average semantic information content of each individual data point in all industrial multimodal data is taken to obtain the average semantic entropy of the entire dataset in the large language model coding space. By using a nonlinear mapping method, the average semantic entropy is mapped to the initial retention ratio of industrial semantic compression, and the generated initial retention ratio is used as the first retention ratio.

5. The industrial multimodal data extraction method based on a large language model according to claim 4, characterized in that, The process of generating several second retention ratios within the search range of the first retention ratio, centered on the first retention ratio, using the Gaussian perturbation generation method, specifically involves: The first retention ratio is used as the center value; The search range of the first retention ratio is used as the boundary condition for the Gaussian perturbation; At the center of the first retention ratio, a Gaussian distribution with the first retention ratio as the mean is constructed, and the standard deviation parameter is set to control the dispersion of the generated ratio; Based on the Gaussian distribution, several values ​​are randomly selected from the probability density function to generate a corresponding number of second retention ratios. The second retention ratios represent the possible configurations of an industrial semantic compression. Perform boundary checks on the generated second retention ratio and remove values ​​that exceed the search range of the first retention ratio or do not meet reasonable constraints; Output the second set of retention ratios after boundary verification.

6. The industrial multimodal data extraction method based on a large language model according to claim 5, characterized in that, The process of sequentially traversing the second retention ratio to compress the industrial multimodal feature vector, and performing multidimensional feature extraction on the compression process and structure, specifically includes: Several second retention ratios are used one by one as compression strength parameters for industrial semantic compression to process industrial multimodal feature vectors in a unified feature space. For each second retention ratio, the original multimodal feature vector is input into the encoding module of the large language model for compression to generate the corresponding low-dimensional embedding vector, thus completing the semantic compression process. For each compression-generated embedding vector, multi-dimensional features are extracted, including dimensional redundancy features, information loss features, distribution sparsity features, and reconstruction consistency features. Multidimensional features are combined and weighted according to preset weights to output a compressed score value.

7. The method for extracting industrial multimodal data based on a large language model according to claim 6, characterized in that, The process of performing two-dimensional mapping and interpolation to complete several selected vectors and then converting them into a two-dimensional image is as follows: Several selection vectors are arranged in order of the second retention ratio. Each vector contains the compressed score value and multi-dimensional feature quantization result corresponding to the ratio, forming a two-dimensional data matrix, where the rows represent different second retention ratios and the columns represent feature scores of each dimension. Using the second retention ratio as the horizontal axis and the corresponding compressed score or multi-dimensional feature comprehensive index as the vertical axis, each screening vector is mapped onto a two-dimensional plane to form a discrete scatter set. Interpolation calculations are performed on discrete scatter points based on bilinear interpolation to generate a continuous two-dimensional scoring surface, which is used to fill in the gaps between the original discrete screening vectors. The interpolated continuous score surface is mapped to a color image, with each pixel position corresponding to a specific second retention ratio and compressed score value. The image color represents the size of the compressed score, resulting in a two-dimensional image.

8. The method for extracting industrial multimodal data based on a large language model according to claim 7, characterized in that, Data analysis is performed on the two-dimensional image to select the optimal second retention ratio as the third retention ratio, and semantic compression is then performed, specifically as follows: The compression score value corresponding to each pixel in the two-dimensional image is scanned to identify all local minima. The identified local minima are spatially clustered, and continuous low-score regions are divided into independent candidate regions based on the compression score trend of adjacent pixels. Each region corresponds to a set of close second retention ratios. Statistical analysis is performed on the pixel compression score values ​​of each candidate region to calculate the region's average score, score variance, and minimum score, thereby evaluating the overall compression effect and stability of the region. Prioritize selecting the region with the lowest average score and the smallest score variance. Within the preferred region, select the second retention ratio corresponding to the pixel with the smallest compression score as the final third retention ratio for actual industrial semantic compression operations.

9. A system using the industrial multimodal data extraction method based on a large language model as described in any one of claims 1-8, characterized in that, It includes a data acquisition module, a retention ratio search range constraint module, a retention ratio initialization module, an enumeration data acquisition module, a compression evaluation module, and a filtering application module; The data acquisition module is used to collect industrial multimodal data, extract high-dimensional features based on convolutional neural networks, and normalize and concatenate them into industrial multimodal feature vectors in a unified feature space. The retention ratio search range constraint module is used to obtain the first retention ratio search range of industrial semantic compression in parallel constraints based on PCA cumulative variance and reconstruction error elbow. The retention ratio initialization module is used to generate the first retention ratio based on the semantic information density of industrial multimodal data in the large language model coding space; The enumeration data acquisition module is used to generate several second retention ratios within the search range of the first retention ratio, centered on the first retention ratio, based on the Gaussian perturbation generation method. The compression evaluation module is used to compress the industrial multimodal feature vector by going through the second retention ratio one by one, and to extract multidimensional features from the compression process and structure. Based on the multidimensional features, a weighted score is calculated to generate a compression score value. The second retention ratio is mapped to the corresponding compression score value, and a filtering vector is formed; The filtering application module is used to perform two-dimensional mapping and interpolation to complete several filtering vectors into a two-dimensional image, and to perform data analysis on the two-dimensional image to select the best second retention ratio as the third retention ratio for semantic compression.

Images