A method and system for determining the melting state of plastics based on multimodal feature fusion

Abstract

This invention provides a method and system for determining the melting state of plastics based on multimodal feature fusion, relating to the field of state recognition technology. The method includes: collecting multimodal data of the plastic melting process; constructing a cross-modal covariance matrix based on the multimodal data to describe the cooperative change states among the various modal data; constructing reference covariance matrices representing different melting states using the convex lower bound constraint of the Jørgenson inequality; constructing a divergence feature vector with a length consistent with the number of melting states, based on the matrix logarithmic determinant divergence between the cross-modal covariance matrix and each reference covariance matrix; inputting the divergence feature vector into a random forest based on Fisher information and decision diversity gain, and outputting the posterior probability of each melting state; and outputting the melting state corresponding to the maximum posterior probability as the plastic melting state determination result.

Description

Technical Field

[0001] This invention relates to the field of state recognition technology, and in particular to a method and system for determining the melting state of plastics based on multimodal feature fusion. Background Technology

[0002] The molten state of plastic refers to the different stages in which plastic gradually changes from a solid to a liquid state during the heating process. It can generally be divided into the unmelted state (solid, with tightly packed molecular chains), the initial melting state (some molecules begin to flow, and the plastic softens), and the fully molten state (the plastic is completely liquefied, and the molecular chains are in full motion).

[0003] Identifying the molten state of plastics is directly related to the quality of plastic processing and product performance. Accurately determining the molten state can effectively control process parameters such as injection molding and extrusion, avoid defects caused by premature or delayed molding, improve production efficiency and material utilization, reduce energy consumption and production costs, and ensure that the mechanical properties, appearance, and durability of the final product meet design requirements.

[0004] However, existing technologies for judging the melting state of plastics often rely on single modal data or directly determine the melting state by subjectively setting the heating time. This makes it difficult to capture the coordinated changes between different physical signals, resulting in low judgment accuracy and large errors, and making it impossible to achieve real-time and reliable state monitoring and processing quality control. Summary of the Invention

[0005] To address the technical problem that existing technologies rely heavily on single-modal data or subjectively set heating time to determine the melting state of plastics, making it difficult to capture the coordinated changes between different physical signals, resulting in low accuracy, large errors, and the inability to achieve real-time and reliable state monitoring and processing quality control, this invention provides a method and system for determining the melting state of plastics based on multimodal feature fusion.

[0006] The technical solutions provided by the embodiments of the present invention are as follows: The first aspect of this invention provides a method for determining the melting state of plastics based on multimodal feature fusion, comprising: S1: Acquire multimodal data of the plastic melting process; S2: Based on multimodal data, construct a cross-modal covariance matrix to describe the cooperative change state among various modal data; S3: Using the convex lower bound constraint of the Jørnsen inequality, construct a reference covariance matrix representing different melting states; S4: Based on the matrix logarithmic determinant divergence between the cross-modal covariance matrix and each reference covariance matrix, construct a divergence feature vector with a length consistent with the number of molten states. S5: Input the divergence feature vector into a random forest built based on Fisher information and decision diversity gain, and output the posterior probability of each melt state; S6: Output the melting state corresponding to the maximum posterior probability as the result of plastic melting state discrimination.

[0007] A second aspect of the present invention provides a plastic melt state discrimination system based on multimodal feature fusion, comprising: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the plastic melt state discrimination method based on multimodal feature fusion as described in the first aspect.

[0008] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the plastic melt state discrimination method based on multimodal feature fusion as described in the first aspect.

[0009] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: This invention overcomes the limitations of single-modality limitations by capturing the collaborative dynamic changes of multimodal data during plastic melting through a cross-modal covariance matrix. A melting state reference matrix is ​​constructed using the Jørgensen inequality, essentially establishing statistical prototype templates for different melting stages. Then, the difference between real-time data and each state prototype is quantified using matrix logarithmic determinant divergence, transforming high-dimensional matrix relationships into low-dimensional discriminable feature vectors. This method amplifies the statistical differences between different melting states, as the determinant divergence is highly sensitive to small changes in the covariance structure, accurately characterizing phase transition critical points. Subsequently, a random forest based on Fisher information and decision diversity gains is used for classification. By enhancing the target nature of feature selection and the complementarity of base classifiers, robust classification capabilities for nonlinear and high-noise data are improved, taking into account the importance of various features and improving the robustness and accuracy of discrimination. Finally, the most probable melting state is selected through posterior probability, achieving stable and accurate identification of the plastic melting state. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a method for determining the molten state of plastics based on multimodal feature fusion, provided in an embodiment of the present invention.

[0012] Figure 2 This is a schematic diagram of a plastic melting state discrimination system based on multimodal feature fusion, provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0014] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0015] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0016] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0017] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0018] Reference manual attached Figure 1 The diagram shows a flowchart of a plastic melting state discrimination method based on multimodal feature fusion provided by an embodiment of the present invention.

[0019] This invention provides a method for determining the molten state of plastics based on multimodal feature fusion. This method can be implemented by a device for determining the molten state of plastics based on multimodal feature fusion, which can be a terminal or a server. The processing flow of the method for determining the molten state of plastics based on multimodal feature fusion may include the following steps: S1: Collect multimodal data of the plastic melting process.

[0020] Multimodal data of the plastic melting process refers to the multiple physical and process signals simultaneously collected by different types of sensors or detection methods during the heating, melting, and flow of plastic. This data can cover different aspects of plastic melting, thus comprehensively reflecting the dynamic characteristics of the molten state.

[0021] In one possible implementation, the multimodal data includes a time-synchronized sequence of visual images, an infrared thermal image sequence, and a sequence of acoustic emission time-series signals.

[0022] S2: Based on multimodal data, construct a cross-modal covariance matrix to describe the cooperative change state among various modal data.

[0023] The cross-modal covariance matrix is ​​a mathematical representation used to characterize the interrelationships and coordinated changes between different modal data. In this process, the matrix not only preserves the fluctuation information of each mode itself but also captures the synchronous changes and correlations between modes. Its principle lies in quantifying the linkage characteristics of different signals through covariance and cross-correlation, enabling the system to identify minute but critical state changes during the melting process. This allows the discrimination method to integrate complementary information from multiple signals, thereby significantly improving the accuracy and robustness of melt state discrimination and ensuring real-time and reliable monitoring of complex dynamic processes.

[0024] In one possible implementation, S2 specifically includes: S201: Extract the transient changes of each modality in the multimodal data.

[0025] In one possible implementation, the transient change is specifically the difference between the statistical eigenvectors at the first and second time points.

[0026] The first time point and the second time point are adjacent time points.

[0027] The statistical feature vectors include visual image sequence features, infrared thermal image sequence features, and acoustic emission time-series signal sequence features.

[0028] The visual image sequence features include the image RGB mean, image RGB variance, and image histogram peak value. The infrared thermal image sequence features include the average thermal image temperature, maximum thermal image temperature, and thermal image temperature standard deviation. The acoustic emission time-series signal sequence features include the signal dominant frequency, signal spectral centroid, and signal bandwidth.

[0029] S202: Based on the transient changes, establish a cross-modal covariance matrix, where each element in the cross-modal covariance matrix is ​​a normalized outer product matrix between the statistical eigenvectors corresponding to any two modal data.

[0030] The formula for the cross-modal covariance matrix is ​​as follows: .

[0031] .

[0032] .

[0033] in, Indicating the plastic melting process t Time-varying cross-modal covariant matrix, and They represent the first m The modal data and the first n Modal data in t The rate of change of the statistical eigenvector at time t is the transient change, and the subscript is... Indicates transpose. This represents the total number of modes in multimodal data. This indicates the avoidance of tiny positive numbers with a denominator of zero. Represents the Euclidean norm. and They represent the first m The modal data and the first n Modal data in t Statistical feature vector at time step and They represent the first m The modal data and the first n Modal data in t Statistical eigenvector at time -1.

[0034] Specifically, by extracting transient changes from each modal data and constructing a cross-modal covariance matrix, the system can quantify the synchronous changes and interrelationships between different signals. The matrix elements reflect the normalized correlation of the changes in the eigenvectors of each modality, enabling the method to not only retain the fluctuation information of a single mode but also capture the cooperative features between modes. In this way, the discrimination method can comprehensively utilize the complementary information of multimodal data to effectively identify minute key changes in the plastic melting process, thereby significantly improving the accuracy, robustness, and real-time performance of melt state discrimination.

[0035] S3: Using the convex lower bound constraint of the Jørnson inequality, construct a reference covariance matrix representing different melting states.

[0036] The Chinsen inequality convex lower bound constraint is used to construct a stable and convex lower bound in the covariance matrix space, thereby ensuring that the generated matrix has good numerical properties and discriminability. It guarantees that the reference matrix is ​​mathematically positive definite and stable, avoiding anomalies or irreversible situations. The reference covariance matrix is ​​a statistical prototype matrix established for each melting state, representing the ideal cooperative variation pattern among modal data under that state. It serves as a mathematical template for the melting state and is used for subsequent comparison and discrimination.

[0037] Based on the convex lower bound constraint of the Chinsen inequality, statistical analysis is performed on historical or training data of different melting stages to generate a reference covariance matrix corresponding to each melting state. By constructing stable and distinguishable state prototypes, the system can clearly define the characteristic range of different states in a high-dimensional covariance space. Typical cooperative patterns of each melting state are accurately characterized, amplifying the differences between states and improving the accuracy and robustness of subsequent discrimination, thereby ensuring that the system can reliably identify melting states when faced with complex real-time data.

[0038] In one possible implementation, the molten state includes an unmolten state, a partially molten state, and a fully molten state. S3 specifically includes: S301: Obtain the set of time indices belonging to each melting state from the historical labeled multimodal data sample set.

[0039] S302: Establish the historical cross-modal covariance matrix corresponding to the multimodal data of the same melting state in the time index set.

[0040] S303: Calculate the matrix logarithm of each historical cross-modal covariance matrix. S304: Perform arithmetic mean and matrix exponent mapping operations on the calculation results sequentially to obtain the initial reference covariance matrix. S305: Under the constraint of the convex lower bound of the Jørnson inequality, add a regularization term to the initial reference covariance matrix to obtain a reference covariance matrix that is always positive definite.

[0041] The specific formula for the reference covariant matrix is ​​as follows: .

[0042] .

[0043] in, Indicates belonging to c A set of time indices for a molten state. Represents the natural logarithm function. Representing historical moments The corresponding historical cross-modal covariance matrix, This indicates a convex lower bound constraint for the Chinson inequality. Represents the determinant of a matrix. This represents the natural exponential function. express c The reference covariance matrix corresponding to the molten state. Indicates the update symbol, Represents the regularization coefficient. Representing dimensions and A consistent identity matrix.

[0044] It should be noted that by taking the logarithm and arithmetic mean of the historical cross-modal covariance matrices, and then generating the reference covariance matrix through exponential mapping, a stable positive definite matrix can be obtained while maintaining the matrix structure characteristics. Regularization terms further enhance numerical stability, and the convex lower bound constraint of the Jørnsen inequality ensures that the generated reference matrix will not exhibit singular or irreversible conditions. This approach can accurately characterize the typical cooperative modes between modes under each melting state, making subsequent divergence calculations more reliable, effectively distinguishing different melting states, and improving discrimination accuracy and robustness.

[0045] Specifically, this constraint is used to constrain the regularization coefficient. The size of the regularization coefficient is adjusted to avoid the resulting reference covariance matrix being singular or nearly singular. Specifically, after each addition of a regularization term, it is checked whether the convex lower bound constraint of the Chimsen inequality is satisfied. If satisfied, the reference covariance matrix with the added regularization term is directly output as the result. Otherwise, the regularization coefficient is automatically reduced, and the judgment logic is re-entered until the convex lower bound constraint of the Chimsen inequality is satisfied, ensuring that the resulting reference covariance matrix is ​​positive definite.

[0046] It should be noted that in actual calculations, to avoid matrix singularities (determinant approaching 0, leading to errors in subsequent calculations), a regularization term is added to the initial reference covariance matrix. This involves adding a small positive number to all diagonal elements, which directly changes the matrix's determinant. After regularization, the determinant of the reference covariance matrix increases, and the corresponding convex function... It will also grow larger, and may very well break the inequality, becoming This violates the convex lower bound constraint of the Jönsson inequality (the function value of a convex function at the average of a set of variables is always less than or equal to the average of the individual function values ​​of the set of variables). This directly causes the divergence of the subsequently calculated logarithmic determinant to become negative, meaning "the difference between two identical matrices is negative." The divergence completely loses its meaning as a measure of difference. Since the divergence, a core feature for classification, is incorrect, the final determination of the plastic's molten state will inevitably be wrong (for example, misclassifying fully molten as unmolten, directly affecting processing quality control). Therefore, the convex lower bound constraint of the Jönsson inequality takes effect in this situation. That is, once a violation of the Jönsson inequality constraint is found after regularization, the regularization coefficient is automatically reduced and recalculated until the constraint is satisfied, thus avoiding numerical errors at the source. This ensures that the determination of the plastic's molten state is accurate and reliable.

[0047] Specifically, this process involves statistical analysis of historically labeled multimodal data. First, it extracts the cross-modal covariance matrix corresponding to each melting state as the original feature representation. Then, it calculates the logarithm of these matrices and performs an arithmetic mean, obtaining the initial reference matrix through matrix exponential mapping. Based on this, it applies the convex lower bound constraint of the Jørnson inequality and adds appropriate regularization terms to ensure that the reference covariance matrix is ​​always mathematically positive definite and numerically stable, thus avoiding singular or non-invertible matrix conditions. This method can generate stable and distinguishable state prototype matrices, accurately characterizing the typical cooperative modes of each melting state. It enables the system to clearly define the feature range of different states in a high-dimensional feature space, amplifying the differences between states, improving the accuracy and robustness of subsequent discrimination, and achieving real-time and reliable monitoring of complex melting processes.

[0048] S4: Based on the matrix logarithmic determinant divergence between the cross-modal covariance matrix and each reference covariance matrix, construct a divergence eigenvector with a length consistent with the number of molten states.

[0049] Among them, matrix logarithmic determinant divergence is a measure of the difference between two positive definite matrices in their overall covariance structure. It quantifies the statistical difference between real-time data and reference states by comparing the determinant and logarithmic information of the matrices, and is highly sensitive to small changes in the covariance structure. The divergence feature vector is a vector composed of the divergence values ​​between the cross-modal covariance matrix and each reference covariance matrix. Its length corresponds to the number of melt states, and each component reflects the similarity between the real-time state and the corresponding reference state, providing directly usable input features for subsequent classification.

[0050] This process uses divergence to quantify the statistical differences between different states, transforming high-dimensional complex matrix relationships into low-dimensional discriminative features, thereby highlighting subtle differences in the molten state. It effectively compresses multimodal co-variance information and enhances the distinguishability between states, making it easier for the discriminative model to identify critical phase transition points and achieving high-precision and robust molten state discrimination.

[0051] In one possible implementation, S4 specifically includes: S401: Calculate the matrix logarithmic determinant divergence for the corresponding melting state by taking the product of the cross-modal covariance matrix and the reference covariance matrix corresponding to different melting states as input.

[0052] S402: Concatenate the logarithmic determinant divergence of the matrices corresponding to each melting state to obtain the divergence eigenvector.

[0053] The divergence feature vector is specifically: .

[0054] .

[0055] in, express t The divergence eigenvector at time step 1. , and They represent t The logarithmic determinant divergence of the matrix corresponding to the unmelted state, the beginning of the melting state, and the fully molten state at time points. express t The logarithmic determinant divergence of the matrix corresponding to the c-th type of molten state at time c, with subscripts Indicates transpose. This represents the total dimension of the statistical features of multimodal data. Represents the trace of a matrix. Represents the determinant of a matrix.

[0056] It should be noted that this divergence eigenvector quantifies state deviation by comparing the overall structural differences between the real-time covariance matrix and the reference state matrix. The trace operation captures the total energy or overall change of the matrix, while the logarithmic determinant reflects the distribution characteristics of the covariance. The combination of the two can sensitively reflect small cooperative changes. By subtracting the correction for the total dimension of the statistical features, the resulting divergence value can intuitively represent the difference between the real-time state and the ideal molten state, thus providing clear and distinguishable features for the classification model and improving the accuracy and robustness of molten state discrimination.

[0057] Specifically, this process compares the real-time cross-modal covariance matrix with the reference covariance matrix corresponding to each molten state, calculates the matrix logarithmic determinant divergence, and quantifies the difference between the current state and each reference state in the overall covariance structure. Each divergence value reflects the similarity between the real-time data and a template of a certain molten state. Then, all divergence values ​​are concatenated to form a divergence feature vector, with a length consistent with the number of molten states. This process compresses complex high-dimensional covariance information into low-dimensional features that can be directly used for classification, while enhancing the distinguishability between different states. Since divergence is highly sensitive to small changes in covariance, this method can highlight key transition points in the melting process, enabling the discriminant model to more accurately and robustly identify states such as unmelted, beginning to melt, and fully melted, thereby ensuring the reliability of real-time monitoring and processing quality control.

[0058] S5: Input the divergence feature vector into a random forest built based on Fisher information and decision diversity gain, and output the posterior probability of each melting state.

[0059] Random forest is an ensemble learning model composed of multiple decision trees, which improves prediction accuracy and robustness by voting or averaging the results of multiple trees. Fisher information is used to quantify the sensitivity of model parameters to the predicted output, helping to optimize the hyperparameters of the random forest in this method, enabling the model to more effectively distinguish the melt state when faced with different input features. Decision diversity gain measures the prediction differences between decision trees, improving overall classification performance by encouraging the complementarity of decisions among trees. Posterior probability represents the likelihood of each melt state occurring given the observed features, used to determine the most likely melt stage of the plastic.

[0060] By inputting the divergence feature vector into the optimized random forest, the model can integrate the judgments of multiple decision trees, combine Fisher information-guided hyperparameter tuning and decision diversity gains, and output the posterior probability of each molten state, thereby achieving high-precision, stable, and robust molten state discrimination. This method can fully utilize the differences between features and the advantages of model ensemble, effectively reducing misjudgments and adapting to complex changes in the melting process.

[0061] In one possible implementation, a random forest includes an input layer, an ensemble layer, and an output layer connected in sequence, wherein the ensemble layer includes multiple parallel decision trees.

[0062] As is understandable, a random forest consists of three parts: an input layer that receives divergence feature vectors, feeding multimodal collaborative variation information into the model; an ensemble layer composed of multiple parallel decision trees, each trained independently to capture discrimination patterns under different feature combinations; and an output layer that votes on or weights the predictions of all decision trees to generate the posterior probability of the fused state. During training, multiple subsets are generated by randomly sampling the training set with replacement, and a decision tree is trained on each subset. Simultaneously, at each node split, a subset of features is randomly selected for optimal splitting, thus forming a diverse set of trees. Ensemble training reduces overfitting of individual trees and improves overall prediction performance.

[0063] S5 specifically includes: S501: With the goal of ensuring decision diversity among decision trees in the random forest, the hyperparameter set of the random forest is determined by combining Fisher information. The hyperparameter set includes the temporal forgetting factor, the number of decision trees in the random forest, and the depth of the decision trees.

[0064] In one possible implementation, S501 specifically includes: S5011: Obtain training data, which includes the sample to be judged and the corresponding melt state label.

[0065] S5012: Encode different hyperparameter groups of the random forest into population individuals and initialize the population individuals.

[0066] S5013: Establish a fitness function for the decision diversity gain of random forests.

[0067] The fitness function is as follows: .

[0068] .

[0069] .

[0070] in, Indicates hyperparameter group The fitness function value under the following conditions These represent the temporal forgetting factor, the number of decision trees in the random forest, and the depth of the decision tree, respectively. Indicates hyperparameter group Random forest ensemble accuracy under the following conditions Indicates hyperparameter group The next j The individual prediction accuracy of a decision tree. Indicates hyperparameter group The gain in decision diversity, Indicates the weight of decision diversity gain. This represents the weight of the random forest complexity penalty.

[0071] Among them, the accuracy of random forest ensemble It is the overall accuracy rate after voting by multiple decision trees. The fitness function represents the average accuracy of a single decision tree, reflecting its average performance. It's important to note that this fitness function evaluates the quality of the hyperparameter combination by comprehensively considering the ensemble accuracy of the random forest, the average accuracy of individual decision trees, and the decision diversity gain. While maintaining overall prediction accuracy, it encourages differentiated judgments among decision trees, thereby improving the robustness and generalization ability of the ensemble model. Simultaneously, it penalizes complexity to prevent over-complexity. In this way, the optimized hyperparameter combination enables the random forest to be both efficient and stable in handling fused state discrimination, fully utilizing the differential information of multimodal features to improve classification accuracy and reliability.

[0072] S5014: Input the sample to be judged into the random forest and output the predicted label of the melting state.

[0073] S5015: Calculate the fitness function value of each individual in the population. If the change in fitness function value is greater than the preset change in fitness function value or the number of iterations is greater than the preset number of iterations, proceed to step S5019; otherwise, proceed to step S5016.

[0074] It should be noted that those skilled in the art can set the preset fitness function value change amount and the preset number of iterations according to actual needs, and this invention does not limit these settings.

[0075] S5016: Calculate the fitness gradient of the current population.

[0076] The specific formula for calculating the fitness gradient is as follows: .

[0077] in, Indicates the first i Hyperparameter set The corresponding fitness function, Indicates the hyperparameter set Find the gradient.

[0078] S5017: Establish an empirical Fisher information matrix based on fitness gradient to quantify the sensitivity of each hyperparameter to the fitness function.

[0079] The specific formula for the empirical Fisher information matrix is ​​as follows: .

[0080] in, Represents the empirical Fisher information matrix. Indicates the population size, i.e., the total number of hyperparameter sets, indicated by the subscript. This indicates transpose.

[0081] S5018: Determine the hyperparameter set search direction based on the empirical Fisher information matrix, update the population individuals according to the hyperparameter set search direction, and return to step S5014.

[0082] The formula for the hyperparameter search direction is as follows: .

[0083] in, Represents the inverse of the empirical Fisher information matrix. Indicates the first i Hyperparameter set The corresponding hyperparameter update amount, The plus or minus sign indicates the search direction for the hyperparameter group.

[0084] It's important to note that by calculating the gradient of the fitness function for each hyperparameter group, the direction and magnitude of each hyperparameter's impact on model performance can be quantified. Based on these gradients, an empirical Fisher information matrix can be built to assess the sensitivity of different hyperparameters to fitness changes, thereby determining the most effective optimization direction. This information is then used to update the hyperparameter combinations, allowing the random forest to gradually optimize towards high ensemble accuracy, high decision diversity, and moderate complexity during training. This method efficiently guides hyperparameter search, ensures a stable and reliable optimization process, and improves the model's accuracy and robustness in identifying melt states.

[0085] S5019: Output the current hyperparameter set.

[0086] Specifically, the process first acquires the samples to be judged and their melt state labels through training data, providing supervision information for the random forest model. Then, different combinations of hyperparameters are represented as individuals in the population, and a fitness function is established. This function comprehensively considers ensemble accuracy, single-tree performance, and decision diversity gains, while penalizing model complexity, thus balancing accuracy and generalization ability. Next, by calculating the fitness gradient, an empirical Fisher information matrix is ​​established to quantify the sensitivity of each hyperparameter to the fitness function, and the hyperparameter update direction is determined accordingly, gradually optimizing the individuals in the population. After multiple iterations, the final output is a hyperparameter combination that balances decision diversity and overall performance. This method can systematically optimize random forests, enabling each decision tree to complement each other and enhance classification performance while maintaining independent judgment capabilities, thereby improving the model's accuracy, robustness, and adaptability to complex dynamic features in melt state recognition.

[0087] S502: Update the random forest according to the determined number and depth of decision trees.

[0088] S503: Establish a temporal input vector for the divergence feature vector according to the determined temporal forgetting factor.

[0089] .

[0090] in, express t Timing input vector at each time step, The time window length represents the time-series input vector. Indicates the distance from the current time step. k The temporal forgetting factor at each time step, , This represents the temporal forgetting factor at the current time step.

[0091] S504: Input the time series input vector into the updated random forest and output the posterior probability of each melt state.

[0092] Specifically, the process first establishes a random forest model using training data, combining multiple decision trees into an ensemble structure to enhance prediction stability and robustness. To improve the model's ability to distinguish different melt states, Fisher information is used to quantify the sensitivity of hyperparameters to prediction performance, thereby guiding hyperparameter optimization and enabling the model to more effectively identify state differences when faced with different input features. Simultaneously, decision diversity gain is used to evaluate the complementarity between decision trees, encouraging diverse judgments among trees and avoiding consistent misjudgments in the overall model. After training, the divergence feature vectors are time-weighted and then input into the random forest. The model outputs the posterior probability of each melt state, representing the likelihood of each state under the current observation conditions. This process fully utilizes the differential information of multimodal features and the advantages of model ensemble, making melt state discrimination more accurate, stable, and adaptable to complex dynamic processes, thus ensuring real-time monitoring and quality control of the processing.

[0093] S6: Output the melting state corresponding to the maximum posterior probability as the result of plastic melting state discrimination.

[0094] In practical applications, the entire process of plastic melting state discrimination first comprehensively acquires visual, thermal, and acoustic information of the plastic heating and melting process through multimodal data acquisition. Then, transient changes in each modal data are extracted, and a cross-modal covariance matrix is ​​constructed to quantify the collaborative dynamic changes between different modes and capture subtle but crucial state features. Subsequently, a reference covariance matrix for each melting state is generated using Jørgensen's inequality and regularization techniques, establishing a stable and distinguishable state prototype template to ensure matrix positive definiteness and numerical stability. By calculating the logarithmic determinant divergence between the real-time matrix and the reference matrix, high-dimensional covariance information is compressed into a discriminative low-dimensional feature vector, highlighting the differences between states and phase transition critical points. Finally, this feature vector is input into a random forest optimized with Fisher information and enhanced decision diversity. The ensemble judgment of multiple decision trees outputs the posterior probability of each melting state, and the most probable state is selected as the discrimination result. This method achieves high-precision, robust, and real-time discrimination of complex dynamic melting processes through multimodal information fusion, stable state template construction, and sensitive divergence feature quantization, which can effectively improve the reliability of processing quality control.

[0095] In this invention, the collaborative dynamic changes of multimodal data during plastic melting are captured through a cross-modal covariance matrix, overcoming the limitations of a single modality. A melting state reference matrix is ​​constructed using the Jørgensen inequality, essentially establishing statistical prototype templates for different melting stages. The difference between real-time data and each state prototype is then quantified using matrix logarithmic determinant divergence, transforming high-dimensional matrix relationships into low-dimensional discriminable feature vectors. This method amplifies the statistical differences between different melting states; because determinant divergence is highly sensitive to small changes in the covariance structure, it can accurately characterize the phase transition critical point. Subsequently, a random forest based on Fisher information and decision diversity gains is used for classification. By enhancing the target nature of feature selection and the complementarity of base classifiers, robust classification capabilities for nonlinear and high-noise data are improved, taking into account the importance of various features and improving the robustness and accuracy of discrimination. Finally, the most probable melting state is selected through posterior probability, achieving stable and accurate identification of the plastic melting state.

[0096] Reference manual attached Figure 2 The diagram shows a schematic of a plastic melting state discrimination system based on multimodal feature fusion provided by the present invention.

[0097] The present invention also provides a plastic melt state discrimination system 20 based on multimodal feature fusion, applied to the above-mentioned plastic melt state discrimination method based on multimodal feature fusion, comprising: Processor 201.

[0098] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, they implement the plastic melt state discrimination method based on multimodal feature fusion as described in the method embodiment.

[0099] The plastic melting state discrimination system 20 based on multimodal feature fusion provided by the present invention can perform the above-mentioned plastic melting state discrimination method based on multimodal feature fusion and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate further.

[0100] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0101] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0102] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0103] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0104] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0105] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0106] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0108] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0111] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the plastic melting state discrimination method based on multimodal feature fusion as described in the method embodiment.

[0113] The present invention provides a computer-readable storage medium that can implement the steps and effects of the plastic melting state discrimination method based on multimodal feature fusion in the above-described method embodiments. To avoid repetition, the present invention will not elaborate further.

[0114] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0115] The following points need to be explained: (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0116] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0117] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0118] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for determining the melting state of plastics based on multimodal feature fusion, characterized in that, include: S1: Acquire multimodal data of the plastic melting process; S2: Based on the multimodal data, construct a cross-modal covariance matrix to describe the cooperative change state between the various modal data; S3: Using the convex lower bound constraint of the Jørnsen inequality, construct a reference covariance matrix representing different melting states; S4: Based on the matrix logarithmic determinant divergence between the cross-modal covariance matrix and each of the reference covariance matrices, construct a divergence feature vector with a length consistent with the number of molten states. S5: Input the divergence feature vector into a random forest built based on Fisher information and decision diversity gain, and output the posterior probability of each melt state; S6: Output the melting state corresponding to the maximum posterior probability as the result of plastic melting state discrimination.

2. The method for determining the molten state of plastics based on multimodal feature fusion according to claim 1, characterized in that, The multimodal data includes time-synchronized visual image sequences, infrared thermal image sequences, and acoustic emission time-series signal sequences.

3. The method for determining the molten state of plastics based on multimodal feature fusion according to claim 2, characterized in that, S2 specifically includes: S201: Extract the transient changes of each modality in the multimodal data respectively; S202: Based on the transient change, establish the cross-modal covariance matrix, wherein each element in the cross-modal covariance matrix is ​​a normalized outer product matrix between the statistical eigenvectors corresponding to any two modal data.

4. The method for determining the melting state of plastics based on multimodal feature fusion according to claim 3, characterized in that, The transient change is specifically the difference between the statistical feature vectors at the first and second time points; Wherein, the first time point and the second time point are adjacent time points; The statistical feature vector includes visual image sequence features, infrared thermal image sequence features, and acoustic emission time-series signal sequence features; The visual image sequence features include the image RGB mean, image RGB variance, and image histogram peak value; the infrared thermal image sequence features include the thermal image average temperature, thermal image maximum temperature, and thermal image temperature standard deviation; and the acoustic emission time-series signal sequence features include the signal dominant frequency, signal spectrum centroid, and signal bandwidth.

5. The method for determining the melting state of plastics based on multimodal feature fusion according to claim 1, characterized in that, The molten state includes an unmolten state, a beginning-to-melt state, and a fully molten state; S3 specifically includes: S301: Obtain the set of time indices belonging to each of the aforementioned melting states from the historical labeled multimodal data sample set; S302: Establish the historical cross-modal covariance matrix corresponding to the multimodal data of the same molten state in the time index set; S303: Calculate the matrix logarithm of each of the aforementioned historical cross-modal covariance matrices; S304: Perform arithmetic mean operation and matrix exponential mapping operation on the calculation results in sequence to obtain the initial reference covariant matrix; S305: Under the constraint of the convex lower bound of the Jørnson inequality, add a regularization term to the initial reference covariant matrix to obtain the reference covariant matrix that is always positive definite.

6. The method for determining the melting state of plastics based on multimodal feature fusion according to claim 1, characterized in that, S4 specifically includes: S401: Using the product of the cross-modal covariance matrix and the reference covariance matrix corresponding to different melting states as input, calculate the logarithmic determinant divergence of the matrix for the corresponding melting state; S402: Concatenate the matrix logarithmic determinant divergence corresponding to each of the molten states to obtain the divergence feature vector.

7. The method for determining the molten state of plastics based on multimodal feature fusion according to claim 1, characterized in that, The random forest comprises an input layer, an ensemble layer, and an output layer connected in sequence, wherein the ensemble layer comprises multiple parallel decision trees; S5 specifically includes: S501: To ensure that each decision tree in the random forest has decision diversity, the hyperparameter set of the random forest is determined by combining the Fisher information, wherein the hyperparameter set includes the temporal forgetting factor, the number of random forest decision trees, and the depth of decision trees; S502: Update the random forest according to the determined number of random forest decision trees and the depth of the decision trees; S503: Establish a temporal input vector for the divergence feature vector according to the determined temporal forgetting factor; S504: Input the time-series input vector into the updated random forest and output the posterior probability of each of the melt states.

8. The method for determining the molten state of plastics based on multimodal feature fusion according to claim 7, characterized in that, S501 specifically includes: S5011: Acquire training data, wherein the training data includes samples to be judged and corresponding melt state labels; S5012: Encode the different hyperparameter groups of the random forest into population individuals, and initialize the population individuals; S5013: Establish a fitness function for the decision diversity gain of the random forest; S5014: Input the sample to be judged into the random forest and output the melt state prediction label; S5015: Calculate the fitness function value of each individual in the population. If the change in fitness function value is greater than the preset change in fitness function value or the number of iterations is greater than the preset number of iterations, proceed to step S5019; otherwise, proceed to step S5016. S5016: Calculate the fitness gradient of the current population; S5017: Establish an empirical Fisher information matrix based on the fitness gradient to quantify the sensitivity of each hyperparameter to the fitness function; S5018: Determine the hyperparameter set search direction based on the empirical Fisher information matrix, update the population individuals according to the hyperparameter set search direction, and return to step S5014; S5019: Output the current hyperparameter set.

9. A plastic melt state discrimination system based on multimodal feature fusion, characterized in that, include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the plastic melt state discrimination method based on multimodal feature fusion as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the plastic melting state discrimination method based on multimodal feature fusion as described in any one of claims 1 to 8.

Images