Computer information screening system and method based on artificial intelligence

Abstract

The invention discloses a computer information screening system and method based on artificial intelligence, and particularly relates to the technical field of information screening. Analyzing the multi-modal feature vector of the to-be-screened information, and evaluating the distribution density in the high-dimensional feature space through the feature projection network; dimensionality reduction mapping is carried out on the multi-modal feature vectors, a distance matrix between feature clusters after dimensionality reduction is calculated, and the local folding degree of the feature space is evaluated; then, based on the feature distribution density and the local folding degree, whether the distortion degree of the feature space exceeds a preset threshold value or not is judged; when the distortion degree exceeds a threshold value, the feature loss degree is evaluated by calculating mutual information between the original features and the reconstructed features; according to the method, the distribution density, the local folding degree and the feature loss degree of the multi-modal features are integrated, the overall reliability of the information screening result is evaluated, whether the screening strategy needs to be adjusted or not is judged, the accuracy and stability of complex data screening are effectively improved, and the method is suitable for large-scale data analysis and anomaly detection scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of information screening, and more specifically, to a computer information screening system and method based on artificial intelligence. Background Art

[0002] With the development of artificial intelligence technology, multimodal feature vectors have been widely used in the representation and processing of complex data to improve the accuracy and efficiency of computer information screening. However, in high-dimensional feature space, the dimension of the feature vector increases, causing the geometric structure of the feature space to become complex and easy to distort. In this high-dimensional environment, information that may have been irrelevant may be incorrectly mapped to similar feature space positions, resulting in feature space folding. This feature space folding phenomenon not only reduces the reliability of the screening results, but also increases the risk of false positives and negatives. Traditional methods, such as simply increasing the amount of training data, cannot effectively solve the problem of feature space folding.

[0003] In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present invention provide a computer information screening system and method based on artificial intelligence to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A computer information screening method based on artificial intelligence comprises the following steps: Analyze the multimodal feature vectors of the information to be screened, and evaluate the distribution density of the multimodal feature vectors in the high-dimensional feature space through the feature projection network; Perform dimension reduction mapping on the multimodal feature vectors of the information to be screened, analyze the distance matrix between feature clusters after dimension reduction, and evaluate the degree of local folding of the feature space; Based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the local folding degree of the feature space, determining whether the distortion degree of the feature space exceeds a preset threshold; When the degree of feature space distortion exceeds a preset threshold, the degree of feature loss of the information to be screened is evaluated by analyzing the mutual information between the original features and the reconstructed features of the information to be screened; The distribution density of the modal feature vector in the high-dimensional feature space, the degree of local folding of the feature space, and the degree of feature loss of the information to be screened are comprehensively analyzed to evaluate the reliability of the information screening results and determine whether the screening strategy needs to be adjusted.

[0006] In a preferred embodiment, the multimodal feature vector of the information to be screened is analyzed, and the distribution density of the multimodal feature vector in the high-dimensional feature space is evaluated by a feature projection network, specifically: Obtain a multimodal data set to be screened and extract the original feature vectors under each modality, including text, image and numerical feature vectors; Normalize each modal eigenvector; The feature fusion method is used to combine the features of each modality into a joint multimodal feature vector; Based on the joint multimodal feature vector, a feature projection network is used for nonlinear mapping to generate high-dimensional feature representation; Kernel density estimation is used to calculate the distribution density of high-dimensional feature representation in the feature space after feature projection.

[0007] In a preferred embodiment, the multimodal feature vector of the information to be screened is subjected to dimensionality reduction mapping, the distance matrix between the feature clusters after dimensionality reduction is analyzed, and the degree of local folding of the feature space is evaluated, specifically: Select a dimensionality reduction method based on geometry preservation to map the high-dimensional feature vector set to a low-dimensional feature space; Calculate the change in Euclidean distance between feature clusters before and after dimensionality reduction, and establish a distance matrix comparison before and after dimensionality reduction; Based on the rate of change of the distance matrix, the degree of separation between feature clusters after dimensionality reduction is evaluated; The overlap degree of feature points in the local neighborhood is counted to evaluate the local folding degree of the feature space.

[0008] In a preferred embodiment, based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the degree of local folding of the feature space, it is determined whether the distortion degree of the feature space exceeds a preset threshold, specifically: Preset the kernel density estimate threshold and compare the kernel density estimate with the kernel density estimate threshold: When the kernel density estimate is greater than or equal to the kernel density estimate threshold, it indicates that the feature vectors are highly clustered in the high-dimensional feature space; When the kernel density estimate is less than the kernel density estimate threshold, it means that the feature vector is dispersed in the high-dimensional feature space; Preset the overlap coefficient threshold and compare the overlap coefficient with the overlap coefficient threshold: When the overlap coefficient is greater than the overlap coefficient threshold, it indicates that the local folding degree of the feature space is high; When the overlap coefficient is less than or equal to the overlap coefficient threshold, it indicates that the local folding degree of the feature space is low; When the kernel density estimation value is less than the kernel density estimation value threshold and the overlap coefficient is greater than the overlap coefficient threshold, it is determined that the distortion degree of the feature space exceeds the preset threshold; otherwise, it is determined that the distortion degree of the feature space does not exceed the preset threshold.

[0009] In a preferred embodiment, the feature loss degree of the information to be screened is evaluated by analyzing the mutual information between the original features and the reconstructed features of the information to be screened, specifically: Extract the original feature vector from the information to be screened, and generate the corresponding reconstructed feature vector through a pre-trained model or an autoencoder; Define the joint probability distribution between the original feature vector and the reconstructed feature vector; The entropy estimation method is used to calculate the joint entropy and marginal entropy of the original features and the reconstructed features, and the mutual information value between the original features and the reconstructed features is obtained; The calculated mutual information value is compared with the preset threshold to evaluate the degree of information loss in feature reconstruction: feature loss index, calculated as: ;in, is the feature loss index; is the mutual information value between the original eigenvector and the reconstructed eigenvector; is the mutual information threshold.

[0010] In a preferred embodiment, the distribution density of the multimodal feature vector in the high-dimensional feature space, the degree of local folding of the feature space, and the degree of feature loss of the information to be screened are comprehensively analyzed to evaluate the reliability of the information screening results and determine whether the screening strategy needs to be adjusted, specifically: The normalized kernel density estimate, overlap index, and feature loss index are calculated to obtain the reliability index, which is calculated as follows: ;in, is the reliability index; For point The kernel density estimate at ; is the feature loss index; is the overlap index; is the hyperparameter of the feature loss index; is the hyperparameter of the overlap index; A reliability index threshold is preset, and the reliability index is compared with the reliability index threshold: When the reliability index is greater than or equal to the reliability index threshold, it indicates that the reliability of the information screening result is high and there is no need to adjust the screening strategy; When the reliability index is less than the reliability index threshold, it indicates that the reliability of the information screening result is low and the screening strategy needs to be adjusted.

[0011] On the other hand, the present invention provides a computer information screening system based on artificial intelligence, including a distribution density assessment module, a folding degree assessment module, a distortion degree determination module, a characteristic loss assessment module, and a reliability assessment module; The distribution density evaluation module analyzes the multimodal feature vectors of the information to be screened and evaluates the distribution density of the multimodal feature vectors in the high-dimensional feature space through the feature projection network; The folding degree evaluation module performs dimension reduction mapping on the multimodal feature vectors of the information to be screened, analyzes the distance matrix between feature clusters after dimension reduction, and evaluates the local folding degree of the feature space; The distortion degree determination module determines whether the distortion degree of the feature space exceeds a preset threshold based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the local folding degree of the feature space; When the degree of feature space distortion exceeds a preset threshold, the feature loss assessment module assesses the degree of feature loss of the information to be screened by analyzing the mutual information between the original features and the reconstructed features of the information to be screened; The reliability assessment module conducts a comprehensive analysis of the distribution density of the modal feature vector in the high-dimensional feature space, the degree of local folding of the feature space, and the degree of feature loss of the information to be screened, evaluates the reliability of the information screening results, and determines whether the screening strategy needs to be adjusted.

[0012] The technical effects and advantages of the computer information screening system and method based on artificial intelligence of the present invention are as follows: By performing distribution density analysis and dimensionality reduction mapping on multimodal feature vectors, the degree of local folding of the feature space can be quantified to avoid false correlations caused by feature overlaps between samples of different categories. In addition, by calculating the mutual information between the original features and the reconstructed features, the degree of feature loss can be quantified, and data integrity can be further evaluated from the perspective of information fidelity. By combining the multimodal feature distribution density, local folding degree and feature loss index, a reliability index can be obtained, which can guide further adjustments to the screening strategy when the screening results do not meet the standards, improve the stability of feature screening and the scientific nature of screening decisions. It can more comprehensively capture the data distribution characteristics in high-dimensional space, significantly reduce the risk of false positives and negatives, and is suitable for complex fine-grained classification and anomaly detection. It can effectively alleviate the problem of feature space folding in high-dimensional feature space and improve the reliability and accuracy of screening results. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a computer information screening method based on artificial intelligence of the present invention; Figure 2 The present invention is a schematic diagram of the structure of an artificial intelligence-based computer information screening system. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] Example 1 Figure 1 The present invention provides a computer information screening method based on artificial intelligence, which comprises the following steps: Analyze the multimodal feature vectors of the information to be screened, and evaluate the distribution density of the multimodal feature vectors in the high-dimensional feature space through the feature projection network; Perform dimension reduction mapping on the multimodal feature vectors of the information to be screened, analyze the distance matrix between feature clusters after dimension reduction, and evaluate the degree of local folding of the feature space; Based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the local folding degree of the feature space, determining whether the distortion degree of the feature space exceeds a preset threshold; When the degree of feature space distortion exceeds a preset threshold, the degree of feature loss of the information to be screened is evaluated by analyzing the mutual information between the original features and the reconstructed features of the information to be screened; The distribution density of the modal feature vector in the high-dimensional feature space, the degree of local folding of the feature space, and the degree of feature loss of the information to be screened are comprehensively analyzed to evaluate the reliability of the information screening results and determine whether the screening strategy needs to be adjusted.

[0016] Specifically, the multimodal feature vectors of the information to be screened are analyzed, and the distribution density of the multimodal feature vectors in the high-dimensional feature space is evaluated through the feature projection network, including: Obtain the multimodal data set to be screened and extract the original feature vectors under each mode, including text, image and numerical feature vectors: Obtain a multimodal data set from the information to be screened. The multimodal data set contains multiple data forms such as text, image and numerical values. For text data, extract word embedding vectors through natural language processing technology; for image data, use convolutional neural network to extract image feature vectors; for numerical data, directly select or calculate relevant features. The extracted feature vectors are recorded as , and ; Among them, the subscript Represent text, image and numerical features respectively, Indicates the sample index, and the sample is the data in the information to be screened. The specific extraction process is: Text feature extraction: Use a pre-trained word embedding model to convert text into a vector. For example, use Word2Vec, BERT, and other models to generate vectors with dimensions of The word vector of ; Image feature extraction: Use convolutional neural networks (such as ResNet or VGG) to extract image features and obtain a dimension of Vector ; Numerical feature extraction: Calculate the statistical features of numerical data directly, and obtain the dimension Vector .

[0017] in, For sample The text feature vector of is the dimension of the text feature, that is, the number of components of the text feature vector; represents the field of real numbers; For sample The image feature vector of is the dimension of the image feature, that is, the number of components of the image feature vector; For sample The numerical eigenvector of ; is the dimension of the numerical feature, that is, the number of components of the numerical feature vector.

[0018] Normalize each modal feature vector: The extracted modal feature vectors have different value ranges and distributions. In order to make them comparable in the same feature space, each type of feature vector needs to be normalized. Definition , the normalization process adopts maximum and minimum normalization.

[0019] When using maximum and minimum normalization, for the text feature vector dimension , calculate the normalized value , the calculation formula is: ;in, For sample The text feature vector of The normalized value of the feature components; For sample The text feature vector of The original value of the feature components; For all samples The minimum value of the characteristic components; For all samples The maximum value of the feature components.

[0020] The image feature vector and numerical feature vector are also normalized to ensure that each modal feature is analyzed at a unified scale.

[0021] After normalization, each modal eigenvector is recorded as , and , corresponding to the normalized text feature vector, image feature vector and numerical feature vector respectively.

[0022] Use feature fusion method to combine the features of each modality into a joint multimodal feature vector: Based on the normalized multimodal feature vector, the feature fusion method is used to integrate the features from different modalities into a joint feature vector for unified analysis. It can be expressed as: .

[0023] The fused joint feature vector Dimensions for: , which retains the information characteristics from each mode and provides a sufficient data basis for subsequent nonlinear mapping and density evaluation; among them, is the total dimension of the multimodal joint feature vector.

[0024] Based on the joint multimodal feature vector, a feature projection network is used for nonlinear mapping to generate high-dimensional feature representation: , through the feature projection network, nonlinear mapping is performed to generate a new high-dimensional feature representation. Assume that the feature projection network is a multi-layer feedforward neural network with input , the output is a high-dimensional feature vector ,in, is the dimension of the output high-dimensional feature vector.

[0025] The mapping process of the feature projection network can be expressed as: ;in, For sample The high-dimensional feature representation vector generated after the feature projection network; The feature projection network The weight matrix of the layer has the dimension ; The feature projection network The bias vector of the layer, dimension is ; The feature projection network The number of neurons in the layer, indicating the hierarchical structure of the feature projection network; represents a non-linear activation function.

[0026] Kernel density estimation is used to calculate the distribution density of the high-dimensional feature representation in the feature space after feature projection: high-dimensional feature representation is obtained After that, in order to calculate the distribution density of the high-dimensional feature representation after feature projection in the feature space, the kernel density estimation method is used. The kernel density estimation method calculates the density function by smoothing the distribution of data points in the feature space, which is specifically: Define a point in high-dimensional feature space as , then the kernel density estimate It can be defined as: ;in, For point The kernel density estimate at represents the distribution density in the feature space; is the number of samples; is the bandwidth parameter, which controls the smoothness of the kernel function; is the dimension of the high-dimensional feature vector; is the kernel function; For sample The high-dimensional feature representation vector generated after the feature projection network; is a point in the high-dimensional feature space.

[0027] The larger the kernel density estimate, the higher the distribution density of the multimodal feature vector in the high-dimensional feature space, which means that more other samples are gathered around the location of the sample, indicating that its feature vector has a high similarity and correlation with a large number of data points. Specifically, a high kernel density estimate reflects a high density of data points in the high-dimensional feature space, representing a normal or expected behavior pattern with a high degree of credibility. In computer information screening, samples located in high-density areas are considered to meet normal standards.

[0028] Specifically, the multimodal feature vectors of the information to be screened are mapped to a reduced dimension, the distance matrix between the feature clusters after the reduced dimension is analyzed, and the degree of local folding of the feature space is evaluated, including: Select a dimensionality reduction method based on geometry preservation to map the high-dimensional feature vector set to a low-dimensional feature space: Establish a high-dimensional feature vector set , composed of high-dimensional feature vectors of each mode Based on the high-dimensional feature vector set, a dimensionality reduction method that can maintain the geometric structure of the original data is selected, such as principal component analysis. Principal component analysis projects the high-dimensional feature vector set into a low-dimensional space through linear transformation, retaining the direction with the largest data variance. Let the feature vector after dimensionality reduction be , then the dimension reduction mapping function can be expressed as: ;in, For the The eigenvector after modal dimensionality reduction; It is a dimensionality reduction mapping matrix used to map high-dimensional features to low-dimensional space.

[0029] Calculate the change in Euclidean distance between feature clusters before and after dimensionality reduction, and establish a distance matrix comparison before and after dimensionality reduction: For the high-dimensional feature vector set before and after dimensionality reduction and , respectively calculate the Euclidean distance between feature clusters. Suppose the distance matrix in the original high-dimensional space is , whose elements Indicates and The Euclidean distance between feature clusters: ;in, In high-dimensional space and The Euclidean distance between feature clusters; In high-dimensional space feature vectors; In high-dimensional space feature vectors.

[0030] The distance matrix in the low-dimensional space after dimensionality reduction is , whose elements Indicates and The Euclidean distance between feature clusters: ; In low-dimensional space and The Euclidean distance between feature clusters; In low-dimensional space feature vectors; In low-dimensional space feature vectors.

[0031] By comparing the distance matrix in the original high-dimensional space and the distance matrix in the low-dimensional space after dimensionality reduction, a comparison of the distance matrices before and after dimensionality reduction is established to evaluate the impact of the dimensionality reduction process on the distance between feature clusters.

[0032] Based on the change rate of the distance matrix, the degree of separation between feature clusters after dimensionality reduction is evaluated: In order to quantify the impact of dimensionality reduction on the degree of separation between feature clusters, the distance change rate is introduced. , defined as the ratio of the distance after dimensionality reduction to the distance before dimensionality reduction: ;in, Indicates and The rate of change of distance between feature clusters.

[0033] By calculating the distance change rate of all feature cluster pairs , can evaluate the degree of separation between feature clusters after dimensionality reduction. Specifically, if the distance change rate If it is close to 1, it means that the relative distance between the pair of feature clusters remains unchanged during the dimensionality reduction process; if the distance change rate If the distance change rate is greater than 1, it means that the distance between the feature clusters increases after dimensionality reduction, and the degree of separation increases. If it is less than 1, it means that the distance is reduced and the degree of separation is reduced.

[0034] Count the overlap of feature points in the local neighborhood and evaluate the local folding degree of the feature space: Local folding means that in the process of dimensionality reduction, feature vectors that were originally adjacent in the high-dimensional space are no longer adjacent in the low-dimensional space, resulting in local distortion of the feature space structure. In order to evaluate the degree of local folding, it is necessary to define the local neighborhood of each feature point. Assume that in the high-dimensional space, the high-dimensional feature vector of The neighbor set is ; In low-dimensional space, the eigenvector of The neighbor set is .

[0035] The overlap index is defined as the high-dimensional feature vector And the feature vector after dimension reduction The intersection size and high-dimensional feature vector The ratio of size is calculated as follows: ;in, is the overlap index; is the number of samples; is a high-dimensional feature vector; is the feature vector after dimensionality reduction.

[0036] The larger the overlap index, the greater the degree of local collapse of the feature space, which means that in the process of dimensionality reduction mapping, the feature clusters that originally had obvious distinction in the high-dimensional space have significant overlap in the low-dimensional space. The boundaries between different categories or different feature clusters become blurred, making it difficult to effectively distinguish and identify them in the low-dimensional space. This local collapse phenomenon may lead to a decline in the performance of downstream tasks such as classification and clustering because the discriminative ability of the features is weakened. In addition, a larger overlap index may also reflect that the selected dimensionality reduction method is insufficient in maintaining the original data structure and fails to fully retain the key geometric and topological information in the high-dimensional space. Therefore, when the overlap index is large, it is usually necessary to re-evaluate and select a more suitable dimensionality reduction method, or adjust the parameters of the existing method, in order to better maintain the structural integrity and discriminative ability of the feature space while reducing the dimension, thereby improving the effect of subsequent data processing and analysis tasks. Furthermore, a larger overlap index may indicate the need to combine other technical means, such as feature selection, feature extraction or data enhancement, to optimize the dimensionality reduction effect, ensure that in the process of multimodal feature fusion, the features of each modality can effectively cooperate in the low-dimensional space, and improve the quality of the overall data representation and application performance.

[0037] Specifically, based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the local folding degree of the feature space, determining whether the distortion degree of the feature space exceeds a preset threshold includes: Preset the kernel density estimate threshold and compare the kernel density estimate with the kernel density estimate threshold: When the kernel density estimate is greater than or equal to the kernel density estimate threshold, it indicates that the feature vectors are highly clustered in the high-dimensional feature space, which usually indicates that the feature vectors have a high frequency and consistency in practical applications. Density estimates above the threshold indicate that the similarity between data points is high, and the screening system can regard these feature vectors as stable and reliable data groups, which helps reduce false positives and false correlations; When the kernel density estimate is less than the kernel density estimate threshold, it indicates that the feature vectors are dispersed in the high-dimensional feature space; low-density areas usually correspond to uncommon or marginal data distributions, which may contain noise, erroneous data, or rare events; when the screening system detects these low-density areas, further analysis is required to determine their nature and source, and additional measures may be required to verify or exclude them. This mechanism helps to improve the sensitivity of the screening system, ensuring that potential abnormalities can be discovered and handled in a timely manner, thereby improving the accuracy and reliability of the overall screening; The setting of the kernel density estimate threshold is usually based on statistical analysis, empirical rules or specific application requirements to ensure that the screening system can effectively identify data areas with significant density differences. Determining an appropriate kernel density estimate threshold requires considering factors such as the overall distribution characteristics of the data set, the number of samples, and the feature dimension.

[0038] Preset the overlap coefficient threshold and compare the overlap coefficient with the overlap coefficient threshold: When the overlap coefficient is greater than the overlap coefficient threshold, it means that the local folding degree of the feature space is high, indicating that there is a high degree of local folding in the feature space after dimensionality reduction, and the overlap between feature clusters exceeds the expected acceptable range; it means that some key structural information in the original high-dimensional space has not been effectively retained during the dimensionality reduction mapping process, which may lead to a decrease in the discriminative ability of the data; therefore, when the overlap index exceeds the threshold, measures need to be taken to improve the dimensionality reduction effect. Specific measures include reselecting the dimensionality reduction algorithm, such as switching from linear dimensionality reduction methods to nonlinear dimensionality reduction methods to better capture the complex structure of the data; adjusting the parameters of the dimensionality reduction algorithm, such as increasing the number of principal components retained or adjusting the neighborhood size to enhance the preservation of the local structure of the data; or using a combination of multiple dimensionality reduction methods to comprehensively utilize the advantages of different methods; When the overlap coefficient is less than or equal to the overlap coefficient threshold, it means that the local folding degree of the feature space is low, which means that in the dimensionality reduction mapping process, the key structural information in the original high-dimensional space is well preserved, and the distinction between different categories or different feature clusters is effectively maintained. The reduced dimensionality feature space can better reflect the intrinsic structure and similarity relationship of the original data. At the same time, the low local folding degree also indicates that the selected dimensionality reduction method and its parameter setting are reasonable, which can maximize the retention of the data's discriminative ability and structural information while reducing the data dimension.

[0039] The setting of the overlap coefficient threshold can be set based on historical data experience and the theoretical performance of the dimensionality reduction algorithm, and is usually expressed as a value between 0 and 1. The overlap index threshold is used to provide a standard line for judging the quality of dimensionality reduction, reflecting the acceptable range of the degree of feature space folding.

[0040] When the kernel density estimation value is less than the kernel density estimation value threshold and the overlap coefficient is greater than the overlap coefficient threshold, it is determined that the distortion degree of the feature space exceeds the preset threshold; otherwise, it is determined that the distortion degree of the feature space does not exceed the preset threshold.

[0041] Specifically, by analyzing the mutual information between the original features and the reconstructed features of the information to be screened, the feature loss degree of the information to be screened is evaluated, including: Extract the original feature vector from the information to be screened, and generate the corresponding reconstructed feature vector through the pre-trained model or autoencoder: the original feature vector is the joint feature vector after multimodal data fusion .

[0042] Use the autoencoder to reconstruct the joint feature vector and generate the corresponding reconstructed feature vector. The specific steps are as follows: Encoding process: the original feature vector Input into the autoencoder and obtain the potential feature representation through nonlinear mapping: ;in, For sample The latent feature representation of is the encoder function; is the parameter of the encoder; The dimension represented by the latent feature.

[0043] Decoding process: Representing latent features Input the decoder to generate the reconstructed feature vector: ;in, To reconstruct the feature vector; is the decoder function; is the parameter of the decoder; is the dimension of the reconstructed feature vector.

[0044] Define the joint probability distribution between the original feature vector and the reconstructed feature vector: In order to calculate the mutual information between the original feature and the reconstructed feature, it is necessary to define the joint probability distribution, specifically: Defining random variables Represents the original feature vector The value of random variable Represents the reconstructed feature vector The value of , then the random variable and The joint probability distribution of is: ;in, The original eigenvector and the reconstructed eigenvector have specific values The joint probability distribution under .

[0045] The marginal probability distribution of the original eigenvector is defined as: ;in, is the marginal probability distribution of the original eigenvector.

[0046] The marginal probability distribution of the reconstructed eigenvector is defined as: ;in, is the marginal distribution probability of the reconstructed eigenvector.

[0047] The entropy estimation method is used to calculate the joint entropy and marginal entropy of the original features and the reconstructed features, and the mutual information value between the original features and the reconstructed features is obtained: the entropy concept in information theory is used to quantify the relationship between the original features and the reconstructed features, and the mutual information between the original features and the reconstructed features is calculated, including: Joint Entropy Calculation: Joint Entropy Weighing random variables and The joint uncertainty of is defined as: ;in, Represents the original feature vector and reconstruct the eigenvector The joint entropy of .

[0048] By traversing all possible Yes, according to its joint probability distribution Calculate uncertainty.

[0049] Marginal entropy calculation: Calculate the marginal entropy of the original eigenvector and the reconstructed eigenvector respectively: Marginal entropy of the original eigenvector: ;in, represents the marginal entropy of the original eigenvector; Reconstruct the marginal entropy of the eigenvector: ;in, represents the marginal entropy of the reconstructed eigenvector.

[0050] Mutual information value calculation: Mutual information is defined as the amount of information shared by the original feature vector and the reconstructed feature vector. Its calculation formula is: ;in, is the mutual information value between the original feature vector and the reconstructed feature vector.

[0051] Compare the calculated mutual information value with the preset threshold to evaluate the degree of information loss in feature reconstruction: set up is the minimum acceptable mutual information value for the feature reconstruction process, i.e., the mutual information threshold. The mutual information threshold is determined based on historical data, experimental results, or domain expert recommendations.

[0052] If the mutual information value is greater than or equal to the mutual information threshold, it is considered that the feature reconstruction is good and the information loss can be ignored.

[0053] If the mutual information value is less than the mutual information threshold, it means that there is obvious information loss in the feature reconstruction process, and the feature loss index needs to be further calculated: ;in, is the feature loss index, ranging from 0 to 1; is the mutual information value between the original eigenvector and the reconstructed eigenvector; is the mutual information threshold.

[0054] The larger the feature loss index, the more serious the feature loss of the information to be screened. A larger feature loss index indicates that the reconstructed feature vector fails to effectively retain the important information in the original feature vector, which may lead to performance degradation in subsequent data analysis or classification tasks. This information loss may be due to a variety of reasons, including insufficient capacity of the feature extraction model, insufficient training of the autoencoder or decoder, or over-simplification in the feature mapping process. In addition, a larger feature loss index may also reflect the presence of complex nonlinear relationships or high-dimensional structures in the original data, and the current reconstruction method fails to fully capture and retain these complex features. Therefore, when the feature loss index is large, it may be necessary to re-evaluate and optimize the feature extraction and reconstruction methods, such as increasing the complexity of the model, adjusting the dimension of the latent feature space, or adopting more advanced dimensionality reduction techniques to improve the fidelity of the reconstructed features.

[0055] Specifically, the distribution density of multimodal feature vectors in high-dimensional feature space, the degree of local folding of feature space, and the degree of feature loss of the information to be screened are comprehensively analyzed to evaluate the reliability of the information screening results and determine whether the screening strategy needs to be adjusted, including: The kernel density estimate corresponding to the distribution density of the multimodal feature vector in the high-dimensional feature space, the overlap index corresponding to the local folding degree of the feature space, and the feature loss index corresponding to the feature loss degree of the information to be screened are normalized respectively. The normalized kernel density estimate, overlap index and feature loss index are calculated to obtain the reliability index, which is used to evaluate the reliability of the information screening results and determine whether the screening strategy needs to be adjusted.

[0056] The calculation formula of reliability index is: ;in, is the reliability index; For point The kernel density estimate at ; is the feature loss index; is the overlap index; is the hyperparameter of the feature loss index; is a hyperparameter of the overlap index.

[0057] A reliability index threshold is preset, and the reliability index is compared with the reliability index threshold: When the reliability index is greater than or equal to the reliability index threshold, it indicates that the reliability of the information screening result is high, and there is no need to adjust the screening strategy; When the reliability index is less than the reliability index threshold, it indicates that the reliability of the information screening result is low, and the screening strategy needs to be adjusted, including: increasing the number of training samples, optimizing feature extraction and dimensionality reduction algorithms, improving the complexity of the autoencoder, introducing a more sophisticated screening threshold, etc. By optimizing these screening strategies, feature integrity can be improved, feature loss index can be reduced, and the reliability index can be returned to above the reliability threshold, ensuring the robustness and accuracy of the screening system.

[0058] The determination of the reliability index threshold needs to be based on historical data analysis. Through statistical evaluation of past screening results, we can understand the screening effects and accuracy levels corresponding to different reliability index thresholds, and select a reasonable threshold that can effectively distinguish high reliability from low reliability. Secondly, business needs and application scenarios are also important bases for setting thresholds. For example, in scenarios with high security requirements, such as financial fraud detection or security monitoring, it may be necessary to set a higher reliability index threshold to ensure the high reliability of screening results and reduce the risk of false positives and false negatives; in scenarios with relatively low accuracy requirements, such as large-scale information filtering, the threshold can be appropriately lowered to improve screening efficiency. Expert experience and industry standards are also important reference factors. Through collaboration with domain experts and combining industry best practices and standards, a scientific and reasonable reliability index threshold can be determined.

[0059] Example 2 The difference between Example 2 of the present invention and Example 1 is that this example introduces a computer information screening system based on artificial intelligence.

[0060] Figure 2 A structural schematic diagram of a computer information screening system based on artificial intelligence of the present invention is given, and a computer information screening system based on artificial intelligence includes a distribution density assessment module, a folding degree assessment module, a distortion degree determination module, a characteristic loss assessment module and a reliability assessment module; The distribution density evaluation module analyzes the multimodal feature vectors of the information to be screened and evaluates the distribution density of the multimodal feature vectors in the high-dimensional feature space through the feature projection network; The folding degree evaluation module performs dimension reduction mapping on the multimodal feature vectors of the information to be screened, analyzes the distance matrix between feature clusters after dimension reduction, and evaluates the local folding degree of the feature space; The distortion degree determination module determines whether the distortion degree of the feature space exceeds a preset threshold based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the local folding degree of the feature space; When the degree of feature space distortion exceeds a preset threshold, the feature loss assessment module assesses the degree of feature loss of the information to be screened by analyzing the mutual information between the original features and the reconstructed features of the information to be screened; The reliability assessment module conducts a comprehensive analysis of the distribution density of the modal feature vector in the high-dimensional feature space, the degree of local folding of the feature space, and the degree of feature loss of the information to be screened, evaluates the reliability of the information screening results, and determines whether the screening strategy needs to be adjusted.

[0061] The above formulas are dimensionless and numerical calculations. The formula is a formula obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions. The parts not described in the present invention are applicable to the prior art.

Claims

1. A computer information screening method based on artificial intelligence, characterized in that: The steps include: Analyze the multimodal feature vectors of the information to be screened, and evaluate the distribution density of the multimodal feature vectors in the high-dimensional feature space through the feature projection network; Perform dimension reduction mapping on the multimodal feature vectors of the information to be screened, analyze the distance matrix between feature clusters after dimension reduction, and evaluate the degree of local folding of the feature space; Based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the local folding degree of the feature space, determining whether the distortion degree of the feature space exceeds a preset threshold; When the degree of feature space distortion exceeds a preset threshold, the degree of feature loss of the information to be screened is evaluated by analyzing the mutual information between the original features and the reconstructed features of the information to be screened; The distribution density of the modal feature vector in the high-dimensional feature space, the degree of local folding of the feature space, and the degree of feature loss of the information to be screened are comprehensively analyzed to evaluate the reliability of the information screening results and determine whether the screening strategy needs to be adjusted.

2. The computer information screening method based on artificial intelligence according to claim 1, characterized in that: Analyze the multimodal feature vectors of the information to be screened, and evaluate the distribution density of the multimodal feature vectors in the high-dimensional feature space through the feature projection network, specifically: Obtain a multimodal data set to be screened and extract the original feature vectors under each modality, including text, image and numerical feature vectors; Normalize each modal eigenvector; The feature fusion method is used to combine the features of each modality into a joint multimodal feature vector; Based on the joint multimodal feature vector, a feature projection network is used for nonlinear mapping to generate high-dimensional feature representation; Kernel density estimation is used to calculate the distribution density of high-dimensional feature representation in the feature space after feature projection.

3. The computer information screening method based on artificial intelligence according to claim 2 is characterized in that: The multimodal feature vectors of the information to be screened are mapped to a reduced dimension, the distance matrix between the feature clusters after the reduced dimension is analyzed, and the degree of local folding of the feature space is evaluated, specifically: Select a dimensionality reduction method based on geometry preservation to map the high-dimensional feature vector set to a low-dimensional feature space; Calculate the change in Euclidean distance between feature clusters before and after dimensionality reduction, and establish a distance matrix comparison before and after dimensionality reduction; Based on the rate of change of the distance matrix, the degree of separation between feature clusters after dimensionality reduction is evaluated; The overlap degree of feature points in the local neighborhood is counted to evaluate the local folding degree of the feature space.

4. The computer information screening method based on artificial intelligence according to claim 3 is characterized in that: Based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the degree of local folding of the feature space, it is determined whether the distortion degree of the feature space exceeds a preset threshold, specifically: Preset the kernel density estimate threshold and compare the kernel density estimate with the kernel density estimate threshold: When the kernel density estimate is greater than or equal to the kernel density estimate threshold, it indicates that the feature vectors are highly clustered in the high-dimensional feature space; When the kernel density estimate is less than the kernel density estimate threshold, it means that the feature vector is dispersed in the high-dimensional feature space; Preset the overlap coefficient threshold and compare the overlap coefficient with the overlap coefficient threshold: When the overlap coefficient is greater than the overlap coefficient threshold, it indicates that the local folding degree of the feature space is high; When the overlap coefficient is less than or equal to the overlap coefficient threshold, it indicates that the local folding degree of the feature space is low; When the kernel density estimate is less than the kernel density estimate threshold, and the overlap coefficient is greater than the overlap coefficient threshold, it is determined that the distortion degree of the feature space exceeds the preset threshold; otherwise, it is determined that the distortion degree of the feature space does not exceed the preset threshold.

5. The computer information screening method based on artificial intelligence according to claim 4 is characterized in that: By analyzing the mutual information between the original features and the reconstructed features of the information to be screened, the degree of feature loss of the information to be screened is evaluated, specifically: Extract the original feature vector from the information to be screened, and generate the corresponding reconstructed feature vector through a pre-trained model or an autoencoder; Define the joint probability distribution between the original feature vector and the reconstructed feature vector; The entropy estimation method is used to calculate the joint entropy and marginal entropy of the original features and the reconstructed features, and the mutual information value between the original features and the reconstructed features is obtained; The calculated mutual information value is compared with the preset threshold to evaluate the degree of information loss in feature reconstruction: feature loss index, calculated as: ;in, is the feature loss index; is the mutual information value between the original eigenvector and the reconstructed eigenvector; is the mutual information threshold.

6. The computer information screening method based on artificial intelligence according to claim 5 is characterized in that: Comprehensively analyze the distribution density of multimodal feature vectors in high-dimensional feature space, the degree of local folding of feature space, and the degree of feature loss of the information to be screened, evaluate the reliability of the information screening results, and determine whether the screening strategy needs to be adjusted. Specifically: The normalized kernel density estimate, overlap index, and feature loss index are calculated to obtain the reliability index, which is calculated as follows: ;in, is the reliability index; For point The kernel density estimate at ; is the feature loss index; is the overlap index; is the hyperparameter of the feature loss index; is the hyperparameter of the overlap index; A reliability index threshold is preset, and the reliability index is compared with the reliability index threshold: When the reliability index is greater than or equal to the reliability index threshold, it indicates that the reliability of the information screening result is high and there is no need to adjust the screening strategy; When the reliability index is less than the reliability index threshold, it indicates that the reliability of the information screening result is low and the screening strategy needs to be adjusted.

7. An artificial intelligence-based computer information screening system, used to implement an artificial intelligence-based computer information screening method according to any one of claims 1 to 6, characterized in that: It includes a distribution density assessment module, a folding degree assessment module, a distortion degree determination module, a characteristic loss assessment module and a reliability assessment module; The distribution density evaluation module analyzes the multimodal feature vectors of the information to be screened and evaluates the distribution density of the multimodal feature vectors in the high-dimensional feature space through the feature projection network; The folding degree evaluation module performs dimension reduction mapping on the multimodal feature vectors of the information to be screened, analyzes the distance matrix between feature clusters after dimension reduction, and evaluates the local folding degree of the feature space; The distortion degree determination module determines whether the distortion degree of the feature space exceeds a preset threshold based on the distribution density of the multimodal feature vector in the high-dimensional feature space and the local folding degree of the feature space; When the degree of feature space distortion exceeds a preset threshold, the feature loss assessment module assesses the degree of feature loss of the information to be screened by analyzing the mutual information between the original features and the reconstructed features of the information to be screened; The reliability assessment module conducts a comprehensive analysis of the distribution density of the modal feature vector in the high-dimensional feature space, the degree of local folding of the feature space, and the degree of feature loss of the information to be screened, evaluates the reliability of the information screening results, and determines whether the screening strategy needs to be adjusted.

Images