A big data analysis system and method for computer images

Abstract

This invention provides a big data analysis system and method for computer images, relating to the field of image analysis technology. This invention improves the accuracy, robustness, and automation of image analysis by using a feature space reference model constructed from an external image database to guide the clustering analysis process of a target image set. By generating external pattern clusters and calculating their statistical characteristics, a semantically rich reference framework is established, providing an objective standard for the analysis of target images. By dynamically linking clustering parameters with probability density calculations, a probability density function and a likelihood function are constructed, transforming the selection of clustering parameters into a quantifiable mathematical optimization problem. This allows the parameter optimization process to automatically search for the optimal solution through iterative algorithms, rather than relying on human experience, ensuring the objectivity and repeatability of the analysis results.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to a big data analysis system and method for computer images. Background Technology

[0002] In the current field of computer image analysis technology, traditional methods are typically limited to processing specific target image sets independently, lacking the ability to utilize abundant external image data resources to guide and analyze target images. This results in significant deficiencies in the accuracy and robustness of image analysis results, especially when dealing with complex and variable image content. Traditional methods often rely on human experience to preset key parameters, making the analysis process highly subjective and difficult to adapt to the inherent characteristics of different image sets. Furthermore, due to the lack of external prior knowledge, traditional methods exhibit poor adaptability to problems such as variations in image noise distribution and class imbalance, making it difficult to learn general patterns from massive amounts of existing image data and transform them into effective analytical bases. The typical approach is to directly extract features from the target image and then perform cluster analysis—a closed-loop processing method that ignores the statistical regularities and distribution information provided by external big data. This limits the theoretical foundation and performance ceiling of image analysis models. On the other hand, although big data resources are becoming increasingly abundant, existing technologies have not provided a systematic solution on how to transform these resources into a quantifiable knowledge model that can directly guide specific image analysis tasks. As a result, the value of big data resources has not been fully explored and applied in the field of computer image analysis. Therefore, there is an urgent need for a new technical solution that can introduce the statistical characteristics of external big data into the image analysis process, thereby achieving more intelligent and accurate big data analysis of computer images.

[0003] Therefore, it is necessary to provide a big data analysis system and method for computer images to solve the above-mentioned technical problems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a big data analysis system and method for computer images, achieving the beneficial effect of more intelligent and accurate big data analysis of computer images.

[0005] This invention provides a big data analysis method for computer images, comprising:

[0006] S1: Use a pre-trained deep convolutional neural network to extract the feature vector of each image in the target computer image set, and form a target feature vector set;

[0007] S2: Generate external pattern clusters from the large image database using a clustering algorithm, calculate the cluster center vector and covariance matrix of each external pattern cluster, and construct an external feature space reference model based on all external pattern clusters;

[0008] S3: Construct a probability density function with clustering parameters within a predefined range as independent variables and the set of probability density values ​​obtained by matching the clustering results of the target feature vector set based on these clustering parameters with the external feature space reference model as dependent variables.

[0009] S4: Based on the probability density function, construct the likelihood function of the target feature vector set relative to the external feature space reference model;

[0010] S5: Use an iterative optimization algorithm to search for clustering parameters within a predefined range that maximize the likelihood function and use them as the final clustering parameters;

[0011] S6: Use the final clustering parameters to perform unsupervised clustering analysis on the target feature vector set to obtain the clustering analysis results of the target computer image.

[0012] Preferably, in step S2, the step of generating the external pattern cluster includes:

[0013] The feature vectors of each image in the image database are extracted using a pre-trained deep convolutional neural network to form an external feature vector set.

[0014] An unsupervised clustering analysis was performed on the set of external feature vectors using a clustering algorithm, and the resulting clusters were taken as external pattern clusters.

[0015] Preferably, in step S2, the cluster center vector of each external pattern cluster is the mean vector obtained by calculating the arithmetic mean of all feature vectors in the external pattern cluster.

[0016] Preferably, in step S2, the covariance matrix of each external pattern cluster is the sample covariance matrix calculated based on all the eigenvectors constituting the external pattern cluster.

[0017] Preferably, after step S2 and before step S3, the method further includes a step of dimensionality reduction of the external feature space reference model when the dimension of the cluster center vector in the external feature space reference model is higher than a preset dimension threshold. The dimensionality reduction adopts the principal component analysis method, and the cluster center vector and covariance matrix are recalculated for the dimensionality-reduced external feature space reference model.

[0018] Preferably, in step S3, the lower limit of the predefined value range is 2, and the upper limit of the predefined value range is determined based on the square root of the size of the target feature vector set.

[0019] Preferably, in step S3, the probability density function calculates the probability density value based on a multivariate Gaussian distribution, wherein the mean vector of the multivariate Gaussian distribution is the cluster center vector of the external model cluster, and the covariance matrix is ​​the covariance matrix of the external model cluster.

[0020] Preferably, in step S4, the likelihood function is defined as the sum of the natural logarithms of all probability density values ​​output by the probability density function.

[0021] Preferably, in step S5, the convergence condition of the iterative optimization algorithm is that the increase in the likelihood function value after a preset number of consecutive iterations is less than a preset increase threshold.

[0022] This invention provides a big data analysis system for computer images, applied to a big data analysis method for computer images, comprising:

[0023] The target feature extraction module is used to extract the feature vector of each image in the target computer image set using a pre-trained deep convolutional neural network, forming a target feature vector set.

[0024] The external reference model construction module is used to generate external pattern clusters from a large image database using a clustering algorithm, and to calculate the cluster center vector and covariance matrix of each external pattern cluster, and to construct an external feature space reference model based on all external pattern clusters.

[0025] The probability density function definition module is used to construct a probability density function with clustering parameters within a predefined range as independent variables and the set of probability density values ​​obtained by matching the clustering results of the target feature vector set based on the clustering parameters with the external feature space reference model as dependent variables.

[0026] The likelihood function construction module is used to construct the likelihood function of the target feature vector set relative to the external feature space reference model based on the probability density function.

[0027] The parameter optimization search module is used to search for clustering parameters that maximize the likelihood function within a predefined range of clustering parameters using an iterative optimization algorithm, and then use these as the final clustering parameters.

[0028] The clustering analysis execution module is used to perform unsupervised clustering analysis on the target feature vector set using the final clustering parameters to obtain the clustering analysis results of the target computer image.

[0029] Compared with related technologies, the big data analysis system and method for computer images provided by this invention have the following beneficial effects:

[0030] This invention improves the accuracy, robustness, and automation of image analysis by using a feature space reference model constructed from an external image database to guide the clustering analysis process of a target image set. Traditional methods typically perform isolated analysis only on the target image set itself, lacking the ability to utilize external prior knowledge. In contrast, this invention establishes a semantically rich reference framework by generating external pattern clusters and calculating their statistical properties. This framework provides an objective benchmark for the analysis of target images. Specifically, this invention constructs a probability density function and a likelihood function by dynamically linking clustering parameters with probability density calculations, thereby transforming the selection of clustering parameters into a quantifiable mathematical optimization problem. This allows the parameter optimization process to automatically search for the optimal solution through iterative algorithms instead of relying on human experience, ensuring the objectivity and repeatability of the analysis results. Furthermore, due to the introduction of statistical regularities from external big data, this invention exhibits stronger adaptability to processing noisy images, imbalanced images, and images with complex backgrounds, effectively identifying potential patterns and distribution features in target images. Attached Figure Description

[0031] Figure 1 This is a flowchart of a big data analysis method for computer images according to the present invention;

[0032] Figure 2 This is a module structure diagram of a big data analysis system for computer images according to the present invention. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not all structures. Moreover, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0034] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as being processed sequentially, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0035] Example 1

[0036] A big data analysis method for computer images, in its specific implementation process, such as... Figure 1 As shown, it illustrates a flowchart of a big data analysis method for computer images, including:

[0037] Step S1: Use a pre-trained deep convolutional neural network to extract the feature vector of each image in the target computer image set, and form a target feature vector set.

[0038] In the specific implementation process, firstly, for example, the pre-trained deep convolutional neural network uses a ResNet-50 model trained on the ImageNet dataset. This ResNet-50 model contains 50 deep convolutional layers, including four sets of residual modules. Each set of modules contains multiple convolutional layers, batch normalization layers, and ReLU activation functions. Global average pooling layers are used instead of fully connected layers before the final output to preserve spatial feature information. The training process of this ResNet-50 model uses a stochastic gradient descent optimizer with an initial learning rate of 0.1, decaying by a factor of one-tenth every 30 training epochs, for a total of 120 training epochs. The cross-entropy loss function is used as the optimization objective during training, and weights are used... To prevent overfitting, decay regularization is used. During the training data preprocessing stage, all input images are uniformly scaled to 256 x 256 pixels and then cropped to 224 x 224 pixels. Pixel values ​​are also normalized to ensure a normal distribution with a mean of 0.5 and a standard deviation of 0.5. The same preprocessing procedure is followed for each image in the target computer image set. The processed images are then input into the trained ResNet-50 network to extract feature maps from the output of the last convolutional layer. These feature maps are then transformed into 1024-dimensional feature vectors using global average pooling. Finally, the feature vectors of all images are combined in sequence to form the target feature vector set, which serves as the basic input data for subsequent steps.

[0039] Step S2: Generate external pattern clusters from the large image database using a clustering algorithm, and calculate the cluster center vector and covariance matrix of each external pattern cluster. Construct an external feature space reference model based on all external pattern clusters.

[0040] Specifically, in step S2, the generation of the external pattern cluster includes:

[0041] The feature vectors of each image in a large image database are extracted using a pre-trained deep convolutional neural network to form an external feature vector set.

[0042] An unsupervised clustering analysis of the external feature vector set is performed using a clustering algorithm, and the resulting clusters are taken as external pattern clusters.

[0043] Specifically, in step S2, the cluster center vector of each external pattern cluster is the mean vector obtained by calculating the arithmetic mean of all feature vectors in the external pattern cluster.

[0044] Specifically, in step S2, the covariance matrix of each external pattern cluster is the sample covariance matrix calculated based on all the eigenvectors constituting that external pattern cluster.

[0045] In the specific implementation process, firstly, feature vectors of all images need to be extracted from a large image database. This database contains a vast amount of diverse image resources, including, but not limited to, the publicly available ImageNet dataset. Specifically, a ResNet-50 model trained on the ImageNet dataset is used to extract features from all images. All extracted feature vectors are combined to form an external feature vector set. Then, an unsupervised clustering algorithm is used to analyze this external feature vector set. For example, the K-Means clustering algorithm is used, including but not limited to. The K-Means clustering algorithm randomly initializes cluster centers, then iteratively calculates the Euclidean distance from each feature vector to each cluster center and assigns it to the nearest cluster. The mean of each cluster is then recalculated as the new cluster center. This process is repeated until the cluster assignments no longer change or the preset maximum number of iterations is reached, thereby dividing the entire external feature vector set into multiple clusters. These clusters are formally defined as external feature vector sets. Next, for each external pattern cluster, its cluster center vector is calculated. Specifically, all feature vectors contained in the external pattern cluster are summed and divided by the total number of vectors to obtain the arithmetic mean vector. This arithmetic mean vector is the cluster center vector representing the core feature of the external pattern cluster. Then, the covariance matrix is ​​calculated based on all feature vectors within the same pattern cluster. First, the difference matrix between each feature vector and the cluster center vector is calculated. Then, these difference matrices are transposed, multiplied, and all products are summed. Finally, the result is divided by the number of vectors minus 1 to achieve unbiased estimation, thus obtaining the sample covariance matrix of the external pattern cluster, i.e., the covariance matrix of the external pattern cluster. This covariance matrix describes the distribution shape of feature vectors within the external pattern cluster and the linear correlation between features in each dimension. Finally, the cluster center vectors and covariance matrices of all external pattern clusters are combined to form a complete external feature space reference model. This external feature space reference model serves as a reference framework containing rich statistical information, providing a benchmark and basis for subsequent analysis.

[0046] Step S3: Using clustering parameters within a predefined range as independent variables, and the set of probability density values ​​obtained by matching the clustering results of the target feature vector set with the external feature space reference model after performing clustering analysis based on these clustering parameters as dependent variables, construct a probability density function.

[0047] Specifically, after step S2 and before step S3, the method further includes a step of dimensionality reduction of the external feature space reference model when the dimension of the cluster center vector in the external feature space reference model is higher than a preset dimension threshold. The dimensionality reduction adopts the principal component analysis method, and the cluster center vector and covariance matrix are recalculated for the dimensionality-reduced external feature space reference model.

[0048] Specifically, in step S3, the lower limit of the predefined value range is 2, and the upper limit of the predefined value range is determined based on the square root of the size of the target feature vector set.

[0049] Specifically, in step S3, the probability density function calculates the probability density value based on the multivariate Gaussian distribution, where the mean vector of the multivariate Gaussian distribution is the cluster center vector of the external model cluster, and the covariance matrix is ​​the covariance matrix of the external model cluster.

[0050] In the specific implementation process, firstly, a dimensionality check is required before constructing the probability density function. If the dimension of the cluster center vectors in the external feature space reference model is higher than a preset dimensionality threshold, principal component analysis is used to reduce the dimensionality of the external feature space reference model, and all cluster center vectors and covariance matrices are recalculated on the dimensionality-reduced external feature space reference model to ensure the feasibility and efficiency of subsequent calculations. Next, the predefined range of values ​​for the clustering parameters is determined. The lower limit of the range of values ​​for the independent variables of the clustering parameters is fixed at 2, and the upper limit is determined based on the square root of the size of the target feature vector set, thus defining a clear input domain for the function. Based on this, the structure of the probability density function is formally defined. The probability density function is essentially a mapping relationship that maps each given... The clustering parameter values ​​are mapped to a set of probability density values. The mathematical basis of this is a multivariate Gaussian distribution model. The mean vector of each Gaussian distribution directly adopts the cluster center vector of the external pattern cluster, and the covariance matrix adopts the covariance matrix of the external pattern cluster. These two key parameters fully inherit the statistical properties inherent in the external feature space reference model. For any clustering parameter value within a predefined range, the density distribution representation of the target feature vector in the probability space defined by the external feature space reference model can be obtained through this probability density function. This provides a theoretical basis and computational framework for the construction of the likelihood function in subsequent steps, achieving the goal of transforming external big data knowledge into an internal clustering quality assessment standard. The expression for the probability density function is:

[0051] in, For target feature vectors The probability density value, For the target feature vector, Let be the dimension value of the target feature vector. The center vector of the outer pattern cluster. Let be the covariance matrix of the external mode cluster. Let be the determinant of the covariance matrix. It is the inverse of the covariance matrix. Represents an exponential function. This is the transpose operator.

[0052] Step S4: Based on the probability density function, construct the likelihood function of the target feature vector set relative to the external feature space reference model.

[0053] Specifically, in step S4, the likelihood function is defined as the sum of the natural logarithms of all probability density values ​​output by the probability density function.

[0054] In the specific implementation process, the core task is first to construct a likelihood function for the target feature vector set relative to the external feature space reference model. The design principle of the likelihood function is based on the maximum likelihood estimation theory in probability theory, aiming to create a global index that can quantitatively evaluate the degree of matching between the target feature vector set and the external reference model under different clustering parameters. The likelihood function is explicitly defined as the sum of the natural logarithms of all probability density values ​​output by the probability density function. The mathematical implication lies in transforming the product relationship of probability density values ​​into an additive linear relationship through logarithmic transformation, thereby greatly simplifying the mathematical complexity of subsequent optimization problems. This likelihood function is a high-order function that takes clustering parameters as input variables and obtains the corresponding set of probability density values ​​by internally calling the probability density function. Then, a natural logarithmic transformation is applied to all elements in the probability density value set, and the results are summed to output a scalar value representing the overall goodness of fit. The likelihood function mathematically links the probabilistic framework provided by the external feature space reference model with the selection of clustering parameters, allowing each candidate clustering parameter to be evaluated using a unified, computable scalar index. The constructed likelihood function essentially defines a mapping from the clustering parameter space to the set of real numbers. Its value directly reflects the degree of fit between the overall probability distribution of the target feature vector set and the ideal distribution described by the external feature space reference model, given the clustering parameters. This provides a clear and differentiable mathematical objective function for subsequent parameter optimization search steps. The expression for the likelihood function is:

[0055] in, For clustering parameters Likelihood of time For clustering parameters, The size of the target feature vector set. It is the natural logarithm function. Indicates the first The nth target feature vector, i.e., the nth target feature vector in the target computer image set. The target feature vector extracted from the image. Let be the matching function, representing the first... The target feature vectors are assigned to the clusters that match the external pattern clusters. For the index of the external schema cluster, For the external feature space reference model, the first The cluster center vector of each external pattern cluster. For the external feature space reference model, the first The covariance matrix of the external pattern clusters.

[0056] Step S5: Use an iterative optimization algorithm to search for clustering parameters that maximize the likelihood function within a predefined range of values, and use these as the final clustering parameters.

[0057] Specifically, in step S5, the convergence condition of the iterative optimization algorithm is that the growth rate of the likelihood function value after a preset number of consecutive iterations is less than a preset growth rate threshold.

[0058] In the specific implementation process, based on the constructed likelihood function, an iterative optimization algorithm is initially used to search for clustering parameters within a predefined range that maximize the likelihood function, using these as the final clustering parameters. For example, a Bayesian optimization algorithm, including but not limited to, is selected as the iterative optimization tool. The Bayesian optimization algorithm fits the relationship between the likelihood function and the clustering parameters by constructing a Gaussian process surrogate model, and uses a data acquisition function to guide the next step in selecting clustering parameters. During algorithm initialization, several initial candidate clustering parameter values ​​are randomly selected within the predefined range. For each candidate clustering parameter value, a complete calculation process needs to be executed to obtain the corresponding likelihood function value. For example, this complete calculation... The process includes: first, using the candidate clustering parameter values ​​to perform clustering analysis on the target feature vector set, dividing the target feature vector set into multiple clusters using the K-means algorithm, with the number of clusters equal to the candidate clustering parameter values; then, matching each cluster obtained from the clustering with an external pattern cluster in the external feature space reference model, finding the most similar external pattern cluster for each cluster by calculating the cosine similarity between the cluster center vectors; next, for each target feature vector in the target feature vector set, calculating the probability density value of the target feature vector belonging to the external pattern cluster matched by its cluster, based on the multivariate Gaussian distribution formula, where the mean vector is taken from the matched external pattern cluster. The cluster center vector and covariance matrix are obtained by using the covariance matrix of the matched external pattern cluster. After obtaining the probability density values ​​of all target feature vectors, these probability density values ​​are grouped into a set. Then, the natural logarithm of each probability density value in this set is taken, and all natural logarithm results are summed to obtain the likelihood function value corresponding to the candidate clustering parameter value. The correspondence between the candidate clustering parameter value and the likelihood function value is recorded. After initialization, an iterative loop is entered. In each iteration, the Gaussian process surrogate model is updated based on the evaluated clustering parameter values ​​and their likelihood function values. Then, the expected improvement value of each potential candidate clustering parameter is calculated through the acquisition function, and the candidate clustering parameter value with the largest expected improvement is selected as the next evaluation. Estimate parameter values ​​to form a complete calculation process; for the newly selected candidate clustering parameter values, repeat the above complete calculation process, that is, use the candidate clustering parameter values ​​to perform cluster analysis on the target feature vector set, match external pattern clusters, calculate the probability density value of each feature vector, and sum the natural logarithms to obtain the likelihood function value; record the new value and update the Gaussian process model; after obtaining a new likelihood function value each time, check the convergence condition, calculate the growth rate of the current likelihood function value and the likelihood function value obtained in the previous iteration. If the growth rate of the current likelihood function value is less than the preset growth rate threshold of one-thousandth for a consecutive preset number of iterations, the algorithm is determined to have converged, the iteration is terminated, and the clustering parameter with the largest current likelihood function value is selected as the final clustering parameter;If the convergence condition is not met, the iterative search continues until the preset maximum number of iterations is reached. Finally, the clustering parameter values ​​that maximize the likelihood function are output for subsequent clustering analysis steps.

[0059] Step S6: Use the final clustering parameters to perform unsupervised clustering analysis on the target feature vector set to obtain the clustering analysis results of the target computer image.

[0060] In the specific implementation process, based on the obtained final clustering parameters, unsupervised clustering analysis is performed on the target feature vector set to obtain the clustering analysis results of the target computer image. For example, the final clustering parameter value is used as the input for the number of clusters in the K-means clustering algorithm. The K-means clustering algorithm performs the following iterative optimization process: First, K cluster centers are randomly initialized, where the value of K is equal to the final clustering parameter value; then, an iterative loop is entered, and in each iteration, the Euclidean distance from each target feature vector in the target feature vector set to each cluster center is calculated, and each target feature vector is assigned to the cluster containing the nearest cluster center according to the nearest neighbor principle; after all feature vectors are assigned, the arithmetic mean of each cluster is recalculated as the new cluster center; then, the changes in the old and new cluster centers are checked. The algorithm calculates the Euclidean distance of each cluster center's movement. If the movement distance of all cluster centers is less than one ten-thousandth of the preset convergence threshold or the number of iterations reaches the maximum number of iterations, the iteration process terminates; otherwise, the next round of iteration allocation and update process continues. When the algorithm converges, it outputs stable clustering results, including the cluster label of each feature vector and the final cluster center vector coordinates of each cluster. Finally, based on these clustering results, a clustering analysis report of the target computer image is generated. This report includes statistical distribution information of each category, typical representative images, and visualizations. The entire implementation process is fully automated without human intervention, ensuring the consistency and objectivity of the analysis results. It provides a reliable final output for big data analysis of computer images, realizing a big data analysis method for computer images.

[0061] The working principle of the big data analysis method for computer images provided by this invention is as follows:

[0062] This invention first extracts feature vectors from a target image set and all images in an external database using a pre-trained deep convolutional neural network, forming a target feature vector set and an external feature vector set. Next, unsupervised clustering analysis is performed on the external feature vector set to generate multiple external pattern clusters. For each pattern cluster, its cluster center vector and covariance matrix are calculated, thereby constructing an external feature space reference model. This model serves as a statistical knowledge base representing the distribution characteristics of external data. Subsequently, a probability density function is constructed with clustering parameters as independent variables. This probability density function performs clustering analysis on the target feature vector set according to the given clustering parameters and matches the clustering results with the external feature space reference model, based on multivariate Gaussian distribution. The algorithm calculates the probability density value of each target feature vector under its matching external pattern cluster distribution, and all probability density values ​​constitute the output set. Based on this probability density function, a likelihood function is further constructed, defined as the sum of the natural logarithms of all probability density values. This likelihood function quantifies the overall matching degree between the target dataset and the external reference model under different clustering parameters. An iterative optimization algorithm is used to automatically search for the clustering parameters that maximize the likelihood function within the predefined range of clustering parameter values, and uses these as the final clustering parameters. This search process gradually approaches the optimal solution through intelligent balancing exploration and utilization. Finally, the optimized clustering parameters are used to perform unsupervised clustering analysis on the target feature vector set to obtain stable and reliable image classification results.

[0063] Example 2

[0064] A big data analysis system for computer images, applied to a big data analysis method for computer images, in the specific implementation process, such as... Figure 2 As shown, it illustrates the modular structure of a big data analysis system for computer images, including:

[0065] The target feature extraction module 100 is used to extract the feature vector of each image in the target computer image set using a pre-trained deep convolutional neural network, forming a target feature vector set.

[0066] The external reference model construction module 200 is used to generate external pattern clusters from a large image database through a clustering algorithm, calculate the cluster center vector and covariance matrix of each external pattern cluster, and construct an external feature space reference model based on all external pattern clusters.

[0067] The probability density function definition module 300 is used to construct a probability density function with clustering parameters within a predefined range as independent variables and the set of probability density values ​​obtained by matching the clustering results of the target feature vector set after clustering analysis based on the clustering parameters with the external feature space reference model as dependent variables.

[0068] The likelihood function construction module 400 is used to construct the likelihood function of the target feature vector set relative to the external feature space reference model based on the probability density function.

[0069] The parameter optimization search module 500 is used to search for clustering parameters that maximize the likelihood function within a predefined range of clustering parameters using an iterative optimization algorithm, and then use these clustering parameters as the final clustering parameters.

[0070] The clustering analysis execution module 600 is used to perform unsupervised clustering analysis on the target feature vector set using the final clustering parameters to obtain the clustering analysis results of the target computer image.

[0071] The working principle of the big data analysis system for computer images provided by this invention is as follows:

[0072] This invention employs a target feature extraction module 100 to extract feature vectors from an input set of target computer images using a pre-trained deep convolutional neural network, thus constructing a target feature vector set. An external reference model construction module 200 generates external pattern clusters from a large image database using a clustering algorithm, calculates the cluster center vector and covariance matrix of each pattern cluster, and ultimately constructs an external feature space reference model, thereby establishing a knowledge base. A probability density function definition module 300 uses clustering parameters as independent variables, performs clustering analysis on the target feature vector set based on these parameters, and compares the obtained clustering results with the external feature space reference model. The system performs a set of probability density values ​​to construct a probability density function. The likelihood function construction module 400 constructs a likelihood function of the target feature vector set relative to the external feature space reference model by summing all probability density values ​​output by the probability density function using the natural logarithm. The parameter optimization search module 500 uses an iterative optimization algorithm to automatically search for clustering parameters within a predefined range that maximize the likelihood function, as the final output. The clustering analysis execution module 600 uses the optimized final clustering parameters to perform unsupervised clustering analysis on the target feature vector set and obtains the final image analysis results.

[0073] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0074] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0075] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A big data analysis method for computer images, characterized in that, The analytical method includes the following steps: S1: Use a pre-trained deep convolutional neural network to extract the feature vector of each image in the target computer image set, and form a target feature vector set; S2: Generate external pattern clusters from the large image database using a clustering algorithm, calculate the cluster center vector and covariance matrix of each external pattern cluster, and construct an external feature space reference model based on all external pattern clusters; S3: Construct a probability density function with clustering parameters within a predefined range as independent variables and the set of probability density values ​​obtained by matching the clustering results of the target feature vector set based on these clustering parameters with the external feature space reference model as dependent variables. S4: Based on the probability density function, construct the likelihood function of the target feature vector set relative to the external feature space reference model; S5: Use an iterative optimization algorithm to search for clustering parameters within a predefined range that maximize the likelihood function and use them as the final clustering parameters; S6: Use the final clustering parameters to perform unsupervised clustering analysis on the target feature vector set to obtain the clustering analysis results of the target computer image; In step S3, the probability density function calculates the probability density value based on a multivariate Gaussian distribution, where the mean vector of the multivariate Gaussian distribution is the cluster center vector of the external model cluster, and the covariance matrix is ​​the covariance matrix of the external model cluster. The expression for the probability density function is: in, For target feature vectors The probability density value, For the target feature vector, Let be the dimension value of the target feature vector. The center vector of the outer pattern cluster. Let be the covariance matrix of the external mode cluster. Let be the determinant of the covariance matrix. It is the inverse of the covariance matrix. Represents an exponential function. It is the transpose operator; In step S4, the likelihood function is defined as the sum of the natural logarithms of all probability density values ​​output by the probability density function, where the expression for the likelihood function is: in, For clustering parameters Likelihood of time For clustering parameters, The size of the target feature vector set. It is the natural logarithm function. Indicates the first The nth target feature vector, i.e., the nth target feature vector in the target computer image set. The target feature vector extracted from the image. Let be the matching function, representing the first... The target feature vectors are assigned to the clusters that match the external pattern clusters. For the index of the external schema cluster, For the external feature space reference model, the first The cluster center vector of each external pattern cluster. For the external feature space reference model, the first The covariance matrix of the external pattern clusters.

2. The big data analysis method for computer images according to claim 1, characterized in that, In step S2, the generation of the external pattern cluster includes: The feature vectors of each image in the image database are extracted using a pre-trained deep convolutional neural network to form an external feature vector set. An unsupervised clustering analysis was performed on the set of external feature vectors using a clustering algorithm, and the resulting clusters were taken as external pattern clusters.

3. The big data analysis method for computer images according to claim 2, characterized in that, In step S2, the cluster center vector of each external pattern cluster is the mean vector obtained by calculating the arithmetic mean of all feature vectors in that external pattern cluster.

4. The big data analysis method for computer images according to claim 3, characterized in that, In step S2, the covariance matrix of each external pattern cluster is the sample covariance matrix calculated based on all the eigenvectors constituting that external pattern cluster.

5. The big data analysis method for computer images according to claim 4, characterized in that, After step S2 and before step S3, the method further includes a step of dimensionality reduction of the external feature space reference model when the dimension of the cluster center vector in the external feature space reference model is higher than a preset dimension threshold. The dimensionality reduction process adopts the principal component analysis method, and the cluster center vector and covariance matrix are recalculated for the dimensionality-reduced external feature space reference model.

6. The big data analysis method for computer images according to claim 5, characterized in that, In step S3, the lower limit of the predefined value range is 2, and the upper limit of the predefined value range is determined based on the square root of the size of the target feature vector set.

7. The big data analysis method for computer images according to claim 6, characterized in that, In step S5, the convergence condition of the iterative optimization algorithm is that the growth rate of the likelihood function value after a preset number of consecutive iterations is less than a preset growth rate threshold.

8. A big data analysis system for computer images, characterized in that, The big data analysis method for computer images as described in any one of claims 1 to 7, wherein the analysis system comprises: The target feature extraction module is used to extract the feature vector of each image in the target computer image set using a pre-trained deep convolutional neural network, forming a target feature vector set. The external reference model construction module is used to generate external pattern clusters from a large image database using a clustering algorithm, and to calculate the cluster center vector and covariance matrix of each external pattern cluster, and to construct an external feature space reference model based on all external pattern clusters. The probability density function definition module is used to construct a probability density function with clustering parameters within a predefined range as independent variables and the set of probability density values ​​obtained by matching the clustering results of the target feature vector set based on the clustering parameters with the external feature space reference model as dependent variables. The likelihood function construction module is used to construct the likelihood function of the target feature vector set relative to the external feature space reference model based on the probability density function. The parameter optimization search module is used to search for clustering parameters that maximize the likelihood function within a predefined range of clustering parameters using an iterative optimization algorithm, and then use these as the final clustering parameters. The clustering analysis execution module is used to perform unsupervised clustering analysis on the target feature vector set using the final clustering parameters to obtain the clustering analysis results of the target computer image. In the probability density function definition module, the probability density function calculates the probability density value based on a multivariate Gaussian distribution, where the mean vector of the multivariate Gaussian distribution is the cluster center vector of the external model cluster, and the covariance matrix is ​​the covariance matrix of the external model cluster. The expression for the probability density function is: in, For target feature vectors The probability density value, For the target feature vector, Let be the dimension value of the target feature vector. The center vector of the outer pattern cluster. Let be the covariance matrix of the external mode cluster. Let be the determinant of the covariance matrix. It is the inverse of the covariance matrix. Represents an exponential function. It is the transpose operator; In the likelihood function construction module, the likelihood function is defined as the sum of the natural logarithms of all probability density values ​​output by the probability density function. The expression for the likelihood function is: in, For clustering parameters Likelihood of time For clustering parameters, The size of the target feature vector set. It is the natural logarithm function. Indicates the first The nth target feature vector, i.e., the nth target feature vector in the target computer image set. The target feature vector extracted from the image. Let be the matching function, representing the first... The target feature vectors are assigned to the clusters that match the external pattern clusters. For the index of the external schema cluster, For the external feature space reference model, the first The cluster center vector of each external pattern cluster. For the external feature space reference model, the first The covariance matrix of the external pattern clusters.

Images