Parallel DCNN fast scene matching and recognition method based on Spark and feature clustering

Abstract

The present invention proposes a parallel DCNN fast scene matching and recognition method based on Spark and feature clustering, comprising the following steps: S1, parallel pooling selection: adaptively adjusting the next pooling method by calculating the mutual information coefficient of the feature graphs before and after pooling, thereby solving the problem of decreased accuracy of the feature graphs. S2, feature parallel clustering: quickly locating the density clustering parameters through particle swarm optimization, sampling and identifying cluster features in the fully connected layer, thereby solving the problem of excessive redundant feature calculations. S3, feature dynamic load: dynamically calculating the computational overhead of the feature graphs of each node in the distributed system, and dynamically distributing data between groups based on this overhead, thereby achieving dynamic load balancing of data and solving the problem of low parallel recognition efficiency. The present invention solves the problems of decreased accuracy of feature graphs, excessive redundant feature calculations, and low parallel recognition efficiency in the process of deep parallel convolutional neural networks (DCNNs) in recognizing large-scale images.

Description

Technical Field

[0001] The present invention relates to the field of scene recognition, and in particular to a parallel DCNN fast scene matching and recognition method based on Spark and feature clustering. Background Art

[0002] Scene recognition is an important computer vision task that analyzes input images or videos to identify objects, people, actions, and other elements within the scene, thereby understanding the semantic meaning of the entire scene. Scene recognition essentially extracts meaningful features from images or videos and then classifies these features into predefined scene categories. This enables scene recognition technology to accurately perceive the behavioral characteristics of scene elements and predict potential behaviors, thereby enabling intelligent decision-making. It has already played a vital role in fields such as public safety, road traffic, and autonomous driving. Breakthroughs in storage media technology and the development of IoT devices have generated massive, complex, and difficult-to-process data sets in the public scene recognition field. These data possess the "four V" characteristics of high volume, high velocity, high dimensionality, and high value. These four V characteristics make it difficult for scene recognition to uncover the potential features of this big data, which in turn affects scene recognition accuracy.

[0003] To accelerate scene feature extraction, many studies have combined scene recognition with deep convolutional neural networks (DCNNs). Leveraging DCNN's automatic feature extraction, hierarchical model structure, and model parameter sharing, this approach significantly reduces the computational complexity of scene recognition and enhances recognition accuracy. Herranz et al. first proposed a convolutional neural network-based scene recognition method (Scene Recognition with CNNs: Objects, Scales, and Dataset Bias). By constructing a CNN architecture that simultaneously models both object and scene information, the method can handle scenes with objects of varying scales. They also proposed a dataset bias adaptation method to mitigate the poor scene recognition performance caused by training data bias. Zhang et al. proposed a novel multi-level ensemble network for scene recognition (Multi-Level Ensemble Network for Scene Recognition). By constructing global, local, and fusion layers to capture global scene information and focus on local scene details, respectively, the method combines the outputs of the first two layers for final scene classification. This method achieves superior performance in scene recognition tasks compared to traditional single-network approaches. Khan et al. proposed a multi-layer feature fusion method (Integrating Multilayer Features of Convolutional Neural Networks for Remote Sensing Scene Classification) that integrates a pre-trained CNN model. This method extracts multi-layer features from an image using a pre-trained CNN model, fuses these features, and classifies them to obtain the final result. This literature represents a recent trend in scene recognition algorithm research. By integrating multi-scale features, it increases recognition robustness and accuracy, while also accelerating feature extraction by combining deep convolutional neural network models. However, these algorithms focus on multi-scale feature extraction and fusion within deep convolutional neural networks, without addressing the computational process of DCNNs. This results in the algorithm being unable to quickly and accurately perform scene recognition in large-scale environments.

[0004] Because traditional deep convolutional neural networks struggle to respond quickly to big data, introducing parallel computing frameworks into DCNNs has become a new research direction. Spark, a memory-based distributed computing framework, is widely used for big data processing and analysis due to its ease of programming, real-time response, load balancing, and support for multiple data sources. Many DCNN recognition optimization algorithms based on the Spark distributed computing framework have also been extensively studied. Wang et al. proposed SASTCNN, a distributed DCNN recognition algorithm based on Hadoop. By constructing data partitions and parallelizing the recognition convolutional neural network, they achieved a parallelized DCNN recognition process. Based on this, Bello et al. proposed a bidirectional network BDCNN (Deep neural network to extract high-level features and labels in multi-label classification problems) composed of stacked associated pooling layers. This algorithm associates neurons with pooling layers and calculates the correlation between the two neurons based on the Pearson correlation between variables, thereby dynamically selecting the strategy of the pooling layer. However, since the model parameters are fixed during the recognition process, only the Dropout strategy affects the correlation between the front and back neurons, which reduces the accuracy of the feature map during the pooling process. Boulila et al. proposed a distributed recognition algorithm based on DCNN, RS-DCNN (A novel distributed convolutional-neural-networks based-approach for big remote-sensing image This algorithm is used for large-scale image classification. It splits large images into smaller ones and applies the maximum likelihood classification supervision method to prepare for dataset screening. Although this method segments the image and processes it using a distributed parallel framework, it uses classification methods to screen data features. For datasets with non-fixed classes, it cannot solve the problem of redundant feature calculations. Mao et al. proposed the Parallel Deep Convolutional-Neural-Networks Optimization Algorithm (PDCNNO) based on the modified secant conjugate gradient method (CGMSE). This algorithm obtains local classification results by designing the modified secant conjugate gradient method (CGMSE), achieving rapid convergence of the network model classification results. It also designs a load balancing strategy (LBRLA) to control the load rate and obtain global classification results. However, the algorithm's final global load balancing causes the load of abnormal nodes to affect the global load, resulting in increased load overhead for global nodes, and still cannot solve the problem of low parallel recognition efficiency.

[0005] In summary, although the above algorithms have achieved certain results, in the process of parallel DCNN scene recognition, problems such as decreased accuracy of feature maps, excessive feature redundant calculations, and low parallel recognition efficiency are still urgently needed to be solved. Summary of the Invention

[0006] The present invention aims to at least solve the technical problems existing in the prior art, and particularly innovatively proposes a parallel DCNN fast scene matching and recognition method based on Spark and feature clustering.

[0007] In order to achieve the above-mentioned object of the present invention, the present invention provides a parallel DCNN fast scene matching and recognition method based on Spark and feature clustering, comprising the following steps:

[0008] S0, obtain scene;

[0009] S1, parallel pooling selection: By calculating the mutual information coefficient of the feature maps before and after pooling, the next pooling method is adaptively adjusted to solve the problem of decreased accuracy of the feature maps.

[0010] S2, feature parallel clustering: Through particle swarm optimization, density clustering parameters are quickly located, and cluster features are sampled and identified in the fully connected layer, solving the problem of redundant feature calculations.

[0011] S3, dynamic feature load: By dynamically calculating the computational overhead of the feature graph of each node in the distributed system and dynamically distributing data between groups based on this overhead, dynamic load balancing of data is achieved, solving the problem of low parallel recognition efficiency.

[0012] S4, obtain the scene recognition result, and identify the objects, people, actions and other elements that appear in the scene.

[0013] Furthermore, the S1 includes the following steps:

[0014] (1) Feature comparison: During the recognition process, the mutual information correlation coefficients of the feature maps before and after pooling are calculated and compared in parallel. Provide indicators for pooling method selection;

[0015] (2) Pooling selection: Each node reads the mutual information correlation coefficient of the previous pooling layer And calculate the mutual information correlation coefficient of this pooling layer Two times in the compute node Aggregate the difference to evaluate the adaptability of the previous pooling method and select the pooling method suitable for the current data set, thereby solving the problem of decreased accuracy of the feature map.

[0016] Furthermore, the feature comparison includes the following steps:

[0017] First, the image file is divided into several batches. OpenCV is used to convert the input initial image data format and mark the index. It is stored in the distributed file system HDFS, and the HDFS file is input into the Spark job.

[0018] Then, the data slicing function is called to run a Mapper task for each data slice, and the trained DCNN network model is assigned to the data slice, and the data in the slice is input into the DCNN network model;

[0019] Slicing in Spark is an abstraction built on top of RDD (Resilient Distributed Dataset) partitioning. A slice can be a set of contiguous or disjoint partitions. Slicing helps Spark optimize data distribution and task scheduling during computations. We only set the minimum number of slices; the rest is handled by the system.

[0020] Mapper is a function that performs transformation operations on RDD (Resilient Distributed Dataset) and is a stage of Spark task.

[0021] Then, every time the data reaches the pooling layer of the DCNN model, the mutual information correlation coefficient of all feature maps before and after the Mapper is calculated. And the mutual information correlation coefficient of the system as a whole Perform aggregation processing;

[0022] Finally, compare the overall system of the last pooling with that of this pooling The difference in aggregation values ​​is compared to determine the next pooling method selection.

[0023] Furthermore, the improved mutual information correlation coefficient The calculation formula is as follows:

[0024]

[0025] in express The joint probability density of

[0026] Respectively and The probability density of

[0027] express arrive Projection distance;

[0028] Represents the one-dimensional vector representation of the feature map before pooling;

[0029] Represents the one-dimensional vector representation of the feature map after pooling.

[0030] Furthermore, the pooling selection includes:

[0031] First, set the pooling matrices of 1*1, 2*2, 3*3 and 4*4, and set the step size of the pooling matrix to 1, 1, 2, and 2 respectively. Initially, the pooling matrix of size 4*4 is selected for operation;

[0032] Then the last pooling layer The value is the same as the value of this pooling layer The difference between the aggregated values ​​of each node is calculated to get the value of This indicates that the feature accuracy has not dropped significantly this time. Keep the pooling matrix and step size this time. If This indicates that the feature accuracy has dropped significantly, and the matrix size and its corresponding step size need to be reduced in sequence until the data is calculated into a one-dimensional feature vector.

[0033] Furthermore, the S2 includes the following steps:

[0034] (1) Feature selection clustering: Adaptive particle velocity and feature graph fitness function (CGFF) are used to quickly locate density clustering parameters and thus cluster feature vectors.

[0035] (2) Identification between feature vector groups: After completing the clustering of the feature vectors, a full connection operation is performed on a sample of data from each category, and the final classification top-5 accuracy is evaluated to determine the category of the entire group. The top-5 accuracy is a more reliable evaluation metric than the top-1 accuracy because it takes into account the fact that some images may be inherently ambiguous, and it measures the model's ability to identify the correct label among a set of possible alternatives.

[0036] Furthermore, the feature selection clustering includes:

[0037] First, the Master node reads all the feature vector sets of the algorithm in parallel in the previous stage from HDFS, and determines the initialization range and number of the particle swarm based on the number of feature vectors. represents a particle that has evolved for the tth time in the initialized particle swarm. Represents the evolution of the t-th clustering parameter; and randomly generates the position and velocity vectors of the particles;

[0038] Then calculate the fitness function value of the particle according to the characteristic graph fitness function CGFF, and set the maximum number of iterations Maxiter, according to the adaptive particle speed v t+1The formula calculates the particle velocity and updates the particle velocity and position. If the maximum number of iterations Maxiter is reached or the CGFF value reaches the predetermined accuracy requirement, the algorithm stops. Otherwise, it returns to the CGFF calculation and continues to pass the particle v t+1 The formula updates the particle's velocity and position;

[0039] Finally, after obtaining the density clustering parameters ε and minpts, all feature vectors are clustered, and the recorded categories and feature vectors are stored in HDFS to complete feature selection clustering.

[0040] Furthermore, the adaptive particle velocity v t+1 The calculation formula is as follows:

[0041]

[0042] is the velocity of the i-th particle at the t-th iteration;

[0043] c1 and c2 are learning factors, representing individual learning factors and social learning factors respectively;

[0044] p i ,g i They represent the best position of a particle in the first t iterations and the best position of all particles in the first t iterations respectively;

[0045] represents the position of the i-th particle in the t-th iteration;

[0046] represents the modulus of the two clusters formed by the eigenvectors;

[0047] represents two clusters formed by eigenvectors,

[0048] r() usually returns a random number in the range [0,1].

[0049] Furthermore, the calculation formula of the characteristic graph fitness function CGFF is as follows:

[0050]

[0051] q represents the total number of clusters;

[0052] C z represents the total average value of all eigenvector moduli in the zth cluster;

[0053] p represents a feature vector within the cluster;

[0054] represents the modulus of the two clusters formed by the eigenvectors;

[0055] Represent the two clusters formed by the feature vectors.

[0056] Furthermore, the step S3 includes the following steps:

[0057] First, several DCNN models are assigned to each node. Only one DCNN model is set to enabled, and data is input into these neural networks for calculation, so that the node reaches the maximum load at the beginning. After each round of convolution and pooling, the load of each node and the load to its LVV (s) are calculated. d );

[0058] Next, another model in the node is enabled. When the load is lower than the load threshold η, new data is added to this node for calculation. The amount of input data is LVV(s d ) to achieve dynamic load balancing;

[0059] Finally, each node calculates and obtains the final recognition result.

[0060] In summary, due to the adoption of the above technical solution, the present invention can ensure the accuracy of feature maps, avoid redundant feature calculations, and improve recognition efficiency in the parallel DCNN scene recognition process.

[0061] The method proposed in the present invention has significantly improved both operational efficiency and model accuracy. Through this method, scene recognition can be quickly performed on big data. It can be applied to the fields of intelligent transportation, e-commerce, human-computer interaction, industrial inspection, etc. with large-scale data sets, helping people to process image information data more efficiently and improve efficiency.

[0062] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0064] Figure 1 is the speedup ratio of each algorithm on SVHN, CIFAR10, MS-COCO, and ImageNet 1K datasets, where Figure 1 (a) is the speedup ratio of each algorithm on the SVHN dataset, Figure 1 (b) is the speedup ratio of each algorithm on the CIFAR10 dataset. Figure 1 (c) is the acceleration ratio of each algorithm on the MS-COCO dataset, Figure 1 (d) is the speedup ratio of each algorithm on the ImageNet 1K dataset.

[0065] Figure 2 is the running time of each algorithm on the SVHN, CIFAR10, MS-COCO, and ImageNet 1K datasets. DETAILED DESCRIPTION

[0066] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0067] The present invention proposes a parallel DCNN fast scene matching and recognition method FIRM-PDCNN based on Spark and feature clustering, comprising: setting batch_size and dividing data according to batch_size and the number of nodes; inputting the divided data into a cluster for scene recognition, during which the pooling method is dynamically selected through the MI-IPSS strategy when the data passes through the pooling layer; then, before all data are calculated to the fully connected layer, all one-dimensional feature vectors in the cluster are transmitted to the Master node, and the DCPSO-FSS strategy is used to select features for the fully connected layer calculation; and during the Spark operation, the CCG-LBS strategy is used to achieve dynamic load balancing of data.

[0068] The main contributions of the FIRM-PDCNN of the present invention are: (1) proposing a pooling selection strategy MI-IPSS based on improved mutual information, which adaptively adjusts the next pooling method by calculating the mutual information coefficient of the feature graphs before and after pooling, thereby solving the problem of decreased accuracy of the feature graphs; (2) designing a feature selection strategy DCPSO-FSS based on density clustering and particle swarm optimization, which quickly locates the density clustering parameters through particle swarm optimization, and samples and identifies cluster features in the fully connected layer, thereby solving the problem of redundant feature calculation; (3) designing a load balancing strategy CCG-LBS based on cluster feature graphs, which dynamically calculates the computational overhead of the feature graphs of each node in the distributed system, and dynamically distributes data between groups according to this overhead, thereby achieving dynamic load balancing of data and solving the problem of low parallel recognition efficiency.

[0069] 1. Parallel pooling selection

[0070] Currently, in the field of scene recognition, the accuracy of feature maps decreases during the recognition process of parallel DCNN algorithms. To solve this problem, a pooling selection strategy based on improved mutual information, MI-IPSS, is proposed. This strategy mainly includes two steps: (1) Feature comparison: an improved mutual information correlation coefficient is proposed. During the recognition process, the mutual information correlation coefficients of the feature maps before and after pooling are calculated and compared in parallel. Provide indicators for pooling method selection; (2) Pooling selection: Compare with the previous pooling layer coefficient value, and the entire distributed system The values ​​are aggregated and passed twice Aggregate the difference to evaluate the adaptability of the previous pooling method and select the pooling method suitable for the current data set, thereby solving the problem of decreased accuracy of the feature map.

[0071] 1.1 Feature Comparison

[0072] In order to select a pooling method suitable for the current data set, it is necessary to compare the feature maps before and after the operation of each pooling layer of the distributed DCNN. Therefore, an improved mutual information correlation coefficient IMIC(a,b) is proposed to judge the adaptability of the current pooling method to the current data set by comparing the similarity of related features before and after pooling. The specific process is as follows: First, the image file is divided into several batches, and the input initial image data is converted into a format using OpenCV, and the index is marked and stored in the distributed file system HDFS, and the HDFS file is input into the Spark job. Then, a Mapper is run for each slice, and a trained DCNN network model is assigned to it, and the data in the slice is input into the network model. Then, whenever the data reaches the pooling layer, the mutual information correlation coefficients before and after all feature maps in the Mapper are calculated. And the mutual information correlation coefficient of the system as a whole Perform aggregation processing. Finally, compare the overall performance of the last pooling with that of this pooling system. The difference in aggregation values ​​is compared to determine the next pooling method selection.

[0073] Theorem 1 (Improved mutual information correlation coefficient ): Known Represents the one-dimensional vector representation of the feature map before pooling, Represents the one-dimensional vector representation of the feature map after pooling. Mutual information correlation coefficient The calculation formula is as follows:

[0074]

[0075] in express The joint probability density of Respectively and The probability density of express arrive projection distance.

[0076] prove: yes The correlation coefficient between eigenvectors is expressed as and When they are not similar, approaches infinity, making Infinitesimal, and and probability density of simultaneous occurrence will also become smaller, and their mutual information It also becomes smaller, The value of is at a low level; on the contrary, when and When similar, approaches 0, making Approaching 1, and and probability density of simultaneous occurrence will also become larger, and their mutual information It also grew bigger, The value of is at a high level, so when and When they are not similar, When the value of is small, and When similar, The value of is large, which can be used as an indicator to measure the correlation coefficient of the eigenvector.

[0077] Proof

[0078] 1.2 Pooling Selection

[0079] In completing the system After the value is calculated, the entire distributed system Value aggregation, get That is, the aggregate value of this time is saved in the variable and compared with the aggregate value of the previous time. Compare and select the pooling method that is suitable for the current data. The specific process is as follows: First, set the pooling matrix of 1*1, 2*2, 3*3 and 4*4, and set the step size of the pooling matrix to 1, 1, 2, and 2 respectively. Initially, the pooling matrix of size 4*4 is selected for operation. Then the mutual information correlation coefficient of two consecutive poolings is calculated. The difference of the aggregate value is calculated. This indicates that the feature accuracy has not dropped significantly this time, so the pooling method is retained. If This indicates that the feature accuracy has dropped significantly. It is necessary to reduce the matrix size and its corresponding step size in sequence until the data is calculated to a one-dimensional feature vector. Then, the next step is to use the feature selection strategy DCPSO-FSS based on density clustering and particle swarm optimization.

[0080] 2. Feature Parallel Clustering

[0081] Currently, in the field of scene recognition, during the recognition process of the DCNN algorithm, a large amount of data is parallelized and processed to the fully connected layer. The feature vectors of repeated category data will generate a large amount of redundant calculations, resulting in the problem of redundant feature calculations in a big data environment. In order to solve this problem, a feature selection strategy DCPSO-FSS based on density clustering and particle swarm optimization is designed. The strategy first proposes a particle swarm optimization algorithm for adaptive feature selection to find the clustering parameters of the feature vectors of the parallel DCNN model before the operation to the fully connected layer, and then clusters and samples these feature vectors to the fully connected layer for recognition. The strategy mainly includes two steps: (1) Feature selection clustering: Adaptive particle velocity v is proposed t+1 (Adaptive particle speed), adaptively select particle speed according to clustering feature vectors, and design characteristic graph fitness function CGFF (Fitness function of characteristic graph) to quickly obtain accurate density clustering parameters; (2) Identification between feature vector groups: After completing the clustering of feature vectors in the previous stage, perform full connection operation on the data samples of each category, and evaluate the final classification top 5 accuracy to determine the category of the entire group.

[0082] 2.1 Feature Selection Clustering

[0083] In order to reduce the redundant calculation of a large number of repeated data feature vectors, the design proposes an adaptive particle velocity v t +1 The density clustering parameters ε and minpts are quickly located using the feature graph fitness function CGFF to increase the accuracy of feature clustering and reduce the amount of recognition operations. The specific process is as follows: First, the Master node reads all the feature vector sets of the algorithm in parallel in the previous stage from HDFS, and determines the initialization range and number of the particle swarm based on the number of feature vectors. represents a particle that has evolved for the tth time in the initialized particle swarm. Represents the evolution of the t-th clustering parameter; and randomly generates the position and velocity vector of the particle. Then, the fitness function value of the particle is calculated according to the characteristic graph fitness function CGFF, and the maximum number of iterations Maxiter is set. According to the adaptive particle velocity v t+1 The formula calculates the particle velocity and updates the particle velocity and position. If the maximum number of iterations Maxiter is reached or the CGFF value reaches the predetermined accuracy requirement, the algorithm stops. Otherwise, it returns to the CGFF calculation and continues to pass the particle v t+1The formula updates the particle's velocity and position. Finally, after obtaining the density clustering parameters ε and minpts, all feature vectors are clustered and the recorded categories and feature vectors are stored in HDSF to complete the feature selection clustering.

[0084] Theorem 2 (Adaptive particle velocity v t+1 ): Known is the speed of the i-th particle at the t-th iteration, c1, c2 are learning factors, representing the individual learning factor and the social learning factor, respectively, p i ,g i They represent the best position of a particle in the first t iterations and the best position of all particles in the first t iterations, represents the position of the i-th particle in the t-th iteration, represents the module of the two clusters formed by the eigenvectors, Represents two clusters formed by eigenvectors, then the adaptive particle speed v t+1 The calculation formula is as follows:

[0085]

[0086] r() usually returns a random number in the range [0,1].

[0087] Proof: v t+1 is an adaptive particle velocity based on eigenvectors, The calculation results show that The Euclidean distance between the two clusters is formed when When the particle group is very similar, it needs a large inertia to push the particle to update its position. The value of is large, which meets the need of large inertia of the particle group; when When they are not similar, it means that the particle swarm needs to reduce inertia to search for the optimal position. The value of is very small, which meets the need of small inertia of particle group, so v t+1 It can well meet the needs of speed update of particle swarm algorithm.

[0088] Proof

[0089] Theorem 3 (Characteristic Graph Fitness Function CGFF): Given a total number of clusters q, C z represents the total average value of all eigenvectors in the zth cluster, q represents the number of clusters, and p represents a eigenvector in the cluster. Represents the modulus of the two clusters formed by the eigenvectors. The calculation formula of the feature graph fitness function CGFF is as follows:

[0090]

[0091] X,Y∈C z , Represents two clusters formed by eigenvectors

[0092] Proof: CGFF is a function that describes the fitness of the feature graph clustering effect. This function is based on the principle that the similarity within the cluster is large and the similarity between clusters is small. For dist(c z ,p) 2 , when the modulus of the cluster eigenvector is closer to the average eigenvector of the current cluster, dist(c z ,p) 2 The smaller the value, the closer it is to the minimum value of CGFF. and The more dissimilar they are, Therefore, CGFF meets the requirements of the fitness function for the feature graph and can be used to select the density clustering parameters of the feature graph.

[0093] Proof

[0094] 2.2 Feature vector group identification

[0095] After completing feature selection clustering, the algorithm uses the Spark parallel framework to perform the final fully connected layer recognition on the cluster feature vectors. The specific process is as follows: First, a 20% sample of each category is distributed to the distributed computing nodes for fully connected operations to obtain the data feature prediction results. The top 5 accuracy of the current category recognition is then verified in the validation set. If the accuracy exceeds 90%, the current cluster features are considered to belong to the same category. If it does not exceed the predetermined value, the algorithm returns to feature selection clustering and continues clustering until the top 5 accuracy reaches the predetermined value.

[0096] 3. Characteristic dynamic load

[0097] In the current scene recognition field, in the recognition process of parallel DCNN algorithms, the load balancing strategy of traditional distributed systems generally adopts polling or least connection strategy. This strategy can achieve better load balancing in tasks with a certain amount of calculation, but for parallel DCNN recognition algorithms, deep pooling and convolution will cause huge changes in the amount of calculation of each distributed node, which in turn leads to the problem of low parallel recognition efficiency. To solve this problem, a load balancing strategy CCG-LBS based on cluster feature graphs is designed. By dynamically calculating the computational overhead of the feature graphs of each node in the distributed system and dynamically distributing data between groups based on this overhead, dynamic load balancing of data is achieved. The main steps of this strategy are as follows: First, several DCNN models are assigned to each node, only one DCNN model is set to enabled, and data is input into these neural networks for calculation, so that the node reaches the maximum load at the initial time, and the load of each node is calculated after each round of convolution and pooling, as well as the load LVV (s d )(Load Versus Volume); Then, another model in the node is enabled. When the load is lower than the load threshold η, new data is added to this node for calculation. The input data volume is LVV(s d ) to achieve dynamic load balancing, and finally each node calculates the final recognition result.

[0098] Theorem 4 (Load to its quantity LVV(s d )): Known FML(s d ) represents the computational load of all feature graphs of the dth computation node, s d Represents feature map, FML(s max ) represents the maximum node load, Indicates the total load of all nodes, N indicates the total number of nodes. Load LVV(s d ) is calculated as follows:

[0099]

[0100] Proof: LVV(s d ) is used to supplement input data and balance the system load when the distributed system is unbalanced due to the convolution pooling operation of the feature map. d ) formula can be divided into two parts, for the incremental part FML(s max )-FML(s d ) represents the difference between the maximum node and the current node's computing load. This difference means that all nodes in the system can still operate at the maximum load, and this difference can fill the gap in the nodes. In addition, when the node load is lower than the load threshold η, the maximum load node is not running at full load. Based on the current system average load, the shortage of current nodes can be supplemented.

[0101] 4. Analysis of the Effectiveness of the FIRM-PDCNN Algorithm

[0102] To verify the performance of the FIRM-PDCNN algorithm, we applied the FIRM-PDCNN method to four datasets: MS-COCO, ImageNet 1K, SVHN, and CIFAR10. The specific information is shown in Table 1. BDCNN, RS-DCNN, and PDCNNO were compared in terms of algorithm parallel performance, classification accuracy, and running time.

[0103] Table 1 Dataset details

[0104]

[0105]

[0106] 4.1 Analysis of FIRM-PDCNN Algorithm Acceleration Ratio

[0107] To verify the scene recognition performance of the FIRM-PDCNN algorithm in a big data environment, we used the acceleration ratio as a measurement indicator to compare it with the PDCNNO, RS-DCNN, and BDCNN algorithms based on the MS-COCO, ImageNet1K, SVHN, and CIFAR10 datasets. At the same time, to ensure the accuracy of the experimental results, we took the average of 10 running times of each algorithm to calculate the acceleration ratio as the final experimental result. The experimental results are as follows: Figure 1 As shown:

[0108] from Figure 1 (a) Figure 1 (b) It can be seen that when processing relatively small-scale datasets such as SVHN and CIFAR10, the speedup ratios of the four algorithms increase slowly with the increase of the number of nodes. Among them, when the number of cluster nodes is 4, the speedup ratio of FIRM-PDCNN is 1.93, 1.05, 0.22, and 0.68 lower than that of BDCNN and RS-DCNN algorithms with low parallelization degree, respectively; but Figure 1 (c) Figure 1In (d), when the algorithm copes with relatively large datasets such as MS-COCO and ImageNet 1K, the acceleration ratio of the FIRM-PDCNN algorithm increases significantly, reaching 8.98 and 9.07 when the number of cluster nodes is 8, which are 1.47, 2.31, 1.89 and 1.41, 3.43, 3.39 higher than the PDCNNO, RS-DCNN and BDCNN algorithms, respectively. The reasons for these results are: when the FIRM-PDCNN algorithm is processing relatively small datasets such as SVHN and CIFAR10, the distribution of data to each computing node will cause the communication time overhead between nodes to increase rapidly, and the running speed improvement obtained through parallel operations is extremely limited; when the FIRM-PDCNN algorithm is processing relatively large datasets such as MS-COCO and ImageNet 1K, because of its designed pooling selection strategy MI-IPSS based on improved mutual information, it judges the adaptability of the current pooling method to the current dataset by comparing the similarity of related features before and after pooling, and thus adaptively adjusts the next pooling method to solve the problem of decreased feature map accuracy and improve the algorithm's acceleration ratio. Experiments show that the parallelization capability of the FIRM-PDCNN algorithm is significantly enhanced with the increase in the number of cluster nodes. It is suitable for model parallelization recognition of deep convolutional neural networks in scene recognition and has good performance.

[0109] 4.2 Experimental Analysis of FIRM-PDCNN Algorithm Accuracy and FLOPs

[0110] To verify the accuracy and model optimization effect of the FIRM-PDCNN algorithm in a big data environment, the MS-COCO, ImageNet 1K, SVHN, and CIFAR10 datasets are used to calculate the top 5 accuracy and FLOPs of the Baseline, FIRM-PDCNN, PDCNNO, RS-DCNN, and BDCNN, respectively. The Baseline is the benchmark data of the VGG16 model at 1 / 8 the data load. The experimental results are shown in Table 2:

[0111] Table 2 Top 5 accuracy and FLOPs of each algorithm on four datasets

[0112]

[0113] As can be seen from Table 2, when processing relatively small datasets such as SVHN and CIFAR10, the floating-point operations of each algorithm are reduced to varying degrees. Among them, the floating-point operations of FIRM-PDCNN are reduced by 14%, 28%, and 19% and 19%, 37%, and 27% respectively compared with PDCNNO, RS-DCNN, and BDCNN algorithms; when processing larger datasets such as MS-COCO and ImageNet1K, the top5 accuracy and floating-point operations of the FIRM-PDCNN algorithm are better than the other three algorithms. Among them, the top5 accuracy of the FIRM-PDCNN algorithm is 3.07%, 2.01%, and 6.02% higher than that of PDCNNO, RS-DCNN, and BDCNN algorithms, and the floating-point operations are reduced by 50%, 52%, and 55% and 47%, 48%, and 60% respectively. These results are due to the FIRM-PDCNN algorithm's design, which uses a feature selection strategy called DCPSO-FSS (Dense Clustering and Particle Swarm Optimization) based on density clustering and particle swarm optimization. The algorithm uses particle swarm optimization to rapidly find the clustering parameters for the parallel DCNN model's feature vectors before they are applied to the fully connected layer. These feature vectors are then clustered and sampled for recognition in the fully connected layer, resolving the issue of redundant feature computation. Experimental data demonstrates that FIRM-PDCNN achieves higher convergence speed and accuracy than the other three parallel algorithms, making it suitable for model parallelization of deep convolutional neural networks in scene recognition.

[0114] 4.3 Experimental analysis of FIRM-PDCNN algorithm running time

[0115] In order to verify the impact of the FIRM-PDCNN optimization method on the running time in a big data environment, this paper selects PDCNNO, RS-DCNN and BDCNN algorithms to compare with FIRM-PDCNN. The running time taken by the algorithms to reach 90% accuracy in the top 5 of MS-COCO, ImageNet 1K, SVHN and CIFAR10 datasets is compared. The experimental results are shown in the figure. Figure 2 As shown:

[0116] from Figure 2It can be seen that when the algorithm processes smaller datasets such as SVHN and CIFAR10, the recognition time spent by each algorithm is not much different. However, in the big data environment of MS-COCO and ImageNet 1K, the running time of the FIRM-PDCNN algorithm is reduced by 6581s, 7396s, 6970s and 5698s, 7253s, 5789s respectively compared with PDCNNO, RS-DCNN and BDCNN. It can be seen from the figure that as the scale of training data increases, the training time of the FIRM-PDCNN algorithm is greatly reduced compared with the time spent by PDCNNO, RS-DCNN and BDCNN algorithms. The reason for these results is that the FIRM-PDCNN algorithm proposes a load balancing strategy CCG-LBS based on cluster feature graphs. By dynamically calculating the computational overhead of the feature graphs of each node in the distributed system and dynamically distributing data between groups based on this overhead, dynamic load balancing of data is achieved, avoiding the overall algorithm speed being slowed down by the uneven distribution of computing resources. This significantly reduces the running time of the FIRM-PDCNN algorithm compared to the PDCNNO, RS-DCNN, and BDCNN algorithms. Therefore, it can be concluded that FIRM-PDCNN is superior to the PDCNNO, RS-DCNN, and BDCNN algorithms, and is suitable for model parallelization of deep convolutional neural networks in scene recognition.

[0117] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A parallel DCNN fast scene image matching and recognition method based on Spark and feature clustering, characterized by: The following steps are involved: S0, obtain scene image; S1, parallel pooling selection: adaptively adjust the next pooling method by calculating the mutual information coefficient of the feature map before and after pooling, S2, feature parallel clustering: quickly locate density clustering parameters through particle swarm optimization, and sample and identify cluster features in the fully connected layer. S2 includes the following steps: (1) Feature selection clustering: Adaptive particle velocity and feature graph fitness function CGFF are used to quickly locate density clustering parameters, thereby clustering feature vectors; the adaptive particle velocity v t+1 The calculation formula is as follows: is the velocity of the i-th particle at the t-th iteration; c1 and c2 are learning factors, representing individual learning factors and social learning factors respectively; p i ,g i They represent the best position of a particle in the first t iterations and the best position of all particles in the first t iterations respectively; represents the position of the i-th particle in the t-th iteration; represents the modulus of the two clusters formed by the eigenvectors; represents two clusters formed by eigenvectors, r() means that a random number in the range of [0,1] is usually returned; The calculation formula of the characteristic graph fitness function CGFF is as follows: dist(,) represents distance; q represents the total number of clusters; C z represents the total average value of all eigenvector moduli in the zth cluster; p represents a feature vector within the cluster; represents the modulus of the two clusters formed by the eigenvectors; Represent the two clusters formed by the feature vectors; (2) Identification between feature vector groups: After completing the clustering of feature vectors, perform full-connection operations on the data samples of each category, and evaluate the final top 5 classification accuracy to determine the category of the entire group; S3, dynamic load balancing of features: This dynamically calculates the computational overhead of the feature graph of each node in the distributed system and dynamically distributes data between groups based on this overhead, achieving dynamic load balancing of data. S4, obtain image recognition results, and identify objects, people, and action elements appearing in the scene image.

2. The method of fast scene image matching and recognition based on Spark and feature clustering using parallel DCNN according to claim 1, characterized in that: Said S1 comprises the following steps: (1) Feature comparison: During the recognition process, the mutual information correlation coefficients of the feature maps before and after pooling are calculated and compared in parallel. Provide indicators for pooling method selection; (2) Pooling selection: Each node reads the mutual information correlation coefficient of the previous pooling layer And calculate the mutual information correlation coefficient of this pooling layer Two times in the compute node Aggregate the difference to evaluate the adaptability of the previous pooling method and select the pooling method suitable for the current data set, thereby solving the problem of decreased accuracy of the feature map.

3. The method for fast scene image matching and recognition based on Spark and feature clustering using parallel DCNN according to claim 2, characterized in that: The feature comparison comprises the following steps: First, the image file is divided into several batches. The input initial image data is converted into a certain format using OpenCV, and the index is marked. The data is then stored in the distributed file system HDFS, and the HDFS file is input into the Spark job. Then, the data slicing function is called to run a Mapper task for each data slice, and the trained DCNN network model is assigned to the data slice, and the data in the slice is input into the DCNN network model; Then, every time the data reaches the pooling layer of the DCNN model, the mutual information correlation coefficient of all feature maps before and after the Mapper is calculated. And the mutual information correlation coefficient of the system as a whole Perform aggregation processing; Finally, compare the overall system of the last pooling with that of this pooling The difference in aggregation values ​​is compared to determine the next pooling method selection.

4. The method of claim 2 for fast scene image matching and recognition based on Spark and feature clustering using parallel DCNN, characterized in that: The mutual information correlation coefficient The calculation formula is as follows: in express The joint probability density of Respectively and The probability density of express arrive Projection distance; Represents the one-dimensional vector representation of the feature map before pooling; Represents the one-dimensional vector representation of the feature map after pooling.

5. The method of fast scene image matching and recognition based on Spark and feature clustering using parallel DCNN according to claim 1, characterized in that: Pooling options include: First, set the pooling matrices of 1*1, 2*2, 3*3 and 4*4, and set the step size of the pooling matrix to 1, 1, 2, and 2 respectively. Initially, the pooling matrix of size 4*4 is selected for operation; Then the last pooling layer The value is the same as the value of this pooling layer The difference between the aggregated values ​​of each node is calculated to get the value of This indicates that the feature accuracy has not dropped significantly this time. Keep the pooling matrix and step size this time. If This indicates that the feature accuracy has dropped significantly, and the matrix size and its corresponding step size need to be reduced in sequence until the data is calculated into a one-dimensional feature vector.

6. The method of claim 1 for fast scene image matching and recognition based on Spark and feature clustering using parallel DCNN, characterized in that: The feature selection clustering includes: First, the Master node reads all the feature vector sets of the algorithm in parallel in the previous stage from HDFS, and determines the initialization range and number of the particle swarm based on the number of feature vectors. represents a particle that has evolved for the tth time in the initialized particle swarm. Represents the evolution of the t-th clustering parameter; and randomly generates the position and velocity vectors of the particles; Then calculate the fitness function value of the particle according to the characteristic graph fitness function CGFF, and set the maximum number of iterations Maxiter, according to the adaptive particle speed v t+1 The formula calculates the particle velocity and updates the particle velocity and position. If the maximum number of iterations Maxiter is reached or the CGFF value reaches the predetermined accuracy requirement, the algorithm stops. Otherwise, it returns to the CGFF calculation and continues to pass the particle v t+1 The formula updates the particle's velocity and position; Finally, after obtaining the density clustering parameters ε and minpts, all feature vectors are clustered, and the recorded categories and feature vectors are stored in HDFS to complete feature selection clustering.

7. The method of claim 1 for fast scene image matching and recognition based on Spark and feature clustering using parallel DCNN, characterized in that: The S3 includes the following steps: First, several DCNN models are assigned to each node. Only one DCNN model is set to enabled, and data is input into these neural networks for calculation, so that the node reaches the maximum load at the beginning. After each round of convolution and pooling, the load of each node and the load to its LVV (s) are calculated. d ); Next, another model in the node is enabled. When the load is lower than the load threshold η, new data is added to this node for calculation. The amount of input data is LVV(s d ) to achieve dynamic load balancing; Finally, each node calculates and obtains the final recognition result.

Images