Apple producing area identification method based on multi-modal data fusion
Through multimodal data fusion and cascade classifier architecture, combined with adaptive genetic algorithm parameter optimization, the problem of insufficient recognition accuracy of single modal data is solved, and efficient and accurate identification of apple origin is achieved.
Patent Information
- Application Number
- CN202510799244.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-16
- Publication Date
- 2025-09-26
AI Technical Summary
In the existing technology, the feature information dimension of the apple origin identification method based on single modal data is limited, which makes it difficult to fully reflect the various differences in apples. In addition, the single model has high computational complexity, which makes it difficult to meet the needs of fast online detection, and the model parameter optimization efficiency is low.
A multimodal data fusion method is adopted to construct a cascade classifier architecture, which combines convolutional neural networks and support vector machines. The parameters are optimized through an adaptive genetic algorithm to achieve multidimensional feature extraction and efficient classification.
It significantly improves the feature discriminability and computational efficiency of apple origin identification, improves recognition accuracy and model generalization ability, adapts to differences in different samples, and reduces computing resource consumption.
Smart Images
Figure CN120705679A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of apple origin tracing, and more particularly to an apple origin identification method based on multimodal data fusion. Background Art
[0002] Apple origin traceability is a key technical area in agricultural product quality and safety supervision and geographical indication protection. Accurately identifying apple origin is crucial for combating counterfeiting and inferior products, protecting regional brand value, safeguarding consumer rights, and implementing precise supply chain management.
[0003] Currently, existing technical solutions often rely on a single type of data for feature extraction and origin identification. This single-modal approach captures limited and incomplete feature information, making it difficult to fully reflect the comprehensive differences in apples from different origins, such as their internal chemical composition and external physical form. This results in insufficient discriminative features and a weak ability to characterize complex origin environments, limiting further improvements in recognition accuracy.
[0004] Furthermore, existing methods typically employ a single machine learning model for classification. A single SVM model can be computationally complex when processing high-dimensional, multimodal fusion features, and its feature extraction capabilities are limited. While a single CNN model can effectively process high-dimensional data such as images, it may be less adaptable to fusion of small samples or multimodal data with widely varying feature dimensions, and its complex model is prone to overfitting. Furthermore, the lack of an effective computational resource allocation mechanism forces the use of a complex model for all samples, resulting in low overall computational efficiency and difficulty meeting the requirements of rapid online detection.
[0005] Furthermore, existing technologies for key parameters of classification models rely heavily on manual tuning based on experience or optimization methods such as standard genetic algorithms. Manual tuning is highly subjective and inefficient. Standard genetic algorithms suffer from fixed crossover and mutation probabilities, making it difficult to strike a good balance between global search and local development. They are prone to falling into local optima or premature convergence, making it impossible to efficiently and adaptively find the optimal parameter combination for the model, impacting the model's generalization performance and robustness.
[0006] Therefore, how to design an apple origin identification method based on multimodal data fusion that can overcome the defects of limited representation ability of single-modal data and adaptive optimization of key parameters of the model to achieve efficient and accurate identification of apple origin is an urgent problem that technical personnel in this field need to solve. Summary of the Invention
[0007] In view of this, the present invention provides an apple origin identification method based on multimodal data fusion. By fusing multimodal sensor data and constructing an intelligent cascade classification model, it realizes automatic and accurate traceability of apple origin, thereby solving the problem of insufficient discrimination accuracy of traditional single detection methods and providing reliable technical support for agricultural product quality supervision and origin protection.
[0008] In order to achieve the above object, the present invention adopts the following technical solutions:
[0009] A method for identifying apple origin based on multimodal data fusion includes the following steps:
[0010] S1. Obtain multimodal data of sample apples, including near-infrared spectroscopy, visible light imaging, and laser-induced breakdown spectroscopy;
[0011] S2. Extracting features from the multimodal data and generating a multidimensional standardized feature vector through fusion processing;
[0012] S3, constructing a cascade classifier including a first-level convolutional neural network and a second-level support vector machine, and using the standardized feature vector to train the cascade classifier, and optimizing network parameters in combination with an adaptive genetic algorithm;
[0013] S4. Input the standardized feature vector of the apple to be tested into the optimized cascade classifier and output the corresponding origin classification result.
[0014] Preferably, the S1 includes:
[0015] Near-infrared spectrum acquisition: scanning six surface areas of the apple, wavelength range 900-1700nm;
[0016] Visible light image acquisition: 1920 × 1080 pixel RGB images were acquired under D65 standard light source;
[0017] Laser-induced breakdown spectroscopy acquisition: 50 mJ ± 5 mJ breakdown energy was used to acquire spectra in the 350-900 nm band.
[0018] Preferably, in S2, performing feature extraction includes:
[0019] Near-infrared spectral characteristics: After Savitzky-Golay filtering and standard normal variable transformation, characteristic wavelengths are screened using the continuous projection algorithm;
[0020] Visible light image features: Convert to HSV space to calculate the first-order moment, second-order moment, and third-order moment of the H / S / V channels, and use rotation-invariant local binary patterns to extract texture features;
[0021] Laser-induced breakdown spectroscopy characteristics: The baseline was corrected using the adaptive iterative reweighted penalized least squares method, and the Mg / Ca intensity ratio, K / Na intensity ratio, and Fe element characteristic peaks were calculated.
[0022] Preferably, in S2, generating a multidimensional normalized feature vector through fusion processing includes:
[0023] The screened near-infrared characteristic wavelength, HSV moment features and texture features of visible light images, and element intensity ratio features of laser-induced breakdown spectroscopy are cascaded and normalized by Z-score to generate a multidimensional feature vector.
[0024] Preferably, in S3, the first-level convolutional neural network includes:
[0025] Input layer: reshapes the multidimensional normalized feature vector into a two-dimensional matrix;
[0026] Convolutional layer: uses multi-channel 3×3 convolution kernel and ReLU activation function;
[0027] Pooling layer: 2×2 maximum pooling;
[0028] Fully connected layer: a fully connected network with Dropout mechanism;
[0029] Output layer: Softmax function generates category probability distribution.
[0030] Preferably, in S3, the decision function of the second-level support vector machine is:
[0031]
[0032] Among them, N SV represents the number of support vectors, α i ∈[0,C], represents the Lagrange multiplier, C represents the penalty factor, y i Represents the sample category label, and b represents the classification hyperplane bias item.
[0033] Preferably, in S3, the kernel function of the second-level support vector machine is a radial basis function:
[0034] K(x i , x j )=exp(-γ||x i -x j || 2 )
[0035] Among them, x i 、x j represents the sample feature vector in the training data set, and γ represents the kernel width parameter.
[0036] Preferably, in S3, the working logic of the cascade classifier is:
[0037] The first-level convolutional neural network outputs the primary classification result and confidence p, and directly outputs the result when the confidence p ≥ the confidence threshold;
[0038] When the confidence level p < confidence threshold, the second-level support vector machine is activated to make the final decision.
[0039] Preferably, the confidence threshold is set to 0.85, which is determined by maximizing the classification accuracy of the validation set.
[0040] Preferably, in S3, optimizing network parameters by combining an adaptive genetic algorithm includes:
[0041] The convolution kernel size and number of fully connected nodes in the first-level convolutional neural network, as well as the C and γ parameters in the second-level support vector machine are encoded as chromosomes to construct an optimizable gene sequence;
[0042] Dynamically adjust the crossover probability Pc and mutation probability Pm, with high exploration in the early stage and strong development in the later stage, and trigger mutation based on fitness stagnation;
[0043] The validation set classification accuracy Acc is used as the fitness evaluation criterion to quantify the quality of chromosomes;
[0044] The roulette wheel selection, arithmetic crossover and Gaussian mutation operations drive the population to iteratively update and generate better parameter combinations;
[0045] Set the maximum number of iterations and the fitness stability threshold, and output the optimal parameter combination.
[0046] It can be seen from the above technical solution that compared with the prior art, the technical solution of the present invention has the following advantages:
[0047] Beneficial effects:
[0048] 1. This method fuses three modal data types: near-infrared spectroscopy, visible light imaging, and laser-induced breakdown spectroscopy. This method leverages the chemical composition, physical texture, and optical properties of apples to construct a multidimensional, standardized feature vector. The complementary nature of these multi-source data significantly enhances the discriminative information in the feature space, overcoming the limited representational capabilities of single-modal data and providing a more comprehensive feature basis for origin identification.
[0049] 2. A cascaded classifier architecture combined with a two-stage decision-making mechanism is employed: the first-stage CNN processes high-dimensional features for rapid initial screening, directly outputting results when the confidence level is ≥0.85; the second-stage SVM is activated only for low-confidence samples for refined discrimination. This design dynamically allocates computing resources based on confidence thresholds, significantly reducing redundant computation while ensuring high accuracy and improving system real-time performance.
[0050] 3. An adaptive genetic algorithm is introduced to optimize key parameters. By dynamically adjusting the crossover / mutation probability, a balance between global search and local development is achieved. The accuracy of the validation set is used as the fitness function to drive parameter evolution. This effectively avoids the subjectivity of manual parameter adjustment and generates a better parameter combination, thereby improving the model's adaptability to sample differences and reducing the risk of overfitting. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0052] Figure 1 A flow chart of a method for identifying apple origin based on multimodal data fusion provided by an embodiment of the present invention;
[0053] Figure 2 A schematic diagram of the process of optimizing network parameters by combining an adaptive genetic algorithm provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0055] like Figure 1 As shown, this embodiment provides an apple origin identification method based on multimodal data fusion, including the following steps:
[0056] S1. Obtain multimodal data of sample apples, including near-infrared spectroscopy, visible light imaging, and laser-induced breakdown spectroscopy;
[0057] S2. Extracting features from the multimodal data and generating a multidimensional standardized feature vector through fusion processing;
[0058] S3, constructing a cascade classifier including a first-level convolutional neural network and a second-level support vector machine, and using the standardized feature vector to train the cascade classifier, and optimizing network parameters in combination with an adaptive genetic algorithm;
[0059] S4. Input the standardized feature vector of the apple to be tested into the optimized cascade classifier and output the corresponding origin classification result.
[0060] It constructs a multi-dimensional standardized feature vector by fusing multimodal data of near-infrared spectroscopy, visible light images and laser-induced breakdown spectroscopy, significantly improving the comprehensiveness and discriminability of feature representation; adopts a cascaded classification architecture of CNN and SVM combined with a two-level decision-making mechanism to optimize computational efficiency while ensuring high classification accuracy; and introduces an adaptive genetic algorithm to dynamically optimize network and model parameters, effectively enhancing the generalization ability and robustness of the model, and achieving efficient and accurate identification of apple origin.
[0061] The following further describes each step in the above method in detail;
[0062] In this embodiment S1, multimodal data of a sample apple is obtained, including near-infrared spectrum, visible light image, and laser-induced breakdown spectroscopy; including:
[0063] Near-infrared spectrum acquisition: scanning six surface areas of the apple, wavelength range 900-1700nm;
[0064] Visible light image acquisition: 1920 × 1080 pixel RGB images were acquired under D65 standard light source;
[0065] Laser-induced breakdown spectroscopy acquisition: using 50mJ±5mJ breakdown energy, collecting spectra in the 350-900nm band;
[0066] In this step, the three types of data collaboratively cover the chemical composition, physical form and elemental fingerprint characteristics of apples, providing highly reliable basic data for subsequent multimodal fusion.
[0067] In this embodiment S2, feature extraction is performed on the multimodal data, and a multidimensional normalized feature vector is generated through fusion processing;
[0068] Among them, feature extraction includes:
[0069] Near-infrared spectral characteristics: After Savitzky-Golay filtering and standard normal variable transformation, the characteristic wavelength is screened by the continuous projection algorithm; specifically, min||X is iteratively solved based on the continuous projection algorithm. k || and satisfies ||r k-1 -X k b k || 2 <10 -5 , screen out multiple characteristic wavelengths; among them, r k-1 represents the feature variables selected before the kth iteration, X k represents the projection matrix, b k represents the regression coefficient;
[0070] Visible light image features: Convert to HSV space to calculate the first-order moment, second-order moment, and third-order moment of the H / S / V channels, and use the rotation-invariant local binary pattern to extract texture features; specifically, calculate the mean of the first-order moment Second moment standard deviation Third moment skewness Among them, h i Expressed as H, S, V channel values, N is the total number of pixels;
[0071] The texture features are calculated using rotation-invariant local binary patterns with a neighborhood radius of R = 3 pixels and the number of neighborhood sampling points P = 8;
[0072] Laser-induced breakdown spectroscopy characteristics: The baseline was corrected using the adaptive iterative reweighted penalized least squares method, and the Mg / Ca intensity ratio, K / Na intensity ratio, and Fe element characteristic peaks were calculated.
[0073] Furthermore, generating a multi-dimensional normalized feature vector through fusion processing includes:
[0074] The screened near-infrared characteristic wavelength, HSV moment features and texture features of visible light images, and element intensity ratio features of laser-induced breakdown spectroscopy are concatenated and normalized by Z-score to generate a multidimensional feature vector.
[0075] This process not only retains the complementary advantages of near-infrared chemical composition characteristics, visible light morphological texture characteristics and LIBS element fingerprint characteristics, but also eliminates dimensional differences through standardization, providing a unified and robust high-quality input for subsequent cascade classifiers, significantly improving the feature representation ability and model generalization of origin identification.
[0076] In this embodiment S3, a cascade classifier including a first-level convolutional neural network and a second-level support vector machine is constructed, and the cascade classifier is trained using the standardized feature vector, and the network parameters are optimized in combination with an adaptive genetic algorithm;
[0077] Among them, the first-level convolutional neural network includes:
[0078] Input layer: reshapes the multidimensional normalized feature vector into a two-dimensional matrix;
[0079] Convolutional layer: uses multi-channel 3×3 convolution kernel and ReLU activation function;
[0080] Pooling layer: 2×2 maximum pooling;
[0081] Fully connected layer: a fully connected network with Dropout mechanism;
[0082] Output layer: Softmax function generates category probability distribution.
[0083] Furthermore, the decision function of the second-level support vector machine is:
[0084]
[0085] Among them, N SV represents the number of support vectors, α i ∈[0,C], represents the Lagrange multiplier, C represents the penalty factor, y i Represents the sample category label, and b represents the classification hyperplane bias item.
[0086] Furthermore, the kernel function of the second-level support vector machine is the radial basis function:
[0087] K(x i , x j )=exp(-γ||x i -x j || 2 )
[0088] Among them, x i 、x j represents the sample feature vector in the training data set, and γ represents the kernel width parameter.
[0089] Furthermore, the working logic of the cascade classifier is:
[0090] The first-level convolutional neural network outputs the primary classification result and confidence p, and directly outputs the result when the confidence p ≥ the confidence threshold;
[0091] When the confidence level p < confidence threshold, the second-level support vector machine is activated to make the final decision.
[0092] Furthermore, the confidence threshold was set to 0.85, which was determined by maximizing the classification accuracy of the validation set. In specific experiments, the threshold was adjusted in the range of 0.6-0.95 with a step size of 0.05 to evaluate the balance between the classification accuracy and computational efficiency at each level. When the threshold was 0.85, the overall accuracy of the validation set reached 98.2% and the call ratio of the second-level support vector machine was stabilized between 17% and 23%. This not only ensured the decision-making effectiveness of the CNN main classifier, but also avoided the computing resource consumption caused by excessive participation of the SVM auxiliary layer, achieving the optimal ratio of model accuracy and computational efficiency.
[0093] like Figure 2 As shown in Figure 2, the optimization of network parameters by combining adaptive genetic algorithm includes:
[0094] The convolution kernel size and number of fully connected nodes in the first-level convolutional neural network, as well as the C and γ parameters in the second-level support vector machine are encoded as chromosomes to construct an optimizable gene sequence;
[0095] Dynamically adjust the crossover probability Pc and mutation probability Pm, with high exploration in the early stage and strong development in the later stage, and trigger mutation based on fitness stagnation;
[0096] The validation set classification accuracy Acc is used as the fitness evaluation criterion to quantify the quality of chromosomes;
[0097] The roulette wheel selection, arithmetic crossover and Gaussian mutation operations drive the population to iteratively update and generate better parameter combinations;
[0098] Set the maximum number of iterations and the fitness stability threshold, and output the optimal parameter combination.
[0099] By constructing a CNN-SVM cascade classifier architecture and introducing a two-level decision-making mechanism, the computational efficiency is significantly improved while ensuring high-precision classification, and only difficult samples trigger SVM fine discrimination. By combining an adaptive genetic algorithm to dynamically optimize key parameters, with the accuracy of the validation set as the fitness function, the global-local search balance is achieved by dynamically adjusting the crossover / mutation probability, and the optimal parameter combination is automatically generated, effectively enhancing the generalization and robustness of the model.
[0100] In this embodiment, S4, the standardized feature vector of the apple to be tested is input into the optimized cascade classifier, and the corresponding origin classification result is output; this specifically includes:
[0101] The standardized feature vector of the apple to be tested, generated by preprocessing, is input into the optimized cascade classifier. First, the feature vector flows through the first-level convolutional neural network for processing, which outputs the primary classification result for the preset origin category and its corresponding confidence probability value p. A confidence threshold optimized by the validation set is preset. The decision-making mechanism is as follows: If the confidence p output by the CNN is ≥ the threshold, the primary classification result is considered to be highly reliable, and the system directly adopts this result as the final origin classification output without activating the second-level model; on the contrary, if the confidence p is < the threshold, it is determined that the sample has classification uncertainty, and the system automatically triggers the second-level support vector machine (SVM) for fine discrimination. At this time, the same standardized feature vector will be routed to the SVM model, which will make the final classification decision based on the learned decision boundary.
[0102] Regardless of whether the final decision comes from the direct output of the first-level CNN or the judgment of the second-level SVM, the result is uniformly output as the origin classification label of the apple being tested. The core advantage of this two-level decision-making architecture is the intelligent allocation of computing resources: for samples that are easy to distinguish, only the CNN with relatively controllable computational complexity needs to be run to quickly draw reliable conclusions; only for a few samples that are more difficult to distinguish, the SVM with potentially higher computational complexity is called for in-depth analysis. This dynamic routing mechanism based on confidence thresholds significantly reduces the average processing time while ensuring overall classification accuracy, optimizes the system's computational efficiency and response speed, and makes it more suitable for practical application scenarios.
[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. References to the same or similar parts between the various embodiments are sufficient. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For relevant parts, refer to the method description.
[0104] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for identifying apple origin based on multimodal data fusion, characterized in that: The following steps are involved: S1. Obtain multimodal data of sample apples, including near-infrared spectroscopy, visible light imaging, and laser-induced breakdown spectroscopy; S2. Extracting features from the multimodal data and generating a multidimensional standardized feature vector through fusion processing; S3, constructing a cascade classifier including a first-level convolutional neural network and a second-level support vector machine, and using the standardized feature vector to train the cascade classifier, and optimizing network parameters in combination with an adaptive genetic algorithm; S4. Input the standardized feature vector of the apple to be tested into the optimized cascade classifier and output the corresponding origin classification result.
2. The apple origin identification method based on multimodal data fusion according to claim 1, characterized in that: Said S1 comprises: Near-infrared spectrum acquisition: scanning six surface areas of the apple, wavelength range 900-1700nm; Visible light image acquisition: 1920 × 1080 pixel RGB images were acquired under D65 standard light source; Laser-induced breakdown spectroscopy acquisition: 50 mJ ± 5 mJ breakdown energy was used to acquire spectra in the 350-900 nm band.
3. The apple origin identification method based on multimodal data fusion according to claim 1, characterized in that: In S2, feature extraction includes: Near-infrared spectral characteristics: After Savitzky-Golay filtering and standard normal variable transformation, characteristic wavelengths are screened using the continuous projection algorithm; Visible light image features: Convert to HSV space to calculate the first-order moment, second-order moment, and third-order moment of the H / S / V channels, and use rotation-invariant local binary patterns to extract texture features; Laser-induced breakdown spectroscopy characteristics: The baseline was corrected using the adaptive iterative reweighted penalized least squares method, and the Mg / Ca intensity ratio, K / Na intensity ratio, and Fe element characteristic peaks were calculated.
4. The method for identifying apple origin based on multimodal data fusion according to claim 1, characterized in that: In S2, generating a multi-dimensional normalized feature vector through fusion processing includes: The screened near-infrared characteristic wavelength, HSV moment features and texture features of visible light images, and element intensity ratio features of laser-induced breakdown spectroscopy are cascaded and normalized by Z-score to generate a multidimensional feature vector.
5. The method for identifying apple origin based on multimodal data fusion according to claim 1, characterized in that: In S3, the first-level convolutional neural network includes: Input layer: reshapes the multidimensional normalized feature vector into a two-dimensional matrix; Convolutional layer: uses multi-channel 3×3 convolution kernel and ReLU activation function; Pooling layer: 2×2 maximum pooling; Fully connected layer: a fully connected network with Dropout mechanism; Output layer: Softmax function generates category probability distribution.
6. The method for identifying apple origin based on multimodal data fusion according to claim 1, characterized in that: In S3, the decision function of the second-level support vector machine is: Among them, N SV represents the number of support vectors, α i ∈[0,C], represents the Lagrange multiplier, C represents the penalty factor, y i Represents the sample category label, and b represents the classification hyperplane bias item.
7. The method for identifying apple origin based on multimodal data fusion according to claim 6, characterized in that: In S3, the kernel function of the second-level support vector machine is the radial basis function: K(x i ,x j )=exp(-γ||x i -x j || 2 ) Among them, x i 、x j represents the sample feature vector in the training data set, and γ represents the kernel width parameter.
8. The method for identifying apple origin based on multimodal data fusion according to claim 1, characterized in that: In S3, the working logic of the cascade classifier is: The first-level convolutional neural network outputs the primary classification result and confidence p, and directly outputs the result when the confidence p ≥ the confidence threshold; When the confidence level p < confidence threshold, the second-level support vector machine is activated to make the final decision.
9. The method for identifying apple origin based on multimodal data fusion according to claim 8, characterized in that: The confidence threshold is set to 0.85, which is determined by maximizing the classification accuracy of the validation set.
10. The method for identifying apple origin based on multimodal data fusion according to claim 1, characterized in that: In S3, optimizing network parameters by combining an adaptive genetic algorithm includes: The convolution kernel size and number of fully connected nodes in the first-level convolutional neural network, as well as the C and γ parameters in the second-level support vector machine are encoded as chromosomes to construct an optimizable gene sequence; Dynamically adjust the crossover probability Pc and mutation probability Pm, with high exploration in the early stage and strong development in the later stage, and trigger mutation based on fitness stagnation; The validation set classification accuracy Acc is used as the fitness evaluation criterion to quantify the quality of chromosomes; The roulette wheel selection, arithmetic crossover and Gaussian mutation operations drive the population to iteratively update and generate better parameter combinations; Set the maximum number of iterations and the fitness stability threshold, and output the optimal parameter combination.