A nonlinear optical scattering neural network method and system based on multi-order harmonic feature joint optimization

Abstract

This invention discloses a nonlinear optical scattering neural network method and system based on joint optimization of multi-level harmonic features. It constructs three parallel feature extraction paths—fundamental frequency linear scattering, second-harmonic nonlinear scattering, and third-harmonic nonlinear scattering—to acquire optical speckle features of different orders. These multi-order features are then jointly optimized and classified in the digital domain. This invention reveals for the first time a strong complementarity between fundamental frequency linear scattering features and second-harmonic nonlinear scattering features, achieving a synergistic enhancement effect through feature splicing and fusion and joint optimization strategies. Furthermore, comparative analysis reveals high information redundancy between second-harmonic and third-harmonic features, providing clear hardware optimization guidance for multi-channel detection design in practical optical systems. This method improves the performance of complex image classification tasks, offering advantages such as low training complexity, high computational efficiency, and controllable hardware costs, making it suitable for multi-class, small-sample applications.

Description

Technical Field

[0001] This invention belongs to the field of optical computing and artificial intelligence technology, and particularly relates to a nonlinear optical scattering neural network method and system based on joint optimization of multi-level harmonic features. Background Technology

[0002] In optical computing, optical neural networks based on disordered scattering media have attracted widespread attention due to their simple structure and high computational efficiency. When coherent light passes through a scattering medium, multiple scattering effects produce complex speckle patterns. This process can be modeled as a high-dimensional random projection, mapping the input data to a high-dimensional feature space. Such networks are often called scattering neural networks or optical reservoir computing. Their core advantage lies in the fact that the weights of the hidden layers are fixed by the physical medium, requiring no training; only the linear classifier of the output layer needs to be trained, significantly reducing training complexity.

[0003] However, scattering in purely disordered media is essentially a linear process; even under high-power or pulsed light illumination, its input-output mapping still primarily exhibits a linear transformation. This limits the overall expressive power of optical neural networks based on disordered scattering media, making it difficult to handle complex pattern recognition tasks, similar to single-layer linear neural networks. To address the limitation of expressive power by linear transformation, existing research attempts to introduce nonlinearity into scattering neural networks. It is generally assumed that higher-order nonlinearity necessarily brings performance gains; however, existing nonlinear optical scattering neural networks mostly employ single-order nonlinear effects (such as utilizing only the second harmonic), lacking systematic research on the complementarity between nonlinear features of different orders. This fails to fully utilize the rich feature information contained in multi-order harmonics, leading to a blind pursuit of higher-order nonlinearity in hardware design, increasing system complexity and cost. Furthermore, the feature vectors formed after detecting the fundamental frequency light and each order of harmonic light are often used independently, lacking an effective joint optimization mechanism, making it difficult to leverage the synergistic effect between features of different orders, thus limiting classification performance. Summary of the Invention

[0004] Purpose of the invention: This invention provides a nonlinear optical scattering neural network method and system based on joint optimization of multi-level harmonic features. It aims to solve the technical defects of existing nonlinear optical scattering neural networks, such as insufficient utilization of features and lack of synergistic optimization among different order harmonic features. By constructing a multi-channel feature extraction path, the fundamental frequency light, second harmonic, and third harmonic features are jointly optimized to improve classification performance. It is especially suitable for complex image classification tasks such as multi-class and small sample.

[0005] Technical solution: This invention provides a nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features, comprising:

[0006] The input image is acquired, divided into a training set and a test set, and each image in the input image is preprocessed and encoded to obtain the input light field of each image.

[0007] The input light field of each image is input into the fundamental frequency linear scattering path to obtain the fundamental frequency output light field of each image; the input light field of each image is input into the second harmonic nonlinear scattering path to obtain the second harmonic output light field of each image; the input light field of each image is input into the third harmonic nonlinear scattering path to obtain the third harmonic output light field of each image.

[0008] After intensity detection and normalization of the fundamental frequency output light field, second harmonic output light field, and third harmonic output light field of each image, the fundamental frequency feature vector, second harmonic feature vector, and third harmonic feature vector of each image are obtained.

[0009] A joint optimization strategy is determined, and the fundamental frequency feature vector, second harmonic feature vector, and third harmonic feature vector of each image are jointly optimized based on the joint optimization strategy to obtain the joint feature vector of each image.

[0010] Combine the joint feature vectors of each image to obtain the feature matrix of the training set and the feature matrix of the test set; obtain the label matrix of the training set; input the feature matrix and label matrix of the training set into the ridge regression classifier, solve the optimization problem of the ridge regression classifier, and obtain the analytical solution of the output weight matrix of the ridge regression classifier.

[0011] The predicted category of each image in the test set is calculated by combining the analytical solution of the weight matrix output by the classifier with the feature matrix of the test set.

[0012] Furthermore, the encoding employs pure phase modulation, as shown in the formula:

[0013] ;

[0014] in, Here, j represents the pixel values ​​of the preprocessed image, where j is the imaginary unit. For input light field.

[0015] Furthermore, the fundamental frequency output light field It can be obtained through the following formula:

[0016] ;

[0017] in, For the transmission matrix;

[0018] The second harmonic output light field It can be obtained through the following formula:

[0019] ;

[0020] in and All are complex Gaussian random matrices;

[0021] The third harmonic output light field It can be obtained through the following formula:

[0022] .

[0023] Furthermore, the intensity detection formula is as follows:

[0024] ;

[0025] in, This indicates taking the modulo of each element. For the fundamental frequency output light field The light intensity vector, Second harmonic output light field Light intensity vector Output light field for third harmonic The light intensity vector.

[0026] Furthermore, the joint optimization strategy includes any one of the following four methods:

[0027] Method 1: Concatenate the fundamental frequency eigenvector and the second harmonic eigenvector to obtain a joint eigenvector;

[0028] Method 2: Concatenate the second harmonic eigenvector with the third harmonic eigenvector to obtain a joint eigenvector;

[0029] Method 3: Take half of the nodes from the fundamental frequency feature vector and the second harmonic feature vector, and then concatenate them to obtain the joint feature vector;

[0030] Method 4: Take half of the nodes from the second harmonic feature vector and the third harmonic feature vector, then concatenate them to form a joint feature vector.

[0031] Furthermore, the optimization problem of the ridge regression classifier is as follows:

[0032] ;

[0033] in, For regularization parameters, To output the weight matrix, The feature matrix of the training set, The label matrix of the training set;

[0034] The analytical solution for:

[0035] ;

[0036] in This is the joint feature vector.

[0037] Furthermore, the calculation to obtain the predicted category for each image in the test set includes:

[0038] Calculate the predicted label matrix for the test set. The formula is:

[0039] ;

[0040] in, The feature matrix of the test set;

[0041] The category corresponding to the maximum value in each row of the predicted label matrix is ​​taken as the predicted category of the image in the test set corresponding to that row.

[0042] This invention also provides a nonlinear optical scattering neural network system based on joint optimization of multi-level harmonic features, comprising:

[0043] The image acquisition and encoding module is used to acquire input images, divide the input images into training and testing sets, preprocess and encode each image in the input images, and obtain the input light field of each image.

[0044] The scattering path module is used to input the input light field of each image into the fundamental frequency linear scattering path to obtain the fundamental frequency output light field of each image; input the input light field of each image into the second harmonic nonlinear scattering path to obtain the second harmonic output light field of each image; and input the input light field of each image into the third harmonic nonlinear scattering path to obtain the third harmonic output light field of each image.

[0045] The intensity detection module is used to perform intensity detection and normalization on the fundamental frequency output light field, second harmonic output light field and third harmonic output light field of each image in sequence to obtain the fundamental frequency feature vector, second harmonic feature vector and third harmonic feature vector of each image.

[0046] The joint optimization module is used to determine the joint optimization strategy, and to perform joint optimization on the fundamental frequency feature vector, second harmonic feature vector and third harmonic feature vector of each image based on the joint optimization strategy to obtain the joint feature vector of each image.

[0047] The analytical solution module is used to combine the joint feature vectors of each image to obtain the feature matrix of the training set and the feature matrix of the test set; obtain the label matrix of the training set; input the feature matrix and label matrix of the training set into the ridge regression classifier, solve the optimization problem of the ridge regression classifier, and obtain the analytical solution of the output weight matrix of the ridge regression classifier.

[0048] The prediction category module is used to calculate the predicted category of each image in the test set by taking the analytical solution of the weight matrix output by the classifier and the feature matrix of the test set.

[0049] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0050] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0051] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. Significantly Improved Classification Performance: This invention achieves a significant improvement in classification performance by jointly optimizing fundamental frequency and second harmonic features, effectively utilizing their strong complementarity. This technical solution enables the joint features to produce a synergistic enhancement effect in classification tasks, significantly outperforming the use of either order feature alone, and is particularly suitable for complex image classification tasks such as multi-class and small-sample tasks.

[0053] 2. Clear Hardware Design Guidance and Controllable Cost: This invention, through a systematic study of the information redundancy of second and third harmonic characteristics, reveals for the first time that there is no significant performance gain when the two are fused under the same feature dimension. This discovery provides clear guidance for the multi-channel detection design of practical optical systems: if hardware resources allow for expanding the feature dimension, multiple harmonic signals can be acquired simultaneously to obtain additional gain; if hardware resources are limited, it is not necessary to acquire both simultaneously, and acquiring only the second harmonic signal can achieve similar performance. This avoids the increased hardware complexity and cost caused by blindly pursuing higher-order nonlinearities, achieving an optimal balance between performance and cost.

[0054] 3. High information utilization and enhanced feature representation: This invention employs a multi-order feature joint optimization strategy, which automatically identifies and retains more discriminative feature components under the same feature dimensionality. Under dimensionality expansion, it fully mines the complementary information contained in features of different orders, maximizing information utilization. Simultaneously, the introduction of high-order feature mixing through a nonlinear scattering path enhances feature representation, effectively handling challenging tasks such as high-dimensionality and small sample sizes.

[0055] 4. Low training complexity and high computational efficiency: In this invention, the transfer matrices of the optical feature extraction module are all fixed random matrices, requiring no training; only the linear classifier of the digital readout layer needs to obtain an analytical solution through ridge regression, significantly reducing training complexity and computational resource consumption. Simultaneously, the optical feature extraction process is completed entirely in the optical domain, possessing inherent advantages of parallel processing, low latency, and low power consumption, providing a feasible solution for realizing a high-performance, low-power optical intelligent computing system. Attached Figure Description

[0056] Figure 1 This is a flowchart of the method of the present invention.

[0057] Figure 2 This is a schematic diagram of the nonlinear optical scattering neural network based on the joint optimization of multi-level harmonic features of the present invention.

[0058] Figure 3 This is the joint optimization result of taking N nodes each for the fundamental frequency feature vector and the second harmonic feature vector in the experiment of this invention.

[0059] Figure 4 This is the joint optimization result of taking N / 2 nodes for both the fundamental frequency eigenvector and the second harmonic eigenvector in the experiment of this invention.

[0060] Figure 5 This is the joint optimization result of taking N nodes each for the second harmonic eigenvector and the third harmonic eigenvector in the experiment of this invention.

[0061] Figure 6 This is the joint optimization result of taking N / 2 nodes for each of the second harmonic eigenvector and the third harmonic eigenvector in the experiment of this invention. Detailed Implementation

[0062] like Figure 1 and Figure 2 As shown, the nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features described in this invention includes:

[0063] Obtain the input images, including the training set and the test set; perform the following steps for each image in the input images:

[0064] Normalization is performed to obtain a normalized image. The pixel values ​​of the normalized image are then analyzed. ( , The total number of pixels in the input image is encoded as a complex amplitude light field. ( As the input optical field, the encoding method uses pure phase modulation, and the formula is:

[0065] ;

[0066] Where j is the imaginary unit, this step simulates the phase loading process of the spatial light modulator on the incident coherent light.

[0067] To simulate the linear and nonlinear scattering processes of light in disordered LN media, we constructed three scattering paths: the fundamental frequency linear scattering path, the second harmonic nonlinear scattering path, and the third harmonic nonlinear scattering path.

[0068] Input light field By inputting a fundamental frequency linear scattering path, the multiple linear scattering process of light in a disordered scattering medium is simulated to obtain the fundamental frequency output light field. , N is the number of output patterns, i.e., the feature dimension, and the formula is:

[0069] ;

[0070] in, The transfer matrix is ​​a complex Gaussian distribution in which the elements are independently and identically distributed. The elements are randomly generated and fixed in one experiment. It is used to simulate the linear transmission characteristics of a disordered scattering medium, since the internal conditions of the disordered medium are unknown but fixed.

[0071] In practice, LN crystals trigger nonlinear effects that generate second harmonics. This second-order nonlinearity creates a squared relationship between the output and input. Introducing nonlinearity into optical neural networks can enhance their expressive power; therefore, the input optical field... The input is a second harmonic nonlinear scattering path, which simulates a second-order nonlinear effect through two layers of linear scattering and a nonlinear square operation. The second harmonic output light field is obtained. , The formula is:

[0072] ;

[0073] in and All are independent and identically distributed complex Gaussian random matrices, whose elements are randomly generated and fixed in a single experiment. Used to simulate the first linear scattering. Used to simulate the second linear scattering; This represents squaring each complex element in the vector to simulate the nonlinear frequency transformation generated by the second harmonic. As an intermediate layer dimension, this embodiment takes... ;

[0074] Input light field The input is a third harmonic nonlinear scattering path, which simulates the third-order nonlinear effect through two layers of linear scattering and nonlinear cubic operations. The third harmonic output light field is obtained. , The formula is:

[0075] ;

[0076] in This represents taking the cube of each complex element in the vector to simulate the nonlinear frequency transformation generated by the third harmonic.

[0077] By first scattering through a fundamental frequency linear scattering path, then through a second harmonic nonlinear scattering path, and finally through a third harmonic nonlinear scattering path, the nonlinear effect can be demonstrated, while also reducing the burden on computer simulation.

[0078] For the fundamental frequency output light field Second harmonic output light field Third harmonic output light field Intensity detection is performed sequentially to simulate the recording process of a photodetector (such as a CCD) and obtain the fundamental frequency output light field. Light intensity vector, second harmonic output light field Light intensity vector, third harmonic output light field The light intensity vector is given by the formula:

[0079] ;

[0080] in, This indicates taking the modulo of each element.

[0081] For the fundamental frequency output light field Light intensity vector Second harmonic output light field Light intensity vector Third harmonic output light field Light intensity vector Normalization is performed sequentially to obtain the fundamental frequency feature vector. ( ), second harmonic eigenvector ( ), Third harmonic eigenvector ( The range of element values ​​for each feature vector is... .

[0082] For fundamental frequency eigenvectors Second harmonic eigenvector Third harmonic eigenvector Perform joint optimization to construct a joint feature vector; the joint optimization can be any one of the following four methods:

[0083] Method 1: Directly concatenate the fundamental frequency eigenvector and the second harmonic eigenvector to obtain a joint eigenvector. ;

[0084] Method 2: Directly concatenate the second harmonic eigenvector and the third harmonic eigenvector to obtain a joint eigenvector: ;

[0085] Method 3: Take the fundamental frequency eigenvector and the second harmonic eigenvector respectively. Each node is then concatenated to obtain the joint feature vector: ;

[0086] Method 4: Take the second harmonic eigenvector and the third harmonic eigenvector respectively. Each node is then concatenated to form a joint feature vector: .

[0087] Obtain the label matrix of the training set , To determine the number of categories, the labels use one-hot encoding;

[0088] The joint feature vectors of the training set are used to form the feature matrix of the training set. , K is the number of samples in the training set. If the joint feature vector of each image in the training set is obtained by method one or method two, then D=2N. If the joint feature vector of each image in the training set is obtained by method three or method four, then D=N.

[0089] The feature matrix and label matrix of the training set are input into the ridge regression classifier, and the output weight matrix is ​​obtained by solving the optimization problem of the ridge regression classifier. Analytical solution , The optimization problem is:

[0090] ;

[0091] in, This is a regularization parameter used to prevent overfitting;

[0092] By solving the optimization problem, the optimal output weight matrix can be obtained. :

[0093] ;

[0094] The joint feature vectors of the test set are used to form the feature matrix of the test set. The predicted label matrix for the test set is calculated by combining the analytical solution of the weight matrix output by the classifier with the feature matrix of the test set. The formula is:

[0095] ;

[0096] The category corresponding to the maximum value in each row of the predicted label matrix is ​​taken as the predicted category of the image in the test set corresponding to that row.

[0097] The nonlinear optical scattering neural network system based on joint optimization of multi-level harmonic features described in this invention includes:

[0098] The image acquisition and encoding module is used to acquire input images, divide the input images into training and testing sets, preprocess and encode each image in the input images, and obtain the input light field of each image.

[0099] The scattering path module is used to input the input light field of each image into the fundamental frequency linear scattering path to obtain the fundamental frequency output light field of each image; input the input light field of each image into the second harmonic nonlinear scattering path to obtain the second harmonic output light field of each image; and input the input light field of each image into the third harmonic nonlinear scattering path to obtain the third harmonic output light field of each image.

[0100] The intensity detection module is used to perform intensity detection and normalization on the fundamental frequency output light field, second harmonic output light field and third harmonic output light field of each image in sequence to obtain the fundamental frequency feature vector, second harmonic feature vector and third harmonic feature vector of each image.

[0101] The joint optimization module is used to determine the joint optimization strategy, and to perform joint optimization on the fundamental frequency feature vector, second harmonic feature vector and third harmonic feature vector of each image based on the joint optimization strategy to obtain the joint feature vector of each image.

[0102] The analytical solution module is used to combine the joint feature vectors of each image to obtain the feature matrix of the training set and the feature matrix of the test set; obtain the label matrix of the training set; input the feature matrix and label matrix of the training set into the ridge regression classifier, solve the optimization problem of the ridge regression classifier, and obtain the analytical solution of the output weight matrix of the ridge regression classifier.

[0103] The prediction category module is used to calculate the predicted category of each image in the test set by taking the analytical solution of the weight matrix output by the classifier and the feature matrix of the test set.

[0104] The computer device of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0105] The computer-readable storage medium of the present invention stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0106] To verify the effectiveness of the method of the present invention, the following describes the technical solution of this embodiment in conjunction with... Figures 3 to 6 This will detail the experimental process of applying the technical solution to a specific numerical simulation experiment and the technical effects of the technical solution.

[0107] This invention conducts numerical simulation experiments on the Sign Language Digits (SLD) dataset. The SLD dataset contains 10 digit categories. The input images are divided into training and test sets. The training set contains 1,649 images, and the test set contains 413 images. The image size is 64×64 pixels, and the input dimension M=4096.

[0108] Each image in the input image is normalized and encoded to obtain the input light field set of each image;

[0109] The input light field of each image is input into the fundamental frequency linear scattering path to obtain the fundamental frequency output light field of each image; the input light field of each image is input into the second harmonic nonlinear scattering path to obtain the second harmonic output light field of each image; the input light field of each image is input into the third harmonic nonlinear scattering path to obtain the third harmonic output light field of each image.

[0110] After intensity detection and normalization of the fundamental frequency output light field, second harmonic output light field, and third harmonic output light field of each image, the fundamental frequency feature vector, second harmonic feature vector, and third harmonic feature vector of each image are obtained.

[0111] A joint optimization strategy is determined, and the fundamental frequency feature vector, second harmonic feature vector, and third harmonic feature vector of each image are jointly optimized based on the joint optimization strategy to obtain the joint feature vector of each image.

[0112] Combine the joint feature vectors of each image to obtain the feature matrix of the training set and the feature matrix of the test set; obtain the label matrix of the training set; input the feature matrix and label matrix of the training set into the ridge regression classifier, solve the optimization problem of the ridge regression classifier, and obtain the analytical solution of the output weight matrix of the ridge regression classifier.

[0113] By combining the analytical solution of the weight matrix output by the classifier with the feature matrix of the test set, the predicted category of each image in the test set is calculated, and the accuracy of the test set is calculated.

[0114] For different joint optimization strategies, this invention conducted four sets of experiments. The simulation parameters for each set of experiments were set as follows: the number of output modes N was 500, 1000, 1500, 2000, 2500, 3000, and 3500; the intermediate layer dimension H=N; and the regularization parameter... =30, each parameter group was repeated 10 times in an independent experiment and the average value was taken. The results are expressed as mean accuracy ± standard deviation.

[0115] Experiment 1: The joint optimization strategy involves taking N nodes from both the fundamental frequency eigenvector and the second harmonic eigenvector for joint optimization.

[0116] Figure 3 This demonstrates the joint optimization results when N nodes are taken for both the fundamental frequency eigenvector and the second harmonic eigenvector (total dimension 2N), and compares them with those obtained by using the fundamental frequency eigenvector (N nodes) and the second harmonic eigenvector alone. The accuracy curves using the fundamental frequency eigenvector are compared (or SHG, N-node). The graph shows the accuracy curves using the fundamental frequency eigenvector as Linear, the second harmonic eigenvector as Nonliner+SHG, and the joint eigenvector as Linear+SHG.

[0117] The results show that when N=3500, the accuracy using the joint eigenvector is 81.21%±0.82%, the accuracy using the fundamental frequency eigenvector alone is 78.06%±1.13%, and the accuracy using the second harmonic eigenvector alone is 80.75%±1.05%. Among the individual eigenvectors, the accuracy using the second harmonic eigenvector is the best. The accuracy using the joint eigenvector is about 0.5 percentage points higher than the best individual eigenvector accuracy. When N=500, the accuracy using the joint eigenvector is 73.92%±1.76%, which is about 5.4 percentage points higher than the accuracy using the second harmonic eigenvector alone (68.47%±1.50%).

[0118] The results reveal the following pattern: there is a significant complementarity between fundamental frequency features and second harmonic features. Fundamental frequency features, due to linear scattering, preserve the original spatial structure information of the input image, while second harmonic features, through the introduction of a product term between input modes via a squaring operation, achieve nonlinear scattering and realize higher-order feature mixing. After fusion, the joint feature retains the stability of linear features while gaining the discriminative power of nonlinear features, thus achieving a synergistic enhancement effect of "1+1>2" in classification tasks. Furthermore, under low-dimensional conditions (N=500), the gain of the joint feature is more pronounced, indicating that when feature dimensions are limited, fusing complementary features can effectively compensate for the insufficient expressive power of a single feature. This has significant guiding implications for applications with limited hardware resources.

[0119] Experiment 2: The joint optimization strategy involves taking N / 2 nodes for both the fundamental frequency eigenvector and the second harmonic eigenvector and performing joint optimization.

[0120] Figure 4 This paper presents the joint optimization results when the fundamental frequency feature vector and the second harmonic feature vector each have N / 2 nodes (total dimension N). The total dimension of the joint features is the same as when used individually, aiming to fairly compare the performance of feature fusion under the same dimension. The accuracy curves using the fundamental frequency feature vector are shown as Linear, the accuracy curves using the second harmonic feature vector are shown as Nonliner+SHG, and the accuracy curves using the joint feature vector are shown as Linear+SHG.

[0121] The results show that when N=3500, the accuracy of the joint feature is 79.56%±1.00%, which is between that of using the second harmonic feature alone (80.75%±1.05%) and using the fundamental frequency feature alone (78.06%±1.13%), and is closer to the accuracy of the better second harmonic feature.

[0122] The results reveal the following pattern: under the same feature dimension constraint, the joint feature does not simply take the average of the two, but tends to inherit the dominant advantage of the second harmonic feature with higher information content. This indicates that the joint optimization strategy adopted in this invention can automatically identify and retain the more discriminative feature component, reflecting the nonlinear characteristics of information fusion. Simultaneously, the results also confirm that the second harmonic feature has a higher information content than the fundamental frequency feature in classification tasks, providing a quantitative basis for setting feature priorities in practical systems.

[0123] Experiment 3: The joint optimization strategy is to take N nodes from each of the second harmonic eigenvector and the third harmonic eigenvector for joint optimization;

[0124] Figure 5 This demonstrates the joint optimization results when N nodes are taken from both the second and third harmonic eigenvectors (total dimension 2N), and compares them with those obtained by using the second harmonic eigenvector (N nodes) and the third harmonic eigenvector alone. Compare the accuracy curves (or THG, N nodes) using the second harmonic eigenvector. The example graphs for using the second harmonic eigenvector are: Nonliner+SHG, Nonliner+THG, and SHG+THG.

[0125] The results show that when N=3500, the accuracy of the joint feature is 82.37%±0.89%, the accuracy of the second harmonic feature alone is 80.87%±1.00%, and the accuracy of the third harmonic feature alone is 80.41%±0.94%. The accuracy of the joint feature is about 1.5 percentage points higher than that of the second harmonic feature alone. When N=500, the accuracy of the joint feature is 73.63%±1.04%, which is about 4.9 percentage points higher than that of the second harmonic feature alone (68.77%±1.97%).

[0126] The results reveal the following pattern: although the performance of the second and third harmonics is similar when used alone, they still have a certain degree of information complementarity, and performance gains can still be obtained after fusion through dimensional expansion.

[0127] Experiment 4: The joint optimization strategy is to take N / 2 nodes for each of the second harmonic eigenvector and the third harmonic eigenvector and perform joint optimization.

[0128] Figure 6 The diagram illustrates the joint optimization results when N / 2 nodes (total dimension N) are selected for both the second and third harmonic eigenvectors. In this case, the joint feature dimension is the same as when used individually. The accuracy curves using the second harmonic eigenvector are shown as follows: Nonliner+SHG; Nonliner+THG; and SHG+THG.

[0129] As can be observed from the figure, the accuracy curve of the combined feature almost completely overlaps with the accuracy curve of SHG alone or THG alone, showing a slight improvement but no significant advantage.

[0130] The results reveal the following pattern: a high degree of information redundancy exists between the second and third harmonic features. This is because the second harmonic (square operation) and the third harmonic (cubic operation) are both nonlinear frequency transforms, and they share similar structural characteristics in their feature representation. When the feature dimension is limited, redundant information cannot be converted into effective performance gain. This finding provides clear guidance for the design of multi-channel detection systems in practical optical systems: if hardware resources allow for expanding the feature dimension, both second and third harmonic signals can be acquired simultaneously to obtain additional gain; if hardware resources are limited (e.g., the feature dimension is fixed), it is not necessary to acquire both simultaneously; acquiring only the second harmonic signal can achieve similar performance, thereby reducing system complexity and cost.

[0131] When the task complexity is low or the hardware conditions are limited, the classification performance can still be better than that of a single feature by using only the fundamental frequency linear scattering path and the second harmonic nonlinear scattering path for joint optimization.

[0132] When the system focuses on nonlinear feature extraction, the fundamental frequency optical path can be omitted, and only the second and third harmonic features are used for joint optimization. Although the gain is limited, the basic functions can still be achieved.

[0133] The direct splicing method used in this invention can be replaced by other feature fusion methods such as weighted summation and attention mechanism, while still achieving the basic goal of joint optimization.

[0134] The classifier used in this invention is a ridge regression classifier (which has an analytical solution that can be directly calculated). Its optimization capability is limited, but it better reflects the role of nonlinear scattering and joint optimization. It can also be replaced by other linear classifiers (such as logistic regression, support vector machine, etc.) or shallow neural networks (including input layer, hidden layer, and output layer, with 1 or 2 hidden layers). By training with iterative optimization algorithms such as gradient descent, the classification task can still be completed.

Claims

1. A nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features, characterized in that, include: The input image is acquired, divided into a training set and a test set, and each image in the input image is preprocessed and encoded to obtain the input light field of each image. The input light field of each image is input into the fundamental frequency linear scattering path to obtain the fundamental frequency output light field of each image; the input light field of each image is input into the second harmonic nonlinear scattering path to obtain the second harmonic output light field of each image; the input light field of each image is input into the third harmonic nonlinear scattering path to obtain the third harmonic output light field of each image. After intensity detection and normalization of the fundamental frequency output light field, second harmonic output light field, and third harmonic output light field of each image, the fundamental frequency feature vector, second harmonic feature vector, and third harmonic feature vector of each image are obtained. A joint optimization strategy is determined, and the fundamental frequency feature vector, second harmonic feature vector, and third harmonic feature vector of each image are jointly optimized based on the joint optimization strategy to obtain the joint feature vector of each image. Combine the joint feature vectors of each image to obtain the feature matrix of the training set and the feature matrix of the test set; obtain the label matrix of the training set; input the feature matrix and label matrix of the training set into the ridge regression classifier, solve the optimization problem of the ridge regression classifier, and obtain the analytical solution of the output weight matrix of the ridge regression classifier. The predicted category of each image in the test set is calculated by combining the analytical solution of the weight matrix output by the classifier with the feature matrix of the test set.

2. The nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features according to claim 1, characterized in that, The encoding uses pure phase modulation, and the formula is: ； in, Here, j represents the pixel values ​​of the preprocessed image, where j is the imaginary unit. For input light field.

3. The nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features according to claim 2, characterized in that, The fundamental frequency output light field It can be obtained through the following formula: ； in, For the transmission matrix; The second harmonic output light field It can be obtained through the following formula: ； in and All are complex Gaussian random matrices; The third harmonic output light field It can be obtained through the following formula: 。 4. The nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features according to claim 3, characterized in that, The intensity detection formula is as follows: ； in, This indicates taking the modulo of each element. For the fundamental frequency output light field The light intensity vector, Second harmonic output light field Light intensity vector Output light field for third harmonic The light intensity vector.

5. The nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features according to claim 4, characterized in that, The joint optimization strategy includes any one of the following four methods: Method 1: Concatenate the fundamental frequency eigenvector and the second harmonic eigenvector to obtain a joint eigenvector; Method 2: Concatenate the second harmonic eigenvector with the third harmonic eigenvector to obtain a joint eigenvector; Method 3: Take half of the nodes from the fundamental frequency feature vector and the second harmonic feature vector, and then concatenate them to obtain the joint feature vector; Method 4: Take half of the nodes from the second harmonic feature vector and the third harmonic feature vector, then concatenate them to form a joint feature vector.

6. The nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features according to claim 5, characterized in that, The optimization problem of the ridge regression classifier is: ； in, For regularization parameters, To output the weight matrix, The feature matrix of the training set, The label matrix of the training set; The analytical solution for: ； in This is the joint feature vector.

7. The nonlinear optical scattering neural network method based on joint optimization of multi-level harmonic features according to claim 6, characterized in that, The calculation obtains the predicted category for each image in the test set, including: Calculate the predicted label matrix for the test set. The formula is: ； in, The feature matrix of the test set; The category corresponding to the maximum value in each row of the predicted label matrix is ​​taken as the predicted category of the image in the test set corresponding to that row.

8. A nonlinear optical scattering neural network system based on joint optimization of multi-level harmonic features, characterized in that, include: The image acquisition and encoding module is used to acquire input images, divide the input images into training and testing sets, preprocess and encode each image in the input images, and obtain the input light field of each image. The scattering path module is used to input the input light field of each image into the fundamental frequency linear scattering path to obtain the fundamental frequency output light field of each image; input the input light field of each image into the second harmonic nonlinear scattering path to obtain the second harmonic output light field of each image; and input the input light field of each image into the third harmonic nonlinear scattering path to obtain the third harmonic output light field of each image. The intensity detection module is used to perform intensity detection and normalization on the fundamental frequency output light field, second harmonic output light field and third harmonic output light field of each image in sequence to obtain the fundamental frequency feature vector, second harmonic feature vector and third harmonic feature vector of each image. The joint optimization module is used to determine the joint optimization strategy, and to perform joint optimization on the fundamental frequency feature vector, second harmonic feature vector and third harmonic feature vector of each image based on the joint optimization strategy to obtain the joint feature vector of each image. The analytical solution module is used to combine the joint feature vectors of each image to obtain the feature matrix of the training set and the feature matrix of the test set; obtain the label matrix of the training set; input the feature matrix and label matrix of the training set into the ridge regression classifier, solve the optimization problem of the ridge regression classifier, and obtain the analytical solution of the output weight matrix of the ridge regression classifier. The prediction category module is used to calculate the predicted category of each image in the test set by taking the analytical solution of the weight matrix output by the classifier and the feature matrix of the test set.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images