A spectral data feature screening method, computer device and storage medium
Through Fourier decomposition and LeBerger spatial transformation, the target matrix is calculated to decompose the spectral data, and a partial least squares regression model is constructed, which solves the problem of feature selection in small samples and improves the accuracy of spectral detection.
Patent Information
- Application Number
- CN202411141620.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-20
AI Technical Summary
In the case of small samples, existing spectral signal analysis methods are difficult to adaptively select representative features, resulting in inaccurate detection results.
Fourier decomposition and LeBerger spatial transformation are used to decompose the spectral data into multiple layers by calculating the target matrix, construct a partial least squares regression model, and adaptively select the characteristics of the spectral data.
It improves the accuracy of spectral characteristics, making the detection results more reliable in small sample scenarios, and is suitable for testing of crops, soil composition, etc.
Smart Images

Figure CN118656619B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of spectral data processing, and in particular to a spectral data feature screening method, a computer device and a storage medium. Background Art
[0002] Hyperspectral technology has high spectral resolution and can quickly obtain hyperspectral reflectance information of continuous bands of the detection target. It is a simple, fast and non-destructive detection method. It has been widely used in crop detection, soil component content detection, food component detection and water plankton detection.
[0003] The researchers extracted crop spectral feature variables and used one or more of the methods such as stepwise regression, principal component analysis, support vector basis, random forest algorithm and continuous wavelet transform to detect indicators such as crop nutrient content, moisture content, chlorophyll content, soil heavy metal content and pH value. In spectral signal analysis, commonly used feature extraction methods include stepwise projection method and vegetation index construction method. In the case of small samples, it is impossible to adaptively select features. In response to the above problems, the present invention proposes a spectral data feature screening method, which can extract more representative target features according to sample values, construct a target component estimation model, and use the target component estimation model to adaptively select the features of the spectral data of the target detection object. Summary of the invention
[0004] In order to overcome the problems existing in the related art, one of the purposes of the present invention is to provide a method for screening spectral data features. The method is based on Fourier decomposition and changes the selection method of the target matrix to extract the features of the spectral data, so that the features of the screened spectral data can construct a stable estimation model, and use the target component estimation model to adaptively select the features of the spectral data of the target detection object.
[0005] A method for screening spectral data features, comprising:
[0006] Acquire original spectral data of the target object, preprocess the original spectral data, and obtain a preprocessed spectral curve;
[0007] Transforming the preprocessed spectral curve into Lebesgue space to obtain a unit circle;
[0008] Calculating a target matrix based on the unit circle;
[0009] Decomposing the preprocessed spectral curve into m layers according to the target matrix to obtain k reconstructed spectral functions;
[0010] The i-th partial least squares regression model corresponding to the i-th group of samples is constructed, the maximum value of the determination coefficient is screened out from all partial least squares regression models, and the target matrix corresponding to the maximum value of the determination coefficient is used as the final spectral feature.
[0011] In a preferred technical solution of the present invention, the step of transforming the preprocessed spectral curve into the Lebesgue space to obtain a unit circle comprises:
[0012] The preprocessed spectral curve is subjected to Hilbert transformation to obtain a unit circle in Lebesgue space.
[0013] In a preferred technical solution of the present invention, the step of calculating the target matrix based on the unit circle includes:
[0014] Divide the unit circle into n equidistant concentric circles; wherein n≥2;
[0015] Dividing each of the equidistant concentric circles into n equally divided sectors;
[0016] The intersection points of the n equidistant concentric circles and the n equally divided sectors form a target matrix.
[0017] In a preferred technical solution of the present invention, before decomposing the preprocessed spectral curve into m layers according to the target matrix, the method further includes:
[0018] Perform Fourier decomposition on the preprocessed spectrum curve to obtain m decomposition items; wherein the basis function of each decomposition item is determined by the elements in the target matrix.
[0019] In a preferred technical solution of the present invention, the preprocessed spectral curve is decomposed into m layers according to the target matrix to obtain k reconstructed spectral functions, including:
[0020] Substitute each element in the target matrix into the corresponding decomposition term;
[0021] Check whether all elements in the target matrix have been substituted into the corresponding decomposition items. If so, k reconstructed spectral functions are obtained; wherein, .
[0022] In a preferred technical solution of the present invention, before selecting the maximum value of the determination coefficient from all partial least squares regression models and taking the target matrix corresponding to the maximum value of the determination coefficient as the final spectral feature, the method further includes:
[0023] The coefficient of determination of the partial least squares regression model corresponding to each group of samples is calculated according to the following formula:
[0024] ;
[0025] in, is the predicted value of the j-th reconstructed spectral function in the i-th group of samples, is the average value of the reconstructed spectral function of the i-th group of samples, is the actual value of the jth reconstructed spectral function in the i-th group of samples, n is the total number of samples in the i-th group of samples, is the coefficient of determination of the partial least squares regression model corresponding to the i-th group of samples.
[0026] In a preferred technical solution of the present invention, the preprocessing of the original spectral data to obtain a preprocessed spectral curve includes:
[0027] Performing SG filtering or wavelet denoising on the original spectral data to obtain pre-processed spectral data;
[0028] The preprocessed spectral data is plotted into a preprocessed spectral curve.
[0029] In a preferred technical solution of the present invention, after obtaining k reconstructed spectral functions, the method further includes:
[0030] Detecting the similarity between k reconstructed spectral functions and the original spectral function; wherein the original spectral function is obtained by drawing the original spectral data;
[0031] If the similarity is less than the similarity threshold, the target matrix is recalculated, and the preprocessed spectral curve is decomposed into m layers according to the target matrix to obtain k reconstructed spectral functions until the similarity is greater than or equal to the similarity threshold.
[0032] A second object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of any of the above-mentioned spectral data feature screening methods when executing the computer program.
[0033] A third object of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the spectral data feature screening method described in any one of the above items are implemented.
[0034] The beneficial effects of the present invention are:
[0035] The spectral data feature screening method provided by the present invention includes obtaining the original spectral data of the target detection object, preprocessing the original spectral data, removing the noise of the original spectral data, and obtaining a preprocessed spectral curve. The preprocessed spectral curve is transformed into the Lebesgue space to obtain a unit circle. The preprocessed spectral curve is decomposed into m layers according to the target matrix to obtain k reconstructed spectral functions, and the reconstructed spectral function has a high similarity with the original spectral data. The i-th partial least squares regression model corresponding to the i-th group of samples is constructed, and the maximum value of the determination coefficient is screened out from all partial least squares regression models, and the target matrix corresponding to the maximum value of the determination coefficient is used as the final spectral feature. The above method changes the selection method of the target matrix, calculates the target matrix in the orthogonal rational system of the unit circle, and constructs the partial least squares regression model. The use of the partial least squares regression model can adaptively select the characteristics of the spectral data of the target detection object, and the accuracy of the final spectral feature obtained is high, which is suitable for small sample application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a schematic flow chart of the spectral data feature screening method of the present invention;
[0037] Figure 2 is a schematic diagram of the original spectral function and the reconstructed spectral function of the present invention;
[0038] Figure 3 Schematic diagram of Fourier decomposition of the present invention. DETAILED DESCRIPTION
[0039] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.
[0040] Example 1
[0041] like Figure 1 As shown, this embodiment provides a method for screening spectral data features, including:
[0042] S1: Acquire original spectral data of the target object, preprocess the original spectral data, and obtain a preprocessed spectral curve.
[0043] Step S1 includes the following steps:
[0044] S11: Acquire the original spectral data of the target object.
[0045] S12: Perform SG filtering or wavelet denoising on the original spectral data to obtain pre-processed spectral data.
[0046] S13: Plotting the preprocessed spectral data into a preprocessed spectral curve.
[0047] The SG filter algorithm is used to smooth the signal. The algorithm uses the least squares method to fit the local data segment, and then uses the fitted function to estimate the value of each data point to achieve smoothing. The SG filter algorithm can effectively filter out high-frequency noise and has good adaptability to nonlinear signals. The SG filter algorithm has a fast calculation speed and does not require frequency domain conversion, so it is suitable for real-time signal processing.
[0048] Wavelet denoising originates from Fourier analysis. Wavelet refers to a wave in a small area, which has non-zero values only in a small range. Wavelets can be translated forward and backward along the time axis, or stretched and compressed proportionally to obtain low-frequency and high-frequency wavelets. The constructed wavelet function can be used to filter or compress signals, thereby extracting useful signals from noisy signals.
[0049] With wavelength as the horizontal axis and amplitude as the vertical axis, the preprocessed spectral data is plotted into a preprocessed spectral curve.
[0050] S2: transforming the preprocessed spectral curve into the Lebesgue space to obtain a unit circle.
[0051] The preprocessed spectral curve is subjected to Hilbert transformation to obtain a unit circle in Lebesgue space.
[0052] The Hilbert transform is a linear operator that produces a function H(u(t)) with the same domain as the function u(t). The Hilbert transform is very important in signal processing. It can derive the analytical representation of the signal u(t), thereby expanding the real signal to a complex signal so that it satisfies the Cauchy-Riemann equation.
[0053] The Hilbert transform is defined for periodic functions, i.e. functions on a circle, in which case the Hilbert transform is the convolution of a real signal with the Hilbert kernel.
[0054] S3: Calculate a target matrix based on the unit circle.
[0055] Step S3 includes the following steps S31-S33:
[0056] S31: Divide the unit circle into n equidistant concentric circles; wherein n≥2.
[0057] S32: Divide each of the equidistant concentric circles into n equally divided sectors.
[0058] S33: The intersection points of the n equidistant concentric circles and the n equally divided sectors form a target matrix.
[0059] As an example, obtain the unit circle, multiply the radius of the unit circle by 0.2, 0.4, 0.6, and 0.8 to obtain multiple proportional radii. With the center of the unit circle as the center point, draw four equidistant concentric circles according to the proportional radius, plus the unit circle, a total of 5 equidistant concentric circles are formed, that is, n=5. After forming 5 equidistant concentric circles, each equidistant concentric circle is divided into 5 sectors. The intersection of each equidistant concentric circle and the sector dividing line is the matrix element, and all matrix elements form the target matrix.
[0060] The size of the target matrix is ,For example or The target matrix is constructed using the intersection of equidistant concentric circles and sector dividing lines, so that the target matrix has more matrix elements and can better reflect the spectral characteristics of the target detection object.
[0061] S4: Decomposing the preprocessed spectral curve into m layers according to the target matrix to obtain k reconstructed spectral functions.
[0062] Substitute each element in the target matrix into the corresponding decomposition term;
[0063] Check whether all elements in the target matrix have been substituted into the corresponding decomposition items. If so, k reconstructed spectral functions are obtained; wherein, .
[0064] Each matrix element in the target matrix is substituted into the decomposition term of the adaptive Fourier decomposition, and the original spectral function is decomposed into m layers, so that the similarity between the reconstructed spectral function and the original spectral function reaches more than 98%.
[0065] Each original spectral function has The reconstructed spectral function corresponds to The decomposition terms of each group are calculated according to the following formula: m groups of decomposition functions corresponding to each group of decomposition terms in the unitized Szego kernel e ak Projection on:
[0066] ;
[0067] in, <f l ,e ak > is the decomposition function in the unitized Szego kernel e ak The projection on l is the lth decomposition term, e ak is the unitized Szego kernel, f k is the kth decomposition term, is the unitized Szego kernel e ak The conjugation of.
[0068] The matrix elements of the target matrix are selected in an orthogonal rational system of the unit circle, where the key is the Lebesgue space L at a point a in the unit circle on the complex plane. 2 The Szego kernel with internal unit modeling.
[0069] S5: Construct the i-th partial least squares regression model corresponding to the i-th group of samples, screen out the maximum value of the determination coefficient from all partial least squares regression models, and use the target matrix corresponding to the maximum value of the determination coefficient as the final spectral feature.
[0070] There are g groups of samples in total, and the spectral functions corresponding to each group of samples are subjected to data processing of steps S1-S4 to construct a partial least squares regression model for each group of samples, where i≤g.
[0071] The determination coefficient was used to evaluate the partial least squares model results, and the target matrix corresponding to the maximum value of the determination coefficient was taken as the final spectral feature.
[0072] The spectral data feature screening method of the present embodiment includes obtaining the original spectral data of the target detection object, preprocessing the original spectral data, removing the noise of the original spectral data, and obtaining a preprocessed spectral curve. The preprocessed spectral curve is transformed into the Lebesgue space to obtain a unit circle. The preprocessed spectral curve is decomposed into m layers according to the target matrix to obtain k reconstructed spectral functions, and the reconstructed spectral function has a high similarity with the original spectral data. The i-th partial least squares regression model corresponding to the i-th group of samples is constructed, and the maximum value of the determination coefficient is screened out from all partial least squares regression models, and the target matrix corresponding to the maximum value of the determination coefficient is used as the final spectral feature. The above method changes the selection method of the target matrix, calculates the target matrix in the orthogonal rational system of the unit circle, and constructs the partial least squares regression model. The use of the partial least squares regression model can adaptively select the characteristics of the spectral data of the target detection object, and the accuracy of the final spectral feature obtained is high, which is suitable for small sample application scenarios.
[0073] Example 2
[0074] This embodiment provides a method for screening spectral data features. This embodiment only describes the differences from Embodiment 1. Before decomposing the preprocessed spectral curve into m layers according to the target matrix, it also includes:
[0075] Perform Fourier decomposition on the preprocessed spectrum curve to obtain m decomposition items; wherein the basis function of each decomposition item is determined by the elements in the target matrix.
[0076] Substitute each matrix element in the target matrix into the decomposition term of the adaptive Fourier decomposition, and decompose the original spectral function into m layers, so that the similarity between the reconstructed spectral function and the original spectral function reaches more than 98%. The basis function of the adaptive Fourier decomposition is a trigonometric function, which is determined according to the matrix elements in the target matrix.
[0077] The method further includes: selecting the maximum value of the determination coefficient from all partial least squares regression models, and taking the target matrix corresponding to the maximum value of the determination coefficient as the final spectral feature.
[0078] The coefficient of determination of the partial least squares regression model corresponding to each group of samples is calculated according to the following formula:
[0079] ;
[0080] in, is the predicted value of the j-th reconstructed spectral function in the i-th group of samples, is the average value of the reconstructed spectral function of the i-th group of samples, is the actual value of the jth reconstructed spectral function in the i-th group of samples, n is the total number of samples in the i-th group of samples, is the coefficient of determination of the partial least squares regression model corresponding to the i-th group of samples.
[0081] The coefficient of determination represents the numerical characteristics of the relationship between a random variable and multiple random variables. It is a statistical indicator used to reflect the reliability of changes in the dependent variable under the regression model. The coefficient of determination is defined as the ratio of the variation of the independent variable explained by all independent variables in the model to the total variation of the independent variable.
[0082] The larger the determination coefficient, the higher the reliability of the dependent variable changes in the regression model and the higher the accuracy of the screened spectral data features.
[0083] After obtaining k reconstructed spectral functions, the method further includes:
[0084] Detecting the similarity between k reconstructed spectral functions and the original spectral function; wherein the original spectral function is obtained by drawing the original spectral data;
[0085] If the similarity is less than the similarity threshold, the target matrix is recalculated, and the preprocessed spectral curve is decomposed into m layers according to the target matrix to obtain k reconstructed spectral functions until the similarity is greater than or equal to the similarity threshold.
[0086] The similarity threshold is 95%-98%. In this embodiment, the similarity threshold is 95%, the target object is tea, there are 30 samples in total, and the target matrix size is For example.
[0087] The target matrix is expressed as:
[0088] ;
[0089] like Figure 2 and Figure 3 As shown, the number of decomposition layers is ten, and a partial least squares regression model is constructed for each set of features. The square of the maximum value of the coefficient of determination R 2 It is 0.82.
[0090] After obtaining k reconstructed spectral functions, the present embodiment also includes detecting the similarity between the k reconstructed spectral functions and the original spectral functions; wherein the original spectral function is obtained by drawing the original spectral data. If the similarity is less than the similarity threshold, the target matrix is recalculated, and the preprocessed spectral curve is decomposed into m layers according to the target matrix to obtain k reconstructed spectral functions until the similarity is greater than or equal to the similarity threshold. If the similarity is less than the similarity threshold, it means that the credibility of the target matrix is low, and the restoration degree of the preprocessed spectral curve after decomposing the preprocessed spectral curve using the target matrix is low. Therefore, the value of n is adjusted, and the unit circle is re-divided into n equidistant concentric circles, and each equidistant concentric circle is divided into n equally divided sectors, and the target matrix is updated until the similarity between the k reconstructed spectral functions reconstructed according to the target matrix and the original spectral function is greater than or equal to the similarity threshold. At this time, the credibility of the target matrix is high, and the restoration degree of the preprocessed spectral curve after decomposing the preprocessed spectral curve using the target matrix is high.
[0091] Example 3
[0092] A computer device is also provided in an embodiment of the present application, which may be a server, wherein the computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target word vectors, etc. The network interface of the computer device is used to communicate with an external terminal via a network connection. Furthermore, the above-mentioned computer device may also be provided with an input device and a display screen, etc.
[0093] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method for cross-modal data representation is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.
[0094] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.
[0095] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A method for screening spectral data features, characterized in that: include: Acquire original spectral data of the target object, preprocess the original spectral data, and obtain a preprocessed spectral curve; Transforming the preprocessed spectral curve into Lebesgue space to obtain a unit circle; Calculating a target matrix based on the unit circle; Decomposing the preprocessed spectral curve into m layers according to the target matrix to obtain k reconstructed spectral functions; Construct the i-th partial least squares regression model corresponding to the i-th group of samples, screen out the maximum value of the determination coefficient from all partial least squares regression models, and use the target matrix corresponding to the maximum value of the determination coefficient as the final spectral feature; The step of transforming the preprocessed spectral curve into the Lebesgue space to obtain a unit circle includes: Performing Hilbert transformation on the preprocessed spectral curve to obtain a unit circle in Lebesgue space; The calculating of the target matrix based on the unit circle comprises: Divide the unit circle into n equidistant concentric circles; wherein n≥2; Dividing each of the equidistant concentric circles into n equally divided sectors; The intersection points of the n equally spaced concentric circles and the n equally divided sectors form a target matrix; Before decomposing the preprocessed spectral curve into m layers according to the target matrix, the method further includes: Perform Fourier decomposition on the preprocessed spectrum curve to obtain m decomposition items; wherein the basis function of each decomposition item is determined by the elements in the target matrix.
2. The method for screening spectral data features according to claim 1, characterized in that: The step of decomposing the preprocessed spectral curve into m layers according to the target matrix to obtain k reconstructed spectral functions includes: Substitute each element in the target matrix into the corresponding decomposition term; Check whether all elements in the target matrix have been substituted into the corresponding decomposition items. If so, k reconstructed spectral functions are obtained; wherein, .
3. The method for screening spectral data features according to claim 1, characterized in that: The method further includes: selecting the maximum value of the determination coefficient from all partial least squares regression models, and taking the target matrix corresponding to the maximum value of the determination coefficient as the final spectral feature. The coefficient of determination of the partial least squares regression model corresponding to each group of samples is calculated according to the following formula: ; in, is the predicted value of the j-th reconstructed spectral function in the i-th group of samples, is the average value of the reconstructed spectral function of the i-th group of samples, is the actual value of the jth reconstructed spectral function in the i-th group of samples, n is the total number of samples in the i-th group of samples, is the determination coefficient of the partial least squares regression model corresponding to the i-th group of samples.
4. The method for screening spectral data features according to claim 1, characterized in that: The preprocessing of the original spectral data to obtain a preprocessed spectral curve includes: Performing SG filtering or wavelet denoising on the original spectral data to obtain pre-processed spectral data; The preprocessed spectral data is plotted into a preprocessed spectral curve.
5. The method for screening spectral data features according to claim 1, characterized in that: After obtaining k reconstructed spectral functions, the method further includes: Detecting the similarity between k reconstructed spectral functions and the original spectral function; wherein the original spectral function is obtained by drawing the original spectral data; If the similarity is less than the similarity threshold, the target matrix is recalculated, and the preprocessed spectral curve is decomposed into m layers according to the target matrix to obtain k reconstructed spectral functions until the similarity is greater than or equal to the similarity threshold.
6. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the spectral data feature screening method according to any one of claims 1 to 5 are implemented.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the spectral data feature screening method according to any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
A hyperspectral remote sensing image segmentation method based on spectral distance of spectral curve
CN109377507A
Fourier infrared spectrometer data processing and analyzing system
CN115561193A
Spectral data preprocessing method and device, computer equipment and storage medium
CN115797318A
Method of and system for blind extraction of more pure components than mixtures in id and 2d NMR spectroscopy and mass spectrometry by means of combined sparse component analysis and detection of single component points
WO2011030172A1
Near infrared spectral feature extraction method and system based on functional principal component analysis
WO2020232959A1
