Micro fiber spectrometer spectral calibration method, system, electronic device and medium

Abstract

The present application relates to a kind of micro fiber spectrometer spectral calibration method, system, electronic equipment and medium, specifically relates to measuring instrument calibration technical field.The method includes obtaining reference dataset, to be calibrated dataset and intercept dataset;According to intercept dataset, determine first threshold;The spectrum value in to be calibrated dataset greater than first threshold is determined as effective data;The spectrum value corresponding to effective data in reference dataset is determined as reserved data;According to the form of the RGB value of reserved data and effective data, determine data combination, obtain dataset according to data combination;Using dataset, train and verify neural network to obtain calibration model, and using calibration model carries out calibration.The present application can improve the measurement precision and accuracy of micro fiber spectrometer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of measuring instrument calibration, in particular to a micro fiber spectrometer spectrum calibration method and system, electronic equipment and medium. BACKGROUND

[0002] The micro fiber spectrometer has the advantages of small size, low price and light weight. Its detection process is simple, and it can be detected online and in real time, and has a wider application environment. However, due to the limitations of existing manufacturing processes, integration, and micro-optical machine technologies, the micro fiber spectrometer will be affected by stray light, thermal noise, pulse broadening, etc., resulting in measurement errors and lower accuracy than large integrating sphere spectrometers. Therefore, while ensuring the advantages of small size and low cost of the micro fiber spectrometer, it is of great research significance to improve the measurement precision and accuracy. The existing calibration method is highly targeted and has poor generalization ability. SUMMARY

[0003] The purpose of the present application is to provide a micro fiber spectrometer spectrum calibration method, system, electronic equipment and medium, which can improve the generalization ability of the calibration method of the micro fiber spectrometer.

[0004] To achieve the above purpose, the present application provides the following scheme:

[0005] A micro fiber spectrometer spectrum calibration method, comprising:

[0006] obtaining a reference data set, a to-be-calibrated data set and a cut-off data set; the reference data set includes a spectrum data set of a reference integrating sphere spectrometer at each set wavelength; the to-be-calibrated data set includes a spectrum data set of a to-be-calibrated micro fiber spectrometer at each set wavelength; the cut-off data set includes a spectrum data set of the reference integrating sphere spectrometer at each set wavelength without light; one set wavelength corresponds to one spectrum data set; one spectrum data set includes spectrum values at different RGB values;

[0007] determining a first threshold value according to the cut-off data set; the first threshold value is twice the maximum spectrum value in the cut-off data set;

[0008] determining the spectrum values greater than the first threshold value in the to-be-calibrated data set as effective data;

[0009] determining the spectrum values in the reference data set corresponding to the same set wavelength and the same RGB value as the effective data as reserved data;

[0010] determining forms of each RGB value according to the RGB value; the forms include R=G=0, B=n, R=B=0, G=n, and B=G=0, R=n, n being a positive integer;

[0011] determining reserved data corresponding to the RGB values of the same form as a group to obtain reserved combination data corresponding to each form of RGB value;

[0012] determining effective data corresponding to the RGB values of the same form as a group to obtain to-be-calibrated combination data corresponding to each form of RGB value;

[0013] selecting one spectral value in the reserved combination data corresponding to each form of RGB value to form a reserved data combination to obtain a plurality of reserved data combinations; the spectral values in one of the reserved data combinations correspond to the same set wavelength;

[0014] selecting one spectral value in the to-be-calibrated combination data corresponding to each form of RGB value to form a to-be-calibrated data combination to obtain a plurality of to-be-calibrated data combinations; the spectral values in one of the to-be-calibrated data combinations correspond to the same set wavelength;

[0015] for any one reserved data combination, adding each spectral value in the reserved data combination to obtain a spectral value corresponding to the reserved data combination;

[0016] for any one to-be-calibrated data combination, adding each spectral value in the to-be-calibrated data combination to obtain a spectral value corresponding to the to-be-calibrated data combination;

[0017] training and verifying the neural network using a data set to obtain a calibration model, and calibrating a to-be-tested spectral value using the calibration model; the data set includes spectral values corresponding to all to-be-calibrated data combinations and spectral values corresponding to all reserved data combinations.

[0018] Optionally, the training and verifying the neural network using a data set to obtain a calibration model specifically includes:

[0019] dividing the data set into a verification set and a training set according to a set proportion; the training set and the verification set each include a plurality of spectral data groups, one spectral data group including reference data values and to-be-calibrated data values of the same set wavelength and the same RGB value, the reference data values being spectral values corresponding to a reference integrating sphere spectrometer, and the to-be-calibrated data values being spectral values corresponding to a to-be-calibrated micro optical fiber spectrometer;

[0020] training the neural network using the training set to obtain a trained neural network;

[0021] verifying the trained neural network using the verification set to obtain a calibration model.

[0022] Optionally, the training of the neural network with the data set and the verification to obtain the calibration model, and the calibration of the to-be-tested spectrum value with the calibration model further comprise:

[0023] performing confidence verification on the calibration data value to determine whether the calibration data value reaches a confidence threshold; the calibration data value is obtained by calibrating the to-be-tested spectrum value with the calibration model;

[0024] if the calibration data value reaches the confidence threshold, taking the to-be-tested spectrum value and the calibration data value as a new training set, and training and verifying the calibration model with the new training set and the verification set.

[0025] Optionally, the confidence verification on the calibration data value to determine whether the calibration data value reaches the confidence threshold comprises:

[0026] obtaining a target wavelength, a to-be-tested spectrum value, and a target predicted spectrum value; the target wavelength is a wavelength corresponding to the to-be-tested spectrum value, and the target predicted spectrum value is obtained by inputting the to-be-tested spectrum value into the calibration model;

[0027] selecting spectrum values corresponding to the target wavelength from the to-be-calibrated data set and the reference data set respectively to obtain a first to-be-calibrated data set and a first reference data set;

[0028] sequentially arranging the spectrum values in the first to-be-calibrated data set and the first reference data set in ascending order to obtain a to-be-calibrated data set sequence and a reference data set sequence;

[0029] selecting a first to-be-calibrated spectrum value and a second to-be-calibrated spectrum value from the to-be-calibrated data set; the first to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the to-be-calibrated data set sequence; the second to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the to-be-calibrated data set sequence;

[0030] selecting a first reference spectrum value and a second reference spectrum value from the reference data set; the first reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the reference data set sequence; the second reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the reference data set sequence;

[0031] determining whether a product of a first difference value and a second difference value is less than 0 to obtain a first determination result; the first difference value is a difference between the to-be-tested spectrum value and the target predicted spectrum value; the second difference value is a difference between the first to-be-calibrated spectrum value and the first reference spectrum value;

[0032] if the first determination result is no, determining whether an absolute value of the first difference value is greater than an absolute value of the second difference value and an absolute value of a third difference value to obtain a second determination result; the third difference value is a difference between the second to-be-calibrated spectrum value and the second reference spectrum value;

[0033] if the second determination result is yes, determining that the calibration data value reaches a confidence threshold.

[0034] A micro fiber spectrum instrument spectrum calibration system, comprising:

[0035] an acquisition module, configured to acquire a reference data set, a to-be-calibrated data set and a cut-off data set; the reference data set comprises spectrum data sets of a reference integrating sphere spectrometer at each set wavelength; the to-be-calibrated data set comprises spectrum data sets of a to-be-calibrated micro fiber spectrum instrument at each set wavelength; the cut-off data set comprises spectrum data sets of the reference integrating sphere spectrometer at each set wavelength without light; one set wavelength corresponds to one spectrum data set; one spectrum data set comprises spectrum values at different RGB values;

[0036] a first threshold determination module, configured to determine a first threshold according to the cut-off data set; the first threshold is twice a spectrum value with a maximum value in the cut-off data set;

[0037] an effective data determination module, configured to determine, as effective data, spectrum values in the to-be-calibrated data set greater than the first threshold;

[0038] a reserved data determination module, configured to determine, as reserved data, spectrum values in the reference data set corresponding to the effective data and having the same set wavelength and the same RGB value as the effective data;

[0039] an RGB value form determination module, configured to determine a form of each RGB value according to the RGB value; the form comprises R=G=0, B=n, R=B=0, G=n and B=G=0, R=n, n being a positive integer;

[0040] a reserved combination data determination module, configured to determine, as reserved combination data corresponding to each form of RGB value, reserved data corresponding to the same form of RGB value;

[0041] The to-be-calibrated combination data determination module is configured to determine effective data corresponding to RGB values of the same form as a set of to-be-calibrated combination data corresponding to RGB values of each form;

[0042] The reserved data combination determination module is configured to select one spectrum value from the reserved combination data corresponding to RGB values of each form to form a reserved data combination, so as to obtain a plurality of reserved data combinations; the spectrum values in one reserved data combination correspond to the same set wavelength;

[0043] The to-be-calibrated data combination determination module is configured to select one spectrum value from the to-be-calibrated combination data corresponding to RGB values of each form to form a to-be-calibrated data combination, so as to obtain a plurality of to-be-calibrated data combinations; the spectrum values in one to-be-calibrated data combination correspond to the same set wavelength;

[0044] The reserved spectrum value updating module is configured to, for any one reserved data combination, add the spectrum values in the reserved data combination to obtain a spectrum value corresponding to the reserved data combination;

[0045] The to-be-trained spectrum value updating module is configured to, for any one to-be-calibrated data combination, add the spectrum values in the to-be-calibrated data combination to obtain a spectrum value corresponding to the to-be-calibrated data combination;

[0046] The training, verification and calibration module is configured to train and verify a neural network by using a data set to obtain a calibration model, and calibrate a to-be-tested spectrum value by using the calibration model; the data set includes spectrum values corresponding to all to-be-calibrated data combinations and spectrum values corresponding to all reserved data combinations.

[0047] Optionally, the training, verification and calibration module specifically includes:

[0048] The verification set and training set determination unit is configured to divide the data set into a verification set and a training set according to a set proportion; the training set and the verification set each include a plurality of spectrum data groups, and one spectrum data group includes reference data values and to-be-calibrated data values of the same RGB values and the same set wavelength, the reference data values are spectrum values corresponding to a reference integrating sphere spectrometer, and the to-be-calibrated data values are spectrum values corresponding to a to-be-calibrated micro optical fiber spectrometer;

[0049] The training unit is configured to train the neural network by using the training set to obtain a trained neural network;

[0050] The verification unit is configured to verify the trained neural network by using the verification set to obtain a calibration model.

[0051] Optionally, the micro optical fiber spectrometer spectrum calibration system further includes:

[0052] The confidence threshold judgment module is configured to perform confidence verification on the calibration data value to determine whether the calibration data value reaches a confidence threshold.

[0053] The training set updating module is configured to, if the calibration data value reaches the confidence threshold, take the to-be-tested spectrum value and the calibration data value as a new training set, and train and verify the calibration model by using the new training set and the verification set.

[0054] Optionally, the confidence threshold judgment module specifically includes:

[0055] The basic data acquisition unit is configured to acquire a target wavelength, a to-be-tested spectrum value, and a target predicted spectrum value; the target wavelength is a wavelength corresponding to the to-be-tested spectrum value, and the target predicted spectrum value is obtained by inputting the to-be-tested spectrum value into the calibration model;

[0056] The first selection unit is configured to select spectrum values corresponding to the target wavelength from the to-be-calibrated data set and the reference data set respectively to obtain a first to-be-calibrated data set and a first reference data set;

[0057] The sorting unit is configured to sort the spectrum values in the first to-be-calibrated data set and the first reference data set in ascending order respectively to obtain a to-be-calibrated data set sequence and a reference data set sequence;

[0058] The second selection unit is configured to select a first to-be-calibrated spectrum value and a second to-be-calibrated spectrum value from the to-be-calibrated data set; the first to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the to-be-calibrated data set sequence; and the second to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the to-be-calibrated data set sequence;

[0059] The third selection unit is configured to select a first reference spectrum value and a second reference spectrum value from the reference data set; the first reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the reference data set sequence; and the second reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the reference data set sequence;

[0060] The first judgment unit is configured to determine whether a product of a first difference value and a second difference value is less than 0 to obtain a first judgment result; the first difference value is a difference between the to-be-tested spectrum value and the target predicted spectrum value; and the second difference value is a difference between the first to-be-calibrated spectrum value and the first reference spectrum value;

[0061] A second judging unit is configured to, if the first judging result is no, judge whether the absolute value of the first difference value is greater than the absolute value of a second difference value and the absolute value of a third difference value, to obtain a second judging result; the third difference value is a difference value between the second to-be-calibrated spectrum value and the second reference spectrum value.

[0062] A confidence threshold judging unit is configured to, if the second judging result is yes, determine that the calibration data value reaches a confidence threshold.

[0063] An electronic device comprises:

[0064] A memory is configured to store a computer program, and a processor is configured to run the computer program to enable the electronic device to execute the micro fiber spectrum calibrating method.

[0065] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the micro fiber spectrum calibrating method.

[0066] According to the embodiments of the present application, the following technical effects are achieved: the calibration model is used to calibrate the spectrum data, the calibration of the spectrum instrument is realized without changing the hardware structure of the measuring instrument, and the measurement precision and accuracy of the micro fiber spectrum instrument are improved. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0068] Figure 1 The micro fiber spectrum calibrating method provided by the embodiments of the present application is shown in the specific flowchart.

[0069] Figure 2 The micro fiber spectrum calibrating method provided by the embodiments of the present application is shown in the general flowchart.

[0070] Figure 3 The neural network structure provided by the embodiments of the present application is shown in the general flowchart.

[0071] Figure 4 The confidence verification flowchart provided by the embodiments of the present application is shown in the general flowchart.

[0072] Figure 5 The flowchart of the adaptive iterative optimization according to the real working condition data provided by the embodiments of the present application is shown in the general flowchart. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0074] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0075] The present application proposes a new solution idea, introduces a deep learning algorithm, improves the generalization of the calibration method from the data perspective, and makes it applicable to various measurement tasks. At the same time, the semi-supervised learning idea is introduced to realize the continuous automatic optimization of the calibration model in the application and improve the accuracy.

[0076] As shown in Figure 1 The present application provides a micro fiber spectrometer spectrum calibration method, which comprises the following steps:

[0077] S101: Obtain a reference data set, a to-be-calibrated data set, and a cut-off data set. The reference data set comprises spectrum data sets of a reference integrating sphere spectrometer at each set wavelength; the to-be-calibrated data set comprises spectrum data sets of a to-be-calibrated micro fiber spectrometer at each set wavelength; the cut-off data set comprises spectrum data sets of the reference integrating sphere spectrometer at each set wavelength without light; one set wavelength corresponds to one spectrum data set; and one spectrum data set comprises spectrum values at different RGB values.

[0078] S102: Determine a first threshold value according to the cut-off data set. The first threshold value is twice the maximum spectrum value in the cut-off data set.

[0079] S103: Determine the spectrum values greater than the first threshold value in the to-be-calibrated data set as effective data.

[0080] S104: Determine the spectrum values in the reference data set, which are the same as the set wavelength corresponding to the effective data and the same as the RGB value corresponding to the effective data, as reserved data.

[0081] S105: Determine the form of each RGB value according to the RGB value. The form comprises R=G=0, B=n, R=B=0, G=n, and B=G=0, R=n, wherein n is a positive integer.

[0082] S106: Determine the reserved data corresponding to the RGB values of the same form as a group to obtain the reserved combination data corresponding to the RGB values of each form.

[0083] S107: Determine the effective data corresponding to the RGB values of the same form as a group to obtain the to-be-calibrated combination data corresponding to the RGB values of each form.

[0084] S108: Select one spectrum value in the reserved combination data corresponding to the RGB values of each form to form a reserved data combination, to obtain a plurality of reserved data combinations; the spectrum values in one of the reserved data combinations correspond to the same set wavelength.

[0085] S109: Select one spectrum value in the to-be-calibrated combination data corresponding to the RGB values of each form to form a to-be-calibrated data combination, to obtain a plurality of to-be-calibrated data combinations; the spectrum values in one of the to-be-calibrated data combinations correspond to the same set wavelength.

[0086] S110: For any one reserved data combination, add the spectrum values in the reserved data combination to obtain the spectrum value corresponding to the reserved data combination.

[0087] S111: For any one to-be-calibrated data combination, add the spectrum values in the to-be-calibrated data combination to obtain the spectrum value corresponding to the to-be-calibrated data combination.

[0088] S112: Train and verify the neural network using a data set to obtain a calibration model, and calibrate the to-be-tested spectrum value using the calibration model; the data set includes the spectrum values corresponding to all to-be-calibrated data combinations and the spectrum values corresponding to all reserved data combinations.

[0089] In actual application, the training and verification of the neural network using a data set to obtain a calibration model specifically includes:

[0090] The data set is divided into a verification set and a training set according to a set proportion; the training set and the verification set each include a plurality of spectrum data groups, and one spectrum data group includes reference data values and to-be-calibrated data values of the same RGB values and the same set wavelength, the reference data values are the spectrum values corresponding to a reference integrating sphere spectrometer, and the to-be-calibrated data values are the spectrum values corresponding to a to-be-calibrated micro optical fiber spectrometer.

[0091] The training set is used to train the neural network to obtain a trained neural network.

[0092] The verification set is used to verify the trained neural network to obtain a calibration model.

[0093] In practical applications, the training and verification of the neural network by using the data set, and the calibration of the to-be-tested spectrum value by using the calibration model further comprise:

[0094] The calibration data value is subjected to confidence verification to determine whether the calibration data value reaches a confidence threshold.

[0095] If the calibration data value reaches the confidence threshold, the to-be-tested spectrum value and the calibration data value are used as a new training set, and the calibration model is trained and verified by using the new training set and the verification set.

[0096] In practical applications, the confidence verification of the calibration data value to determine whether the calibration data value reaches a confidence threshold specifically comprises:

[0097] The target wavelength, the to-be-tested spectrum value, and the target predicted spectrum value are obtained; the target wavelength is a wavelength corresponding to the to-be-tested spectrum value, and the target predicted spectrum value is obtained by inputting the to-be-tested spectrum value into the calibration model.

[0098] Spectrum values corresponding to the target wavelength in the to-be-calibrated data set and the reference data set are selected to obtain a first to-be-calibrated data set and a first reference data set.

[0099] The spectrum values in the first to-be-calibrated data set and the first reference data set are sorted in ascending order to obtain a to-be-calibrated data set sequence and a reference data set sequence.

[0100] The first to-be-calibrated spectrum value and the second to-be-calibrated spectrum value are selected in the to-be-calibrated data set; the first to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the to-be-calibrated data set sequence; and the second to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the to-be-calibrated data set sequence.

[0101] The first reference spectrum value and the second reference spectrum value are selected in the reference data set; the first reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the reference data set sequence; and the second reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the reference data set sequence.

[0102] determining whether a product of a first difference value and a second difference value is less than 0 to obtain a first determination result; the first difference value is a difference between the to-be-tested spectrum value and the target predicted spectrum value; the second difference value is a difference between the first to-be-calibrated spectrum value and the first reference spectrum value.

[0103] if the first determination result is no, determining whether an absolute value of the first difference value is greater than an absolute value of the second difference value and an absolute value of a third difference value to obtain a second determination result; the third difference value is a difference between the second to-be-calibrated spectrum value and the second reference spectrum value.

[0104] if the second determination result is yes, determining that the calibration data value reaches a confidence threshold.

[0105] The application further provides a specific embodiment for further illustrating the above method, and the specific embodiment is as follows:

[0106] S1: obtaining an original data set.

[0107] The original data set is divided into two parts, which are spectrum values of a reference spectrometer (a reference integrating sphere spectrometer) in a wavelength interval of 380nm-780nm (380nm, 381nm,..., 780nm) and RGB values of 0 / 0 / 80, 0 / 0 / 160, 0 / 0 / 240, 0 / 80 / 0, 0 / 160 / 0, 0 / 240 / 0, 80 / 0 / 0, 160 / 0 / 0 and 240 / 0 / 0, and spectrum values of a to-be-calibrated spectrometer (a to-be-calibrated micro optical fiber spectrometer) under the same illumination condition, which are defined as “reference data” and “to-be-calibrated data” respectively in the embodiment. One “to-be-calibrated data” and one corresponding “reference data” form a group of data, and a total of 3609 groups of data are collected, wherein the same illumination condition is that the same LED, the same illumination RGB value modulation and the same external illumination environment are adopted.

[0108] S2: performing data preprocessing on the above original data set.

[0109] The data preprocessing in the embodiment specifically includes:

[0110] The reference spectrometer is used to collect spectrum data under the condition that the LED is not lit in the wavelength interval of 380nm-780nm, and the maximum spectrum data is selected and recorded as M.

[0111] All spectrum values greater than 2M (a first threshold value) in all to-be-calibrated data values are retained as effective data, and spectrum values less than 2M are deleted, and reference data with the same wavelength and RGB value are deleted as retained data.

[0112] The data of RGB=0 / 0 / n (R=G=0, B=n) is defined as group 1, the data of RGB=0 / n / 0 (R=B=0, G=n) is defined as group 2, and the data of RGB=n / 0 / 0 (B=G=0, R=n) is defined as group 3, and the valid data and the reserved data are respectively operated as follows: the data of each wavelength corresponding to one RGB in the three groups of data is extracted and added to form a data combination, which is used as a data set for preliminary training and verification of the neural network, and the ratio of the verification set to the training set is divided into 3:7 during training; the verification set needs to be evenly distributed to each wavelength; the training epoch parameter is 10000; the training batch size is 256, and the final verification set loss value is A.

[0113] According to the data combination, the process of preliminary training and verification of the neural network is as follows: using the to-be-calibrated data (shown as "training sample X" in Figure 2 ) and the reference data (shown as "training sample Y" in Figure 2 ) as training data, the neural network shown in Figure 3 is trained, the ratio of the verification set to the training set is divided into 3:7 during training, the verification set is evenly distributed to each wavelength, the training epoch parameter is 10000, the training batch size is 256, and a calibration model is obtained, as shown in Figure 3 , the neural network in the present application adopts a five-layer neural network structure, which is divided into an input layer, a hidden layer and an output layer, the hidden layer is a three-layer fully connected network structure, the number of neurons in the first layer is 32, the number of neurons in the second layer is 32, and the number of neurons in the third layer is 64. The input size and the output size are both 1, the activation function adopts a Relu function; the optimizer selects an Adam function (learning rate=0.0001); L2 regularization is added (regularization parameter=0.03); and the input data is not standardized or normalized.

[0114] S3: performing a spectral radiation calibration work step.

[0115] When the spectral radiation calibration work step is performed, it is actually put into actual working conditions, and a to-be-calibrated spectrometer of the same model as that used during training needs to be used for testing and measurement, the input signal is the spectral data of the to-be-calibrated spectrometer, i.e. the to-be-tested spectral value, and the output signal is the calibrated data, i.e. the target predicted spectral value.

[0116] As shown in Figure 2 , a micro fiber spectrometer adaptive optimization radiation calibration method process is roughly as follows: a to-be-calibrated sample for testing, i.e. a to-be-tested spectral value (shown as "test sample X'" in Figure 2 and Figure 5 ), is obtained, and the data is output by the calibration model to obtain calibrated data, i.e. a target predicted spectral value (shown as "predicted result Y'" in Figure 2 and Figure 5 ).

[0117] The prediction result Y' is subjected to confidence verification to determine whether it reaches a threshold value, if not, it is output as a normal result, if it reaches the confidence threshold value, it is updated and trained as a new training set for the calibration model together with the corresponding test sample X', the validation set is unchanged during training, the training epoch parameter is 10000, the training batch size is 256, a trained calibration model is obtained, and the next batch of test samples based on iteration optimization is carried out.

[0118] In practical application, the calibration model is updated and trained as a new training set together with the corresponding test sample X', the specific steps are as follows, Figure 5 As shown in the figure, when the number of prediction results Y' exceeding the confidence threshold value reaches 1 / 3 of the training sample Y, the neural network model is verified and trained once, that is, in actual work, when the number of Y' reaches 1 / 3 of the training sample Y, the calibration model is trained and verified once, the calibration model weight parameter is used as the pre-training model, the test sample X' and the prediction result Y' are used as the training set for training, and the validation set is unchanged; the training epoch parameter is 10000, the training batch size is 256, and whether the current validation accuracy is lower than the original validation accuracy is observed after training; if it is less than the validation loss value A of the last training, the current validation accuracy replaces the original A, the training is retained, which is called one iteration optimization, and the trained calibration model becomes a new calibration model; when the final validation set loss value, that is, the validation set loss value after training, is greater than or equal to A, A remains unchanged, and the training is discarded.

[0119] As shown in the figure, Figure 4 The confidence verification method in the application comprises the following steps:

[0120] The first step is to set the wavelength of the to-be-tested spectrum value as W, the spectrum data corresponding to the wavelength as X', and the predicted spectrum value as Y' 。

[0121] The second step is to retrieve all spectrum values corresponding to the wavelength in the original data set, and sort the spectrum values in the to-be-calibrated data from small to large as X o1 , X o2 , X o3 , …X on , the spectrum values in the reference data from small to large as Y o1 , Y o2 , Y o3 , …Y on , n is the original data amount; the specific position of X' in the to-be-calibrated data is found according to the value size, so that X om <X’<X om+1 ; the difference between X' and Y' is calculated, if (X om -Y om<0, Y' is discarded; in (X'-Y')*(X om - Y om > 0, if |(Y om - X om | < |(X'-Y')| < |(X om+1 - Y om+1 |, Y' reaches the confidence threshold, and is stored in a temporary database of new training samples, for preparing the next iteration of training, otherwise, Y' is discarded.

[0122] The present application provides a miniature fiber spectrometer spectral calibration system for the above method, comprising:

[0123] An acquisition module is configured to acquire a reference dataset, a to-be-calibrated dataset, and a cut-off dataset; the reference dataset comprises spectral datasets of a reference integrating sphere spectrometer at each set wavelength; the to-be-calibrated dataset comprises spectral datasets of a to-be-calibrated miniature fiber spectrometer at each set wavelength; the cut-off dataset comprises spectral datasets of the reference integrating sphere spectrometer at each set wavelength without light; one set wavelength corresponds to one spectral dataset; one spectral dataset comprises spectral values at different RGB values.

[0124] A first threshold determination module is configured to determine a first threshold according to the cut-off dataset; the first threshold is twice the maximum spectral value in the cut-off dataset.

[0125] An effective data determination module is configured to determine spectral values greater than the first threshold in the to-be-calibrated dataset as effective data.

[0126] A reserved data determination module is configured to determine spectral values in the reference dataset that are the same as the effective data in terms of set wavelength and the same as the effective data in terms of RGB value as reserved data.

[0127] An RGB value form determination module is configured to determine the form of each RGB value according to the RGB value; the form comprises R=G=0, B=n, R=B=0, G=n, and B=G=0, R=n, n being a positive integer.

[0128] A reserved combination data determination module is configured to determine the reserved data corresponding to the RGB values of the same form as a group to obtain reserved combination data corresponding to each form of RGB value.

[0129] A to-be-calibrated combination data determination module is configured to determine the effective data corresponding to the RGB values of the same form as a group to obtain to-be-calibrated combination data corresponding to each form of RGB value.

[0130] The reserved data combination determination module is configured to select one spectral value from each form of RGB value corresponding reserved combination data to form a reserved data combination, thereby obtaining a plurality of reserved data combinations; the spectral values in one of the reserved data combinations correspond to the same set wavelength.

[0131] The to-be-calibrated data combination determination module is configured to select one spectral value from each form of RGB value corresponding to-be-calibrated combination data to form a to-be-calibrated data combination, thereby obtaining a plurality of to-be-calibrated data combinations; the spectral values in one of the to-be-calibrated data combinations correspond to the same set wavelength.

[0132] The reserved spectral value updating module is configured to, for any one reserved data combination, add the spectral values in the reserved data combination to obtain a spectral value corresponding to the reserved data combination.

[0133] The to-be-trained spectral value updating module is configured to, for any one to-be-calibrated data combination, add the spectral values in the to-be-calibrated data combination to obtain a spectral value corresponding to the to-be-calibrated data combination.

[0134] The training, verification and calibration module is configured to train and verify a neural network using a data set to obtain a calibration model, and calibrate a to-be-tested spectral value using the calibration model; the data set includes spectral values corresponding to all to-be-calibrated data combinations and spectral values corresponding to all reserved data combinations.

[0135] In actual application, the training, verification and calibration module specifically includes:

[0136] The verification set and training set determination unit is configured to divide the data set into a verification set and a training set according to a set proportion; the training set and the verification set each include a plurality of spectral data groups, and one spectral data group includes a reference data value and a to-be-calibrated data value with the same RGB value and the same set wavelength; the reference data value is a spectral value corresponding to a reference integrating sphere spectrometer; and the to-be-calibrated data value is a spectral value corresponding to a to-be-calibrated micro fiber spectrometer.

[0137] The training unit is configured to train the neural network using the training set to obtain a trained neural network.

[0138] The verification unit is configured to verify the trained neural network using the verification set to obtain a calibration model.

[0139] In actual application, the micro fiber spectrometer spectral calibration system further includes:

[0140] The confidence threshold determination module is configured to perform confidence verification on a calibration data value to determine whether the calibration data value reaches a confidence threshold; the calibration data value is obtained by calibrating a to-be-tested spectral value using the calibration model.

[0141] The training set updating module is configured to, if the calibration data value reaches the confidence threshold, take the to-be-tested spectrum value and the calibration data value as a new training set, and train and verify the calibration model by using the new training set and the verification set.

[0142] In actual applications, the confidence threshold determination module specifically includes:

[0143] The basic data acquisition unit is configured to acquire a target wavelength, a to-be-tested spectrum value, and a target predicted spectrum value; the target wavelength is a wavelength corresponding to the to-be-tested spectrum value, and the target predicted spectrum value is obtained by inputting the to-be-tested spectrum value into the calibration model.

[0144] The first selection unit is configured to select spectrum values corresponding to the target wavelength from the to-be-calibrated data set and the reference data set respectively to obtain a first to-be-calibrated data set and a first reference data set.

[0145] The sorting unit is configured to sort the spectrum values in the first to-be-calibrated data set and the first reference data set in ascending order respectively to obtain a to-be-calibrated data set sequence and a reference data set sequence.

[0146] The second selection unit is configured to select a first to-be-calibrated spectrum value and a second to-be-calibrated spectrum value from the to-be-calibrated data set; the first to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the to-be-calibrated data set sequence; and the second to-be-calibrated spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the to-be-calibrated data set sequence.

[0147] The third selection unit is configured to select a first reference spectrum value and a second reference spectrum value from the reference data set; the first reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and smaller than the to-be-tested spectrum value in the reference data set sequence; and the second reference spectrum value is a spectrum value adjacent to the to-be-tested spectrum value and greater than the to-be-tested spectrum value in the reference data set sequence.

[0148] The first determination unit is configured to determine whether a product of a first difference value and a second difference value is less than 0 to obtain a first determination result; the first difference value is a difference between the to-be-tested spectrum value and the target predicted spectrum value; and the second difference value is a difference between the first to-be-calibrated spectrum value and the first reference spectrum value.

[0149] The second judging unit is configured to, if the first judging result is no, judge whether the absolute value of the first difference value is greater than the absolute value of a second difference value and the absolute value of a third difference value, to obtain a second judging result, wherein the third difference value is a difference value between the second to-be-calibrated spectrum value and the second reference spectrum value.

[0150] The confidence threshold judging unit is configured to, if the second judging result is yes, determine that the calibration data value reaches a confidence threshold.

[0151] The embodiment of the present application also provides an electronic device, comprising:

[0152] The memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to execute the micro fiber spectrum calibrating method of the above-mentioned embodiment.

[0153] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the micro fiber spectrum calibrating method of the above-mentioned embodiment.

[0154] The present application has the following advantages:

[0155] The original data set is collected, data preprocessing is performed, the initial structure of the neural network is built, the calibration model is obtained by preliminary training of the teacher network model, i.e., the initial neural network, the spectrum radiation calibration work is performed, the confidence verification is performed on the calibration result, and the student neural network model is iteratively trained and the adaptive optimization model is obtained, i.e., the adaptive iterative optimization is continuously performed according to the real working condition data in actual application. The neural network model based on the deep learning algorithm can convert the to-be-calibrated spectrum distribution data into data close to the spectrum distribution reference value in real time. The neural network is trained according to a small amount of artificial annotation data set to solve the measurement inaccuracy problem of the micro fiber spectrum caused by stray light, thermal noise, pulse broadening and the like.

[0156] The measurement error of the micro fiber spectrum can be reduced, so that the measurement precision is improved while the advantages are retained, and the measurement precision is close to that of a large integral sphere spectrometer.

[0157] The real-time performance is high, and a large amount of data can be calibrated rapidly.

[0158] The data annotation demand is small, and the adaptive iterative optimization can be continuously performed according to the real working condition data in actual application.

[0159] The cost is low, and the use is simple. While the advantages of small size and low cost of the micro fiber spectrum are ensured, the data offset problem of the spectrum caused by various design defects is solved to a certain extent without changing the hardware structure of the spectrum measuring instrument.

[0160] The various embodiments described in this specification are presented for the purpose of illustrating the principles of the present application and its best mode of operation. Each of the embodiments described in this specification has been provided for the purpose of illustration only and the various embodiments are not intended to limit the present application in any way unless otherwise specifically indicated. The same parts and / or features of the various embodiments described in this specification can be referenced using the same reference numerals for the ease of understanding of the present application.

[0161] The principles and implementations of the present application have been described in the above embodiments, which are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation and application range of the present application can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A spectral calibration method for a miniature fiber optic spectrometer, characterized in that, include: Obtain the baseline dataset, the dataset to be calibrated, and the truncated dataset; The reference dataset includes: spectral datasets of the reference integrating sphere spectrometer at each set wavelength; the dataset to be calibrated includes: spectral datasets of the miniature fiber optic spectrometer to be calibrated at each set wavelength; the cut-out dataset includes spectral datasets of the reference integrating sphere spectrometer at each set wavelength under conditions without light; one set wavelength corresponds to one spectral dataset; one spectral dataset includes spectral values ​​under multiple different RGB values; A first threshold is determined based on the extracted dataset; the first threshold is twice the largest spectral value in the extracted dataset. Spectral values ​​in the dataset to be calibrated that are greater than the first threshold are determined as valid data; The spectral values ​​in the benchmark dataset that have the same set wavelength as the valid data and the same RGB value as the valid data are determined as retained data; The form of each RGB value is determined based on the RGB values; the forms include R=G=0,B=n, R=B=0,G=n and B=G=0,R=n, where n is a positive integer; By identifying the retained data corresponding to RGB values ​​of the same form as a group, we obtain the retained combination data corresponding to RGB values ​​of various forms; The valid data corresponding to RGB values ​​of the same form are identified as a group to obtain the calibration combination data corresponding to RGB values ​​of each form; In each of the reserved data combinations corresponding to RGB values ​​of various forms, a spectral value is selected to form a reserved data combination, resulting in multiple reserved data combinations; the spectral values ​​in a single reserved data combination correspond to the same set wavelength. In each of the RGB value combinations to be calibrated, a spectral value is selected to form a data combination to be calibrated, resulting in multiple data combinations to be calibrated; the spectral values ​​in a data combination to be calibrated correspond to the same set wavelength. For any given set of retained data, the spectral values ​​of each data set are added together to obtain the spectral value corresponding to the data set. For any combination of data to be calibrated, the spectral values ​​of each spectral value in the combination of data to be calibrated are added together to obtain the spectral value corresponding to the combination of data to be calibrated. A calibration model is obtained by training and validating the neural network using a dataset, and the calibration model is then used to calibrate the spectral values ​​to be tested; the dataset includes spectral values ​​corresponding to all combinations of data to be calibrated and spectral values ​​corresponding to all combinations of retained data. The neural network is trained and validated using a dataset to obtain a calibration model, which is then used to calibrate the spectral values ​​to be tested. The process also includes: The confidence verification of the calibration data values ​​is performed to determine whether the calibration data values ​​have reached a confidence threshold, specifically including: The target wavelength, the spectral value to be tested, and the target predicted spectral value are obtained; the target wavelength is the wavelength corresponding to the spectral value to be tested, and the target predicted spectral value is obtained by inputting the spectral value to be tested into the calibration model. The first dataset to be calibrated and the first reference dataset are obtained by selecting spectral values ​​corresponding to the target wavelength from the dataset to be calibrated and the reference dataset, respectively. The spectral values ​​in the first dataset to be calibrated and the first reference dataset are sorted in ascending order to obtain the dataset to be calibrated sequence and the reference dataset sequence, respectively. In the dataset to be calibrated, a first spectral value to be calibrated and a second spectral value to be calibrated are selected; the first spectral value to be calibrated is the spectral value in the dataset sequence that is adjacent to the spectral value to be tested and is smaller than the spectral value to be tested; the second spectral value to be calibrated is the spectral value in the dataset sequence that is adjacent to the spectral value to be tested and is larger than the spectral value to be tested. In the benchmark dataset, a first benchmark spectral value and a second benchmark spectral value are selected; the first benchmark spectral value is the spectral value in the benchmark dataset sequence that is adjacent to the spectral value to be tested and is smaller than the spectral value to be tested; the second benchmark spectral value is the spectral value in the benchmark dataset sequence that is adjacent to the spectral value to be tested and is larger than the spectral value to be tested. Determine whether the product of the first difference and the second difference is less than 0 to obtain a first determination result; the first difference is the difference between the spectral value to be tested and the target predicted spectral value; the second difference is the difference between the first spectral value to be calibrated and the first reference spectral value; If the first judgment result is negative, then it is determined whether the absolute value of the first difference is greater than the absolute value of the second difference and the absolute value of the third difference to obtain the second judgment result; the third difference is the difference between the second spectral value to be calibrated and the second reference spectral value. If the second judgment result is yes, then it is determined that the calibration data value has reached the confidence threshold; the calibration data value is obtained by calibrating the test spectral value using the calibration model. If the calibration data value reaches the confidence threshold, the test spectral value and the calibration data value are used as a new training set, and the calibration model is trained and validated using the new training set and validation set.

2. The spectral calibration method for a miniature fiber optic spectrometer according to claim 1, characterized in that, The process of training and validating the neural network using a dataset to obtain a calibration model specifically includes: The dataset is divided into a validation set and a training set according to a set ratio. Both the training set and the validation set include multiple sets of spectral data. Each set of spectral data includes reference data values ​​with the same wavelength and the same RGB values ​​and data values ​​to be calibrated. The reference data values ​​are the spectral values ​​corresponding to the reference integrating sphere spectrometer. The data values ​​to be calibrated are the spectral values ​​corresponding to the miniature fiber optic spectrometer to be calibrated. The neural network is trained using the training set to obtain the trained neural network; The trained neural network is validated using the validation set to obtain a calibration model.

3. A spectral calibration system for a miniature fiber optic spectrometer, characterized in that, include: The acquisition module is used to acquire the benchmark dataset, the dataset to be calibrated, and the truncated dataset; The reference dataset includes: spectral datasets of the reference integrating sphere spectrometer at each set wavelength; the dataset to be calibrated includes: spectral datasets of the miniature fiber optic spectrometer to be calibrated at each set wavelength; the cut-out dataset includes spectral datasets of the reference integrating sphere spectrometer at each set wavelength under conditions without light; one set wavelength corresponds to one spectral dataset; one spectral dataset includes spectral values ​​under multiple different RGB values; The first threshold determination module is used to determine a first threshold based on the extracted dataset; the first threshold is twice the largest spectral value in the extracted dataset. The valid data determination module is used to determine the spectral values ​​in the dataset to be calibrated that are greater than the first threshold as valid data; The data retention determination module is used to determine the spectral values ​​in the benchmark dataset that have the same set wavelength as the valid data and the same RGB value as the valid data as the data to be retained. The RGB value format determination module is used to determine the format of each RGB value based on the RGB value; the format includes R=G=0,B=n, R=B=0,G=n and B=G=0,R=n, where n is a positive integer; The module for determining retained combination data is used to determine the retained data corresponding to RGB values ​​of the same form as a group to obtain the retained combination data corresponding to RGB values ​​of various forms. The module for determining the combined data to be calibrated is used to determine the valid data corresponding to RGB values ​​of the same form as a group to obtain the combined data to be calibrated corresponding to RGB values ​​of various forms. The data combination determination module is used to select a spectral value from the data combinations corresponding to each form of RGB value to form a data combination, thereby obtaining multiple data combinations; the spectral values ​​in a data combination have the same set wavelength. The data combination determination module is used to select a spectral value from the data combinations to be calibrated corresponding to each form of RGB value to form a data combination to be calibrated, thereby obtaining multiple data combinations to be calibrated; the spectral values ​​in a data combination to be calibrated correspond to the same set wavelength. The spectral value update module is used to add up the spectral values ​​in any given spectral data combination to obtain the spectral value corresponding to the spectral data combination. The training spectral value update module is used to add up the spectral values ​​in any combination of data to be calibrated to obtain the spectral value corresponding to the combination of data to be calibrated. The training, validation, and calibration module is used to train and validate the neural network using a dataset to obtain a calibration model, and to calibrate the spectral values ​​to be tested using the calibration model; the dataset includes spectral values ​​corresponding to all combinations of data to be calibrated and spectral values ​​corresponding to all combinations of retained data. The confidence threshold judgment module is used to perform confidence verification on the calibration data value to determine whether the calibration data value reaches the confidence threshold; the calibration data value is obtained by calibrating the calibration model to the test spectral value; The training set update module is used to, if the calibration data value reaches the confidence threshold, use the test spectral value and the calibration data value as a new training set, and use the new training set and validation set to train and validate the calibration model. The confidence threshold determination module specifically includes: The basic data acquisition unit is used to acquire the target wavelength, the spectral value to be tested, and the target predicted spectral value; the target wavelength is the wavelength corresponding to the spectral value to be tested, and the target predicted spectral value is obtained by inputting the spectral value to be tested into the calibration model. The first selection unit is used to select the spectral values ​​corresponding to the target wavelength from the dataset to be calibrated and the reference dataset respectively to obtain the first dataset to be calibrated and the first reference dataset. The sorting unit is used to sort the spectral values ​​in the first dataset to be calibrated and the first reference dataset in ascending order, respectively, to obtain the dataset to be calibrated sequence and the reference dataset sequence. The second selection unit is used to select a first spectral value to be calibrated and a second spectral value to be calibrated in the dataset to be calibrated; the first spectral value to be calibrated is a spectral value in the dataset sequence that is adjacent to the spectral value to be tested and is smaller than the spectral value to be tested; the second spectral value to be calibrated is a spectral value in the dataset sequence that is adjacent to the spectral value to be tested and is larger than the spectral value to be tested. The third selection unit is used to select a first reference spectral value and a second reference spectral value from the reference dataset; the first reference spectral value is a spectral value in the reference dataset sequence that is adjacent to the spectral value to be tested and is smaller than the spectral value to be tested; the second reference spectral value is a spectral value in the reference dataset sequence that is adjacent to the spectral value to be tested and is larger than the spectral value to be tested. The first judgment unit is used to determine whether the product of the first difference and the second difference is less than 0, and to obtain a first judgment result; the first difference is the difference between the spectral value to be tested and the target predicted spectral value; the second difference is the difference between the first spectral value to be calibrated and the first reference spectral value; The second judgment unit is used to determine whether the absolute value of the first difference is greater than the absolute value of the second difference and the absolute value of the third difference if the first judgment result is negative, and to obtain the second judgment result; the third difference is the difference between the second spectral value to be calibrated and the second reference spectral value. A confidence threshold determination unit is used to determine that the calibration data value has reached a confidence threshold if the second determination result is yes.

4. The spectral calibration system for a miniature fiber optic spectrometer according to claim 3, characterized in that, The training, validation, and calibration module specifically includes: The validation set and training set determination unit is used to divide the dataset into a validation set and a training set according to a set ratio. Both the training set and the validation set include multiple sets of spectral data. Each set of spectral data includes reference data values ​​with the same wavelength and the same RGB values ​​and data values ​​to be calibrated. The reference data values ​​are the spectral values ​​corresponding to the reference integrating sphere spectrometer. The data values ​​to be calibrated are the spectral values ​​corresponding to the miniature fiber optic spectrometer to be calibrated. A training unit is used to train the neural network using the training set to obtain a trained neural network. A verification unit is used to verify the trained neural network using the verification set to obtain a calibration model.

5. An electronic device, characterized in that, include: A memory and a processor, the memory for storing a computer program, the processor for running the computer program to cause the electronic device to perform the spectral calibration method for a miniature fiber optic spectrometer according to any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the spectral calibration method for a miniature fiber optic spectrometer as described in any one of claims 1 to 2.

Images