Online detection method and device for dairy product quality, electronic equipment and storage medium

Abstract

The invention provides an online detection method and device for dairy product quality, electronic equipment and a storage medium, and the method comprises the following steps: in a dairy product production process, respectively obtaining a plurality of first dairy product dynamic spectrums containing moisture disturbance at different moments and a plurality of second dairy product dynamic spectrums containing temperature disturbance at different moments; obtaining a first dairy product spectral feature image without water molecule disturbance based on the plurality of first dairy product dynamic spectrums, and obtaining a second dairy product spectral feature image without temperature disturbance based on the plurality of second dairy product dynamic spectrums; and inputting the first dairy product spectral feature image and the second dairy product spectral feature image into a pre-trained neural network model, so that a dairy product quality detection result output by the neural network model can be obtained online, the working efficiency is improved, and the detection cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of dairy product analysis, and in particular to an online detection method, device, electronic equipment and storage medium for dairy product quality. Background Art

[0002] The production process of dairy products requires strict control of the quality of raw materials and products. For example, the concentration or quality of whey protein, lactose and fat in dairy products has a great impact on the taste, texture and nutritional value of the product.

[0003] It is known from relevant technologies that in order to ensure the quality and taste of dairy products, the concentration changes of these main ingredients need to be accurately monitored and controlled during the production process, so that the product quality of dairy products can be detected.

[0004] Currently, quality testing of dairy products usually requires sampling and laboratory analysis, which is time-consuming, labor-intensive and has high testing costs. Summary of the invention

[0005] The present invention provides an online detection method, device, electronic equipment and storage medium for dairy product quality, which can realize online detection of dairy product quality, thereby improving work efficiency and reducing detection costs.

[0006] The present invention provides an online detection method for dairy product quality, the method comprising: in a dairy product production process, respectively acquiring a plurality of first dairy product dynamic spectra containing water disturbance at different times, and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times; based on the plurality of first dairy product dynamic spectra, obtaining a first dairy product spectrum characteristic image with water molecule disturbance removed, and based on the plurality of second dairy product dynamic spectra, obtaining a second dairy product spectrum characteristic image with temperature disturbance removed; inputting the first dairy product spectrum characteristic image and the second dairy product spectrum characteristic image into a pre-trained neural network model, and obtaining online a product quality detection result of the dairy product output by the neural network model.

[0007] According to an online detection method for dairy product quality provided by the present invention, the first dairy product spectral characteristic image with water molecule disturbance removed is obtained based on multiple first dairy product dynamic spectra, specifically comprising: based on the multiple first dairy product dynamic spectra, determining the characteristic peak change information of the water molecules through a two-dimensional correlation analysis algorithm; based on the multiple first dairy product dynamic spectra and the characteristic peak change information of the water molecules, obtaining the first dairy product spectral characteristic image with water molecule disturbance removed.

[0008] According to an online detection method for dairy product quality provided by the present invention, the method obtains a second dairy product spectral characteristic image with temperature disturbance removed based on multiple second dairy product dynamic spectra, specifically comprising: performing second-order derivative spectrum operations on the multiple second dairy product dynamic spectra to obtain second-order derivative spectra of the second dairy product dynamic spectra in different bands; and obtaining the second dairy product spectral characteristic image with temperature disturbance removed by a two-dimensional correlation analysis algorithm based on the second-order derivative spectra of the second dairy product dynamic spectra in different bands.

[0009] According to an online detection method for dairy product quality provided by the present invention, the neural network model includes a backbone feature extraction network, a feature pyramid network and a detector; the neural network model outputs the product quality detection result of the dairy product in the following manner: based on the backbone feature extraction network, feature extraction is performed on the first dairy product spectral feature image and the second dairy product spectral feature image to obtain a first feature extraction result corresponding to the first dairy product spectral feature image and a second feature extraction result corresponding to the second dairy product spectral feature image, respectively; based on the feature pyramid network, feature fusion is performed on the first feature extraction result and the second feature extraction result to obtain a fused result; based on the detector, calculation is performed in combination with the fused result to obtain the product quality detection result of the dairy product.

[0010] According to an online detection method for dairy product quality provided by the present invention, before the feature pyramid network, the neural network model also includes a coordinate attention module, which is used to capture the first feature extraction results and / or the second feature extraction results of different sizes based on the adaptive receptive field size of the coordinate attention module.

[0011] According to an online detection method for dairy product quality provided by the present invention, the neural network model also includes a sliding window deformer self-attention mechanism module, which is used to capture the first dairy product spectral feature image and / or the second dairy product spectral feature image with different magnitude changes based on the sliding window deformer self-attention mechanism module.

[0012] According to an online detection method for dairy product quality provided by the present invention, the first dairy product dynamic spectrum containing moisture disturbance is acquired in the following manner: when a change in moisture in the dairy product is detected, the first dairy product dynamic spectrum containing moisture disturbance is collected based on an infrared spectrum probe immersed in the dairy product; the second dairy product dynamic spectrum containing temperature disturbance is acquired in the following manner: when a change in temperature of the dairy product is detected, the second dairy product dynamic spectrum containing temperature disturbance is collected based on an infrared spectrum probe immersed in the dairy product.

[0013] The present invention also provides an online detection device for dairy product quality, the device comprising: an acquisition module, used to acquire multiple first dairy product dynamic spectra containing water disturbance at different times and multiple second dairy product dynamic spectra containing temperature disturbance at different times during the dairy product production process; a processing module, used to obtain a first dairy product spectrum characteristic image with water molecule disturbance removed based on the multiple first dairy product dynamic spectra, and to obtain a second dairy product spectrum characteristic image with temperature disturbance removed based on the multiple second dairy product dynamic spectra; a generation module, used to input the first dairy product spectrum characteristic image and the second dairy product spectrum characteristic image into a pre-trained neural network model, and obtain the product quality detection result of the dairy product output by the neural network model online.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the online detection method for the quality of dairy products as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the online detection method for the quality of dairy products as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the online detection method for the quality of dairy products as described above is implemented.

[0017] The online detection method, device, electronic device and storage medium of dairy product quality provided by the present invention respectively obtain a plurality of first dairy product dynamic spectra containing water disturbance at different times and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times during the dairy product production process; and obtain a first dairy product spectrum characteristic image with water molecule disturbance removed based on the plurality of first dairy product dynamic spectra, and obtain a second dairy product spectrum characteristic image with temperature disturbance removed based on the plurality of second dairy product dynamic spectra; and then input the first dairy product spectrum characteristic image and the second dairy product spectrum characteristic image into a pre-trained neural network model, so that the product quality detection result of the dairy product output by the neural network model can be obtained online, thereby improving work efficiency and reducing detection costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of the process of the online detection method of dairy product quality provided by the present invention;

[0020] Figure 2 It is a schematic diagram of a process of obtaining a first dairy product spectral characteristic image with water molecule disturbance removed based on a plurality of first dairy product dynamic spectra provided by the present invention;

[0021] Figure 3 It is a schematic diagram of a process of obtaining a second dairy product spectral characteristic image with temperature disturbance removed based on a plurality of second dairy product dynamic spectra provided by the present invention;

[0022] Figure 4 It is a schematic diagram of a process of outputting the product quality test results of dairy products by the neural network model provided by the present invention;

[0023] Figure 5 This is a schematic diagram of an application scenario of the online dynamic spectrum measurement device for dairy products provided by the present invention;

[0024] Figure 6 It is a schematic structural diagram of an online detection device for dairy product quality provided by the present invention;

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention.

[0026] Description of reference numerals:

[0027] 1: Light source; 2: Fiber optic probe for near-infrared spectrum measurement;

[0028] 3: Mid-infrared spectrum measurement fiber probe; 4: Temperature sensor;

[0029] 5: moisture sensor; 6: near infrared spectrometer;

[0030] 7: Mid-infrared spectrometer; 8: Data acquisition module;

[0031] 9: Processor; 10: Dairy production tank. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Figure 1 It is a schematic diagram of the process of the online detection method of dairy product quality provided by the present invention.

[0034] The following will be combined Figure 1 The process of the online detection method for dairy product quality provided by the present invention is described.

[0035] In an exemplary embodiment of the present invention, Figure 1 It can be seen that the online detection method for dairy product quality may include steps 110 to 130, and each step will be introduced below.

[0036] In step 110, during the dairy product production process, a plurality of first dairy product dynamic spectra containing water disturbance at different times and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times are acquired respectively.

[0037] In one embodiment, during the production of dairy products, multiple first dairy product dynamic spectra containing moisture disturbance at different times and multiple second dairy product dynamic spectra containing temperature disturbance at different times may be acquired in different process scenarios.

[0038] The multiple first dairy product dynamic spectra containing water disturbance at different times may be two first dairy product dynamic spectra; the multiple second dairy product dynamic spectra containing temperature disturbance at different times may be two second dairy product dynamic spectra.

[0039] In another exemplary embodiment of the present invention, the first dairy product dynamic spectrum containing water disturbance can be obtained in the following manner:

[0040] When a change in the water content in the dairy product is detected, a first dairy product dynamic spectrum including the water content disturbance is collected based on an infrared spectrum probe immersed in the dairy product;

[0041] The second dairy product dynamic spectrum including temperature disturbance can be obtained in the following way:

[0042] When a temperature change of the dairy product is detected, a second dairy product dynamic spectrum including temperature disturbance is collected based on the infrared spectrum probe immersed in the dairy product.

[0043] In one embodiment, a first dairy product dynamic spectrum including water disturbance and a second dairy product dynamic spectrum including temperature disturbance can be obtained based on an online dynamic spectrum measuring device for dairy products.

[0044] In one example, combining Figure 5 It can be seen that the online dynamic spectrum measurement device for dairy products may include a light source 1, an infrared spectrum probe, a temperature sensor 4, a moisture sensor 5, an infrared spectrometer and a data acquisition module 8. Among them, the infrared spectrum probe may include a near-infrared spectrum measurement optical fiber probe 2 and a mid-infrared spectrum measurement optical fiber probe 3. The infrared spectrometer may include a near-infrared spectrometer 6 and a mid-infrared spectrometer 7.

[0045] The light source 1 is a halogen lamp light source with continuous spectrum emission characteristics, and the spectral range covers the near-infrared and mid-infrared regions.

[0046] The near-infrared spectrum measurement optical fiber probe 2 collects the transmitted light absorbed and scattered by the sample after the light source 1 is incident on the sample (corresponding to the dairy product), and transmits the optical signal to the near-infrared spectrometer 6 for spectral measurement to obtain the first dairy product dynamic spectrum containing water disturbance and\or the second dairy product dynamic spectrum containing temperature disturbance. The near-infrared spectrometer 6 is used to measure the near-infrared band spectrum of the sample, and the measurement range is 900-1700nm.

[0047] Based on the same principle, the mid-infrared spectrum measurement optical fiber probe 3 collects the transmitted light absorbed and scattered by the sample after the light source 1 is incident on the sample, and transmits the optical signal to the mid-infrared spectrometer 7 for spectral measurement to obtain the first dairy dynamic spectrum containing moisture disturbance and\or the second dairy dynamic spectrum containing temperature disturbance. The mid-infrared spectrometer 7 is used to measure the mid-infrared band spectrum of the sample, and the measurement range is 2500-10000nm. Among them, when to use the near-infrared spectrum measurement optical fiber probe 2 or the mid-infrared spectrum measurement optical fiber probe 3 is adjusted according to the actual situation.

[0048] The temperature sensor 4 is used to measure the real-time temperature of the sample (corresponding to the dairy product) and quantify the temperature disturbance of the sample in the dynamic spectrum.

[0049] The moisture sensor 5 is used to measure the moisture content of the sample (corresponding to the dairy product) and quantify the moisture disturbance of the sample in the dynamic spectrum.

[0050] The data acquisition module 8 is used to transmit the temperature and moisture sensor data to the processor 9, wherein the processor 9 may be a computer.

[0051] The dairy product production tank 10 can be used for dairy product storage and production.

[0052] In the application process, the online dynamic spectrum measurement device for dairy products is used to obtain mid-infrared and near-infrared spectra of temperature, moisture, and other conditions in different production links (corresponding to the second dairy dynamic spectrum containing temperature disturbances, and the first dairy dynamic spectrum containing moisture disturbances, respectively). According to the results of the temperature and moisture sensors, the temperature and moisture changes corresponding to the collected spectrum can be determined. Further, according to the absorption spectrum line characteristics and intensity change characteristics caused by different conditions, based on the spectrum preprocessing, decomposition and feature extraction methods, the decomposed spectrum is subjected to two-dimensional correlation processing to obtain the disturbance dynamic spectrum characteristics (corresponding to the first dairy spectrum characteristic image without water molecule disturbances, and the second dairy spectrum characteristic image without temperature disturbances), and the parameter variables related to the quality in the dairy production process are extracted to serve as the basis for quality judgment and analysis.

[0053] In step 120, a first dairy product spectrum characteristic image with water molecule disturbance removed is obtained based on a plurality of first dairy product dynamic spectra, and a second dairy product spectrum characteristic image with temperature disturbance removed is obtained based on a plurality of second dairy product dynamic spectra.

[0054] In one embodiment, a first dairy product spectral characteristic image with water molecule disturbance removed can be obtained based on multiple, for example two, first dairy product dynamic spectra, and a second dairy product spectral characteristic image with temperature disturbance removed can be obtained based on multiple, for example two, second dairy product dynamic spectra, thereby laying a foundation for product quality detection results of dairy products based on online measurement of dynamic spectra.

[0055] In step 130, the first dairy product spectral characteristic image and the second dairy product spectral characteristic image are input into a pre-trained neural network model, and the product quality detection result of the dairy product output by the neural network model is obtained online.

[0056] In another embodiment, after extracting the dynamic spectral features of moisture disturbance and temperature disturbance in the dairy production process, a convolutional neural network can be used to establish a quality discrimination model based on the dynamic disturbance spectrum and the two-dimensional correlation spectrum of the heterogeneous spectrum. In order to highlight the potential asynchronous spectral behavior of different component changes reflected by different types of spectra, the asynchronous cross peaks of the heterogeneous spectrum are used to identify highly overlapping absorption features and separate the spectral features of different component changes. According to the intensity changes of the characteristic peaks in the dual-spectrum asynchronous spectrum, the two-dimensional correlation spectrum matrix is ​​converted into a two-dimensional image (corresponding to the first dairy product spectral feature image and = the second dairy product spectral feature image) as the input based on the convolutional neural network, so that the product quality detection results of the dairy product output by the neural network model can be accurately and quickly obtained online.

[0057] The online detection method for dairy product quality provided by the present invention obtains a plurality of first dairy product dynamic spectra containing water disturbance at different times and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times during the dairy product production process; and based on the plurality of first dairy product dynamic spectra, obtains a first dairy product spectrum characteristic image with water molecule disturbance removed, and based on the plurality of second dairy product dynamic spectra, obtains a second dairy product spectrum characteristic image with temperature disturbance removed; and then inputs the first dairy product spectrum characteristic image and the second dairy product spectrum characteristic image into a pre-trained neural network model, so that the product quality detection result of the dairy product output by the neural network model can be obtained online, thereby improving work efficiency and reducing detection costs.

[0058] Figure 2 It is a schematic diagram of a process for obtaining a first dairy product spectral characteristic image with water molecule disturbance removed based on a plurality of first dairy product dynamic spectra provided by the present invention.

[0059] In an exemplary embodiment of the present invention, Figure 2 It can be seen that obtaining the first dairy product spectrum characteristic image without water molecule disturbance based on multiple first dairy product dynamic spectra may include step 210 and step 220, and each step will be described below.

[0060] In step 210, based on a plurality of first dairy product dynamic spectra, characteristic peak change information of water molecules is determined by a two-dimensional correlation analysis algorithm.

[0061] In step 220, based on a plurality of first dairy product dynamic spectra and characteristic peak change information of water molecules, a first dairy product spectrum characteristic image with water molecule disturbance removed is obtained.

[0062] In the application process, the characteristic peak change information of water molecules can be determined based on multiple dynamic spectra of the first dairy product at different times through a two-dimensional correlation analysis algorithm. Furthermore, based on multiple dynamic spectra of the first dairy product and the characteristic peak change information of water molecules, the characteristic spectrum image of the first dairy product without water molecule disturbance can be obtained. In this way, the spectrum image of the dairy product without water molecule disturbance can be obtained more accurately, laying a foundation for accurately and quickly obtaining the product quality test results of dairy products online.

[0063] It should be noted that water is the main component of dairy products, and the water content changes greatly during the production process of dairy products, which is also an important factor affecting the quality of dairy products. Water molecules in dairy products have characteristic absorption peaks at certain wavelengths and will form hydrogen bonds with molecules such as proteins and fats. Using water molecules in dairy products as perturbation factors, the dynamic spectral changes of dairy products during the production process are measured. Through dual-spectrum two-dimensional (2T2D) correlation analysis, the absorption peak changes of water molecules themselves and the spectral characteristics of hydrogen bond changes formed by proteins and fat molecules with different water molecule contents are obtained. Then, the slice spectrum intensity integration within the wavelength range of water molecule hydrogen bond changes is used to obtain the first dairy product spectral characteristic image after removing the water molecule disturbance.

[0064] In one example, an external disturbance t can be applied to the sample to be tested (corresponding to dairy products). The external disturbance can be any form of physical or chemical quantity. In this embodiment, the external disturbance t can be a change in water molecules. min ,T max ] changes within a certain range, collect the spectrum that changes with the disturbance of variable t (corresponding to the first dairy product dynamic spectrum) y(v,t), where v is the wave number of the measured absorption spectrum. According to the characteristics of hydrogen bonds formed by water molecules and main components such as protein and fat in dairy products, the dynamic spectrum is decomposed using the wavelet transform method to obtain a decomposed spectrum that can characterize the absorption characteristics of different molecules (which can correspond to the first dairy product spectrum characteristic image without water molecule disturbance), as shown in formulas (1)-(2):

[0065]

[0066]

[0067] Where M is the number of spectral wavelengths, f(n) is the spectral decomposition coefficient, Ω j,v (n) is the detail wavelet basis function, Θ j,v (n) is the approximate wavelet basis function. For each decomposed spectrum, the dynamic spectrum induced by the disturbance factor is defined as formula (3):

[0068]

[0069] Among them, It is the reference spectrum, which is used to centralize multiple dynamic spectra of each sample. Its definition is not strictly regulated and unified. It can generally be replaced by the average spectrum, as shown in formula (4):

[0070]

[0071] The synchronization correlation intensity Φ(v1,v2) is the vector product of the dynamic spectra at frequencies v1 and v2. According to the above definition of the dynamic spectrum, the synchronization spectrum can be expressed as follows:

[0072]

[0073] The asynchronous correlation intensity Ψ(v1,v2) is the vector product of the Hilbert-Noda matrices of the dynamic spectral intensities at frequencies v1 and v2, which can be expressed as follows:

[0074]

[0075] Where N is an m-order square matrix, called an m-matrix, which is used to calculate the two-dimensional correlated asynchronous spectrum. Its elements are given by formula (7):

[0076]

[0077] Figure 3 It is a schematic diagram of a process for obtaining a second dairy product spectral characteristic image with temperature disturbance removed based on a plurality of second dairy product dynamic spectra provided by the present invention.

[0078] The following will be combined Figure 3 The process of obtaining the second dairy product spectral characteristic image with temperature disturbance removed based on a plurality of second dairy product dynamic spectra is described.

[0079] In an exemplary embodiment of the present invention, Figure 3 It can be seen that obtaining the second dairy product spectrum characteristic image with temperature disturbance removed based on multiple second dairy product dynamic spectra may include step 310 and step 320, and each step will be described below.

[0080] In step 310, a second-order derivative spectrum operation is performed on a plurality of second dairy product dynamic spectra to obtain second-order derivative spectra of the second dairy product dynamic spectra in different bands;

[0081] In step 320, based on the second-order derivative spectra of the second dairy product dynamic spectrum in different bands, a two-dimensional correlation analysis algorithm is used to obtain a second dairy product spectrum characteristic image with temperature disturbance removed.

[0082] In one embodiment, a second-order derivative spectrum operation can be performed based on multiple second dairy product dynamic spectra at different times to obtain second-order derivative spectra of the second dairy product dynamic spectrum in different bands; and then based on the second-order derivative spectra of the second dairy product dynamic spectrum in different bands, a two-dimensional correlation analysis algorithm is used to obtain a second dairy product spectrum characteristic image with temperature disturbance removed. In this way, the spectrum image of the dairy product with temperature disturbance removed can be obtained more accurately, laying a foundation for accurately and quickly obtaining the product quality detection results of the dairy product online.

[0083] It should also be noted that the perturbation under the action of temperature will have a certain impact on the vibration frequency and amplitude of molecular bonds, thereby changing the morphology and intensity of the absorption spectrum. The main components related to quality in dairy products, such as -OH, -CH, -NH and other molecular bonds contained in components such as protein and fat, have obvious changes in spectral characteristics under different temperature perturbations. Since hydrogen bonds weaken with increasing temperature, water clusters dissociate and break, that is, the free water content increases with increasing temperature, while hydrogen-bonded water decreases with increasing temperature. Since the combined frequency and overtone absorption lines caused by different molecular bond vibration modes are prone to overlap, in the invention, the spectral characteristics of different bands are measured separately, and the molecular absorption fundamental frequency band and overtone band and Xining heterogeneous spectrum are analyzed by two-dimensional correlation spectroscopy to obtain different types of spectral correlation asynchronous spectral characteristics, and extract the spectral characteristics of molecular bond changes in components such as protein and fat.

[0084] Since the near-infrared spectrum is mostly molecular frequency combination and frequency doubling absorption, the spectral peaks overlap seriously, and small absorption peaks or overlapping spectral peaks cannot be directly distinguished from the original spectrum. The spectral characteristics of the mid-infrared spectrum are easily affected by light source drift and changes in sample scattering characteristics, resulting in baseline drift. In order to highlight the molecular bond absorption characteristics contained in components such as protein and fat under temperature disturbance, the original spectrum (corresponding to the second dairy product dynamic spectrum) is subjected to second-order derivative spectrum calculation, as shown in formula (8):

[0085]

[0086] Where λ1 and λ2 are the central wavelengths respectively; Δλ is the step size for calculating the second-order derivative spectrum;

[0087] The second-order derivative spectra of different bands are used to perform heterogeneous two-dimensional correlation spectroscopy, which can be achieved using the following formula (9):

[0088]

[0089] in, and represents two different types of dynamic spectra; t represents the external disturbance variable (here it can refer to temperature); m represents the number of groups of external disturbance variables; the peak value of Φ(v1, v2) in the synchronization spectrum indicates the correlation strength between the bands belonging to the same secondary structure elements in the two spectra.

[0090] Figure 4 It is a schematic diagram of a process of outputting product quality test results of dairy products by the neural network model provided by the present invention.

[0091] The following will be combined Figure 4 The process of the neural network model outputting the product quality test results of dairy products is explained.

[0092] In an exemplary embodiment of the present invention, the neural network model may include a backbone feature extraction network, a feature pyramid network, and a detector. Figure 4 It can be seen that the neural network model outputting the product quality detection results of dairy products may include steps 410 to 430, and each step will be introduced below.

[0093] In step 410, feature extraction is performed on the first dairy product spectral feature image and the second dairy product spectral feature image based on the backbone feature extraction network to obtain a first feature extraction result corresponding to the first dairy product spectral feature image and a second feature extraction result corresponding to the second dairy product spectral feature image.

[0094] In step 420, feature fusion is performed on the first feature extraction result and the second feature extraction result based on the feature pyramid network to obtain a fused result.

[0095] In step 430, calculations are performed based on the detector and the fused results to obtain a product quality test result of the dairy product.

[0096] It should be noted that after extracting the dynamic spectral characteristics of water molecule disturbance and temperature disturbance in the dairy production process, a quality discrimination model based on the dynamic disturbance spectrum and the two-dimensional correlation spectrum of the heterogeneous spectrum can be arbitrarily established using a convolutional neural network. In order to highlight the potential asynchronous spectral behavior of different component changes reflected by different types of spectra, the asynchronous cross peaks of the heterogeneous spectrum can be used to identify highly overlapping absorption features and separate the spectral characteristics of different component changes. According to the intensity changes of the characteristic peaks in the dual-spectrum asynchronous spectrum, the two-dimensional correlation spectrum matrix is ​​converted into a two-dimensional image (corresponding to the first dairy product spectrum characteristic image and the second dairy product spectrum characteristic image) as an input based on a convolutional neural network (corresponding to the neural network model in the present invention), thereby improving the quality discrimination and analysis accuracy.

[0097] In one embodiment, the deep convolutional neural network structure is mainly divided into three parts: a trunk, a neck, and a detector, which can correspond to a trunk feature extraction network, a feature pyramid network, and a detector, respectively.

[0098] In the application process, feature extraction can be performed on the first dairy product spectral feature image and the second dairy product spectral feature image based on the backbone feature extraction network, and a first feature extraction result corresponding to the first dairy product spectral feature image and a second feature extraction result corresponding to the second dairy product spectral feature image can be obtained respectively. The backbone feature extraction network can be the backbone structure of CSPDarknet, and the backbone feature extraction network is not specifically limited in the present invention.

[0099] In another embodiment, the first feature extraction result and the second feature extraction result may be fused based on the feature pyramid network to obtain a fused result; and based on the detector, calculations may be performed in combination with the fused result to obtain a product quality detection result of the dairy product.

[0100] In one example, the feature pyramid network can be a feature pyramid structure of FPN (Feature Pyramid Network) + PAN (Path Aggregation Network). In the application process, feature fusion can be performed based on the feature pyramid structure of FPN (Feature Pyramid Network) + PAN (Path Aggregation Network). Among them, the detector can use CIOU_loss as the loss function and calculate in combination with the fusion result to obtain the product quality detection result of dairy products.

[0101] In another exemplary embodiment of the present invention, continuing with the above Figure 4 The embodiment is described as an example. Before the feature pyramid network, the neural network model may further include a coordinate attention module, which is used to capture the first feature extraction results and / or the second feature extraction results of different sizes based on the adaptive receptive field size of the coordinate attention module.

[0102] In one embodiment, in view of the large differences in dynamic spectral features and the complex molecular absorption spectral features, a coordinate attention module with adaptive receptive field size can be added to the neck of the neural network model to improve the neural network model's ability to extract features of targets of different scales and enhance the network's ability to locate targets in complex backgrounds.

[0103] In the coordinate attention module, the information of the convolution outputs of different sizes can be added to obtain the fused element U, and then the spatial channel pooling kernels (H, 1) and (1, W) are used to pool the U element in the height and width channel directions, respectively, to obtain features with dimensions of C×1×W and C×H×1. For the features after spatial channel pooling, the output of the cth channel at height h can be expressed as formula (10):

[0104]

[0105] Among them, x c (h,i) represents the value of the input variable at height h and point i; W represents the calculation width;

[0106] The output of the cth channel at width w can be expressed as formula (11):

[0107]

[0108] Among them, x c (j,w) represents the value of the input variable at width w and point j; H represents the height.

[0109] It should be noted that the input variable may refer to the first feature extraction results and / or the second feature extraction results of different sizes as described above.

[0110] After the above transformation is completed, the two-directional features can be concatenated, followed by convolution transformation and activation using a nonlinear activation function, as shown in formula (12):

[0111]

[0112] in, represents concatenation processing; F1(.) represents convolution transformation processing; δ represents a nonlinear activation function; f represents the intermediate feature of encoding spatial information.

[0113] Then, f is convolved in height and width directions and activated with Sigmoid function to obtain two outputs of coordinate attention, which are expressed as formulas (13)-(14):

[0114] g h =σ(F h (f h )) (13)

[0115] g w =σ(F w (f w )) (14)

[0116] Among them, g h represents the coordinates in the height direction of the first feature extraction result and / or the second feature extraction result after being processed by the standard attention module; g w represents the coordinates in the width direction of the first feature extraction result and / or the second feature extraction result after being processed by the label attention module; (f h ) represents the intermediate feature encoding spatial information in the height direction; (f w ) represents the intermediate feature encoding spatial information in the width direction; F h (.) indicates the convolution transformation process in the height direction; F w (.) represents the convolution transformation processing in the width direction; σ(.) represents the Sigmoid function activation processing.

[0117] It is understandable that based on g h and g w It is possible to capture first feature extraction results and / or second feature extraction results of different sizes.

[0118] In this embodiment, the coordinate attention module is based on the separation and selection mechanism to achieve the effect of dynamically adjusting the size of the module receptive field according to the size of the input target, thereby improving the multi-scale feature extraction capability. In the coordinate attention module, spatial features are aggregated along two directions respectively to generate a pair of feature maps with direction perception. While preserving the position information in one direction, capturing the long-term dependency in another spatial direction helps the network to locate the target of interest more accurately.

[0119] In another exemplary embodiment of the present invention, continuing with the above Figure 4 The embodiment is described as an example. The neural network model may also include a sliding window deformer self-attention mechanism module, which is used to capture the first dairy product spectral feature image and / or the second dairy product spectral feature image with different magnitude changes based on the sliding window deformer self-attention mechanism module.

[0120] In one embodiment, a sliding window deformer self-attention mechanism module can be added to enhance the network's ability to capture subtle changes in two-dimensional correlation spectra in view of the overlapping characteristics of absorption peaks in the spectrum.

[0121] According to the above description, the online detection method of dairy product quality provided by the present invention obtains multiple first dairy product dynamic spectra containing water disturbance at different times and multiple second dairy product dynamic spectra containing temperature disturbance at different times during the dairy product production process; and based on the multiple first dairy product dynamic spectra, obtains the first dairy product spectral characteristic image with water molecule disturbance removed, and based on the multiple second dairy product dynamic spectra, obtains the second dairy product spectral characteristic image with temperature disturbance removed; and then inputs the first dairy product spectral characteristic image and the second dairy product spectral characteristic image into a pre-trained neural network model, so that the product quality detection result of the dairy product output by the neural network model can be obtained online, thereby improving work efficiency and reducing detection costs.

[0122] Based on the same concept, the present invention also provides an online detection device for dairy product quality.

[0123] The online detection device for dairy product quality provided by the present invention is described below. The online detection device for dairy product quality described below and the online detection method for dairy product quality described above can be referred to each other.

[0124] Figure 6 It is a structural schematic diagram of the online detection device for dairy product quality provided by the present invention.

[0125] In an exemplary embodiment of the present invention, Figure 6It can be seen that the online detection device for dairy product quality may include an acquisition module 610, a processing module 620 and a generation module 630, and each module will be introduced below.

[0126] The acquisition module 610 may be configured to acquire, during the production process of dairy products, a plurality of first dairy product dynamic spectra containing water disturbance at different times, and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times;

[0127] The processing module 620 may be configured to obtain a first dairy product spectrum characteristic image with water molecule disturbance removed based on a plurality of first dairy product dynamic spectra, and to obtain a second dairy product spectrum characteristic image with temperature disturbance removed based on a plurality of second dairy product dynamic spectra;

[0128] The generation module 630 may be configured to input the first dairy product spectral characteristic image and the second dairy product spectral characteristic image into a pre-trained neural network model, and obtain online the product quality detection result of the dairy product output by the neural network model.

[0129] In an exemplary embodiment of the present invention, the processing module 620 can obtain the first dairy product spectrum characteristic image with water molecule disturbance removed based on multiple first dairy product dynamic spectra in the following manner:

[0130] Based on multiple first-class dairy product dynamic spectra, the characteristic peak change information of water molecules is determined through a two-dimensional correlation analysis algorithm;

[0131] Based on a plurality of first dairy product dynamic spectra and characteristic peak change information of water molecules, a first dairy product spectrum characteristic image with water molecule disturbance removed is obtained.

[0132] In an exemplary embodiment of the present invention, the processing module 620 can obtain the second dairy product spectrum characteristic image with temperature disturbance removed based on multiple second dairy product dynamic spectra in the following manner:

[0133] Performing second-order derivative spectrum calculation on a plurality of second dairy product dynamic spectra to obtain second-order derivative spectra of the second dairy product dynamic spectra in different bands;

[0134] Based on the second-order derivative spectrum of the dynamic spectrum of the second dairy product in different bands, a spectral characteristic image of the second dairy product with temperature disturbance removed is obtained through a two-dimensional correlation analysis algorithm.

[0135] In an exemplary embodiment of the present invention, the neural network model may include a backbone feature extraction network, a feature pyramid network and a detector; the generation module 630 may implement the neural network model to output the product quality detection result of the dairy product in the following manner:

[0136] Based on the backbone feature extraction network, feature extraction is performed on the first dairy product spectral feature image and the second dairy product spectral feature image, and a first feature extraction result corresponding to the first dairy product spectral feature image and a second feature extraction result corresponding to the second dairy product spectral feature image are obtained respectively;

[0137] Based on the feature pyramid network, feature fusion is performed on the first feature extraction result and the second feature extraction result to obtain a fused result;

[0138] Based on the detector, the fusion results are combined for calculation to obtain the product quality test results of dairy products.

[0139] In an exemplary embodiment of the present invention, before the feature pyramid network, the neural network model also includes a coordinate attention module, which is used to capture the first feature extraction results and / or second feature extraction results of different sizes based on the adaptive receptive field size of the coordinate attention module.

[0140] In an exemplary embodiment of the present invention, the neural network model also includes a sliding window deformer self-attention mechanism module, which is used to capture the first dairy product spectral feature image and / or the second dairy product spectral feature image with different magnitude changes based on the sliding window deformer self-attention mechanism module.

[0141] In an exemplary embodiment of the present invention, the acquisition module 610 may acquire the first dairy product dynamic spectrum containing water disturbance in the following manner:

[0142] When a change in the water content in the dairy product is detected, a first dairy product dynamic spectrum including the water content disturbance is collected based on an infrared spectrum probe immersed in the dairy product;

[0143] The acquisition module 610 may acquire the second dairy product dynamic spectrum including temperature disturbance in the following manner:

[0144] When a temperature change of the dairy product is detected, a second dairy product dynamic spectrum including temperature disturbance is collected based on the infrared spectrum probe immersed in the dairy product.

[0145] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the online detection method of the quality of dairy products, the method comprising: in the production process of dairy products, respectively obtaining a plurality of first dairy product dynamic spectra containing water disturbance at different times, and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times; based on the plurality of first dairy product dynamic spectra, obtaining a first dairy product spectrum characteristic image with water molecule disturbance removed, and based on the plurality of second dairy product dynamic spectra, obtaining a second dairy product spectrum characteristic image with temperature disturbance removed; inputting the first dairy product spectrum characteristic image and the second dairy product spectrum characteristic image into a pre-trained neural network model, and obtaining the product quality detection result of the dairy product output by the neural network model online.

[0146] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0147] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the online detection method of dairy product quality provided by the above methods, the method including: in the dairy product production process, respectively obtaining a plurality of first dairy product dynamic spectra containing water disturbance at different times, and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times; based on the plurality of first dairy product dynamic spectra, obtaining a first dairy product spectral characteristic image with water molecule disturbance removed, and based on the plurality of second dairy product dynamic spectra, obtaining a second dairy product spectral characteristic image with temperature disturbance removed; inputting the first dairy product spectral characteristic image and the second dairy product spectral characteristic image into a pre-trained neural network model, and obtaining the product quality detection result of the dairy product output by the neural network model online.

[0148] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the online detection method for dairy product quality provided by the above-mentioned methods, the method comprising: in a dairy product production process, obtaining a plurality of first dairy product dynamic spectra containing water disturbance at different times, and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times; based on the plurality of first dairy product dynamic spectra, obtaining a first dairy product spectral characteristic image with water molecule disturbance removed, and based on the plurality of second dairy product dynamic spectra, obtaining a second dairy product spectral characteristic image with temperature disturbance removed; inputting the first dairy product spectral characteristic image and the second dairy product spectral characteristic image into a pre-trained neural network model, and obtaining online the product quality detection result of the dairy product output by the neural network model.

[0149] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0151] It is further understood that, although the operations are described in a specific order in the accompanying drawings in the embodiments of the present invention, it should not be understood as requiring the operations to be performed in the specific order or serial order shown, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An online detection method for dairy product quality, characterized in that: The method comprises: In the dairy product production process, a plurality of first dairy product dynamic spectra containing water disturbance at different times and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times are respectively obtained; Based on the plurality of first dairy product dynamic spectra, a first dairy product spectrum characteristic image with water molecule disturbance removed is obtained, and based on the plurality of second dairy product dynamic spectra, a second dairy product spectrum characteristic image with temperature disturbance removed is obtained; The first dairy product spectral characteristic image and the second dairy product spectral characteristic image are input into a pre-trained neural network model, and the product quality detection result of the dairy product output by the neural network model is obtained online.

2. The online detection method for dairy product quality according to claim 1, characterized in that: The method of obtaining the first dairy product spectral characteristic image with water molecule disturbance removed based on the plurality of first dairy product dynamic spectra specifically includes: Based on the multiple dynamic spectra of the first dairy product, determining the characteristic peak change information of the water molecules through a two-dimensional correlation analysis algorithm; Based on the plurality of first dairy product dynamic spectra and the characteristic peak change information of the water molecules, a first dairy product spectrum characteristic image with water molecule disturbance removed is obtained.

3. The online detection method for dairy product quality according to claim 1, characterized in that: The method of obtaining a second dairy product spectral characteristic image with temperature disturbance removed based on a plurality of second dairy product dynamic spectra specifically includes: Performing second-order derivative spectrum calculation on a plurality of second dairy product dynamic spectra to obtain second-order derivative spectra of the second dairy product dynamic spectra in different bands; Based on the second-order derivative spectrum of the second dairy product dynamic spectrum in different wavebands, a spectral characteristic image of the second dairy product with temperature disturbance removed is obtained through a two-dimensional correlation analysis algorithm.

4. The online detection method for dairy product quality according to any one of claims 1 to 3, characterized in that: The neural network model includes a backbone feature extraction network, a feature pyramid network and a detector; the neural network model outputs the product quality detection result of the dairy product in the following manner: Based on the backbone feature extraction network, feature extraction is performed on the first dairy product spectral feature image and the second dairy product spectral feature image to obtain a first feature extraction result corresponding to the first dairy product spectral feature image and a second feature extraction result corresponding to the second dairy product spectral feature image, respectively; Based on the feature pyramid network, feature fusion is performed on the first feature extraction result and the second feature extraction result to obtain a fused result; Based on the detector, calculation is performed in combination with the fused result to obtain the product quality test result of the dairy product.

5. The online detection method for dairy product quality according to claim 4, characterized in that: Before the feature pyramid network, the neural network model also includes a coordinate attention module, which is used to capture the first feature extraction results and / or the second feature extraction results of different sizes based on the adaptive receptive field size of the coordinate attention module.

6. The online detection method for dairy product quality according to claim 4, characterized in that: The neural network model also includes a sliding window deformer self-attention mechanism module, which is used to capture the first dairy product spectral feature image and / or the second dairy product spectral feature image with different magnitude changes based on the sliding window deformer self-attention mechanism module.

7. The online detection method for dairy product quality according to claim 1, characterized in that: The first dairy product dynamic spectrum containing water disturbance is obtained in the following manner: When a change in the water content in the dairy product is detected, collecting the first dairy product dynamic spectrum containing the water content disturbance based on an infrared spectrum probe immersed in the dairy product; The second dairy product dynamic spectrum containing temperature disturbance is obtained in the following manner: When a change in the temperature of the dairy product is detected, the second dairy product dynamic spectrum containing the temperature disturbance is collected based on the infrared spectrum probe immersed in the dairy product.

8. An online detection device for dairy product quality, characterized in that: The device comprises: An acquisition module, used for acquiring a plurality of first dairy product dynamic spectra containing water disturbance at different times and a plurality of second dairy product dynamic spectra containing temperature disturbance at different times during the dairy product production process; A processing module, for obtaining a first dairy product spectrum characteristic image with water molecule disturbance removed based on a plurality of first dairy product dynamic spectra, and obtaining a second dairy product spectrum characteristic image with temperature disturbance removed based on a plurality of second dairy product dynamic spectra; A generation module is used to input the first dairy product spectral characteristic image and the second dairy product spectral characteristic image into a pre-trained neural network model, and obtain the product quality detection result of the dairy product output by the neural network model online.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the online detection method for dairy product quality according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the online detection method for dairy product quality according to any one of claims 1 to 7 is implemented.

Images