Deep learning network model for laser granularity inversion and training method
By constructing a deep learning network model based on Transformer and multilayer perceptron, and combining staged training and cosine annealing methods, the accuracy and stability problems in particle size measurement and inversion by laser particle size analyzer were solved, and efficient and accurate particle size distribution inversion was achieved.
Patent Information
- Application Number
- CN202411493883.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2025-01-10
AI Technical Summary
Existing laser particle size analyzer particle size measurement inversion algorithms suffer from problems such as low inversion accuracy, long calculation time, and unstable calculation, especially in situations with high noise levels where it is difficult to maintain high accuracy.
We employ an encoder and decoder architecture based on the Transformer module, combined with the bridging mechanism of the multilayer perceptron module, and construct a deep learning network model through a multi-head self-attention mechanism and a position-fully connected feedforward network. We also use a staged training method and a cosine annealing method with preheating to dynamically adjust the learning rate.
It improves the measurement accuracy of particle size distribution and the noise resistance of the model, reduces training time and resource consumption, enhances the model's generalization ability on different data distributions, and ensures efficient training and inversion results.
Smart Images
Figure CN119294460A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of deep learning technology, and in particular relates to a deep learning network model and training method for laser particle size inversion. Background Art
[0002] Particles generally refer to geometric bodies with a size ranging from nanometers to millimeters and a specific shape, which can be divided into gas, liquid and solid particles. One of the most important physical properties of particles is particle size distribution, which is closely related to many fields such as combustion, fire protection, metallurgy, petroleum, chemical industry, materials, medicine, environmental protection, etc. The laser particle size analyzer is a particle size measurement instrument based on the principle of light scattering. It is necessary to invert the particle size distribution from the scattered light energy distribution measured by the multi-ring photoelectric detector. In essence, it is to solve the first kind of Fredholm integral, which is a typical ill-posed problem and has poor noise resistance. Commonly used inversion algorithms include nonlinear iterative algorithm, least squares iterative algorithm, singular value decomposition algorithm, BP neural network algorithm, etc. At present, the commonly used algorithms still have problems such as low inversion accuracy, long calculation time, and unstable calculation. The deep learning network model has extremely strong nonlinear regression ability, generalization ability and parallel computing ability. Therefore, there is an urgent need for a deep learning network model and training method to solve the above problems for the inversion calculation of particle size distribution of laser particle size analyzer. Summary of the invention
[0003] In order to solve the above technical problems, the present invention provides a deep learning network model and training method for laser particle size inversion, which is particularly suitable for the inversion calculation of particle size distribution of laser particle size analyzer.
[0004] The technical solution adopted by the present invention is: in the first aspect, a deep learning network model for laser particle size inversion is provided, comprising:
[0005] The encoder module, the decoder module, the bridge module, the upward interpolation module, the fully connected layer module and the soft maximum layer module are used to input the light energy distribution vector, perform interpolation processing through the upward interpolation module, then perform encoding processing through the encoder module, perform bridging processing through the bridge module, perform decoding processing through the decoder module, and finally perform dimensionality change processing through the fully connected layer module and then output the particle size distribution vector after processing by the soft maximum layer module.
[0006] Furthermore, the encoder module and the decoder module are Transformer modules consisting of a residual-connected multi-head self-attention sub-module and a position-fully-connected feed-forward network.
[0007] Furthermore, the bridge module is a multi-layer perceptron module consisting of multiple fully connected networks and layer normalization layers.
[0008] In a second aspect, a training method for a deep learning network model for laser particle size inversion is provided, comprising:
[0009] The loss function is defined as Where C represents the number of particle size ranges in the particle size distribution, p p,i is the i-th component of the particle size distribution calculated by the neural grid model, p T,i is the i-th component of the true value of the particle size distribution;
[0010] Generate unimodal distribution data sets and mixed distribution data sets and divide them into training data sets and test data sets according to the proportion;
[0011] Use the unimodal distribution training data set to train the network model and obtain the first parameter of the model;
[0012] The unimodal distribution training data set and the mixed distribution training data set are combined, and the network model is trained based on the first model parameter to obtain the second model parameter.
[0013] The trained model is evaluated using the test dataset.
[0014] Furthermore, the cosine annealing method with preheating is used to dynamically adjust the learning rate when training the model.
[0015] In a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method of the deep learning network model for laser particle size inversion provided in the present disclosure.
[0016] In a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the training method of the deep learning network model for laser particle size inversion provided by the present disclosure.
[0017] In a fifth aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the training method of the deep learning network model for laser particle size inversion provided in the present disclosure.
[0018] The advantages and positive effects of the present invention are as follows: due to the adoption of the technical scheme, a complete encoding-decoding architecture is formed, the encoder module and the decoder module adopt the Transformer module and are connected by a bridge module, thereby ensuring efficient transmission of information; the multi-layer perceptron submodule and the dynamic learning rate adjustment method in the bridge module enable the model to have stronger noise resistance, and can maintain high accuracy even in the case of large noise; through the multi-head self-attention mechanism, the model can more accurately invert the particle size distribution from the light energy distribution vector measured by the laser particle size analyzer, thereby improving the measurement accuracy; the position fully connected feedforward network further enhances the nonlinear transformation ability of the model, thereby ensuring accurate inversion of the particle size distribution; the staged training method enables the model to perform well not only on unimodal distribution data, but also on mixed distribution data, thereby improving the generalization ability of the model; the cosine annealing method with preheating helps the model to converge faster, thereby reducing the training time and resource consumption, thereby improving the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 This is a schematic diagram of the deep learning network model structure of an embodiment of the present invention
[0020] Figure 2 This is a flowchart of a training method for a deep learning network model according to an embodiment of the present invention.
[0021] Figure 3 This is a schematic diagram of the process of loss function decline during training of a unimodal distribution data set according to an embodiment of the present invention.
[0022] Figure 4 Schematic diagram of the inversion of narrow unimodal distribution particle size and comparison with the true value according to an embodiment of the present invention
[0023] Figure 5 Schematic diagram of the inversion of the particle size of a wide unimodal distribution and the comparison with the true value according to an embodiment of the present invention
[0024] Figure 6 This is a schematic diagram of the loss function descent process during mixed distribution data set training according to an embodiment of the present invention.
[0025] Figure 7 Schematic diagram of particle size inversion of non-obvious bimodal mixed distribution and comparison with the true value according to an embodiment of the present invention
[0026] Figure 8 Schematic diagram of the inversion of particle size of obviously bimodal mixed distribution and comparison with the true value according to an embodiment of the present invention DETAILED DESCRIPTION
[0027] The present disclosure is described more fully below with reference to the accompanying drawings, in which exemplary embodiments of the present disclosure are described. The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0028] like Figure 1 As shown, the present invention provides a deep learning network model for laser particle size inversion, comprising:
[0029] The encoder module 1100, the decoder module 1200, the bridge module 1300, the upward interpolation module 1400, the fully connected layer module 1500 and the soft maximum layer module 1600 are used to input the light energy distribution vector and perform interpolation processing through the upward interpolation module 1400, then perform encoding processing through the encoder module 1100, perform bridge processing through the bridge module 1300, perform decoding processing through the decoder module 1200, and finally perform dimensionality change processing through the fully connected layer module 1500 and then output the particle size distribution vector after processing by the soft maximum layer module 1600.
[0030] The overall structure of the deep learning network model provided by the present invention is a deep learning network model based on the self-attention mechanism and the encoder-decoder structure, which is composed of multiple encoder modules 1100, decoder modules 1200 and bridge modules 1300. The module types of the encoder module 1100 and the decoder module 1200 are Transformer modules composed of multi-head self-attention submodules connected by residuals and position fully connected feedforward networks. The bridge module 1300 is a multi-layer perceptron module composed of multiple fully connected networks and layer normalization layers. By inputting the light energy distribution vector E B×1×c The upward interpolation module 1400 becomes E B×1×2c , and then pass through each encoder module 1100 and decoder module 1200 in turn, where B is the batch size of the network model hyperparameters, c is the number of components in the light energy distribution vector, and the output of the corresponding encoder module 1100 is summed with the output of the corresponding decoder module 1200 after passing through the bridge module 1300. The output of the decoder module 1200 in the last ring is connected to the fully connected layer module 1500 to make the dimension become B×1×c, and then connected to the soft maximum layer module 1600 to invert the particle size distribution.
[0031] Preferably, the total number of layers of the deep learning network model provided by the present invention is 362, and the total number of parameters is 23004128. The operating hardware environment has a memory size of 64G, a GPU of RTX3090, and a video memory of 24G; the operating software environment is an operating system of Windows 10, Python 3.7.4, PyTorch 1.10.0, and CUDA 11.1.
[0032] By adopting the above technical solution, a complete encoding-decoding architecture is formed. The encoder module and the decoder module adopt Transformer modules and are connected by bridge modules to ensure efficient transmission of information. The position fully connected feedforward network further enhances the nonlinear transformation capability of the model and ensures accurate inversion of particle size distribution.
[0033] In one embodiment, the encoder module 1100 and the decoder module 1200 are Transformer modules composed of residual-connected multi-head self-attention sub-modules and position-fully-connected feed-forward networks.
[0034] Preferably, the number of Transformer modules K in the encoder module 1100 and the decoder module 1200 is 9, the number of heads of the multi-head self-attention submodule in the Transformer module is 8, the function of the multi-head self-attention submodule adopts nn.linear, the function of the normalization layer adopts nn.Layernorm, and the function of the positional fully connected feedforward network adopts nn.linear and nn.GELU.
[0035] Through the multi-head self-attention mechanism, the model can more accurately invert the particle size distribution from the light energy distribution vector measured by the laser particle size analyzer, thereby improving the measurement accuracy.
[0036] In one embodiment, the bridge module 1300 is a multi-layer perceptron module consisting of a plurality of fully connected networks and layer normalization layers.
[0037] Preferably, the function of the fully connected layer in the bridge module 1300 adopts nn.Layernorm, and the function of the nonlinear activation layer adopts nn.GELU.
[0038] The multi-layer perceptron submodule in the bridge module and the dynamic learning rate adjustment method make the model more resistant to noise and maintain high accuracy even in the presence of large noise.
[0039] like Figure 2 As shown, in order to facilitate the training of the network model provided by the present disclosure, the present disclosure also provides a training method for a deep learning network model for laser particle size inversion, including:
[0040] S2100, define the loss function as Where C represents the number of particle size ranges in the particle size distribution, p p,i is the i-th component of the particle size distribution calculated by the neural grid model, p T,i is the i-th component of the true value of the particle size distribution;
[0041] S2200, generating a unimodal distribution data set and a mixed distribution data set and dividing them into a training data set and a test data set according to a proportion;
[0042] Use unimodal and mixed distribution data sets generated based on specific laser particle size analyzer parameters, Fraunhofer diffraction theory, and specific particle size distribution functions.
[0043] S2300, training the network model using a unimodal distribution training data set to obtain a first model parameter;
[0044] The unimodal distribution data set and the mixed distribution data set are divided into training data set and test data set according to the ratio of r:1-r.
[0045] S2400, merging the unimodal distribution training data set and the mixed distribution training data set, and continuing to train the network model based on the first model parameter to obtain the second model parameter;
[0046] The phased training method enables the model to perform well not only on unimodal distribution data, but also to achieve excellent performance on mixed distribution data, thus improving the generalization ability of the model.
[0047] S2500, use the test dataset to evaluate the trained model.
[0048] like Figure 3-8 As shown, preferably, the hyperparameters of the training model are set to batch size B = 256, training epoch = 500, and the learning rate is dynamically adjusted using the cosine annealing method with preheating, with an initial learning rate of 1×e -6 , preheat 5 times, maximum learning rate 2×e -4 . The optical structure parameters of the laser particle size analyzer are obtained to generate a data set, assuming that the particle size distribution satisfies the Rosin-Rammler (RR) function. The unimodal distribution data set contains 20,000 data pairs, and the mixed distribution data set contains 100,000 data pairs. Each data pair consists of a particle size distribution vector and a light energy distribution vector. Both the particle size distribution vector and the light energy distribution vector contain 32 components, i.e. c = 32. Let r = 0.8 to generate a training data set and a test data set. The neural network model is trained using a unimodal distribution training data set, and the loss function descent process is as follows: Figure 3 As shown in Figure 2, when training to 500 epochs, the loss function drops to 1×10 -5The trained neural network model is tested using a unimodal distribution test data set. Figure 4 and Figure 5 The inversion results of narrow unimodal distribution and wide unimodal distribution particle size and the examples of comparison with the corresponding true values are given respectively. Taking the first parameter of the model as the initial parameter, the unimodal distribution training data set and the mixed distribution training data set are combined to continue training the neural network model. The loss function descent process is shown in Figure 6 As shown in Figure 2, when training to 500 epochs, the loss function drops to 2×10 -8 It basically no longer changes, and the second parameter of the model is obtained. Figure 7 and Figure 8 The particle size inversion results of non-obvious bimodal and obvious bimodal mixed distributions are given, as well as the comparison examples with the corresponding true values. Finally, the trained model is evaluated using the test data set.
[0049] In one embodiment, a cosine annealing method with preheating is used to dynamically adjust the learning rate when training the model.
[0050] The cosine annealing method with preheating helps the model converge faster, reduces training time and resource consumption, and improves training efficiency.
[0051] Based on the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0052] An electronic device includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the unified coding method for the Internet of Things collection of campus equipment assets provided by the present disclosure.
[0053] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0054] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the unified coding method for collecting campus equipment assets using the Internet of Things provided by the present disclosure.
[0055] Various embodiments of the present disclosure may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0056] A computer program product includes a computer program / instruction, and the computer program / instruction is executed by a processor to implement the unified coding method for collecting campus equipment assets using the Internet of Things provided by the present invention.
[0057] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0058] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0059] The embodiments of the present invention are described in detail above, but the contents described are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.
Claims
1. A deep learning network model for laser particle size inversion, characterized in that: include: The encoder module, the decoder module, the bridge module, the upward interpolation module, the fully connected layer module and the soft maximum layer module are used to input the light energy distribution vector, perform interpolation processing through the upward interpolation module, then perform encoding processing through the encoder module, perform bridging processing through the bridge module, perform decoding processing through the decoder module, and finally perform dimensionality change processing through the fully connected layer module and then output the particle size distribution vector after processing by the soft maximum layer module.
2. The network model according to claim 1, characterized in that: The encoder module and decoder module are Transformer modules consisting of residual-connected multi-head self-attention sub-modules and position-fully-connected feed-forward networks.
3. The network model according to claim 1, characterized in that: The bridge module is a multi-layer perceptron module consisting of multiple fully connected networks and layer normalization layers.
4. A training method for a deep learning network model for laser particle size inversion, characterized in that: include: The loss function is defined as Where C represents the number of particle size ranges in the particle size distribution, p p,i is the i-th component of the particle size distribution calculated by the neural grid model, p T,i is the i-th component of the true value of the particle size distribution; Generate unimodal distribution data sets and mixed distribution data sets and divide them into training data sets and test data sets according to the proportion; Use the unimodal distribution training data set to train the network model and obtain the first parameter of the model; The unimodal distribution training data set and the mixed distribution training data set are combined, and the network model is trained based on the first model parameter to obtain the second model parameter. The trained model is evaluated using the test dataset.
5. The training method according to claim 4, characterized in that: When training the model, the cosine annealing method with preheating is used to dynamically adjust the learning rate.
6. An electronic device comprising: at least one processor; as well as a memory communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor so that the at least one processor can execute the method according to any one of claims 4 to 5.
7. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 4 to 5.
8. A computer program product, comprising a computer program / instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 4 to 5.