A particle detection method and system based on precursor network

Abstract

The present invention relates to the field of machine vision technology, and discloses a particle detection method and system based on a precursor network. The method comprises: obtaining a microscope image of a precursor to be processed, and preprocessing the microscope image; inputting the preprocessed microscope image into the precursor network, obtaining the category of the precursor to be processed, the precursor target segmentation area, and a multimodal array for identifying the precursor of this category, wherein the precursor network is a network model with dual-branch outputs constructed based on a multi-branch feature extraction block, and the multi-branch feature extraction block segments the features of the microscope image according to a predetermined ratio, processes a part of the features, and then re-integrates with other parts of the features to obtain enhanced features. The present invention solves the problems of existing manual detection of multi-category recognition and quality detection of precursor particles.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a particle detection method and system based on a precursor network. Background Art

[0002] In the production process of high-performance materials and chemicals, the size, morphology and other microscopic characteristics of precursor particles have a decisive influence on the quality and characteristics of the final product. Different types of precursor particles have different sizes and morphologies. Although there is automated or intelligent detection of different types of particles at this stage, there are still many limitations.

[0003] Taking CN118552569A as an example, this method uses a neural network image segmentation model to perform multi-layer extraction and fusion of image features, thereby improving segmentation accuracy and efficiency. However, this technology mainly focuses on the accuracy of image segmentation, and is still insufficient in the automatic identification of multiple categories of particles and the detection of particles with complex morphology. In addition, its model relies on a specific feature fusion strategy and cannot fully adapt to complex and diverse industrial application scenarios, especially the characterization of unknown particles. Similarly, the method in CN116840108A realizes the synchronous detection of particle size and shape through optical image analysis. Although it improves real-time performance, its detection objects are limited to continuously synthesized precursor particles, and its in-depth analysis capabilities for multimodal characteristics and particles with complex morphology are limited.

[0004] Furthermore, existing mainstream technologies such as YOLO (especially YOLOv5 and YOLOv8) and UNet also face certain limitations. YOLOv5 and YOLOv8 have excellent real-time performance in target detection, but the grid prediction mechanism they adopt performs poorly when dealing with particles with blurred boundaries or complex shapes, and the detection frame is prone to offset or incomplete coverage. In addition, the YOLO model has weak generalization ability when dealing with particles with large differences in size and morphology, especially when detecting particles of unknown categories. UNet achieves feature fusion through encoding-decoding structure and jump connection. Although it is suitable for high-resolution target segmentation tasks, it has limited feature extraction and fusion capabilities when dealing with complex particle morphology in microscopic images. At the same time, the computational overhead is large in high-resolution scenarios and the real-time performance is insufficient. Therefore, YOLO and UNet are difficult to fully meet the needs of complex particle detection tasks. Summary of the invention

[0005] The present invention provides a particle detection method and system based on a precursor network to solve the problems of multi-category recognition and quality detection of precursor particles in existing manual detection.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] In a first aspect, the present invention provides a particle detection method based on a precursor network, wherein the precursor network comprises an extraction module for extracting features, a multi-branch module for enhancing features, a fusion module for fusing features, and a dual-branch output module for outputting detection results, wherein the multi-branch module and the fusion module are constructed based on the multi-branch feature extraction block, and the dual-branch output module comprises a category segmentation branch for outputting a category of a precursor and a category segmentation region, and a multi-modal branch for outputting a multi-modal array of a precursor;

[0008] The method comprises the following steps:

[0009] Step 1: Obtain a microscope image of a precursor to be processed, and preprocess the microscope image;

[0010] Step 2: Input the preprocessed microscope image into the precursor network to obtain the category of the precursor to be processed, the precursor target segmentation area, and the multimodal array for identifying the precursor of this category;

[0011] The precursor network is a network model with dual-branch outputs constructed based on a multi-branch feature extraction block, wherein the multi-branch feature extraction block segments the features of the microscope image according to a predetermined ratio, processes a portion of the segmented features, and then re-integrates with other portions of the features to obtain enhanced features;

[0012] The multi-branch feature extraction block includes an input layer, an SE block, a fusion layer, an output layer, a segmentation layer for segmenting features according to a predetermined ratio, and a depth-separable convolutional layer for extracting features;

[0013] The segmentation layer obtains the features from the input layer, segments the features according to a predetermined ratio, and then transmits some of the features to the depthwise separable convolutional layer. The depthwise separable convolutional layer extracts the segmented features, enhances the expression through the SE block, and finally fuses them with another part of the features through the fusion layer and outputs them through the output layer.

[0014] Among them, whether the precursor meets the requirements is determined by the last identified category combined with the segmented area, and the multimodal array of the precursor of this category is the corresponding feature array when identifying the unknown precursor. The multimodal array can be stored for the next identification of the unknown precursor.

[0015] Further, the multi-branch module includes a first trunk for increasing the receptive field, a second trunk for enhancing feature extraction, and a low-level feature extraction node for supplementing characteristic information;

[0016] The first trunk is connected to the extraction module and the second trunk respectively, the second trunk is connected to the low-level feature extraction node, and the first trunk and the low-level feature extraction node are both connected to the fusion module;

[0017] The first trunk and the second trunk each include a multi-branch feature extraction block.

[0018] Furthermore, the first backbone includes a convolution layer with a convolution kernel of 3 and a stride of 2 and a multi-branch feature extraction block;

[0019] The second backbone includes a convolution layer with a convolution kernel of 3 and a stride of 2 and a multi-branch feature extraction block;

[0020] The low-level feature extraction node includes a convolution layer with a convolution kernel of 1.

[0021] Furthermore, the fusion module includes a multi-scale fusion unit and a dimension reduction unit, and the multi-scale fusion unit performs fusion enhancement on the features and then transmits the features to the dimension reduction unit to perform fusion and dimension reduction processing on the features;

[0022] The first trunk and the low-level feature extraction nodes are both connected to the multi-scale fusion unit, the multi-scale fusion unit is connected to the dimension reduction unit, and the low-level feature extraction nodes and the dimension reduction unit are both connected to the dual-branch output module.

[0023] Furthermore, the multi-scale fusion unit includes a convolution layer with a convolution kernel of 3, a multi-branch feature extraction block, and a convolution layer with a convolution kernel of 1;

[0024] The dimension reduction unit includes a splicing layer and a convolution layer with a convolution kernel of 3 in sequence.

[0025] Furthermore, the category segmentation branch and the multimodal branch both include a convolution layer with a convolution kernel of 1.

[0026] Furthermore, a hierarchical adaptive initialization method is used in the construction process of the precursor network, and the precursor network is optimized in the training process by at least one of previous propagation, loss calculation, back propagation, validation set evaluation, early stopping strategy and hyperparameter adjustment;

[0027] The loss calculation includes: the loss calculation includes: the category of the precursor, the target segmentation area, and the multimodal array obtained by the precursor loss function constraint;

[0028] The precursor loss function expression is:

[0029] ;

[0030] in, represents the precursor loss function; represents branch loss; represents the classification loss; represents mask loss; represents the consistency loss; represents the attention weighted loss; , , , , Represents the weight coefficient.

[0031] In a second aspect, the present invention also provides a precursor network-based particle detection system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned methods when executing the computer program.

[0032] Beneficial effects:

[0033] The present invention provides a particle detection method and system based on a precursor network. The precursor network with dual-branch outputs respectively obtains the category of the precursor to be detected, the segmented area, and the multimodal array of the precursor of the category. The multimodal array can be stored for the next identification of the unknown precursor, and the multi-branch feature extraction block performs segmentation and fusion on the features, which reduces the amount of calculation while retaining the efficient features, strengthens the feature output, ensures the accuracy of the particle detection classification results, and effectively solves the problems of the existing manual detection of precursor particles in multi-category identification and quality detection.

[0034] The accuracy and diversity recognition capabilities of particle detection have been significantly improved. A dual-branch output precursor network model is introduced to extract particle categories, segment regions, and generate multimodal arrays, respectively, to achieve efficient recognition of unknown particles and subsequent storage support. Secondly, the multi-branch feature extraction block segments and fuses the features of the microscope image according to a predetermined ratio, which not only reduces the amount of calculation, but also enhances the performance of feature output, ensuring efficient feature extraction and retention;

[0035] This method can not only automatically process particle morphology, distribution and category, but also support the fine detection of complex particle morphology, which is especially suitable for multi-category detection needs in industrial production. Through the introduction of multimodal arrays, this method can build an adaptable feature representation mechanism, laying a solid foundation for the identification of unknown particles. At the same time, the application of hierarchical adaptive initialization and optimization strategies further improves the robustness and generalization ability of the network model, thereby maintaining high detection accuracy in dynamic environments;

[0036] The features are segmented according to a predetermined ratio through a multi-branch feature extraction block, and a part of the features is processed and then re-fused with other parts to obtain enhanced features, thereby achieving selective enhancement of features and efficient extraction and retention of features. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1A flowchart of a particle detection method based on a precursor network according to an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of the structure of the precursor network in an embodiment of the present invention;

[0039] Figure 3 Schematic diagram of the structure of a multi-branch feature extraction block in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the usual meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate quantity restrictions, but indicate the existence of at least one. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0042] See also Figure 1 The embodiment of the present invention provides a particle detection method based on a precursor network, the method comprising the following steps:

[0043] Step 1: Obtain a microscope image of the precursor to be processed and preprocess the microscope image;

[0044] Preprocessing includes standardized resizing and pixel normalization of each image (for example, normalizing pixel values ​​to the range of [0,1]), and dividing the data into training, validation, and test sets. In addition, data augmentation methods such as rotation, random cropping, and brightness adjustment are used to further expand the dataset, thereby improving the robustness of the model under different input conditions.

[0045] Step 2: Input the preprocessed microscope image into the precursor network to obtain the category of the precursor to be processed, the precursor target segmentation area, and the multimodal array for identifying the precursor of this category;

[0046] Among them, the precursor network is a network model with dual-branch outputs constructed based on a multi-branch feature extraction block. The multi-branch feature extraction block segments the features of the precursor according to a predetermined ratio, processes a portion of the segmented features, and then re-integrates with other portions of the features to obtain enhanced features.

[0047] See also Figure 2 ,The precursor network includes an extraction module for extracting features, a multi-branch module for strengthening features, a fusion module for fusing features, and a dual-branch output module for outputting detection results;

[0048] Among them, the multi-branch module and the fusion module are built based on the multi-branch feature extraction block.

[0049] exist Figure 2 There are 15 nodes in total, among which node 1 is the extraction module, nodes 2 to 6 are multi-branch modules, nodes 7 to 11 are fusion modules, and nodes 12 to 15 are dual-branch output modules.

[0050] The multi-branch module includes a first trunk that increases the receptive field, a second trunk that enhances feature extraction, and a low-level feature extraction node that supplements feature information. Figure 2 In the figure, the first trunk is nodes 2 and 3, the second trunk is nodes 4 and 5, and the low-level feature extraction node is node 6;

[0051] The output end of the extraction module is connected to the input end of the first trunk, the output end of the first trunk is connected to the input end of the second trunk, the output end of the second trunk is connected to the input end of the low-level feature extraction node, and the output ends of the first trunk and the low-level feature extraction node are both connected to the input end of the fusion module.

[0052] The introduction of residual connections and splicing operations between different feature layers enhances the fusion capability of feature maps at multiple scales and avoids the problem of gradient vanishing. It has strong sensitivity and robustness in the expression of multi-scale features and is more suitable for detecting tiny particles with complex morphology.

[0053] The first backbone consists of a convolutional layer with a kernel size of 3 and a stride of 2, and a multi-branch feature extraction block, which can capture information with a larger receptive field.

[0054] The second backbone consists of a convolutional layer with a kernel of 3 and a stride of 2 and a multi-branch feature extraction block to enhance the diversity of feature extraction;

[0055] The low-level feature extraction node includes a convolution layer with a convolution kernel of 1 to increase the detail information of the features.

[0056] The fusion module includes a multi-scale fusion unit and a dimensionality reduction unit. The multi-scale fusion unit performs fusion enhancement on the features and then transmits them to the dimensionality reduction unit for fusion and dimensionality reduction processing. Figure 2 The medium-scale fusion units are nodes 7 to 9, and the dimension reduction units are nodes 10 to 11;

[0057] The output ends of the first trunk and the low-level feature extraction nodes are connected to the input end of the multi-scale fusion unit, the output end of the multi-scale fusion unit is connected to the input end of the dimensionality reduction unit, and the output ends of the low-level feature extraction nodes and the dimensionality reduction unit are connected to the input end of the dual-branch output module.

[0058] The multi-scale fusion unit includes a convolution layer with a convolution kernel of 3, a multi-branch feature extraction block, and a convolution layer with a convolution kernel of 1;

[0059] The enhancement unit consists of a concatenation layer and a convolution layer with a convolution kernel of 3.

[0060] The dual-branch output module includes a category segmentation branch and a multimodal branch;

[0061] Among them, the category segmentation branch is used to output the category of the precursor and the target segmentation area, and the multimodal branch is used to output the multimodal array of the precursor;

[0062] Both the category segmentation branch and the multimodal branch include a convolution layer with a convolution kernel of 1.

[0063] For details about the multi-branch feature extraction blocks in the multi-branch module and the fusion module, see Figure 3 , the multi-branch feature extraction block includes an input layer, a segmentation layer for segmenting features according to a predetermined ratio, a depth-separable convolutional layer for extracting features, an SE block, a fusion layer, and an output layer;

[0064] The segmentation layer obtains the features from the input layer and then segments the features according to a predetermined ratio, and then transmits some of the features to the depthwise separable convolutional layer. The depthwise separable convolutional layer extracts the segmented features, and then enhances the expression through the SE block. Finally, it is fused with another part of the features through the fusion layer and output through the output layer.

[0065] In this embodiment, the segmentation layer divides the feature into two parts, one part is , part of , for the split Features are processed through a deep separable convolutional layer to reduce parameters and computational complexity, while maintaining sensitivity to spatial features to achieve the extraction of fine-grained features. After processing, SE is used for adaptive weighting to enhance the expression of important features, and finally the fusion layer is used to combine the unprocessed features. After feature fusion, the results are output through the output layer.

[0066] Through multi-branch feature extraction blocks, it can fully capture the information of particles of different scales, effectively improve the ability to resolve complex backgrounds and tiny particles, retain a large amount of original information and highlight key features, greatly improving detection accuracy and detail recognition capabilities.

[0067] In the present invention, a hierarchical adaptive initialization method is used in the construction process of the precursor network, and the precursor network is optimized through prior propagation, loss calculation, back propagation, validation set evaluation, early stopping strategy and hyperparameter adjustment during training.

[0068] The loss calculation includes: the category of the precursor, the target segmentation area, and the multimodal array obtained through the precursor loss function constraint;

[0069] The precursor loss function expression is:

[0070] ;

[0071] in, represents the precursor loss function; represents branch loss; represents the classification loss; represents mask loss; represents the consistency loss; represents the attention weighted loss; , , , , Represents the weight coefficient.

[0072] The branch loss is used to supervise and encourage the effective expression of features such as particle morphology, size, and defects, while the classification loss ensures the correct classification of particles through cross entropy. The mask loss is used for contour segmentation and can be used. Loss or The loss is used to optimize the edge detection of particles and enhance the details of defect recognition. The consistency loss is used to constrain the consistency of feature expression between multiple branches, where the The norm constrained distance ensures the consistency of feature expression of different branches. The attention weighted loss and the SE block that integrates the attention mechanism in the multi-branch feature extraction block are further combined with the attention score to assign different loss weights to different feature areas, so that the network pays more attention to key areas during back propagation. The precursor loss function constructed by the above losses makes full use of the multi-task and multi-branch structure of the precursor network, and also enhances the innovation of the loss function through feature consistency and attention mechanism, which helps to improve the performance of the network in complex particle detection tasks.

[0073] Finally, the multimodal array serves as a feature array corresponding to the identification of an unknown precursor, and the multimodal array can be stored for the next identification of the unknown precursor.

[0074] After executing a particle detection method based on a precursor network through the precursor network provided in this embodiment, Yolov8 is used to process the same microscope image, and after obtaining the detection results, the results are compared horizontally, see Table 1;

[0075] Table 1: Horizontal comparison between Yolov8 and precursor network

[0076]

[0077] Among them, cls represents category, mask represents segmentation area. According to Table 1, the precursor network provided by the present invention is superior to Yolov8 in both particle category and segmentation area, and can detect and identify precursor particles more accurately and effectively, and can better meet the needs of precursor particle size quality control.

[0078] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A particle detection method based on a precursor network, characterized in that: The precursor network includes an extraction module for extracting features, a multi-branch module for strengthening features, a fusion module for fusing features, and a dual-branch output module for outputting detection results, wherein the multi-branch module and the fusion module are constructed based on a multi-branch feature extraction block, and the dual-branch output module includes a category segmentation branch for outputting the category of the precursor and the target segmentation area, and a multi-modal branch for outputting a multi-modal array of the precursor; The method comprises the following steps: Step 1: Obtain a microscope image of a precursor to be processed, and preprocess the microscope image; Step 2: Input the preprocessed microscope image into the precursor network to obtain the category of the precursor to be processed, the precursor target segmentation area, and the multimodal array for identifying the precursor of this category; The precursor network is a network model with dual-branch outputs constructed based on a multi-branch feature extraction block, wherein the multi-branch feature extraction block segments the features of the microscope image according to a predetermined ratio, processes a portion of the segmented features, and then re-integrates with other portions of the features to obtain enhanced features; The multi-branch feature extraction block includes an input layer, an SE block, a fusion layer, an output layer, a segmentation layer for segmenting features according to a predetermined ratio, and a depth-separable convolutional layer for extracting features; The segmentation layer obtains the features from the input layer, segments the features according to a predetermined ratio, and then transmits some of the features to the depthwise separable convolutional layer. The depthwise separable convolutional layer extracts the segmented features, enhances the expression through the SE block, and finally fuses them with another part of the features through the fusion layer and outputs them through the output layer.

2. The particle detection method based on precursor network according to claim 1, characterized in that: The multi-branch module includes a first trunk that increases the receptive field, a second trunk that enhances feature extraction, and a low-level feature extraction node that supplements feature information; The first trunk is connected to the second trunk, the second trunk is connected to the low-level feature extraction node, and the second trunk and the low-level feature extraction node are both connected to the fusion module; The first trunk and the second trunk each include a multi-branch feature extraction block.

3. The particle detection method based on precursor network according to claim 2, characterized in that: The first backbone includes a convolution layer with a convolution kernel of 3 and a stride of 2 and a multi-branch feature extraction block; The second backbone includes a convolution layer with a convolution kernel of 3 and a stride of 2 and a multi-branch feature extraction block; The low-level feature extraction node includes a convolution layer with a convolution kernel of 1.

4. The particle detection method based on precursor network according to claim 2, characterized in that: The fusion module includes a multi-scale fusion unit and a dimension reduction unit. The multi-scale fusion unit performs fusion enhancement on the features and then transmits them to the dimension reduction unit to perform fusion and dimension reduction processing on the features. The second trunk and the low-level feature extraction nodes are connected to the multi-scale fusion unit, the multi-scale fusion unit is connected to the dimension reduction unit, and the low-level feature extraction nodes and the dimension reduction unit are connected to the dual-branch output module.

5. The particle detection method based on precursor network according to claim 4, characterized in that: The multi-scale fusion unit includes a convolution layer with a convolution kernel of 3, a multi-branch feature extraction block and a convolution layer with a convolution kernel of 1; The dimension reduction unit includes a splicing layer and a convolution layer with a convolution kernel of 3 in sequence.

6. The particle detection method based on precursor network according to claim 2, characterized in that: The category segmentation branch and the multimodal branch both include a convolution layer with a convolution kernel of 1.

7. The particle detection method based on precursor network according to any one of claims 1 to 6, characterized in that: The precursor network is constructed by using a hierarchical adaptive initialization method, and the precursor network is optimized by at least one of previous propagation, loss calculation, back propagation, validation set evaluation, early stopping strategy, and hyperparameter adjustment during training; The loss calculation includes: the category of the precursor, the target segmentation area and the multimodal array obtained by the precursor loss function constraint; The precursor loss function expression is: ； in, represents the precursor loss function; represents branch loss; represents the classification loss; represents mask loss; represents the consistency loss; represents the attention weighted loss; , , , , Represents the weight coefficient.

8. A particle detection system based on a precursor network, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images