Probe distribution target detection method and device based on multichannel neural network model

Abstract

The invention discloses a probe distribution target detection method based on a multichannel neural network model, and belongs to the technical field of image processing. The method comprises the following steps: acquiring G channel data and R channel data of a fluorescence image of a cell to be detected; inputting the G channel data into the trained yov8-SCC deep neural network model to obtain a green fluorescence probe distribution diagram, and inputting the R channel data into the trained yov8-SCC deep neural network model to obtain a red fluorescence probe distribution diagram; combining the green fluorescent probe distribution diagram and the red fluorescent probe distribution diagram to obtain a relation diagram of to-be-detected cells and probe distribution; wherein the yov8-SCC deep neural network model comprises a Backbone network, a Neck network and a Head network, the Backbone network comprises an SPFF module, the Neck network comprises a feature pyramid network and a path aggregation network, the feature pyramid network comprises a first CE module, the path aggregation network comprises a second CE module and a first SPD module, and the method improves the accuracy and efficiency of cell detection.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to a probe distribution target detection method and device based on a multi-channel neural network model. Background Art

[0002] With the widespread application of indirect fluorescence immunoassay (IFIA) technology, ANA (antinuclear antibody) fluorescence image analysis has become an important basis for the diagnosis of autoimmune diseases. Although IFIA is considered the "gold standard" for ANA detection due to its high sensitivity and specificity, its results still rely heavily on manual microscopic observation and subjective interpretation. Traditional manual interpretation methods are subject to visual fatigue, significant subjective bias, and insufficient ability to capture weak fluorescence signals. This is particularly prone to missed detections or misinterpretations when detecting rare karyotypes (such as centromere and nuclear dot types), limiting the accuracy and efficiency of clinical diagnosis.

[0003] In recent years, deep learning models based on convolutional neural networks (CNNs) have been introduced into the field of medical image analysis due to their powerful feature learning capabilities. However, they still face challenges in detecting ANA fluorescence images. General-purpose object detection networks (such as the YOLO series) are not optimized for the sparsity, low signal-to-noise ratio, and karyotype texture specificity of fluorescence images. This results in insufficient learning of small fluorescent targets and rare karyotype features, leading to high rates of missed detection. Furthermore, existing algorithms often use single-scale feature extraction strategies, making it difficult to effectively integrate global cellular morphology with local fluorescence details, further limiting the robustness of detection in complex samples.

[0004] Although the improved algorithm based on the YOLO architecture improves detection efficiency through multi-scale prediction, its default network structure still has limitations in its ability to represent weak intracellular fluorescence signals and rare nuclear textures. For example, problems such as uneven feature response caused by differences in fluorescence staining intensity and target confusion caused by overlapping fluorescent regions between cells will increase the risk of the model missing low-contrast targets. The current method lacks a mechanism to focus attention on key fluorescent regions during the feature extraction stage and is unable to adaptively enhance deep semantic features that are strongly associated with rare nuclear types. This results in the model's insufficient ability to suppress complex background interference, resulting in problems such as missed detection of fluorescent targets and low detection accuracy when detecting cell fluorescence images. Summary of the Invention

[0005] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a probe distribution target detection method and device based on a multi-channel neural network model, which improves the accuracy and efficiency of cell detection.

[0006] In a first aspect, the present application provides a probe distribution target detection method based on a multi-channel neural network model, the method comprising:

[0007] Obtain G channel data and R channel data of the fluorescence image of the cell to be tested;

[0008] The G channel data is input into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, and the R channel data is input into the trained yolov8-SCC deep neural network model to obtain a red fluorescent probe distribution map;

[0009] Merging the green fluorescent probe distribution map and the red fluorescent probe distribution map to obtain a relationship map between the cells to be tested and the probe distribution;

[0010] Among them, the yolov8-SCC deep neural network model includes a Backbone network, a Neck network and a Head network. The Backbone network includes an SPFF module, the Neck network includes a feature pyramid network and a path aggregation network, the feature pyramid network includes a first CE module, and the path aggregation network includes a second CE module and a first SPD module.

[0011] According to one embodiment of the present application, the G channel data is input into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, including:

[0012] Input the G channel data into the Backbone network, obtain a first feature map through the SPFF module, and obtain a second feature map, a third feature map, and a fourth feature map through the second SPD module, the third SPD module, and the fourth SPD module of the Backbone network;

[0013] Inputting the first feature map, the second feature map, the third feature map, and the fourth feature map into a feature pyramid network, obtaining a fifth feature map and a sixth feature map through a first C2f module and a second C2f module of the feature pyramid network, and obtaining a seventh feature map through a first CE module of the feature pyramid network;

[0014] Inputting the first, fifth, sixth, and seventh feature maps into a path aggregation network, obtaining an eighth and ninth feature maps through a third and fourth C2f modules of the path aggregation network, and obtaining a tenth feature map through a second CE module of the path aggregation network;

[0015] The fifth feature map, the eighth feature map, the ninth feature map and the tenth feature map are input into the Head network to obtain a green fluorescent probe distribution map.

[0016] According to one embodiment of the present application, the convolution module of the first CE module includes an ECA attention mechanism module, which performs global average pooling on the output of the convolution layer in the first CE module, compresses the spatial information of each channel into a single value, and obtains a feature vector with the same number of channels;

[0017] The feature vector is processed by one-dimensional convolution and then input into the Sigmoid activation function to obtain the channel attention weight.

[0018] According to one embodiment of the present application, the first feature map, the second feature map, the third feature map, and the fourth feature map are input into a feature pyramid network, a fifth feature map and a sixth feature map are obtained through a first C2f module and a second C2f module of the feature pyramid network, and a seventh feature map is obtained through a first CE module of the feature pyramid network, including:

[0019] Upsampling the first feature map and fusing it with the fourth feature map, extracting features through a first C2f module to obtain a fifth feature map;

[0020] Upsampling the fifth feature map and fusing it with the third feature map, and passing it through the first CE module to obtain a seventh feature map;

[0021] The seventh feature map is upsampled and fused with the second feature map, and features are extracted by the second C2f module to obtain the sixth feature map.

[0022] According to one embodiment of the present application, the first feature map, the fifth feature map, the sixth feature map, and the seventh feature map are input into the path aggregation network, the eighth feature map and the ninth feature map are obtained by the third C2f module and the fourth C2f module of the path aggregation network, and the tenth feature map is obtained by the second CE module of the path aggregation network, including:

[0023] The sixth feature map is processed by the convolution operation and the fifth SPD module of the path aggregation network and then fused with the seventh feature map. After extracting features through the third C2f, the eighth feature map is obtained.

[0024] The eighth feature map is processed by a convolution operation and a sixth SPD module of a path aggregation network, and then fused with the fifth feature map, and the tenth feature map is obtained by passing it through a second CE module;

[0025] The tenth feature map is processed by the convolution operation and the seventh SPD module of the path aggregation network and then fused with the first feature map, and then passed through the fourth C2f module to obtain the ninth feature map.

[0026] According to one embodiment of the present application, the eighth feature map is processed by a convolution operation and a sixth SPD module of a path aggregation network and then fused with the fifth feature map, and a tenth feature map is obtained by a second CE module, including:

[0027] The fused feature map is input into the second CE module to obtain the first sub-feature map and the second sub-feature map;

[0028] The second sub-feature map is convolved once and then input into the ECA attention mechanism module, and the third sub-feature map is obtained through a global average pooling.

[0029] Calculating the convolution kernel size of the third sub-feature map based on the inverse mapping function;

[0030] Convolving the third sub-feature map to obtain a fourth sub-feature map;

[0031] After performing a convolution operation on the fourth sub-feature map with a convolution kernel size of a preset value, a fifth sub-feature map is obtained;

[0032] The fifth sub-feature map and the second sub-feature map are skip-connected to obtain a sixth sub-feature map;

[0033] The sixth sub-feature map is convolved once and then input into the ECA attention mechanism module, and the above operation is repeated N times to obtain the Nth sub-feature map;

[0034] After concatenating the first sub-feature map, the sixth sub-feature map, and the Nth sub-feature map, a tenth feature map is obtained through convolution.

[0035] According to one embodiment of the present application, the Head network includes multiple Detect layers, each Detect layer includes a regression branch and a classification branch, the regression branch is used to output the position and size of the bounding box, and the classification branch is used to output the category and confidence.

[0036] In a second aspect, the present application provides a probe distribution target detection device based on a multi-channel neural network model, the device comprising:

[0037] An acquisition module is used to acquire G channel data and R channel data of the fluorescence image of the cell to be tested;

[0038] A processing module is used to input the G channel data into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, and input the R channel data into the trained yolov8-SCC deep neural network model to obtain a red fluorescent probe distribution map;

[0039] a merging module, configured to merge the green fluorescent probe distribution map and the red fluorescent probe distribution map to obtain a relationship map between the cells to be tested and the probe distribution;

[0040] Among them, the yolov8-SCC deep neural network model includes a Backbone network, a Neck network and a Head network. The Backbone network includes an SPFF module, the Neck network includes a feature pyramid network and a path aggregation network, the feature pyramid network includes a first CE module, and the path aggregation network includes a second CE module and a first SPD module.

[0041] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the processor executes the computer program, it implements the probe distribution target detection method based on the multi-channel neural network model as described in the first aspect above.

[0042] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the probe distribution target detection method based on a multi-channel neural network model as described in the first aspect above.

[0043] In the fifth aspect, the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the probe distribution target detection method based on the multi-channel neural network model as described in the first aspect.

[0044] In a sixth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the probe distribution target detection method based on a multi-channel neural network model as described in the first aspect above.

[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application.

[0046] The present invention provides a probe distribution target detection method based on a multi-channel neural network model, which has the following advantages over the prior art:

[0047] (1) The present invention obtains the G channel and R channel data of the cells to be tested and inputs them into the trained YOLOv8-SCC model respectively to obtain the green and red fluorescent probe distribution maps. By utilizing the collaborative work of the SPFF module, CE module and SPD module in the YOLOv8-SCC model, the detection accuracy of the cell probe distribution is effectively improved, and the distribution of probes inside the cells can be more accurately displayed, which assists further biological analysis and can improve the cell detection accuracy and processing speed. It is suitable for complex image scenes, realizes the analysis of various medical images, improves the efficiency of medical diagnosis, reduces human errors, and enhances the cell analysis and pathology detection capabilities in scientific research and clinical applications.

[0048] (2) The present invention adds an ECA attention mechanism module to the CE module and uses one-dimensional convolution to process the feature vector after global average pooling, which reduces dimensionality reduction and retains more information. The adaptive adjustment of the channels of the feature map through one-dimensional convolution can enable the model to pay more attention to the most important features for the current task, realize cross-channel interaction in channel attention learning, and significantly improve the performance of the model with very little additional computational effort, focusing more on the display of small targets, thereby improving the accuracy and robustness of the yolov8-SCC deep neural network model detection.

[0049] (3) The present invention introduces the CE module and the SPD module into the path aggregation network. The CE module can adaptively enhance the features, effectively improving the sensitivity of the yolov8-SCC deep neural network model to key features, thereby improving the accuracy of target detection in complex scenarios. The SPD module further enhances the detection ability of the yolov8-SCC deep neural network model for targets of different scales through multi-scale spatial feature processing, especially when processing small targets and edge targets. By combining the CE module and the SPD module, the yolov8-SCC deep neural network model can perform feature fusion and information extraction at a higher level, improve the cell detection accuracy and processing speed, effectively process complex image scenes, and realize the analysis of various medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 This is one of the flow diagrams of the probe distribution target detection method based on the multi-channel neural network model provided in the embodiment of the present application;

[0052] Figure 2 This is one of the structural diagrams of the YOLOv8-SCC model provided in the embodiments of the present application;

[0053] Figure 3 This is the working mechanism of the SPD module provided in the embodiment of the present application;

[0054] Figure 4 This is a schematic diagram of the structure of the CE module provided in the embodiment of the present application;

[0055] Figure 5 This is a comparison chart of the results of the yolov8-SCC model provided in the examples of this application and other models;

[0056] Figure 6 This is the second flow chart of the probe distribution target detection method based on the multi-channel neural network model provided in the embodiment of the present application;

[0057] Figure 7 This is the second structural diagram of the YOLOv8-SCC model provided in the embodiment of the present application;

[0058] Figure 8 Schematic diagram of the structure of the ECA module provided in the embodiment of the present application;

[0059] Figure 9 Schematic diagram of the structure of a probe distribution target detection device based on a multi-channel neural network model provided in an embodiment of the present application;

[0060] Figure 10 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0062] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0063] Below, in combination with the accompanying drawings, the probe distribution target detection method based on a multi-channel neural network model, the probe distribution target detection device based on a multi-channel neural network model, the electronic device and the readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0064] Among them, the probe distribution target detection method based on the multi-channel neural network model can be applied to the terminal, and can be specifically executed by hardware or software in the terminal.

[0065] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer having a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0066] In the following embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and a joystick.

[0067] The embodiment of the present application provides a probe distribution target detection method based on a multi-channel neural network model. The execution subject of the probe distribution target detection method based on a multi-channel neural network model can be an electronic device or a functional module or functional entity in the electronic device that can implement the probe distribution target detection method based on a multi-channel neural network model. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablets, computers, cameras and wearable devices, etc. The probe distribution target detection method based on a multi-channel neural network model provided in the embodiment of the present application is explained below using an electronic device as an example of the execution subject.

[0068] Figure 1 This is one of the flow charts of the probe distribution target detection method based on the multi-channel neural network model provided in the embodiment of the present application, such as Figure 1 As shown, the probe distribution target detection method based on the multi-channel neural network model includes: step 110, step 120 and step 130.

[0069] Step 110: Acquire G channel data and R channel data of the fluorescence image of the cell to be tested;

[0070] It is easy to understand that a fluorescence image of the cell to be tested is captured by a microscope or imaging device, and G channel data and R channel data are extracted from the fluorescence image of the cell to be tested, wherein the G channel data is green fluorescence channel data and the R channel data is red fluorescence channel data.

[0071] Step 120: input the G channel data into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, and input the R channel data into the trained yolov8-SCC deep neural network model to obtain a red fluorescent probe distribution map;

[0072] Step 130: merging the green fluorescent probe distribution map and the red fluorescent probe distribution map to obtain a relationship map between the cells to be tested and the probe distribution;

[0073] Among them, the yolov8-SCC deep neural network model includes a Backbone network, a Neck network and a Head network. The Backbone network includes an SPFF module, the Neck network includes a feature pyramid network and a path aggregation network, the feature pyramid network includes a first CE module, and the path aggregation network includes a second CE module and a first SPD module.

[0074] It is easy to understand that the G channel data and the R channel data are input into the trained yolov8-SCC deep neural network model to obtain the green fluorescent probe distribution map and the red fluorescent probe distribution map. The yolov8-SCC deep neural network model is a deep neural network model based on yolov8 by adding SPFF (Spatial Pyramid Feature Fusion) module, CE (Cross-Entropy) module and SPD (Self-ensembling Prediction) module.

[0075] In some embodiments, the training process of the yolov8-SCC deep neural network model is as follows:

[0076] (1) Construct a dataset for the yolov8-SCC deep neural network model;

[0077] a. Obtain 1231 high-resolution cell images to form an original high-resolution cell image set I, I∈(I1,I2,…,I i ,…,I n ), I i is the i-th high-resolution cell image, i∈(1,…,n); n=1231.

[0078] b. Randomly divide the high-resolution cell image set I into the training set I according to the ratio of 8:1:1. t and validation set I v and test set For training set I tThe i-th image in, i∈(1,…,m), m is the training set I t The number of images in For the validation set I v The i-th image in , i∈(1,…,u), u is the number of images in the validation set, For the test set I w The i-th image in , i∈(1,…,w), w is the number of images in the validation set, w=124;

[0079] c. The training set I t Individual high-resolution cell images and validation set I v Individual high-resolution cell images and test set I w Each high-resolution cell image in is preprocessed to obtain the preprocessed training set h t And the preprocessed validation set is the preprocessed training set h t The i-th preprocessed cell image in is the preprocessed validation set h v The i-th preprocessed cell image in ;

[0080] The image is preprocessed for data augmentation and data filtering. Data filtering includes extracting the R channel and G channel images of the image. Data augmentation includes performing the following operations on the R channel and G channel images: random cropping, horizontal flipping, vertical flipping, random rotation, and changing the image attributes, where the image attributes include brightness, contrast, saturation or hue.

[0081] (2) Construct a yolov8-SCC deep neural network model, input the preprocessed cell image dataset into the yolov8-SCC deep neural network model for training, and obtain a trained yolov8-SCC deep neural network model.

[0082] The yolov8-SCC deep neural network model is improved based on the original YOLO v8. The improvement process is as follows:

[0083] A feature extraction network based on YOLO v8 is constructed, consisting of three components: the Backbone network, the Neck network, and the Head network. The Backbone network is a convolutional network used for feature extraction, responsible for extracting multi-level, multi-scale features from the input image and providing them to subsequent processing. The Neck network is used to further fuse and process the features extracted by the Backbone network. This multi-scale feature fusion enables the model to better detect objects of different sizes. The Head network is responsible for classifying and localizing objects based on the fused features, and outputs include the object category, bounding box coordinates, and confidence level.

[0084] Figure 2 This is one of the structural diagrams of the YOLOv8-SCC model provided in an embodiment of the present application, where conv is a convolution module, C2f is a C2f (Crossover Two-level Fusion) module in YOLO v8, Concat is a Concat module in YOLO v8, and CE is a CE module.

[0085] To preserve the spatial resolution of the input image, the stride of the convolutional modules in the Backbone network is first reduced to 1, preventing feature map downsampling and reducing the number of downsampling cycles required for the model. While reducing the size of the feature maps, the spatial resolution of the input data is preserved, minimizing the loss of detail during downsampling for small objects, which occupy fewer pixels in the image. The Backbone module consists of five Conv modules, four C2f modules, and one SPFF module.

[0086] After each feature layer of the Backbone network is generated, the feature layer is input into the SPD (Space-to-Depth) module to improve the detection ability of small targets in each feature layer. Each feature layer includes Figure 2 The first characteristic layer P1, the second characteristic layer P2, the third characteristic layer P3, and the fourth characteristic layer P4. Figure 3 This is the working mechanism of the SPD module provided in the embodiment of the present application, such as Figure 3As shown, the SPD module evenly divides the feature maps input by the first feature layer P1, the second feature layer P2, the third feature layer P3, and the fourth feature layer P4, so that the feature maps extract features evenly and equidistantly in height and width. The features extracted in each direction are divided into two channels, thereby halving the height and width of the feature map while increasing the number of channels fourfold. The feature map is then divided into small blocks after being classified into each channel and then combined together to form a new feature map with more channels. This converts the spatial dimensions (height and width) of the input feature map into the depth dimension (number of channels), downsampling the feature map while retaining all information in the channel dimension, so no information is lost. The SPD module of the improved backbone network can better retain the features of small targets, thereby increasing the receptive field of the model while maintaining spatial resolution.

[0087] In the Backbone network, the C2f module splits the feature map output by the SPD module into two feature maps, X0 and Y0, along channel C. X0 undergoes no processing, while Y0 undergoes a series of convolutions and skip connections. Ultimately, it is concatenated with X0 and the Y series of feature maps processed through multiple convolutions and skip connections, and then a 1×1 convolution is performed to obtain the feature map output by the C2f module. The C2f module enhances the expressiveness of features through convolutions and skip connections. Through the segmentation of the SPD module and the convolution processing of the C2f module, the model can better capture and utilize important information in the feature map.

[0088] In order to avoid channel dimensionality reduction and realize cross-channel interaction in channel attention learning, CE modules are introduced into the feature pyramid network and path aggregation network in the Neck network respectively. Figure 4 This is a schematic diagram of the structure of the CE module provided in the embodiment of the present application. Figure 4 As shown in Figure 1, an ECA attention mechanism module is added between the convolution modules in the CE module of the Neck network.

[0089] It should be noted that the process of the C2f module processing the feature map is as follows:

[0090] like Figure 2 As shown, the feature map Input into the C2f module and transform the feature map T along the direction of channel C 11 Divided into 2 feature maps and feature Figure X 0 Without any processing, the feature map Y0 is convolved twice with a convolution kernel of size 3×3 to obtain the feature map Then perform a jump connection with the feature map Y0 to obtain the feature map The feature map Y1 is convolved twice with a convolution kernel of size 3×3 to obtain the feature map Then perform a jump connection with the feature map Y1 to obtain the feature map The feature map Y2 is convolved twice with a convolution kernel of size 3×3 to obtain the feature map Then perform a jump connection with the feature map Y2 to obtain the feature map After n times, the feature map is obtained Then the features Figure X 0 and Y1 to Y n After splicing along the channel direction, a 1×1 convolution operation is performed to obtain the feature map The specific formula is as follows:

[0091]

[0092] Among them, T1 is divided into two parts, X0 and Y0, conv 3×3 For the convolution operation with a convolution kernel size of 3×3, conv 1×1 is a convolution operation with a convolution kernel size of 1×1, and Concat is the feature map channel C obtained. out Directional dimension stitching.

[0093] The process of SPFF module processing feature map is as follows:

[0094] like Figure 2 As shown, the feature map is input into the SPFF module and passed through a 3×3 convolution kernel W2 to obtain the feature map S1 and perform maximum pooling to obtain the feature map S 11 Perform maximum pooling to obtain S 12 After performing a maximum pooling, the feature map is obtained After three times of maximum pooling, three feature maps of different resolutions are obtained. These three feature maps are spliced ​​with the feature map along the channel direction and then a 1×1 convolution operation is performed to obtain the feature map. The specific formula is as follows:

[0095]

[0096] Among them, conv 3×3 To perform convolution operation on the feature map, MaxPool2d is the maximum pooling operation, conv 1×1 It is a convolution operation with a convolution kernel size of 1×1.

[0097] The process of SPD module processing feature map is as follows:

[0098] Feature blocks are extracted from the input feature map by different starting positions and step sizes, and the feature map is divided into sub-feature sequence blocks F through the SPD module. n,m, where n represents the horizontal coordinate and m represents the vertical coordinate;

[0099] F 0,0 =F[0:H:step,0:W:step],F 1,0

[0100] =F[1:H:step,0:W:step],...,F step-1,0

[0101] =F[step-1:H:step,0:W:step]

[0102] F 0,step-1 =F[0:H:step, step-1:W:step], F 1,step-1

[0103] =F[1:H:step,step-1:W:step],...,F step-1,step-1

[0104] =F[step-1:H:step, step-1:W:step]

[0105] Among them, F 0,0 To extract the feature block from the feature map F(0,0) coordinate, that is, the upper left corner of the feature map, with a step size of step=2, H and W are the height and width of the input feature map.

[0106] These sub-feature sequences are separated along channel C out direction to obtain an image of size feature map.

[0107] It should be noted that the SPD module is introduced into the path aggregation network in the Neck network to transfer features layer by layer from the bottom feature layer to the top (i.e. Figure 2 When transferring the second feature layer P2 to the fourth feature layer P4 in the image processing, the features of small targets in the underlying feature layer are retained to increase the receptive field of the model while maintaining the spatial resolution.

[0108] It is worth noting that setting the MPD (Mean Pairwise Distance Loss) loss function to replace the original loss function solves the problem of limiting convergence speed and accuracy when the predicted box and the true box have the same aspect ratio but different actual width and height by minimizing the vertex distance between the predicted box and the true box, thereby achieving better detection results.

[0109] By improving the original YOLOv8 model described above, we developed the YOLOv8-SCC deep neural network model to enhance model detection accuracy. The training set was fed into the YOLOv8-SCC deep neural network model, and training batches were set for multiple training runs. The model then underwent supervised learning based on labeled data from the validation set, adjusting parameters accordingly. The initial weights of the YOLOv8-SCC model were automatically adjusted based on model training. After model training, the weights were transferred to the YOLOv8-SCC model prediction module, where the model's image recognition accuracy, recognition time, and real-time performance were analyzed and compared.

[0110] Then, through the following steps, the collected and processed image data is used as model input data to analyze the fluorescence in situ hybridization image results:

[0111] (1) The R channel original image of the cell image to be detected is used as input and input into the yolov8-SCC deep neural network model for detection to obtain the red fluorescent probe distribution map;

[0112] (2) The G channel original image of the cell image to be detected is used as input and input into the yolov8-SCC deep neural network model for detection to obtain the green fluorescent probe distribution map;

[0113] (3) The obtained red fluorescence distribution map and green fluorescence distribution map are combined into one map to obtain the relationship map between cell and probe distribution.

[0114] Figure 5 This is a comparison chart of the results of the yolov8-SCC model provided in the embodiment of this application and other models. After the cell image to be detected is processed by the above (1)-(3), Figure 5 The detection result shown in (b) can identify the distribution of fluorescent probes. The same cell image to be detected cannot be identified by the Unet++ image segmentation model, as shown in Figure 5 As shown in (a) in the figure, the same cell image to be detected can only identify the distribution of a very small number of fluorescent probes after YOLOv8 processing, such as Figure 5 As shown in (d) in the figure, the same cell image to be detected can only identify the distribution of a very small number of fluorescent probes after YOLOv5 processing, such as Figure 5 As shown in (e) in the figure, the same cell image to be detected can only identify the distribution of a very small number of fluorescent probes after YOLOv3 processing, such as Figure 5 As shown in (f) in the figure, the same cell image to be detected can only identify the distribution of a very small number of fluorescent probes after being processed by YOLOv8P2, such as Figure 5 As shown in (c), the number of fluorescent probes identified is 20% to 40% less than that of the method in (b).

[0115] Figure 6 This is the second flow chart of the probe distribution target detection method based on the multi-channel neural network model provided in the embodiment of the present application, such as Figure 6 As shown, the green fluorescent probe distribution map and the red fluorescent probe distribution map are merged to obtain a relationship map between the cells to be tested and the probe distribution.

[0116] According to the probe distribution target detection method based on a multi-channel neural network model provided in an embodiment of the present application, by obtaining the G channel and R channel data of the cell to be tested and inputting them into the trained YOLOv8-SCC model respectively, green and red fluorescent probe distribution maps are obtained. By utilizing the collaborative work of the SPFF module, CE module and SPD module in the YOLOv8-SCC model, the detection accuracy of the cell probe distribution is effectively improved, and the distribution of probes inside the cell can be more accurately displayed, assisting further biological analysis, and improving cell detection accuracy and processing speed. It is suitable for complex image scenes, realizes the analysis of various medical images, improves the efficiency of medical diagnosis, reduces human errors, and enhances cell analysis and pathology detection capabilities in scientific research and clinical applications.

[0117] In some embodiments, inputting the G channel data into a trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map includes:

[0118] Input the G channel data into the Backbone network, obtain a first feature map through the SPFF module, and obtain a second feature map, a third feature map, and a fourth feature map through the second SPD module, the third SPD module, and the fourth SPD module of the Backbone network;

[0119] Inputting the first feature map, the second feature map, the third feature map, and the fourth feature map into a feature pyramid network, obtaining a fifth feature map and a sixth feature map through a first C2f module and a second C2f module of the feature pyramid network, and obtaining a seventh feature map through a first CE module of the feature pyramid network;

[0120] Inputting the first, fifth, sixth, and seventh feature maps into a path aggregation network, obtaining an eighth and ninth feature maps through a third and fourth C2f modules of the path aggregation network, and obtaining a tenth feature map through a second CE module of the path aggregation network;

[0121] The fifth feature map, the eighth feature map, the ninth feature map and the tenth feature map are input into the Head network to obtain a green fluorescent probe distribution map.

[0122] In some embodiments, as Figure 2As shown in the figure, the yolov8-SCC deep neural network model is built using YOLOv8 as the basic network, reducing the stride of the convolution module of the YOLOv8 Backbone network to 1. The Backbone network includes 5 Conv modules at layers 0, 1, 4, 7, and 10, 4 SPD modules at layers 2, 5, 8, and 11, 4 C2f modules at layers 3, 6, 9, and 12, and 1 SPFF module at layer 13.

[0123] The outputs of layers 2, 5, and 8 of the Backbone network are fed into the Concat module of the feature pyramid network of the Neck network of YOLOv8 to further fuse feature information. The CE module is introduced into the feature pyramid network and path aggregation network of the Neck network of YOLOv8, respectively. The ECA attention mechanism module is added between the convolution modules in the CE module. The ECA attention mechanism module first performs global average pooling on the output of the convolution layer in the CE module, compressing the spatial information of each channel into a single value, thereby obtaining a feature vector with the same number of channels. Then, the compressed feature vector is processed using one-dimensional convolution to capture the dependencies between channels.

[0124] Finally, the channel attention weight is generated through the Sigmoid activation function; the SPD module is introduced in the path aggregation network in the Neck network. The SPD module is used to retain the features of small targets in the bottom feature layer when transferring features from the bottom feature layer to the top layer, so as to increase the receptive field of the model while maintaining spatial resolution; the top layer of the feature pyramid network of the Neck network and the feature maps output by the CE module and C2f module in the path aggregation network are input into the Detect layer of the Head network. The Head network is used to convert the feature maps provided by the Neck network into the final detection results, including the coordinates of the bounding box, the confidence of the bounding box, and the predicted probability of each category.

[0125] Figure 7 This is the second structural diagram of the YOLOv8-SCC model provided in the embodiment of the present application, such as Figure 7As shown in the figure, the Backbone network of the YOLOv8-SCC model extracts features from the training set and implements downsampling of the feature map, specifically including: resizing the image data in the training set to W×H×C, performing two convolution operations to obtain the feature layer F1; inputting the feature layer F1 into the SPD module to obtain the sub-feature sequence block F(n, m), and then splicing to obtain the feature layer F2 (W / 2×H / 2×4Cout); inputting the feature layer F2 into the C2f module, dividing it into X0 and Y0, and Y0 undergoes multiple 3×3 convolutions and jump connections to obtain Yn, and finally splicing it with X0, and obtaining the feature layer F3 (W / 2×H / 2×4Cout) through 1×1 convolution. After convolution of the feature layer F3, the second feature layer P2 (W / 2×H / 2×4Cout) is obtained; the second feature layer P2 is input into the SPD module to obtain the feature layer F4; F4 is input to the C2f module to obtain the feature layer F5 (W / 4×H / 4×8Cout). After convolution of the feature layer F5, the third feature layer P3 (W / 4×H / 4×8Cout) is obtained. The third feature layer P3 is input to the SPD module to obtain the feature layer F6. The feature layer F6 is input to the C2f module to obtain the feature layer F7 (W / 8×H / 8×16Cout). After convolution of the feature layer F7, the fourth feature layer P4 (W / 8×H / 8×16Cout) is obtained. The fourth feature layer P4 is input to the SPD module to obtain the feature layer F8. The feature layer F8 is input to the C2f module to obtain the feature layer F9 (W / 16×H / 16×32Cout). The feature layer F9 is passed through the SPFF module, through 3×3 convolution and multiple maximum pooling, to obtain the updated fourth feature layer P4' (W / 16×H / 16×32Cout).

[0126] Depend on Figure 2It can be seen that the three major branches of the Backbone network output part output three scale feature maps, which correspond to P1, P2, P3, and P4' of the Backbone network output mentioned above from bottom to top, and the scales are stacked in a pyramid shape from large to small. In the Neck network, the first step is to start the top-down path fusion from the top feature layer P4', upsampling layer by layer, and fusion of the upsampled features of each layer with the corresponding low-level features to supplement spatial information and enhance semantic information. During horizontal fusion, 1X1 convolution is used to adjust the number of channels so that the upsampled features are consistent with the number of channels of the low-level features. Feature fusion is performed by layer-by-layer addition. That is, the upsampled features of the fourth feature layer P4' are upsampled and fused with the feature layer F6. The feature map T3 (W / 8×H / 8×16Cout) is obtained through 1×1 convolution. After the feature map T3 is processed by the C2f module, the features are upsampled and fused with the feature layer F4. The feature map T2 (W / 4×H / 4×8Cout) is obtained through 1×1 convolution. The feature map T2 passes through the CE module, and in the CE module, it passes through the ECA module and multiple convolutions. The upsampled features are then fused with the feature layer F2 to obtain the feature map T1 (W / 2×H / 2×4Cout). The second step starts with the bottom-up path fusion from the bottom-up feature. The feature map of each layer is fused with the high-level feature map in the top-down path through the bottom-up path, and the features are passed upward layer by layer to ensure that the features of each layer contain information of different scales. That is, the feature map T1 is processed by the C2f module to obtain the feature map D1 (W / 2×H / 2×4Cout). The feature map D1 is processed by the 3×3 convolution operation and the SPD module to obtain the feature, and then fused with the feature map T2 to obtain the feature map M2. The feature map M2 is processed by the C2f module to obtain the feature map D2 (W / 8×H / 8×8Cout). Feature map D2 is processed by a 3×3 convolution operation and the SPD module to obtain features, which are then fused to obtain feature map M3. Feature map M3 is processed by the CE module and weightedly combined with feature map T3 to obtain feature map D3 (W / 8×H / 8×16Cout). Feature map D3 is processed by a 3×3 convolution operation and the SPD module to obtain features, which are then fused with the fourth feature layer P4' to obtain feature map M4. Feature map M4 is processed by the C2f module to obtain feature map D4 (W / 16×H / 16×32Cout). Feature maps D1, D2, D3, and D4 are sent to the Detect head for detection.

[0127] The feature maps D1, D2, D3, and D4 are input to the Detect layer, which contains two branches: the regression branch and the classification branch. The regression branch outputs the position and size of the bounding box, and the classification branch outputs the category and confidence to determine whether the predicted box is a fluorescent point or background. The feature maps D1, D2, D3, and D4 are sent to the Detect head for detection, and the sizes can be obtained respectively. The bounding box position and size (usually four parameters of the bounding box are output), category and confidence (confidence is the probability that the bounding box contains the target and the accuracy of the prediction).

[0128] In this embodiment, the G channel data is input into the trained YOLOv8-SCC deep neural network model and multi-level processing is performed. The first feature map is obtained by the SPFF module in the Backbone network, and the second, third, and fourth feature maps are extracted by continuous SPD modules. These feature maps are input into the feature pyramid network, and high-level features are further extracted by multi-layer C2f modules and CE modules to obtain the fifth to seventh feature maps, which further enhances the expression of information and improves the multi-scale feature learning ability of the network. After optimization processing by the path aggregation network, the eighth to tenth feature maps are obtained, and finally the green fluorescent probe distribution map is generated by the Head network, which effectively improves the perception ability of the yolov8-SCC deep neural network model of probe distributions of different scales, enhances the ability to capture detail information, and can effectively improve the feature extraction ability of the model.

[0129] In some embodiments, the convolution module of the first CE module includes an ECA attention mechanism module, which performs global average pooling on the output of the convolution layer in the first CE module, compresses the spatial information of each channel into a single value, and obtains a feature vector with the same number of channels;

[0130] The feature vector is processed by one-dimensional convolution and then input into the Sigmoid activation function to obtain the channel attention weight.

[0131] Figure 8 This is a schematic diagram of the structure of the ECA module provided in the embodiment of the present application. Figure 8As shown in the figure, the ECA attention mechanism module first inputs a 3D feature map of size H×W×C, where H is the height, W is the width, and C is the number of channels. It then performs global average pooling on each channel of the input feature map, compressing the spatial information into a single value. This step aims to transform the spatial information of each channel into a global feature, resulting in a 1×1×C feature vector with the same number of channels as the input 3D feature map. Global average pooling reduces the continuous feature dimensions by calculating the mean of the feature map, thereby compressing the spatial information into a single output. This is an effective method for reducing overfitting and extracting features. The compressed feature vector is then processed using a one-dimensional convolution. The kernel size k of this one-dimensional convolution is adaptively selected to determine the coverage of local cross-channel interactions. The purpose of the one-dimensional convolution is to capture inter-channel dependencies. In this way, the ECA module can effectively capture inter-channel dependencies, thereby enabling cross-channel interactions in channel attention learning, significantly improving model performance with minimal additional computation. Finally, the output of the one-dimensional convolution generates a 1×1×C channel attention weight through the Sigmoid activation function. These weights represent the importance of each channel. The higher the weight, the more attention the channel should be given in the subsequent feature map. These channel attention weights are then multiplied channel by channel with the original input feature map to achieve a weighted combination of different channels. After channel weighting, the final output feature map is obtained, whose size is still H×W×C, but the features of each channel have been reweighted to improve the model's ability to capture key features.

[0132] It is worth noting that in the ECA module, the kernel size of the one-dimensional convolution is adaptive, which means that it can be adjusted according to the number of channels of the input feature map to determine the coverage of local cross-channel interactions. This adaptive kernel size allows the model to establish connections between different channels, thereby learning which channels should be given higher weights. This cross-channel interaction enables the model to focus more on the features that are most important to the current task, rather than simply treating all channels equally.

[0133] In this embodiment, by adding an ECA attention mechanism module to the CE module and using one-dimensional convolution to process the feature vector after global average pooling, the dimensionality reduction is reduced and more information is retained. The one-dimensional convolution adaptively adjusts the channels of the feature map so that the model can pay more attention to the most important features of the current task, realizes cross-channel interaction in channel attention learning, and can significantly improve the performance of the model with very little additional computational effort, focusing more on the display of small targets, thereby improving the accuracy and robustness of the yolov8-SCC deep neural network model detection.

[0134] In some embodiments, the step of inputting the first feature map, the second feature map, the third feature map, and the fourth feature map into a feature pyramid network, obtaining a fifth feature map and a sixth feature map through a first C2f module and a second C2f module of the feature pyramid network, and obtaining a seventh feature map through a first CE module of the feature pyramid network includes:

[0135] Upsampling the first feature map and fusing it with the fourth feature map, extracting features through a first C2f module to obtain a fifth feature map;

[0136] Upsampling the fifth feature map and fusing it with the third feature map, and passing it through the first CE module to obtain a seventh feature map;

[0137] The seventh feature map is upsampled and fused with the second feature map, and features are extracted by the second C2f module to obtain the sixth feature map.

[0138] by Figure 2 For example, in the feature pyramid network of the Neck network, the feature map output by the SPFF module is upsampled and fused with the feature map output by the 8th layer of the Backbone network. After a C2f module extracts features, the 16th layer feature map is obtained, which is upsampled again and fused with the feature map output by the 5th layer of the Backbone network. After a CE module increases the channel attention weight, the 19th layer feature map is obtained, which is upsampled again and fused with the feature map output by the 2nd layer of the Backbone network. After a C2f module extracts features, the 22nd layer feature map is obtained and output to the path aggregation network of the Neck network and the Detect layer of the Head network. The feature maps output by the 16th, 19th layers and SPFF modules are also output to the path aggregation network of the Neck network.

[0139] In this embodiment, by fusing the first feature map and the fourth feature map and extracting features through the first C2f module, the fifth feature map is obtained, which enhances the model's ability to jointly learn low-level and high-level features. By fusing the fifth feature map with the third feature map and processing it through the first CE module, useful channel features are further extracted to obtain the seventh feature map. By adding the CE module to the feature pyramid network, the feature learning process of the yolov8-SCC deep neural network model is optimized, the discrimination of complex fluorescence image features is enhanced, the detection of probe distribution is more accurate, and the detection capability of the yolov8-SCC deep neural network model for targets in multi-scale images is effectively improved.

[0140] In some embodiments, the inputting the first feature map, the fifth feature map, the sixth feature map, and the seventh feature map into the path aggregation network, obtaining the eighth feature map and the ninth feature map through the third C2f module and the fourth C2f module of the path aggregation network, and obtaining the tenth feature map through the second CE module of the path aggregation network includes:

[0141] The sixth feature map is processed by the convolution operation and the fifth SPD module of the path aggregation network and then fused with the seventh feature map. After extracting features through the third C2f, the eighth feature map is obtained.

[0142] The eighth feature map is processed by a convolution operation and a sixth SPD module of a path aggregation network, and then fused with the fifth feature map, and the tenth feature map is obtained by passing it through a second CE module;

[0143] The tenth feature map is processed by the convolution operation and the seventh SPD module of the path aggregation network and then fused with the first feature map, and then passed through the fourth C2f module to obtain the ninth feature map.

[0144] by Figure 2 For example, the 22nd layer feature map is processed by the convolution operation and the SPD module and then fused with the 18th layer feature map. After the features are extracted by the C2f module once, the 26th layer feature map is obtained. After the convolution operation and the SPD module, it is fused with the 16th layer feature map to obtain the 30th layer feature map. After the convolution operation and the SPD module, it is fused with the feature map output by the SPFF module. After the features are extracted by the C2f module once, the 34th layer feature map is obtained. The feature maps of the 22nd, 26th, 30th, and 34th layers are output to the Detect layer of the Head network.

[0145] In this embodiment, by introducing the CE module and the SPD module into the path aggregation network, the CE module can adaptively enhance the features, effectively improving the sensitivity of the yolov8-SCC deep neural network model to key features, thereby improving the accuracy of target detection in complex scenarios. The SPD module further enhances the yolov8-SCC deep neural network model's detection capability for targets of different scales through multi-scale spatial feature processing, especially when processing small targets and edge targets. By combining the CE module and the SPD module, the yolov8-SCC deep neural network model can perform feature fusion and information extraction at a higher level, improve cell detection accuracy and processing speed, effectively process complex image scenes, and realize the analysis of various medical images.

[0146] In some embodiments, the eighth feature map is processed by a convolution operation and a sixth SPD module of a path aggregation network and then fused with the fifth feature map, and a tenth feature map is obtained by a second CE module, including:

[0147] The fused feature map is input into the second CE module to obtain the first sub-feature map and the second sub-feature map;

[0148] The second sub-feature map is convolved once and then input into the ECA attention mechanism module, and the third sub-feature map is obtained through a global average pooling.

[0149] Calculating the convolution kernel size of the third sub-feature map based on the inverse mapping function;

[0150] Convolving the third sub-feature map to obtain a fourth sub-feature map;

[0151] After performing a convolution operation on the fourth sub-feature map with a convolution kernel size of a preset value, a fifth sub-feature map is obtained;

[0152] The fifth sub-feature map and the second sub-feature map are skip-connected to obtain a sixth sub-feature map;

[0153] The sixth sub-feature map is convolved once and then input into the ECA attention mechanism module, and the above operation is repeated N times to obtain the Nth sub-feature map;

[0154] After concatenating the first sub-feature map, the sixth sub-feature map, and the Nth sub-feature map, a tenth feature map is obtained through convolution.

[0155] The process of CE module processing feature map is as follows:

[0156] (1) Figure 2 The feature map T2 or T3 in the input is input to the CE module to divide the feature map into two feature maps and S0 is not processed in any way, and f0 is input into the ECA module after a convolution operation.

[0157] (2) f0 first undergoes a global average pooling to obtain f 01 (1×1×4C out );

[0158] (3) Then f 01 Perform a one-dimensional convolution, the size of the one-dimensional convolution kernel is adaptively determined according to the number of channels C, and use the inverse mapping function To infer the appropriate convolution kernel size K from the number of channels C;

[0159] Among them, the nonlinear function is C = φ(k) = 2 γ×K-b , K is the size of the one-dimensional convolution kernel, C is the number of channels, φ is the mapping function, b is the offset in the nonlinear function, γ is the scale coefficient in the nonlinear function, ψ is the inverse mapping function, |t| oddis the odd number closest to t;

[0160] (4) After obtaining the convolution kernel size, f 01 Perform convolution to obtain

[0161] (5) Then f 02 Perform a convolution operation with a convolution kernel size of 3×3 to obtain

[0162] (6)f 03 Then make a jump connection with f0 to get

[0163] (7) After another convolution operation, f1 is input into the ECA module and the above steps (1) to (6) are repeated until the result is

[0164] (8) Then transform the feature map S0 and f1 to f n Along Channel C out After the directions are spliced, a 1×1 convolution operation is performed to the feature map The specific formula is as follows:

[0165]

[0166] Among them, T1 is divided into two parts, S0 and f0, conv 3×3 is a convolution operation with a convolution kernel size of 3×3, conv is a one-dimensional convolution operation, conv 1×1 is a convolution operation with a convolution kernel size of 1×1, and Concat is the feature map f0 to f n Along the channel direction C out Perform splicing.

[0167] f1 repeats the above steps (1) to (6) to obtain the feature map The process is as follows:

[0168] f1 obtains the feature map after a global average pooling Then the feature map f 11 Perform a one-dimensional convolution to get Then a convolution operation with a convolution kernel size of 3×3 is performed to obtain the feature map Then f 13 Perform a jump connection with the feature map f1 to obtain the feature map f2 undergoes another convolution operation and is input into the ECA module, and a global average pooling operation is performed to obtain the feature map. Then the feature map f 21 Perform a one-dimensional convolution to get Then a convolution operation with a convolution kernel size of 3×3 is performed to obtain the feature map Then f 23 Perform a jump connection with the feature map f2 to obtain the feature map After n times, the feature map is obtained

[0169] In this embodiment, by adding a CE module to the feature pyramid network, the feature learning process of the yolov8-SCC deep neural network model is optimized, the discrimination of complex fluorescence image features is enhanced, the detection of probe distribution is made more accurate, and the detection capability of the yolov8-SCC deep neural network model in multi-scale images is effectively improved. Through layer-by-layer feature fusion and refined upsampling processing, the yolov8-SCC deep neural network model can more accurately capture targets of different sizes and positions in cell images, improving the accuracy and robustness of cell detection.

[0170] In some embodiments, the Head network includes multiple Detect layers, each Detect layer includes a regression branch and a classification branch, the regression branch is used to output the position and size of the bounding box, and the classification branch is used to output the category and confidence.

[0171] It should be noted that the Detect layer contains regression branches and classification branches. The regression branch outputs the position and size of the bounding box, and the classification branch outputs the category and confidence to determine whether the predicted box is a fluorescent point or background. The size can be obtained as After obtaining the category, confidence and bounding box coordinates of the classification branch output of different sizes and the regression branch output, the confidence of each bounding box is sorted in descending order, and the bounding box with the highest confidence is selected from the sorted list, marked as selected, and added to the final detection result list. The intersection-over-union ratio of each remaining bounding box to the selected bounding box is calculated, and the non-maximum suppression method is used to filter out some bounding boxes. The remaining bounding boxes are the retained high-confidence bounding boxes.

[0172] After obtaining the categories, confidences, and bounding box coordinates of the classification branches of different sizes and the regression branches, sort them in descending order according to the confidence of each bounding box. Select the bounding box with the highest confidence from the sorted list, mark it as selected, and add it to the final detection result list. For each remaining bounding box, if the intersection-of-union ratio with the selected bounding box is greater than the set threshold, such as 0.7, the low-confidence bounding box is discarded because the two boxes are considered to be the same target. Otherwise, the low-confidence bounding box is retained.

[0173] Continue to select the low-confidence bounding box with the highest confidence among the remaining bounding boxes, repeatedly calculate the intersection-over-union ratio of the low-confidence bounding box with the selected bounding box, and repeat the elimination process until all remaining bounding boxes have been checked;

[0174] The final output bounding box list contains only the bounding boxes that have been processed by non-maximum suppression. There will not be too much overlap between them. The calculation formula of the intersection-over-union ratio is as follows:

[0175] The coordinates of the upper left corner and lower right corner of the prediction box and the true label box are and MPDIoU is a loss function based on the intersection-over-union ratio. The calculation formula of MPDIoU is as follows;

[0176]

[0177] Among them, IoU is the intersection over union ratio of the predicted box and the true label box, d1 and d2 are the Euclidean distances between the upper left corner and lower right corner of the predicted box and the true box, respectively, and h and w are the height and width of the image.

[0178] MPDIoU is used to calculate the minimum point distance intersection-over-union (IoU) between the predicted bounding box and the ground-truth bounding box, focusing on minimizing the distances to the top-left and bottom-right corners of the predicted and ground-truth bounding boxes. The MPDIoU calculation takes into account the overlap (i.e., IoU), the center point distance, and the width and height deviations of the two bounding boxes. This approach directly minimizes the top-left and bottom-right distances between the predicted and ground-truth bounding boxes, providing a more comprehensive and accurate measure of bounding box similarity.

[0179] L MPDIoU It is a loss function based on MPDIoU, which is used to optimize the model during training so that the predicted bounding box matches the real bounding box more accurately. The calculation formula of the MPD loss function is as follows:

[0180] The coordinates of the upper left and lower right corners are and The width and height of the image are H and W,

[0181]

[0182] L MPDIoU =1-MPDIoU

[0183] Among them, H and W represent the height and width of the image respectively, L MPDIoU is the MPD loss function, and MPDIoU is the MPD intersection-over-union ratio.

[0184] In this embodiment, by introducing multiple Detect layers in the Head network, each Detect layer includes a regression branch and a classification branch. The regression branch outputs the position and size of the bounding box, and the classification branch outputs the category and confidence. This effectively improves the target detection accuracy. Through multi-level feature extraction and classification regression optimization, it can more accurately locate the target and identify its category, thereby improving the robustness and accuracy of target detection, realizing the analysis of various medical images, improving the efficiency of medical diagnosis, reducing human errors, and enhancing the cell analysis and pathology detection capabilities in scientific research and clinical applications.

[0185] The probe distribution target detection method based on a multi-channel neural network model provided in the embodiments of the present application can be executed by a probe distribution target detection device based on a multi-channel neural network model. In the embodiments of the present application, the probe distribution target detection method based on a multi-channel neural network model is executed by a probe distribution target detection device based on a multi-channel neural network model as an example to illustrate the probe distribution target detection device based on a multi-channel neural network model provided in the embodiments of the present application.

[0186] The present application also provides a probe distribution target detection device based on a multi-channel neural network model. Figure 9 As shown, the probe distribution target detection device based on the multi-channel neural network model includes: an acquisition module 910, a first processing module 920, a second processing module 930, a third processing module 940 and an output module 950.

[0187] An acquisition module 910 is used to acquire G channel data and R channel data of a fluorescence image of a cell to be tested;

[0188] Processing module 920, used to input the G channel data into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, and input the R channel data into the trained yolov8-SCC deep neural network model to obtain a red fluorescent probe distribution map;

[0189] A merging module 930 is configured to merge the green fluorescent probe distribution map and the red fluorescent probe distribution map to obtain a relationship map between cells to be tested and probe distributions;

[0190] Among them, the yolov8-SCC deep neural network model includes a Backbone network, a Neck network and a Head network. The Backbone network includes an SPFF module, the Neck network includes a feature pyramid network and a path aggregation network, the feature pyramid network includes a first CE module, and the path aggregation network includes a second CE module and a first SPD module.

[0191] According to the probe distribution target detection method based on a multi-channel neural network model provided in an embodiment of the present application, by obtaining the G channel and R channel data of the cell to be tested and inputting them into the trained YOLOv8-SCC model respectively, green and red fluorescent probe distribution maps are obtained. By utilizing the collaborative work of the SPFF module, CE module and SPD module in the YOLOv8-SCC model, the detection accuracy of the cell probe distribution is effectively improved, and the distribution of probes inside the cell can be more accurately displayed, assisting further biological analysis, and improving cell detection accuracy and processing speed. It is suitable for complex image scenes, realizes the analysis of various medical images, improves the efficiency of medical diagnosis, reduces human errors, and enhances cell analysis and pathology detection capabilities in scientific research and clinical applications.

[0192] The probe distribution target detection device based on the multi-channel neural network model provided in the embodiment of the present application can achieve Figures 1 to 8 To reduce repetition, the various processes implemented in the embodiment of the probe distribution target detection method based on the multi-channel neural network model are not repeated here.

[0193] In some embodiments, as Figure 10 As shown, an embodiment of the present application also provides an electronic device 1000, including a processor 1001, a memory 1002, and a computer program stored in the memory 1002 and executable on the processor 1001. When the program is executed by the processor 1001, each process of the embodiment of the probe distribution target detection method based on the multi-channel neural network model is implemented, and the same technical effect can be achieved. To reduce repetition, it will not be repeated here.

[0194] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0195] An embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various processes of the above-mentioned probe distribution target detection method embodiment based on the multi-channel neural network model and can achieve the same technical effect. To reduce repetition, it will not be repeated here.

[0196] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0197] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned probe distribution target detection method based on the multi-channel neural network model.

[0198] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0199] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, which are coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the probe distribution target detection method based on the multi-channel neural network model, and can achieve the same technical effect. To reduce repetition, it will not be repeated here.

[0200] It should be understood that the chip mentioned in the embodiments of the present application can also be called a device-level chip, a device chip, a chip device, or an on-chip device chip, etc.

[0201] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0202] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD-ROM), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the probe distribution target detection method based on the multi-channel neural network model of each embodiment of the present application.

[0203] In the description of this application, "first feature" and "second feature" may include one or more such features.

[0204] In the description of this application, “plurality” means two or more.

[0205] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0206] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0207] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A probe distribution target detection method based on a multi-channel neural network model, characterized in that: The method comprises: Obtain G channel data and R channel data of the fluorescence image of the cell to be tested; The G channel data is input into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, and the R channel data is input into the trained yolov8-SCC deep neural network model to obtain a red fluorescent probe distribution map; Merging the green fluorescent probe distribution map and the red fluorescent probe distribution map to obtain a relationship map between the cells to be tested and the probe distribution; Among them, the yolov8-SCC deep neural network model includes a Backbone network, a Neck network and a Head network. The Backbone network includes an SPFF module, the Neck network includes a feature pyramid network and a path aggregation network, the feature pyramid network includes a first CE module, and the path aggregation network includes a second CE module and a first SPD module.

2. The probe distribution target detection method based on a multi-channel neural network model according to claim 1, characterized in that: The G channel data is input into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, including: Input the G channel data into the Backbone network, obtain a first feature map through the SPFF module, and obtain a second feature map, a third feature map, and a fourth feature map through the second SPD module, the third SPD module, and the fourth SPD module of the Backbone network; Inputting the first feature map, the second feature map, the third feature map, and the fourth feature map into a feature pyramid network, obtaining a fifth feature map and a sixth feature map through a first C2f module and a second C2f module of the feature pyramid network, and obtaining a seventh feature map through a first CE module of the feature pyramid network; Inputting the first, fifth, sixth, and seventh feature maps into a path aggregation network, obtaining an eighth and ninth feature maps through a third and fourth C2f modules of the path aggregation network, and obtaining a tenth feature map through a second CE module of the path aggregation network; The fifth feature map, the eighth feature map, the ninth feature map and the tenth feature map are input into the Head network to obtain a green fluorescent probe distribution map.

3. The probe distribution target detection method based on a multi-channel neural network model according to claim 1, characterized in that: The convolution module of the first CE module includes an ECA attention mechanism module, which performs global average pooling on the output of the convolution layer in the first CE module, compresses the spatial information of each channel into a single value, and obtains a feature vector with the same number of channels; The feature vector is processed by one-dimensional convolution and then input into the Sigmoid activation function to obtain the channel attention weight.

4. The probe distribution target detection method based on a multi-channel neural network model according to claim 2, characterized in that: The step of inputting the first feature map, the second feature map, the third feature map, and the fourth feature map into a feature pyramid network, obtaining a fifth feature map and a sixth feature map through a first C2f module and a second C2f module of the feature pyramid network, and obtaining a seventh feature map through a first CE module of the feature pyramid network includes: Upsampling the first feature map and fusing it with the fourth feature map, extracting features through a first C2f module to obtain a fifth feature map; Upsampling the fifth feature map and fusing it with the third feature map, and passing it through the first CE module to obtain a seventh feature map; The seventh feature map is upsampled and fused with the second feature map, and features are extracted by the second C2f module to obtain the sixth feature map.

5. The probe distribution target detection method based on a multi-channel neural network model according to claim 2, characterized in that: The step of inputting the first, fifth, sixth, and seventh feature maps into a path aggregation network, obtaining an eighth and ninth feature maps through a third C2f module and a fourth C2f module of the path aggregation network, and obtaining a tenth feature map through a second CE module of the path aggregation network includes: The sixth feature map is processed by the convolution operation and the fifth SPD module of the path aggregation network and then fused with the seventh feature map. After extracting features through the third C2f, the eighth feature map is obtained. The eighth feature map is processed by a convolution operation and a sixth SPD module of a path aggregation network, and then fused with the fifth feature map, and the tenth feature map is obtained by passing it through a second CE module; The tenth feature map is processed by the convolution operation and the seventh SPD module of the path aggregation network and then fused with the first feature map, and then passed through the fourth C2f module to obtain the ninth feature map.

6. The probe distribution target detection method based on a multi-channel neural network model according to claim 5, characterized in that: The eighth feature map is processed by a convolution operation and a sixth SPD module of a path aggregation network and then fused with the fifth feature map, and a tenth feature map is obtained by a second CE module, including: The fused feature map is input into the second CE module to obtain the first sub-feature map and the second sub-feature map; The second sub-feature map is convolved once and then input into the ECA attention mechanism module, and the third sub-feature map is obtained through a global average pooling. Calculating the convolution kernel size of the third sub-feature map based on the inverse mapping function; Convolving the third sub-feature map to obtain a fourth sub-feature map; After performing a convolution operation on the fourth sub-feature map with a convolution kernel size of a preset value, a fifth sub-feature map is obtained; The fifth sub-feature map and the second sub-feature map are skip-connected to obtain a sixth sub-feature map; The sixth sub-feature map is input into the ECA attention mechanism module after a convolution, and the above operation is repeated N times to obtain the Nth sub-feature map; After concatenating the first sub-feature map, the sixth sub-feature map, and the Nth sub-feature map, a tenth feature map is obtained through convolution.

7. The probe distribution target detection method based on a multi-channel neural network model according to claim 1, characterized in that: The Head network includes multiple Detect layers, each Detect layer includes a regression branch and a classification branch, the regression branch is used to output the position and size of the bounding box, and the classification branch is used to output the category and confidence.

8. A probe distribution target detection device based on a multi-channel neural network model, implemented using the probe distribution target detection method based on a multi-channel neural network model according to any one of claims 1 to 7, characterized in that: The device comprises: An acquisition module is used to acquire G channel data and R channel data of the fluorescence image of the cell to be tested; A processing module is used to input the G channel data into the trained yolov8-SCC deep neural network model to obtain a green fluorescent probe distribution map, and input the R channel data into the trained yolov8-SCC deep neural network model to obtain a red fluorescent probe distribution map; a merging module, configured to merge the green fluorescent probe distribution map and the red fluorescent probe distribution map to obtain a relationship map between the cells to be tested and the probe distribution; Among them, the yolov8-SCC deep neural network model includes a Backbone network, a Neck network and a Head network. The Backbone network includes an SPFF module, the Neck network includes a feature pyramid network and a path aggregation network, the feature pyramid network includes a first CE module, and the path aggregation network includes a second CE module and a first SPD module.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the probe distribution target detection method based on the multi-channel neural network model is implemented as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the probe distribution target detection method based on a multi-channel neural network model is implemented.

Images