A method, apparatus, device, medium, and product for counting optic nerve axons across species

Abstract

The application discloses a cross-species optic nerve axon counting method, device, equipment, medium and product, and relates to the field of image processing. The method comprises the following steps: acquiring an optic nerve microscopic image to be processed; obtaining a classification confidence score by using a trained counting model according to the optic nerve microscopic image to be processed; in the trained counting model, a feature extraction network is a VGG-16 network after removing a full connection layer and a maximum pooling layer of a fifth convolution block; the third convolution block and the fourth convolution block of the VGG-16 network both perform dilated convolution; in the training process of the counting model, a Hungarian algorithm and a domain adversarial adaptation algorithm based on gradient reversal are jointly optimized; points with a classification confidence score greater than a set threshold are extracted, and an absolute coordinate set of the points is determined; and the total number of axons is obtained by counting according to the absolute coordinate set of the points. The application can improve counting efficiency, counting accuracy and scalability.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a method, apparatus, device, medium and product for cross-species optic nerve axon counting. Background Technology

[0002] Thin sections of the optic nerve are a crucial matrix for quantifying the severity of nerve damage and assessing the effectiveness of neuroprotective or regenerative therapies. In clinical and basic research, experiments are often conducted using multi-species models, such as rats, goats, and rabbits. In these cross-species experiments, axonal number and density are fundamental structural outcome indicators for quantifying damage and assessing treatment efficacy.

[0003] However, current methods and automation techniques for counting optic nerve axons face the following significant drawbacks and technical bottlenecks in practical applications: Manual counting is inefficient and subject to subjective bias: Despite recent advancements in automated image analysis techniques, manual counting remains widely used in practice. Because axons are densely packed small targets, and because microscopic images have complex and heterogeneous backgrounds, manual counting is not only extremely time-consuming and labor-intensive, but also exhibits significant inter-evaluator variability.

[0004] Existing automated algorithms struggle to handle high-density, adherent targets: Current automated axon counting methods mainly fall into two categories: segmentation-based methods and density map regression methods. On the one hand, segmentation-based methods often fail when faced with high-density regions due to target adhesion and image artifacts. On the other hand, traditional counting often employs density map regression (generating Gaussian heatmaps), but in optic nerve images, density maps can cause signal merging between adjacent axons, making effective segmentation impossible and thus coupling localization errors with counting errors.

[0005] Severe cross-species domain shift leads to poor model generalization: a core challenge in cross-species axon counting is the significant domain shift problem. First, there are substantial differences in axon diameter distribution, myelin thickness, and texture patterns among different species. Second, tissue processing and staining procedures introduce systematic changes in color, contrast, and background texture, while differences in microscope systems and acquisition settings further alter resolution and noise characteristics.

[0006] The combined effect of these factors means that models trained on single species or experimental conditions cannot generalize to new species and platforms, increasing false positives and false negatives and leading to severe counting bias. This necessitates costly relabeling and retraining when introducing each new species into the analysis, severely limiting the scalability of cross-species research. Even previously developed point regression models show significant performance degradation when faced with appearance bias and collection differences across species.

[0007] In summary, there is an urgent need to develop an automated axon counting method that can overcome the challenges of high-density crowding, avoid intermediate density map artifacts, and effectively eliminate cross-species microscopic image domain offset bias, thereby providing a consistent, comparable, and highly scalable measurement scheme for cross-species research. Summary of the Invention

[0008] The purpose of this application is to provide a cross-species optic nerve axon counting method, apparatus, device, medium, and product that can improve counting efficiency, counting accuracy, and scalability.

[0009] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a cross-species optic nerve axon counting method, comprising: Acquire microscopic images of the optic nerve to be processed; The classification confidence score is obtained using a trained counting model based on the optic nerve microscopic image to be processed. The trained counting model includes a feature extraction network, a lightweight feature pyramid network, and a two-branch prediction head. The feature extraction network is a VGG-16 network after removing the fully connected layers and the max-pooling layer of the fifth convolutional block. The third and fourth convolutional blocks of the VGG-16 network both perform dilated convolutions. During the training of the counting model, the Hungarian algorithm and a gradient inversion-based domain adversarial adaptation algorithm are used for joint optimization. Extract the points whose classification confidence scores are greater than a set threshold and determine the set of absolute coordinates of the points; The total number of axons is obtained by statistically analyzing the set of absolute coordinates of the points.

[0010] In one embodiment, a classification confidence score is obtained based on the optic nerve microscopic image to be processed using a trained counting model, specifically including: Based on the optic nerve microscopic image to be processed, the third and fourth convolutional blocks of the feature extraction network are used to output feature maps for the inference stage, respectively. The feature maps output from the third and fourth convolutional blocks during the inference stage are fused using a lightweight feature pyramid network to obtain shared feature maps for the inference stage. Based on the shared feature map of the inference stage, classification is performed using the classification branch of the two-branch prediction head to obtain a classification confidence score.

[0011] In one embodiment, the training process of the counting model specifically includes: Acquire microscopic images of the optic nerves of multiple species for training and annotate the real axons based on the microscopic images of the optic nerves of multiple species for training to obtain the training set; Based on the optic nerve microscopic images used for training from the multi-species network, the third and fourth convolutional blocks of the feature extraction network are used to output feature maps for the training phase, respectively. The feature maps output from the third and fourth convolutional blocks during the training phase are fused using a lightweight feature pyramid network to obtain shared feature maps during the training phase. Based on the shared feature map of the training phase, the regression branch of the two-branch prediction head is used to make predictions, and a set of candidate prediction points is obtained. Based on the shared feature map of the training phase and the set of candidate prediction points, the classification branch of the dual-branch prediction head is used to classify the candidate prediction points and obtain the confidence score of each candidate prediction point belonging to the true axon. Construct a cost matrix based on the candidate prediction point set, the true axons, and the confidence scores; The point matching loss is determined based on the cost matrix, geometric constraint threshold, and Hungarian algorithm. Global average pooling is performed based on the shared feature maps from the training phase, and the loss is propagated using a multilayer perceptron domain discriminator containing a gradient reversal layer to obtain the domain adaptation loss. The total loss function is determined based on the point matching loss and the domain adaptation loss. By using backpropagation and the Adam optimizer to optimize the weight parameters in the counting model based on the total loss function, a well-trained counting model is obtained.

[0012] In one embodiment, the cost matrix is ​​expressed as: ; in, Here is the cost matrix. For regression cost weights, For the prediction point, To be more realistic, For classification cost weights, This is the confidence score for the model's predictions.

[0013] In one embodiment, the point matching loss is a weighted sum of regression loss and classification loss; The regression loss The expression is: ; The classification loss The expression is: ; in, The balance factor is M, where M is the total number of actual axonal points. To be more realistic, To focus parameters, Let i be the predicted point, N be the total number of candidate predicted points, and i and j be the indexes. Indicates the prediction point number. For the first The confidence level at which a predicted point is classified as an axon is given, where t refers to the label category.

[0014] In one embodiment, the expression for the domain adaptation loss is: ; The domain adaptation loss is calculated simultaneously on both source and target domain data, wherein For domain adaptation loss, The mathematical expectation of the source domain data distribution. It represents the mathematical expectation of the distribution of data in the source domain.

[0015] Secondly, this application provides a cross-species optic nerve axon counting device, comprising: The acquisition module is used to acquire the optic nerve microscopic image to be processed; The prediction module is used to obtain a classification confidence score based on the optic nerve microscopic image to be processed using a trained counting model. The trained counting model includes a feature extraction network, a lightweight feature pyramid network, and a two-branch prediction head. The feature extraction network is a VGG-16 network after removing the fully connected layers and the max-pooling layer of the fifth convolutional block. The third and fourth convolutional blocks of the VGG-16 network both perform dilated convolutions. During the training of the counting model, the Hungarian algorithm and a gradient-reversed domain adversarial adaptation algorithm are used for joint optimization. The extraction module is used to extract points whose classification confidence scores are greater than a set threshold and determine the set of absolute coordinates of the points; The statistics module is used to perform statistics based on the set of absolute coordinates of the points to obtain the total number of axons.

[0016] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cross-species optic nerve axon counting method described above.

[0017] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cross-species optic nerve axon counting method.

[0018] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the cross-species optic nerve axon counting method.

[0019] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a cross-species optic nerve axon counting method, device, equipment, medium, and product. During the training process of the counting model, the Hungarian algorithm is used to achieve a strict one-to-one matching between predicted points and real points, effectively solving the counting conflict problem caused by high-density target overlap and adhesion. Through a gradient-inverted domain adversarial adaptation algorithm, the model is effectively prevented from failing in cross-domain applications, significantly reducing the counting bias dependent on specific AND, and improving the scalability of the model. By removing the fully connected layer and the max-pooling layer of the fifth convolutional block, the dilated convolution is performed on the third and fourth convolutional blocks of the VGG-16 network, which exponentially expands the effective receptive field of the convolutional kernel without increasing the number of model parameters, so as to better distinguish dense axons and background glial cells, thereby improving the counting accuracy. The entire process realizes automated counting, avoiding the problems of low efficiency and subjective bias in manual counting. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a cross-species optic nerve axon counting method in one embodiment of this application; Figure 2 A schematic diagram of a cross-species optic nerve axon counting method provided in an embodiment of this application; Figure 3 This is a diagram showing the counting effect; Figure 4 A schematic diagram of the functional modules of a cross-species optic nerve axon counting device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] In one exemplary embodiment, such as Figure 1 As shown, a cross-species optic nerve axon counting method is provided. This method is executed by a computer device, specifically by a computer device such as a terminal or server alone, or by a terminal and server together. In the embodiments of this application, the method includes the following steps.

[0025] Step 101: Obtain the microscopic image of the optic nerve to be processed.

[0026] Step 102: Using the trained counting model, obtain the classification confidence score based on the optic nerve microscopic image to be processed. The trained counting model includes a feature extraction network, a lightweight feature pyramid network, and a two-branch prediction head. The feature extraction network is a VGG-16 network after removing the fully connected layers and the max-pooling layer of the fifth convolutional block. The third and fourth convolutional blocks of the VGG-16 network both perform dilated convolutions. During the training of the counting model, the Hungarian algorithm and a gradient-reversed domain adversarial adaptation algorithm are used for joint optimization. In practical applications, the final output of the trained counting model includes the predicted point location, the classification confidence score of the predicted point location, and the domain classification result.

[0027] Step 103: Extract the points whose classification confidence scores are greater than a set threshold and determine the set of absolute coordinates of the points.

[0028] Step 104: Statistically calculate the total number of axons based on the set of absolute coordinates of the points.

[0029] In an exemplary embodiment, the classification confidence score is obtained using a trained counting model based on the optic nerve micrograph to be processed. Specifically, this includes: using the third and fourth convolutional blocks of the feature extraction network to output feature maps for the inference stage based on the optic nerve micrograph to be processed; using a lightweight feature pyramid network to perform multi-scale feature fusion on the feature maps output by the third and fourth convolutional blocks to obtain shared feature maps for the inference stage; and using the classification branch of a dual-branch prediction head to perform classification based on the shared feature maps for the inference stage to obtain the classification confidence score.

[0030] In an exemplary embodiment, the training process of the counting model specifically includes: The training set is obtained by acquiring microscopic images of the optic nerves of multiple species for training and labeling the real axons based on these images.

[0031] Based on the optic nerve microscopic images used for training from the multi-species network, the third and fourth convolutional blocks of the feature extraction network are used to output feature maps for the training phase.

[0032] The feature maps output from the third and fourth convolutional blocks during the training phase are fused using a lightweight feature pyramid network to obtain shared feature maps for the training phase.

[0033] Based on the shared feature map from the training phase, the regression branch of the dual-branch prediction head is used to make predictions, resulting in a set of candidate prediction points.

[0034] Based on the shared feature map from the training phase and the set of candidate prediction points, the classification branch of the dual-branch prediction head is used to classify each candidate prediction point and obtain a confidence score that the candidate prediction point belongs to the true axon.

[0035] A cost matrix is ​​constructed based on the candidate prediction point set, the true axons, and the confidence score.

[0036] The point matching loss is determined based on the cost matrix, geometric constraint threshold, and Hungarian algorithm.

[0037] Global average pooling is performed based on the shared feature maps from the training phase, and the loss is propagated using a multilayer perceptron domain discriminator containing a gradient reversal layer to obtain the domain adaptation loss.

[0038] The total loss function is determined based on the point matching loss and the domain adaptation loss.

[0039] By using backpropagation and the Adam optimizer to optimize the weight parameters in the counting model based on the total loss function, a well-trained counting model is obtained.

[0040] In one exemplary embodiment, the point matching loss is a weighted sum of regression loss and classification loss.

[0041] Regression loss The calculation is performed only on successfully matched positive samples, using mean squared error (MSE) to measure the deviation between the predicted coordinates and the ground truth coordinates, driving the model to learn accurate sub-pixel localization. .

[0042] Classification loss In this case, since the number of background points (negative samples) far exceeds the number of axon points (positive samples), the classes are extremely imbalanced. Therefore, Focal Loss is used to reduce the weight of easily classified negative samples and focus on learning the difficult samples. .

[0043] in, This is the balancing factor, used to adjust the weights of positive and negative samples. This is the focusing parameter; the larger the value, the stronger the suppression of easily distinguishable samples. i and j are both ordinal numbers. Indicates the prediction point number. For the first The confidence level at which a predicted point is classified as an axon is given, where t refers to the label category.

[0044] The cost matrix is ​​a cost table that measures how poorly the i-th predicted point matches the j-th true point. The geometric constraint threshold applies a hard filter to the cost matrix, forcibly prohibiting matching if the distance is too great. The Hungarian algorithm, on the other hand, finds a strict one-to-one allocation on the cost matrix that minimizes the total cost, producing a matching set.

[0045] Finally, the matching results obtained using the Hungarian algorithm are used for backpropagation training through a loss function.

[0046] In one exemplary embodiment, the domain adaptation loss is the standard binary cross entropy loss in a domain classifier: .

[0047] The loss is calculated simultaneously on both the source and target domain data, where The mathematical expectation representing the distribution of data in the source domain. It represents the mathematical expectation of the distribution of data in the source domain.

[0048] In another exemplary embodiment, a specific process for the cross-species optic nerve axon counting method in practical application is also provided. This method uses a computer or a device with data processing capabilities as the execution entity; for example, a computer device... Figure 2 As shown, it includes the following steps.

[0049] Step 1: Acquire and preprocess optic nerve microscopic images of multiple species.

[0050] ① Obtain microscopic images of semi-thin sections of optic nerves from multiple species (e.g., goats, rats, and rabbits) from a computer device. These microscopic images of semi-thin sections of optic nerves from multiple species are the input images.

[0051] ② During the model training phase, the acquired images are subjected to data augmentation processing, including random scaling with scaling factors from 0.7 to 1.3, random cropping to 128×128 pixel image blocks, and random horizontal flipping with a probability of 0.5, in order to enhance the robustness of the model.

[0052] ③ During the model inference stage, the computer equipment can directly receive complete microscopic images of any size as input.

[0053] Step 2: Extract high-frequency features and fuse them with multiple scales using an improved morphological backbone network.

[0054] ① The computer equipment inputs the preprocessed image into the improved VGG-16 feature extraction network.

[0055] ② This feature extraction network removes the fully connected layers and the final max pooling layer of the standard VGG-16, retaining the first 13 convolutional layers, making it a fully convolutional network.

[0056] ③ The computer device performs dilated convolution operations in the third and fourth convolutional blocks of the network, with dilation rates set to 2 and 4, respectively. The purpose of this step is to exponentially expand the effective receptive field without increasing the number of parameters, while fixing the output stride of the feature map to 1 / 8 of the input resolution, thereby maximizing the preservation of high-frequency edge details of the axons.

[0057] ④ The computer device performs a lightweight feature pyramid network operation to perform multi-scale feature fusion. Specifically, the feature map output from the 4th convolutional block is adjusted in channel dimension through a 1×1 convolution, and then concatenated and fused with the feature map output from the 3rd convolutional block to output a shared feature map.

[0058] Step 3: Generation of end-to-end candidate points based on the dual-branch prediction head.

[0059] ① The computer device inputs the shared feature map obtained in step S2 into two sub-network branches in parallel: the regression branch and the classification branch.

[0060] ② Regression branch execution steps: For each grid point on the feature map, calculate and predict its coordinate offset relative to the true axon center. The location of the predicted point is determined by the coordinate offset, and a set of all candidate predicted points is output. , where N is the total number of candidate prediction points. This is the prediction point.

[0061] ③ Classification branch execution steps: Perform classification evaluation on the above candidate prediction point set and shared feature map, and output the confidence score of each candidate prediction point belonging to the true axon. .

[0062] Step 4: Joint training of the model based on bipartite graph matching and multi-domain adversarial adaptation.

[0063] During the model training phase, the computer equipment uses the labeled set of real axon points. (M is the total number of actual axon points,) (For real points), perform the joint optimization process: Step 4.1: One-to-one matching based on the Hungarian algorithm.

[0064] ①Constructing a cost matrix for computer equipment The elements of the cost matrix The matching cost between the i-th predicted point and the j-th true point is defined. The calculation formula is: .

[0065] in, This represents the Euclidean distance between the predicted point and the actual point. The confidence score for the model's prediction; and These are the regression cost weights and classification cost weights (preset hyperparameters).

[0066] ② Introducing geometric constraint thresholds for computer equipment (e.g., average axon radius), when At that time, a mandatory cost is set. It is infinitely large.

[0067] ③ The computer equipment uses the Hungarian algorithm to find the allocation scheme with the minimum total matching cost, so as to achieve a strict one-to-one matching between the predicted points and the actual points.

[0068] Step 4.2: Perform gradient-inverted domain adversarial adaptation.

[0069] ① The computer device takes the features of the fourth convolutional block output by the backbone network in step 2, performs global average pooling, and then feeds them in parallel into the multilayer perceptron domain discriminator containing the gradient reversal layer (GRL).

[0070] ② Forward propagation computation: GRL performs identity transformation. The domain discriminator is composed of a fully connected neural network, which contains two hidden layers and one output layer. Batch normalization is performed in the hidden layers, using the ReLU activation function. The domain discriminator predicts the species category (e.g., K multi-class domains such as goat, rat, and rabbit) based on the input feature map.

[0071] ③ Backpropagation calculation: GRL reverses the gradient from the domain discriminator and multiplies it by the negative fitness factor. The data is then passed back to the feature extraction network. This step forces the feature extraction network to maximize domain classification error while minimizing axon counting error, thereby discarding species-specific staining and texture features and learning structurally invariant features shared across species.

[0072] Step 4.3: Calculate the total loss function and update the network weights.

[0073] ① Computer equipment calculates the joint optimization total loss function : .

[0074] in: The mean squared error regression loss is for positive samples (successfully matched points). This is the classification loss calculated based on FocalLoss, used to suppress the weights of a large number of easily classified negative samples (background points). The standard cross-entropy loss is calculated simultaneously on the source and target domain data by the domain discriminator. Assign coefficients to the weights of the loss function; where A dynamic scheduling strategy is adopted, which gradually increases from 0 to 1 as the training progresses, to prevent the model from crashing in the early stages of training.

[0075] ② Computer devices utilize the backpropagation mechanism and the Adam optimizer, based on Update the weight parameters of the entire network framework.

[0076] Step 5: Perform automated inference and axon counting.

[0077] ① Computer equipment inputs microscopic images of unknown species or new cross-domain optic nerves into the trained network.

[0078] ② The computer device extracts the points whose classification confidence scores from the prediction head are greater than a preset threshold, and outputs the set of absolute coordinates of these points.

[0079] ③ The computer equipment performs statistical calculations on the final output set of coordinate points, directly outputting the total number of axons in the image and their spatial distribution, thus completing the automated quantitative assessment. The effect is as follows: Figure 3 As shown. Among them Figure 3 Image (a) shows the results of optic nerve counting in goats. Figure 3 (b) in the figure shows the results of rat optic nerve counting. Figure 3 (c) in the figure shows the results of the rabbit's optic nerve counting.

[0080] Compared to existing technologies, this application demonstrates significant advantages in automated quantitative analysis of optic nerve axons from multiple species, including high precision, high robustness, and high cross-domain scalability. Specifically, one or more technical advantages of this application and their intrinsic connection with corresponding technical points are detailed below: Advantage 1: It solves the problems of undercounting and positioning errors in high-density, adherent scenes, achieving high-precision counting of dense targets. This advantage stems from the "end-to-endpoint prediction mechanism based on bipartite graph matching" mentioned in step 4.1 of this application. In existing technologies, segmentation-based methods are prone to failure in dense regions due to target adhesion and image artifacts, while density map regression methods often merge signals from adjacent axons, leading to coupling of positioning and counting errors, resulting in severe undercounting in extremely high-density scenes. This application completely eliminates artifact-prone representations such as intermediate density maps by introducing an end-to-endpoint prediction mechanism. This mechanism utilizes the Hungarian algorithm to achieve a strict one-to-one matching between predicted points and actual labeled points. This technique not only effectively solves the counting conflict problem caused by overlapping and adhesion of high-density targets, but also successfully prevents counting saturation when facing extremely high-density image samples such as rat optic nerves, maintaining a quasi-normal residual distribution centered at zero, and achieving the lowest mean absolute error and the narrowest consistency limit.

[0081] Advantage 2: Significantly reduces the cost of cross-domain model transfer, achieving high scalability for cross-species generalization without re-labeling. This advantage stems from the "gradient-inverted joint multi-domain adversarial adaptation training method" mentioned in step 4.2 of this application. In cross-species optic nerve research, different species exhibit inherent differences in axonal diameter and myelin sheath thickness, and their staining and microscopic imaging systems differ, leading to severe "domain shift" phenomena in color, contrast, and background texture. Existing models typically cannot generalize to data from unseen species, necessitating costly re-labeling and retraining. This application introduces a domain adversarial adaptation module containing a gradient inversion layer, inverting the gradients from the domain classifier during the backpropagation phase of training. This forces the feature extraction network to discard species-specific staining hues and background noise "style" features, instead extracting cross-species shared, species-independent axonal "structural" features. This mechanism effectively prevents model failure in cross-domain applications, significantly reduces domain-dependent counting bias, and provides a highly consistent, comparable, and scalable measurement tool for cross-species neuroprotection and regeneration research.

[0082] Advantage 3: Effectively overcomes the destruction of spatial information of small targets by downsampling in deep networks, improving the extraction of high-frequency details and sub-pixel-level localization accuracy. This advantage stems from the "high-frequency feature extraction method based on morphological adaptation and dilated convolution" mentioned in step 2 of this application. The visual definition of optic nerve axons mainly relies on the high-frequency edge features of the myelin sheath. Traditional deep residual networks often suffer from the loss of spatial information of small targets due to multiple pooling and downsampling operations, thus reducing localization accuracy. This application, by employing a morphologically adapted VGG-16 backbone network and introducing dilated convolution, can exponentially expand the effective receptive field of the convolution kernel without increasing the number of model parameters, thereby better distinguishing dense axons from background glial cells. Simultaneously, this design successfully fixes the output stride of the feature map to 1 / 8 of the input resolution (instead of 1 / 32 of the standard network), maximizing the preservation of the spatial resolution of the microscopic image and the texture details of the axon edges. This provides an indispensable underlying feature foundation for subsequent branches to achieve sub-pixel-level high-precision center coordinate regression.

[0083] Based on the same inventive concept, this application also provides a video tag processing apparatus for implementing the video tag processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more video tag processing apparatus embodiments provided below can be found in the limitations of the video tag processing method described above, and will not be repeated here.

[0084] In one exemplary embodiment, such as Figure 4 As shown, a cross-species optic nerve axon counting device is provided, comprising: The acquisition module is used to acquire the optic nerve microscopic image to be processed.

[0085] The prediction module is used to obtain a classification confidence score based on the optic nerve microscopic image to be processed using a trained counting model. The trained counting model includes a feature extraction network, a lightweight feature pyramid network, and a two-branch prediction head. The feature extraction network is a VGG-16 network after removing the fully connected layers and the max pooling layer of the fifth convolutional block. The third and fourth convolutional blocks of the VGG-16 network both perform dilated convolutions. During the training of the counting model, the Hungarian algorithm and the domain adversarial adaptation algorithm based on gradient inversion are used for joint optimization.

[0086] The extraction module is used to extract points whose classification confidence scores are greater than a set threshold and determine the set of absolute coordinates of the points.

[0087] The statistics module is used to perform statistics based on the set of absolute coordinates of the points to obtain the total number of axons.

[0088] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 5 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores video tag processing data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a video tag processing method.

[0089] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include, but are not limited to, the following: Figure 5 The embodiments show more or fewer components, combinations of certain components, or different component arrangements. In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the method embodiments described above.

[0090] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method embodiments.

[0091] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method embodiments.

[0092] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0093] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0094] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0095] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0096] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A cross-species optic nerve axon counting method, characterized in that, include: Acquire microscopic images of the optic nerve to be processed; The classification confidence score is obtained using a trained counting model based on the optic nerve microscopic image to be processed. The trained counting model includes a feature extraction network, a lightweight feature pyramid network, and a two-branch prediction head. The feature extraction network is a VGG-16 network after removing the fully connected layers and the max-pooling layer of the fifth convolutional block. The third and fourth convolutional blocks of the VGG-16 network both perform dilated convolutions. During the training of the counting model, the Hungarian algorithm and a gradient inversion-based domain adversarial adaptation algorithm are used for joint optimization. Extract the points whose classification confidence scores are greater than a set threshold and determine the set of absolute coordinates of the points; The total number of axons is obtained by statistically analyzing the set of absolute coordinates of the points.

2. The cross-species optic nerve axon counting method according to claim 1, characterized in that, Based on the optic nerve microscopic image to be processed, a training counting model is used to obtain a classification confidence score, specifically including: Based on the optic nerve microscopic image to be processed, the third and fourth convolutional blocks of the feature extraction network are used to output feature maps for the inference stage, respectively. The feature maps output from the third and fourth convolutional blocks during the inference stage are fused using a lightweight feature pyramid network to obtain shared feature maps for the inference stage. Based on the shared feature map of the inference stage, classification is performed using the classification branch of the two-branch prediction head to obtain a classification confidence score.

3. The cross-species optic nerve axon counting method according to claim 1, characterized in that, The training process of the counting model specifically includes: Acquire microscopic images of the optic nerves of multiple species for training and annotate the real axons based on the microscopic images of the optic nerves of multiple species for training to obtain the training set; Based on the optic nerve microscopic images used for training from the multi-species network, the third and fourth convolutional blocks of the feature extraction network are used to output feature maps for the training phase, respectively. The feature maps output from the third and fourth convolutional blocks during the training phase are fused using a lightweight feature pyramid network to obtain shared feature maps during the training phase. Based on the shared feature map of the training phase, the regression branch of the two-branch prediction head is used to make predictions, and a set of candidate prediction points is obtained. Based on the shared feature map of the training phase and the set of candidate prediction points, the classification branch of the dual-branch prediction head is used to classify the candidate prediction points and obtain the confidence score of each candidate prediction point belonging to the true axon. Construct a cost matrix based on the candidate prediction point set, the true axons, and the confidence scores; The point matching loss is determined based on the cost matrix, geometric constraint threshold, and Hungarian algorithm. Global average pooling is performed based on the shared feature maps from the training phase, and the loss is propagated using a multilayer perceptron domain discriminator containing a gradient reversal layer to obtain the domain adaptation loss. The total loss function is determined based on the point matching loss and the domain adaptation loss. By using backpropagation and the Adam optimizer to optimize the weight parameters in the counting model based on the total loss function, a well-trained counting model is obtained.

4. The cross-species optic nerve axon counting method according to claim 3, characterized in that, The expression for the cost matrix is: ； in, Here is the cost matrix. For regression cost weights, For the prediction point, To be more realistic, For classification cost weights, This is the confidence score for the model's predictions.

5. The cross-species optic nerve axon counting method according to claim 3, characterized in that, The point matching loss is a weighted sum of the regression loss and the classification loss; The regression loss The expression is: ； The classification loss The expression is: ； in, The balance factor is M, where M is the total number of actual axonal points. To be more realistic, To focus parameters, Let i be the predicted point, N be the total number of candidate predicted points, and i and j be the indexes. Indicates the prediction point number. For the first The confidence level at which a predicted point is classified as an axon is given, where t refers to the label category.

6. The cross-species optic nerve axon counting method according to claim 3, characterized in that, The expression for the domain adaptation loss is: ； The domain adaptation loss is calculated simultaneously on both source and target domain data, wherein For domain adaptation loss, The mathematical expectation of the distribution of data in the source domain. It represents the mathematical expectation of the distribution of data in the source domain.

7. A cross-species optic nerve axon counting device, characterized in that, include: The acquisition module is used to acquire the optic nerve microscopic image to be processed; The prediction module is used to obtain classification confidence scores based on the optic nerve microscopic image to be processed using a trained counting model. The trained counting model includes a feature extraction network, a lightweight feature pyramid network, and a two-branch prediction head. The feature extraction network is a VGG-16 network after removing the fully connected layers and the max-pooling layer of the fifth convolutional block. The third and fourth convolutional blocks of the VGG-16 network both perform dilated convolutions. During the training of the counting model, the Hungarian algorithm and a gradient-reversed domain adversarial adaptation algorithm are used for joint optimization. The extraction module is used to extract points whose classification confidence scores are greater than a set threshold and determine the set of absolute coordinates of the points; The statistics module is used to perform statistics based on the set of absolute coordinates of the points to obtain the total number of axons.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the cross-species optic nerve axon counting method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the cross-species optic nerve axon counting method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the cross-species optic nerve axon counting method as described in any one of claims 1-6.

Images