Auditory neuron target extraction and positioning method based on vertical model

Abstract

The invention discloses an auditory neuron target extraction and positioning method based on a vertical model. Comprising the following steps: 1) preprocessing multi-modal data; 2) analyzing clinical symptom description in the current medical history by means of a Gemini vertical model; 3) constructing a multi-modal feature fusion module; and 4) dynamically analyzing auditory neurons of neuron images of different age groups. According to the method, screening and information identification are carried out on the neuron features of the specific area in the image, the difficulty that in the prior art, the positioning precision of auditory neurons in the image is low and the individual differences of patients cannot be dynamically adapted is overcome, and the analysis accuracy and efficiency are remarkably improved. Meanwhile, the dynamic adaptation capability of the technology can optimize an analysis strategy according to image signal characteristics of special crowds such as young people and old people, and fine analysis of an auditory nerve mechanism is promoted.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of neuroscience and artificial intelligence, specifically relating to a method for target extraction and localization of auditory neurons based on a vertical model. Background Technology

[0002] Currently, vertical models demonstrate significant advantages in medical prediction: they can automatically extract nonlinear features from multimodal data through deep neural network architectures, overcoming the limitations of traditional statistical methods that rely on manual feature engineering. Simultaneously, by incorporating a vertical model into the loss function for dynamic adjustment, the loss weights can be dynamically adjusted to address individual heterogeneity and noise interference in medical data, effectively reducing the bias in predicting key clinical indicators and solving the problem of insufficient adaptability of traditional fixed loss functions. This further enhances the model's ability to capture weakly correlated features in medical data, improving the reliability of prediction results in scenarios such as rare diseases and complex complications, and providing better technical support for precise clinical diagnosis and treatment decisions. Furthermore, dynamically optimizing feature fusion analysis for special populations significantly improves generalization performance compared to traditional analysis methods. Moreover, the continuous learning mechanism of large language models allows them to dynamically adapt to new clinical data distributions, while traditional methods often require remodeling.

[0003] One of the core pathogenic mechanisms of sensorineural hearing loss is damage to the auditory nerve conduction, which depends on the normal electrophysiological activity of spiral ganglion neurons (SGNs) in the cochlea. When specific SGNs malfunction (e.g., abnormal firing frequency, conduction delay, or signal transmission interruption), auditory signals cannot be properly transmitted to the central nervous system, leading to hearing loss. Current diagnostic techniques, such as auditory brainstem response (ABR) and electrocochleography (ECochG), can only determine whether cochlear nerve conduction is damaged, but cannot precisely pinpoint "which one or which group of SGNs is malfunctioning." Traditional artificial intelligence models (such as CNN and LSTM) attempt to locate abnormal neurons through electrophysiological signals, but they have two major drawbacks: First, the models use a fixed loss function, which cannot adapt to individual differences in the electrophysiological signals of different auditory neurons (e.g., signal-to-noise ratio, conduction delay fluctuations, and varying degrees of abnormal signal weakness), resulting in insufficient accuracy in locating some auditory neurons. Second, the models struggle to capture the complex correlation between SGN electrophysiological signals and auditory dysfunction, failing to achieve a precise mapping between "lesioned neurons and hearing abnormality symptoms." Summary of the Invention

[0004] To address the problems in the background art, this invention proposes a method for extracting and locating auditory neurons based on a vertical model. By integrating a large language model with a deep learning network, it dynamically adapts to individual patient data and accurately locates diseased auditory neurons in images.

[0005] Therefore, the present invention adopts the following technical solution: A method for target extraction and localization of auditory neurons based on a vertical model Includes the following steps: 1) Multimodal data preprocessing: Construct a U-Net network with residual attention module to achieve accurate segmentation of neurons in the target region; further, integrate the Qianwen hybrid reasoning model into the loss function through feature weight output to optimize the weight allocation of the hybrid loss function; 2) Using Gemini vertical model to analyze clinical symptom descriptions in present medical history: Using the natural language processing technology of Gemini large language model, the symptom descriptions in clinical medical history text are analyzed in a structured manner (automatically identifying key elements such as symptom duration and severity), and a temporal attention mechanism is used to establish the spatiotemporal relationship between symptoms and generate standardized clinical terminology codes. 3) Construct a multimodal feature fusion network: Employ a cross-modal attention mechanism to achieve spatial alignment of clinical text keywords in segmented regions; 4) Dynamic analysis of auditory nerve lesions in neuronal images of different age groups: Images containing dynamic data of auditory cortex neuronal firing are collected, and the neuronal images are divided into three types: juvenile, adult and elderly based on the recognition results of step 2). Specific preprocessing is performed on images in different types, and then combined with steps 1) to 3), the location of the lesion neurons in the image, the cause of the lesion neurons and clinical symptoms are finally output.

[0006] In step 1), the specific steps of multimodal data preprocessing are as follows: Step 1.1) Construct a U-Net network consisting of an encoder and a decoder. The encoder path employs a three-level hierarchical feature extraction module. Each module contains two stages of 3×3 convolutional layers. After each convolutional layer, batch normalization, ELU activation, and Dropout regularization (probability 0.3) are performed sequentially. Then, 2×2 max pooling with a stride of 2 is used to progressively compress the feature map space dimension to 1 / 4 of the original input, while simultaneously expanding the channel dimension to 256, thus achieving progressive extraction of contextual information. The decoder path achieves feature reconstruction by combining bilinear interpolation with transposed convolution (2×2 kernel, stride 2): at each upsampling stage, the spatial dimension of the low-resolution feature map is first expanded with 2×2 transposed convolution, and then skipped with the high-resolution features of the corresponding layer of the encoder; the fused features after skipping are processed by 16-channel and 8-channel 3×3 convolution kernels in sequence, and then the segmentation result is generated by 1×1 convolutional affine transformation and sigmoid activation; Step 1.2) Calculate the segmentation error based on the real-time segmentation output of the U-Net network (e.g., when the Dice coefficient of the edge region is <0.85), and dynamically adjust the loss function in combination with the Qianwen hybrid inference model to optimize the weight distribution of the hybrid loss function (e.g., the Dice loss weight of the edge region is increased to 0.6). This solves the problem of low segmentation accuracy of the traditional U-Net for blurred edge regions, avoids the model from missegmenting surrounding tissues due to image noise, and ultimately significantly improves the segmentation accuracy of the target region. Step 1.3) Calculate the activity curve of each neuron in the segmented and denoised neuron image using the following formula:

[0007] in, This represents the sum of all pixel values ​​in the region labeled by the current neuron in the current frame. express The average value; Step 1.4) Compare the calcium signal emission of all neurons in the image, and determine whether the neurons in the image are diseased neurons based on the degree of calcium signal emission.

[0008] Step 1.4) specifically refers to: If the calcium signal emission of a neuron in an image is reduced by more than 50% compared to the signal emission of other neurons, then the neuron is determined to have weakened calcium signal and is identified as a diseased neuron, with the cause of the disease being weakened signal emission. If the calcium signal emission of a neuron in an image is more than 50% stronger than that of other neurons, then the neuron is considered to have enhanced calcium signal and is identified as a diseased neuron, with the cause of the disease being excessive signal emission. If the calcium signal firing cycle of a neuron in an image increases by more than twice the firing cycle of other neurons, then its neuronal activity is determined to be inhibited, and the neuron is determined to be a diseased neuron, with the cause of the disease being the inhibition of neuronal activity. If the calcium signal firing cycle of a neuron in an image is significantly shorter than that of other neurons, and less than twice as long, then the neuron is considered to be overexcited, and the neuron is identified as a diseased neuron, with the cause of the disease being overexcitement.

[0009] Step 2) specifically refers to: Step 2.1) The Gemini vertical model is used to intelligently process the patient's present medical history text. Relying on the medical knowledge base and contextual understanding ability of the model pre-trained, the clinical entities in the text are automatically identified and a three-dimensional semantic association network of symptoms-signs-disease course is established. The medical knowledge base contains rules relating symptoms and signs of auditory nerve diseases; the clinical entities include symptoms, signs, and disease course (e.g., symptom: tinnitus, sign: decreased hearing threshold, disease course: 3 months). Step 2.2) Utilize a multi-head self-attention mechanism to achieve deep semantic deconstruction of the current medical history text through parallel multi-dimensional dynamic weight allocation.

[0010] Step 2.2) specifically involves: establishing dynamic connection weights for each text through a query-key value interaction system; obtaining the dynamic connection weights and calculating the position vector using sine and cosine functions; then injecting absolute temporal information into the position vector to enable the Gemini vertical model to accurately perceive the time span in the symptom description; in addition, using different attention heads to process the input text through division of labor and collaboration to solve the long-distance dependency characteristics unique to medical texts and output a semantic representation of clinical texts. The functions of the different attention points are as follows: focusing on the relationship between symptoms and signs, tracking the semantic scope of negation words, and establishing logical connections across sentences; For example, the co-occurrence relationship between "fever" and "chills" is captured by a specific attention head, while the temporal progression relationship between "three days ago" and "this morning" is handled by another set of attention heads. The query-key-value interaction system has the following functions: it maps each text to a unique key based on a hash function, calculates the associated dynamic weight value through the dot product attention of the query vector and the pre-stored key vector, and stores it in an in-memory database; it further optimizes query efficiency by using a Bloom filter and integrates an LRU caching strategy to manage frequently accessed data, ensures the real-time performance and accuracy of weight calculation through a distributed consistency protocol, and finally establishes dynamic connection weights.

[0011] The construction of the multimodal feature fusion network in step 3) specifically involves: A cross-modal attention mechanism is adopted to spatially align and weight the preprocessed auditory nerve calcium signal in step 1) with the semantic features of clinical text after entity recognition and feature normalization in step 2): first, the temporal features of calcium signal and the temporal sequence of text symptoms are initially aligned by the dynamic time warping (DTW) algorithm, and then the spatial mapping accuracy is optimized by using attention scores. If the neuronal calcium signal in the image is a juvenile neuronal calcium signal, the spatial alignment and spatial mapping between the juvenile neuronal calcium signal and the semantic features of the clinical text can be enhanced by adjusting the parameters of the cross-modal attention mechanism.

[0012] In step 4), the dynamic analysis of auditory nerve lesions in different age groups is carried out as follows: Images containing dynamic data of auditory cortex neuron firing are acquired, and the neuron images in the images are classified into three patterns: juvenile, adult, and elderly based on the recognition results of step 2). For images of auditory cortex neuronal firing in young children, noise reduction is first performed. Then, the method in step 1) is used to segment and identify the calcium signals of cochlear neurons in the image. Then, the structured clinical text is obtained through text recognition and feature normalization in step 2). Then, the correlation between neuronal calcium signals and clinical text is obtained based on step 3). Finally, the location of the diseased neurons in the image, the cause of the diseased neurons, and the clinical symptoms are output. The noise reduction process specifically includes: A dual-optical-path imaging framework was constructed. Images obtained from the low signal-to-noise ratio (SNR) side optical path were used as training source data. A non-continuous frame skipping sampling strategy was adopted, using intervening frames as input and intermediate frames as supervision targets to establish a self-supervised training paradigm that does not require manual annotation. The training source data was input into a convolutional neural network based on a symmetric topology for preliminary denoising. In the downsampling stage, each unit of the convolutional neural network based on the symmetric topology consisted of two convolutional layers with 3×3 kernels and a 2×2 max-pooling layer. In the upsampling stage, two 3×3 convolutional layers were set in conjunction with transposed convolutions to improve resolution. Nonlinear feature transformation was introduced between all layers through the ReLU activation function, and a Dropout layer with a probability of 0.2 was used to prevent overfitting. Furthermore, an evaluation system was constructed using a benchmark image obtained from a high SNR reference optical path. The peak SNR between the benchmark image and the preliminary denoising result was calculated for quantitative verification, thereby achieving noise suppression of the neuron image.

[0013] For images of auditory cortex neuronal firing in adults, the method in step 1) is used to segment and identify the calcium signals of cochlear neurons in the image. Then, the structured clinical text is obtained through text recognition and feature normalization in step 2). Then, the correlation between neuronal calcium signals and clinical text is obtained based on step 3). Finally, the location of the diseased neurons in the image, the cause of the diseased neurons, and the clinical symptoms are output. For images of auditory cortex neuronal firing in the elderly, a temporal alignment correction network is used. A three-dimensional convolutional layer is used to extract the spatiotemporal features of calcium signals. Temporal integrity is maintained by spatial pooling that preserves the temporal dimension. Simultaneously, a bidirectional LSTM layer captures the contextual dependencies of the signals. Finally, a temporally corrected sequence is output by a temporally distributed fully connected layer. Then, the method in step 1) is used to segment and identify the extracted cochlear neuronal calcium signals from the images. Following this, text recognition and feature normalization in step 2) yields structured clinical text. Further, based on step 3), the correlation between neuronal calcium signals and clinical text is obtained. Finally, the location of the diseased neurons in the image, the cause of the disease, and the clinical symptoms are output.

[0014] II. A target extraction and localization device for auditory neurons based on a vertical model Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the method for locating auditory neuron lesions based on a vertical model as described in any one of claims 1 to 8.

[0015] III. A computer-readable storage medium The storage medium stores a computer program, which, when executed by a processor, implements a method for extracting and locating auditory neurons based on a vertical model.

[0016] The beneficial effects of this invention are: This invention targets and identifies neuronal features in specific regions, overcoming the difficulties of low accuracy in target extraction and localization of auditory neurons in existing technologies, and significantly improving accuracy and efficiency. Simultaneously, its dynamic adaptability can optimize analysis for issues such as low signal-to-noise ratio and large signal transmission delay fluctuations in image signals, promoting a more refined understanding of auditory neural mechanisms. Furthermore, by establishing a mapping relationship between "image features and neuronal function," it provides a high-precision localization tool for neuronal image analysis, filling the technological gap in research on the correlation between auditory neuron image features and function, and providing a new technological paradigm for the field of biomedical image analysis. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process for the target extraction and localization method of auditory neurons based on a vertical model provided by the present invention.

[0018] Figure 2 This invention provides a diagram of a network structure for adjusting a dynamic loss function based on a vertical model.

[0019] Figure 3 This is a schematic diagram of a denoising convolutional neural network based on a symmetric topology provided by the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] The present invention provides a method for locating auditory neuron lesions based on a vertical model, the specific steps of which are as follows: (1) Multimodal data preprocessing: Construct a U-Net network with residual attention module to achieve accurate segmentation of neurons in the target region; further integrate the Qianwen hybrid reasoning model into the loss function through the “feature weight output” method, and reduce the segmentation error of weak feature regions by real-time output adjustment coefficients, thereby improving the segmentation accuracy.

[0022] (2) Using Gemini vertical model to analyze clinical symptom descriptions in present medical history: Using the natural language processing technology of Gemini large language model, the symptom descriptions in the clinical present medical history text are analyzed in a structured manner, key elements such as symptom duration and severity are automatically identified, the temporal attention mechanism is used to establish the spatiotemporal relationship between symptoms, and standardized clinical terminology codes are generated.

[0023] (3) Construct a multimodal feature fusion network: adopt a cross-modal attention mechanism to achieve spatial alignment of clinical text keywords in segmented regions.

[0024] (4) Dynamic analysis of auditory nerve lesions in different age groups: collect dynamic data of auditory cortex neurons, and divide the neuron images into three typical patterns of childhood, adulthood and old age based on the recognition results of step 2). Specific preprocessing is carried out for different patterns, and then combined with steps 1 to 3 to finally output the location of the lesion neurons in the image, the cause of the lesion neurons and clinical symptoms.

[0025] The multimodal data preprocessing in step (1) is performed according to the following steps: Step 1.1) Construct a U-Net network consisting of an encoder and a decoder. The encoder path employs a three-level hierarchical feature extraction module, with each module containing two stages of 3×3 convolutions. After each convolutional layer, batch normalization, ELU activation, and Dropout regularization (probability 0.3) are performed sequentially. The feature map space dimension is progressively compressed to 1 / 4 of the original input using 2×2 max pooling with a stride of 2, while simultaneously expanding the channel dimension to 256, achieving progressive extraction of contextual information. The decoder path achieves feature reconstruction through a combination of bilinear interpolation and transposed convolutions (2×2 kernels, stride 2): at each upsampling stage, the low-resolution feature map space dimension is first expanded using 2×2 transposed convolutions, and then skipped to the corresponding high-resolution feature map of the encoder layer. After the fused features are processed by 16-channel and 8-channel 3×3 convolution kernels, a segmentation result is generated through 1×1 convolutional affine transformation and sigmoid activation.

[0026] Step 1.2) Calculate the segmentation error based on the real-time segmentation output of the U-Net network (e.g., when the Dice coefficient of the edge region is <0.85), and dynamically adjust the loss function in conjunction with the Qianwen hybrid inference model to optimize the weight allocation of the hybrid loss function (e.g., the Dice loss weight of the edge region is increased to 0.6, which solves the problem of low segmentation accuracy of the traditional U-Net for blurred edge regions, avoids the model from missegmenting surrounding tissues due to image noise, and ultimately significantly improves the segmentation accuracy of the target region).

[0027] Step 1.3) Calculate the activity curve of each neuron in the segmented and denoised neuron image using the following formula:

[0028] in, This represents the sum of all pixel values ​​in the region labeled by the current neuron in the current frame. express The average value.

[0029] Step 1.4) Compare the calcium signal emission of all neurons in the image. If the calcium signal emission of one neuron is reduced by more than 50% compared to the emission of other neurons, it is determined that its calcium signal is weakened, and this neuron may be a diseased neuron, with the cause being weakened signal emission. If the calcium signal emission of one neuron is increased by more than 50% compared to the emission of other neurons, it is determined that its calcium signal is enhanced, and this neuron may be a diseased neuron, with the cause being excessive signal emission. If the calcium signal emission cycle of one neuron is longer than the emission cycle of other neurons, and is more than twice as long, it is determined that its neuronal activity is inhibited, and this neuron may be a diseased neuron, with the cause being inhibited neuronal activity. If the calcium signal emission cycle of one neuron is significantly shorter than the emission cycle of other neurons, and is less than twice as long, it is determined that its neuronal activity is overexcited, and this neuron may be a diseased neuron, with the cause being overexcited neuronal activity.

[0030] In step 2), the clinical symptom descriptions in the present medical history are analyzed using the Gemini vertical model, as follows.

[0031] Step 2.1) The Gemini vertical model is used to intelligently process the patient's current medical history text. Relying on the model's pre-trained medical knowledge base (including symptom-sign association rules for cochlear nerve diseases) and contextual understanding capabilities, it automatically identifies clinical entities in the text (such as the symptom "tinnitus", the sign "hearing threshold decline", and the course of the disease "3 months"), and establishes a three-dimensional semantic association network of "symptom-sign-course of the disease". Using a multi-head self-attention mechanism, deep semantic deconstruction of the current medical history text is achieved through parallel multi-dimensional dynamic weight allocation. Specifically, a non-linear association network between words is constructed. Each text establishes dynamic connection weights through a query-key value interaction system. For example, the co-occurrence relationship between "fever" and "chills" is captured by a specific attention head, while the temporal progression relationship between "three days ago" and "this morning" is handled by another set of attention heads.

[0032] The dynamic connection weights are calculated using sine and cosine functions to obtain position vectors. Absolute temporal information is injected through these position vectors, enabling the model to accurately perceive the time span in symptom descriptions. By leveraging the collaborative functions of different attention heads, the unique long-distance dependency characteristic of medical texts is addressed, resulting in a semantic representation of clinical texts. The functions of these different attention heads are: focusing on the relationship between symptom and sign modifiers, tracking the semantic scope of negation words, and establishing logical connections across sentences.

[0033] In step 3), constructing a multimodal feature fusion network includes the following steps: Step 3.1) Employing a cross-modal attention mechanism, the preprocessed auditory nerve calcium signal is spatially aligned and weighted with the clinical text semantic features obtained through entity recognition and feature normalization in Step 2. First, a dynamic time warping (DTW) algorithm is used to achieve initial alignment between the temporal features of the calcium signal and the temporal sequence of the text symptoms. Then, attention scores are used to optimize the spatial mapping accuracy. For children, a specific population, the spatial alignment and mapping between neuronal calcium signals and clinical text semantic features are enhanced by adjusting the parameters of the cross-modal attention mechanism.

[0034] In step 4), the dynamic analysis of auditory nerve lesions in different age groups is carried out as follows: Step 4.1) Collect dynamic data of auditory cortex neuron firing, and classify neuron images into three typical patterns: juvenile, adult, and elderly based on the recognition results of Step 2).

[0035] For images of auditory cortex neuronal firing in young children, such as Figure 3 As shown, before segmentation in step 1), denoising is performed using a convolutional neural network based on a symmetric topology. Specifically, a dual-optical-path imaging framework is first constructed. The image obtained from the low signal-to-noise ratio side optical path is used as the training source data. A non-continuous frame skipping sampling strategy is adopted, using the interval frame as input and the intermediate frame as the supervised target, establishing a self-supervised training paradigm that does not require manual annotation. The training source data is then input into the convolutional neural network based on a symmetric topology for preliminary denoising. In the downsampling stage, each unit of the convolutional neural network based on a symmetric topology consists of two convolutional layers with 3×3 kernels and a 2×2 max-pooling layer in sequence. In the upsampling stage, two 3×3 convolutional layers are set in conjunction with transposed convolution to improve resolution. Nonlinear feature transformation is introduced between all layers through the ReLU activation function, and a Dropout layer with a probability of 0.2 is used to prevent overfitting. Furthermore, an evaluation system is constructed using a reference image obtained from a high signal-to-noise ratio reference optical path. The peak signal-to-noise ratio between the reference image and the preliminary denoising result is calculated for quantitative verification, thereby achieving noise suppression of the neuron image.

[0036] Subsequently, the calcium signals of cochlear neurons in the image are extracted by segmentation and recognition in step 1), and then the structured clinical text is obtained by text recognition and feature normalization in step 2). In addition, the correlation between neuronal calcium signals and clinical text is obtained based on step 3). Finally, the location of the diseased neurons in the image, the cause of the diseased neurons, and the clinical symptoms are output.

[0037] For images of auditory cortex neuronal firing in adults, the calcium signals of cochlear neurons in the image are extracted by segmentation and recognition in step 1). Then, the structured clinical text is obtained by text recognition and feature normalization in step 2). Finally, the correlation between neuronal calcium signals and clinical text is obtained based on step 3). The location of the diseased neurons in the image, the cause of the diseased neurons, and the clinical symptoms are finally output.

[0038] For auditory cortical neurons in the elderly, a temporal alignment correction network is employed. This network utilizes 3D convolutional layers to extract spatiotemporal features of calcium signals, preserving temporal integrity through spatial pooling that retains the temporal dimension. Simultaneously, a bidirectional LSTM layer captures the contextual dependencies of the signals. Finally, a temporally corrected sequence is output by a temporally distributed fully connected layer. Subsequently, the segmentation and recognition in step 1) extracts calcium signals from cochlear neurons in the image. Then, text recognition and feature normalization in step 2) yields structured clinical text. Further, based on step 3), the correlation between neuronal calcium signals and clinical text is obtained. Finally, the location of diseased neurons in the image, the cause of the disease, and clinical symptoms are output. Specific implementation examples: The method uses acquired auditory neuron images as input and compares and analyzes the calcium signal firing levels of all neurons in the images to identify and locate them. Based on the calcium signal firing levels, it determines whether the neurons in the image are diseased. Then, the clinical symptom text description is input into the Gemini vertical model for parsing. Further, a cross-modal attention mechanism is used to achieve spatial alignment between text keywords and image segmentation regions. Finally, the location, functional characteristics, and clinical symptoms of the target neurons are output. Experimental results show that this method significantly improves the efficiency and accuracy of neuron feature extraction and localization, and the output localization results and functional characteristics are highly consistent with manually labeled and analyzed results.

Claims

1. A method for target extraction and localization of auditory neurons based on a vertical model, characterized in that, Includes the following steps: 1) Multimodal data preprocessing: Construct a U-Net network with residual attention module to achieve accurate segmentation of neurons in the target region; further, integrate the Qianwen hybrid reasoning model into the loss function through feature weight output to optimize the weight allocation of the hybrid loss function; 2) Analyze clinical symptom descriptions in present medical history using Gemini vertical model: Utilize the natural language processing technology of Gemini large language model to perform structured analysis of symptom descriptions in clinical medical history texts, establish spatiotemporal relationships between symptoms using temporal attention mechanism, and generate standardized clinical terminology codes; 3) Construct a multimodal feature fusion network: Employ a cross-modal attention mechanism to achieve spatial alignment of clinical text keywords in segmented regions; 4) Dynamic analysis of auditory nerve lesions in neuronal images of different age groups: Images containing dynamic data of auditory cortex neuronal firing are collected, and the neuronal images are divided into three types: juvenile, adult and elderly based on the recognition results of step 2). Specific preprocessing is performed on images in different types, and then combined with steps 1) to 3), the location of the lesion neurons in the image, the cause of the lesion neurons and clinical symptoms are finally output.

2. The method for target extraction and localization of auditory neurons based on a vertical model according to claim 1, characterized in that, In step 1), the specific steps of multimodal data preprocessing are as follows: Step 1.1) Construct a U-Net network consisting of an encoder and a decoder. The encoder path employs a three-level hierarchical feature extraction module. Each module contains two stages of 3×3 convolutional layers. After each convolutional layer, batch normalization, ELU activation, and Dropout regularization are performed sequentially. Then, 2×2 max pooling with a stride of 2 is used to progressively compress the feature map space dimension to 1 / 4 of the original input, while simultaneously expanding the channel dimension to 256, thus achieving progressive extraction of contextual information. The decoder path achieves feature reconstruction by combining bilinear interpolation and transposed convolution: in each upsampling stage, the spatial dimension of the low-resolution feature map is first expanded by 2×2 transposed convolution, and then it is connected to the high-resolution feature map of the corresponding layer of the encoder. The fused features after the skip connections are processed sequentially by 16-channel and 8-channel 3×3 convolution kernels, and then the segmentation results are generated by 1×1 convolutional affine transformation and sigmoid activation. Step 1.2) Calculate the segmentation error based on the real-time segmentation output of the U-Net network, and dynamically adjust the loss function in conjunction with the Qianwen hybrid inference model to optimize the weight allocation of the hybrid loss function; Step 1.3) Calculate the activity curve of each neuron in the segmented and denoised neuron image using the following formula: in, This represents the sum of all pixel values ​​in the region labeled by the current neuron in the current frame. express The average value; Step 1.4) Compare the calcium signal emission of all neurons in the image, and determine whether the neurons in the image are diseased neurons based on the degree of calcium signal emission.

3. The method for target extraction and localization of auditory neurons based on a vertical model according to claim 2, characterized in that, Step 1.4) specifically refers to: If the calcium signal emission of a neuron in an image is reduced by more than 50% compared to the signal emission of other neurons, then the neuron is determined to have weakened calcium signal and is identified as a diseased neuron, with the cause of the disease being weakened signal emission. If the calcium signal emission of a neuron in an image is more than 50% stronger than that of other neurons, then the neuron is considered to have enhanced calcium signal and is identified as a diseased neuron, with the cause of the disease being excessive signal emission. If the calcium signal firing cycle of a neuron in an image increases by more than twice the firing cycle of other neurons, then its neuronal activity is determined to be inhibited, and the neuron is determined to be a diseased neuron, with the cause of the disease being the inhibition of neuronal activity. If the calcium signal firing cycle of a neuron in an image is significantly shorter than that of other neurons, and less than twice as long, then the neuron is considered to be overexcited, and the neuron is identified as a diseased neuron, with the cause of the disease being overexcitement.

4. The method for target extraction and localization of auditory neurons based on a vertical model according to claim 1, characterized in that, Step 2) specifically refers to: Step 2.1) The Gemini vertical model is used to intelligently process the patient's present medical history text. Relying on the medical knowledge base and contextual understanding ability of the model pre-trained, the clinical entities in the text are automatically identified and a three-dimensional semantic association network of symptoms-signs-disease course is established. The medical knowledge base contains rules relating symptoms and signs of auditory nerve diseases; the clinical entities include symptoms, signs, and disease course. Step 2.2) Utilize a multi-head self-attention mechanism to achieve deep semantic deconstruction of the current medical history text through parallel multi-dimensional dynamic weight allocation.

5. The method for target extraction and localization of auditory neurons based on a vertical model according to claim 4, characterized in that, Step 2.2) specifically involves: establishing dynamic connection weights for each text through a query-key value interaction system; obtaining the dynamic connection weights and calculating the position vector using sine and cosine functions; then injecting absolute temporal information through the position vectors to enable the Gemini vertical model to accurately perceive the time span in the symptom description; in addition, using different attention heads to process the input text through division of labor and collaboration to output a clinical text semantic representation. The functions of the different attention points are as follows: focusing on the relationship between symptoms and signs, tracking the semantic scope of negation words, and establishing logical connections across sentences; The query-key-value interaction system has the following functions: it maps each text to a unique key based on a hash function, calculates the associated dynamic weight value through the dot product attention of the query vector and the pre-stored key vector, and stores it in an in-memory database; it further optimizes query efficiency by using a Bloom filter and integrates an LRU caching strategy to manage frequently accessed data, ensures the real-time performance and accuracy of weight calculation through a distributed consistency protocol, and finally establishes dynamic connection weights.

6. The method for target extraction and localization of auditory neurons based on a vertical model according to claim 1, characterized in that, The construction of the multimodal feature fusion network in step 3) specifically involves: A cross-modal attention mechanism is adopted to spatially align and weight the preprocessed auditory nerve calcium signal in step 1) with the semantic features of clinical text after entity recognition and feature normalization in step 2): first, the temporal features of calcium signal and the temporal sequence of text symptoms are initially aligned by dynamic time warping algorithm, and then the spatial mapping accuracy is optimized by attention score. If the neuronal calcium signal in the image is a juvenile neuronal calcium signal, the spatial alignment and spatial mapping between the juvenile neuronal calcium signal and the semantic features of the clinical text can be enhanced by adjusting the parameters of the cross-modal attention mechanism.

7. The method for target extraction and localization of auditory neurons based on a vertical model according to claim 1, characterized in that, In step 4), the dynamic analysis of auditory nerve lesions in different age groups is carried out as follows: The steps for dynamically analyzing auditory nerve lesions in neuronal images of different age groups are as follows: Images containing dynamic data of auditory cortex neuron firing are acquired, and the neuron images in the images are classified into three patterns: juvenile, adult, and elderly based on the recognition results of step 2). For images of auditory cortex neuronal firing in young children, noise reduction is first performed. Then, the method in step 1) is used to segment and identify the calcium signals of cochlear neurons in the image. Then, the structured clinical text is obtained through text recognition and feature normalization in step 2). Then, the correlation between neuronal calcium signals and clinical text is obtained based on step 3). Finally, the location of the diseased neurons in the image, the cause of the diseased neurons, and the clinical symptoms are output. For images of auditory cortex neuronal firing in adults, the method in step 1) is used to segment and identify the calcium signals of cochlear neurons in the image. Then, the structured clinical text is obtained through text recognition and feature normalization in step 2). Then, the correlation between neuronal calcium signals and clinical text is obtained based on step 3). Finally, the location of the diseased neurons in the image, the cause of the diseased neurons, and the clinical symptoms are output. For images of auditory cortex neuronal firing in the elderly, a temporal alignment correction network is used. A three-dimensional convolutional layer is used to extract the spatiotemporal features of calcium signals. Temporal integrity is maintained by spatial pooling that preserves the temporal dimension. Simultaneously, a bidirectional LSTM layer captures the contextual dependencies of the signals. Finally, a temporally corrected sequence is output by a temporally distributed fully connected layer. Then, the method in step 1) is used to segment and identify the extracted cochlear neuronal calcium signals from the images. Following this, text recognition and feature normalization in step 2) yields structured clinical text. Further, based on step 3), the correlation between neuronal calcium signals and clinical text is obtained. Finally, the location of the diseased neurons in the image, the cause of the disease, and the clinical symptoms are output.

8. The method for target extraction and localization of auditory neurons based on a vertical model according to claim 7, characterized in that, For images of auditory cortex neuron firing in young children, the noise reduction process specifically involves: A dual-optical-path imaging framework was constructed. Images obtained from the low signal-to-noise ratio (SNR) side optical path were used as training source data. A non-continuous frame skipping sampling strategy was adopted, using intervening frames as input and intermediate frames as supervision targets to establish a self-supervised training paradigm that does not require manual annotation. The training source data was input into a convolutional neural network based on a symmetric topology for preliminary denoising. In the downsampling stage, each unit of the convolutional neural network based on the symmetric topology consisted of two convolutional layers with 3×3 kernels and a 2×2 max-pooling layer. In the upsampling stage, two 3×3 convolutional layers were set in conjunction with transposed convolutions to improve resolution. Nonlinear feature transformation was introduced between all layers through the ReLU activation function, and a Dropout layer with a probability of 0.2 was used to prevent overfitting. Furthermore, an evaluation system was constructed using a benchmark image obtained from a high SNR reference optical path. The peak SNR between the benchmark image and the preliminary denoising result was calculated for quantitative verification, thereby achieving noise suppression of the neuron image.

9. A device for target extraction and localization of auditory neurons based on a vertical model, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the method for locating auditory neuron lesions based on a vertical model as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method for extracting and locating auditory neurons based on a vertical model as described in any one of claims 1 to 8.

Images