Method, device, system and medium for processing image sequences for two-photon microscopy

Abstract

The present application relates to a method, device, system and medium for processing image sequences for two-photon microscopy. The processing method includes dividing the image sequence taken by the two-photon microscope after correction and noise reduction into multiple image subsequences, determining the time mask that characterizes the position and morphology of potential nerve cells in each subsequence; removing the potential nerve cells that are repeatedly identified based on the number of shared pixels between potential nerve cells in the time mask, and generating a spatial mask that characterizes the position and morphology of potential nerve cells in the entire image sequence; combining the image sequence after correction and noise reduction, performing calcium signal feature analysis on the potential nerve cells in the spatial mask to generate a cell mask that characterizes the position and morphology of real nerve cells and extracting the real nerve cell event signal of each real nerve cell in the cell mask. The present application can efficiently, automatically and more accurately identify real nerve cells from a two-photon image sequence for use in nerve cell activation analysis, etc.

Description

Technical Field

[0001] This application belongs to the field of neuroimaging, and particularly relates to a method, device, system, and medium for processing image sequences of a two-photon microscope. Background Art

[0002] In the field of neurobiology, accurately recording and analyzing the subtle changes in signals in neurons and their subcellular structures constitutes the core foundation for understanding the brain's encoding and reconstruction mechanisms. Calcium imaging technology, especially when combined with a two-photon microscope, shows great potential in the study of in vivo animal models in static and even dynamic behaviors.

[0003] However, the existing technology lacks the ability to process the images / image sequences captured by a two-photon microscope. Regarding the accurate identification of nerve cells in two-photon calcium imaging images, one processing method relies on experienced professionals to manually define the contours of the included nerve cells in the images. This method that highly depends on scarce human resources not only has low efficiency but also cannot guarantee the consistency of the identification results, far from meeting the needs of the technological development in this field. On the other hand, there are also some technologies in the field that attempt to identify nerve cells through image processing. Although they can identify some nerve cells and their cell structures, there are often problems of over-detection. That is to say, the number of nerve cells they identify even has an order-of-magnitude deviation compared with the manual identification results used as a benchmark, with low algorithm accuracy and poor practicability.

[0004] Therefore, in this field, there is no existing technology that can accurately identify the real nerve cells contained in the image sequences of a two-photon microscope and detect the nerve signals associated with the nerve cells through automated processing and analysis of two-photon imaging data. Summary of the Invention

[0005] This application is provided to solve the above-mentioned defects existing in the prior art. There is a need for a method, device, system, and medium for processing image sequences of a two-photon microscope, which can accurately detect the real nerve cells and their related nerve cell events contained therein through automated processing of the image sequences captured by the two-photon microscope.

[0006] According to a first aspect of the present application, there is provided a method for processing an image sequence for a two-photon microscope, including: receiving an image sequence obtained by imaging with the two-photon microscope, the image sequence including a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel point in an image frame reflects the calcium signal intensity of a nerve cell in the region where it is located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cell; performing correction and noise reduction processing on each frame image in the image sequence; dividing the image sequence after correction and noise reduction processing into a plurality of image subsequences corresponding to a first time period, and determining a time mask corresponding to each image subsequence in association with the image features in each image subsequence, wherein the time mask characterizes the positions and morphologies of potential nerve cells identified based on the corresponding image subsequence; removing repeatedly identified potential nerve cells based on the number of common pixel points between potential nerve cells in different time masks to generate a spatial mask for the image sequence, wherein the spatial mask characterizes the positions and morphologies of potential nerve cells identified based on the complete image sequence; combining the image sequence after correction and noise reduction processing, and performing calcium signal feature analysis on each potential nerve cell in the spatial mask to screen the potential nerve cells and generate a cell mask characterizing the positions and morphologies of real nerve cells included in the image sequence, and extracting real nerve cell event signals of each real nerve cell at each moment in the cell mask for use in analyzing the activation state and activation degree of each real nerve cell.

[0007] According to a second aspect of the present application, there is provided a processing device for an image sequence for a two-photon microscope, the processing device including an interface and a processor, the interface being configured to receive an image sequence obtained by imaging with the two-photon microscope, the image sequence including a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel point in an image frame reflects the calcium signal intensity of a nerve cell in the region where it is located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cell; the processor being configured to execute the method for processing an image sequence for a two-photon microscope according to each embodiment of the present application.

[0008] According to a third aspect of the present application, there is provided a processing system for an image sequence for a two-photon microscope, the processing system including a two-photon microscope and the processing device for an image sequence for a two-photon microscope according to each embodiment of the present application.

[0009] According to a fourth aspect of the present application, there is provided a non-transitory computer-readable storage medium storing a program, the program causing a processor to execute the method for processing an image sequence for a two-photon microscope according to each embodiment of the present application.

[0010] The processing methods, devices, systems, and media for image sequences of a two-photon microscope provided by various embodiments of the present application segment a long-time image sequence into image subsequences, so as to more accurately identify potential nerve cells in the image subsequences with lower complexity and generate time masks corresponding to each image subsequence. Based on multiple time masks, potential nerve cells that are repeatedly identified are removed through pixel-level processing to generate a spatial mask representing the position and morphology of potential nerve cells in the entire image sequence. Then, in combination with the image sequence after correction and noise reduction, calcium signal feature analysis is performed on the potential nerve cells in the spatial mask, thereby finally achieving accurate identification and detection of real nerve cells and nerve cell events at each moment. The present application can provide a systematic, automated, and high-precision one-stop processing flow for the analysis of calcium imaging data obtained by a two-photon microscope, which helps to promote the development of research in the field of neuroscience and other related fields, and has profound scientific value and application prospects.

[0011] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the following specific embodiments of the present application are specifically exemplified.

[0012] It should be understood that the foregoing general description and the following detailed description are merely illustrative and explanatory, and are not restrictive of the claimed invention. Brief Description of the Drawings

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0014] Figure 1 The flowchart shows the processing method for an image sequence of a two-photon microscope according to an embodiment of the present application.

[0015] Figure 2(a) shows a flowchart of the correction process for each image frame based on a periodically updated correction template according to an embodiment of the present application.

[0016] Figure 2(b) shows a schematic diagram of the generation method of the initial template of the correction template according to an embodiment of the present application.

[0017] Figure 2(c) shows a schematic diagram of the improvement of nerve cell clarity after correcting an image frame using the correction template according to an embodiment of the present application.

[0018] FIG. 3(a) shows a flowchart of denoising an image frame according to an embodiment of the present application.

[0019] FIG. 3(b) shows a comparison schematic diagram of a single image frame before and after denoising according to an embodiment of the present application.

[0020] FIG. 4(a) shows a schematic diagram of steps for determining corresponding temporal masks based on image features in each image subsequence according to an embodiment of the present application.

[0021] FIG. 4(b) shows a schematic diagram of the generation process of an initial temporal mask according to an embodiment of the present application.

[0022] Figure 5 A schematic diagram showing a temporal mask, a spatial mask, and a cellular mask according to an embodiment of the present application.

[0023] Figure 6 A schematic diagram showing the process of generating a spatial mask based on each temporal mask according to an embodiment of the present application.

[0024] Figure 7 A schematic diagram showing a processing flow for screening each potential nerve cell in the spatial mask and generating a cellular mask according to an embodiment of the present application.

[0025] FIG. 8(a) shows a schematic diagram of a typical nerve cell and its corresponding background region according to an embodiment of the present application.

[0026] FIG. 8(b) shows a signal curve of a nerve cell and its background region in a time series according to an embodiment of the present application.

[0027] FIG. 8(c) shows the fluorescence signal of potential nerve cells after removing background fluorescence and de-linearization processing according to an embodiment of the present application.

[0028] FIG. 8(d) shows a fluorescence signal distribution curve of potential nerve cells according to an embodiment of the present application.

[0029] FIG. 8(e) shows a calcium signal curve of real nerve cells according to an embodiment of the present application.

[0030] FIG. 8(f) shows a schematic diagram of real nerve cell event signals of real nerve cells at each moment according to an embodiment of the present application.

[0031] Figure 9 A schematic diagram showing a partial composition of a processing device for an image sequence of a two-photon microscope according to an embodiment of the present application.

[0032] Figure 10 A schematic diagram showing a partial composition of a processing system for an image sequence of a two-photon microscope according to an embodiment of the present application. Detailed implementation manners

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions of the embodiments of this application in conjunction with the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of this application without creative efforts fall within the scope of protection of this application.

[0034] Unless otherwise defined, the technical terms or scientific terms used in this application shall have the ordinary meaning understood by those of ordinary skill in the art to which this application pertains. Words such as "including" or "comprising" and the like mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects.

[0035] The "first", "second", and similar words used in this application do not denote any order, quantity, or importance, but are only used for distinction. Words such as "including" or "comprising" and the like mean that the elements before this word cover the elements listed after this word, and do not exclude the possibility of also covering other elements. The execution order of each step in the methods described in this application in conjunction with the accompanying drawings is not limited. As long as the logical relationship between each step is not affected, several steps can be integrated into a single step, a single step can be decomposed into multiple steps, or the execution order of each step can be adjusted according to specific requirements.

[0036] It should also be understood that the term "and / or" in this application is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application generally represents an "or" relationship between the associated objects before and after.

[0037] To keep the following description of the embodiments of this application clear and concise, the detailed descriptions of known functions and known components are omitted in this application.

[0038] The working principle of the two-photon microscope in the embodiments of this application is based on two-photon excitation, which is a non-linear optical phenomenon. During this process, a fluorophore simultaneously absorbs two low-energy photons (usually in the near-infrared band) to achieve a transition from the ground state to the excited state, and subsequently emits fluorescence. This process only occurs in the sample within the laser focus area. Due to its inherent spatial selectivity, it significantly reduces background noise and enhances image contrast. In addition, the use of a pulsed near-infrared light source not only provides a tissue penetration depth of about 1 mm or even deeper, but also effectively reduces the effects of photodamage and photobleaching, making it an ideal tool for in-vivo tissue imaging. Different from confocal microscopy, the two-photon microscope does not rely on a pinhole to limit signal acquisition outside the focal plane, so it can accurately reconstruct three-dimensional structures while maintaining high resolution. In the field of neuroscience research, when the two-photon microscope is combined with calcium-sensitive fluorescent probes, it has brought transformative progress. Calcium imaging technology realizes the dynamic tracking of nerve cell activity by monitoring the changes in intracellular calcium concentration caused by neuronal activity. With its excellent deep-tissue imaging ability, the two-photon microscope allows researchers to directly observe these transient calcium dynamics in the brains of living animals, even under free-behavior conditions. This provides profound insights into understanding the communication mechanisms between neurons and the operating modes of neural networks under physiological and pathological conditions. With single-cell and even subcellular-level resolution, the two-photon microscope has become a key technology platform for exploring the microscopic architecture of the brain and its functional connections.

[0039] Figure 1 The flowchart shows a method for processing an image sequence for a two-photon microscope according to an embodiment of the present application.

[0040] As Figure 1 shown, first, in step 101, an image sequence obtained by imaging with the two-photon microscope is received. The image sequence includes a plurality of two-dimensional image frames corresponding to different moments in time domain. Among them, the fluorescence intensity of the pixel points in the image frame reflects the calcium signal intensity of the nerve cells in the area where they are located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cells.

[0041] In the embodiments of this application, the above image sequence generally refers to the calcium imaging data continuously acquired by the two-photon microscope within a period of time during a calcium imaging experiment on a sample containing nerve cells of a test object, and the image sequence containing neuron calcium imaging data generated therefrom.

[0042] Then, in step 102, each frame image in the image sequence can be corrected and denoised.

[0043] Although two-photon microscopy has a relatively high signal-to-noise ratio, the rigid and non-rigid displacements in the images caused by the movement of the experimental subject and the physiological activities of nerve cells, as well as image noise from other sources, still have a very adverse impact on the subsequent nerve cell recognition process. Therefore, appropriate algorithms must be used to correct the rigid and non-rigid displacements of each frame in the image sequence and obtain as clear image frames as possible through noise suppression, so as to provide a good image quality basis for the subsequent nerve cell recognition process.

[0044] Next, in step 103, multiple temporal masks will be generated. First, the image sequence after correction and noise reduction is segmented into multiple image subsequences corresponding to a first time duration. Then, temporal masks corresponding to each image subsequence are determined in association with the image features in each image subsequence. Thus, the information on the positions and morphologies of potential nerve cells identified based on the corresponding image subsequences is included in each temporal mask.

[0045] Considering that the number of all nerve cells contained in the image sequence taken for a long time is large, and they are adjacent to each other or even partially overlapped. If the image sequence is regarded as a whole to try to identify each nerve cell, the difficulty will undoubtedly be very great. However, the inventor found during the practice of submitting this application that not all nerve cells will be active simultaneously at any given time period, that is to say, only some active nerve cells can be observed during a specific period. Therefore, dividing the image sequence into multiple image subsequences can ensure that different nerve cell activity patterns can be accurately captured in different time windows. Processing each image subsequence containing only some nerve cells separately can effectively reduce the complexity of nerve cell recognition, thereby improving the recognition accuracy and overall recognition efficiency.

[0046] In step 104, potential nerve cells that are repeatedly recognized are removed based on the number of common pixel points between potential nerve cells in different temporal masks to generate the spatial mask of the image sequence, where the spatial mask represents the positions and morphologies of potential nerve cells identified based on the complete image sequence.

[0047] Finally, in step 105, in combination with the image sequence after correction and noise reduction, by analyzing the calcium signal characteristics of each potential nerve cell in the spatial mask, the potential nerve cells are screened and a cell mask representing the positions and morphologies of the real nerve cells contained in the image sequence is generated, and the real nerve cell event signals of each real nerve cell at each moment in the cell mask are extracted for analyzing the activation state and activation degree of each real nerve cell.

[0048] A method for processing an image sequence for a two-photon microscope according to an embodiment of the present application first divides a relatively long image sequence into image subsequences, so that more accurate identification of potential nerve cells can be performed in different time windows in a manner with lower complexity and parallel processing; on the basis of generating a time mask corresponding to each image subsequence, pixel-level processing is used to remove repeatedly identified potential nerve cells to generate a spatial mask including potential nerve cells in the entire image sequence, and the image sequence after correction and noise reduction processing is combined again to perform calcium signal feature analysis on each potential nerve cell, thereby finally achieving accurate identification and detection of real nerve cells and their nerve cell events. The present application provides a systematic, automated, and high-precision one-stop processing flow for the image sequence captured by a two-photon microscope for the first time, which helps to promote the development of research in the field of neuroscience and other related fields, and has profound scientific value and application prospects.

[0049] In the images captured by a two-photon microscope, the motion displacement of the images and the existence of various Poisson noises including thermal noise and shot noise will have a very adverse impact on the subsequent identification of nerve cells. Therefore, image correction and noise reduction processing are very important steps. First, correction processing needs to be performed on each frame of the image.

[0050] FIG. 2(a) shows a schematic flow chart of correcting each image frame based on a periodically updated correction template according to an embodiment of the present application. FIG. 2(b) shows a schematic diagram of the generation method of the initial template of the correction template according to an embodiment of the present application. FIG. 2(c) shows a schematic diagram of the improvement of the clarity of nerve cells after correcting the image frame using the correction template according to an embodiment of the present application.

[0051] As shown in FIG. 2(a), in step 201, starting from the first frame of the image sequence, the first number of image frames are selected at preset intervals until the entire image sequence is covered.

[0052] In some embodiments, the selection of the preset interval size mainly considers within what range the imaging quality remains highly consistent, including but not limited to image jitter, decay of fluorescent proteins, etc., and can also be selected according to the characteristics of the two-photon microscope and the behavioral characteristics of the observed object such as a mouse. For example, if the two-photon microscope has good stability within 1 minute, then when the imaging frequency is 10 Hz, 500 frames can be selected as the preset interval.

[0053] In step 202, the image frame with the largest fluorescence signal among the selected image frames is used as the reference frame.

[0054] In step 203, after aligning each selected image frame with the reference frame, average projection is performed to generate an initial template of the correction template. By way of example only, the alignment method can be, for example, central alignment, or any other applicable alignment method, and the present application does not make specific limitations thereto.

[0055] The image sequence shown in Fig. 2(b) contains 36,000 frames, the preset interval is 500 frames, and the first quantity is 100 frames. That is, starting from the first frame, 100 frames are selected first, and then 100 frames are selected every 500 frames until the end of the image sequence. The frame with the largest fluorescence signal is selected from each of the selected image frames as the reference frame. Among the 6 frames of images f1-f6 schematically shown in Fig. 2(b), the fluorescence of image frame f1 is relatively larger. In the specific implementation process, the fluorescence magnitude of each image frame can be conveniently judged by the method of visual observation, or can be determined by any other applicable image processing algorithm, and the present application does not limit this. IM shown in Fig. 2(b) is the initial template of the correction template generated by performing average projection after aligning each selected image frame with the reference frame. It can be easily seen that the fluorescence signal of the initial template IM is larger than that of any single frame image. Therefore, the initial template determined by the method in the embodiment of the present application summarizes as much as possible the characteristics of all potential nerve cell imaging states in the entire image sequence. Using such an initial template as the key image in the subsequent image correction and registration process is very beneficial for accurately and without omission determining the effective image area for subsequent nerve cell analysis.

[0056] Then, in step 204, based on the initial template, frame-by-frame correction is performed starting from the first frame image in the image sequence. Specifically, for example, an algorithm framework based on the Fast Fourier Transform (FFT) technology and upsampling + Discrete Fourier Transform (DFT) can be used to correct the rigid displacement and non-rigid displacement between images, where FFT is used to correct the rigid displacement of the image, and upsampling + DFT is used to correct the non-rigid offset in the image. In some other embodiments, other correction algorithms can also be used, and the present application does not make specific limitations thereto, as long as it can correct the rigid and non-rigid displacements in the image so that each frame image is aligned as much as possible with a common reference.

[0057] Furthermore, in step 205, the correction template can also be updated based on a preset period using the corrected image sequence, so as to correct subsequent image frames with the updated correction template. The dynamic update of the correction template can enable the correction effect of the image sequence to be improved more quickly.

[0058] As shown in Fig. 2(c), P1 is the average projection of all image frames before image correction using the correction template of the embodiment of the present application, and P2 is the average projection of all image frames after image correction using the correction template of the embodiment of the present application. It can be easily seen from P1 and its partial enlarged view P1', and P2 and its partial enlarged view P2' that before image correction, the average projection shows obvious blurring, while after image correction, the clarity of the average projection is significantly improved, and even the structures of individual nerve cells can be clearly identified.

[0059] According to the image correction method of the embodiment of the present application, especially the generation and update method of the correction template, not all nerve cells in the image sequence captured by the two-photon microscope remain active throughout the entire shooting period. Therefore, compared with the prior art method of only using the initial several frames (e.g., the first 200 frames) of the image sequence to generate the correction template, the method in the embodiment of the present application takes into account the offset of the images and the natural decay of fluorescence during the entire imaging process, and covers as many features in all image frames as possible, without missing possible changes at all times, such as nerve cells that become active only in the later period. In addition, by regularly updating the correction template, the accuracy of the image features in the template is further improved, thus making the overall correction effect of the image sequence better.

[0060] After the correction of each image frame is completed, further image denoising is required. Only as an example, a deep learning network constructed based on algorithms such as DeepInterpolation can be used to perform denoising processing on the corrected image frames. In the neural network of the DeepInterpolation framework, a design of encoder-decoder with skip connections is adopted. By learning and processing the image features of multiple adjacent frames of the central frame to be processed, the overall features of the image sequence where the central frame is located are learned, so that the features of the central frame can be accurately predicted, and then the effective suppression of the image noise of the central frame can be realized. Since existing such trained deep learning networks may only be applicable to images / image sequences with specific parameters, in the embodiment of the present application, a method of performing transfer learning on the trained deep learning network is specifically proposed, so that the deep learning network can adapt to the noise characteristics in the image sequence of the present application, in order to achieve a better denoising effect.

[0061] Fig. 3(a) shows a flowchart of performing denoising processing on an image frame according to an embodiment of the present application.

[0062] Since the existing trained deep learning networks require customized network configurations for different image sequence parameters such as imaging frequencies, it is necessary to use the two-photon microscope image sequence in this application for transfer training. Specifically, as shown in FIG. 3(a), in step 301, for example, first, from all the image frames of the corrected image sequence, every preset number of image frames, a second number of image frames are extracted as training samples to perform transfer training on the deep learning network, so that the trained deep learning network can adaptively fit the relevant parameters of the image sequence including the imaging frequency, where at least one of the preset number and the second number is associated with the imaging frequency. Only as an example, for example, when the imaging frequency is 10 Hz, samples can be extracted every 100 frames, and each time a segment containing 61 image frames is extracted, where the first 30 frames and the last 30 frames are used as the input of the neural network, and the middle frame is used as the target output. Through rapid iterative training, a neural network model optimized for a specific imaging frequency can be obtained in a relatively short time.

[0063] Next, in step 302, the trained deep learning network can be used to perform noise reduction processing on each frame image in the corrected image sequence, thereby obtaining the corrected and noise-reduced image sequence. Only as an example, when performing noise reduction processing on a target image frame in the corrected image sequence, a second number of image frames centered on the target image frame can be symmetrically selected from the image sequence and input into the trained deep learning network to obtain the noise-reduced target image frame.

[0064] FIG. 3(b) shows a comparison schematic diagram of a single image frame before and after noise reduction processing according to an embodiment of the present application. In FIG. 3(b), image frame f31 is the image frame before noise reduction processing, and image frame f32 is the image frame after noise reduction processing. It can be clearly seen from image frame f31 and its enlarged view f31', and image frame f32 and its enlarged view f32' that after the noise reduction processing described in the embodiment of the present application, the Poisson noise (white noise points) in image frame f31 is basically effectively eliminated, and at the same time, the contrast between nerve cells and the background in image frame f32 is also significantly enhanced.

[0065] In some embodiments, the first duration for dividing the image sequence into multiple image subsequences can be preset according to experience. In other embodiments, it can also be determined in the following manner:

[0066] First, multiple alternative first durations can be set, and a corresponding set of time masks are generated based on each alternative first duration; then, in combination with each set of time masks, the number of nerve cells contained in the time masks corresponding to each image subsequence and the similarity degree of the nerve cells recognized in adjacent time masks are used to determine the finally selected first duration.

[0067] For example, when generating the time mask according to the first alternative first duration, the number of neurons included in each time mask is too large, which means that the division of the image subsequence may not effectively reduce the complexity of image processing. Therefore, the appropriate first duration should be smaller. On the other hand, for example, when generating the time mask according to the second alternative first duration, the neurons included in adjacent time masks are very similar and have a high degree of overlap, which may mean that the selected first duration is too short, so that the same group of neurons is active in multiple time masks. In this case, a longer first duration should be selected. Otherwise, not only will the image processing efficiency be reduced, but the same group of neurons may also produce inconsistent recognition results in different time masks, increasing the computational amount of subsequent processing and possibly leading to a decrease in recognition accuracy. That is to say, when the first duration is appropriately selected, the number of neurons included in the time masks corresponding to each image subsequence is appropriate, and the neurons included in adjacent time masks have an appropriate similarity / distinctiveness. The specific settings of the number of neurons included in a single time mask and the similarity of neurons in adjacent time masks are not specifically limited in this application and can be appropriately set according to experience or experimental results.

[0068] FIG. 4(a) shows a schematic diagram of the steps for determining the corresponding time mask based on the image features in each image subsequence according to an embodiment of the present application. FIG. 4(b) shows a schematic diagram of the generation process of the initial time mask according to an embodiment of the present application.

[0069] As shown in FIG. 4(a), first, in step 401, based on each image frame in the image subsequence, the maximum projection and the average projection of the fluorescence intensity of each pixel point are obtained. Based on the difference between the maximum projection and the average projection of the fluorescence intensity of each pixel point, the difference value projection of the fluorescence intensity of this pixel point is generated, and the difference value projection of the image subsequence is composed of the difference value projections of the fluorescence intensity of each pixel point.

[0070] Among them, the average projection can help smooth random noise and highlight continuous nerve signals, while the maximum projection helps to retain the highest intensity performance of each neuron during this time period. By comparing the differences between these two projections, the cell structure can be more clearly distinguished from the background of non-cell structures, thereby helping to improve the accuracy of neuron detection. In some other embodiments, the difference value projection of the image subsequence can also be normalized to further emphasize the edges of different neurons.

[0071] Then, in step 402, for example, by combining adaptive computer vision technology, etc., identify the morphological features of potential nerve cells contained in the difference value projection of a single image subsequence, and generate an initial time mask for the corresponding image subsequence that contains the initial potential nerve cells. Moreover, in a single initial time mask, assign the same unique ID to all pixel points contained in each initial potential nerve cell, and pixel points belonging to different initial potential nerve cells have different unique IDs.

[0072] As can be seen from Fig. 4(b), the positions and morphologies (shown as white bright blocks) of the potential nerve cells contained in the initial time mask 1, initial time mask 2, and initial time mask 3 generated based on different image subsequences spaced apart in time are not exactly the same. By processing and identifying some nerve cells in different time windows respectively, it is possible to significantly improve the accuracy of potential nerve cell recognition while reducing the computational complexity and increasing the parallelism of processing.

[0073] Next, in step 403, combine each initial time mask and generate a time mask corresponding to a single image subsequence based on the similarity degree between pixel points in a single initial time mask.

[0074] More specifically, the generation of the time mask corresponding to a single image subsequence can be divided into the following sub-steps:

[0075] In sub-step 4031, pixel points whose total number of times identified as initial potential nerve cells in each initial time mask is lower than the recognition threshold can be determined as background pixel points (i.e., positions without nerve cell structures, neuropil), and other pixel points are used as non-background pixel points, and mark the identified non-background pixel points in each time mask.

[0076] In sub-step 4032, the spatial position of non-background pixel points, the probability of being identified as non-background pixel points, and the unique ID sequence of this pixel point on each initial time mask can be used as the eigenvalue of this non-background pixel point, and construct a potential nerve cell feature matrix based on the eigenvalues of each non-background pixel point.

[0077] Then, in sub-step 4033, based on the constructed potential nerve cell feature matrix, the preset number of cell categories, and the maximum number of pixel points of a single nerve cell, cluster each non-background pixel point into each cell category, and identify the central pixel points of each cell category. By determining the similarity between each non-background pixel point and each central pixel point, identify each potential nerve cell with each central pixel point as the core pixel point, thereby generating a time mask corresponding to a single image subsequence.

[0078] In some embodiments, dimensionality reduction techniques such as principal component analysis (PCA) and t-distributed stochastic neighbor embedding (t-SNE) can be used to reduce the data dimensionality of the latent neural cell feature matrix. For example, these core pixel points can be projected onto a two-dimensional plane according to their features. On this two-dimensional plane, the closer the pixels are, the higher their similarity. In this way, the problem becomes more tractable. In some embodiments, for example, the K-means clustering algorithm in unsupervised learning can be used to classify the non-background pixel point data contained in the dimensionality-reduced latent neural cell feature matrix. Only by way of example, the preset number of cell categories can be set to 9000, and each category contains at least 10 pixel points, which can ensure that each category has sufficient representativeness and can also cover all potential neural cells as much as possible. Once the classification is completed, the central pixel points of each cell category can be determined. These central pixel points can be regarded as the representative points of each neural cell region. Then, around these central pixel points, the similarity between them and other pixel points in their respective regions is calculated, and thus a time mask corresponding to a single image subsequence can be quickly generated. According to the above method of the embodiments of the present application, compared with the method of calculating the complex Jaccard correlation coefficient on a larger scale to quantify whether two pixel points on two initial time masks tend to appear in the same cell region, the computational complexity is greatly reduced, and key information can be effectively and quickly extracted from a large amount of data, and a higher-quality time mask can be obtained.

[0079] Figure 5 Schematic diagrams showing a time mask, a spatial mask, and a cell mask according to an embodiment of the present application. In Figure 5 In the time mask shown in the left figure of, different colors represent different categories of potential neural cells with different positions and morphologies contained in the time mask.

[0080] On the basis of generating the time masks corresponding to the respective image subsequences, potential neural cells that are repeatedly identified can be further removed based on the number of common pixel points between the potential neural cells in different time masks to generate the spatial mask of the image sequence. Figure 6 Schematic diagram showing the process of generating a spatial mask based on each time mask according to an embodiment of the present application.

[0081] As Figure 6As shown, first in step 601, for example, based on the position information of two potential nerve cells from different time masks, it can be determined whether the number of common pixel points between the two potential nerve cells is greater than a repetition rate threshold (which can be set to 80% for example). When the determination result in step 601 is "yes", that is to say, the two potential nerve cells are respectively recognized in different time masks and their positions and morphologies are relatively close, then step 602 is entered for further discrimination. In some embodiments, the repetition rate threshold can be set according to experience or experimental results, and this application does not limit this.

[0082] In step 602, it is determined that the above two potential nerve cells belong to the same potential nerve cell that is repeatedly recognized. Here, it is necessary to determine all potential nerve cells in each time mask that may belong to the same potential nerve cell. Specifically, for example, a specific potential nerve cell in one time mask can be compared pairwise with potential nerve cells within a certain proximity range in other time masks and a determination result is given. And, when this time mask does not cover all potential nerve cells, the identity determination can be further carried out based on the undetermined potential nerve cells in other time masks, ultimately ensuring that all potential nerve cells separately recognized in all time masks are subjected to identity determination with other potential nerve cells.

[0083] Next, in step 603, fusion processing is performed on the same potential nerve cells that are repeatedly recognized in each time mask to generate the spatial mask of the image sequence. Specific fusion processing methods can, for example, simply use the union of all pixel points in the same potential nerve cell in each time mask as the spatial mask of this potential nerve cell. In other embodiments, it can also be further determined according to determination rules such as "2σ" the probability of each pixel point in the above union being recognized, or it can also simply set a specific number of times of being recognized (for example, at least 3 times) to remove some pixel points that are recognized less from the union. In other embodiments, since the data volume of the union of all pixel points in the same potential nerve cell is usually not large, therefore, here, correlation analysis between pixel points can be used to more reasonably and accurately determine the spatial mask of each potential nerve cell. Specifically, for example, the Jaccard correlation coefficient (also denoted as JI) between pairwise pixels in the union can be calculated. The JI threshold can be set to 0.7 for example to determine the possibility of their being recognized simultaneously; next, for a certain pixel point, if there are more than a screening threshold (typically can be set to 50%) of pixel pairs in the correlation analysis with other pixel points, then it can be used as a pixel point in this potential nerve cell. The JI threshold and the screening threshold can also be specifically set according to experience or experimental results, and this application does not limit this. Figure 5 The middle figure in is according toFigure 6 The method described above generates a spatial mask based on each temporal mask, which contains potential neurons in the entire image sequence, and uses different color blocks to identify the specific positions and morphologies of each potential neuron. The boundaries between different color blocks represent the distinctions between different potential neurons.

[0084] In the spatial mask generated according to the Figure 6 steps shown above, only the positions and morphologies of potential neurons in the entire image sequence are included. Due to the dynamics and complexity of neuron activities, these potential neurons obtained by processing relatively static images are not necessarily real neurons. Therefore, it is also necessary to further combine the image sequence after correction and noise reduction, and perform calcium signal feature analysis on each potential neuron in the spatial mask to screen the potential neurons and generate a cell mask representing the positions and morphologies of the real neurons included in the image sequence, and extract the real neuron event signals of each real neuron at each moment in the cell mask. Figure 7 FIG. 8 shows a schematic processing flow diagram for screening each potential neuron in the spatial mask and generating a cell mask according to an embodiment of the present application. The following will describe the identification process of real neurons in detail with reference to FIGS. 8(a)-8(f). FIG. 8(a) shows a schematic diagram of a typical neuron and its corresponding background region according to an embodiment of the present application.

[0085] As Figure 7 shown, in step 1, the average fluorescence intensity of the region of this potential neuron in each image frame and the average fluorescence intensity of the background region of this potential neuron are extracted, where the background region includes the surrounding region closest to this potential neuron and having the same number of pixels as this potential neuron. Taking FIG. 8(a) as an example, the part marked in light green is the neuron, and the part marked in dark green is the corresponding background region of this neuron. Further, when the average fluorescence intensity of the region of this potential neuron is higher than three times the standard deviation of the average value of the fluorescence intensity of its background region within the full time domain for 3 consecutive frames, it is determined that the fluorescence signal of this potential neuron is valid, and the average value of the fluorescence intensity of this neuron in this image frame and the fluorescence intensity of its background region is used as the fluorescence signal value of this potential neuron at the corresponding moment in this image frame.

[0086] FIG. 8(b) shows the signal curves of the neuron and its background region in the time series according to an embodiment of the present application. FIG. 8(b) takes the potential neuron in FIG. 8(a) as an example, the light green is the fluorescence signal curve of this potential neuron, the dark green is the fluorescence signal curve of its background region, and the black line represents three times the standard deviation (3σ).

[0087] In step 2, the gradient descent method is used to remove the influence of the background signal on the fluorescence signal of potential nerve cells, so as to obtain the fluorescence signal of potential nerve cells with the background signal removed. Specifically, the relationship between the fluorescence signal value of potential nerve cells, the true fluorescence signal of the potential nerve cells, and the fluorescence contribution brought by the background can be expressed as the following formula (1):

[0088] F1 = F2 + r * F0 (1)

[0089] Among them, F1 represents the fluorescence signal value of potential nerve cells, F2 represents the true fluorescence signal of the potential nerve cells, F0 represents the fluorescence intensity of the background area of the potential nerve cells, and r represents the influence degree of background fluorescence. Using the gradient descent method, based on the fluorescence signal value F1 of each potential nerve cell at each moment on the time axis and the fluorescence intensity value F0 of the background area calculated in step 1, the pollution degree of the background to the fluorescence signal of each potential nerve cell can be quickly determined, and corrected accordingly, so as to effectively remove the influence of the background signal.

[0090] In step 3, the polynomial attenuation trend of the fluorescence signal caused by the natural decay of the fluorescent protein during the imaging process is corrected to obtain the fluorescence signal of potential nerve cells with the background signal removed and decay compensated, denoted as F. Since there is a phenomenon of natural decay of the fluorescent protein during the two-photon microscope calcium imaging process, therefore, in the embodiment of the present application, it is also necessary to perform a de-linearization process on the fluorescence signal of nerve cells to correct the influence brought by this decay. Figure 8(c) shows the fluorescence signal of potential nerve cells after removing background fluorescence and de-linearization processing according to the embodiment of the present application.

[0091] In step 4, based on the baseline fluorescence signal value of the potential nerve cell, the fluorescence signal of the potential nerve cell is normalized to obtain the fluorescence signal curve of the potential nerve cell after normalization as the initial calcium signal curve of the potential nerve cell, where the baseline fluorescence signal value is determined based on the distribution of the fluorescence signal of the potential nerve cell in the full time domain.

[0092] Due to the differences in the expression levels and spatial distributions of fluorescent proteins within each nerve cell, it is necessary to normalize the signals of each potential nerve cell (Normalization). During the normalization process, it is first necessary to determine the baseline value of the fluorescent signal according to the distribution curve of the fluorescent signal, that is, the F0 value in Equation (1). Figure 8(d) shows the distribution curve of the fluorescent signal of potential nerve cells according to an embodiment of the present application. In the embodiment of the present application, considering the characteristics that neurons do not continuously excite and are in a resting state most of the time, the fluorescent signal intensity value with the highest occurrence probability (marked by a black line in Figure 8(d)) is determined as the baseline fluorescent signal value F0. As shown in Figure 8(d), the occurrence probability of the fluorescent signal intensity value is represented by kernel density estimation. Among them, kernel density estimation is a non-parametric method for estimating probability density functions. It constructs a continuous density curve by placing a smooth kernel function around the data points and performing weighted averaging. The absolute magnitude of its ordinate has no practical significance, but the abscissa corresponding to the highest or lowest point represents the data that is theoretically most concentrated or least concentrated in this distribution. As mentioned above, neurons are in a resting state most of the time throughout the time period, that is, near the baseline. Therefore, the highest point of the kernel density estimation can be used as the target of the baseline fluorescent signal value F0. Thus, the fluorescent signals of each potential nerve cell are normalized relative to F0. The normalized fluorescent signal (usually denoted as ΔF / F in the art), then ΔF / F can be calculated according to the following Equation (2):

[0093] ΔF / F=(F - F0) / F0 (2)

[0094] Where, ΔF / F is the normalized fluorescent signal, F represents the intensity of the fluorescent signal of the potential nerve cell after removing the background signal and compensating for attenuation, and F0 is the baseline fluorescent signal value.

[0095] In step 5, the initial calcium signal curve of the potential nerve cell is divided into a positive part and a negative part. A positive threshold is determined based on the positive part, and a negative threshold is determined based on the negative part; an initial calcium signal curve with a difference greater than a preset ratio between the duration when the positive part exceeds the negative threshold and the duration when the negative part exceeds the positive threshold is determined as the calcium signal curve of an effective nerve cell. Only as an example, the positive threshold can be set to three times the standard deviation (3σ) of the positive part, the negative threshold can be set to three times the standard deviation of the negative part, and the preset ratio can be set to 10, etc. The present application does not make specific limitations on this. Each potential nerve cell with a calcium signal curve of an effective nerve cell is used as a real nerve cell and the cell mask is generated, and the negative part of the calcium signal curve of each real nerve cell is set to zero to obtain the calcium signal curve of each real nerve cell. Figure 8(e) shows the calcium signal curve of the real nerve cell processed according to step 5 according to an embodiment of the present application.Figure 5 The right-middle figure shows a cell mask containing real nerve cells generated after being processed according to Step 5, where some potential nerve cells with calcium signal curves that could not be determined as valid nerve cells are removed. A large number of experimental results show that the cell mask automatically generated by the processing method of the present application is very close to the results of manual recognition and annotation. Therefore, the processing method in the present application can efficiently and accurately identify the position and morphological information of real nerve cells contained in a two-photon microscope image sequence in an automated manner.

[0096] In Step 6, in combination with the imaging frequency of the image sequence, the influence of the calcium signal of real nerve cells at previous times on the calcium signal of real nerve cells at the current time in the calcium signal curve of real nerve cells is removed, that is, deconvolution is performed to obtain the real nerve cell event signal at the current time, and the real nerve cell event signal of the real nerve cell at each time is plotted. Fig. 8(f) shows a schematic diagram of the real nerve cell event signal of a real nerve cell at each time according to an embodiment of the present application. As shown in Fig. 8(f), when the value of the real nerve cell event signal is 0, it indicates that the real nerve cell is in a resting state, and the part higher than 0 indicates that the real nerve cell is in an activated state.

[0097] As in the above Steps 1 - Step 6, the standardization of different nerve signals is achieved, and the automatic recognition of calcium transients and nerve events is realized. These processing steps are crucial for accurately interpreting nerve activities. In other embodiments, in combination with Figure 5 the cell mask of the real nerve cells in the right figure, and the real nerve cell event signal curves of each real nerve cell, more comprehensive and in-depth calculations, analyses and studies can also be carried out. The systematic and standardized analysis process provided by the embodiments of the present application provides a unified methodology for neuroscience researchers, promotes the comparability and repeatability of research results, provides strong scientific research support for biologists, and further promotes the development of the entire field in a more in-depth, refined and healthy direction.

[0098] According to an embodiment of the present application, there is also provided a processing device for an image sequence of a two-photon microscope. Figure 9 Fig. shows a partial composition schematic diagram of a processing device for an image sequence of a two-photon microscope according to an embodiment of the present application. As Figure 9 shown, the processing device 900 includes at least an interface 901 and a processor 902. Among them, the interface 901 can be configured to receive, for example, an image sequence obtained by imaging with a two-photon microscope. The image sequence includes a plurality of two-dimensional image frames corresponding to different times in the time domain. Among them, the fluorescence intensity of the pixel points in the image frame reflects the calcium signal intensity of the nerve cells in the area where they are located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cells.

[0099] In some embodiments, the interface 901 may directly receive the image sequence obtained by imaging with the two-photon microscope from the two-photon microscope, or may obtain the pre-stored image sequence obtained by imaging with the two-photon microscope from other storage media, and the present application does not limit this. In some embodiments, the interface 901 may include a network adapter, a cable connector, a serial connector, a USB connector, a parallel connector, a high-speed data transfer adapter (such as an optical fiber, USB 3.0, Thunderbolt interface, etc.), a wireless network adapter (such as a WiFi adapter), a telecommunications (3G, 4G / LTE, etc.) adapter, etc., and the present application does not limit this.

[0100] The processor 902 may be configured to execute, for example, the steps of the method for processing the image sequence of the two-photon microscope described in various embodiments of the present application.

[0101] In some embodiments, the processor 902 may be a processing device including more than one general-purpose processing device, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be more than one dedicated processing device, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a system-on-chip (SoC), etc.

[0102] In some other embodiments, the processing device 900 may also include a memory (not shown) for storing, for example, the image sequence obtained by imaging with the two-photon microscope, and executable programs that enable the processor 902 to execute various operations to implement the method for processing the image sequence of the two-photon microscope described in various embodiments of the present application, and the like, which are not listed one by one here.

[0103] The memory may be, for example, a read-only memory (ROM), a random access memory (RAM), a phase change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), an electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), a flash drive or other forms of flash memory, a cache, a register, a static memory, a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD) or other optical memory, a cassette tape or other magnetic storage device, or any other possible non-transitory medium used to store information or instructions that can be accessed by a computer device.

[0104] Embodiments of the present application also provide a processing system for an image sequence of a two-photon microscope. Figure 10 FIG. shows a schematic diagram of a partial composition of a processing system for an image sequence of a two-photon microscope according to an embodiment of the present application.

[0105] As Figure 10 shown, the processing system 1000 may include a two-photon microscope 1001 and a processing device 1002 for an image sequence of a two-photon microscope as described in various embodiments of the present application. In some embodiments, the two-photon microscope 1001 may be configured to acquire calcium imaging data and generate an image sequence including the calcium imaging data. The processing device 1002 may then be configured to receive, using an interface (not shown) included therein, the image sequence obtained by imaging with the two-photon microscope 1001, and execute, using a processor (not shown) included therein, the steps of the processing method for an image sequence of a two-photon microscope as described in various embodiments of the present application. Specific implementation manners have been described in detail above in connection with the processing method for an image sequence of a two-photon microscope and the processing device for an image sequence of a two-photon microscope according to embodiments of the present application, and will not be elaborated herein.

[0106] Embodiments of the present application also provide a non-transitory computer-readable storage medium storing a program, the program causing a processor to execute various operations of the processing method for an image sequence of a two-photon microscope as described in various embodiments of the present application.

[0107] In some embodiments, the non-transitory computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a phase change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), an electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), a flash drive or other forms of flash memory, a cache, a register, a static memory, a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD) or other optical memory, a cassette tape or other magnetic storage device, or any other possible non-transitory medium used to store information or instructions accessible by a computer device.

[0108] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application that have equivalent elements, modifications, omissions, combinations (e.g., schemes that cross various embodiments), adaptations, or alterations. The elements in the claims will be broadly interpreted based on the language employed in the claims and are not limited to the examples described in this specification or during the implementation of the present application, and the examples will be construed as non-exclusive. Thus, the specification and examples are intended to be considered only as examples, and the true scope and spirit are indicated by the full scope of the claims and their equivalents.

[0109] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of their aspects) can be used in combination with each other. For example, other embodiments can be used by those of ordinary skill in the art upon reading the above description. Additionally, in the above detailed description, various features can be grouped together to simplify the present application. This should not be construed as an intention that any feature disclosed that is not claimed is necessary for any claim. On the contrary, the subject matter of the present application can be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein as examples or embodiments into the detailed description, where each claim independently serves as a separate embodiment, and it is contemplated that these embodiments can be combined with each other in various combinations or permutations. The scope of the present application should be determined with reference to the full scope of the claims and their equivalents to which these claims are entitled.

[0110] The above embodiments are only exemplary embodiments of the present application and are not used to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions within the essence and protection scope of the present application, and such modifications or equivalent substitutions should also be regarded as falling within the protection scope of the present application.

Claims

1. A method for processing image sequences for two-photon microscopy, characterized in that: include: Receiving an image sequence obtained by imaging with the two-photon microscope, wherein the image sequence includes a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel point in the image frame reflects the calcium signal intensity of a nerve cell in the region where the pixel point is located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cell; Correct and reduce noise on each frame of the image sequence; The image sequence after correction and noise reduction processing is divided into a plurality of image subsequences corresponding to the first time length, and the time mask corresponding to each image subsequence is determined in association with the image features in each image subsequence, wherein the time mask represents the position and morphology of potential nerve cells identified based on the corresponding image subsequence; the first time length is determined according to the following steps: a plurality of candidate first time lengths are set, and a corresponding set of time masks is generated based on each candidate first time length; the first time length finally selected is determined by combining the number of nerve cells contained in the time mask corresponding to each image subsequence in each set of time masks and the similarity of nerve cells identified in adjacent time masks; removing the repeatedly identified potential nerve cells based on the number of shared pixels between the potential nerve cells in different time masks to generate a spatial mask of the image sequence, wherein the spatial mask represents the position and morphology of the potential nerve cells identified based on the complete image sequence; In combination with the image sequence after correction and noise reduction processing, calcium signal feature analysis is performed on each potential nerve cell and its background area in the spatial mask to screen the potential nerve cells and generate a cell mask representing the position and morphology of the real nerve cells contained in the image sequence, and the real nerve cell event signal of each real nerve cell in the cell mask at each moment is extracted for analysis of the activation state and activation degree of each real nerve cell.

2. The processing method according to claim 1, characterized in that: The correction and noise reduction processing of each frame image in the image sequence further includes: Each frame image in the image sequence is corrected based on a periodically updated correction template, specifically including: Starting from the first frame of the image sequence, selecting a first number of image frames at preset intervals until the entire image sequence is covered; The image frame with the largest fluorescence signal among the selected image frames is used as the reference frame; Aligning each selected image frame with the reference frame and then performing average projection to generate an initial template of the correction template; Based on the initial template, performing frame-by-frame correction starting from the first frame image in the image sequence; Based on a preset period, the correction template is updated using the corrected image sequence, and subsequent image frames are corrected using the updated correction template.

3. The processing method according to claim 2, characterized in that: The correction and noise reduction processing of each frame image in the image sequence further includes: From all the image frames of the corrected image sequence, every preset number of image frames, extract a second number of image frames as training samples to perform migration training on the deep learning network, so that the trained deep learning network can adaptively adapt to relevant parameters of the image sequence including imaging frequency, wherein the preset number and / or the second number are associated with the imaging frequency; The trained deep learning network is used to perform denoising on each frame image in the rectified image sequence.

4. The processing method according to any one of claims 1 to 3, characterized in that: Determining the time mask corresponding to each image subsequence in association with the image features in each image subsequence further includes: Based on each image frame in the image subsequence, a maximum value projection and an average value projection of the fluorescence intensity of each pixel are obtained, and a difference value projection of the fluorescence intensity of the pixel is generated based on the difference between the maximum value projection and the average value projection of the fluorescence intensity of each pixel, and the difference value projections of the fluorescence intensity of each pixel constitute the difference value projection of the image subsequence; Identify the morphological features of potential neural cells contained in the difference value projection of a single image subsequence, and generate an initial time mask containing the initially identified potential neural cells of the corresponding image subsequence, and in the single initial time mask, assign the same unique ID to all pixels contained in each initially identified potential neural cell, and pixels belonging to different initially identified potential neural cells have different unique IDs; The initial temporal masks are combined to generate a temporal mask corresponding to a single image subsequence based on the similarity between the pixels in a single initial temporal mask.

5. The processing method according to claim 4, characterized in that: The step of combining the initial time masks and generating a time mask corresponding to a single image subsequence based on the similarity between pixels in a single initial time mask further includes: The pixels whose total number of times of being identified as initially recognized potential neural cells in each initial time mask is lower than the recognition threshold are determined as background pixels, and the other pixels are regarded as non-background pixels, and the identified non-background pixels are marked in each time mask; The spatial position of the non-background pixel, the probability of being identified as a non-background pixel, and the unique ID sequence of the pixel on each initial time mask are taken as the feature value of the non-background pixel, and a potential neural cell feature matrix is ​​constructed based on the feature values ​​of each non-background pixel; Based on the constructed potential neural cell feature matrix, the preset number of cell categories and the maximum number of pixels of a single neural cell, each non-background pixel is clustered into each cell category, and the central pixel of each cell category is identified. By determining the similarity between each non-background pixel and each central pixel, each potential neural cell with each central pixel as the core pixel is identified, thereby generating a time mask corresponding to a single image subsequence.

6. The processing method according to any one of claims 1 to 3, characterized in that: The removing of the repeatedly identified potential neural cells based on the number of common pixels between the potential neural cells in different time masks to generate the spatial mask of the image sequence further comprises: When the number of common pixels between two potential neurons from different time masks is greater than the repetition rate threshold, the two potential neurons are determined to belong to the same potential neuron that is repeatedly identified; The same potential neural cells repeatedly identified in each temporal mask are fused to generate a spatial mask of the image sequence.

7. The processing method according to any one of claims 1 to 3, characterized in that: The method of combining the image sequence after correction and noise reduction, screening the potential nerve cells and generating a cell mask representing the position and morphology of the real nerve cells contained in the image sequence by analyzing the calcium signal characteristics of each potential nerve cell in the spatial mask, and extracting the real nerve cell event signal of each real nerve cell in the cell mask at each moment further includes performing the following processing steps on each potential nerve cell in the spatial mask: Step 1: extracting the average fluorescence intensity of the potential nerve cell region in each image frame, and the average fluorescence intensity of the background region of the potential nerve cell, wherein the background region includes the surrounding region that is closest to the potential nerve cell and has an equal number of pixels as the potential nerve cell; when the average fluorescence intensity of the potential nerve cell region is higher than three times the standard deviation of the average fluorescence intensity of its background region in the entire time domain for three consecutive frames, the fluorescence signal of the potential nerve cell is determined to be valid, and the average of the average fluorescence intensity of the nerve cell in the image frame and the average fluorescence intensity of its background region is used as the fluorescence signal value of the potential nerve cell at the corresponding moment of the image frame; Step 2: Using the gradient descent method to remove the influence of the background signal on the fluorescence signal of the potential nerve cells, so as to obtain the fluorescence signal of the potential nerve cells with the background signal removed; Step 3: Correct the polynomial attenuation trend of the fluorescence signal caused by the natural attenuation of the fluorescent protein during the imaging process to obtain the fluorescence signal of the potential neural cells after removing the background signal and attenuation compensation; Step 4: based on the baseline fluorescence signal value of the potential nerve cell, normalizing the fluorescence signal of the potential nerve cell to obtain a normalized fluorescence signal curve of the potential nerve cell as the initial calcium signal curve of the potential nerve cell, wherein the baseline fluorescence signal value is determined based on the distribution of the fluorescence signal of the potential nerve cell in the full time domain; Step 5: Divide the initial calcium signal curve of the potential nerve cell into a positive part and a negative part, determine the positive threshold value based on the positive part, and determine the negative threshold value based on the negative part; determine the initial calcium signal curve whose difference between the duration when the positive part exceeds the negative threshold value and the duration when the negative part exceeds the positive threshold value is greater than a preset ratio as a valid calcium signal curve of the nerve cell; regard each potential nerve cell with a valid calcium signal curve of the nerve cell as a real nerve cell and generate the cell mask, and set the negative part of the calcium signal curve of each real nerve cell to zero to obtain the calcium signal curve of each real nerve cell; Step 6: In combination with the imaging frequency of the image sequence, remove the influence of the real nerve cell calcium signal at the previous moment on the real nerve cell calcium signal at the current moment in the calcium signal curve of the real nerve cell, so as to obtain the real nerve cell event signal at the current moment, and draw the real nerve cell event signal of the real nerve cell at each moment.

8. The processing method according to claim 2 or 3, characterized in that: The correction includes correcting rigid displacement and non-rigid displacement.

9. The processing method according to claim 3, characterized in that: The step of performing noise reduction processing on each frame image in the image sequence by using the trained deep learning network specifically includes: When performing noise reduction processing on the target image frame, a second number of image frames centered on the target image frame are symmetrically selected from the image sequence and input into the trained deep learning network to obtain the target image frame after noise reduction processing.

10. A device for processing image sequences of a two-photon microscope, characterized in that: include: An interface configured to receive an image sequence obtained by imaging with the two-photon microscope, wherein the image sequence includes a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel point in the image frame reflects the calcium signal intensity of a nerve cell in the region where the pixel point is located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cell; and A processor configured to execute the method for processing an image sequence for a two-photon microscope as claimed in any one of claims 1 to 9.

11. A system for processing image sequences of a two-photon microscope, characterized in that: The processing system comprises a two-photon microscope and a processing device for image sequences of a two-photon microscope as claimed in claim 10 .

12. A non-transitory computer-readable storage medium storing a program, wherein the program causes a processor to execute the method for processing an image sequence for a two-photon microscope according to any one of claims 1 to 9.

Images