Image sequence processing method, device and system for two-photon microscope and medium

Abstract

The invention relates to an image sequence processing method, device and system for a two-photon microscope and a medium. The processing method comprises the following steps: segmenting a corrected and denoised image sequence shot by a two-photon microscope into a plurality of image subsequences, and determining a time mask representing the position and form of potential nerve cells in each subsequence; based on the number of common pixel points among the potential nerve cells in the time mask, removing the potential nerve cells which are repeatedly identified, and generating a space mask representing the positions and forms of the potential nerve cells in the whole image sequence; and in combination with the corrected and denoised image sequence, performing calcium signal feature analysis on the potential nerve cells in the spatial mask to generate a cell mask representing the positions and forms of real nerve cells, and extracting real nerve cell event signals of the real nerve cells in the cell mask. According to the method, the real nerve cells can be efficiently, automatically and more accurately identified from the two-photon image sequence, so that nerve cell activation analysis and the like are facilitated.

Description

Technical Field

[0001] The present application belongs to the field of neuroimaging, and in particular, relates to a method, device, system and medium for processing image sequences for two-photon microscopy. Background Art

[0002] In the field of neurobiology, accurately recording and analyzing the subtle changes in signal dynamics in neurons and their subcellular structures constitutes the core cornerstone of understanding the brain's encoding and reconstruction mechanisms. Calcium imaging technology, especially when combined with two-photon microscopy, has shown great potential in the study of living animal models in static and even dynamic behaviors.

[0003] However, the existing technology lacks the ability to process images / image sequences taken by two-photon microscopes. As for the accurate identification of nerve cells in two-photon calcium imaging images, one of the processing methods relies on experienced professionals to manually define the outlines of the nerve cells contained in the image. This method, which relies heavily on scarce human resources, is not only inefficient, but also cannot guarantee the consistency of the identification results, and is far from meeting the needs of technological development in this field. On the other hand, there are also some technologies in the field that attempt to identify nerve cells through image processing. Although they can identify some nerve cells and their cell structures, they often have the problem of over-detection. In other words, the number of nerve cells identified is even an order of magnitude different from the manual identification results used as the benchmark, and the algorithm has low accuracy and poor practicality.

[0004] Therefore, in this field, there is no existing technology that can accurately identify the nerve cells actually contained in the two-photon microscope image sequence and detect the neural signals associated with the nerve cells by automatically processing and analyzing the two-photon imaging data. Summary of the invention

[0005] The present application is provided to solve the above-mentioned defects existing in the prior art. A method, device, system and medium for processing an image sequence of a two-photon microscope are needed, which can accurately detect the real nerve cells and related nerve cell events contained therein by automatically processing the image sequence taken by the two-photon microscope.

[0006] According to a first scheme of the present application, a method for processing an image sequence for a two-photon microscope is provided, comprising: receiving an image sequence obtained by imaging with the two-photon microscope, the image sequence comprising a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel in the image frame reflects the calcium signal intensity of a nerve cell in the region in which it is located, and the calcium signal intensity is associated with the activation state and degree of activation of the nerve cell; correcting and denoising each frame of the image in the image sequence; dividing the image sequence after the correction and denoising into a plurality of image subsequences corresponding to a first time length, and determining a time mask corresponding to each image subsequence in association with the image features in each image subsequence, wherein the time mask represents the image features identified based on the corresponding image subsequence. The method comprises the following steps: determining the position and morphology of potential nerve cells identified by the image sequence; removing the repeatedly identified potential nerve cells based on the number of common pixels between the potential nerve cells in different time masks to generate a spatial mask of the image sequence, wherein the spatial mask represents the position and morphology of the potential nerve cells identified based on the complete image sequence; combining the image sequence after correction and noise reduction processing, performing calcium signal feature analysis on each potential nerve cell in the spatial mask to screen the potential nerve cells and generate a cell mask representing the position and morphology of the real nerve cells contained in the image sequence, and extracting the real nerve cell event signals of each real nerve cell in the cell mask at each moment, so as to be used for analyzing the activation state and activation degree of each real nerve cell.

[0007] According to a second scheme of the present application, a device for processing image sequences for a two-photon microscope is provided, the processing device comprising an interface and a processor, the interface being configured to receive an image sequence obtained by imaging with the two-photon microscope, the image sequence comprising a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel in the image frame reflects the calcium signal intensity of a nerve cell in the region in which it is located, and the calcium signal intensity is associated with the activation state and degree of activation of the nerve cell; the processor being configured to execute the method for processing image sequences for a two-photon microscope as described in the various embodiments of the present application.

[0008] According to a third embodiment of the present application, a system for processing image sequences for a two-photon microscope is provided. The processing system includes a two-photon microscope and a device for processing image sequences for a two-photon microscope as described in the various embodiments of the present application.

[0009] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing a program is provided, wherein the program enables a processor to execute the method for processing an image sequence for a two-photon microscope as described in the various embodiments of the present application.

[0010] The processing methods, devices, systems and media for image sequences of two-photon microscopes provided in various embodiments of the present application divide a long image sequence into image subsequences so as to more accurately identify potential nerve cells in the image subsequences with lower complexity and generate time masks corresponding to each image subsequence. On the basis of multiple time masks, the potential nerve cells that are repeatedly identified are removed by pixel-level processing to generate a spatial mask that characterizes the position and morphology of the potential nerve cells in the entire image sequence, and then combined with the image sequence after correction and noise reduction, the calcium signal characteristics of the potential nerve cells in the spatial mask are analyzed, thereby finally realizing the accurate identification and detection of real nerve cells and their nerve cell events at various times. The present application can provide a systematic, automated, high-precision one-stop processing flow for the analysis of calcium imaging data obtained by a two-photon microscope, which is helpful to promote the development of research in the field of neuroscience and other related fields, and has far-reaching scientific value and application prospects.

[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0012] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0014] Figure 1 A schematic flow chart showing a method for processing an image sequence for a two-photon microscope according to an embodiment of the present application.

[0015] FIG. 2( a ) is a schematic diagram showing a flow chart of performing correction processing on each image frame based on a periodically updated correction template according to an embodiment of the present application.

[0016] FIG2( b ) is a schematic diagram showing a method for generating an initial template of a correction template according to an embodiment of the present application.

[0017] FIG2( c ) is a schematic diagram showing the improvement of the clarity of nerve cells after the image frame is corrected using a correction template according to an embodiment of the present application.

[0018] FIG3( a ) shows a flowchart of performing noise reduction processing on an image frame according to an embodiment of the present application.

[0019] FIG3( b ) is a schematic diagram showing a comparison of a single image frame before and after noise reduction processing according to an embodiment of the present application.

[0020] FIG4( a ) is a schematic diagram showing steps of determining a corresponding temporal mask based on image features in each image subsequence according to an embodiment of the present application.

[0021] FIG4( b ) is a schematic diagram showing a process of generating an initial time mask according to an embodiment of the present application.

[0022] Figure 5 A schematic diagram showing a time mask, a space mask, and a cell mask according to an embodiment of the present application.

[0023] Figure 6 A schematic diagram of a process of generating a spatial mask based on various time masks according to an embodiment of the present application is shown.

[0024] Figure 7 A schematic diagram of a processing flow for screening potential neural cells in a spatial mask and generating a cell mask according to an embodiment of the present application is shown.

[0025] FIG8( a ) shows a schematic diagram of a typical nerve cell and its corresponding background region according to an embodiment of the present application.

[0026] FIG8( b ) shows a signal curve of a nerve cell and its background area in a time series according to an embodiment of the present application.

[0027] FIG8( c ) shows the fluorescence signal of potential nerve cells after background fluorescence is removed and delinearization is performed according to an embodiment of the present application.

[0028] FIG8( d ) shows a fluorescence signal distribution curve of potential nerve cells according to an embodiment of the present application.

[0029] FIG8( e ) shows a calcium signal curve of a real nerve cell according to an embodiment of the present application.

[0030] FIG8( f ) is a schematic diagram showing event signals of a real nerve cell at various moments according to an embodiment of the present application.

[0031] Fig. 9 A schematic diagram showing the partial composition of a device for processing image sequences for a two-photon microscope according to an embodiment of the present application.

[0032] Fig.10 A schematic diagram showing the partial composition of a system for processing image sequences for a two-photon microscope according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiment of the present application clearer, the technical solution of the embodiment of the present application will be clearly and completely described in conjunction with the drawings of the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, not all of the embodiments. Based on the described embodiment of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] Unless otherwise defined, the technical terms or scientific terms used in this application shall have the common meanings understood by persons with ordinary skills in the field to which this application belongs. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, but do not exclude other elements or objects.

[0035] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. Words such as "include" or "comprise" and similar words mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also covering other elements. The execution order of the various steps in the method described in this application in conjunction with the accompanying drawings is not intended to be limiting. As long as it does not affect the logical relationship between the various steps, several steps can be integrated into a single step, a single step can be decomposed into multiple steps, and the execution order of the various steps can be changed according to specific needs.

[0036] It should also be understood that the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0037] In order to keep the following description of the embodiments of the present application clear and concise, the present application omits detailed descriptions of known functions and known components.

[0038] The working principle of the two-photon microscope in the embodiment of the present application is based on two-photon excitation, which is a nonlinear optical phenomenon in which the fluorophore simultaneously absorbs two low-energy photons (usually in the near-infrared band) to achieve a transition from the ground state to the excited state and subsequently emit fluorescence. This process only occurs in the sample within the laser focus area, and due to its inherent spatial selectivity, the background noise is greatly reduced and the image contrast is enhanced. In addition, the use of a pulsed near-infrared light source not only provides a tissue penetration depth of about 1 mm or even deeper, but also effectively reduces the effects of photodamage and photobleaching, making it an ideal tool for imaging living tissues. Unlike confocal microscopy, two-photon microscopy does not rely on pinholes to limit signal acquisition outside the focal plane, so it can accurately reconstruct three-dimensional structures while maintaining high resolution. In the field of neuroscience research, when two-photon microscopy is combined with calcium ion-sensitive fluorescent probes, it has brought revolutionary progress. Calcium imaging technology achieves dynamic tracking of nerve cell activity by monitoring changes in intracellular calcium concentration caused by neuronal activity. With its excellent deep tissue imaging capabilities, two-photon microscopy allows researchers to directly observe these transient calcium dynamics in the brains of living animals, even under freely behaving conditions. This provides deep insights into the communication mechanisms between neurons and the operation modes of neural networks under physiological and pathological conditions. With single-cell and even subcellular resolution, two-photon microscopy has become a key technology platform for exploring the microscopic architecture of the brain and its functional connections.

[0039] Figure 1 A schematic flow chart showing a method for processing an image sequence for a two-photon microscope according to an embodiment of the present application.

[0040] like Figure 1 As shown, first, in step 101, an image sequence obtained by imaging with the two-photon microscope is received, wherein the image sequence includes a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of the pixel points in the image frame reflects the calcium signal intensity of the nerve cells in the area, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cells.

[0041] In the embodiments of the present application, the above-mentioned image sequence generally refers to an image sequence containing neuronal calcium imaging data generated by continuously acquiring calcium imaging data over a period of time by a two-photon microscope during a calcium imaging experiment on a sample of a test object containing nerve cells.

[0042] Then, in step 102, correction and noise reduction processing may be performed on each frame image in the image sequence.

[0043] Although two-photon microscopy has a relatively high signal-to-noise ratio, the rigid and non-rigid displacements in the image caused by the movement of the experimental object and the physiological activities of nerve cells, as well as image noise from other sources, will still have a very adverse effect on the subsequent nerve cell recognition process. Therefore, it is necessary to use a suitable algorithm to perform rigid and non-rigid displacement correction on each frame in the image sequence, and to obtain the clearest possible image frames through noise suppression, so that the subsequent nerve cell recognition process has a good image quality foundation.

[0044] Next, in step 103, a plurality of time masks are generated. First, the image sequence after correction and noise reduction processing is divided into a plurality of image subsequences corresponding to the first time length. Then, the time mask corresponding to each image subsequence is determined in association with the image features in each image subsequence. Thus, each time mask contains information on the position and morphology of potential nerve cells identified based on the corresponding image subsequence.

[0045] Considering that the number of all nerve cells contained in the image sequence taken over a long period of time is large, and they are adjacent to each other or even partially overlapped, it will undoubtedly be very difficult to try to identify each nerve cell in the image sequence as a whole. However, the inventor discovered in the practice of proposing this application that not all nerve cells are active at the same time in any given time period, that is, only some of the active nerve cells can be observed in a specific period of time. Therefore, dividing the image sequence into multiple image subsequences can ensure that different nerve cell activity patterns can be accurately captured in different time windows. Separately processing each image subsequence that only contains some nerve cells can effectively reduce the complexity of nerve cell recognition, thereby improving the recognition accuracy and overall recognition efficiency.

[0046] In step 104, the potential nerve cells that are repeatedly identified are removed based on the number of common pixels between the potential nerve cells in different time masks to generate a spatial mask of the image sequence, wherein the spatial mask represents the position and morphology of the potential nerve cells identified based on the complete image sequence.

[0047] Finally, in step 105, in combination with the image sequence after correction and noise reduction processing, the potential nerve cells are screened by analyzing the calcium signal characteristics of each potential nerve cell in the spatial mask, and a cell mask representing the position and morphology of the real nerve cells contained in the image sequence is generated, and the real nerve cell event signals of each real nerve cell in the cell mask at each moment are extracted for analysis of the activation state and activation degree of each real nerve cell.

[0048] According to the method for processing image sequences for two-photon microscopy of the embodiment of the present application, firstly, a longer image sequence is divided into image subsequences, so that potential nerve cells can be more accurately identified in different time windows with lower complexity and parallel processing; on the basis of generating a time mask corresponding to each image subsequence, the potential nerve cells that are repeatedly identified are removed by pixel-level processing to generate a spatial mask containing potential nerve cells in the entire image sequence, and then combined with the image sequence after correction and noise reduction processing, each potential nerve cell performs calcium signal feature analysis, thereby finally achieving accurate identification and detection of real nerve cells and their nerve cell events. This application provides a systematic, automated, and high-precision one-stop processing flow for image sequences taken by two-photon microscopy for the first time, which will help promote the development of research in the field of neuroscience and other related fields, and has far-reaching scientific value and application prospects.

[0049] In the images taken by the two-photon microscope, the motion displacement of the image and the presence of various Poisson noises including thermal noise and shot noise will have a very adverse effect on the subsequent recognition of nerve cells. Therefore, image correction and noise reduction are very important steps. First, each frame of the image needs to be corrected.

[0050] FIG2 (a) is a schematic diagram showing a process of correcting each image frame based on a periodically updated correction template according to an embodiment of the present application. FIG2 (b) is a schematic diagram showing a method of generating an initial template of a correction template according to an embodiment of the present application. FIG2 (c) is a schematic diagram showing improvement in the clarity of nerve cells after correcting an image frame using a correction template according to an embodiment of the present application.

[0051] As shown in FIG. 2( a ), in step 201 , starting from the first frame of the image sequence, a first number of image frames are selected at preset intervals until the entire image sequence is covered.

[0052] In some embodiments, the selection of the preset interval size mainly considers the range within which the imaging quality remains highly consistent, including but not limited to image jitter, attenuation of fluorescent proteins, etc., and can also be selected based on the characteristics of the two-photon microscope and the behavioral characteristics of the observed objects such as mice. For example, if the two-photon microscope has relatively good stability within 1 minute, then when the imaging frequency is 10 Hz, 500 frames can be selected as the preset interval.

[0053] In step 202, the image frame with the largest fluorescence signal among the selected image frames is used as a reference frame.

[0054] In step 203, each selected image frame is aligned with the reference frame and then averaged and projected to generate an initial template of the correction template. As an example only, the alignment method may be central alignment or any other applicable alignment method, which is not specifically limited in the present application.

[0055] The image sequence shown in Figure 2 (b) contains 36,000 frames, the preset interval is 500 frames, and the first number is 100 frames, that is: starting from the first frame, first select 100 frames, and then select 100 frames every 500 frames until the end of the image sequence, and select the frame with the largest fluorescence signal from the selected image frames as the reference frame. Among the six frames of images f1-f6 schematically shown in Figure 2 (b), the fluorescence of image frame f1 is relatively larger. In the specific implementation process, the fluorescence size of each image frame can be conveniently judged by naked eye observation, or by any other applicable image processing algorithm, and this application does not limit this. The IM shown in Figure 2 (b) is the initial template of the correction template generated by aligning the selected image frames with the reference frame and then projecting the average value. It can be easily seen that the fluorescence signal of the initial template IM is larger than that of any frame of the image. Therefore, the initial template determined according to the method in the embodiment of the present application summarizes the characteristics of all potential nerve cell imaging states in the entire image sequence as much as possible. Using such an initial template as the key image in the subsequent image correction and registration process is very helpful in accurately and without omission determining the effective image area for subsequent nerve cell analysis.

[0056] Then, in step 204, based on the initial template, frame-by-frame correction is performed starting from the first frame image in the image sequence. Specifically, for example, an algorithm framework based on Fast Fourier Transform (FFT) technology and upsampling + Discrete Fourier Transform (DFT) can be used to correct rigid and non-rigid displacements between images, wherein FFT is used to correct rigid displacements of images, and upsampling + DFT is used to correct non-rigid offsets in images. In other embodiments, other correction algorithms may also be used, and the present application does not impose specific restrictions on this, as long as it can correct rigid and non-rigid displacements in the image so that each frame of the image is aligned as much as possible with a common reference.

[0057] Furthermore, in step 205, the correction template can be updated based on a preset period using the corrected image sequence, so that subsequent image frames can be corrected using the updated correction template. Dynamic updating of the correction template can improve the correction effect of the image sequence more quickly.

[0058] As shown in Figure 2 (c), P1 is the average projection of all image frames before image correction using the correction template of the embodiment of the present application, and P2 is the average projection of all image frames after image correction using the correction template of the embodiment of the present application. It can be easily seen from P1 and its local enlarged image P1', as well as P2 and its local enlarged image P2' that before image correction, the average projection shows obvious blurring, while after image correction, the clarity of the average projection is significantly improved, and even the structure of each nerve cell can be clearly identified.

[0059] According to the image correction method of the embodiment of the present application, in particular, the method for generating and updating the correction template, not all nerve cells in the image sequence captured by the two-photon microscope remain active throughout the entire shooting cycle. Therefore, compared with the prior art method of only using the initial frames of the image sequence (e.g., the first 200 frames) to generate the correction template, the method in the embodiment of the present application takes into account the image offset and the natural attenuation of fluorescence during the entire imaging process, and covers the features in all image frames as much as possible, and will not miss possible changes in all time periods, such as nerve cells that become active only in the later time period. In addition, by regularly updating the correction template, the accuracy of the image features in the template is further improved, thereby making the overall correction effect of the image sequence better.

[0060] After the correction of each frame of the image is completed, it is necessary to further perform image denoising. As an example only, a deep learning network constructed based on algorithms such as DeepInterpolation can be used to perform denoising on the corrected image frames. In the neural network of the DeepInterpolation framework, an encoder-decoder with skip connections is designed to learn and process the image features of multiple adjacent frames of the central frame to be processed to learn the overall features of the image sequence in which the central frame is located. This can accurately predict the features of the central frame, thereby achieving effective suppression of the image noise of the central frame. Since such existing trained deep learning networks may only be applicable to images / image sequences with specific parameters, in an embodiment of the present application, a method for transfer learning of a trained deep learning network is particularly proposed, so that the deep learning network can adapt to the noise features in the image sequence of the present application, so as to achieve a better denoising effect.

[0061] FIG3( a ) shows a flowchart of performing noise reduction processing on an image frame according to an embodiment of the present application.

[0062] Since the existing trained deep learning networks of this type require customized network configurations for different image sequence parameters such as imaging frequencies, it is necessary to use the two-photon microscope image sequence in the present application for transfer training. Specifically, as shown in FIG3 (a), in step 301, for example, a second number of image frames can be first extracted from all the image frames of the corrected image sequence, every preset number of image frames, as training samples to perform transfer training on the deep learning network, so that the trained deep learning network can adaptively adapt to the relevant parameters of the image sequence including the imaging frequency, wherein at least one of the preset number and the second number is associated with the imaging frequency. As an example only, for example, when the imaging frequency is 10 Hz, a sample can be extracted every 100 frames, and each time a segment containing 61 frames of images is extracted, wherein the first 30 frames and the last 30 frames are used as the input of the neural network, and the intermediate frames are used as the target output. Through rapid iterative training, a neural network model optimized for a specific imaging frequency can be obtained in a relatively short time.

[0063] Next, in step 302, the trained deep learning network can be used to perform noise reduction processing on each frame image in the corrected image sequence, thereby obtaining a corrected and noise-reduced image sequence. As an example only, when the target image frame in the corrected image sequence is to be subjected to noise reduction processing, a second number of image frames centered on the target image frame can be symmetrically selected from the image sequence and input into the trained deep learning network to obtain the target image frame after noise reduction processing.

[0064] FIG3 (b) shows a schematic diagram of a comparison of a single image frame before and after noise reduction processing according to an embodiment of the present application. In FIG3 (b), image frame f31 is an image frame before noise reduction processing, and image frame f32 is an image frame after noise reduction processing. It can be clearly seen from image frame f31 and its enlarged image f31', and image frame f32 and its enlarged image f32' that after the noise reduction processing described in the embodiment of the present application, the Poisson noise (white noise) in image frame f31 is basically effectively eliminated after the noise reduction processing. At the same time, the contrast between the nerve cells and the background in image frame f32 is also significantly enhanced.

[0065] In some embodiments, the first duration of dividing the image sequence into multiple image subsequences may be preset based on experience. In other embodiments, it may also be determined in the following manner: First, multiple alternative first time lengths can be set, and a corresponding set of time masks can be generated based on each alternative first time length; then, the first time length that is finally selected can be determined by combining the number of neurons contained in the time masks corresponding to each image subsequence in each set of time masks and the degree of similarity of the neurons identified in adjacent time masks.

[0066] Just as an example, for example, when the time mask is generated according to the first alternative first duration, the number of nerve cells contained in each time mask is too large, which means that the division of the image subsequence may not effectively reduce the complexity of image processing. Therefore, the appropriate first duration should be smaller. On the other hand, for example, when the time mask is generated according to the second alternative first duration, the nerve cells contained in the adjacent time masks are very similar and the overlap is very high. This may mean that the first duration is too short, so that the same group of nerve cells is active in multiple time masks. In this case, a longer first duration should be selected, otherwise it will not only reduce the image processing efficiency, but also may cause the same group of nerve cells to produce inconsistent recognition results in different time masks, increase the amount of calculation for subsequent processing, and may also lead to a decrease in recognition accuracy. That is to say, when the first duration is appropriately selected, the number of nerve cells contained in the time mask corresponding to each image subsequence is appropriate, and the nerve cells contained in the adjacent time masks have appropriate similarity / discrimination. The specific settings of the number of nerve cells contained in a single time mask and the similarity of nerve cells in adjacent time masks are not specifically limited in this application, and can be appropriately set according to experience or experimental results.

[0067] Fig. 4 (a) is a schematic diagram showing the steps of determining the corresponding temporal mask based on the image features in each image subsequence according to an embodiment of the present application. Fig. 4 (b) is a schematic diagram showing the process of generating the initial temporal mask according to an embodiment of the present application.

[0068] As shown in FIG4(a), first in step 401, based on each image frame in the image subsequence, the maximum value projection and the average value projection of the fluorescence intensity of each pixel are obtained, and the difference value projection of the fluorescence intensity of each pixel is generated based on the difference between the maximum value projection and the average value projection of the fluorescence intensity of each pixel, and the difference value projection of the fluorescence intensity of each pixel constitutes the difference value projection of the image subsequence.

[0069] Among them, the average projection can help smooth random noise and highlight continuous neural signals, while the maximum projection helps to retain the highest intensity performance of each nerve cell in the time period. By comparing the difference between the two projections, the cell structure can be more clearly distinguished from the background of non-cell structure, which helps to improve the accuracy of nerve cell detection. In other embodiments, the difference value projection of the image subsequence can also be homogenized to further emphasize the edges of different nerve cells.

[0070] Then, in step 402, for example, adaptive computer vision technology can be combined to identify the morphological features of the potential neural cells contained in the difference value projection of a single image subsequence, and an initial time mask containing the initially identified potential neural cells of the corresponding image subsequence is generated, and in a single initial time mask, all pixel points contained in each initially identified potential neural cell are assigned the same unique ID, and the pixel points belonging to different initially identified potential neural cells have different unique IDs.

[0071] As can be seen from Figure 4 (b), the positions and morphologies (shown as white bright blocks) of the potential nerve cells contained in the initial time mask 1, initial time mask 2 and initial time mask 3 generated based on different image subsequences separated in time are not exactly the same. By processing and identifying some nerve cells separately in different time windows, the accuracy of potential nerve cell identification can be significantly improved while reducing the computational complexity and increasing the parallelism of processing.

[0072] Next, in step 403, the initial time masks are combined to generate a time mask corresponding to a single image subsequence based on the similarity between the pixels in a single initial time mask.

[0073] More specifically, the generation of the temporal mask corresponding to a single image subsequence can be divided into the following sub-steps: In sub-step 4031, the pixels whose total number of times identified as initially recognized potential neurons in each initial time mask is lower than the recognition threshold can be determined as background pixels (i.e., locations where there is no neuropil structure), and the other pixels can be determined as non-background pixels, and the identified non-background pixels are marked in each time mask.

[0074] In sub-step 4032, the spatial position of the non-background pixel, the probability of being identified as a non-background pixel, and the unique ID sequence of the pixel on each initial time mask can be used as the feature value of the non-background pixel, and a potential neural cell feature matrix is ​​constructed based on the feature values ​​of each non-background pixel.

[0075] Then, in sub-step 4033, based on the constructed potential neural cell feature matrix, the preset number of cell categories and the maximum number of pixels of a single neural cell, each non-background pixel is clustered into each cell category, and the central pixel of each cell category is identified. By determining the similarity between each non-background pixel and each central pixel, each potential neural cell with each central pixel as the core pixel is identified, thereby generating a time mask corresponding to a single image subsequence.

[0076] In some embodiments, dimensionality reduction techniques such as principal component analysis (PCA) and t-distributed neighborhood embedding (t-SNE) can be used to reduce the data dimension of the potential neural cell feature matrix. For example, these core pixels can be projected onto a two-dimensional plane according to their features. On this two-dimensional plane, the closer the distance, the higher the similarity of pixels, so that the problem becomes easier to handle. In some embodiments, for example, the K-means clustering algorithm in unsupervised learning can be used to classify the non-background pixel data contained in the reduced potential neural cell feature matrix. As an example only, for example, the number of preset cell categories can be set to 9000, and each category contains at least 10 pixels, so that each category can be sufficiently representative and can cover as many potential neural cells as possible. Once the classification is completed, the central pixel points of each cell category can be determined. These central pixel points can be regarded as representative points of each neural cell area. Then, around these central pixel points, their similarity with other pixels in their respective areas is calculated, so that the time mask corresponding to a single image subsequence can be quickly generated. According to the above method of the embodiment of the present application, compared with the method of performing complex Jaccard correlation coefficient calculation in a larger range to quantitatively measure whether two pixel points on two initial time masks tend to appear in the same cell area, the calculation complexity is greatly reduced, and it can also effectively and quickly extract key information from a large amount of data and obtain a higher quality time mask.

[0077] Figure 5 FIG. 1 is a schematic diagram showing a time mask, a space mask and a cell mask according to an embodiment of the present application. Figure 5 In the time mask shown in the left figure, different colors are used to represent different categories of potential neural cells with different positions and shapes contained in the time mask.

[0078] On the basis of generating the temporal masks corresponding to each image subsequence, the potential neural cells that are repeatedly identified can be further removed based on the number of common pixels between potential neural cells in different temporal masks to generate a spatial mask of the image sequence. Figure 6 A schematic diagram of a process of generating a spatial mask based on various time masks according to an embodiment of the present application is shown.

[0079] like Figure 6As shown, first in step 601, for example, based on the position information of two potential nerve cells from different time masks, it can be determined whether the number of common pixels between the two potential nerve cells is greater than a repetition rate threshold (for example, it can be set to 80%). If the result of the judgment in step 601 is "yes", that is, the two potential nerve cells are identified in different time masks and their positions and shapes are relatively close, then step 602 is entered for further determination. In some embodiments, the repetition rate threshold can be set based on experience or experimental results, and the present application does not limit this.

[0080] In step 602, it is determined that the two potential nerve cells mentioned above belong to the same potential nerve cell that is repeatedly identified. Here, it is necessary to determine the potential nerve cells that may belong to the same potential nerve cell in each time mask. Specifically, for example, a specific potential nerve cell in a time mask can be compared with potential nerve cells in a certain adjacent range in other time masks and a determination result can be given. In addition, if the time mask fails to cover all potential nerve cells, the undetermined potential nerve cells in other time masks can be further used as a reference to perform identity determination, and finally ensure that the potential nerve cells identified in all time masks are determined to be identical with other potential nerve cells.

[0081] Next, in step 603, the same potential neural cell repeatedly identified in each time mask is fused to generate a spatial mask of the image sequence. The specific fusion processing method can, for example, simply take the collection of all pixels in the same potential neural cell in each time mask as the spatial mask of the potential neural cell. In other embodiments, the probability of each pixel in the above collection being identified can be further determined based on a determination rule such as "2σ", or a specific number of identifications (for example, at least 3 times) can be simply set to remove some pixels that are less identified from the collection. In other embodiments, since the amount of data of the collection of all pixels in the same potential neural cell is usually not huge, correlation analysis between pixels can be used here to more reasonably and accurately determine the spatial mask of each potential neural cell. For example, the Jaccard correlation coefficient (also denoted as JI) between two pixels in the collection can be calculated, and the JI threshold can be set to 0.7, for example, to determine the possibility of simultaneous identification; next, for a certain pixel point, if there is a pixel pair that exceeds the screening threshold (typically set to 50%) in the correlation analysis with other pixel points, then it can be used as a pixel point in the potential nerve cell. The JI threshold and the screening threshold can also be specifically set based on experience or experimental results, and this application does not limit this. Figure 5 The middle picture is Figure 6 The described method generates a spatial mask based on each temporal mask, which includes potential nerve cells in the entire image sequence, and identifies the specific position and morphology of each potential nerve cell with different color blocks, and the boundaries between different color blocks represent the distinction between different potential nerve cells.

[0082] In press Figure 6 The spatial mask generated by the steps shown only contains the position and morphology of potential nerve cells in the entire image sequence. Due to the dynamic nature and complexity of nerve cell activities, these potential nerve cells obtained by processing relatively static images are not necessarily real nerve cells. Therefore, it is necessary to further combine the image sequence after correction and noise reduction, and perform calcium signal feature analysis on each potential nerve cell in the spatial mask to screen the potential nerve cells and generate a cell mask that characterizes the position and morphology of the real nerve cells contained in the image sequence, and extract the real nerve cell event signals of each real nerve cell in the cell mask at each moment. Figure 7 A schematic diagram of the processing flow of screening each potential nerve cell in the spatial mask and generating a cell mask according to an embodiment of the present application is shown. The identification process of the real nerve cell is described in detail below in conjunction with Figures 8 (a) to 8 (f). Figure 8 (a) shows a schematic diagram of a typical nerve cell and its corresponding background area according to an embodiment of the present application.

[0083] like Figure 7 As shown, in step 1, the average fluorescence intensity of the potential nerve cell area in each image frame and the average fluorescence intensity of the background area of ​​the potential nerve cell are extracted, wherein the background area includes the surrounding area that is closest to the potential nerve cell and has an equal number of pixels as the potential nerve cell. Taking Figure 8 (a) as an example, the part marked with light green is the nerve cell, and the part marked with dark green is the background area corresponding to the nerve cell. Further, when the average fluorescence intensity of the potential nerve cell area is higher than three times the standard deviation of the average fluorescence intensity of its background area in the full time domain for three consecutive frames, the fluorescence signal of the potential nerve cell is determined to be valid, and the average of the average fluorescence intensity of the nerve cell in the image frame and the average fluorescence intensity of its background area is taken as the fluorescence signal value of the potential nerve cell at the corresponding moment of the image frame.

[0084] FIG8(b) shows the signal curves of the nerve cell and its background area in the time series according to an embodiment of the present application. FIG8(b) takes the potential nerve cell in FIG8(a) as an example, the light green is the fluorescence signal curve of the potential nerve cell, the dark green is the fluorescence signal curve of its background area, and the black line represents three times the standard deviation (3σ).

[0085] In step 2, the gradient descent method is used to remove the influence of the background signal on the fluorescence signal of the potential nerve cell, so as to obtain the fluorescence signal of the potential nerve cell with the background signal removed. Specifically, the relationship between the fluorescence signal value of the potential nerve cell, the real fluorescence signal of the potential nerve cell and the fluorescence contribution brought by the background can be expressed as the following formula (1): F1=F2+r*F0(1) Among them, F1 represents the fluorescence signal value of the potential nerve cell, F2 represents the real fluorescence signal of the potential nerve cell, F0 represents the fluorescence intensity of the background area of ​​the potential nerve cell, and r represents the influence of the background fluorescence. Using the gradient descent method, based on the fluorescence signal value F1 of each potential nerve cell at each moment on the time axis and the fluorescence intensity value F0 of the background area calculated in step 1, the degree of background contamination of the fluorescence signal of each potential nerve cell can be quickly determined, and corrections can be made accordingly, thereby effectively removing the influence of the background signal.

[0086] In step 3, the polynomial attenuation trend of the fluorescence signal caused by the natural attenuation of the fluorescent protein during the imaging process is corrected to obtain the fluorescence signal of the potential nerve cell after the background signal is removed and the attenuation is compensated, which is recorded as F. Since the fluorescent protein has a natural attenuation phenomenon during the two-photon microscope calcium imaging, in the embodiment of the present application, it is also necessary to perform a delinearization process on the fluorescence signal of the nerve cell to correct the influence of this attenuation. FIG8 (c) shows the fluorescence signal of the potential nerve cell after the background fluorescence is removed and the delinearization process is performed according to the embodiment of the present application.

[0087] In step 4, based on the baseline fluorescence signal value of the potential nerve cell, the fluorescence signal of the potential nerve cell is normalized to obtain a normalized fluorescence signal curve of the potential nerve cell as the initial calcium signal curve of the potential nerve cell, wherein the baseline fluorescence signal value is determined based on the distribution of the fluorescence signal of the potential nerve cell in the entire time domain.

[0088] Since the expression level and spatial distribution of fluorescent proteins in each nerve cell are different, the signal of each potential nerve cell needs to be normalized. In the normalization process, it is first necessary to determine the baseline value of the fluorescent signal according to the distribution curve of the fluorescent signal, that is, the F0 value in formula (1). FIG8 (d) shows the distribution curve of the fluorescent signal of the potential nerve cell according to the embodiment of the present application. In the embodiment of the present application, considering that neurons will not be continuously excited and are in a resting state for most of the time, the fluorescent signal intensity value with the highest probability of occurrence (marked by a black line in FIG8 (d)) is determined as the baseline fluorescent signal value F0. As shown in FIG8 (d), the probability of occurrence of the fluorescent signal intensity value is expressed by kernel density estimation, wherein kernel density estimation is a non-parametric probability density function estimation method, which constructs a continuous density curve by placing a smooth kernel function around the data points and performing weighted averaging. The absolute size of the ordinate has no practical significance, but the abscissa corresponding to the highest or lowest point represents the most concentrated or least concentrated data in the distribution theory. As mentioned above, neurons are in a resting state for most of the time period, that is, near the baseline, so the highest point of the kernel density estimation can be used as the target of the baseline fluorescence signal value F0. Therefore, the fluorescence signal of each potential nerve cell is normalized relative to F0, and the normalized fluorescence signal (usually denoted as ΔF / F in the art) can be calculated according to the following formula (2): ΔF / F=(F-F0) / F0 (2) Wherein, ΔF / F is the normalized fluorescence signal, F represents the intensity of the fluorescence signal of the potential nerve cells after background signal removal and attenuation compensation, and F0 is the baseline fluorescence signal value.

[0089] In step 5, the initial calcium signal curve of the potential nerve cell is divided into a positive part and a negative part, a positive threshold is determined based on the positive part, and a negative threshold is determined based on the negative part; the initial calcium signal curve whose difference between the duration of the positive part exceeding the negative threshold and the duration of the negative part exceeding the positive threshold is greater than a preset ratio is determined as a valid calcium signal curve of the nerve cell. As an example only, the positive threshold can be set to three times the standard deviation (3σ) of the positive part, the negative threshold can be set to three times the standard deviation of the negative part, the preset ratio can be set to 10, and so on, which is not specifically limited in the present application. Each potential nerve cell with a valid calcium signal curve of a nerve cell is taken as a real nerve cell and the cell mask is generated, and the negative part of the calcium signal curve of each real nerve cell is set to zero to obtain the calcium signal curve of each real nerve cell. Figure 8 (e) shows the calcium signal curve of the real nerve cell after processing according to step 5 according to an embodiment of the present application. Figure 5The middle right figure shows the cell mask containing real nerve cells generated after processing according to step 5, in which some potential nerve cells whose calcium signal curves failed to be determined as valid nerve cells are removed. A large number of experimental results show that the cell mask automatically generated by the processing method of the present application is very close to the result of manual identification and manual annotation. Therefore, the processing method of the present application can efficiently and accurately identify the position and morphological information of real nerve cells contained in the two-photon microscope image sequence in an automated manner.

[0090] In step 6, in combination with the imaging frequency of the image sequence, the influence of the real nerve cell calcium signal at the previous moment on the real nerve cell calcium signal at the current moment in the calcium signal curve of the real nerve cell is removed, that is, deconvolution is performed to obtain the real nerve cell event signal at the current moment, and the real nerve cell event signal of the real nerve cell at each moment is plotted. FIG8 (f) shows a schematic diagram of the real nerve cell event signal of the real nerve cell at each moment according to an embodiment of the present application. As shown in FIG8 (f), when the value of the real nerve cell event signal is 0, it indicates that the real nerve cell is in a resting state, and the part higher than 0 indicates that the real nerve cell is in an activated state.

[0091] As described in steps 1 to 6 above, standardization of different neural signals and automatic identification of calcium transients and neural events are achieved. These processing steps are crucial for accurately interpreting neural activity. Figure 5 The cell masks of the real neurons in the right figure, as well as the real neuronal cell event signal curves of each real neuronal cell, can also be calculated, analyzed and studied from more angles and in more depth. The systematic and standardized analysis process provided in the embodiments of the present application provides a unified methodology for neuroscience researchers, promotes the comparability and repeatability of research results, and provides strong scientific research support for biologists, thereby promoting the development of the entire field in a more in-depth, refined and healthy direction.

[0092] According to an embodiment of the present application, a device for processing an image sequence of a two-photon microscope is also provided. Fig. 9 FIG. 2 is a schematic diagram showing a partial composition of an image sequence processing device for a two-photon microscope according to an embodiment of the present application. Fig. 9 As shown, the processing device 900 at least includes an interface 901 and a processor 902. The interface 901 can be configured to receive an image sequence obtained by two-photon microscopy imaging, wherein the image sequence includes a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of the pixel points in the image frame reflects the calcium signal intensity of the nerve cells in the region, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cells.

[0093] In some embodiments, the interface 901 may directly receive an image sequence obtained by imaging the two-photon microscope from the two-photon microscope, or may obtain a pre-stored image sequence obtained by imaging the two-photon microscope from other storage media, and the present application does not limit this. In some embodiments, the interface 901 may include a network adapter, a cable connector, a serial connector, a USB connector, a parallel connector, a high-speed data transmission adapter (such as an optical fiber, USB 3.0, a thunderbolt interface, etc.), a wireless network adapter (such as a WiFi adapter), a telecommunications (3G, 4G / LTE, etc.) adapter, etc., and the present application does not limit this.

[0094] The processor 902 may be configured, for example, to execute the steps of the method for processing an image sequence for a two-photon microscope as described in various embodiments of the present application.

[0095] In some embodiments, the processor 902 may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0096] In some other embodiments, the processing device 900 may include a memory (not shown) for storing image sequences such as those obtained by the two-photon microscope imaging, and an executable program that enables the processor 902 to execute various operations of the image sequence processing method for two-photon microscope described in various embodiments of the present application, which are not listed one by one here.

[0097] The memory may be, for example, read-only memory (ROM), random access memory (RAM), phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), flash disks or other forms of flash memory, cache, registers, static memory, compact disk read-only memory (CD-ROM), digital versatile disks (DVD) or other optical storage, cassettes or other magnetic storage devices, or any other possible non-transitory medium used to store information or instructions that can be accessed by a computer device.

[0098] According to an embodiment of the present application, a system for processing image sequences of a two-photon microscope is also provided. Fig.10 A schematic diagram showing the partial composition of a system for processing image sequences for a two-photon microscope according to an embodiment of the present application.

[0099] like Fig.10 As shown, the processing system 1000 may include a two-photon microscope 1001 and a processing device 1002 for an image sequence for a two-photon microscope as described in various embodiments of the present application. In some embodiments, the two-photon microscope 1001 may be configured to acquire calcium imaging data and generate an image sequence containing calcium imaging data. The processing device 1002 may be configured to receive an image sequence obtained by imaging the two-photon microscope 1001 using an interface (not shown) included therein, and to perform the steps of the processing method for an image sequence for a two-photon microscope as described in various embodiments of the present application using a processor (not shown) included therein. The specific implementation has been described in detail in the foregoing text in conjunction with the processing method for an image sequence for a two-photon microscope and the processing device for an image sequence for a two-photon microscope according to an embodiment of the present application, and will not be repeated here.

[0100] According to an embodiment of the present application, a non-transitory computer-readable storage medium storing a program is also provided, wherein the program enables a processor to execute various operations of the method for processing an image sequence for a two-photon microscope as described in various embodiments of the present application.

[0101] In some embodiments, the non-transitory computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a phase-change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), an electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), a flash disk or other form of flash memory, a cache, a register, a static memory, a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD) or other optical storage, a cassette tape or other magnetic storage device, or any other possible non-transitory medium used to store information or instructions that can be accessed by a computer device.

[0102] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the execution of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the claims and the full scope of their equivalents.

[0103] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more schemes thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein into the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the claims and the full scope of equivalent forms granted by these claims.

[0104] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for processing image sequences for two-photon microscopy, characterized in that: include: Receiving an image sequence obtained by imaging with the two-photon microscope, wherein the image sequence includes a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel point in the image frame reflects the calcium signal intensity of a nerve cell in the region where the pixel point is located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cell; Correct and reduce noise on each frame of the image sequence; The image sequence after correction and noise reduction is divided into a plurality of image subsequences corresponding to the first time length, and a time mask corresponding to each image subsequence is determined in association with image features in each image subsequence, wherein the time mask represents the position and morphology of potential nerve cells identified based on the corresponding image subsequence; removing the repeatedly identified potential nerve cells based on the number of shared pixels between the potential nerve cells in different time masks to generate a spatial mask of the image sequence, wherein the spatial mask represents the position and morphology of the potential nerve cells identified based on the complete image sequence; In combination with the image sequence after correction and noise reduction processing, the potential nerve cells are screened by analyzing the calcium signal characteristics of each potential nerve cell in the spatial mask, and a cell mask representing the position and morphology of the real nerve cells contained in the image sequence is generated, and the real nerve cell event signals of each real nerve cell in the cell mask at each moment are extracted for analysis of the activation state and activation degree of each real nerve cell.

2. The processing method according to claim 1, characterized in that: The correction and noise reduction processing of each frame image in the image sequence further includes: Each frame image in the image sequence is corrected based on a periodically updated correction template, specifically including: Starting from the first frame of the image sequence, selecting a first number of image frames at preset intervals until the entire image sequence is covered; The image frame with the largest fluorescence signal among the selected image frames is used as the reference frame; Aligning each selected image frame with the reference frame and then performing average projection to generate an initial template of the correction template; Based on the initial template, performing frame-by-frame correction starting from the first frame image in the image sequence; Based on a preset period, the correction template is updated using the corrected image sequence, and subsequent image frames are corrected using the updated correction template.

3. The processing method according to claim 2, characterized in that: The correction and noise reduction processing of each frame image in the image sequence further includes: From all the image frames of the corrected image sequence, every preset number of image frames, extract a second number of image frames as training samples to perform migration training on the deep learning network, so that the trained deep learning network can adaptively adapt to relevant parameters of the image sequence including imaging frequency, wherein the preset number and / or the second number are associated with the imaging frequency; The trained deep learning network is used to perform denoising on each frame image in the rectified image sequence.

4. The processing method according to any one of claims 1 to 3, characterized in that: The first duration is determined according to the following steps: Setting a plurality of candidate first durations, and generating a corresponding set of time masks based on each candidate first duration; The first time length finally selected is determined by combining the number of nerve cells contained in the time mask corresponding to each image subsequence in each group of time masks and the similarity of nerve cells identified in adjacent time masks.

5. The processing method according to any one of claims 1 to 3, characterized in that: Determining the time mask corresponding to each image subsequence in association with the image features in each image subsequence further includes: Based on each image frame in the image subsequence, a maximum value projection and an average value projection of the fluorescence intensity of each pixel are obtained, and a difference value projection of the fluorescence intensity of the pixel is generated based on the difference between the maximum value projection and the average value projection of the fluorescence intensity of each pixel, and the difference value projections of the fluorescence intensity of each pixel constitute the difference value projection of the image subsequence; Identify the morphological features of potential neural cells contained in the difference value projection of a single image subsequence, and generate an initial time mask containing the initially identified potential neural cells of the corresponding image subsequence, and in the single initial time mask, assign the same unique ID to all pixels contained in each initially identified potential neural cell, and pixels belonging to different initially identified potential neural cells have different unique IDs; The initial temporal masks are combined to generate a temporal mask corresponding to a single image subsequence based on the similarity between the pixels in a single initial temporal mask.

6. The processing method according to claim 5, characterized in that: The step of combining the initial time masks and generating a time mask corresponding to a single image subsequence based on the similarity between pixels in a single initial time mask further includes: The pixels whose total number of times of being identified as initially recognized potential neural cells in each initial time mask is lower than the recognition threshold are determined as background pixels, and the other pixels are regarded as non-background pixels, and the identified non-background pixels are marked in each time mask; The spatial position of the non-background pixel, the probability of being identified as a non-background pixel, and the unique ID sequence of the pixel on each initial time mask are taken as the feature value of the non-background pixel, and a potential neural cell feature matrix is ​​constructed based on the feature values ​​of each non-background pixel; Based on the constructed potential neural cell feature matrix, the preset number of cell categories and the maximum number of pixels of a single neural cell, each non-background pixel is clustered into each cell category, and the central pixel of each cell category is identified. By determining the similarity between each non-background pixel and each central pixel, each potential neural cell with each central pixel as the core pixel is identified, thereby generating a time mask corresponding to a single image subsequence.

7. The processing method according to any one of claims 1 to 3, characterized in that: The removing of the repeatedly identified potential neural cells based on the number of common pixels between the potential neural cells in different time masks to generate the spatial mask of the image sequence further comprises: When the number of common pixels between two potential neurons from different time masks is greater than the repetition rate threshold, the two potential neurons are determined to belong to the same potential neuron that is repeatedly identified; The same potential neural cells repeatedly identified in each temporal mask are fused to generate a spatial mask of the image sequence.

8. The processing method according to any one of claims 1 to 3, characterized in that: The method of combining the image sequence after correction and noise reduction, screening the potential nerve cells and generating a cell mask representing the position and morphology of the real nerve cells contained in the image sequence by analyzing the calcium signal characteristics of each potential nerve cell in the spatial mask, and extracting the real nerve cell event signal of each real nerve cell in the cell mask at each moment further includes performing the following processing steps on each potential nerve cell in the spatial mask: Step 1: extracting the average fluorescence intensity of the potential nerve cell region in each image frame, and the average fluorescence intensity of the background region of the potential nerve cell, wherein the background region includes the surrounding region that is closest to the potential nerve cell and has an equal number of pixels as the potential nerve cell; when the average fluorescence intensity of the potential nerve cell region is higher than three times the standard deviation of the average fluorescence intensity of its background region in the entire time domain for three consecutive frames, the fluorescence signal of the potential nerve cell is determined to be valid, and the average of the average fluorescence intensity of the nerve cell in the image frame and the average fluorescence intensity of its background region is used as the fluorescence signal value of the potential nerve cell at the corresponding moment of the image frame; Step 2: Using the gradient descent method to remove the influence of the background signal on the fluorescence signal of the potential nerve cells, so as to obtain the fluorescence signal of the potential nerve cells with the background signal removed; Step 3: Correct the polynomial attenuation trend of the fluorescence signal caused by the natural attenuation of the fluorescent protein during the imaging process to obtain the fluorescence signal of the potential neural cells after removing the background signal and attenuation compensation; Step 4: based on the baseline fluorescence signal value of the potential nerve cell, normalizing the fluorescence signal of the potential nerve cell to obtain a normalized fluorescence signal curve of the potential nerve cell as the initial calcium signal curve of the potential nerve cell, wherein the baseline fluorescence signal value is determined based on the distribution of the fluorescence signal of the potential nerve cell in the full time domain; Step 5: Divide the initial calcium signal curve of the potential nerve cell into a positive part and a negative part, determine the positive threshold value based on the positive part, and determine the negative threshold value based on the negative part; determine the initial calcium signal curve whose difference between the duration when the positive part exceeds the negative threshold value and the duration when the negative part exceeds the positive threshold value is greater than a preset ratio as a valid calcium signal curve of the nerve cell; regard each potential nerve cell with a valid calcium signal curve of the nerve cell as a real nerve cell and generate the cell mask, and set the negative part of the calcium signal curve of each real nerve cell to zero to obtain the calcium signal curve of each real nerve cell; Step 6: In combination with the imaging frequency of the image sequence, remove the influence of the real nerve cell calcium signal at the previous moment on the real nerve cell calcium signal at the current moment in the calcium signal curve of the real nerve cell, so as to obtain the real nerve cell event signal at the current moment, and draw the real nerve cell event signal of the real nerve cell at each moment.

9. The processing method according to claim 2 or 3, characterized in that: The correction includes correcting rigid displacement and non-rigid displacement.

10. The processing method according to claim 3, characterized in that: The step of performing noise reduction processing on each frame image in the image sequence by using the trained deep learning network specifically includes: When performing noise reduction processing on the target image frame, a second number of image frames centered on the target image frame are symmetrically selected from the image sequence and input into the trained deep learning network to obtain the target image frame after noise reduction processing.

11. A device for processing image sequences of a two-photon microscope, characterized in that: include: An interface configured to receive an image sequence obtained by imaging with the two-photon microscope, wherein the image sequence includes a plurality of two-dimensional image frames corresponding to different moments in the time domain, wherein the fluorescence intensity of a pixel point in the image frame reflects the calcium signal intensity of a nerve cell in the region where the pixel point is located, and the calcium signal intensity is associated with the activation state and activation degree of the nerve cell; and A processor configured to execute the method for processing an image sequence for a two-photon microscope as claimed in any one of claims 1 to 10.

12. A system for processing image sequences for a two-photon microscope, characterized in that: The processing system comprises a two-photon microscope and a processing device for image sequences of a two-photon microscope as claimed in claim 11 .

13. A non-transitory computer-readable storage medium storing a program, wherein the program causes a processor to execute the method for processing an image sequence for a two-photon microscope according to any one of claims 1 to 10.

Images