A single-molecule image positioning and processing design platform based on PyQt

Abstract

The application discloses a single-molecule image positioning and processing design platform based on PyQt, which comprises an image preprocessing module, an image positioning module, an image post-processing module and an interactive interface layer built based on PyQt, and the modules are connected with each other and work cooperatively, the target single-molecule image from the interactive interface layer is preprocessed by the image preprocessing module to obtain a processed image, the pre-trained single-molecule positioning deep learning network model is called by the image positioning module to perform forward inference on the processed image, single-molecule fluorescent point positioning and super-resolution reconstruction are completed, and a positioning image is obtained; the target single-molecule image is scaled to the same size as the positioning image by the image post-processing module, and then the positioning result of the single-molecule fluorescent point corresponding to the positioning image is superimposed on the scaled target single-molecule image to obtain a positioning comparison image, and the integration of single-molecule image reading, preprocessing, deep learning positioning, post-processing and result visualization is realized.

Description

Technical Field

[0001] This application relates to the interdisciplinary field of single-molecule localization microscopy imaging technology and computer vision technology, and in particular to a single-molecule image localization and processing design platform based on PyQt. Background Technology

[0002] Single-molecule localization microscopy (SMLM) techniques, including STORM, PALM, and STED, have become a core technology for resolving the spatial distribution and interactions of biomolecules in life science research due to their nanometer-level spatial resolution. They are widely used in cell biology, molecular biology, and medical diagnostics. However, during single-molecule imaging, factors such as the low number of photons from fluorescent molecules, strong background noise interference, and inherent noise in imaging equipment often cause the single-molecule signal to be overwhelmed by noise, leading to reduced single-molecule localization accuracy and low localization efficiency.

[0003] Meanwhile, single-molecule imaging data are mostly stored in proprietary formats such as TIFF and FITS, which have poor compatibility and cause great inconvenience to subsequent data analysis. Currently, single-molecule image processing and localization mainly rely on scattered tools and algorithms: on the one hand, preprocessing operations such as image reading, format conversion, and data type conversion need to be manually completed using software such as ImageJ and MATLAB, which is cumbersome, inefficient, and lacks a unified interactive interface, making it unfriendly to non-technical personnel; on the other hand, the deployment and invocation of single-molecule localization algorithms, especially deep learning-based localization algorithms, often require professional programming skills, making it difficult to integrate with preprocessing and post-processing workflows, resulting in a fragmented single-molecule localization process that cannot achieve automated and efficient processing.

[0004] Although existing deep learning algorithms for single-molecule localization exist and can effectively improve the accuracy of single-molecule localization, these algorithms mostly exist in the form of independent program code and lack a convenient visual interactive platform. They cannot realize the integrated operation of the entire process of image reading, preprocessing, localization, postprocessing and result output, which limits the promotion and application of such excellent localization algorithms and cannot meet the actual needs of researchers for convenience and efficiency in single-molecule image processing.

[0005] Therefore, developing a single-molecule image localization and processing platform that integrates preprocessing, deep learning localization, and post-processing functions, and has a user-friendly interface, has become an urgent technical problem to be solved in the field of single-molecule localization. Summary of the Invention

[0006] Therefore, based on the shortcomings of the prior art, this invention combines PyQt visualization interaction technology with existing deep learning single-molecule localization algorithms to provide a PyQt-based single-molecule image localization and processing design platform. This platform addresses the technical pain points of fragmented processes, cumbersome operations, and lack of an integrated interactive platform in the prior art, achieving automation, visualization, and efficiency of the entire single-molecule image processing process. It promotes the practical application of deep learning localization algorithms and meets the actual needs of researchers.

[0007] The main objective of this application is to provide a PyQt-based single-molecule image localization and processing design platform, including:

[0008] Image preprocessing module, image localization module, image postprocessing module, and interactive interface layer built with PyQt;

[0009] The interactive interface layer is communicatively connected to the image preprocessing module, the image localization module, and the image postprocessing module, respectively, and is used to respond to the user's selection operation, identify and read the target single molecule image, and visually display the target single molecule image and the corresponding processed image, localization image, and localization comparison image.

[0010] The image preprocessing module is used to preprocess the target single-molecule image from the interactive interface layer to obtain the processed image;

[0011] The image localization module is used to call a pre-trained single-molecule localization deep learning network model to perform forward inference on the processed image, complete single-molecule fluorescence point localization and super-resolution reconstruction, and obtain a localization image.

[0012] The image post-processing module includes a positioning display unit, which is used to first scale the target single-molecule image to the same size as the positioning image, and then superimpose the single-molecule fluorescence point positioning result corresponding to the positioning image onto the scaled target single-molecule image to obtain a positioning comparison image.

[0013] In one embodiment, the interactive interface layer includes a file selection control, a parameter setting control, a result display control, and a function triggering control;

[0014] The file selection control is configured to obtain the storage path information of the target single-molecule image file based on the user's file selection operation;

[0015] The parameter setting control is configured to obtain the processing configuration parameters of the target single-molecule image based on the user's parameter configuration operation.

[0016] The result display control is configured to perform visual rendering and display of the received target single-molecule image, processed image, localization image, and localization comparison image;

[0017] The function trigger control is configured to generate standardized processing instructions based on the user's function trigger operation and send them to the corresponding image preprocessing module, image positioning module, or image postprocessing module.

[0018] In one embodiment, the image preprocessing module is specifically configured as follows:

[0019] Used to extract the file extension and automatically identify whether the target single-molecule image file is in TIFF or FITS format based on the storage path information of the target single-molecule image file obtained from the file selection control;

[0020] The system calls the corresponding preset parsing library to complete the reading and writing of image data and performs bidirectional conversion between TIFF and FITS formats. At the same time, it performs adaptive matching conversion of pixel data types on the read image data to obtain basic image data that meets the processing requirements of the image positioning module.

[0021] In one embodiment, the image preprocessing module is further configured to perform pixel value normalization processing on the basic image data, adjust the basic image data to the dimension and format required by the input of the pre-trained single-molecule localization deep learning network model, and output standardized processed image data to the image localization module.

[0022] In one embodiment, the pre-trained single-molecule localization deep learning network model is a feature fusion network based on an encoder-decoder structure, with skip connections and residual connections between the encoder and decoder. The single-molecule localization deep learning network model is configured as follows:

[0023] Multi-scale deep features are extracted from the processed image data using an encoder;

[0024] The adaptive feature fusion module performs adaptive weighted fusion of multi-scale deep features to obtain a fused feature map.

[0025] The decoder, based on the fused feature map, completes the localization feature recovery and single-molecule fluorescence point probability mapping to obtain a single-molecule fluorescence point probability localization map as the localization image.

[0026] In one embodiment, the decoder is configured to perform a four-level 2x upsampling operation on the fused feature map, cumulatively complete 8x upsampling super-resolution reconstruction, and simultaneously complete the probability mapping of single-molecule fluorescent dots to obtain a single-molecule fluorescent dot probability localization map.

[0027] In one embodiment, the image post-processing module further includes a frame overlay unit;

[0028] The frame overlay unit is configured to traverse all single-frame localization images within the range of the user-selected frame number in the processing configuration parameters, and perform element-by-element overlay fusion and normalization processing on the multi-frame single-molecule localization results to filter out single-frame localization noise and obtain optimized localization images.

[0029] In one embodiment, the image post-processing module further includes a location list output unit;

[0030] The location list output unit is configured to extract pixel coordinate information of single-molecule fluorescent points from the location image, combine it with the original size parameters of the target single-molecule image, convert the extracted pixel coordinates into standard physical coordinates that match the original image size, and export a location list file in txt or csv format.

[0031] In one embodiment, it further includes:

[0032] The multi-threaded scheduling and communication module is configured to allocate interactive operations of the interactive interface layer to the main thread for execution through a preset multi-threaded and PyQt signal-slot mechanism, and allocate the algorithm calculation tasks of the image preprocessing module, image localization module and image postprocessing module to independent sub-threads for execution, so as to complete the standardized processing instruction transmission and data interaction between the main thread and the sub-threads.

[0033] In one embodiment, it further includes:

[0034] The resource management and exception handling module is configured to, for multi-frame target single-molecule image data, first read file metadata to obtain the total number of frames and basic image parameters, respond to the user's frame operation command and load only the image data of the corresponding frame into memory, and automatically release memory resources for frame data that have completed the operation and exceeded the preset idle time.

[0035] Furthermore, in the execution flow corresponding to the image preprocessing module, image positioning module, image postprocessing module, and interactive interface layer, preset exception capture nodes are set to capture exception scenarios and extract exception types, and the corresponding generated exception prompt information is sent to the result display control for display.

[0036] This application provides a PyQt-based single-molecule image localization and processing design platform that integrates functions such as single-molecule image reading, preprocessing, deep learning localization, post-processing, and result output. Through a visual interactive interface built with PyQt, it solves the technical pain points of existing deep learning localization algorithms, such as cumbersome deployment, complex operation, and fragmented processes. It lowers the technical threshold for single-molecule localization operations, improves the accuracy and resolution of single-molecule localization, and is suitable for research needs in fields such as biomedicine and life sciences. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure provided in one embodiment of the single-molecule image localization and processing design platform based on PyQt in this application;

[0040] Figure 2 A schematic diagram of the interactive interface layer provided for an embodiment of the single-molecule image localization and processing design platform based on PyQt in this application;

[0041] Figure 3 A schematic diagram of the structure of a single-molecule localization deep learning network model provided in an embodiment of the single-molecule image localization and processing design platform based on PyQt in this application;

[0042] Figure 4 Image reading and display results provided for an embodiment of the single-molecule image localization and processing design platform based on PyQt in this application;

[0043] Figure 5 The model localization output image provided for the embodiment of the single-molecule image localization and processing design platform based on PyQt in this application;

[0044] Figure 6 A comparison of localization data and the original image provided for an embodiment of the single-molecule image localization and processing design platform based on PyQt in this application;

[0045] Figure 7 This is a result image of superimposing model outputs within a certain range, provided as an embodiment of the single-molecule image localization and processing design platform based on PyQt in this application.

[0046] Figure 8 The image shows the output results of the localization list provided for an embodiment of the single-molecule image localization and processing design platform based on PyQt in this application.

[0047] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0049] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0050] This application provides an embodiment of a PyQt-based single-molecule image localization and processing design platform, such as... Figure 1 As shown, it includes:

[0051] Image preprocessing module, image localization module, image postprocessing module, and interactive interface layer built with PyQt;

[0052] The interactive interface layer is communicatively connected to the image preprocessing module, the image localization module, and the image postprocessing module, respectively, and is used to respond to the user's selection operation, identify and read the target single molecule image, and visually display the target single molecule image and the corresponding processed image, localization image, and localization comparison image.

[0053] The image preprocessing module is used to preprocess the target single-molecule image from the interactive interface layer to obtain the processed image;

[0054] The image localization module is used to call a pre-trained single-molecule localization deep learning network model to perform forward inference on the processed image, complete single-molecule fluorescence point localization and super-resolution reconstruction, and obtain a localization image.

[0055] The image post-processing module includes a positioning display unit, which is used to first scale the target single-molecule image to the same size as the positioning image, and then superimpose the single-molecule fluorescence point positioning result corresponding to the positioning image onto the scaled target single-molecule image to obtain a positioning comparison image.

[0056] Specifically, in this embodiment, the platform is constructed using a combination of a main window and multiple area controls, developed based on the PyQt5 or PyQt6 framework, and can run on mainstream operating systems such as Windows, Linux, and macOS.

[0057] The interactive interface layer, image preprocessing module, image positioning module, and image postprocessing module achieve data interaction through standardized data interfaces and transmit instructions through the PyQt signal slot mechanism, eliminating the need for manual intervention by the user in connecting the modules and achieving fully automated workflow.

[0058] In this embodiment and subsequent embodiments, the target single-molecule image refers to: microscopic image data containing single-molecule fluorescence signals acquired by a single-molecule positioning microscope and read into memory, including single-frame images and multi-frame time-series images;

[0059] The target single-molecule image file refers to the original packaged file corresponding to the single-molecule image stored on the device's disk;

[0060] The processed image refers to the image data that has been preprocessed and is adapted to the image positioning module's input requirements.

[0061] The localization image refers to a super-resolution image containing the localization information of single-molecule fluorescent dots after inference by a single-molecule localization deep learning network model.

[0062] The location comparison image refers to the visualized image obtained by overlaying the location result with the original image.

[0063] Furthermore, in an optional implementation, the interactive interface layer includes a file selection control, a parameter setting control, a result display control, and a function triggering control;

[0064] The file selection control is configured to obtain the storage path information of the target single-molecule image file based on the user's file selection operation;

[0065] The parameter setting control is configured to obtain the processing configuration parameters of the target single-molecule image based on the user's parameter configuration operation.

[0066] The result display control is configured to perform visual rendering and display of the received target single-molecule image, processed image, localization image, and localization comparison image;

[0067] The function trigger control is configured to generate standardized processing instructions based on the user's function trigger operation and send them to the corresponding image preprocessing module, image positioning module, or image postprocessing module.

[0068] Specifically, in this embodiment, such as Figure 2 As shown, the four main controls of the interactive interface layer are arranged in the main window of the platform according to their functional areas. The file selection control and parameter setting control are arranged in the preprocessing bar at the top of the main window, the result display control occupies the core area in the middle of the main window, and the function trigger control is arranged at the bottom of the main window. The overall layout conforms to the operating habits of researchers.

[0069] The file selection control uses the getOpenFileName function of the QFileDialo class provided by the PyQt framework to enable interactive selection of target single-molecule image files. When triggered, the pop-up file selection dialog box has preset file filtering rules to only display single-molecule image files with the extensions .tif, .tiff, and .fits, in order to avoid users accidentally selecting non-target files. Finally, the complete storage path information of the target single-molecule image file is obtained.

[0070] The parameter setting controls include a model selection dropdown, a start frame input box, an end frame input box, and a QSlider slider control in the PyQt framework. The model selection dropdown includes preset options for self-developed single-molecule localization deep learning models such as WG-storm. The start and end frame input boxes are used to limit the range of processing frames for multiple images. The QSlider slider control is used to control the frame sequence of the target single-molecule image, switch the current frame, and obtain the frame number. Its value range is automatically matched with the total number of frames of the target single-molecule image. All parameters obtained by the above controls are uniformly the processing configuration parameters for the target single-molecule image.

[0071] The results display control is implemented based on PyQt's QGraphicsView control, which supports adaptive scaling and dragging of images, can fully display single-molecule images of different sizes, and supports simultaneous comparison of the original image and the localization result image.

[0072] The function trigger controls include an open image button, a preprocessing button, a predictive positioning button, a display positioning button, an output positioning button, a frame overlay button, and a save image button. Each button is bound to an independent click signal. After being clicked, the corresponding standardized processing instruction is generated through the PyQt signal slot mechanism and sent to the corresponding functional module.

[0073] Specifically, standardized processing instructions refer to predefined instruction codes that can be uniquely identified and executed by the corresponding functional modules. Each function button corresponds to a unique standardized processing instruction, ensuring the accuracy of instruction transmission.

[0074] By adopting this implementation method, the collaborative work of the four controls enables precise linkage between user operations and platform functions, providing users with a unified, convenient, and intuitive visual interaction entry point. Users do not need to manually input code commands, which further reduces the operating threshold of the platform and improves the user experience.

[0075] Furthermore, in one embodiment, the image preprocessing module is specifically configured as follows:

[0076] Used to extract the file extension and automatically identify whether the target single-molecule image file is in TIFF or FITS format based on the storage path information of the target single-molecule image file obtained from the file selection control;

[0077] The system calls the corresponding preset parsing library to complete the reading and writing of image data and performs bidirectional conversion between TIFF and FITS formats. At the same time, it performs adaptive matching conversion of pixel data types on the read image data to obtain basic image data that meets the processing requirements of the image positioning module.

[0078] Specifically, in this embodiment, TIFF and FITS formats are the two mainstream file encapsulation formats output by single-molecule localization microscope equipment. The platform supports reading, writing, and bidirectional conversion between the two formats. Users can select the target encapsulation format to complete the conversion and export by clicking the Data transform button in the preprocessing menu bar of the interactive interface.

[0079] After the image preprocessing module obtains the storage path information of the target single-molecule image file, the file extension is obtained through the Python string truncation function. If the extension is .tif or .tiff, it is determined to be a TIFF format file, and the imread and imwrite functions of the tifffile library are called to complete the file reading and writing.

[0080] If the file extension is .fits or .fit, it is determined to be a FITS format file, and the open and writeto functions of the astropy.io.fits library are called to complete the file reading and writing.

[0081] Adaptive matching conversion of pixel data types refers to:

[0082] Image pixel data of different numerical types such as uint8, uint16, and float32 are uniformly converted into a standard numerical type compatible with subsequent full-process processing (including image rendering, deep learning model inference, and post-processing analysis). After the conversion, the basic image data is obtained.

[0083] Through the above technical means, automatic recognition, reading, writing and bidirectional conversion of TIFF and FITS format files were realized. At the same time, the universal adaptation of image pixel data types was completed, solving the problems of incompatibility of output file formats and mismatch of pixel data types of different single-molecule imaging devices. This enables the processed data to be compatible with the processing requirements of all stages of the process, greatly improving the platform's versatility and data compatibility.

[0084] Furthermore, in one embodiment, the image preprocessing module is also configured to perform pixel value normalization processing on the basic image data, adjust the basic image data to the dimension and format required by the input of the pre-trained single-molecule localization deep learning network model, and output standardized processed image data to the image localization module.

[0085] Specifically, in this embodiment, pixel value normalization processing refers to linearly mapping the pixel values ​​of the basic image data after the pixel data type adaptive matching conversion to the [0,1] interval, thereby eliminating the influence of differences in pixel value ranges of different imaging devices on model inference;

[0086] Dimension and format adjustment refers to standardizing or padding the image size and number of channels to make the image data fully match the input dimension requirements of the pre-trained single-molecule localization deep learning network model, without requiring the user to manually adjust the image size.

[0087] For target single-molecule images with multiple time frames, the preprocessing module can perform frame-by-frame normalization processing of the corresponding frame images based on the frame number passed by the interactive interface layer, ensuring that the normalized images output in each frame can be directly input into the model for inference.

[0088] This implementation method enables standardized preprocessing specifically for deep learning models, ensuring that the image data input to the localization module fully matches the model input requirements. This avoids model inference anomalies caused by data format and dimension mismatches, and ensures smooth and automated operation of the localization process.

[0089] At the same time, it eliminates the differences in pixel value range between different imaging devices, improving the stability of model inference and the consistency of positioning accuracy.

[0090] Furthermore, in an optional implementation, the pre-trained single-molecule localization deep learning network model is a feature fusion network based on an encoder-decoder structure, with skip connections and residual connections between the encoder and decoder. The single-molecule localization deep learning network model is configured as follows:

[0091] Multi-scale deep features are extracted from the processed image data using an encoder;

[0092] The adaptive feature fusion module performs adaptive weighted fusion of multi-scale deep features to obtain a fused feature map.

[0093] The decoder, based on the fused feature map, completes the localization feature recovery and single-molecule fluorescence point probability mapping to obtain a single-molecule fluorescence point probability localization map as the localization image.

[0094] Specifically, in this embodiment, due to the low signal-to-noise ratio, dense and easily overlapping fluorescence spots in single-molecule microscopic images, a single-molecule localization deep learning network model was constructed based on the PyTorch or TensorFlow deep learning framework, such as... Figure 3 As shown;

[0095] The encoder includes a 7×7 convolutional module and four cascaded encoder feature modules, which are used to extract multi-scale deep features from single-molecule images. The decoder is designed to correspond to the encoder and includes four cascaded decoder feature modules, which are used to recover localization features and map single-molecule fluorescence points.

[0096] The adaptive feature fusion module is an ASF adaptive feature fusion module, which is used to perform scale matching, channel splicing, adaptive attention weight allocation and weighted fusion on the multi-scale feature maps X1, X2, X3 and X4 (X4 is the 8×8×C4 feature map output by the fourth encoder feature module) output by the encoder, to enhance the core features of single-molecule fluorescent dots and suppress background noise.

[0097] More specifically, the 7×7 convolutional module in the encoder is configured with a batch normalization layer, using the ReLU function as the activation function, and the input is a standardized image (size 256×256×1) with a stride of 2. This module is used to perform preliminary feature extraction on the input image, while effectively reducing the spatial size of the input image and reducing the computational load of subsequent networks.

[0098] The four cascaded encoder feature modules in the encoder are configured as follows:

[0099] Each module consists of two 3×3 convolutional layers, a batch normalization layer, a ReLU activation function, a 1×1 convolutional layer, and a residual connection structure. The input data completes local feature extraction in the two 3×3 convolutional layers and doubles the number of feature channels in the 1×1 convolutional layer, thereby improving the feature dimension and enhancing the feature expression capability.

[0100] Meanwhile, the original input features of each module are directly fused with the output features of the module at the element level. Through the shortcut propagation characteristics of residual connections, the model training process is accelerated, avoiding gradient vanishing and feature degradation problems that occur during deep network training.

[0101] More specifically, after each encoder feature module is processed, it undergoes a 2×2 max pooling operation to further reduce the image space size, compress redundant information, and retain core features.

[0102] The output feature maps obtained after processing by the first three encoder feature modules are denoted as X1 (64×64×C1), X2 (32×32×C2), and X3 (16×16×C3), respectively. These three feature maps will be directly transmitted to the corresponding level of the decoder through skip connections, realizing the precise linkage between the shallow and middle layer features of the encoder and the features of the decoder, and providing support for the subsequent improvement of positioning accuracy.

[0103] More specifically, in the adaptive feature fusion module:

[0104] First, a 2x upsampling operation is performed on feature map X2, a 4x upsampling operation is performed on feature map X3, and an 8x upsampling operation is performed on feature map X4 to ensure that the spatial size of all feature maps is consistent with that of X1 (64×64) to achieve scale matching. After upsampling, a 1×1 convolutional layer is used to calibrate the number of channels and the size to ensure that all feature maps are consistent in spatial size and channel dimension (all are 64×64×C).

[0105] Next, the scale-matched X1, X2, X3, and X4 are concatenated along the channel dimension to obtain a comprehensive feature map (64×64×4C). The comprehensive feature map is then input into a 1×1 convolutional layer, and the number of output channels is adjusted to 4 to obtain an attention weight feature map (64×64×4), thus completing the channel concatenation and attention weight generation.

[0106] Then, the attention weight feature map is processed by the Softmax activation function to generate a set of normalized attention weight maps (64×64×4), so that the weights corresponding to each feature map satisfy the constraint that they are non-negative and sum to 1, thereby realizing the adaptive weighted allocation of features at different scales and completing the Softmax normalization.

[0107] Finally, weighted fusion is performed. The attention weights of the four channels output by Softmax are multiplied one channel at a time with the original input feature maps X1, X2, X3, and X4 respectively. The weighted feature maps are then fused element by element to output the final fused feature map (64×64×C). This fused feature map has the ability to enhance features at multiple scales, suppress noise, and focus key information.

[0108] More specifically, for the decoder, its decoder feature module is configured as follows:

[0109] The input to each decoder feature module is divided into two parts: one is the deep features output by the encoder at the corresponding level, and the other is the fused feature map after processing by the adaptive feature fusion module.

[0110] First, the deep features output by the encoder are adaptively weighted and multiplied and summed with the fused feature map to achieve accurate fusion of deep features and fused features, thereby enhancing the core features of fluorescent dots and suppressing background noise.

[0111] Secondly, the fusion result is concatenated with the corresponding feature map passed by the encoder through skip connections to achieve complementary fusion of shallow detail features and deep semantic features, making up for the loss of deep feature details and improving positioning accuracy.

[0112] Then, the concatenated feature map is input to the decoder feature module. Each module consists of two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function, as well as a 1×1 convolutional layer, a batch normalization layer, and a ReLU activation function. In the 1×1 convolutional layer, the number of feature channels is halved, which corresponds to the doubling of the number of channels in the encoder. This achieves a reasonable adjustment of the feature dimension, reduces the amount of computation, and ensures the effectiveness of feature representation.

[0113] Finally, after each decoder feature module has finished processing, a double upsampling operation is performed to gradually restore the spatial size of the image and map the deep features back to the original image scale.

[0114] For the decoder's output layer:

[0115] The feature map obtained after continuous processing by four decoder feature modules is input into a 1×1 convolutional layer. This 1×1 convolutional layer converts the number of feature channels to 1 and outputs a single-channel feature map.

[0116] Subsequently, the single-channel feature map is processed by the Sigmoid activation function, and the output is mapped to the [0,1] interval. Each pixel value in this interval corresponds to the probability that it is a single-molecule fluorescent spot. The closer the probability value is to 1, the higher the probability that the pixel position is a single-molecule fluorescent spot. Finally, the accurate positioning and probability judgment of single-molecule fluorescent spots are achieved, and a single-molecule fluorescent spot probability positioning map is obtained.

[0117] For single-molecule localization deep learning network models, a hybrid dataset of large-scale single-molecule simulated image datasets and real microscopic image datasets can be used for training. The simulated image dataset is generated by single-molecule imaging simulation software and contains single-molecule images with different signal-to-noise ratios, different fluorescence dot densities, and different degrees of overlap. The real microscopic image dataset uses the publicly available STORM and PALM single-molecule localization datasets.

[0118] Regarding the loss function, the mean squared error (MSE) loss function is used to calculate the error between the single-molecule fluorescence point probability localization map output by the model and the actual fluorescence point annotation map;

[0119] Regarding the optimizer, the Adam optimizer was used, with an initial learning rate set to 1e-4. During training, a learning rate decay strategy was adopted, with the learning rate decreasing to 0.5 every 100 training epochs. The model was trained for 500 epochs on a mixed dataset. After each training epoch, the model performance was evaluated on the validation set, and the weight file of the model with the best performance on the validation set was saved.

[0120] By adopting the above implementation method, and combining the encoder-decoder basic structure with residual connections, skip connections and adaptive feature fusion modules, the technical pain points of insufficient feature extraction, low localization accuracy and difficulty in separating overlapping molecules in traditional single-molecule localization models are solved. It can effectively suppress background noise in single-molecule images, accurately identify densely overlapping single-molecule fluorescent dots, and significantly improve the accuracy of single-molecule localization.

[0121] Furthermore, in an optional embodiment, the decoder is configured to perform a four-level 2x upsampling operation on the fused feature map, cumulatively complete 8x upsampling super-resolution reconstruction, and simultaneously complete the probability mapping of single-molecule fluorescent dots to obtain a single-molecule fluorescent dot probability localization map.

[0122] Specifically, in this embodiment, the four decoder feature modules of the decoder perform a 2x upsampling operation after each module finishes processing, and the four modules are cascaded to complete a cumulative 8x upsampling, gradually restoring the 8×8 feature map compressed by the encoder to a high-resolution feature map of 2048×2048, and finally obtaining a high-resolution single-molecule fluorescence point probability localization map through the output layer.

[0123] The image localization module pre-stores the optimal model weight file after training. When the platform starts, it automatically loads the weight file and switches to inference mode. Users do not need to manually load the model or write code to call it. They only need to click the "Predictive Localization" button on the interactive interface to complete the automated forward inference localization.

[0124] In addition, the upsampling factor of each module can be adjusted according to actual needs to achieve super-resolution reconstruction at different magnifications such as 4x and 16x.

[0125] Using this implementation method, 8x super-resolution reconstruction is achieved through four levels of 2x upsampling, which makes up for the insufficient resolution of the original microscopic image and outputs a high-resolution single-molecule localization image. Compared with traditional localization methods, the spatial resolution is significantly improved, meeting the high-precision requirements of researchers for single-molecule localization.

[0126] Furthermore, in an optional implementation, the image post-processing module further includes a frame overlay unit;

[0127] The frame overlay unit is configured to traverse all single-frame localization images within the range of the user-selected frame number in the processing configuration parameters, and perform element-by-element overlay fusion and normalization processing on the multi-frame single-molecule localization results to filter out single-frame localization noise and obtain optimized localization images.

[0128] Specifically, in this embodiment, the frame overlay unit first obtains the start frame and end frame selected by the user from the parameter setting control, and determines the range of frames to be processed.

[0129] Secondly, traverse all frames within this range, obtain the single-molecule localization result of each frame in turn, and then add and fuse the localization result images of all frames element by element.

[0130] Finally, the fused image is normalized to filter out random noise points in the single-frame localization, generating an optimized localization image, which is then transmitted to the result display control for display.

[0131] By employing this implementation method, the deviation caused by single-frame positioning noise is effectively eliminated through the superposition and fusion of multi-frame positioning results, thereby improving the stability and reliability of the positioning results, generating higher-quality super-resolution reconstructed images, and further enhancing the practicality of the platform.

[0132] Furthermore, in an optional implementation, the image post-processing module further includes a positioning list output unit;

[0133] The location list output unit is configured to extract pixel coordinate information of single-molecule fluorescent points from the location image, combine it with the original size parameters of the target single-molecule image, convert the extracted pixel coordinates into standard physical coordinates that match the original image size, and export a location list file in txt or csv format.

[0134] Specifically, in this embodiment, the location list output unit first extracts the pixel coordinates of all fluorescent points from the single-molecule fluorescent point probability localization map using a threshold segmentation algorithm, where these coordinates are the high-resolution coordinates after super-resolution reconstruction; secondly,

[0135] By combining the size parameters of the original image with the physical pixel precision, the super-resolution pixel coordinates are converted into standard physical coordinates that match the size of the original image, in nm.

[0136] Finally, using Python's csv library or file operation functions, information such as single molecule number, frame number, X coordinate, and Y coordinate is written to a file and exported as a standard location list file in txt or csv format, which can be directly imported into biomedical image analysis software for subsequent quantitative analysis.

[0137] This implementation method achieves standardized export of single-molecule positioning coordinates, solving the problems of inconsistent output coordinate formats and inability to be directly adapted to commonly used scientific research software in the prior art. It enables efficient reuse of positioning data and further enhances the practicality of the platform.

[0138] Furthermore, in an optional implementation, it further includes:

[0139] The multi-threaded scheduling and communication module is configured to allocate interactive operations of the interactive interface layer to the main thread for execution through a preset multi-threaded and PyQt signal-slot mechanism, and allocate the algorithm calculation tasks of the image preprocessing module, image localization module and image postprocessing module to independent sub-threads for execution, so as to complete the standardized processing instruction transmission and data interaction between the main thread and the sub-threads.

[0140] Specifically, in this embodiment, the multi-threaded scheduling and communication module runs the interactive operations, window rendering, and control response tasks of the interactive interface layer on the main thread to ensure the real-time performance and smoothness of the interface operation.

[0141] All time-consuming algorithm computation tasks such as image preprocessing, model inference, and multi-frame overlay are assigned to independent sub-threads for execution, so as to avoid blocking the main thread by time-consuming operations.

[0142] The computation status of the child thread (such as "preprocessing", "locating", "locating complete") and the computation result are passed to the main thread through the PyQt signal and slot mechanism to update the display content of the interactive interface.

[0143] This implementation method achieves complete separation between user interface interaction and algorithm calculation, significantly improving the platform's smoothness and user experience, avoiding the problem of unresponsive interfaces caused by the large amount of computation in deep learning, and further enhancing the platform's stability.

[0144] Furthermore, in an alternative implementation, it also includes:

[0145] The resource management and exception handling module is configured to, for multi-frame target single-molecule image data, first read file metadata to obtain the total number of frames and basic image parameters, respond to the user's frame operation command and load only the image data of the corresponding frame into memory, and automatically release memory resources for frame data that have completed the operation and exceeded the preset idle time.

[0146] Furthermore, in the execution flow corresponding to the image preprocessing module, image positioning module, image postprocessing module, and interactive interface layer, preset exception capture nodes are set to capture exception scenarios and extract exception types, and the corresponding generated exception prompt information is sent to the result display control for display.

[0147] Specifically, in this embodiment, the resource management and exception handling module pre-sets logic for on-demand loading and dynamic memory release of multi-frame images and logic for full-process exception capture and prompting.

[0148] The logic of on-demand loading and dynamic memory release of multi-frame images refers to:

[0149] When the resource management and exception handling module reads a multi-frame target single-molecule image file, it does not load all frame data into memory at once. Instead, it first reads the file metadata, including basic parameters such as the total number of frames, image size, and pixel precision. Only when it receives a frame operation command (including frame switching, frame processing, and frame preview) sent by the user through the interactive interface layer will it load the image data of the corresponding frame into memory.

[0150] After processing, if the frame data is not processed again for more than the preset idle time (e.g., 30 seconds), the platform automatically releases the memory resources occupied by the frame data, which greatly reduces the memory usage of the platform during operation, so as to better cope with the characteristics of single-molecule microscopy imaging, which involves multiple frames, time sequence, and large capacity.

[0151] The full-process exception capture and notification logic refers to:

[0152] The resource management and exception handling module has independent exception capture nodes set up at each execution stage of the entire process, including file reading, preprocessing, model inference, postprocessing, and interface interaction.

[0153] When abnormal scenarios such as file corruption, unsupported format, model loading failure, abnormal data type, and insufficient memory are captured, the system automatically extracts the abnormal type, the cause of the abnormality, and the corresponding solution, and sends the corresponding abnormal prompt information to the result display control to display it to the user in the form of a pop-up window.

[0154] This implementation method significantly improves the platform's operational stability, robustness, and fault tolerance, avoiding crashes caused by memory overflows and program anomalies. It also optimizes the display effects of images of different sizes, further enhancing the platform's user experience.

[0155] In one embodiment of the platform, all the above-described implementation methods are included, and its workflow includes the following steps:

[0156] like Figure 2 As shown, users trigger the file selection control by clicking the "Open Image" button in the interactive interface layer, selecting a target single-molecule image file in TIFF or FITS format. The platform automatically recognizes the file format and reads the image data, displaying the results through the result display control. Figure 4 The original image shown;

[0157] Users can select the positioning model and set the processing frame range through the parameter setting control. By clicking the "Preprocessing" button, the image preprocessing module automatically completes the image data type conversion, format conversion and normalization processing, and generates a standardized image, which is displayed in real time on the interactive interface.

[0158] When the user clicks the "Predictive Localization" button, the image localization module invokes a pre-trained single-molecule localization deep learning network model to perform forward inference on the standardized image, completing single-molecule fluorescence point localization and 8x upsampling super-resolution reconstruction. The localized image is then displayed in real-time on the interactive interface. Figure 5 As shown;

[0159] When the user clicks the "Show Location" button, the location display unit of the image post-processing module generates a location comparison image, such as... Figure 6 As shown, the image displays a comparison of the localization results overlaid with the original image. When the user clicks the "Frame Overlay" button, the frame overlay unit overlays and fuses multiple frames of localization results within a selected frame range to generate an optimized super-resolution reconstructed image, as shown below. Figure 7 As shown;

[0160] When the user clicks the "Output Location" button, the location list output unit extracts the location coordinates and converts them into standard physical coordinates, exporting the location list file in txt or csv format. Figure 8 As shown, this image is for subsequent scientific research and analysis. Users can click the "Save Image" button to save the currently displayed image results.

[0161] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A single-molecule image localization and processing design platform based on PyQt, characterized in that, include: Image preprocessing module, image localization module, image postprocessing module, and interactive interface layer built with PyQt; The interactive interface layer is communicatively connected to the image preprocessing module, the image localization module, and the image postprocessing module, respectively, and is used to respond to the user's selection operation, identify and read the target single molecule image, and visually display the target single molecule image and the corresponding processed image, localization image, and localization comparison image. The image preprocessing module is used to preprocess the target single-molecule image from the interactive interface layer to obtain the processed image; The image localization module is used to call a pre-trained single-molecule localization deep learning network model to perform forward inference on the processed image, complete single-molecule fluorescence point localization and super-resolution reconstruction, and obtain a localization image. The image post-processing module includes a positioning display unit, which is used to first scale the target single-molecule image to the same size as the positioning image, and then superimpose the single-molecule fluorescence point positioning result corresponding to the positioning image onto the scaled target single-molecule image to obtain a positioning comparison image.

2. The single-molecule image localization and processing design platform based on PyQt as described in claim 1, characterized in that: The interactive interface layer includes a file selection control, a parameter setting control, a result display control, and a function triggering control; The file selection control is configured to obtain the storage path information of the target single-molecule image file based on the user's file selection operation; The parameter setting control is configured to obtain the processing configuration parameters of the target single-molecule image based on the user's parameter configuration operation. The result display control is configured to perform visual rendering and display of the received target single-molecule image, processed image, localization image, and localization comparison image; The function trigger control is configured to generate standardized processing instructions based on the user's function trigger operation and send them to the corresponding image preprocessing module, image positioning module, or image postprocessing module.

3. The single-molecule image localization and processing design platform based on PyQt as described in claim 2, characterized in that: The image preprocessing module is specifically configured as follows: Used to extract the file extension and automatically identify whether the target single-molecule image file is in TIFF or FITS format based on the storage path information of the target single-molecule image file obtained from the file selection control; The system calls the corresponding preset parsing library to complete the reading and writing of image data and performs bidirectional conversion between TIFF and FITS formats. At the same time, it performs adaptive matching conversion of pixel data types on the read image data to obtain basic image data that meets the processing requirements of the image positioning module.

4. The single-molecule image localization and processing design platform based on PyQt as described in claim 3, characterized in that: The image preprocessing module is further configured to perform pixel value normalization processing on the basic image data, adjust the basic image data to the dimension and format required by the input of the pre-trained single-molecule localization deep learning network model, and output standardized processed image data to the image localization module.

5. The single-molecule image localization and processing design platform based on PyQt as described in claim 1, characterized in that: The pre-trained single-molecule localization deep learning network model is a feature fusion network based on an encoder-decoder structure, with skip connections and residual connections between the encoder and decoder. The single-molecule localization deep learning network model is configured as follows: Multi-scale deep features are extracted from the processed image data using an encoder; The adaptive feature fusion module performs adaptive weighted fusion of multi-scale deep features to obtain a fused feature map. The decoder, based on the fused feature map, completes the localization feature recovery and single-molecule fluorescence point probability mapping to obtain a single-molecule fluorescence point probability localization map as the localization image.

6. The single-molecule image localization and processing design platform based on PyQt as described in claim 5, characterized in that: The decoder is configured to perform a four-level 2x upsampling operation on the fused feature map, cumulatively complete 8x upsampling super-resolution reconstruction, and simultaneously complete the probability mapping of single-molecule fluorescent dots to obtain a single-molecule fluorescent dot probability localization map.

7. The single-molecule image localization and processing design platform based on PyQt as described in claim 2, characterized in that: The image post-processing module also includes a frame overlay unit; The frame overlay unit is configured to traverse all single-frame localization images within the range of the user-selected frame number in the processing configuration parameters, and perform element-by-element overlay fusion and normalization processing on the multi-frame single-molecule localization results to filter out single-frame localization noise and obtain optimized localization images.

8. The single-molecule image localization and processing design platform based on PyQt as described in claim 1, characterized in that: The image post-processing module also includes a location list output unit; The location list output unit is configured to extract pixel coordinate information of single-molecule fluorescent points from the location image, combine it with the original size parameters of the target single-molecule image, convert the extracted pixel coordinates into standard physical coordinates that match the original image size, and export a location list file in txt or csv format.

9. The single-molecule image localization and processing design platform based on PyQt as described in claim 1, characterized in that, Also includes: The multi-threaded scheduling and communication module is configured to allocate interactive operations of the interactive interface layer to the main thread for execution through a preset multi-threaded and PyQt signal-slot mechanism, and allocate the algorithm calculation tasks of the image preprocessing module, image localization module and image postprocessing module to independent sub-threads for execution, so as to complete the standardized processing instruction transmission and data interaction between the main thread and the sub-threads.

10. The single-molecule image localization and processing design platform based on PyQt as described in claim 2, characterized in that, Also includes: The resource management and exception handling module is configured to, for multi-frame target single-molecule image data, first read file metadata to obtain the total number of frames and basic image parameters, respond to the user's frame operation command and load only the image data of the corresponding frame into memory, and automatically release memory resources for frame data that have completed the operation and exceeded the preset idle time. Furthermore, in the execution flow corresponding to the image preprocessing module, image positioning module, image postprocessing module, and interactive interface layer, preset exception capture nodes are set to capture exception scenarios and extract exception types, and the corresponding generated exception prompt information is sent to the result display control for display.

Images