Intelligent region-of-interest monitoring method based on spot scanning imaging system

Abstract

The invention discloses an interested area intelligent monitoring method based on a spot scanning imaging system, which comprises the following steps: positioning an interested area based on full-view-field scanning and generating a scanning mask, and only executing spot scanning imaging on a mask coverage area based on scanning mask control, when physical drift of the imaging area is judged based on the difference feature map between the current imaging data and the initial imaging data, full-field scanning is executed again to update the area of interest; positioning a cell area in the determined imaging area and outputting a silent state and an active state of a cell at the same time, only scanning a geometric center pixel point in the cell area for the silent area, executing full-pixel point scanning for the active area, and if it is monitored that a target in the active area is lost or a signal in the silent area abnormally transitions, determining that the target in the active area is lost or the silent area is abnormally transitioned. And if not, triggering an observation reset signal to forcibly interrupt the current scanning and re-position the cell region. Therefore, the imaging efficiency can be remarkably improved on the premise of ensuring the data integrity.

Description

Technical Field

[0001] This invention belongs to the field of biomedical microscopy and computer vision technology, specifically relating to an intelligent monitoring method for regions of interest based on a point scanning imaging system. Background Technology

[0002] Point scanning imaging systems (such as two-photon microscopes and confocal microscopes) have high resolution, high signal-to-noise ratio and deep tissue imaging capabilities, and are important tools for observing cell morphology and functional activities in neuroscience and biomedical research.

[0003] However, traditional point-scan imaging techniques typically employ raster scanning to acquire data point-by-point across the entire field of view. This approach has significant limitations: First, the imaging speed is limited by the scanning frequency of the mechanical galvanometer and the pixel dwell time. Full-field scanning results in a significant amount of time being wasted in background areas with no useful information, making it difficult to meet the needs of observing millisecond-level transient activities of cells. Second, prolonged full-field scanning generates massive amounts of redundant data, placing enormous pressure on data transmission and storage. Furthermore, excessive laser irradiation may cause photobleaching or phototoxic damage to the samples. Third, although existing region-of-interest (ROI) scanning techniques can improve the imaging frame rate, they lack intelligent closed-loop monitoring mechanisms.

[0004] In cell imaging experiments, samples often experience physical displacement due to cellular physiological activities, leading to misalignment between the pre-defined region of interest and the actual observation target, resulting in the loss of crucial experimental data. Furthermore, cellular activity exhibits significant sparsity and dynamism (i.e., long quiescent periods and short burst periods). Existing scanning strategies often apply a uniform sampling density to all targets, failing to dynamically adjust the scanning strategy based on the target's real-time activity status (e.g., quiescent or active), making it difficult to achieve an optimal balance between spatiotemporal resolution and data efficiency. Summary of the Invention

[0005] In view of the above, and aiming to overcome the shortcomings of existing technologies such as low frame rate of full-field scanning, large data redundancy, and lack of offset correction and dynamic adaptation capabilities in traditional region of interest (ROI) scanning, this invention provides an intelligent ROI monitoring method based on a point scanning imaging system. This method achieves automatic identification and locking of ROI through a deep learning model, and dynamically adjusts the scanning mode in real time according to changes in the physical location and functional activity status of the sample, thereby significantly improving imaging efficiency while ensuring data integrity.

[0006] To achieve the above-mentioned objectives, an embodiment provides an intelligent region-of-interest (ROI) monitoring method based on a point-scan imaging system, which selects one of the following imaging modes according to monitoring requirements, and includes the following steps: Navigation imaging mode: The region of interest is located based on full field of view scanning and a scanning mask of the region of interest is generated. The scanning mask control point scanning imaging system only performs point scanning imaging on the area covered by the mask. When the imaging area is physically offset based on the difference feature map containing time dimension information between the current imaging data and the initial imaging data, the full field of view scanning is re-executed to update the region of interest. Observation Imaging Mode: Locate cell regions within a defined imaging area and simultaneously output the quiescent and active states of cells. For quiescent regions, only the geometric center pixel within the cell region is scanned to generate a one-dimensional temporal signal. For active regions, a full pixel scan is performed to obtain a complete morphological sequence. If the target is lost within the active region or the signal in the quiescent region is abnormally transitioned, an observation reset signal is triggered to forcibly interrupt the current scan and re-locate the cell region.

[0007] Preferably, locating the region of interest based on full-field scanning and generating a scanning mask for the region of interest includes: The imaging system is controlled to perform full-field scanning to acquire images. A trained semantic segmentation model is used to identify regions of interest in the images and obtain scanning masks. The semantic segmentation model outputs a binarized probability map. Thresholding is performed on the binarized probability map to obtain an initial binary mask. A morphological dilation operation is performed on the initial binary mask to extend the mask boundary outward by a preset pixel distance to obtain the scanning mask of the region of interest.

[0008] Preferably, the scanning mask-based control point scanning imaging system performs point scanning imaging only on the area covered by the mask, including: The pixel coordinate index of the scanning mask is parsed and converted into a galvanometer control voltage command in the point scanning imaging system, which then controls the point scanning imaging system to enter a sparse scanning mode that targets only the area covered by the mask.

[0009] Preferably, a difference feature map containing time dimension information is constructed based on the current imaging data and the initial imaging data, including: The current reconstructed image is constructed based on the current imaging data. Specifically, the voltage data sampled by point scanning is backfilled into the corresponding position of the acquired image according to the pixel coordinate index of the scanning mask, and the non-mask-covered area is filled with zeros to obtain the current reconstructed image. The product of the full field-of-view image corresponding to the initial imaging data and the scanning mask is assigned to the G color channel to represent the reference position. The current reconstructed image is assigned to the R color channel to represent the current position. The all-zero matrix is ​​assigned to the B color channel to obtain the difference feature map. The non-overlapping region between the R color channel and the G color channel represents the offset feature of the field of view.

[0010] Preferably, determining a physical shift in the imaging region based on the difference feature map includes: Input the difference feature map into the pre-trained offset judgment model. If the bright spot areas of the R color channel and the G color channel have a high degree of overlap and the image has a yellow tone, the offset judgment model outputs a judgment of stability. If there is a significant misalignment between the R color channel and the G color channel and the image has red or green edges, the offset judgment model outputs a judgment of offset.

[0011] Preferably, locating cell regions within a defined imaging area and simultaneously outputting the quiescent and active states of the cells includes: Within the defined imaging area, acquire continuous data. Frame images; the images to be acquired Frame images are stacked along the channel dimension, constructing a dimension of The multi-channel input tensor, where and These represent the image height and width, respectively; the temporal object detection model improves the first-layer convolutional kernel by inputting a multi-channel input tensor, and the depth of the first-layer convolutional kernel of the temporal object detection model is adjusted to... The model directly extracts spatial features that contain temporal dynamic changes using convolution operations; the temporal object detection model outputs several detection boxes, each containing a category label, confidence score, and coordinate information for either a silent or active state; non-maximum suppression is performed on the output detection boxes to remove overlapping and redundant detection boxes.

[0012] Preferably, if a target loss is detected within the active area, an observation reset signal is triggered, including: For active regions, while performing full pixel scanning, the latest acquired image sequence is continuously input into the temporal target detection model for inference to obtain detection boxes. A class locking mechanism is introduced to perform intersection-union matching between the detection box of the current frame and the target box determined in the previous stage, forcibly inheriting the target category of the previous stage. If the target confidence output by the temporal target detection model is lower than the tracking threshold within a consecutive preset number of frames, or the number of detection boxes is zero, it is determined that the target has shifted or been lost, triggering an observation reset signal.

[0013] Preferably, if an abnormal signal transition is detected in the silent region, an observation reset signal is triggered, including: For silent regions, the voltage amplitude change rate of the timing signal at the center pixel is calculated in real time; if the change rate exceeds the preset active threshold, it is determined that the cell state has transitioned, triggering the observation reset signal.

[0014] Preferably, the semantic segmentation model, offset determination model, and temporal target detection model are deployed on a host computer server, providing services in the form of a microservice architecture and communicating with the point scanning imaging system; wherein the microservice receives image data or voltage data uploaded by the point scanning imaging system and returns galvanometer control signals or reset commands.

[0015] More preferably, the microservice architecture is constructed using a combination of a web framework, a concurrent processing server, and a coroutine network library, and deployed in a host computer environment. The microservice receives reconstructed image data or voltage signal sequences uploaded by the imaging system via a network interface, performs model inference calculations, and returns JSON-formatted result fields containing the mask coordinates of the region of interest, offset determination results, or observation reset instructions.

[0016] More preferably, a multi-process or coroutine mechanism is used to implement concurrent request processing, and the model inference process is accelerated by GPU to meet the high timeliness requirements of the microscopic imaging system for real-time feedback.

[0017] Preferably, the method for constructing the training datasets for the aforementioned neural network models, such as the semantic segmentation model, the offset determination model, and the temporal object detection model, includes: Navigation imaging mode data: Using physical simulation algorithms to simulate the random growth morphology of cells and sinusoidal fluid disturbances, a sequence of images containing displacement deviations is generated as the training set for the semantic segmentation model and the offset determination model. Observational imaging pattern data: The simulation algorithm is used to generate cell image sequences containing different morphologies, different flicker frequencies and different background noises, and the time dimension information is encoded into the channel dimension to construct multi-channel tensor data as the training set for the temporal target detection model.

[0018] Compared with the prior art, the beneficial effects of the present invention include at least the following: To address the shortcomings of existing technologies, such as low full-field scanning frame rate, large data redundancy, and lack of offset correction in traditional scanning, this invention proposes two imaging modes and implements an adaptive sparse scanning strategy, which significantly reduces the transmission and storage pressure of invalid background data. At the same time, this strategy minimizes laser radiation to non-observation areas, effectively delays photobleaching of fluorescent probes and reduces phototoxicity, thus extending the observation window for cell experiments.

[0019] In navigation imaging mode, the system primarily implements dynamic adaptation in the spatial dimension. This mode utilizes a deep learning model to intelligently identify regions of interest and generate sparse scanning masks. The control system performs point scanning only on the effective areas covered by the mask, significantly reducing the number of scanning points and thus increasing the imaging frame rate.

[0020] In observational imaging mode, the system primarily implements dynamic adaptation in the time dimension. This mode utilizes a deep learning model to locate the target imaging region and determine whether it is in a silent or active state, thereby adjusting the scanning method accordingly. For silent targets, the system dynamically switches to scanning only the geometric center pixel to generate a one-dimensional temporal signal; for active regions, it maintains full pixel scanning to obtain a complete image sequence.

[0021] Based on this, the present invention innovatively introduces a dual closed-loop monitoring mechanism, which realizes the monitoring of the target state in two modes respectively, and realizes rescanning when the target state changes, thus solving the problem of easy target loss caused by scanning only the region of interest.

[0022] In navigation imaging mode monitoring, the system constructs a red-green channel difference feature map by fusing the current imaging data with the initial imaging data, dynamically senses the minute physical shifts of the sample in real time, and automatically triggers a full field-of-view reset and rescan when the shift determination model confirms the shift, thereby updating the region of interest.

[0023] In the observation and imaging mode monitoring, the system continuously acquires signals, inputs the signal sequence of the active area into the time-series target monitoring model, and calculates the rate of change of signal amplitude of the signal sequence of the silent area. When the target in the active area is lost or the signal in the silent area undergoes an abnormal transition, the system can immediately observe the sudden change in the state or the deviation in the position of the target, and trigger the observation reset signal to forcibly interrupt the current scan, relocate the target area, and update the region of interest.

[0024] By combining engineering deployment based on a microservice architecture, this invention utilizes a concurrent processing mechanism to decouple imaging control and algorithm inference, ensuring millisecond-level real-time feedback response and system stability. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the region-of-interest intelligent monitoring method based on a point scanning imaging system provided in the embodiment; Figure 2 The embodiments provide a navigation imaging mode construction red-green channel fusion construction difference feature map and example; Figure 3 This is a schematic diagram of the navigation mode neural network model architecture provided in the embodiment; Figure 4 This is a schematic diagram illustrating the temporal data stream and differential scanning strategy principle in the observation imaging mode provided in the embodiment; Figure 5 This is a diagram of the system hardware and microservice deployment architecture provided in the embodiment. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0028] This invention provides an intelligent region-of-interest (ROI) monitoring method based on a point-scan imaging system, primarily comprising two independent operating modes: navigation imaging mode and observation imaging mode. The navigation imaging mode is mainly used for sample localization in the initial stages of an experiment or for retrieving samples after a large area of ​​field of view has been lost. The observation imaging mode is used for high spatiotemporal resolution functional observation of locked targets. One of the following imaging modes is selected and executed according to the monitoring requirements; specifically, the specific process is as follows: Figure 1 As shown, it includes the following steps: S1, Navigation Imaging Mode - Positioning: Based on full-field scanning, locate the region of interest and generate a scanning mask for the region of interest.

[0029] In this embodiment, based on full-field-of-view scanning to locate the region of interest, the imaging system is controlled to perform a standard full-field-of-view point scan imaging, acquiring a large range of raw voltage signal data and reconstructing it into a two-dimensional image. More specifically, the imaging end receives the continuous analog voltage signal output by the photodetector, obtains a one-dimensional discrete voltage sequence after analog-to-digital conversion, and uses a row-column reshaping algorithm to map the one-dimensional discrete voltage sequence into a two-dimensional grayscale matrix as a two-dimensional image.

[0030] In this embodiment, a trained semantic segmentation model is used to identify regions of interest (ROIs) in an image and generate a scanning mask. A specific two-dimensional image is input to the semantic segmentation model, which outputs a binarized probability map of the same size as the input. Thresholding is performed on the binarized probability map to obtain an initial binary mask. A morphological dilation operation is then performed on the initial binary mask to extend the mask boundaries outward by a preset pixel distance (e.g., 3-5 pixels) to obtain the scanning mask for the ROI. This process aims to preserve the background context information of the target edges for subsequent scanning, preventing information loss due to minor target displacement.

[0031] To meet the real-time requirements of microscopic imaging systems, this embodiment specifically designs a lightweight neural network architecture, such as... Figure 3 As shown. First, as Figure 3As shown in the upper part, the semantic segmentation model used to determine the region of interest adopts a simplified U-Net architecture. This model mainly consists of an encoding path, a bottleneck layer, and a decoding path. The input image sequentially passes through three cascaded processing units in the encoding path. Each unit consists of a convolutional module (containing convolution, batch normalization, and activation functions) and a downsampling layer (using max pooling), with the number of channels increasing progressively to extract deep semantic features. At the end of the encoder, the feature map enters the convolutional module of the bottleneck layer to complete contextual information aggregation. Subsequently, the feature map enters the decoding path, passes through an upsampling layer (using transposed convolution) to restore its size, and is then concatenated with the corresponding layer's features in the encoder via a channel cascade structure to fuse shallow spatial details. The concatenated features are then fused again through a convolutional module, and finally, a binarized region of interest mask is output through a 1×1 convolutional layer.

[0032] S2, Navigation Imaging Mode - Monitoring: The scanning mask control point scanning imaging system only performs point scanning imaging on the mask-covered area. When the imaging area is determined to have a physical offset based on the difference feature map containing time dimension information between the current imaging data and the initial imaging data, a full field-of-view scan is re-executed to update the region of interest.

[0033] In this embodiment, the point scanning imaging system based on the scanning mask performs point scanning imaging only on the mask-covered area, including: parsing the pixel coordinate index of the scanning mask, converting the pixel coordinate index into a galvanometer control voltage command in the point scanning imaging system, controlling the point scanning imaging system to switch to navigation imaging mode, and entering a sparse point scanning mode that only targets the mask-covered area to improve the imaging frame rate.

[0034] In this embodiment, during the scanning process in the navigation imaging mode, the current frame imaging data is acquired in real time and fused with the initial frame imaging data through the red and green channels to construct a difference feature map containing time dimension information; the difference feature map is input to the trained offset judgment model to determine whether the imaging area has undergone physical offset; if the judgment result is that an offset has occurred, a reset signal is triggered, and the process returns to step S1 to re-execute the full field of view scan to update the region of interest.

[0035] like Figure 2 As shown in the figure, this embodiment illustrates in detail the construction process of the difference feature map, which aims to transform the displacement changes in the time dimension into color features in the spatial dimension. The specific steps are as follows: During the scanning cycle of the navigation imaging mode, first acquire The region of interest (ROI) point scanning imaging data at time (current time) is reconstructed into a two-dimensional image and assigned to the R color channel to represent the current sample location information. Next, the data is obtained... Full field of view image at time (initial time) and A region of interest (ROI) scan mask is determined at a specific time. The matrix product of the mask and the image is assigned to the G color channel to represent the reference sample position information. Next, a zero-matrix of the same size as the image is constructed and assigned to the B color channel to eliminate interference from irrelevant color information. Finally, the data from the R, G, and B color channels are superimposed and fused to generate a red-green channel difference feature map. The non-overlapping regions between the R and G color channels represent the field-of-view offset characteristics.

[0036] like Figure 3 The lower part shows the offset determination model used to determine visual field shift, which is built based on a pre-trained ResNet18. This model receives the red-green channel difference feature map as input and uses ResNet18 as the backbone network, effectively extracting the difference features of the red-green edges through residual connections. The high-dimensional feature map output by the backbone network is compressed into a feature vector through pooling layers, and then enters a fully connected layer to map the high-dimensional features into a single Logit value. The offset determination model finally outputs a binary classification result: 0 indicates no significant shift, and 1 indicates a significant shift.

[0037] When the sample has not shifted, the pixels in the R color channel and the G color channel are highly overlapping in space. According to the additive color mixing principle, the feature map appears as a bright yellow spot in the region of interest. When the sample physically shifts, the R color channel (current) and the G color channel (reference) will be misaligned in space, and the edge regions of the feature map will appear as independent red (indicating areas newly moved into the field of view) or green (indicating areas moved out of the field of view). The offset determination model identifies the feature map based on these red and green edge features. If the determination result is negative (no offset), the loop is closed and the step of scanning and imaging only the region of interest is continued. If the determination result is positive (offset occurs), a reset is triggered, and the first step is returned to re-execute the full field-of-view point scanning and imaging to update the region of interest.

[0038] S3, Observation Imaging Mode - Localization: Locates cell regions within a defined imaging area and simultaneously outputs the quiescent and active states of the cells.

[0039] In this embodiment, a series of voltage signal data sequences of multiple consecutive frames are acquired within a defined imaging area; a trained temporal target detection model is used to identify cell regions based on the voltage signal data sequences, and simultaneously outputs quiescent and active states.

[0040] Specifically, within a defined imaging area, continuous data acquisition is performed. Two-dimensional image frame; the acquired Two-dimensional images are stacked along the channel dimension to construct a dimension of The multi-channel input tensor, where and These represent the image height and width, respectively; the temporal object detection model improves the first-layer convolutional kernel by inputting a multi-channel input tensor, and the depth of the first-layer convolutional kernel of the temporal object detection model is adjusted to... The model directly extracts spatial features that contain temporal dynamic changes using convolution operations; the temporal object detection model outputs several detection boxes, each containing a category label, confidence score, and coordinate information for either a silent or active state; non-maximum suppression is performed on the output detection boxes to remove overlapping and redundant detection boxes.

[0041] S4, Observation Imaging Mode - Monitoring: For quiescent regions, only the geometric center pixels within the cell region are scanned to generate a one-dimensional temporal signal; for active regions, a full pixel scan is performed to obtain a complete morphological sequence.

[0042] In this embodiment, a differentiated scanning strategy is executed based on the active or silent state identified in step S3: for silent regions, only the geometric center pixel within the silent region is scanned to generate a one-dimensional temporal signal; for active regions, full pixel scanning is performed to obtain a complete morphological sequence.

[0043] S5, Observation Imaging Mode - Reset: If the target is lost in the active area or the signal of the silent area is abnormally transitioned, the observation reset signal is triggered to forcibly interrupt the current scan and re-localize the cell region.

[0044] In this embodiment, during the execution of step S4, the effectiveness of the target status is monitored in real time. If the target in the active area is lost or the signal in the silent area is abnormally transitioned, an observation reset signal is triggered, the current scan is forcibly interrupted, and the process returns to step S3 to reposition the target.

[0045] Specifically, the specific triggering logic for the observation reset is as follows: For active regions, while performing full-pixel scanning, the latest acquired image sequence is continuously input into the temporal target detection model for inference to obtain detection boxes. A class locking mechanism is introduced to perform intersection-union (IoU) matching between the current frame's detection box and the target box determined in the previous stage, forcibly inheriting the target category from the previous stage. If, within a preset number of consecutive frames, the target confidence output by the temporal target detection model is lower than the tracking threshold, or the number of detection boxes drops to zero, it is determined that the target has shifted or been lost, triggering an observation reset signal. Specifically, during the scanning process, active signal detection box loss monitoring is implemented. If the result is negative, it means that the detection box has not been lost, and point scanning continues; if the result is positive, it means that the detection box has been lost, and the target is determined to be abnormal, and the loop returns to the "temporal target detection model confirms silent and active regions" step to reconfirm the region status.

[0046] For silent regions, the rate of change of voltage amplitude of the temporal signal at the center pixel is calculated in real time. If the rate of change exceeds a preset activity threshold, a cell state transition is determined, triggering an observation reset signal. Specifically, during the scanning process, abnormal transitions in silent signals are monitored, and the voltage signal at the center point is monitored in real time. If the result is negative, it indicates no abnormal transition, and the center point scanning is maintained; if the result is positive, it indicates an abnormal signal transition, and the state is determined to have changed. The loop is closed, returning to the "temporal target detection model confirms silent and active regions" step to reconfirm the region state.

[0047] Furthermore, in order to intuitively illustrate the core operating mechanism of the observation imaging mode, Figure 4 The document details the complete data flow from temporal data construction to the execution of differentiated strategies. Firstly, in the data input phase, to enable the 2D detection model to have temporal awareness, the system abandons the traditional single-frame processing mode and instead acquires continuous data. The original grayscale image of the frame. (e.g.) Figure 4 As shown on the left, these images are physically stacked in the channel dimension and recombined into a single dimension. The input tensor is a three-dimensional temporal tensor. This tensor fully preserves the spatial structure information of the region of interest, while encoding the brightness changes in the temporal dimension as pixel value features between channels for subsequent network extraction. To handle this multi-channel input, this embodiment adaptively modifies the first-layer convolutional kernel of the temporal object detection model (such as YOLO), expanding its input channel count to 10; simultaneously, it reuses and initializes the expanded channels using pre-trained RGB weights, thereby achieving rapid model convergence without relying on a large temporal dataset.

[0048] Secondly, in the decision-making and execution phase, the temporal object detection model infers from the input tensor and outputs the locations of all cell regions of interest within the field of view, along with their state labels ("silent" or "active"). Based on these labels, the physical scan path is mapped into two distinct modes: one for regions determined to be in a silent state (e.g., ...). Figure 4 (As shown in the upper right branch), the system automatically switches to center-point scanning mode. At this point, the output is no longer a two-dimensional image, but rather a [dimensional] image. The voltage waveform curve of the geometric center point of the region changing with time is a one-dimensional signal. This not only compresses the amount of data collected to the original... It can also monitor in real time whether a state transition has occurred through waveform analysis. For regions determined to be in an active state (such as...) Figure 4 (As shown in the lower right branch), the system maintains full-pixel scanning mode. At this point, the output is a complete high-resolution image sequence to capture the morphological dynamics of cells. Through this differentiated strategy, the system achieves intelligent optimal allocation of spatiotemporal resolution.

[0049] To support the aforementioned complex dual-modal algorithm and millisecond-level real-time control, this embodiment establishes the following... Figure 5 The hardware-software closed-loop control system shown is mainly composed of three parts: the imaging hardware layer, the host computer service layer, and the data communication layer.

[0050] First, at the imaging hardware layer, such as Figure 5 As shown in the dashed box on the left, the system uses a point scanning microscope as its core physical platform. The photoelectric converter is responsible for capturing the weak fluorescence signal excited by the cell sample and converting it into an analog voltage signal; the data acquisition card is used to digitize the analog signal and transmit the data stream at a high sampling rate; the galvanometer controller receives external commands to drive the X / Y galvanometer to deflect, thereby precisely controlling the physical scanning path of the laser focus.

[0051] Secondly, at the host computer service layer, such as Figure 5 As shown in the dashed box on the right, the algorithm processing end is deployed on a high-performance server, adopting a loosely coupled microservice architecture. Specifically, the system uses a WSGI HTTP server capable of handling high-concurrency network requests and process management to ensure system stability under high load; a lightweight web service framework is used to build a Web API interface, responsible for parsing HTTP requests and routing them to the corresponding algorithm modules; the core deep learning model inference service internally loads the semantic segmentation model in navigation mode, the offset determination model, and the temporal object detection model in observation mode, and uses GPU acceleration to implement the forward inference of the model, ensuring that the calculation results are output in milliseconds.

[0052] Finally, the hardware layer and the service layer interact via the TCP / IP communication protocol to form a real-time closed-loop control, such as... Figure 5 The middle section is shown. Its workflow is as follows: After the imaging hardware serializes the acquired voltage signal, it sends it to the server through the uplink; after the deep learning model service in the server completes the inference calculation, it encapsulates the generated region of interest mask coordinates, differential scanning path or reset command into galvanometer control command, and returns it to the hardware through the downlink, thereby realizing intelligent adaptive scanning.

[0053] The method provided in this embodiment, in navigation mode, utilizes a semantic segmentation model to process the reconstructed voltage signal image, determines the region of interest (ROI), and scans only the ROI. After imaging begins, an offset determination model monitors imaging differences in real time, triggering a ROI reset mechanism. In observation mode, a target detection model identifies cell states (quiet or active), and differentiated scanning strategies (center-point scanning or full-area scanning) are applied to cells in different states. This invention achieves intelligent dynamic adjustment of the imaging region through a deep learning model, effectively reducing invalid scanning areas and significantly improving the frame rate and efficiency of point scan imaging.

[0054] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent monitoring of regions of interest based on a point scanning imaging system, characterized in that, Select one of the following imaging modes to perform based on monitoring requirements, including the following steps: Navigation imaging mode: The region of interest is located based on full field of view scanning and a scanning mask of the region of interest is generated. The scanning mask control point scanning imaging system only performs point scanning imaging on the area covered by the mask. When the imaging area is physically offset based on the difference feature map containing time dimension information between the current imaging data and the initial imaging data, the full field of view scanning is re-executed to update the region of interest. Observation Imaging Mode: Locate cell regions within a defined imaging area and simultaneously output the quiescent and active states of cells. For quiescent regions, only the geometric center pixel within the cell region is scanned to generate a one-dimensional temporal signal. For active regions, a full pixel scan is performed to obtain a complete morphological sequence. If the target is lost within the active region or the signal in the quiescent region is abnormally transitioned, an observation reset signal is triggered to forcibly interrupt the current scan and re-locate the cell region.

2. The intelligent region of interest monitoring method based on a point scanning imaging system according to claim 1, characterized in that, The method for locating the region of interest (ROI) based on full-field scanning and generating a scanning mask for the ROI includes: The imaging system is controlled to perform full-field scanning to acquire images. A trained semantic segmentation model is used to identify regions of interest in the images and obtain scanning masks. The semantic segmentation model outputs a binarized probability map. Thresholding is performed on the binarized probability map to obtain an initial binary mask. A morphological dilation operation is performed on the initial binary mask to extend the mask boundary outward by a preset pixel distance to obtain the scanning mask of the region of interest.

3. The intelligent region-of-interest monitoring method based on a point-scan imaging system according to claim 1, characterized in that, The scanning mask-based control point scanning imaging system performs point scanning imaging only on the area covered by the mask, including: The pixel coordinate index of the scanning mask is parsed and converted into a galvanometer control voltage command in the point scanning imaging system, which then controls the point scanning imaging system to enter a sparse scanning mode that targets only the area covered by the mask.

4. The intelligent region of interest monitoring method based on a point scanning imaging system according to claim 1, characterized in that, A difference feature map containing time dimension information is constructed based on the current imaging data and the initial imaging data, including: The current reconstructed image is constructed based on the current imaging data. Specifically, the voltage data sampled by point scanning is backfilled into the corresponding position of the acquired image according to the pixel coordinate index of the scanning mask, and the non-mask-covered area is filled with zeros to obtain the current reconstructed image. The product of the full field-of-view image corresponding to the initial imaging data and the scanning mask is assigned to the G color channel to represent the reference position. The current reconstructed image is assigned to the R color channel to represent the current position. The all-zero matrix is ​​assigned to the B color channel to obtain the difference feature map. The non-overlapping region between the R color channel and the G color channel represents the offset feature of the field of view.

5. The intelligent region-of-interest monitoring method based on a point-scan imaging system according to claim 4, characterized in that, Based on the difference feature map, physical offset of the imaging region is determined, including: Input the difference feature map into the pre-trained offset judgment model. If the bright spot areas of the R color channel and the G color channel have a high degree of overlap and the image has a yellow tone, the offset judgment model outputs a judgment of stability. If there is a significant misalignment between the R color channel and the G color channel and the image has red or green edges, the offset judgment model outputs a judgment of offset.

6. The intelligent region-of-interest monitoring method based on a point-scan imaging system according to claim 1, characterized in that, Within a defined imaging area, locate cellular regions and simultaneously output the quiescent and active states of the cells, including: Within the defined imaging area, acquire continuous data. Frame images; the images to be acquired Frame images are stacked along the channel dimension, constructing a dimension of The multi-channel input tensor, where and These represent the image height and width, respectively; the temporal object detection model improves the first-layer convolutional kernel by inputting a multi-channel input tensor, and the depth of the first-layer convolutional kernel of the temporal object detection model is adjusted to... The model directly extracts spatial features that contain temporal dynamic changes using convolution operations; the temporal object detection model outputs several detection boxes, each containing a category label, confidence score, and coordinate information for either a silent or active state; non-maximum suppression is performed on the output detection boxes to remove overlapping and redundant detection boxes.

7. The intelligent region-of-interest monitoring method based on a point-scan imaging system according to claim 1, characterized in that, If a target loss is detected within the active area, an observation reset signal is triggered, including: For active regions, while performing full pixel scanning, the latest acquired image sequence is continuously input into the temporal target detection model for inference to obtain detection boxes. A class locking mechanism is introduced to perform intersection-union matching between the detection box of the current frame and the target box determined in the previous stage, forcibly inheriting the target category of the previous stage. If the target confidence output by the temporal target detection model is lower than the tracking threshold within a consecutive preset number of frames, or the number of detection boxes is zero, it is determined that the target has shifted or been lost, triggering an observation reset signal.

8. The intelligent region-of-interest monitoring method based on a point-scan imaging system according to claim 1, characterized in that, If an abnormal signal transition is detected in a silent region, an observation reset signal is triggered, including: For silent regions, the voltage amplitude change rate of the timing signal at the center pixel is calculated in real time; if the change rate exceeds the preset active threshold, it is determined that the cell state has transitioned, triggering the observation reset signal.

9. The intelligent region-of-interest monitoring method based on a point-scan imaging system according to claim 2, 5, or 6, characterized in that, The semantic segmentation model, offset determination model, and temporal target detection model are deployed on the host computer server and provide services in the form of a microservice architecture, communicating with the point scanning imaging system; The microservice receives image or voltage data uploaded by the scanning imaging system and returns mirror control signals or reset commands.

Images