Time-sharing modulation positioning method, device and electronic equipment for multi-target optical imaging

Abstract

The application provides a time-sharing modulation positioning method, device and electronic equipment for multi-target optical imaging, and relates to the technical field of optical imaging precision measurement, aiming to solve the problem of the restriction of multi-target application scenarios on the positioning precision of motion modulation type optical positioning. The method comprises the following steps: evaluating the positioning precision limit of an optical system to obtain the positioning precision limit of a target at each pixel phase; obtaining optical imaging of multiple targets, and performing coarse positioning according to the optical imaging of the multiple targets to obtain initial positioning results of each target; modulating the optical imaging of the multiple targets according to the initial positioning results, sequentially displacing the optical imaging of each target to an optimal pixel phase for positioning to obtain multiple positioning results of each target at multiple pixel phases; determining the confidence weight of each positioning result according to the positioning precision limit, and fusing the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

Description

Technical Field

[0001] This application relates to the field of optical imaging precision measurement technology, and in particular to a time-division modulation positioning method, device and electronic device for multi-target optical imaging. Background Technology

[0002] Optical imaging measurement technology has wide applications in aerospace navigation, astronomical observation, and biological microscopy. Its core is to achieve high-precision position measurement of point targets (such as stars or single molecules) through sub-pixel-level positioning. Traditional fixed optical systems (lens and detector are rigidly connected) cannot modulate the target's imaging position. However, the Cramérault lower bound theory reveals that the positioning accuracy limit fluctuates significantly with the target's sub-pixel position (pixel phase), especially for small-sized point spread function (PSF) optical systems, where the accuracy fluctuation can reach five or even ten times or more. Research shows that, with a fixed number of photons, the positioning accuracy of a small-sized PSF at a specific pixel phase can significantly surpass that of a medium-sized or large-sized PSF. However, fixed systems cannot actively control the target's imaging position, resulting in limited practical accuracy.

[0003] Motion modulation technology, by actively adjusting the relative positions of the optical system or the target, moves the target to the optimal pixel phase, potentially overcoming the accuracy bottleneck of fixed systems. However, in multi-target scenarios (e.g., star sensors need to locate multiple stars simultaneously), the relative positions between targets are fixed. Motion modulation may optimize the positioning accuracy of one target, but leave other targets in a poor pixel phase. Therefore, the fixed relative positions of multiple targets limit the accuracy of motion modulation positioning in optical imaging. Summary of the Invention

[0004] In view of the above problems, embodiments of this application provide a time-division modulation positioning method, apparatus and electronic device for multi-target optical imaging, so as to overcome the above problems or at least partially solve the above problems.

[0005] A first aspect of this application discloses a time-division modulation positioning method for multi-target optical imaging, the method comprising:

[0006] The positioning accuracy limit of the optical system is evaluated to obtain the positioning accuracy limit of the target at each pixel phase, and the positioning accuracy limit characterizes the theoretical uncertainty of positioning.

[0007] Acquire optical images of multiple targets, and perform coarse localization based on the optical images of the multiple targets to obtain the initial localization results of each target;

[0008] Based on the initial positioning result, the optical imaging of the multiple targets is modulated, and the optical imaging of each target is sequentially shifted to the optimal pixel phase for positioning, thereby obtaining multiple positioning results of each target located in multiple pixel phases. The optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit.

[0009] The confidence weight of each positioning result is determined according to the positioning accuracy limit, and the multiple positioning results are fused according to the confidence weight to obtain the final positioning result of each target.

[0010] Optionally, the multiple localization results are fused according to the confidence weights to obtain the final localization result for each target, including:

[0011] The multiple positioning results are fused according to the confidence weights to obtain the fused positioning result of the target;

[0012] If the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, the fused positioning result of the target shall be taken as the final positioning result of the target.

[0013] Optionally, the method further includes:

[0014] If the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fused positioning result of the target is taken as the initial positioning result of the target, and the execution steps are returned: the optical imaging of the multiple targets is modulated according to the initial positioning result, and the optical imaging of each target is sequentially shifted to the optimal pixel phase for positioning, so as to obtain multiple positioning results of each target located in multiple pixel phases.

[0015] Optionally, the positioning accuracy limit of the optical system is evaluated to obtain the positioning accuracy limit of the target at each pixel phase, including:

[0016] An effective PSF model for target imaging in the optical system is constructed. The effective PSF model characterizes the precise correspondence between the target pixel phase and the pixel response. The pixel response is generated after the optical PSF is sampled by the image detector pixels of the optical system.

[0017] Based on the Cramer-Rao lower bound theory, the positioning accuracy limit of the target at each pixel phase is determined according to the target pixel phase, the target imaging area, and the effective PSF model.

[0018] Optionally, the optical imaging of the multiple targets is modulated according to the initial positioning result, and the optical imaging of each target is sequentially shifted to the optimal pixel phase for positioning, resulting in multiple positioning results of each target located at multiple pixel phases, including:

[0019] Based on the optimal pixel phase and the initial positioning result, a control quantity sequence for optical imaging motion modulation is determined. The control quantity in the control quantity sequence represents the displacement required to move the target optical imaging from the position of the initial positioning result to the optimal pixel phase.

[0020] According to the control quantity sequence, each target optical image is sequentially shifted to the optimal pixel phase and imaged with the same exposure time to obtain multiple optical images of multiple targets, and the number of multiple optical images is equal to the number of multiple targets.

[0021] The maximum likelihood method is used to locate the multiple optical images, resulting in multiple positioning results for each target at multiple pixel phases. Each positioning result of the optical image corresponds to a positioning result of one pixel phase.

[0022] Optionally, according to the control quantity sequence, the optical imaging of each target is sequentially shifted to the optimal pixel phase for imaging with the same exposure time, including:

[0023] The lens or image detector of the optical system is fixed by a motion actuator, and the optical system is driven to produce sub-pixel displacement according to the control quantity sequence, so as to sequentially shift the optical imaging of each target to the optimal pixel phase for imaging with the same exposure time.

[0024] Optionally, the confidence weight of each positioning result is determined based on the positioning accuracy limit, including:

[0025] For the localization result of the target in one pixel phase, the confidence weight of the target's localization result is obtained by summing the square of the target's localization accuracy limit in that pixel phase and the reciprocal of the square of the target's localization accuracy limit in each pixel phase; wherein, the sum of the confidence weights of the target's multiple localization results is equal to 1.

[0026] Optionally, the multiple positioning results are fused according to the confidence weights to obtain a fused positioning result of the target, including:

[0027] The difference between each positioning result and the displacement of the corresponding motion actuator relative to the initial positioning result is determined, and the fused positioning result of the target is obtained based on the difference and the confidence weight corresponding to the positioning result.

[0028] A second aspect of this application discloses a time-division modulation positioning device for multi-target optical imaging, the device comprising:

[0029] The first evaluation module is used to evaluate the positioning accuracy limit of the optical system and obtain the positioning accuracy limit of the target in each pixel phase. The positioning accuracy limit characterizes the theoretical uncertainty of positioning.

[0030] The first positioning module is used to acquire optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain the initial positioning results of each target.

[0031] The second positioning module is used to modulate the optical imaging of the multiple targets according to the initial positioning result, and sequentially shift the optical imaging of each target to the optimal pixel phase for positioning, thereby obtaining multiple positioning results of each target located in multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit.

[0032] The first fusion module is used to determine the confidence weight of each positioning result according to the positioning accuracy limit, and to fuse the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

[0033] A third aspect of this application discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the time-division modulation positioning method for multi-target optical imaging described in the first aspect of this application.

[0034] A fourth aspect of this application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the time-division modulation positioning method for multi-target optical imaging described in the first aspect of this application.

[0035] A fifth aspect of this application discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the time-division modulation positioning method for multi-target optical imaging described in the first aspect of this application.

[0036] The embodiments of this application have the following advantages:

[0037] In this embodiment, leveraging the characteristic of a small-sized PSF to achieve optimal positioning accuracy at a specific pixel phase, multi-target imaging is time-division modulated to the optimal pixel phase for positioning, resulting in multiple positioning results for each target at multiple pixel phases. Based on a confidence weight determined by the positioning accuracy limit, these multiple positioning results are fused to optimize the positioning result for each target. Therefore, this method achieves global accuracy optimization for multiple targets. Under the constraint of the relative positional relationships of multiple targets, the time-division modulation of optical imaging surpasses the positioning accuracy of fixed optical measurement techniques, expanding the application scenarios of motion-modulated optical measurement techniques. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the steps of a time-division modulation positioning method for multi-target optical imaging provided in an embodiment of this application.

[0040] Figure 2 This is a schematic diagram of a small-sized PSF and its pixel value distribution generated by a tightly focused imaging optical system according to an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of single-shot subpixel motion modulation and multi-target time-division modulation positioning provided in an embodiment of this application;

[0042] Figure 4 This is a flowchart of another time-division modulation positioning method for multi-target optical imaging provided in this application embodiment;

[0043] Figure 5 This is a schematic diagram of the structure of a time-division modulation positioning device for multi-target optical imaging provided in an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0045] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] To address the limitations on motion-modulated optical positioning accuracy in multi-target application scenarios, this application proposes a time-division modulation positioning method for multi-target optical imaging. This method modulates small PSF images of multiple targets to the optimal pixel phase for measurement, resulting in multiple measurement results (positioning results) for each target at multiple pixel phases. Different confidence weights are assigned based on the accuracy limit estimate, and optimal data fusion is performed, thereby enabling each target to obtain an optimized positioning result.

[0047] Reference Figure 1 As shown, Figure 1This is a flowchart illustrating the steps of a time-division modulation positioning method for multi-target optical imaging provided in an embodiment of this application. Figure 1 As shown, the time-division modulation localization method for multi-target optical imaging may include steps S110 to S140:

[0048] Step S110: Evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target in each pixel phase, wherein the positioning accuracy limit characterizes the theoretical uncertainty of positioning.

[0049] The optical system refers to a small-sized PSF optical system, characterized by a small lens F-number and minimal lens aberrations. This allows for complete focus of the image spot when imaging multiple targets, avoiding the defocusing operations commonly used in sub-pixel positioning of point targets. For example... Figure 2 As shown, Figure 2 The diagram illustrates the small-sized PSF and its pixel value distribution generated by the optical system's tight-focusing imaging. It can be seen that the pixel values ​​in the small-sized PSF are concentrated in a few pixels.

[0050] The positioning accuracy limit characterizes the theoretical uncertainty of positioning. In other words, the smaller the positioning accuracy limit of a target in pixel phase, the higher the reliability of the positioning result in pixel phase; the larger the positioning accuracy limit of a target in pixel phase, the lower the reliability of the positioning result in pixel phase.

[0051] The positioning accuracy limit of an optical system can be evaluated based on the Cramer-Rao lower bound theory, thereby obtaining the positioning accuracy limit (i.e., theoretical uncertainty) of the target at each pixel phase (i.e., sub-pixel position).

[0052] Step S120: Acquire optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain the initial positioning results of each target.

[0053] Optical imaging of multiple targets is acquired through an optical system. The optical imaging of multiple targets contains multiple target image spots (or multiple small-sized PSFs), and then the multiple target image spots are coarsely located based on the optical imaging of multiple targets.

[0054] In some embodiments, coarse localization based on optical imaging of multiple targets can be achieved through a centroid algorithm. Specifically, the target region (i.e., the region where the target is located) is extracted from the optical imaging of multiple targets, and the centroid is calculated based on the pixel value of each pixel in the target region. The pixel position corresponding to the centroid is used as the initial localization result of the target (i.e., the initial position of the target).

[0055] Step S130: Modulate the optical imaging of the multiple targets according to the initial positioning result, and sequentially shift the optical imaging of each target to the optimal pixel phase for positioning, thereby obtaining multiple positioning results of each target in multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit.

[0056] In this embodiment of the application, considering the characteristic that a small-sized PSF produces the best positioning accuracy at a specific pixel phase, the optical imaging of each target is shifted to the optimal pixel phase for positioning, so that each target produces positioning results (measurement results) at multiple pixel phases.

[0057] For each target, the target is moved from its initial location to the optimal pixel phase, and then located at that optimal pixel phase. It's understandable that since different targets may have different optimal pixel phases, moving the optical image of a target to the optimal pixel phase for localization yields the localization result of that target at that optimal pixel phase, as well as the localization results of the other targets at that pixel phase (which may not be the optimal pixel phase for the other targets). For example, given three targets (A, B, and C), if the optical image of target A is moved to the optimal pixel phase for localization, we obtain the localization result of target A at that optimal pixel phase, and the localization results of targets B and C at their corresponding pixel phases.

[0058] In other words, each motion modulation localization (MoM) operation yields a localization result for each target at one pixel phase. By sequentially performing MoM localization on each target, localization results at multiple pixel phases are obtained. For example, if there are N targets, N MoM operations are required, and each MoM operation produces N localization results. Figure 3 As shown, Figure 3 The diagram illustrates the single-shot subpixel motion modulation and multi-target time-division modulation localization process, targeting... Positioning is performed at the optimal pixel phase to obtain the positioning result with the best positioning accuracy, while the remaining targets... and This may not be the most accurate positioning result.

[0059] Step S140: Determine the confidence weight of each positioning result according to the positioning accuracy limit, and fuse the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

[0060] For each pixel phase, the localization result of the target is determined based on the localization accuracy limit of the target in that pixel phase. For example, for localization result 1 where the target is located in pixel phase 1, the confidence weight of localization result 1 can be determined based on the localization accuracy limit of the target in pixel phase 1.

[0061] Specifically, the smaller the positioning accuracy limit of the target in a pixel phase (i.e., the smaller the theoretical uncertainty), the greater the confidence weight of the positioning result of the target in that pixel phase. Then, based on the confidence weight corresponding to each positioning result, multiple positioning results of the target in multiple pixel phases are weighted and fused so that each target can obtain an optimized final positioning result.

[0062] The technical solution implemented in this application utilizes the characteristic of a small-sized PSF to generate optimal positioning accuracy at a specific pixel phase. Multi-target imaging is time-division modulated to the optimal pixel phase for positioning, resulting in multiple positioning results for each target at multiple pixel phases. Based on a confidence weight determined by the positioning accuracy limit, these multiple positioning results are fused to optimize the positioning result for each target. Therefore, this method achieves a comprehensive optimization of global accuracy for multiple targets. Under the constraint of the relative positional relationships of multiple targets, the time-division modulation of optical imaging surpasses the positioning accuracy of fixed optical measurement techniques, expanding the application scenarios of motion-modulated optical measurement techniques.

[0063] In conjunction with the above embodiments, in one embodiment, this application also provides a time-division modulation positioning method for multi-target optical imaging. In this method, the step S110 above, "evaluating the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target in each pixel phase," specifically includes sub-steps S110-1 to S110-2:

[0064] Step S110-1: Construct an effective PSF model for target imaging in the optical system. The effective PSF model characterizes the precise correspondence between the target pixel phase and the pixel response. The pixel response is generated after the optical PSF is sampled by the image detector pixels of the optical system.

[0065] The effective PSF (ePSF) model for target imaging in the optical system can be derived from the convolution of the analytical PSF function of the optical system and the pixel response, or from experimental statistics and modeling of the actual imaging of the target under test.

[0066] In some embodiments, the experimental statistics and modeling process of the actual imaging of the target under test is as follows: the sub-pixel displacement of the target image is generated by the motion actuator, and the image is acquired at different pixel phases; after preprocessing operations such as background removal and average filtering are performed on the acquired image, the precise correspondence between the target pixel phase (target sub-pixel position) and the pixel response is determined by the interpolation technique, that is, the ePSF model is obtained.

[0067] Step S110-2: Based on the Cramer-Rao lower bound theory, determine the positioning accuracy limit of the target at each pixel phase according to the target pixel phase, the target imaging area and the effective PSF model.

[0068] Specifically, when the target optical signal is much larger than the pixel dark noise, the Cramer-Rao lower bound theory is used to estimate the positioning accuracy limit of the target at each pixel phase. For example, the positioning accuracy limit of the target at each pixel phase... It can be represented as:

[0069]

[0070] in, and These represent the pixel phase of the target in the x and y directions, i.e., the sub-pixel position; i and j represent the row number and column number of the pixel, respectively; ROI represents the target imaging region. , Indicates the position of the target center relative to the pixel center; represents the ePSF value (effective PSF model) corresponding to pixel ij; K represents the pixel gain.

[0071] Through the above implementation process, based on the effective PSF model of the optical system target imaging, the Cramer-Rao lower bound theory is used to evaluate the positioning accuracy limit of the optical system, so as to assign confidence weights to the positioning results at multiple pixel phases of each target in the future.

[0072] In conjunction with the above embodiments, in one embodiment, this application also provides a time-division modulation positioning method for multi-target optical imaging. In this method, step S130, "modulating the optical imaging of the multiple targets according to the initial positioning result, sequentially shifting the optical imaging of each target to the optimal pixel phase for positioning, and obtaining multiple positioning results of each target located in multiple pixel phases," specifically includes sub-steps S130-1 to S130-3:

[0073] Step S130-1: Based on the optimal pixel phase and the initial positioning result, determine the control quantity sequence for optical imaging motion modulation. The control quantity in the control quantity sequence represents the displacement required to move the target optical imaging from the position of the initial positioning result to the optimal pixel phase.

[0074] Among them, the pixel phase that minimizes the positioning accuracy limit is taken as the optimal pixel phase. The sequence of control quantities for optical imaging motion modulation, based on the optimal pixel phase and the initial positioning result, refers to determining the displacement for each target to move its optical imaging from the position of the initial positioning result to the optimal pixel phase, and using the displacement as the control quantity for motion modulation of the target.

[0075] Step S130-2: According to the control quantity sequence, each target optical imaging is sequentially shifted to the optimal pixel phase and imaged with the same exposure time to obtain multiple optical imaging of multiple targets, the number of multiple optical imaging is equal to the number of multiple targets.

[0076] Each control quantity in the control quantity sequence corresponds to the motion modulation of a target. By using each control quantity in the control quantity sequence in sequence, the optical imaging is sequentially displaced at the sub-pixel level, thereby shifting the corresponding target optical imaging from the position of the initial positioning result to the optimal pixel phase.

[0077] Specifically, according to the control quantity sequence, each target optical image is sequentially shifted to the optimal pixel phase for imaging with the same exposure time, including: using a motion actuator to fix the lens or image detector of the optical system, and driving the optical system to generate sub-pixel displacement according to the control quantity sequence, thereby sequentially shifting each target optical image to the optimal pixel phase for imaging with the same exposure time.

[0078] Among them, the motion actuator is a high-precision motion actuator. After the motion actuator is fixed to the lens or image detector of the optical system, the motion actuator can drive the optical system to produce sub-pixel displacement, so as to produce the effect of motion modulation of the target image spot on the image detector.

[0079] Step S130-3: Use the maximum likelihood method to locate the multiple optical images and obtain multiple positioning results of each target at multiple pixel phases. Each positioning result of the optical imaging corresponds to the positioning result of one pixel phase.

[0080] Specifically, when establishing the likelihood function based on the maximum likelihood method, the number of photoelectrons detected by the pixel is set to follow a Poisson distribution to form a pixel response probability density function. The probability of the current image response is set to the negative natural logarithm as the maximum likelihood fitting cost function. The position that minimizes the cost function is the image spot localization result, i.e., the target localization result.

[0081] Through the above implementation process, the small PSF imaging of multiple targets is time-division modulated to the optimal pixel phase for measurement, so that each target generates localization results at multiple pixel phases. Subsequently, the optimized final localization result can be determined based on the localization results at multiple pixel phases.

[0082] In conjunction with the above embodiments, in one embodiment, this application also provides a time-division modulation positioning method for multi-target optical imaging. In this method, the step S140 above, "determining the confidence weight of each positioning result according to the positioning accuracy limit," specifically includes:

[0083] For the localization result of the target in one pixel phase, the confidence weight of the target's localization result is obtained by summing the square of the target's localization accuracy limit in that pixel phase and the reciprocal of the square of the target's localization accuracy limit in each pixel phase; wherein, the sum of the confidence weights of the target's multiple localization results is equal to 1.

[0084] For example, if n localization results for each target at n pixel phases are assigned confidence weights according to the localization accuracy limit, then the confidence weight of the k-th localization result for a certain target is... It can be represented as:

[0085]

[0086] in, Indicates the target in pixel phase ( , The positioning accuracy limit on ) Indicates the target in pixel phase ( , The positioning accuracy limit on ).

[0087] The confidence weights of multiple location results for the target satisfy:

[0088]

[0089] in, This indicates the first location result of the target. This indicates the second location result of the target. This represents the nth location result of the target. In other words, the sum of the confidence weights of the multiple location results of the target is equal to 1.

[0090] In some embodiments, step S140 above, "fusing the multiple positioning results according to the confidence weight to obtain the final positioning result of each target," specifically includes sub-steps S140-1 to S140-2:

[0091] Step S140-1: The multiple positioning results are fused according to the confidence weight to obtain the fused positioning result of the target.

[0092] Specifically, the difference between each positioning result and the displacement of the corresponding motion actuator relative to the initial positioning result is determined, and the target's fused positioning result is obtained based on the difference and the confidence weight corresponding to the positioning result.

[0093] For example, the n localization results of the target are fused according to the confidence weights to obtain the fused localization result of the target. It can be represented as:

[0094]

[0095] in, This represents the displacement of the motion actuator relative to the initial positioning result corresponding to the k-th positioning result. This represents the k-th location result.

[0096] Furthermore, the uncertainty of the target fusion positioning results It can be represented as:

[0097]

[0098] Step S140-2: If the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, the fused positioning result of the target shall be taken as the final positioning result of the target.

[0099] In this embodiment, after obtaining the fused positioning result, it is determined whether the difference between the current fused positioning result and the initial positioning result of the target is less than a position threshold. The position threshold is set mainly considering whether the target has an effective measurement result modulated at the optimal pixel phase. If it is less than the threshold, step S140-2 is executed, and the fused positioning result of the target is taken as the final positioning result of the target.

[0100] The location threshold setting primarily considers whether the target has an effective measurement result modulated at the optimal pixel phase. In other words, if the difference between the target's fused localization result and its initial localization result is less than the location threshold, it indicates that the target's fused localization result has an effective measurement result modulated at the optimal pixel phase. Therefore, this fused localization result can be used as the target's final localization result, ensuring that each target obtains an optimized localization result.

[0101] Furthermore, the method further includes step S140-3:

[0102] Step S140-3: If the difference between the fusion positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fusion positioning result of the target is taken as the initial positioning result of the target, and the process returns to step S130: Modulate the optical imaging of the multiple targets according to the initial positioning result, and sequentially shift the optical imaging of each target to the optimal pixel phase for positioning, thereby obtaining multiple positioning results of each target located in multiple pixel phases.

[0103] In this embodiment, if the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, it indicates that the fused positioning result of the target may not have an effective measurement result modulated at the optimal pixel phase. Then, return to step S130 to continue to perform motion modulation positioning on multiple targets until the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, and end the iteration to obtain a high-precision positioning result for multiple targets.

[0104] Through the above implementation process, under the constraint of the relative positional relationship of multiple targets, the positioning accuracy of optical imaging time-division modulation is achieved, surpassing that of fixed optical measurement technology, thus expanding the application scope of motion modulation optical measurement technology.

[0105] The time-division modulation positioning method for multi-target optical imaging in this application will be described below with reference to a specific embodiment. Figure 4 As shown, Figure 4 This is a flowchart illustrating the steps of another time-division modulation localization method for multi-target optical imaging provided in this application embodiment. Specifically, the time-division modulation localization method for multi-target optical imaging includes the following steps S410 to S490:

[0106] Step S410: Construct an effective PSF model for target imaging in the optical system. The effective PSF model characterizes the precise correspondence between the target pixel phase and the pixel response. The pixel response is generated after the optical PSF is sampled by the image detector pixels of the optical system.

[0107] Step S420: Based on the Cramer-Rao lower bound theory, determine the positioning accuracy limit of the target at each pixel phase according to the target pixel phase, the target imaging area and the effective PSF model.

[0108] Step S430: Acquire optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain the initial positioning results of each target.

[0109] Step S440: Based on the optimal pixel phase and the initial positioning result, determine the control quantity sequence for optical imaging motion modulation. The control quantity in the control quantity sequence represents the displacement required to move the target optical imaging from the position of the initial positioning result to the optimal pixel phase. The optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit.

[0110] Step S450: According to the control quantity sequence, each target optical imaging is sequentially shifted to the optimal pixel phase and imaged with the same exposure time to obtain multiple optical imaging of multiple targets, the number of multiple optical imaging is equal to the number of multiple targets.

[0111] Step S460: Use the maximum likelihood method to locate the multiple optical images to obtain multiple positioning results of each target at multiple pixel phases. Each positioning result of the optical image corresponds to the positioning result of one pixel phase.

[0112] Step S470: The multiple positioning results are fused according to the confidence weight to obtain the fused positioning result of the target.

[0113] Step S480: If the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fused positioning result of the target is taken as the initial positioning result of the target, and the process returns to step S440.

[0114] Step S490: If the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, the fused positioning result of the target shall be taken as the final positioning result of the target.

[0115] In this embodiment, leveraging the characteristic of a small-sized PSF to achieve optimal positioning accuracy at a specific pixel phase, multi-target imaging is time-division modulated to the optimal pixel phase for positioning, resulting in multiple positioning results for each target at multiple pixel phases. Based on a confidence weight determined by the positioning accuracy limit, these multiple positioning results are fused to optimize the positioning result for each target. Therefore, this method achieves global accuracy optimization for multiple targets. Under the constraint of the relative positional relationships of multiple targets, the time-division modulation of optical imaging surpasses the positioning accuracy of fixed optical measurement techniques, expanding the application scenarios of motion-modulated optical measurement techniques.

[0116] This application also provides a time-division modulation positioning device for multi-target optical imaging, referring to... Figure 5 As shown, Figure 5 This is a schematic diagram of a time-division modulation positioning device for multi-target optical imaging provided in an embodiment of this application. The device includes:

[0117] The first evaluation module 510 is used to evaluate the positioning accuracy limit of the optical system and obtain the positioning accuracy limit of the target in each pixel phase. The positioning accuracy limit characterizes the theoretical uncertainty of positioning.

[0118] The first positioning module 520 is used to acquire optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain the initial positioning results of each target.

[0119] The second positioning module 530 is used to modulate the optical imaging of the multiple targets according to the initial positioning result, and sequentially shift the optical imaging of each target to the optimal pixel phase for positioning, thereby obtaining multiple positioning results of each target located in multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit.

[0120] The first fusion module 540 is used to determine the confidence weight of each positioning result according to the positioning accuracy limit, and to fuse the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

[0121] It is understood that the time-division modulation positioning device for multi-target optical imaging in the embodiments of this application can realize the time-division modulation positioning method for multi-target optical imaging in the above embodiments. The time-division modulation positioning device for multi-target optical imaging and the time-division modulation positioning method for multi-target optical imaging have the same advantages over the prior art, and will not be repeated here.

[0122] This application also provides an electronic device, see embodiments thereof. Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 6 As shown, the electronic device 600 includes a memory 610 and a processor 620. The memory 610 and the processor 620 are connected via a bus for communication. The memory 610 stores a computer program that can run on the processor 620 to implement the steps of the time-division modulation positioning method for multi-target optical imaging described in the embodiments of this application.

[0123] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the time-division modulation positioning method for multi-target optical imaging described in this application.

[0124] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the time-division modulation positioning method for multi-target optical imaging described in this application.

[0125] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0126] This application describes embodiments of methods and apparatus according to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0129] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0130] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0131] The above provides a detailed description of a time-division modulation positioning method, apparatus, and electronic device for multi-target optical imaging provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A time-division modulation positioning method for multi-target optical imaging, characterized in that, The method includes: The positioning accuracy limit of the optical system is evaluated to obtain the positioning accuracy limit of the target at each pixel phase. This includes: constructing an effective PSF model for target imaging in the optical system, wherein the effective PSF model characterizes the precise correspondence between the target pixel phase and the pixel response, and the pixel response is generated by the optical PSF after pixel sampling by the image detector of the optical system; and determining the positioning accuracy limit of the target at each pixel phase based on the Cramer-Rao lower bound theory, according to the target pixel phase, the target imaging area, and the effective PSF model, wherein the positioning accuracy limit characterizes the theoretical uncertainty of positioning. Acquire optical images of multiple targets, and perform coarse localization based on the optical images of the multiple targets to obtain the initial localization results of each target; The optical imaging of the multiple targets is modulated according to the initial positioning result. Each target's optical imaging is sequentially moved to the optimal pixel phase for positioning, resulting in multiple positioning results for each target at multiple pixel phases. This includes: determining a control sequence for optical imaging motion modulation based on the optimal pixel phase and the initial positioning result, where the control quantity in the control sequence represents the displacement required to move the target's optical imaging from the position of the initial positioning result to the optimal pixel phase; sequentially moving each target's optical imaging to the optimal pixel phase for imaging with the same exposure time based on the control sequence, resulting in multiple optical imagings of the multiple targets, the number of which is equal to the number of targets; and positioning the multiple optical imagings using the maximum likelihood method to obtain multiple positioning results for each target at multiple pixel phases, where each optical imaging positioning result corresponds to a pixel phase positioning result, and the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit. The confidence weight of each positioning result is determined based on the positioning accuracy limit, including: for the positioning result of the target in one pixel phase, the confidence weight of the target positioning result is obtained by summing the square of the positioning accuracy limit of the target in that pixel phase and the reciprocal of the square of the positioning accuracy limit of the target in each pixel phase; wherein the sum of the confidence weights of multiple positioning results of the target is equal to 1; and the multiple positioning results are fused according to the confidence weight to obtain the final positioning result of each target.

2. The time-division modulation positioning method for multi-target optical imaging according to claim 1, characterized in that, The multiple localization results are fused according to the confidence weights to obtain the final localization result for each target, including: The multiple positioning results are fused according to the confidence weights to obtain the fused positioning result of the target; If the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, the fused positioning result of the target shall be taken as the final positioning result of the target.

3. The time-division modulation positioning method for multi-target optical imaging according to claim 2, characterized in that, The method further includes: If the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fused positioning result of the target is taken as the initial positioning result of the target, and the execution steps are returned: the optical imaging of the multiple targets is modulated according to the initial positioning result, and the optical imaging of each target is sequentially shifted to the optimal pixel phase for positioning, so as to obtain multiple positioning results of each target located in multiple pixel phases.

4. The time-division modulation positioning method for multi-target optical imaging according to claim 1, characterized in that, According to the control quantity sequence, the optical imaging of each target is sequentially shifted to the optimal pixel phase for imaging with the same exposure time, including: The lens or image detector of the optical system is fixed by a motion actuator, and the optical system is driven to produce sub-pixel displacement according to the control quantity sequence, so as to sequentially shift the optical imaging of each target to the optimal pixel phase for imaging with the same exposure time.

5. The time-division modulation positioning method for multi-target optical imaging according to claim 4, characterized in that, The multiple localization results are fused according to the confidence weights to obtain the fused localization result of the target, including: The difference between each positioning result and the displacement of the corresponding motion actuator relative to the initial positioning result is determined, and the fused positioning result of the target is obtained based on the difference and the confidence weight corresponding to the positioning result.

6. A time-division modulation positioning device for multi-target optical imaging, characterized in that, The apparatus for implementing the time-division modulation positioning method for multi-target optical imaging according to any one of claims 1-5 includes: The first evaluation module is used to evaluate the positioning accuracy limit of the optical system and obtain the positioning accuracy limit of the target in each pixel phase. The positioning accuracy limit characterizes the theoretical uncertainty of positioning. The first positioning module is used to acquire optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain the initial positioning results of each target. The second positioning module is used to modulate the optical imaging of the multiple targets according to the initial positioning result, and sequentially shift the optical imaging of each target to the optimal pixel phase for positioning, thereby obtaining multiple positioning results of each target located in multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit. The first fusion module is used to determine the confidence weight of each positioning result according to the positioning accuracy limit, and to fuse the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the time-division modulation positioning method for multi-target optical imaging as described in any one of claims 1-5.

Images