Time-sharing modulation positioning method and device for multi-target optical imaging and electronic equipment

Abstract

The invention provides a time-sharing modulation positioning method and device for multi-target optical imaging and electronic equipment, relates to the technical field of optical imaging type precision measurement, and aims to solve the problem of restriction of a multi-target application scene on motion modulation type optical positioning precision. The method comprises the following steps: evaluating the positioning precision limit of an optical system to obtain the positioning precision limit of a target on each pixel phase; multi-target optical imaging is obtained, coarse positioning is carried out according to the multi-target optical imaging, and an initial positioning result of each target is obtained; modulating optical imaging of the multiple targets according to the initial positioning result, sequentially shifting the optical imaging of each target to an optimal pixel phase for positioning, and obtaining a plurality of positioning results of each target in a plurality of pixel phases; and determining the confidence weight of each positioning result according to the positioning precision limit, and fusing the plurality of positioning results according to the confidence weight to obtain the final positioning result of each target.

Description

Technical Field

[0001] The present application relates to the field of optical imaging precision measurement technology, and in particular to a time-sharing modulation positioning method, device and electronic equipment for multi-target optical imaging. Background Art

[0002] Optical imaging measurement technology has wide applications in aerospace navigation, astronomical observation, biological microscopy, and other fields. Its core is to achieve high-precision position measurement of point targets (such as stars and single molecules) through sub-pixel positioning. Traditional fixed optical systems (where the lens and detector are rigidly connected) cannot modulate the target imaging position. However, the Cramer-Rao lower bound theory reveals that the positioning accuracy limit fluctuates significantly with the sub-pixel position (pixel phase) of the target. This is especially true for small-scale point spread function (PSF) optical systems, where the accuracy fluctuation can reach five times or even more than ten times. Research has shown that for a certain number of photons, the positioning accuracy of a small-scale PSF at a specific pixel phase can significantly exceed that of a medium- or large-scale PSF. However, fixed systems cannot actively adjust the target imaging position, resulting in limited actual accuracy.

[0003] Motion modulation technology actively adjusts the relative position of the optical system or target, shifting the target to the optimal pixel phase, potentially breaking through the accuracy bottleneck of fixed systems. However, in multi-target scenarios (for example, a star sensor needs to locate multiple stars simultaneously), where the targets are fixed relative to each other, motion modulation may optimize the positioning accuracy of one target while leaving other targets at a less favorable pixel phase. Therefore, the fixed relative position of multiple targets limits the accuracy of motion modulation positioning in optical imaging. Summary of the Invention

[0004] In view of the above problems, the embodiments of the present application provide a time-sharing modulation positioning method, device and electronic device for multi-target optical imaging to overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect of the embodiments of the present application, a time-sharing modulation positioning method for multi-target optical imaging is disclosed, the method comprising: Evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase, wherein the positioning accuracy limit represents the theoretical uncertainty of positioning; Acquire optical images of multiple targets, and perform coarse positioning based on the optical images of the multiple targets to obtain initial positioning results for each target; Modulating the optical imaging of the multiple targets according to the initial positioning results, sequentially shifting the optical imaging of each target to an optimal pixel phase for positioning, and obtaining multiple positioning results of each target at multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit; The confidence weight of each positioning result is determined according to the positioning accuracy limit, and the multiple positioning results are fused according to the confidence weight to obtain the final positioning result of each target.

[0006] Optionally, fusing the multiple positioning results according to the confidence weight to obtain a final positioning result for each target includes: fusing the multiple positioning results according to the confidence weights to obtain a fused positioning result of the target; When the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, the fused positioning result of the target is used as the final positioning result of the target.

[0007] Optionally, the method further includes: When the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fused positioning result of the target is used as the initial positioning result of the target, and the execution step is returned to: modulating the optical imaging of the multiple targets according to the initial positioning result, shifting the optical imaging of each target to the optimal pixel phase in turn for positioning, and obtaining multiple positioning results of each target at multiple pixel phases.

[0008] Optionally, evaluating the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase includes: Constructing an effective PSF model for target imaging of the optical system, wherein the effective PSF model represents a precise correspondence between a target pixel phase and a pixel response, wherein the pixel response is generated by sampling the optical PSF by pixels of an image detector of the optical system; Based on the Cramer-Rao lower bound theory, the positioning accuracy limit of the target at each pixel phase is determined according to the target pixel phase, the target imaging area and the effective PSF model.

[0009] Optionally, modulating the optical imaging of the multiple targets according to the initial positioning results, sequentially shifting the optical imaging of each target to an optimal pixel phase for positioning, and obtaining multiple positioning results in which each target is located at multiple pixel phases, includes: Determining a control amount sequence for optical imaging motion modulation based on the optimal pixel phase and the initial positioning result, wherein the control amount in the control amount sequence represents a displacement required to move the target optical imaging from the position of the initial positioning result to the optimal pixel phase; According to the control amount sequence, the optical imaging of each target is sequentially shifted to the optimal pixel phase and imaged with the same exposure time to obtain multiple optical imaging of the multiple targets, where the number of the multiple optical imaging is equal to the number of the multiple targets; The multiple optical imagings are positioned using a maximum likelihood method to obtain multiple positioning results of each target at multiple pixel phases, where each optical imaging positioning result corresponds to a positioning result of one pixel phase.

[0010] Optionally, according to the control amount sequence, sequentially shifting the optical imaging of each target to the optimal pixel phase to perform imaging with the same exposure time includes: A motion actuator is used to fix the lens or image detector of the optical system, and drives the optical system to produce sub-pixel displacement according to the control quantity sequence, and sequentially shifts each target optical imaging to the optimal pixel phase for imaging with the same exposure time.

[0011] Optionally, determining a confidence weight of each positioning result according to the positioning accuracy limit includes: For the positioning result of the target at a pixel phase, the confidence weight of the target positioning result is obtained according to the square result of the positioning accuracy limit of the target at the pixel phase and the sum of the reciprocals of the square results of the positioning accuracy limit of the target at each pixel phase; wherein the sum of the confidence weights of multiple positioning results of the target is equal to 1.

[0012] Optionally, fusing the multiple positioning results according to the confidence weight to obtain a fused positioning result of the target includes: The difference between each positioning result and the displacement of the motion actuator corresponding to the positioning result relative to the initial positioning result is determined, and a fused positioning result of the target is obtained based on the difference and the confidence weight corresponding to the positioning result.

[0013] A second aspect of the embodiments of the present application discloses a time-sharing modulation positioning device for multi-target optical imaging, the device comprising: A first evaluation module is used to evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase, wherein the positioning accuracy limit represents the theoretical uncertainty of positioning; A first positioning module is used to obtain optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain initial positioning results for each target; a second positioning module, configured to modulate the optical imaging of the multiple targets according to the initial positioning results, sequentially shift the optical imaging of each target to an optimal pixel phase for positioning, and obtain multiple positioning results in which each target is located at multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit; The first fusion module is used to determine the confidence weight of each positioning result according to the positioning accuracy limit, and fuse the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

[0014] The third aspect of the embodiments of the present application discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the steps of the time-sharing modulation positioning method for multi-target optical imaging described in the first aspect of the embodiments of the present application are implemented.

[0015] The fourth aspect of the embodiments of the present application discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the time-sharing modulation positioning method for multi-target optical imaging described in the first aspect of the embodiments of the present application are implemented.

[0016] The fifth aspect of the embodiments of the present application discloses a computer program product, including a computer program, which, when executed by a processor, implements the steps of the time-sharing modulation positioning method for multi-target optical imaging described in the first aspect of the embodiments of the present application.

[0017] The embodiments of the present application include the following advantages: In the embodiments of this application, the characteristic of a small-scale PSF that produces optimal positioning accuracy at a specific pixel phase is utilized to time-share modulate the imaging of multiple targets to the optimal pixel phase for positioning. This results in multiple positioning results for each target at multiple pixel phases. These multiple positioning results are then fused according to the confidence weights determined by the positioning accuracy limit, thereby optimizing the positioning result for each target. Thus, a method for optimizing the global accuracy of multiple targets is achieved. Under the constraints of the relative positional relationships of multiple targets, the time-share modulation of optical imaging by this method surpasses the positioning accuracy of fixed optical measurement technology and expands the application scenarios of motion-modulated optical measurement technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a flowchart of the steps of a time-sharing modulation positioning method for multi-target optical imaging provided by an embodiment of the present application; Figure 2 Schematic diagram of a small-size PSF and its pixel value distribution generated by tight focus imaging of an optical system provided in an embodiment of the present application; Figure 3 This is a schematic diagram of single sub-pixel motion modulation and multi-target time-sharing modulation positioning provided by an embodiment of the present application; Figure 4 This is a flowchart of another method for time-sharing modulation positioning of multi-target optical imaging provided by an embodiment of the present application; Figure 5 This is a schematic structural diagram of a time-sharing modulation positioning device for multi-target optical imaging provided by an embodiment of the present application; Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] To make the above-mentioned purposes, features, and advantages of this application more clearly understood, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of this application.

[0021] In response to the constraints of multi-target application scenarios on the accuracy of motion-modulated optical positioning, the embodiment of the present application proposes a time-sharing modulation positioning method for multi-target optical imaging, which time-sharing modulates the small PSF imaging of multiple targets to the optimal pixel phase for measurement, so that each target produces measurement results (positioning results) at multiple pixel phases. Different confidence weights are assigned according to the accuracy limit estimate, and optimal data fusion is performed, so that each target obtains an optimized positioning result.

[0022] Reference Figure 1 As shown, Figure 1 This is a flowchart of the steps of a time-sharing modulation positioning method for multi-target optical imaging provided by an embodiment of the present application. Figure 1 As shown, the time-sharing modulation positioning method for multi-target optical imaging may include steps S110 to S140: Step S110: Evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase, where the positioning accuracy limit represents the theoretical uncertainty of positioning.

[0023] The optical system refers to a small-size PSF optical system, which is characterized by a small lens F number and small lens aberration, etc., which makes the image spot fully focused when imaging multiple targets, avoiding the defocusing operation commonly used in sub-pixel positioning of point targets. Figure 2 As shown, Figure 2 The diagram shows the small-size PSF and its pixel value distribution generated by the tightly focused imaging of the optical system. It can be seen that the pixel values ​​in the small-size PSF are concentrated in a few pixels.

[0024] The positioning accuracy limit characterizes the theoretical uncertainty of positioning. That is to say, the smaller the positioning accuracy limit of the target at the pixel phase, the higher the credibility of the positioning result of the target at the pixel phase; the larger the positioning accuracy limit of the target at the pixel phase, the lower the credibility of the positioning result of the target at the pixel phase.

[0025] The positioning accuracy limit of the optical system can be evaluated based on the Cramer-Rao lower bound theory, thereby obtaining the positioning accuracy limit (i.e., theoretical uncertainty) of the target at each pixel phase (i.e., sub-pixel position).

[0026] Step S120: Acquire optical images of multiple targets, and perform coarse positioning based on the optical images of the multiple targets to obtain initial positioning results for each target.

[0027] Optical imaging of multiple targets is acquired through an optical system, wherein the optical imaging of the multiple targets includes multiple target image spots (or multiple small-size PSFs), and then the multiple target image spots are roughly positioned according to the optical imaging of the multiple targets.

[0028] In some embodiments, coarse positioning based on optical imaging of multiple targets can be achieved through a centroid algorithm. Specifically, the target area (i.e., the area where the target position is located) is extracted from the optical imaging of multiple targets, and the centroid is calculated based on the pixel values ​​of each pixel in the target area, and the pixel position corresponding to the centroid is used as the initial positioning result of the target (i.e., the initial position of the target).

[0029] Step S130: Modulate the optical imaging of the multiple targets according to the initial positioning results, and shift the optical imaging of each target to the optimal pixel phase in turn for positioning, to obtain multiple positioning results of each target at multiple pixel phases, where the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit.

[0030] In the embodiment of the present application, considering the characteristic that a small-size PSF produces optimal positioning accuracy at a specific pixel phase, the optical imaging of each target is shifted to the optimal pixel phase for positioning, so that each target produces positioning results (measurement results) at multiple pixel phases.

[0031] For each target, the target is moved from the position of the initial positioning result to the optimal pixel phase, and then positioned at the optimal pixel phase. It is understandable that since the optimal pixel phases corresponding to different targets may be different, when the optical imaging of a target is shifted to the optimal pixel phase for positioning, the positioning result of the target at the optimal pixel phase can be obtained, as well as the positioning results of the remaining targets at that pixel phase (which may not be the optimal pixel phase for the remaining targets). For example, there are three targets (A, B, C). If the optical imaging of target A is shifted to the optimal pixel phase for positioning, the positioning result of target A at the optimal pixel phase is obtained, as well as the positioning results of targets B and C at their corresponding pixel phases.

[0032] That is to say, each time the motion modulation positioning is performed, the positioning result of each target is located at one pixel phase. If the motion modulation positioning is performed on each target in turn, the positioning results of each target at multiple pixel phases will be obtained. For example, if there are N targets, N motion modulations are required, and each motion modulation has N positioning results. Figure 3 As shown, Figure 3 The single sub-pixel motion modulation and multi-target time-sharing modulation positioning process are shown. Positioning is performed at the optimal pixel phase to obtain the positioning result with the best positioning accuracy, while the remaining targets and This is not necessarily the most accurate positioning result.

[0033] Step S140: determining the confidence weight of each positioning result according to the positioning accuracy limit, and fusing the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

[0034] For each pixel phase of the target's positioning result, a corresponding confidence weight is determined based on the target's positioning accuracy limit at that pixel phase. For example, for positioning result 1, where the target is located at pixel phase 1, the confidence weight for positioning result 1 can be determined based on the target's positioning accuracy limit at pixel phase 1.

[0035] The smaller the target's positioning accuracy limit at a pixel phase (i.e., the smaller the theoretical uncertainty), the greater the confidence weight of the positioning result for the target at that pixel phase. Based on the confidence weight corresponding to each positioning result, multiple positioning results for the target at multiple pixel phases are weighted and fused, resulting in an optimized final positioning result for each target.

[0036] The technical solution implemented in this application utilizes the characteristic of a small-scale PSF that produces optimal positioning accuracy at a specific pixel phase. Multi-target imaging is time-modulated to the optimal pixel phase for positioning, resulting in multiple positioning results for each target at multiple pixel phases. These multiple positioning results are then fused based on confidence weights determined by the positioning accuracy limit, thereby optimizing the positioning result for each target. This achieves a method for optimizing global precision while taking into account the relative positional relationships of multiple targets. This method's time-modulation of optical imaging surpasses the positioning accuracy of fixed optical measurement techniques, expanding the application scenarios of motion-modulated optical measurement techniques.

[0037] In combination with the above embodiments, in one embodiment, the present application further provides a time-sharing modulation positioning method for multi-target optical imaging. In this method, the step S110 of "evaluating the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase" specifically includes sub-steps S110-1 to S110-2: Step S110 - 1 : constructing an effective PSF model for target imaging of the optical system, wherein the effective PSF model represents the precise correspondence between the target pixel phase and the pixel response, where the pixel response is generated after the optical PSF is sampled by the image detector pixels of the optical system.

[0038] The effective PSF (ePSF) model of the optical system target imaging may be constructed by convolution of the optical system analytical PSF function and pixel response, or may be obtained by experimental statistics and modeling of actual imaging of the target to be measured.

[0039] In some embodiments, the process of experimental statistics and modeling of actual imaging of the target to be measured is: using a motion actuator to generate sub-pixel displacement of the target image, and collecting images at different pixel phases; after performing pre-processing operations such as background removal and average filtering on the collected image, interpolation technology is used to determine the precise correspondence between the target pixel phase (target sub-pixel position) and the pixel response, thereby obtaining an ePSF model.

[0040] Step S110 - 2 : Based on the Cramer-Rao lower bound theory, according to the target pixel phase, the target imaging area and the effective PSF model, determine the positioning accuracy limit of the target at each pixel phase.

[0041] Specifically, when the target optical signal is much larger than the pixel dark noise, the Cramer-Rao lower bound theory is used to estimate the positioning accuracy limit of the target at each pixel phase. For example, the positioning accuracy limit of the target at each pixel phase is It can be expressed as:

[0042] in, and Respectively represent the pixel phase of the target in the x and y directions, that is, the sub-pixel position; i and j represent the row and column numbers of the pixel, respectively; ROI represents the target imaging area; , Indicates the position of the target center relative to the pixel center; Represents the ePSF value (effective PSF model) corresponding to pixel ij; K represents the pixel gain.

[0043] Through the above implementation process, based on the effective PSF model of target imaging in the optical system, the Cramer-Rao lower bound theory is used to realize the evaluation of the positioning accuracy limit of the optical system, so as to assign confidence weights to the positioning results at multiple pixel phases for each subsequent target.

[0044] In combination with the above embodiments, in one embodiment, the embodiments of the present application further provide a time-sharing modulation positioning method for optical imaging of multiple targets. In this method, the above step S130 of "modulating the optical imaging of the multiple targets according to the initial positioning results, sequentially shifting the optical imaging of each target to the optimal pixel phase for positioning, and obtaining multiple positioning results of each target at multiple pixel phases" specifically includes sub-steps S130-1 to S130-3: Step S130-1: Determine a control quantity sequence for optical imaging motion modulation based on the optimal pixel phase and the initial positioning result, wherein the control quantity in the control quantity sequence represents the displacement required to move the target optical imaging from the position of the initial positioning result to the optimal pixel phase.

[0045] Among them, the pixel phase that makes the positioning accuracy limit achieve the minimum value is taken as the optimal pixel phase , based on the optimal pixel phase and the initial positioning result, determining the control amount sequence of optical imaging motion modulation means: for each target, determining the displacement of moving the target optical imaging from the position of the initial positioning result to the optimal pixel phase, and using the displacement as the control amount for motion modulation of the target.

[0046] Step S130-2: According to the control amount sequence, the optical imaging of each target is shifted to the optimal pixel phase in turn for imaging with the same exposure time, to obtain multiple optical imaging of multiple targets, where the number of the multiple optical imaging is equal to the number of the multiple targets.

[0047] Each control quantity in the control quantity sequence corresponds to the motion modulation of a target. By using each control quantity in the control quantity sequence in turn, the optical imaging produces sub-pixel precision displacement in turn, and the corresponding target optical imaging is shifted from the position of the initial positioning result to the optimal pixel phase.

[0048] Specifically, according to the control amount sequence, each target optical imaging displacement is sequentially moved to the optimal pixel phase for imaging with the same exposure time, including: using a motion actuator to fix the lens or image detector of the optical system, and driving the optical system to generate sub-pixel displacement according to the control amount sequence, and sequentially moving each target optical imaging displacement to the optimal pixel phase for imaging with the same exposure time.

[0049] Among them, the motion actuator is a high-precision motion actuator. After the motion actuator is fixed to the lens or image detector of the optical system, the motion actuator can drive the optical system to produce sub-pixel displacement to produce the effect of motion modulation of the target image spot on the image detector.

[0050] Step S130 - 3 : Using the maximum likelihood method to locate the multiple optical imagings, a plurality of positioning results of each target at a plurality of pixel phases are obtained, wherein each optical imaging positioning result corresponds to a positioning result of a pixel phase.

[0051] Specifically, when establishing the likelihood function based on the maximum likelihood method, the number of photoelectrons detected by the pixel is set to obey the Poisson distribution to form a pixel response probability density function. The probability of the current image response is taken as the negative natural logarithm as the maximum likelihood fitting cost function, and the position that minimizes the cost function is the image spot positioning result, that is, the positioning result of the target.

[0052] Through the above implementation process, the small PSF imaging of multiple targets is time-shared modulated to the optimal pixel phase for measurement, so that each target produces positioning results at multiple pixel phases, so that the optimized final positioning result can be determined based on the positioning results at multiple pixel phases.

[0053] In combination with the above embodiments, in one embodiment, the present application further provides a time-sharing modulation positioning method for multi-target optical imaging. In this method, the step S140 of "determining the confidence weight of each positioning result according to the positioning accuracy limit" specifically includes: For the positioning result of the target at a pixel phase, the confidence weight of the target positioning result is obtained according to the square result of the positioning accuracy limit of the target at the pixel phase and the sum of the reciprocals of the square results of the positioning accuracy limit of the target at each pixel phase; wherein the sum of the confidence weights of multiple positioning results of the target is equal to 1.

[0054] For example, the n positioning results of each target at n pixel phases are assigned confidence weights according to the positioning accuracy limit. Then the confidence weight of the kth positioning result of a target is It can be expressed as:

[0055] in, Indicates the target is in pixel phase ( , ), Indicates the target is in pixel phase ( , ) on the positioning accuracy limit.

[0056] The confidence weights of multiple positioning results of the target satisfy:

[0057] in, Indicates the first positioning result of the target, Indicates the second positioning result of the target, represents the nth positioning result of the target. That is, the sum of the confidence weights of multiple positioning results of the target is equal to 1.

[0058] In some embodiments, the step S140 of “fusing the multiple positioning results according to the confidence weight to obtain the final positioning result of each target” specifically includes sub-steps S140-1 to S140-2: Step S140 - 1 : Fusing the multiple positioning results according to the confidence weight to obtain a fused positioning result of the target.

[0059] Specifically, the difference between each positioning result and the displacement of the motion actuator corresponding to the positioning result relative to the initial positioning result is determined, and the fused positioning result of the target is obtained based on the difference and the confidence weight corresponding to the positioning result.

[0060] For example, the n positioning results of the target are fused according to the confidence weight to obtain the fused positioning result of the target It can be expressed as:

[0061] in, Indicates the displacement of the motion actuator corresponding to the kth positioning result relative to the initial positioning result, represents the kth positioning result.

[0062] Furthermore, the uncertainty of the target's fusion positioning result It can be expressed as:

[0063] Step S140 - 2 : When the difference between the fused positioning result of the target and the initial positioning result of the target is less than a position threshold, the fused positioning result of the target is used as the final positioning result of the target.

[0064] In this embodiment, after obtaining the fused positioning result, it is determined whether the difference between the fused positioning result and the initial positioning result of the target is less than a position threshold. The setting of the position threshold primarily considers whether there is a valid measurement result modulated at the optimal pixel phase for the target. If so, step S140-2 is executed, and the fused positioning result of the target is used as the final positioning result of the target.

[0065] The position threshold is set based on whether a valid measurement result exists for the target modulated at the optimal pixel phase. In other words, if the difference between the target's fused positioning result and its initial positioning result is less than the position threshold, it indicates that the fused positioning result contains a valid measurement result modulated at the optimal pixel phase. Therefore, this fused positioning result can be used as the final positioning result for the target, ensuring that each target obtains an optimized positioning result.

[0066] Furthermore, the method further includes step S140-3: Step S140-3: When the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fused positioning result of the target is used as the initial positioning result of the target, and the process returns to execute step S130: modulate the optical imaging of the multiple targets according to the initial positioning result, and shift the optical imaging of each target to the optimal pixel phase in turn for positioning, so as to obtain multiple positioning results of each target at multiple pixel phases.

[0067] In an embodiment of the present application, if the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, it means that the fused positioning result of the target at this time may not have a valid measurement result modulated at the optimal pixel phase, then return to execute step S130 to continue motion modulation positioning of multiple targets until the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, end the iteration, and obtain a high-precision positioning result for multiple targets.

[0068] Through the above implementation process, under the constraints of the relative position relationship of multiple targets, through optical imaging time-sharing modulation, the positioning accuracy that exceeds that of fixed optical measurement technology is achieved, and the application scope of motion-modulated optical measurement technology is expanded.

[0069] The following describes the time-sharing modulation positioning method for multi-target optical imaging in this application in conjunction with a specific embodiment. Figure 4 As shown, Figure 4 This is a flowchart of another method for time-sharing modulation positioning of multi-target optical imaging provided by an embodiment of the present application. Specifically, the method for time-sharing modulation positioning of multi-target optical imaging includes the following steps S410 to S490: Step S410: constructing an effective PSF model for target imaging of the optical system, wherein the effective PSF model represents the precise correspondence between the target pixel phase and the pixel response, wherein the pixel response is generated after the optical PSF is sampled by the image detector pixels of the optical system; Step S420: Based on the Cramer-Rao lower bound theory, according to the target pixel phase, the target imaging area and the effective PSF model, the positioning accuracy limit of the target at each pixel phase is determined.

[0070] Step S430: Acquire optical images of multiple targets, and perform coarse positioning based on the optical images of the multiple targets to obtain initial positioning results for each target.

[0071] Step S440: Determine a control quantity sequence for optical imaging motion modulation based on the optimal pixel phase and the initial positioning result, wherein the control quantity in the control quantity sequence represents the displacement required to move the target optical imaging from the position of the initial positioning result to the optimal pixel phase, and the optimal pixel phase is the pixel phase corresponding to the minimum value satisfied by the positioning accuracy limit.

[0072] Step S450: according to the control amount sequence, the optical imaging of each target is sequentially shifted to the optimal pixel phase for imaging with the same exposure time, thereby obtaining multiple optical imaging of multiple targets, where the number of the multiple optical imaging is equal to the number of the multiple targets.

[0073] Step S460: locating the multiple optical imagings using the maximum likelihood method to obtain multiple positioning results of each target at multiple pixel phases, where each optical imaging positioning result corresponds to a positioning result of a pixel phase.

[0074] Step S470: fusing the multiple positioning results according to the confidence weight to obtain a fused positioning result of the target.

[0075] Step S480: When the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fused positioning result of the target is used as the initial positioning result of the target, and the process returns to step S440.

[0076] Step S490: When the difference between the fused positioning result of the target and the initial positioning result of the target is less than a position threshold, the fused positioning result of the target is used as the final positioning result of the target.

[0077] In the embodiments of this application, the characteristic of a small-scale PSF that produces optimal positioning accuracy at a specific pixel phase is utilized to time-share modulate the imaging of multiple targets to the optimal pixel phase for positioning. This results in multiple positioning results for each target at multiple pixel phases. These multiple positioning results are then fused according to the confidence weights determined by the positioning accuracy limit, thereby optimizing the positioning result for each target. Thus, a method for optimizing the global accuracy of multiple targets is achieved. Under the constraints of the relative positional relationships of multiple targets, the time-share modulation of optical imaging by this method surpasses the positioning accuracy of fixed optical measurement technology and expands the application scenarios of motion-modulated optical measurement technology.

[0078] The present application also provides a time-sharing modulation positioning device for multi-target optical imaging, referring to Figure 5 As shown, Figure 5 : This is a schematic structural diagram of a time-sharing modulation positioning device for multi-target optical imaging provided by an embodiment of the present application, the device comprising: A first evaluation module 510 is configured to evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase, wherein the positioning accuracy limit represents the theoretical uncertainty of positioning; A first positioning module 520 is configured to obtain optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain an initial positioning result for each target; A second positioning module 530 is configured to modulate the optical imaging of the multiple targets according to the initial positioning results, sequentially shift the optical imaging of each target to an optimal pixel phase for positioning, and obtain multiple positioning results of each target at multiple pixel phases, where the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit; The first fusion module 540 is configured to determine a confidence weight of each positioning result according to the positioning accuracy limit, and fuse the multiple positioning results according to the confidence weight to obtain a final positioning result of each target.

[0079] It can be understood that the time-sharing modulation positioning device for multi-target optical imaging in the embodiment of the present application can realize the time-sharing modulation positioning method for multi-target optical imaging in the above-mentioned embodiment. The time-sharing modulation positioning device for multi-target optical imaging and the above-mentioned time-sharing modulation positioning method for multi-target optical imaging have the same advantages over the prior art, which will not be repeated here.

[0080] The present application also provides an electronic device, Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6As shown, the electronic device 600 includes: a memory 610 and a processor 620. The memory 610 and the processor 620 are connected via a bus communication. A computer program is stored in the memory 610. The computer program can be run on the processor 620 to implement the steps of the time-sharing modulation positioning method for multi-target optical imaging described in the embodiment of the present application.

[0081] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the time-sharing modulation positioning method for multi-target optical imaging described in the embodiment of the present application are implemented.

[0082] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the time-sharing modulation positioning method for multi-target optical imaging described in the embodiment of the present application.

[0083] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0084] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0087] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0088] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0089] The above is a detailed introduction to the time-sharing modulation positioning method, device and electronic equipment for multi-target optical imaging provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A time-sharing modulation positioning method for multi-target optical imaging, characterized in that: The method comprises: Evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase, wherein the positioning accuracy limit represents the theoretical uncertainty of positioning; Acquire optical images of multiple targets, and perform coarse positioning based on the optical images of the multiple targets to obtain initial positioning results for each target; Modulating the optical imaging of the multiple targets according to the initial positioning results, sequentially shifting the optical imaging of each target to an optimal pixel phase for positioning, and obtaining multiple positioning results of each target at multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit; The confidence weight of each positioning result is determined according to the positioning accuracy limit, and the multiple positioning results are fused according to the confidence weight to obtain the final positioning result of each target.

2. The time-sharing modulation positioning method for multi-target optical imaging according to claim 1, characterized in that: The multiple positioning results are fused according to the confidence weights to obtain a final positioning result for each target, including: fusing the multiple positioning results according to the confidence weights to obtain a fused positioning result of the target; When the difference between the fused positioning result of the target and the initial positioning result of the target is less than the position threshold, the fused positioning result of the target is used as the final positioning result of the target.

3. The time-sharing modulation positioning method for multi-target optical imaging according to claim 2, characterized in that: The method further comprises: When the difference between the fused positioning result of the target and the initial positioning result of the target is not less than the position threshold, the fused positioning result of the target is used as the initial positioning result of the target, and the execution step is returned to: modulating the optical imaging of the multiple targets according to the initial positioning result, shifting the optical imaging of each target to the optimal pixel phase in turn for positioning, and obtaining multiple positioning results of each target at multiple pixel phases.

4. The time-sharing modulation positioning method for multi-target optical imaging according to claim 1, characterized in that: Evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase, including: Constructing an effective PSF model for target imaging of the optical system, wherein the effective PSF model represents a precise correspondence between a target pixel phase and a pixel response, wherein the pixel response is generated by sampling the optical PSF by pixels of an image detector of the optical system; Based on the Cramer-Rao lower bound theory, the positioning accuracy limit of the target at each pixel phase is determined according to the target pixel phase, the target imaging area and the effective PSF model.

5. The time-sharing modulation positioning method for multi-target optical imaging according to any one of claims 1 to 4, characterized in that: Modulating the optical imaging of the multiple targets according to the initial positioning results, sequentially shifting the optical imaging of each target to an optimal pixel phase for positioning, and obtaining multiple positioning results in which each target is located at multiple pixel phases, including: Determining a control amount sequence for optical imaging motion modulation based on the optimal pixel phase and the initial positioning result, wherein the control amount in the control amount sequence represents a displacement required to move the target optical imaging from the position of the initial positioning result to the optimal pixel phase; According to the control amount sequence, the optical imaging of each target is sequentially shifted to the optimal pixel phase and imaged with the same exposure time to obtain multiple optical imaging of the multiple targets, where the number of the multiple optical imaging is equal to the number of the multiple targets; The multiple optical imagings are positioned using a maximum likelihood method to obtain multiple positioning results of each target at multiple pixel phases, where each optical imaging positioning result corresponds to a positioning result of one pixel phase.

6. The time-sharing modulation positioning method for multi-target optical imaging according to claim 5, characterized in that: According to the control amount sequence, each target optical imaging position is sequentially shifted to the optimal pixel phase for imaging with the same exposure time, including: A motion actuator is used to fix the lens or image detector of the optical system, and drives the optical system to produce sub-pixel displacement according to the control quantity sequence, and sequentially shifts each target optical imaging to the optimal pixel phase for imaging with the same exposure time.

7. The time-sharing modulation positioning method for multi-target optical imaging according to claim 1, characterized in that: Determining the confidence weight of each positioning result according to the positioning accuracy limit includes: For the positioning result of the target at a pixel phase, the confidence weight of the target positioning result is obtained according to the square result of the positioning accuracy limit of the target at the pixel phase and the sum of the reciprocals of the square results of the positioning accuracy limit of the target at each pixel phase; wherein the sum of the confidence weights of multiple positioning results of the target is equal to 1.

8. The time-sharing modulation positioning method for multi-target optical imaging according to claim 2, characterized in that: The plurality of positioning results are fused according to the confidence weights to obtain a fused positioning result of the target, including: The difference between each positioning result and the displacement of the motion actuator corresponding to the positioning result relative to the initial positioning result is determined, and a fused positioning result of the target is obtained based on the difference and the confidence weight corresponding to the positioning result.

9. A time-sharing modulation positioning device for multi-target optical imaging, characterized in that: The device comprises: A first evaluation module is used to evaluate the positioning accuracy limit of the optical system to obtain the positioning accuracy limit of the target at each pixel phase, wherein the positioning accuracy limit represents the theoretical uncertainty of positioning; A first positioning module is used to obtain optical images of multiple targets and perform coarse positioning based on the optical images of the multiple targets to obtain initial positioning results for each target; a second positioning module, configured to modulate the optical imaging of the multiple targets according to the initial positioning results, sequentially shift the optical imaging of each target to an optimal pixel phase for positioning, and obtain multiple positioning results in which each target is located at multiple pixel phases, wherein the optimal pixel phase is the pixel phase corresponding to the minimum value of the positioning accuracy limit; The first fusion module is used to determine the confidence weight of each positioning result according to the positioning accuracy limit, and fuse the multiple positioning results according to the confidence weight to obtain the final positioning result of each target.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the time-sharing modulation positioning method for multi-target optical imaging according to any one of claims 1 to 8 are implemented.

Images