Automatic focusing method for optical fiber identification camera of multi-target optical fiber spectrum astronomical telescope
By quantizing image sharpness using adaptive variance and combining it with a deep learning-based focus search algorithm, the problem of low focusing accuracy and efficiency of LAMOST fiber optic recognition cameras under large temperature differences is solved, achieving fast and accurate autofocus.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-03-24
AI Technical Summary
The LAMOST fiber optic identification camera cannot accurately locate itself in environments with large temperature differences. Existing autofocus methods are computationally complex and inefficient, failing to meet the high-efficiency and precise focusing requirements of multi-object fiber optic spectroscopic telescopes.
An adaptive variance quantization method is used to quantize image sharpness and combine it with a deep learning-based focus search algorithm. By extracting spot feature information through CNN and using RNN for temporal modeling, fast and accurate focusing can be achieved.
While ensuring focusing accuracy, it significantly improves focusing efficiency, achieving a single-shot focusing success rate of 96.67%, thus solving the problems of high iteration count and low efficiency in traditional methods.
Smart Images
Figure CN121721806A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of multi-target fiber-optic spectroscopic telescopes, and particularly relates to an automatic focusing method for a fiber recognition camera of a multi-target fiber-optic spectroscopic telescope, an electronic device, and a storage medium. BACKGROUND
[0002] The Large Sky Area Multi-Object Fiber Spectroscopy Telescope (LAMOST) is a multi-target fiber-optic spectroscopic telescope with a large aperture and a large field of view. The telescope uses optical fibers to receive concentrated starlight, and therefore, fiber positioning is crucial for LAMOST observation. The LAMOST uses a photogrammetry-based fiber closed-loop positioning system. The fiber closed-loop positioning system detects the position of the optical fiber in real time through a fiber recognition camera and feeds back to the fiber unit control system, controls the optical fiber to move multiple rounds of compensation approach to the target position, and thus realizes precise positioning of the optical fiber. In the LAMOST fiber closed-loop positioning system, six fiber recognition cameras are installed 20 m away from the focal plane. In theory, under the condition that all external conditions remain unchanged, the fiber recognition camera only needs to complete the initial focusing to perform the optical fiber position detection work for a long time. However, the geographical location of the LAMOST has a diurnal temperature difference of up to 40℃ during the observation period. Large temperature changes will produce double optical effects: first, the change of the air refractive index causes a nonlinear shift in the propagation path of the light spot; second, the thermal expansion effect causes micro-deformation of the optical elements and mechanical structures inside the camera lens, resulting in a shift in the focal plane position. These combined factors make it impossible for the fiber recognition camera to focus on the fiber end face, and thus affect the fiber position detection. Therefore, the automatic focusing is added to the LAMOST fiber closed-loop positioning system to ensure that thousands of optical fibers are on the focal plane of the fiber recognition camera, which is the basis for the normal operation of the entire closed-loop positioning system.
[0003] Existing autofocus methods can be broadly categorized into two types: active autofocus based on distance feedback and passive autofocus based on image analysis. Active autofocus relies on the coordinated operation of an external ranging device and the camera system. It acquires object distance parameters in real time through laser or infrared ranging devices to drive the focusing mechanism for position compensation, and is generally suitable for single-target scenarios. Passive autofocus evaluates focus quality by analyzing image sharpness and uses an iterative search algorithm to achieve focus. Because the LAMOST fiber optic identification camera needs to clearly image thousands of fibers, it employs passive autofocus. Passive autofocus comprises two core components: an efficient and accurate image sharpness evaluation method and a fast and accurate focus search algorithm. In the image sharpness evaluation method, the raw images output by the LAMOST fiber optic identification camera have unique properties: thousands of discrete light spots are distributed within a high-resolution image of 7920×6004 pixels, with each spot occupying only about 30 pixels. Existing sharpness assessment methods require analyzing the entire image, processing approximately 48 million pixels, resulting in exponentially increasing computational complexity. Furthermore, global analysis fails to identify feature regions, and feature value changes are not readily apparent. While classic hill-climbing algorithms can achieve final focus through gradual approximation, they require dozens of iterations to complete, severely impacting focusing efficiency. Therefore, for the specific scenario of the LAMOST fiber optic closed-loop positioning system, employing targeted image sharpness assessment methods and efficient focus search algorithms can achieve good results in both focusing accuracy and speed. Summary of the Invention
[0004] To address the problems existing in the prior art, the present invention aims to provide an autofocus method, electronic device, and storage medium for a fiber optic identification camera of a multi-object fiber optic spectroscopic telescope. This method quantifies image sharpness through an image sharpness evaluation method and achieves fast and accurate focusing of the fiber optic identification camera based on a deep learning-based focus search algorithm.
[0005] This method includes: acquiring an initial image from the fiber optic recognition camera; preprocessing the initial image to obtain a preprocessed image; inputting the preprocessed image into a trained deep learning model to predict the defocus state of the fiber optic recognition camera and outputting the defocus amount; and controlling the focusing component to complete the focusing of the fiber optic recognition camera based on the defocus amount. The autofocus method proposed in this invention significantly improves focusing efficiency while ensuring the focusing accuracy of the fiber optic recognition camera. This method is not only applicable to the fiber optic recognition camera of multi-object fiber optic spectroscopic telescopes, but can also be widely applied to various passive focusing camera imaging systems, providing a novel approach to autofocus technology for such systems.
[0006] An autofocus method for a fiber optic identification camera in a multi-object fiber optic spectroscopic telescope is disclosed. The following steps are applicable to a single fiber optic identification camera; six fiber optic identification cameras respectively achieve autofocus on the fibers in their respective fields of view through this method, thereby completing autofocus on all fibers on the focal plane. The steps include: (1) Acquiring the initial image of the fiber optic identification camera: For each fiber optic identification camera in the multi-object fiber optic spectroscopic telescope, the original image of the fiber optic spot corresponding to the field of view on the focal plane is captured by the fiber optic identification camera in the current focus ring state. After rotating the focus ring counterclockwise by a single step length T, the original image of the fiber optic spot is captured. Each fiber optic identification camera is positioned around the spherical primary mirror of the multi-object fiber optic spectroscopic telescope. Each fiber optic identification camera is used to photograph a set area of the focal plane, and the set areas photographed by each fiber optic identification camera are combined to cover the focal plane.
[0007] (2) Preprocess the initial image to obtain a preprocessed image: [Image description missing] and The centroid of the fiber optic spot in two images is determined using an optical centroid algorithm, and the images are then compared based on the minimum Euclidean distance. Center of gravity of the light spot and the image The centroid of the light spot is matched, and the successfully matched light spot is extracted from the image. and The surrounding pixels are used as the light spot area. After normalizing them to a gray value of 0-255, they are saved as JPG images, resulting in a pair of JPG images with the same light spot.
[0008] (3) Input the preprocessed image into the trained deep learning model to predict the defocus state of the fiber optic recognition camera and output the defocus amount. (4) Control the focusing component of the fiber optic identification camera to rotate according to the defocus amount, and complete the focusing of the fiber optic identification camera.
[0009] Preferably, in step (3), the deep learning network model, see [link to relevant documentation]. Figure 2 It includes a convolutional neural network (CNN) as a feature extraction module and a recurrent neural network (RNN) as a temporal modeling module; the CNN module adopts the ResNet-18 neural network architecture, and the RNN module adopts the gated recurrent unit (GRU) neural network architecture.
[0010] Preferably, the method for obtaining the trained deep learning model in step (3) includes: (3.1) constructing a training image dataset; (3.2) preprocessing the training image dataset to obtain a preprocessed training image dataset; and (3.3) training the deep learning model using the preprocessed training image dataset to obtain a trained deep learning model. The training image dataset includes images containing different levels of light spot clarity, acquired simultaneously by rotating the camera focus ring with a specific step size T, causing the optical fiber's imaging within the camera to undergo a process from defocusing to focusing and then back to defocusing.
[0011] Preferably, the preprocessed training image dataset in step (3.1) includes: (3.1.1) Quantifying sharpness by calculating the adaptive variance value of each image, accurately locating the sharp focus position as the position of each spot, and recording the deviation value of other positions relative to the sharp focus position, and using it as the true value of the defocus amount; (3.1.2) Classifying the preprocessed spot images according to the defocus state to form a spatial feature dataset, and then dividing the training set and validation set according to a specific ratio; (3.1.3) Reassembling the JPG images of different sharpness spots corresponding to the same optical fiber (one spot corresponding to each optical fiber) according to the defocus sequence to form a time-series variation dataset, while maintaining its consistency with the spatial feature dataset in the sample division ratio.
[0012] Preferably, step (3.1.1) of calculating the adaptive variance value of the image includes: determining the light spot region in the image through the optical centroid algorithm; calculating the variance value for each light spot region; calculating the mean of the variance values of all light spot regions in the image, which is the adaptive variance value of the image, and using it as the image sharpness evaluation value. The larger the adaptive variance value, the clearer the current image.
[0013] Preferably, the training of the deep learning model using the preprocessed training image dataset in step (3.3) includes: adopting a modular, phased training strategy; first, training the CNN network using the cross-entropy loss function on the spatial feature dataset; minimizing the cross-entropy loss function to minimize the inter-class variance of light spots with different defocus degrees, thus causing the CNN network to converge to its optimal state and completing the discriminative learning of spatial features; after the CNN network is trained, freezing all its network parameters; then, training the RNN network using the L1 loss function on the temporal variation dataset; calculating the loss between the predicted value and the true value using the L1 loss function; minimizing the L1 loss function to cause the RNN network to converge to its optimal state and completing the regression learning of temporal variation patterns. This modular, phased training strategy can achieve precise adaptation of each module task, avoiding mutual interference between the CNN classification task and the RNN regression task during training. At the same time, by freezing the parameters of the already trained CNN network, it ensures that the RNN network can perform temporal modeling based on stable and reliable spatial feature inputs, thus ensuring the training effect of a single module and improving the convergence efficiency and prediction accuracy of the overall model.
[0014] Preferably, in step (4), the focusing component includes: (see...) Figure 2 The focusing assembly mainly consists of a flexible support and an arc-shaped guide rail. The flexible support is used to support the lens of the fiber optic recognition camera, and the motor controls the rotation of the camera's focusing ring by controlling the movement of the arc-shaped guide rail.
[0015] An electronic device includes a memory and a processor, wherein the memory stores a computer program, the computer program including instructions for performing the methods described above according to the present invention, and is executed by the processor.
[0016] A storage medium storing a computer program configured to, when executed by a processor of an electronic device, trigger the processor to perform the method described above in this invention.
[0017] The beneficial technical effects of this invention are reflected in the following aspects: 1. This invention proposes an adaptive variance value to quantify the sharpness of fiber optic spot images with a single background and scattered spot distribution.
[0018] 2. This invention addresses the problems of numerous search iterations and low efficiency in traditional hill-climbing algorithms by proposing a deep learning-based autofocus search algorithm: It extracts feature information from the light spot sequence using a CNN and performs temporal modeling of the feature sequence using an RNN, ultimately outputting the defocus amount. This method, while maintaining focusing accuracy, reduces the traditional algorithm's dozens of iterative search processes to a single iteration, significantly improving efficiency. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of the actual LAMOST observation scenario in an embodiment of the present invention.
[0020] Figure 2 This is a schematic diagram of the focusing component structure.
[0021] Figure 3 This is a flowchart of an embodiment of the present invention.
[0022] Figure 4 This is a diagram of the deep learning model architecture of the present invention.
[0023] Figure 5 This is the result of evaluating image sharpness using adaptive variance as the sharpness evaluation function when using an 800mm lens and a camera focus ring movement resolution of 0.125° in the LAMOST field observation environment of this invention embodiment. Detailed Implementation
[0024] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.
[0025] This embodiment is based on the actual application scenario of LAMOST, taking the F-type fiber optic recognition camera in the lower left corner of the focal plane as an example to describe the autofocus of the present invention in detail. Figure 1 This is a schematic diagram of an actual LMAOST observation scenario. The diameter of the LMAOST focal plane is approximately 1.75 meters, with 4000 fiber optic positioning units evenly distributed across it. Six fiber optic identification cameras are mounted off-axis on an MB truss approximately 20 meters from the focal plane. Each camera is responsible for capturing one-sixth of the focal plane area. The fiber optic identification camera lens has a focal length of 800mm, and the focusing assembly is externally located at the lens focusing ring. See [link / details]. Figure 2 The fiber optic identification camera uses an industrial CMOS camera with a CMV50000 chip, a resolution of 7920×6004, and the image is set to 12-bit grayscale mode with a grayscale value range of 0~4095.
[0026] Figure 3 The flowchart of an embodiment of the present invention is shown below, and the specific steps are as follows.
[0027] (1) Acquire the initial image of the fiber optic recognition camera: Control the focusing assembly to rotate the focusing ring of the fiber optic recognition camera away from the optimal focusing state, record the rotation angle as the true value of the defocus amount in the current state. The camera captures the original image of the fiber optic spot in its current state. After rotating the focus ring counterclockwise by 0.125°, capture the original image of the fiber optic spot. By controlling the focusing assembly to rotate the camera's focusing ring to 20 different states at temperatures of 15℃, 20℃, and 25℃, a total of 60 test experiments were conducted.
[0028] (2) The initial image is preprocessed to obtain a preprocessed image.
[0029] (2.1) Original image of fiber optic spot and The centroid of all light spots in the two images was determined using the optical centroid algorithm. The centroid set of the light spot is ( , ),picture The centroid set of the light spot is ( , ).
[0030] (2.2) Based on the minimum Euclidean distance, the image is processed. Center of gravity of the light spot and the image Matching the center of gravity of the light spot: (2.2.1) Initialize the matching list: Create an empty collection Matches to store valid matching pairs, and initialize the image. Spot Index i and Image The correspondence record of spot index j; (2.2.2) Traversal and Image light spot centroid set: For images The calculated set of centroids of the light spot ( , Each spot of light in ) , Perform the following operations: Initialize the minimum matching distance d_min of the current spot to 10000 pixels and the image. Centroid set of light spots ( , The best matching index j_best in the sample is -1, and all the light spots in it are traversed. , The Euclidean distance is calculated using the following formula: if If it is less than d_min, then update d_min = j_best=j; (2.2.3) Filtering valid matches: Set the distance threshold to 50 pixels. If d_min is less than 50 pixels, then (i,j_best) is determined to be a valid match pair and added to the set Matches.
[0031] (2.3) Obtaining image pairs with the same light spot: For valid matching pairs (i,j_best) in Matches, in the image... Extract from ( , The area centered at a size of 16×16 pixels. In the picture Extract from ( , The area centered at a size of 16×16 pixels. ,Will and After being normalized to a grayscale value of 0-255, it is saved as a JPG image pair.
[0032] (3) Input the preprocessed image into the trained deep learning model to predict the defocus state of the fiber optic recognition camera and output the defocus amount.
[0033] (3.1) Deep learning models: See Figure 4 The CNN feature extraction module adopts the ResNet-18 architecture, the RNN time series modeling module adopts the gated recurrent unit (GRU), and the entire deep learning model adopts a cascaded structure.
[0034] (3.2) Constructing a training image dataset: Control the focusing component to rotate the focusing ring of the fiber optic recognition camera in 0.125° steps, with a total of 20 sampling positions, to obtain 20 original images containing fiber optic spots of different clarity.
[0035] (3.3) Preprocessed training image dataset: The sharpness is quantified by calculating the adaptive variance value of each image, and the position of sharp focus is accurately located, with reference to... Figure 5 The adaptive variance value reaches its maximum at position 10, which is 229.559 pixels, thus position 10 is the in-focus position. The deviation values of other out-of-focus positions relative to the in-focus position are recorded and used as the true values of the out-of-focus amount. For each image, the centroid of the light spot is determined using the optical centroid algorithm. A 16×16 light spot region is extracted around the centroid, and the region is normalized to a grayscale value of 0-255 and saved as a JPG image. All light spot images are divided into 20 categories according to different out-of-focus states to form a spatial feature dataset, with the training and validation sets divided in an 8:2 ratio. JPG images of light spots with different resolutions corresponding to the same optical fiber are reassembled according to the out-of-focus sequence to form a time-series variation dataset, while maintaining consistency with the spatial feature dataset in the sample division ratio.
[0036] (3.4) Preprocessing the training image dataset to train the deep learning model: The CNN is trained using the cross-entropy loss function on the spatial feature dataset. The formula for the cross-entropy loss function is shown below: Where C is the number of classification categories (i.e., the number of spot states), and in this case, C=20. For the actual probability distribution of each category, To predict the probability distribution of each category, the GRU regressor is trained using an L1 loss function on a time-varying dataset. The formula for the L1 loss function is shown below: in the formula To predict the amount of defocus, The actual defocus amount is used; a modular, phased training strategy is implemented: first, the CNN module is trained, then the parameters of the CNN module are frozen to train the GRU module, and the model parameters with the best performance during the training process are saved. Specifically, the training process of the CNN module is set with a learning rate of 0.001, a learning rate decay of 0.1 every 200 epochs, and a batch size of 300. The training process of the RNN module is set with a learning rate of 0.001, a learning rate decay of 0.1 every 100 epochs, and a batch size of 2400. When the loss value of the model on the validation set does not improve significantly within several consecutive training epochs, training is stopped, and the model weights corresponding to the minimum loss value on the validation set are saved to complete the model training.
[0037] (3.5) Predict the defocus state of the fiber optic recognition camera: Load the model parameters with the minimum loss value during the model training process in step (3.4); for each valid match pair in step (2), the corresponding spot area and The input to the CNN network is converted into two 512-dimensional feature vectors. and ,Will As the initial hidden state of GRU. As the current input, the input GRU unit performs temporal feature fusion and outputs the defocus amount of the light spot. The prediction results of all matching pairs Store the defocused values in the set ∆X; perform statistical analysis on the defocused values in ∆X, and use a "majority voting" strategy to select the defocused value that appears most frequently as the final defocused value.
[0038] (4) Control the focus assembly to rotate according to the defocus amount predicted in step (3) to complete the focusing of the fiber optic recognition camera.
[0039] The test results are shown in Table 1. 20 out of 20 tests were successful at 15℃, 19 out of 20 tests were successful at 20℃, and 19 out of 20 tests were successful at 25℃. In 60 tests, 58 single-shot focusing attempts achieved optimal accuracy. This embodiment demonstrates a 96.67% single-shot focusing success rate using the above method, fully validating the high efficiency and high precision of the algorithm of this invention.
[0040] Table 1 Test temperature Predicted success rate (100%) 15℃ 100％ 20℃ 95％ 25℃ 95％ 96.67％ Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.
Claims
1. An autofocus method for a fiber optic recognition camera in a multi-object fiber optic spectroscopic telescope, comprising the following steps: 1) For each fiber identification camera in the multi-object fiber spectroscopic telescope, use the fiber identification camera to capture an original image of the fiber spot in a predetermined area on the focal plane of the multi-object fiber spectroscopic telescope. ; After rotating the focus ring of the fiber optic recognition camera counterclockwise by a single step length T, the original image of the fiber optic spot in the designated area on the focal plane is captured again. Each of the fiber optic identification cameras is respectively arranged around the spherical primary mirror of the multi-object fiber optic spectroscopic telescope. Each fiber optic identification camera is used to photograph a set area of the focal plane. The set areas photographed by each fiber optic identification camera are combined to cover the focal plane. 2) Identify the original images of the two fiber optic spots. , The centroid of the fiber optic spot in the image is determined based on the minimum Euclidean distance from the original image of the fiber optic spot. The centroid of the fiber optic spot and the original image of the fiber optic spot. Matching the centroids of the fiber optic spots in the optical fiber yields several matched spots; , Each matching spot and its surrounding pixels are extracted as the spot region. The two spot regions are normalized to a gray value of 0-255 and saved as a JPG image, which serves as a pair of JPG images of the same matching spot. 3) Input the JPG image of the matching light spot into the trained deep learning model to predict the defocus state of the fiber optic recognition camera, and output the defocus amount; 4) Control the focusing component of the fiber optic identification camera to rotate according to the defocus amount to complete the focusing of the fiber optic identification camera.
2. The method according to claim 1, characterized in that, The deep learning network model includes convolutional neural networks and recurrent neural networks; the method for training the deep learning network model is as follows: 31) Constructing a training image dataset: Multiple images with different sharpness levels of light spots are acquired using a fiber optic recognition camera; the adaptive variance value of each image is calculated to quantify the sharpness, and the position of the sharp focus is located as the position of each light spot. At the same time, the deviation values of other positions relative to the sharp focus position are recorded and used as the true values of the defocus amount; each image is classified according to the defocus state to form a spatial feature dataset, which is then divided into a training set and a validation set; JPG images of light spots with different sharpness corresponding to the same fiber are reassembled according to the defocus sequence to form a time-series variation dataset, while maintaining consistency with the spatial feature dataset in the sample division ratio; 32) A modular, phased training strategy is adopted. First, the convolutional neural network is trained based on the spatial feature dataset using the cross-entropy loss function. By minimizing the cross-entropy loss function, the inter-class variance of light spots with different defocus degrees is minimized, causing the convolutional neural network to converge to the optimal state and completing the discriminative learning of spatial features. After the convolutional neural network is trained, all its network parameters are frozen. Based on the temporal variation dataset, the recurrent neural network is trained using the L1 loss function. The loss between the predicted value and the true value is calculated using the L1 loss function. By minimizing the L1 loss function, the RNN network converges to the optimal state and completes the regression learning of temporal variation patterns.
3. The method according to claim 2, characterized in that, The method for calculating the adaptive variance value is as follows: determine the light spot regions in the image using the optical centroid algorithm, calculate the variance value of each light spot region; calculate the mean of the variance values of all light spot regions in the image, which is the adaptive variance value of the image.
4. The method according to claim 1, 2, or 3, characterized in that, The convolutional neural network adopts the ResNet-18 neural network architecture, and the recurrent neural network adopts the gated recurrent unit neural network architecture.
5. The method according to claim 1, 2, or 3, characterized in that, The method to obtain JPG image pairs with the same matching light spot is as follows: 21) Determine the original image of the fiber optic spot using the optical centroid algorithm. , The centroid of all light spots is obtained The centroid set of the light spot is ( , ), The centroid set of the light spot is ( , ); 22) Based on the minimum Euclidean distance, the set of centroids of the light spot ( , ) and the set of the centroids of the light spot ( , ) Matching: 221) Create an empty set Matches to store valid matching pairs, and initialize the spot centroid set ( , The light spot index i and the set of light spot centroids () , ) The correspondence record of spot index j in ); 222) For the set of spot centroids ( , Each light spot in ) , ), calculate the light spot ( , ) and the set of the centroids of the light spot ( , Each spot of light in ) , Euclidean distance ,if If the distance is less than the minimum matching distance d_min, then update d_min = And update the best matching index j_best = j, where j is the current spot ( , ) index; 223) If d_min is less than the set distance threshold, then (i,j_best) is determined to be a valid matching pair and added to the set Matches; 23) For each valid matching pair (i, j_best) in the set Matches, in Extract from ( , The area centered at a size of 16×16 pixels. ,exist Extract from ( , The area centered at a size of 16×16 pixels. ,Will and After being normalized to a grayscale value of 0-255, it is saved as a light spot. , (A pair of JPG images) 6. The method according to claim 1, characterized in that, The focusing assembly includes a flexible support, an arc-shaped guide rail, and a motor; the flexible support is used to support the lens of the fiber optic recognition camera; the motor is used to control the movement of the arc-shaped guide rail, thereby controlling the rotation of the focusing ring.
7. The method according to claim 1, characterized in that, Six fiber optic recognition cameras are evenly distributed around the spherical primary mirror, and each fiber optic recognition camera is used to capture one-sixth of the focal plane area.
8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, the computer program comprising instructions for performing the method of any one of claims 1 to 7, and is executed by the processor.
9. A storage medium storing a computer program configured to, when executed by a processor of an electronic device, trigger the processor to perform the method of any one of claims 1 to 7.