Deep learning automatic data set acquisition method and device based on super-resolution model, and storage medium

Abstract

The invention relates to the technical field of image acquisition and processing, and particularly discloses a deep learning automatic data set acquisition method and device based on a super-resolution model, and a storage medium, and the method comprises the steps: employing a deep learning technology, building a super-resolution model, and employing a convolutional neural network as a basic network of the super-resolution model; automatic photographing is carried out by using the illuminometer according to a numerical value returned by the illuminometer; through a deep learning technology, the collected data set is utilized to help the EBAPS image to carry out noise reduction and image quality optimization; carrying out image processing by adopting a deep learning automatic data set acquisition method of the super-resolution model; and constructing a CNN classification model, evaluating the image focusing quality, and driving a motor to adjust the focal length in combination with a control instruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image acquisition and processing, in particular to a deep learning automatic data set acquisition method and device based on a super-resolution model and a storage medium. BACKGROUND

[0002] 1) Background introduction of related technology: In the Electron Bombardment Active Pixel Sensor (EBAPS) system, deep learning technology is gradually becoming a core tool to improve image quality. EBAPS amplifies image signals through electron bombardment, but in extremely low light environments, it still faces problems such as noise and detail loss. Deep learning algorithms, especially Convolutional Neural Networks (CNN), can significantly enhance image contrast, reduce noise, and restore details through efficient feature extraction and adaptive capabilities, thereby improving the image performance of EBAPS systems in various complex environments. However, the performance of deep learning algorithms is highly dependent on the quality and diversity of training data sets. Data sets need to cover various lighting conditions, target objects, and noise patterns to ensure that algorithms can generalize to different use scenarios. Therefore, the extensive collection and accurate labeling of data sets have become the foundation for the successful application of deep learning technology in EBAPS. A powerful data set not only improves the robustness of the model, but also optimizes its performance in real-time image processing.2) Solutions of existing technology: Existing image acquisition and processing technologies usually rely on manual operations, such as manual focusing and image acquisition. This method is not only time-consuming and labor-intensive, but also susceptible to human factors such as fatigue, negligence, etc., which can affect the quality and accuracy of the image. In addition, there are some automatic focusing and image acquisition technologies, such as the automatic focusing function of the camera, but these technologies often rely on the hardware performance of the camera and image processing algorithms, and cannot meet the demand for high precision and high efficiency.3) Problems of existing technology: Existing image acquisition and processing technologies have many problems in actual application. First, the manual focusing and image acquisition method is time-consuming and labor-intensive, and cannot meet the demand for large-scale image acquisition. Second, existing automatic focusing and image acquisition technologies rely on hardware and algorithms, and cannot adapt to complex application environments such as low light and moving objects. SUMMARY

[0003] The purpose of the present application is to provide a deep learning automatic data set acquisition method and device based on a super-resolution model and a storage medium to solve the problems in the prior art.

[0004] To achieve the above purpose, the embodiment of the present application provides a deep learning automatic data set acquisition method based on a super-resolution model, comprising: Using deep learning technology, a super-resolution model is established, and a convolutional neural network is used as the base network of the super-resolution model; An illuminometer is used to automatically take a picture according to the value returned by the illuminometer; Through deep learning technology, the collected data set helps to reduce noise and optimize picture quality for EBAPS images; The deep learning automatic data set collection method using the super-resolution model is used for image processing; A CNN classification model is constructed to evaluate the focusing quality of the image and adjust the focal length by combining control instructions to drive the motor.

[0005] Optionally, the deep learning technology is used to establish a super-resolution model, which specifically includes: The architecture of the super-resolution model refers to the design of the Fourier domain attention convolutional neural network, which combines the frequency domain attention mechanism, contains multiple residual blocks and up-sampling layers, and uses the generative adversarial network framework to optimize the training process; The training data set of the super-resolution model is composed of low-resolution images with a resolution lower than 300x300, the training time is 200 epochs, the learning rate is 0.001, the Adam optimizer is used, and the loss function combines mean square error and perceptual loss.

[0006] Optionally, the illuminometer is used to automatically take a picture according to the value returned by the illuminometer, which specifically includes: When the value returned by the illuminometer is less than 10 lux, the automatic photographing function is started to collect real low-light images, and when the value returned by the illuminometer is less than 10 -5 lux, the automatic photographing function is stopped.

[0007] Optionally, the deep learning technology is used to collect data sets to help EBAPS images to reduce noise and optimize picture quality, which specifically includes: A hybrid algorithm of non-local mean denoising and super-resolution reconstruction is used for EBAPS image enhancement; The hybrid algorithm includes: A denoising module, which uses a fast non-local mean algorithm to remove noise by calculating the similarity between image blocks and weighted average, while preserving edge information, A quality optimization module: the quality optimization module is based on a deep residual convolutional network, which performs super-resolution reconstruction on the denoised image, and reconstructs high-frequency details through residual learning.

[0008] Optionally, the model construction process of the quality optimization module includes: the deep residual convolutional network contains multiple residual blocks, each residual block is composed of a convolutional layer, a batch normalization layer and a ReLU activation function; the super-resolution model is used as the basis and fine-tuned by the collected real low-light image data set to adapt to the noise characteristics of the EBAPS sensor; The model training and execution steps of the quality optimization module include: (1) Dataset preparation: Combine the training dataset consisting of low-resolution images and the collected real low-light images to construct a hybrid dataset; (2) Two-stage training: Phase 1: Pre-train the denoising-super-resolution joint model using the aforementioned training dataset. The second stage involves optimizing the model using real low-light images through a self-supervised strategy to avoid dependence on a large amount of labeled data. (3) Inference stage: Input a low-quality EBAPS image, first perform non-local mean denoising, and then output a high-quality image through a super-resolution model.

[0009] Optionally, the deep learning-based automatic dataset acquisition method using the super-resolution model for image processing specifically includes: We adopt a cascaded U-Net structure, reduce redundant parameters by channel pruning and knowledge distillation to compress the model, and introduce separable convolution to replace standard convolution to reduce computational complexity.

[0010] Optionally, the CNN classification model construction and steps include: (1) Focus quality assessment model: Network structure: shallow CNN is adopted, the input is image patch, the output is focus score, training data: the real low light images collected are manually labeled as clear / blurry; (2) Control closed loop: Real-time acquisition of images and input into the focus quality evaluation model. If the score is lower than the threshold, the motor is triggered to fine-tune the focus, and the process is iterated until the score meets the standard.

[0011] Optionally, it also includes: Improve the model's robustness in complex environments through dynamic updates and adversarial training; specifically including: (1) Online learning: After deployment, the model is continuously fine-tuned using real low-light images collected by the automatic photo-taking function, and a small learning rate is used to avoid catastrophic forgetting; (2) Multimodal adaptation: Introducing domain adaptation technology to enhance the model's generalization ability; (3) Periodically test robustness on the validation set and iteratively update network parameters.

[0012] To achieve the above objectives, this application also provides a deep learning-based automatic dataset acquisition device based on a super-resolution model, comprising: a memory; and A processor connected to the memory, the processor being configured to perform the steps of the method described above.

[0013] To achieve the above object, the application further provides a computer storage medium, which stores a computer program, wherein the computer program is executed by a machine to implement the steps of the method.

[0014] The embodiments of the application have the following advantages: 1. High efficiency and automation: the deep learning automatic data set acquisition method based on the super-resolution model of the application realizes full-automatic focusing and data acquisition, greatly improves the efficiency of image acquisition, avoids the tediousness and inefficiency of manual operation, and is suitable for large-scale image acquisition requirements. 2. Strong adaptability: the technical solution of the application can set the illumination of the image to be collected, and realize automatic shooting according to the value returned by the illuminometer, which is suitable for various complex application environments such as low light and moving objects, and improves the accuracy and reliability of image acquisition. 3. Image quality improvement: the technical solution of the application can generate clearer images from low-resolution images, assist in focus judgment, and also help EBAPS image noise reduction and picture quality optimization algorithm development, greatly assisting in improving the quality and clarity of images. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and those skilled in the art can also obtain other drawings according to the provided drawings without creative labor.

[0016] Figure 1 A flow chart of a deep learning automatic data set acquisition method based on a super-resolution model provided by at least one embodiment of the application; Figure 2 A module block diagram of a deep learning automatic data set acquisition device based on a super-resolution model provided by at least one embodiment of the application. DETAILED DESCRIPTION

[0017] The embodiments of the application will be described below by specific specific embodiments, and those skilled in the art can easily understand other advantages and effects of the application from the content disclosed in the specification. Obviously, the described embodiments are part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0018] It should be noted that the steps in the claims and the description of the application can be executed in parallel or in reverse order as appropriate, depending on the functions involved.

[0019] It should also be noted that the terms "step one", "step two", "step three" and the like in the claims and the description of the present application are used to distinguish different steps, and are not intended to describe a specific order or sequence, and it should be understood that these steps can be substantially parallel or in reverse order, depending on the functions involved.

[0020] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0021] An embodiment of the present application provides a deep learning automatic data set acquisition method based on a super-resolution model, which refers to Figure 1 , Figure 1 A flowchart of a deep learning automatic data set acquisition method based on a super-resolution model provided in at least one embodiment of the present application is shown. It should be understood that the method can also include additional frames not shown and / or can omit the frames shown, and the scope of the present application is not limited in this respect. The specific steps are as follows: Step one: using deep learning technology, a super-resolution model is established.

[0022] In some embodiments, a convolutional neural network (CNN) is used as the basic network of the super-resolution model, which realizes the reconstruction of image details and the improvement of clarity by learning end-to-end on a large number of low-resolution images.

[0023] In some embodiments, the architecture of the super-resolution model refers to the design of the Fourier domain attention convolutional neural network (DFCAN), which combines the frequency domain attention mechanism to enhance the recovery ability of high-frequency information. Specifically, the network contains multiple residual blocks and up-sampling layers, and uses the generative adversarial network (GAN) framework to optimize the training process.

[0024] Specifically, the training data set of the super-resolution model is composed of low-resolution images with a resolution lower than 300x300 (such as the public data set DIV2K or the self-built low-light image set), the training time is 200 epochs, the learning rate is 0.001, and the Adam optimizer is used. The loss function combines mean square error (MSE) and perceptual loss to improve visual quality.

[0025] Step two: using a light meter, according to the value returned by the light meter, automatic shooting is performed.

[0026] In some embodiments, when the value returned by the light meter is less than 10 lux, the automatic shooting function is started to collect real low-light images, and when the value returned by the light meter is less than 10 -5Stop auto shooting function at 1 lux, ensure data in valid low light range.

[0027] Specifically, the collected real low light images are automatically stored in RAW format, preserving the original details for subsequent super-resolution and noise reduction processing.

[0028] Step three: Through deep learning technology, use the collected data set to help EBAPS image noise reduction and picture quality optimization.

[0029] In some embodiments, a hybrid algorithm of non-local mean denoising and super-resolution reconstruction is used for the enhancement of EBAPS (electron multiplying back-illuminated CMOS) images.

[0030] Specifically, the hybrid algorithm includes: A denoising module that uses a Fast Non-Local Means algorithm to remove noise by calculating the similarity between image blocks and weighted average, while preserving edge information; A quality optimization module: The quality optimization module is based on a deep residual convolutional network (such as ResNet variants) to perform super-resolution reconstruction on the denoised image, and reconstruct high-frequency details through residual learning.

[0031] In some embodiments, the model construction process of the quality optimization module includes: the deep residual convolutional network contains multiple residual blocks, each residual block is composed of a convolution layer, a batch normalization layer and a ReLU activation function; use the super-resolution model as the basis, and fine-tune (Fine-tuning) through the collected real low light image data set to adapt to the noise characteristics of EBAPS sensor.

[0032] In some embodiments, the model training and execution steps of the quality optimization module include: (1) Data set preparation: Combine the training data set (simulation data) composed of low-resolution images and the collected real low light images (actual data) to construct a hybrid data set.

[0033] (2) Two-stage training: First stage: Pre-train the denoising-super-resolution joint model using the training data set.

[0034] Second stage: Use the real low light images to optimize the model through a self-supervised strategy (such as spatio-temporal interleaving sampling), avoiding dependence on a large amount of labeled data.

[0035] (3) Inference stage: Input low-quality EBAPS images, first denoise through non-local mean, then output high-quality images through the super-resolution model, with a measured clarity improvement of 30%.

[0036] Step 4: Employ the deep learning-based automatic dataset acquisition method of the super-resolution model to perform efficient image processing and reduce the demand for computing resources and storage space.

[0037] Specifically, a cascaded U-Net structure (similar to Real-CUGAN) is adopted, and redundant parameters are reduced through channel pruning and knowledge distillation to compress the model; separable convolution is introduced to replace standard convolution, reducing computational complexity. Step four, image processing, processes the image from step three.

[0038] In some embodiments, the training and execution steps include: (1) Training configuration: End-to-end training is implemented using PaddlePaddle high-level API, with Adam as the optimizer, a learning rate of 0.001, and a batch size of 64.

[0039] (2) Image processing flow: The input low-resolution image is first upsampled to the target size by bicubic interpolation, and the residual details are predicted by lightweight CNN. After superposition, a high-resolution image is output.

[0040] (3) Resource optimization: TensorRT is used to accelerate model deployment, and the actual measured computational resource requirements are reduced by 50% and storage space is reduced by 50%.

[0041] Specifically, the super-resolution model has approximately 1 million parameters, which reduces the computational resource and storage space requirements by 50% compared to traditional image processing algorithms.

[0042] Step 5: Utilize deep learning technology to achieve fully automated focusing and data acquisition, improving work efficiency and reducing labor costs. Specifically, deep learning technology enables automated focus judgment and image acquisition, eliminating the need for manual intervention and increasing work efficiency by 50%.

[0043] Specifically, a CNN classification model is constructed to evaluate the image focus quality (e.g., based on gradient features or frequency domain energy) and combined with control commands to drive the motor to adjust the focal length.

[0044] In some embodiments, the CNN classification model construction and steps include: (1) Focus quality assessment model: Network structure: shallow CNN (e.g., 4 convolutional layers + fully connected layers), input is image patch, output is focus score (0-1 interval), training data: the collected real low light images are manually labeled as clear / blurry.

[0045] (2) Control loop: Real-time acquisition of images and input into the focus quality evaluation model. If the score is lower than the threshold (e.g., 0.8), the motor is triggered to fine-tune the focus, and the process is iterated until the score reaches the target.

[0046] Specifically, the images in the steps of the above embodiments need to be understood as a whole, and the whole is a closed-loop control system. The captured images are processed through a series of models and the like to guide the control of image acquisition, and the role of each step needs to be understood. The "image" in the closed-loop control refers to the image obtained in the previous step.

[0047] In some embodiments, in the selection of the deep learning network model for autofocus, several other network models can also be selected, and each network has its own advantages and disadvantages: 1. Convolutional Neural Network (CNN) Application: CNN is the most commonly used network model in autofocus algorithms. It uses convolutional layers to extract spatial features of images and outputs focus information through fully connected layers.

[0048] Example: A typical CNN structure is used to estimate image sharpness or predict focus offset values and guide lens adjustment.

[0049] Advantages: It is very effective for processing low-level and high-level features of images, and can well capture changes in sharpness and focus.

[0050] 2. Recurrent Neural Network (RNN) and its variants LSTM / GRU Application: RNN or its improved versions such as LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) are suitable for processing sequential data, especially continuous frame autofocus problems.

[0051] Example: In video autofocus, LSTM can learn the temporal relationship between different frames to predict the best focus point of the next frame.

[0052] Advantages: Good at processing time series information, can combine historical information of multiple frames to optimize focus decision.

[0053] 3. Generative Adversarial Network (GAN) Application: The generator of GAN can be used to generate high-quality, sharp images, and the discriminator can be used to distinguish between sharp and blurred images, thereby optimizing autofocus.

[0054] Example: GAN model can generate clear images to guide autofocus operation and also perform post-image sharpness enhancement.

[0055] Advantages: Good at generating clear images from blurred images, improving the effect of autofocus.

[0056] 4. Attention Mechanism Networks Application: Attention mechanisms can focus on key areas of an image, identifying those areas that most require precise focus.

[0057] Example: Attention-based networks can locate high-contrast or target objects in an image, thus better assisting autofocus.

[0058] Advantages: It can effectively focus on important detail areas in an image, improving the accuracy of autofocus.

[0059] Step 6: Improve the model's robustness in complex environments through dynamic updates and adversarial training.

[0060] Specifically, the optimization steps include: (1) Online learning: After deployment, the model is continuously fine-tuned using real low-light images collected by the automatic photo-taking function, and a small learning rate (such as 0.0001) is used to avoid catastrophic forgetting.

[0061] (2) Multimodal adaptation: Introduce domain adaptation technology, such as generating synthetic data under different lighting conditions through GAN to enhance the generalization ability of the model.

[0062] (3) Regularly test robustness (such as PSNR under noise perturbation) on the validation set, iteratively update network parameters, and the measured generalization ability is improved by 30%.

[0063] The above steps were conducted in a laboratory environment. The experimental equipment included a computer equipped with an NVIDIA Tesla V100 graphics card, a lux meter, and corresponding image acquisition equipment. Experimental results show that the deep learning-based automatic dataset acquisition method based on a super-resolution model provided in this embodiment can effectively improve the efficiency and quality of image acquisition and processing, and has good practical value.

[0064] Figure 2 A block diagram of a deep learning-based automatic dataset acquisition device based on a super-resolution model, provided for at least one embodiment of this application. The device includes: Memory 101; and processor 102 connected to the memory 101, the processor 102 being configured to: employ deep learning technology to build a super-resolution model, and use a convolutional neural network as the base network of the super-resolution model; Using a lux meter, take photos automatically based on the values ​​returned by the lux meter; The collected data set is used to help the EBAPS image to reduce noise and optimize picture quality through deep learning technology. The deep learning automatic data set collection method using the super-resolution model is used for image processing. A CNN classification model is constructed to evaluate the image focusing quality and combine control instructions to drive the motor to adjust the focal length.

[0065] In some embodiments, the processor 102 is also configured to use deep learning technology to establish a super-resolution model, specifically including: The architecture of the super-resolution model refers to the design of the Fourier domain attention convolutional neural network, which combines the frequency domain attention mechanism, contains multiple residual blocks and up-sampling layers, and uses the generative adversarial network framework to optimize the training process. The training data set of the super-resolution model is composed of low-resolution images with a resolution lower than 300x300, the training time is 200 epochs, the learning rate is 0.001, the Adam optimizer is used, and the loss function combines mean square error and perceptual loss.

[0066] In some embodiments, the processor 102 is also configured to use an illuminometer to automatically take pictures according to the values returned by the illuminometer, specifically including: When the value returned by the illuminometer is less than 10 lux, start the automatic photographing function to collect real low-light images, and when the value returned by the illuminometer is less than 10 -5 lux, stop the automatic photographing function.

[0067] In some embodiments, the processor 102 is also configured to use deep learning technology to help the EBAPS image to reduce noise and optimize picture quality using the collected data set, specifically including: A hybrid algorithm of non-local mean denoising and super-resolution reconstruction is used for EBAPS image enhancement. The hybrid algorithm includes: A denoising module, which uses a fast non-local mean algorithm to remove noise by calculating the similarity weighted average between image blocks while preserving edge information, A quality optimization module: The quality optimization module uses a deep residual convolutional network to perform super-resolution reconstruction on the denoised image, and reconstructs high-frequency details through residual learning.

[0068] In some embodiments, the processor 102 is further configured to: the model construction process of the quality optimization module includes: the deep residual convolutional network contains multiple residual blocks, each residual block is composed of a convolutional layer, a batch normalization layer and a ReLU activation function; using the super-resolution model as the basis, and fine-tuning through the collected real low-light image dataset to adapt to the noise characteristics of the EBAPS sensor; The model training and execution steps of the quality optimization module include: (1) Data set preparation: combining the training data set composed of low-resolution images and the collected real low-light images to construct a hybrid data set; (2) Two-stage training: First stage: pre-training the noise reduction-super-resolution joint model using the training data set, Second stage: optimizing the model with the real low-light images through a self-supervised strategy to avoid dependence on a large amount of labeled data; (3) Inference stage: input low-quality EBAPS images, first through non-local mean denoising, and then output high-quality images through the super-resolution model.

[0069] In some embodiments, the processor 102 is further configured to: the deep learning automatic data set collection method using the super-resolution model, which specifically includes: Using a cascaded U-Net structure, compressing the model through channel pruning and knowledge distillation to reduce redundant parameters; introducing separable convolution to replace standard convolution to reduce computational complexity.

[0070] In some embodiments, the processor 102 is further configured to: the CNN classification model construction and steps include: (1) Focus quality evaluation model: network structure: using a shallow CNN, inputting image blocks and outputting focus scores, training data: using the collected real low-light images, manually labeling clear / fuzzy labels; (2) Control loop: real-time acquisition of images and input into the focus quality evaluation model, if the score is lower than the threshold, trigger the motor to fine-tune the focal length, iterate until the score meets the standard.

[0071] In some embodiments, the processor 102 is further configured to: further includes: Improving the robustness of the model in complex environments through dynamic updating and adversarial training; specifically including: (1) Online learning: after deployment, continuously fine-tune the model with the real low-light images collected by the automatic photographing function, using a small learning rate to avoid catastrophic forgetting; (2) Multi-modal adaptation: introducing domain adaptation technology to enhance the generalization ability of the model; (3) periodically test robustness on validation set, iteratively update network parameters.

[0072] The specific implementation method refers to the foregoing method embodiment, which will not be described here.

[0073] The present application can be a method, device, system and / or computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon for performing various aspects of the present application.

[0074] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch cards or punched tape, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0075] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0076] Computer readable program instructions for carrying out operations of the present application can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0077] Various aspects of the present application can be described in the general context of methods and apparatuses (systems) and computer program products, respectively. It should be appreciated that the various aspects of the present application can be implemented as one or more computer programs, each of which runs in conjunction with an operating system and / or application program and / or other computer programs. The computer program can be written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer program can be stored in any

[0078] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include non- transitory computer-readable storage media, that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions for causing an apparatus to implement various aspects of the present application as specified in flowchart and / or block diagram block or blocks can comprise an article of manufacture including a computer readable storage medium. As used herein, non-transitory computer-readable storage media

[0079] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0080] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0081] Note that, unless otherwise explicitly stated, all features disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by alternative features for achieving the same, equivalent, or similar purpose. Therefore, unless explicitly stated otherwise, each disclosed feature is merely one example of a set of equivalent or similar features. Where used, "further," "preferably," "even further," and "more preferably" are simply starting points for describing another embodiment based on the foregoing embodiments, the combination of which with the foregoing embodiments constitutes the complete configuration of another embodiment. Any combination of several "further," "preferably," "even further," or "more preferably" settings following the same embodiment constitutes yet another embodiment.

[0082] Although this application has been described in detail above with general descriptions and specific embodiments, some modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of this application fall within the scope of protection claimed in this application.

Claims

1. A deep learning-based automatic dataset acquisition method based on super-resolution models, characterized in that, include: A super-resolution model is established using deep learning technology, and a convolutional neural network is used as the base network of the super-resolution model. Using a lux meter, take photos automatically based on the values ​​returned by the lux meter; Deep learning technology is used to help reduce noise and optimize image quality in EBAPS images by utilizing the collected dataset; Image processing is performed using the deep learning-based automatic dataset acquisition method of the super-resolution model described above; Build a CNN classification model to evaluate image focus quality and combine it with control commands to drive the motor to adjust the focal length.

2. The deep learning-based automatic dataset acquisition method based on super-resolution models according to claim 1, characterized in that, The use of deep learning technology to establish a super-resolution model specifically includes: The architecture of the super-resolution model is based on the design of a Fourier domain attention convolutional neural network. This network combines a frequency domain attention mechanism, contains multiple residual blocks and upsampling layers, and uses a generative adversarial network framework to optimize the training process. The training dataset for the super-resolution model consists of low-resolution images with a resolution lower than 300×300. The training time is 200 epochs, the learning rate is 0.001, the Adam optimizer is used, and the loss function combines mean squared error and perceptual loss.

3. The deep learning-based automatic dataset acquisition method based on super-resolution models according to claim 2, characterized in that, The method of automatically taking pictures based on the values ​​returned by the illuminance meter specifically includes: When the lux meter returns a value less than 10 lux, the automatic photo-taking function is activated to capture a true low-light image. -5 When the value is set to lux, the automatic photo-taking function will be stopped.

4. The deep learning automatic dataset acquisition method based on super-resolution model according to claim 3, characterized in that, The method utilizes deep learning technology and the collected dataset to help EBAPS images reduce noise and optimize image quality, specifically including: A hybrid algorithm combining nonlocal mean denoising and super-resolution reconstruction is used to enhance EBAPS images; The hybrid algorithm includes: The noise reduction module employs a fast nonlocal mean algorithm, which removes noise by calculating a weighted average of similarity between image patches while preserving edge information. Quality optimization module: The quality optimization module is based on a deep residual convolutional network to perform super-resolution reconstruction on the denoised image and reconstruct high-frequency details through residual learning.

5. The deep learning-based automatic dataset acquisition method based on super-resolution models according to claim 4, characterized in that, The model building process of the quality optimization module includes: the deep residual convolutional network contains multiple residual blocks, each residual block consists of a convolutional layer, a batch normalization layer and a ReLU activation function; the super-resolution model is used as the basis and fine-tuned by the collected real low-light image dataset to adapt to the noise characteristics of the EBAPS sensor. The model training and execution steps of the quality optimization module include: (1) Dataset preparation: Combine the training dataset consisting of low-resolution images and the collected real low-light images to construct a hybrid dataset; (2) Two-stage training: Phase 1: Pre-train the denoising-super-resolution joint model using the aforementioned training dataset. The second stage involves optimizing the model using real low-light images through a self-supervised strategy to avoid dependence on a large amount of labeled data. (3) Inference stage: Input a low-quality EBAPS image, first perform non-local mean denoising, and then output a high-quality image through a super-resolution model.

6. The deep learning-based automatic dataset acquisition method based on super-resolution models according to claim 5, characterized in that, The deep learning-based automatic dataset acquisition method using the super-resolution model for image processing specifically includes: We adopt a cascaded U-Net structure, reduce redundant parameters by channel pruning and knowledge distillation to compress the model, and introduce separable convolution to replace standard convolution to reduce computational complexity.

7. The deep learning-based automatic dataset acquisition method based on super-resolution models according to claim 6, characterized in that, The CNN classification model construction and steps include: (1) Focus quality assessment model: Network structure: shallow CNN is adopted, the input is image patch, the output is focus score, training data: the real low light images collected are manually labeled as clear / blurry; (2) Control closed loop: Real-time acquisition of images and input into the focus quality evaluation model. If the score is lower than the threshold, the motor is triggered to fine-tune the focus, and the process is iterated until the score meets the standard.

8. The deep learning automatic dataset acquisition method based on super-resolution model according to claim 1, characterized in that, Also includes: Improve the model's robustness in complex environments through dynamic updates and adversarial training; specifically including: (1) Online learning: After deployment, the model is continuously fine-tuned using real low-light images collected by the automatic photo-taking function, and a small learning rate is used to avoid catastrophic forgetting; (2) Multimodal adaptation: Introducing domain adaptation technology to enhance the model's generalization ability; (3) Periodically test robustness on the validation set and iteratively update network parameters.

9. A deep learning-based automatic dataset acquisition device based on a super-resolution model, characterized in that, include: Memory; as well as A processor connected to the memory, the processor being configured to perform the steps of the method as claimed in any one of claims 1 to 8.

10. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a machine, it implements the steps of the method as described in any one of claims 1 to 8.

Images