A method and system for target detection of DNA nanostructure under AFM scene

Abstract

The application discloses a target detection method and system for DNA nanostructure under an AFM scene, and belongs to the technical field of image recognition. The target detection method comprises the following steps: acquiring an image dataset of DNA nanostructure under an AFM scene; performing data labeling and enhancement on a target AFM image in the image dataset to form a target AFM image dataset; establishing a fusion target detection unit based on a YOLOv5 model; training the fusion target detection unit through the target AFM image dataset to obtain different target detection models; selecting a corresponding trained target detection model as an optimal model according to a current scene and requirements, inputting an AFM image to be detected into the target detection model for image target recognition, and obtaining a target detection result. Therefore, target objects in the image can be accurately recognized, the target detection time is greatly reduced, the flexibility of system selection is enhanced, and the recognition accuracy is significantly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image recognition, and particularly relates to a target detection method and system for DNA nanostructure under an AFM scene. BACKGROUND

[0002] DNA origami is an effective bottom-up technique for building arbitrary predefined two-dimensional (2D) and three-dimensional (3D) structures. DNA origami involves a long scaffold strand that assembles into a pre-designed shape with the help of hundreds of short strands. A single origami structure can be connected to other structures to form a larger and more complex compound, just like bricks connected together. The size of origami ranges from nanometers (single origami) to microns (origami compounds).

[0003] In 1986, Binning, Gerber and Quate first proposed atomic force microscopy (AFM) based on scanning tunneling microscopy (STM). The morphology of nanostructures can be observed and the mechanical properties of the sample can be detected by this method. Atomic force microscopy uses a sensitive probe to "touch" the surface of the origami to characterize its shape and mechanical properties. With the development of atomic force microscopy and DNA nanotechnology, this method has become a standard for DNA detection. To identify DNA origami, it should be tightly attached to a flat substrate (such as a mica plate) in a suitable buffer solution. Appropriate scanning probes, AFM supports and scanning programs are essential for origami imaging. After nearly 30 years of development of atomic force microscopy, various DNA structures have been observed. There are now a large number of AFM origami images available as training data for deep learning origami detection and characterization.

[0004] In the field of nanotechnology. Although deep learning methods have been used to detect nanostructures, they are still rarely used to detect DNA origami, and in view of this, a target detection method for DNA nanostructure under AFM scene is proposed. SUMMARY

[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide a target detection method and system for DNA nanostructure under AFM scene, which solves the problems in the prior art.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A target detection method for DNA nanostructure under AFM scene, comprising the following steps:

[0008] Obtain the image data set of the DNA nanostructure under the AFM scene;

[0009] Data labeling and enhancement are performed on the target AFM image in the image data set to form a target AFM image data set;

[0010] Based on the YOLOv5 model, a fusion target detection unit is established;

[0011] The target AFM image data set is used to train the fusion target detection unit to obtain different target detection models;

[0012] According to the current scene and demand, a trained target detection model is selected as the best model, and the AFM image to be detected is input into the target detection model for image target recognition to obtain a target detection result.

[0013] Further, the step of obtaining the image data set is:

[0014] S11, by pre-setting the required data samples, determine the AFM images with DNA nanostructure to be collected;

[0015] S12, pre-process the target number of AFM images to obtain an image data set.

[0016] Further, the step of pre-processing the AFM image includes:

[0017] S31, read the image data from the data set;

[0018] S32, according to the input requirements of the model, scale the original image to the specified size;

[0019] S33, the image is subjected to data enhancement operation, including random cropping, random flipping and random rotation;

[0020] S34, the image is subjected to normalization processing.

[0021] Further, the network structure of the YOLOv5 model includes a backbone network, a feature pyramid network and a plurality of prediction heads;

[0022] The backbone network adopts CSPDarknet structure, which contains a plurality of CSP modules, each module is composed of a plurality of convolution layers and residual connection, which can effectively extract low-level and middle-level feature information of the image;

[0023] The feature pyramid network is arranged after the backbone network to fuse feature information of different scales; the FPN performs upsampling and fusion operation on the feature maps of different resolutions output by the backbone network;

[0024] The prediction head is provided with a plurality of prediction heads corresponding to feature maps of different scales, which are used to predict the category probability, position and size information of the bounding box of the target; each prediction head maps the feature map to the category and position space of the target through convolution layer and full connection layer operation, to obtain the final detection result.

[0025] Further, the specific steps of training the fusion target detection unit are:

[0026] S41, dividing the target AFM image data set into a training set, a test set and a validation set;

[0027] S42, configuring the model training parameters in the fusion target detection unit;

[0028] S43, starting the training process, training and iteratively optimizing the target detection model and the complete target detection model according to the configured model training parameters;

[0029] The target detection model realizes the detection and positioning function of the target, and is realized by using the YOLOv5 model; the complete target detection model is a target detection system with complete functions and processes, which not only includes the core fusion target detection unit, but also includes data preprocessing, model evaluation, optimization and combination with actual application scenarios;

[0030] S44, by monitoring the training process, inputting the validation set for verification, measuring the performance of the model on the validation set by calculating the evaluation index, and observing the performance of the target detection model and the complete target detection model on the validation set;

[0031] S45, after the model training and optimization are completed, the final performance of the model is evaluated by using the test set data;

[0032] S46, when the target detection model and the complete target detection model reach the satisfactory performance, the corresponding trained target detection model and complete target detection model are saved.

[0033] Further, the target detection result includes the bounding box, category and confidence of the detected target object.

[0034] A target detection system for DNA nanostructure in AFM scene includes:

[0035] An image acquisition module: acquiring image data set of DNA nanostructure in AFM scene;

[0036] An image labeling module: data labeling and enhancement are performed on the target AFM image in the image data set to form a target AFM image data set;

[0037] The detection unit construction module is based on a YOLOv5 model, and a fusion target detection unit is established;

[0038] The detection unit training module trains the fusion target detection unit through the target AFM image dataset, and obtains different target detection models;

[0039] And the target detection module: according to the current scene and the demand, select a corresponding trained target detection model as the best model, and input the AFM image to be detected into the target detection model for image target recognition, and obtain the target detection result.

[0040] A computer storage medium stores a readable program, when the program runs, can execute the above-mentioned one kind of AFM scene under the target detection method for DNA nanostructure.

[0041] An electronic device comprises a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete the communication between each other through the communication bus;

[0042] The memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the operation corresponding to the above-mentioned one kind of AFM scene under the target detection method for DNA nanostructure.

[0043] A computer program product comprises computer instructions, and the computer instructions instruct a computing device to execute the operation corresponding to the above-mentioned one kind of AFM scene under the target detection method for DNA nanostructure.

[0044] The beneficial effects of the present application are:

[0045] The present application scans a plurality of DNA nanostructure pictures with different site information under the AFM image scene, establishes a high-quality AFM image dataset, successfully constructs a feature extraction and recognition model for DNA nanostructure under the AFM image scene based on YOLOv5, realizes good recognition performance of the model in the recognition task of DNA nanostructure under the AFM image scene, combines deep learning and machine vision technology, and constructs a new target detection method for DNA nanostructure under the AFM image scene. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below, and obviously, other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1is the execution flow chart of the target detection method for DNA nanostructure in the AFM image scene of the application;

[0048] Figure 2 is the AFM image schematic diagram of the DNA nanostructure with different site information of the application;

[0049] Figure 3 is the AFM image schematic diagram in the target detection method of the application;

[0050] Figure 4 is the labeling schematic diagram in the target detection method of the application;

[0051] Figure 5 is the file content schematic diagram of the labeling in the target detection method of the application;

[0052] Figure 6 is the training process curve schematic diagram of the target detection method of the application;

[0053] Figure 7 is the detection result schematic diagram of the target detection method of the application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0055] Embodiment 1

[0056] As shown in the figure, a target detection method for DNA nanostructure in an AFM scene includes the following steps: Figure 1

[0057] S1, obtaining an image data set of DNA nanostructure in an AFM scene;

[0058] The step of obtaining the image data set includes:

[0059] S11, determining a plurality of AFM images with DNA nanostructure to be collected by pre-setting required data samples;

[0060] S12, pre-processing a target number of AFM images to obtain an image data set;

[0061] The process of AFM image preprocessing includes:

[0062] 1) image reading, reading image data from the data set. ​

[0063] 2) Image scaling, according to the input requirements of the model, the original image is scaled to the specified size.

[0064] 3) Data enhancement, in order to increase the diversity of data and improve the generalization ability of the model, a series of data enhancement operations are performed on the image, including random cropping, random flipping, random rotation, etc.

[0065] 4) Normalization processing, in order to make the distribution of AFM image data more consistent with the training requirements of the model and speed up the convergence of the model, the image is normalized.

[0066] S2, data labeling and enhancement of target AFM images in the image data set to form a target AFM image data set;

[0067] S3, based on the YOLOv5 model, a fusion target detection unit is established;

[0068] The fusion target detection unit is established based on the YOLOv5 model; the network structure of the YOLOv5 model includes a backbone network, a feature pyramid network and a plurality of prediction heads;

[0069] The backbone network adopts the CSPDarknet structure, which contains a plurality of cross-stage local modules (CSP), each module is composed of a plurality of convolution layers and residual connections, which can effectively extract low-level and middle-level feature information of the image, and provide rich semantic basis for subsequent target detection;

[0070] The feature pyramid network uses a feature pyramid network (FPN) to fuse feature information of different scales after the backbone network; the FPN performs upsampling and fusion operations on the feature maps of different resolutions output by the backbone network, so that the generated feature pyramid contains rich semantic information and high-resolution detail information, thereby being able to better detect targets of different sizes;

[0071] The prediction head sets a plurality of prediction heads corresponding to feature maps of different scales, which are used to predict information such as class probability, position and size of the bounding box of the target; each prediction head maps the feature map to the class and position space of the target through convolution layer and full connection layer operations, to obtain the final detection result.

[0072] S4, training the fusion target detection unit through the target AFM image data set to obtain different target detection models;

[0073] The specific steps for training the fusion target detection unit are:

[0074] S41, divide the target AFM image data set into a training set, a test set and a validation set according to a set proportion (8:1:1);

[0075] S42, configure the YOLOv5 model training parameters in the fusion target detection unit, including the pre-training weight path, the model configuration file path, the parameter file path corresponding to the data set, the number of training rounds, the input data volume of each batch, etc.

[0076] S43, start the training process, and train and iteratively optimize the target detection model and the complete target detection model according to the configured model training parameters;

[0077] The target detection model realizes the detection and positioning functions of the target, and is implemented using the YOLOv5 model, which can find the region of the target object in the input image and judge which category the target object belongs to. The complete target detection model is a target detection system with complete functions and processes, which not only includes the core fusion target detection unit, but also covers data preprocessing, model evaluation, optimization, and combination with actual application scenarios. The iterative optimization specifically uses data enhancement optimization, increasing the number of training rounds and the amount of data to optimize the model.

[0078] S44, by monitoring the training process, inputting the validation set for verification, and calculating various evaluation indicators (such as accuracy, recall rate, average precision mean, confusion matrix, etc.) to measure the performance of the model on the validation set, and observing the performance of the target detection model and the complete target detection model on the validation set.

[0079] S45, the final performance evaluation is realized by testing the test set. The test set is a data set used to objectively and fairly evaluate the final performance of the model after the model training and optimization are completed. The evaluation indicators of the validation set have some similarities with the test set. By calculating various evaluation indicators (such as accuracy, recall rate, average precision mean, intersection over union, etc.) on the test set, the advantages and disadvantages of the model can be accurately evaluated, providing a basis for the application and promotion of the model.

[0080] S46, when the target detection model and the complete target detection model reach satisfactory performance, save the corresponding trained target detection model and complete target detection model.

[0081] S5, according to the current scene and demand, select a trained target detection model as the best model, and input the AFM image to be detected into the target detection model for image target recognition to obtain a target detection result.

[0082] The target detection result includes the bounding box, category, and confidence of the detected target object.

[0083] Based on the similar inventive concept, the embodiment of the present application also provides a computer storage medium, which stores a readable program, and when the program runs, the method for target detection of DNA nanostructure in AFM scene can be executed.

[0084] Based on the similar inventive concept, the embodiment of the present application provides an electronic device, which comprises a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface complete the communication among each other through the communication bus.

[0085] The memory is used for storing at least one executable instruction, and the executable instruction makes the processor execute the operation corresponding to the method for target detection of DNA nanostructure in AFM scene.

[0086] Based on the similar inventive concept, the embodiment of the present application also provides a computer program product, which comprises computer instructions, and the computer instructions instruct the computing device to execute the operation corresponding to the method for target detection of DNA nanostructure in AFM scene.

[0087] Embodiment 2

[0088] In this embodiment, the target detection method proposed by the present application is illustrated through specific examples, which comprises the following steps:

[0089] I. Establishment of data set: the present application uses AFM to scan a large number of high-quality photos of DNA nanostructure with different site information, and through labelme software, four data sets of DNA nanostructure with different site information are made.

[0090] 1) Image acquisition.

[0091] After screening, the images with defects and unclear images are removed, and finally 123 AFM images are selected, about 2100 DNA nanostructure samples are used to construct the data set containing DNA nanostructure with different site information, as shown in Figure 2 , Figure 3 .

[0092] 2) Data labeling.

[0093] Labelme software is used for data labeling of target categories. When labeling, the smallest circumscribed rectangle frame of DNA nanostructure with different site information is selected as the real frame to reduce the background pixels in the real frame; in order to ensure the consistency of labeling.

[0094] The data set uniformly labels DNA nanostructures with different site information as classes "0", "1", "2", and "3", and names the classes "00", "01", "10", and "11". The labeling diagram and labeling file are as shown in Figure 4 and Figure 5 .

[0095] II. Construction of a DNA nanostructure recognition model based on YOLOv5L in an AFM image scenario.

[0096] A feature extraction and recognition model for DNA nanostructures with different site information is successfully constructed using YOLOv5L. The experimental results show that the optimized model achieves higher precision, recall, and average precision (mAP@0.5) on DNA nanostructures with different site information, significantly improving the recognition performance, as shown in Figure 6 It can be seen that the model performance improvement trajectory during training is shown, and as the training period increases, the training loss gradually decreases, and a similar downward trend is shown on the validation set, indicating that the model is continuously learning and gradually improving its prediction ability, achieving an effective convergent model. By comparing the loss function values on the training set and the validation set, it can be found that the loss curves of the two always maintain similar trends during the training process, and there is no significant separation, which indicates that the simulation has not overfit.

[0097] 1) Model construction.

[0098] YOLO stands for you only look once, which means that the class and location of objects in the image can be recognized by looking at it only once. The YOLO series algorithm adopts an end-to-end processing approach, which combines the prediction of candidate frames and target classification into one step, thereby cleverly converting the target detection problem into a regression problem.

[0099] The present application uses the YOLOv5L model, which has an efficient backbone network: adopts the CSPDarknet structure, effectively reduces the computational load through the design of cross-stage local networks, and enhances the feature extraction capability of the network, which can better capture low-level and intermediate feature information in the image, providing a richer semantic basis for subsequent target detection. And it has better performance in multi-scale feature fusion: its feature pyramid network (FPN) can fuse feature information of different scales to generate a feature pyramid with rich semantic information and different receptive fields, thereby improving the detection accuracy of targets of different sizes. Compared with some traditional target detection models, YOLOv5L can more comprehensively utilize the multi-scale information of the image and better detect small, medium, and large targets.

[0100] 2) Model optimization.

[0101] The model training parameters in the target detection system are configured, and the training set is input into the fusion target detection unit,

[0102] In the training process, the target detection model and the complete target detection model are trained and iteratively optimized according to the configured model training parameters, the training process is monitored, the verification set is input for verification, and the performance of the target detection model and the complete target detection model on the verification set is observed.

[0103] III. Test of the target detection method for DNA nanostructure under AFM image scenario.

[0104] The model is tested using a test set containing 18 images, and the ideal detection effect is as shown in Figure 7 .

[0105] The present application scans a variety of DNA nanostructure pictures with different site information under AFM image scenario, and uses Labelme software for labeling, and establishes a high-quality AFM image dataset; based on YOLOv5L, a feature extraction and recognition model for DNA nanostructure under AFM image scenario is successfully constructed, and the experimental results show that the model has good recognition performance on the recognition task of DNA nanostructure under AFM image scenario, and the average precision reaches 98.5%

[0106] The present application combines deep learning and machine vision technology, and constructs a new target detection method for DNA nanostructure under AFM image scenario.

[0107] Embodiment 3

[0108] Based on the target detection method for DNA nanostructure under AFM scenario proposed in embodiment 1, in the present embodiment, a target detection system for DNA nanostructure under AFM scenario is proposed, which comprises:

[0109] An image acquisition module: acquiring an image dataset of DNA nanostructure under AFM scenario;

[0110] An image labeling module: data labeling and enhancement of target AFM images in the image dataset to form a target AFM image dataset;

[0111] A detection unit construction module: based on YOLOv5 model, a fusion target detection unit is established;

[0112] A detection unit training module: the fusion target detection unit is trained through the target AFM image dataset to obtain different target detection models;

[0113] And, the target detection module: according to the current scene and demand, select a corresponding trained target detection model as the best model, and input the AFM image to be detected into the target detection model for image target recognition, to obtain the target detection result.

[0114] The methods of the present application can be implemented in hardware, firmware, or as software or computer code stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or be implemented through computer code downloaded over a network originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium, so that the methods described herein can be processed by such software on a recording medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware such as an ASIC or an FPGA. It is understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code when accessed and executed by the computer, processor, or hardware, to implement the methods described herein. Furthermore, when a general purpose computer accesses code for implementing the methods shown herein, the execution of the code transforms the general purpose computer into a special purpose computer for executing the methods shown herein.

[0115] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A target detection method for DNA nanostructures in an AFM (Automatic Visualization) scenario, characterized in that, Includes the following steps: Obtain an image dataset of DNA nanostructures in an AFM (Automatic Visualization) scenario; Data annotation and enhancement are performed on the target AFM images in the image dataset to form a target AFM image dataset; A fusion target detection unit is established based on the YOLOv5 model; Different target detection models are obtained by training the fused target detection unit using the target AFM image dataset; Based on the current scenario and requirements, select a pre-trained target detection model as the best model, and input the AFM image to be detected into the target detection model to perform image target recognition and obtain the target detection result; The network structure of the YOLOv5 model includes: a backbone network, a feature pyramid network, and multiple prediction heads; The backbone network adopts the CSPDarknet structure, which contains multiple CSP modules. Each module consists of multiple convolutional layers and residual connections, which can effectively extract low-level and mid-level feature information of the image. The Feature Pyramid Network is placed after the backbone network to fuse feature information at different scales; the FPN performs upsampling and fusion operations on the feature maps of different resolutions output by the backbone network. The prediction head has multiple prediction heads set for feature maps of different scales, which are used to predict the class probability of the target, the position and size information of the bounding box; each prediction head maps the feature map to the class and location space of the target through convolutional layer and fully connected layer operation to obtain the final detection result. The specific steps for training the fused target detection unit are as follows: S41, the target AFM image dataset is divided into a training set, a test set, and a validation set; S42, Configure the model training parameters in the fusion target detection unit; S43, During the training process, the target detection model and the complete target detection model are trained and iteratively optimized according to the configured model training parameters; The target detection model realizes the function of target detection and localization, and is implemented using the YOLOv5 model; the complete target detection model is a target detection system with complete functions and processes, including not only the core fusion target detection unit, but also data preprocessing, model evaluation, optimization and integration with actual application scenarios. S44: By monitoring the training process, inputting the validation set for validation, and calculating evaluation metrics to measure the model's performance on the validation set, the performance of the object detection model and the complete object detection model on the validation set is observed. S45, after model training and tuning are completed, uses test set data to evaluate the final performance of the model; S46. When the object detection model and the complete object detection model achieve satisfactory performance, save the corresponding trained object detection model and complete object detection model.

2. The target detection method for DNA nanostructures in an AFM scenario according to claim 1, characterized in that, The steps to obtain the image dataset are as follows: S11, by pre-setting the required data samples, determine the various AFM images with DNA nanostructures that need to be collected; S12, preprocess the target number of AFM images to obtain the image dataset.

3. The target detection method for DNA nanostructures in an AFM scenario according to claim 2, characterized in that, The steps for preprocessing the AFM image include: S31, Read image data from the dataset; S32, scales the original image to the specified size according to the model's input requirements; S33, Perform data augmentation operations on the image, including random cropping, random flipping, and random rotation; S34, normalize the image.

4. The target detection method for DNA nanostructures in an AFM scenario according to claim 1, characterized in that, The target detection results include the bounding box, category, and confidence level of the detected target object.

5. A target detection system for DNA nanostructures in an AFM (Automatic Visualization) scenario, comprising the detection method according to any one of claims 1-4, characterized in that, include: Image acquisition module: Acquires image datasets of DNA nanostructures under AFM conditions; Image annotation module: performs data annotation and augmentation on the target AFM images in the image dataset to form the target AFM image dataset; Detection Unit Construction Module: Based on the YOLOv5 model, a fused target detection unit is established; Detection unit training module: Trains the fused target detection unit using the target AFM image dataset to obtain different target detection models; Additionally, the object detection module selects a pre-trained object detection model as the best model based on the current scene and requirements, and inputs the AFM image to be detected into the object detection model to perform image object recognition and obtain the object detection result.

6. A computer storage medium storing a readable program, characterized in that, When the program runs, it can execute the target detection method for DNA nanostructures in the AFM scenario as described in any one of claims 1-4.

7. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform an operation corresponding to a target detection method for DNA nanostructures in an AFM scenario as described in any one of claims 1-4.

8. A computer program product comprising computer instructions, characterized in that, The computer instructions instruct the computing device to perform the operations corresponding to the target detection method for DNA nanostructures in an AFM scenario as described in any one of claims 1-4.

Images