AFM target point positioning trajectory planning system and method based on deep learning

Abstract

The invention discloses an AFM target point positioning trajectory planning system and method based on deep learning, and relates to the technical field of nanometer manipulation.The AFM target point positioning trajectory planning system based on deep learning comprises the steps that a sample morphology image is obtained through an AFM scanning module, a sample area is determined through processing of an image segmentation module, and a target point positioning trajectory is obtained; the path planning module generates an original path, then the deep learning model module predicts a positioning track, the track comparison module compares differences, then the positioning control module realizes accurate positioning, and finally the mechanical detection module detects mechanical data of a target point.

Description

Technical Field

[0001] The present invention relates to the field of nanomanipulation technology, and in particular to an AFM target point positioning trajectory planning system and method based on deep learning. Background Art

[0002] With the rapid development of nanotechnology, the application of atomic force microscope in the field of micro-nano-scale target point detection and positioning is becoming more and more critical. However, traditional positioning technology faces many difficulties.

[0003] At the micro-nano scale, the surface morphology of the sample is complex, and the displacement of the target point is difficult to control accurately. On the one hand, the image segmentation process is susceptible to noise interference, resulting in inaccurate identification of the sample area; on the other hand, the path planning lacks an efficient strategy, and it is difficult to take into account both the optimal path and the need for obstacle avoidance. Moreover, traditional methods rely on manual experience or simple algorithms, which are difficult to cope with complex and changeable sample characteristics and environmental factors, resulting in low positioning accuracy and efficiency. They are unable to meet the urgent needs for precise positioning of micro-nano-scale target points in fields such as high-precision nanomanufacturing and bio-nanostructure research, and have become a bottleneck for the further development of nanomanipulation technology.

[0004] To solve the above problems, the present invention proposes an AFM target point positioning trajectory planning system and method based on deep learning. Summary of the invention

[0005] The purpose of the present invention is to provide an AFM target point positioning trajectory planning system and method based on deep learning to solve the problems raised in the prior art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The AFM target point positioning trajectory planning system based on deep learning comprises an AFM scanning module, an image segmentation module, a path planning module, a deep learning model module, a trajectory comparison module, a positioning control module, and a mechanical detection module; the AFM scanning module is used to scan the original morphological image; the image segmentation module is used to process the image acquired by the AFM scanning module; the path planning module is used to generate a number of original paths from the starting point to the target point based on the target point determined by the image segmentation module; the deep learning model module is used to predict the positioning trajectory according to the input original path; the trajectory comparison module is used to compare the original path with the predicted trajectory output by the deep learning model module; the positioning control module accurately locates the target point through a feedback control mechanism based on the difference information obtained by the trajectory comparison module; the mechanical detection module is used to obtain the mechanical related data of the target point and perform processing and analysis after the target point is located.

[0008] The AFM scanning module includes a probe driving unit, a signal detection and conversion unit, and a scanning parameter setting and control unit;

[0009] The probe driving unit controls the probe of the atomic force microscope through a motor driving device to scan the sample surface point by point according to a preset scanning path and speed to obtain complete morphological information of the sample;

[0010] The signal detection and conversion unit is responsible for detecting the physical signal generated by the interaction between the probe and the sample surface, and converting it into an electrical signal;

[0011] The scanning parameter setting and control unit is used to set and adjust various parameters of the AFM.

[0012] The image segmentation module includes an image preprocessing unit, a feature extraction unit and a segmentation algorithm execution unit;

[0013] The image preprocessing unit performs preliminary processing on the original image obtained by AFM scanning, and uses a median filtering algorithm to remove noise in the image to improve the image quality;

[0014] The feature extraction unit draws a grayscale histogram according to the grayscale distribution and texture features of the image processed by the image preprocessing unit, extracts image features by combining a grayscale co-occurrence matrix method, and separates the image features from the background;

[0015] The segmentation algorithm execution unit calculates the grayscale features of the image through the grayscale co-occurrence matrix and analyzes the grayscale distribution and texture information; when the image is within a preset grayscale interval, the grayscale difference between the sample and the background exceeds a preset difference threshold and presents a concentrated distribution trend, at this time, the segmentation algorithm execution unit sets a threshold according to the grayscale information analyzed above, and then judges each pixel in the image, and determines the pixels with grayscale values ​​greater than the threshold as sample pixels, and the pixels with grayscale values ​​less than the threshold as background pixels; then the threshold segmentation algorithm is used to segment the image to separate the sample area from the background.

[0016] The path planning module includes an environment modeling unit, a path search unit and a path optimization unit;

[0017] The environment modeling unit receives image data and measurement information from the AFM module, records the three-dimensional coordinates of each sampling point on the sample surface, identifies the contour and features of the sample through a point cloud data processing algorithm, converts this information into a geometric shape representation in the environment model, and marks the position of obstacles that affect the movement of the probe, thereby constructing a virtual environment model containing complete information on the sample surface;

[0018] The path search unit determines the starting point and the target point based on the environment model constructed by the environment modeling unit, uses the A* algorithm to perform path search, marks the distance of the starting point as 0, marks other points as infinity, and then starts from the starting point to search for the point adjacent to the current point with the smallest distance, and updates its distance value, and repeats the above process until the target node is found;

[0019] The path optimization unit optimizes the original path obtained by the path search unit, and the optimization contents are path length, smoothness and obstacle avoidance. For the path length, the path length is obtained by accumulating the Euclidean geometric distances between two adjacent points on the path, and the shortest path is evaluated; for the smoothness optimization of the path, a curve fitting algorithm is used, and the key points on the original path are used as control points of the fitting curve, and a smooth curve is generated by calculation as the optimized path; for the obstacle collision optimization, the collision point between the path and the obstacle in the environment model is checked, and then the position of the path point is adjusted to re-plan the local path to avoid the obstacle.

[0020] The deep learning module includes a data preprocessing unit, a neural network architecture unit, a model training unit and a prediction output unit;

[0021] The data preprocessing unit cleans and normalizes the data input by the path optimization module, and performs random rotation, translation, scaling and other transformation operations on the original path data to increase the diversity of the training data;

[0022] The neural network architecture unit is used to construct a neural network structure suitable for AFM processing positioning trajectory planning tasks, extract data features and make predictions, adopt a recurrent neural network, take the historical trajectory and the original path as time series input, learn the dynamic relationship between them through the recurrent structure, and capture the time-dependent information in path planning;

[0023] The model training unit trains the neural network model using the preprocessed data and adjusts the model parameters to minimize the prediction error. First, the mean square error loss function is defined, and the formula is as follows:

[0024]

[0025] where y i is the actual trajectory coordinate value, is the trajectory coordinate value predicted by the model, and n is the number of samples; then the Adam optimization algorithm is selected. During the training process, the data is divided into a training set, a validation set, and a test set. The training set is used to learn and update the model parameters, and the validation set is used to monitor the model performance during the training process;

[0026] After the model training is completed, the prediction output unit inputs the new original path data into the trained neural network model, outputs the predicted positioning trajectory through the forward propagation calculation of the model, and converts the coordinate value of the prediction result into a format that can be recognized by the AFM device.

[0027] The trajectory comparison module includes an attention weight calculation unit, a weighted difference calculation unit and a visualization unit;

[0028] The attention weight calculation unit compares the original path and the predicted trajectory according to the predicted trajectory and the original path data output by the deep learning model, and calculates the attention weight of each path point. The attention weight calculation method based on distance and curvature is used. For each point i on the path, the average distance d between it and the adjacent points is calculated. i and the local curvature k i , and then calculate the attention weight, the formula is as follows:

[0029] w i =α·exp(-β·d i )+γ·exp(-δ·|k i |)

[0030] Among them, α, β, γ, and δ are learnable parameters;

[0031] The weighted difference unit calculates the difference indicators in spatial position, path length, and path curvature between the original path and the predicted trajectory based on the predicted trajectory and original path data output by the deep learning model, and uses the attention weight to weight the difference of each path point to highlight the difference of key points; for the spatial position difference MSE w , calculated using the mean square error formula, the formula is modified as follows:

[0032]

[0033] where w i is the attention weight of point i, (x i ,y i )and They are the corresponding coordinate points on the original path and the predicted trajectory;

[0034] For the path length difference, first measure the length of the original path and the predicted trajectory path respectively, and for discrete path points, use the Euclidean distance accumulation method to calculate, and finally calculate the path length difference index;

[0035] For the difference in path curvature, the original path and the predicted trajectory are first fitted using a curve fitting algorithm, and then the curvatures of the two curves at each point are calculated using the curvature calculation formula. Finally, the curvature difference index is calculated using the root mean square error formula.

[0036] The visualization unit uses a drawing library to draw the original path and the predicted trajectory in the same coordinate system, and distinguishes the two, while marking key path points and areas with large differences, and analyzes the model performance and path planning results based on the comparison results.

[0037] The positioning control module includes an error analysis and compensation unit, a motion control unit and a calibration and adaptation unit;

[0038] The error analysis and compensation unit receives the difference information between the original path and the predicted trajectory obtained by the trajectory comparison module, determines the current positioning error state of the AFM scanning head by quantifying the deviation of the position, direction and speed; uses the space vector calculation method to analyze the deviation vector between the actual position of the scanning head and the target position in the three-dimensional space, as well as the angle deviation between the actual movement direction and the tangent direction of the ideal path; then, generates a compensation control signal based on these analysis results to correct the movement of the scanning head; in the compensation process, considering the dynamic characteristics and mechanical precision limitations of the AFM system, an algorithm based on the proportional-integral-differential control principle is used to adjust the error in real time;

[0039] The motion control unit drives the motor drive device of the AFM scanning head according to the control signal provided by the error analysis and compensation unit, converts the digital control signal into actual mechanical motion, and controls the displacement and rotation of the scanning head in three-dimensional space; controls the speed and direction of the motor through pulse width modulation technology, and at the same time, the unit also monitors the motion state of the scanning head, including the current position, speed and acceleration, and feeds back this information to the error analysis and compensation unit to form a closed-loop control;

[0040] During the positioning process, the calibration and adaptive unit measures and calibrates the parameters of the AFM system; performs calibration measurements using standard samples, compares the measurement results with known standard values, calculates the system's deviation parameters, and modifies the positioning control model accordingly; at the same time, the unit uses an adaptive control algorithm to automatically adjust the control parameters according to the real-time operating status of the AFM system and environmental changes.

[0041] The mechanical detection module includes a detection mode switching unit, a mechanical data acquisition unit and a data sharing and feedback unit;

[0042] The detection mode switching unit, when the positioning control module accurately positions the AFM scanning head to the target point through the feedback control mechanism according to the difference information of the trajectory comparison module, sends a trigger signal to the mechanical detection module to inform the mechanical detection module that it can start the subsequent mechanical detection operation; after the mechanical detection module receives the trigger signal, the detection mode switching unit selects the mechanical detection mode according to the preset information of the sample;

[0043] The mechanical data acquisition unit starts working in the selected detection mode, collects mechanical data related to the sample surface, and records and stores the positioning information at the time of collection;

[0044] The data sharing and feedback unit, after completing data collection and preliminary processing, the mechanical detection module will feed back some key data to the positioning control module; the positioning control module optimizes subsequent positioning operations based on these feedback information; at the same time, the positioning control module will also inform the mechanical detection module of the information during the positioning process, providing analysis factors for the mechanical detection module in subsequent data processing and analysis.

[0045] The AFM target point positioning trajectory planning method based on deep learning is characterized by comprising the following steps:

[0046] S1, using the probe driving unit in the AFM scanning module, controlling the probe of the atomic force microscope through a motor driving device to scan the sample surface point by point according to a preset scanning path and speed;

[0047] S2, preprocessing the original image transmitted by the scanning module, extracting features from the preprocessed image, and determining the sample area using a segmentation algorithm;

[0048] S3, generating an original path by a path planning module according to the obtained sample data;

[0049] S4, training the generated original path data with a deep model to predict the positioning trajectory;

[0050] S5, input the trained path data, compare the differences in the trajectory comparison module, and then locate the position in the positioning control module;

[0051] S6. Transmit the located data to the mechanical detection module to detect the mechanical data of the target point.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. Accurate positioning accuracy: In terms of positioning accuracy, the deep learning model module deeply analyzes the original path and predicts the positioning trajectory, combined with the precise difference calculation of the trajectory comparison module and the precise feedback control of the positioning control module, it can achieve high-precision positioning of micro-nano-scale target points by atomic force microscope, improve the accuracy of positioning, and overcome the problem of inaccurate positioning of traditional methods at the micro-nano scale.

[0054] 2. Efficient path planning: The environment modeling unit, path search unit and path optimization unit in the path planning module work together to build an environment model based on the point cloud data processing algorithm, and adopt efficient search and optimization algorithms to quickly generate the optimal path from the starting point to the target point, thereby improving the efficiency of path planning and reducing the time cost of AFM operations.

[0055] 3. Comprehensiveness of sample detection: After the target point is located, the mechanical detection module can obtain the mechanical data of the target point in time and process and analyze it, which provides an important basis for studying the mechanical properties of the sample and makes the understanding of the sample more in-depth and comprehensive, which is lacking in traditional AFM technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of the AFM target point positioning trajectory planning system based on deep learning of the present invention;

[0057] Figure 2 This is a schematic diagram of the results of the AFM target point positioning trajectory planning system based on deep learning of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution.

[0060] The AFM target point positioning trajectory planning system based on deep learning comprises an AFM scanning module, an image segmentation module, a path planning module, a deep learning model module, a trajectory comparison module, a positioning control module, and a mechanical detection module; the AFM scanning module is used to scan the original morphological image; the image segmentation module is used to process the image acquired by the AFM scanning module; the path planning module is used to generate a number of original paths from the starting point to the target point based on the target point determined by the image segmentation module; the deep learning model module is used to predict the positioning trajectory according to the input original path; the trajectory comparison module is used to compare the original path with the predicted trajectory output by the deep learning model module; the positioning control module accurately locates the target point through a feedback control mechanism based on the difference information obtained by the trajectory comparison module; the mechanical detection module is used to obtain the mechanical related data of the target point and perform processing and analysis after the target point is located.

[0061] The AFM scanning module includes a probe driving unit, a signal detection and conversion unit, and a scanning parameter setting and control unit;

[0062] The probe driving unit controls the probe of the atomic force microscope through a motor driving device to scan the sample surface point by point according to a preset scanning path and speed to obtain complete morphological information of the sample;

[0063] The signal detection and conversion unit is responsible for detecting the physical signal generated by the interaction between the probe and the sample surface, and converting it into an electrical signal;

[0064] The scanning parameter setting and control unit is used to set and adjust various parameters of the AFM.

[0065] The image segmentation module includes an image preprocessing unit, a feature extraction unit and a segmentation algorithm execution unit;

[0066] The image preprocessing unit performs preliminary processing on the original image obtained by AFM scanning, and uses a median filtering algorithm to remove noise in the image to improve the image quality;

[0067] The feature extraction unit draws a grayscale histogram according to the grayscale distribution and texture features of the image processed by the image preprocessing unit, extracts image features by combining a grayscale co-occurrence matrix method, and separates the image features from the background;

[0068] The segmentation algorithm execution unit calculates the grayscale features of the image through the grayscale co-occurrence matrix and analyzes the grayscale distribution and texture information; when the image is within a preset grayscale interval, the grayscale difference between the sample and the background exceeds a preset difference threshold and presents a concentrated distribution trend, at this time, the segmentation algorithm execution unit sets a threshold according to the grayscale information analyzed above, and then judges each pixel in the image, and determines the pixels with grayscale values ​​greater than the threshold as sample pixels, and the pixels with grayscale values ​​less than the threshold as background pixels; then the threshold segmentation algorithm is used to segment the image to separate the sample area from the background.

[0069] The path planning module includes an environment modeling unit, a path search unit and a path optimization unit;

[0070] The environment modeling unit receives image data and measurement information from the AFM module, records the three-dimensional coordinates of each sampling point on the sample surface, identifies the contour and features of the sample through a point cloud data processing algorithm, converts this information into a geometric shape representation in the environment model, and marks the position of obstacles that affect the movement of the probe, thereby constructing a virtual environment model containing complete information on the sample surface;

[0071] The path search unit determines the starting point and the target point based on the environment model constructed by the environment modeling unit, uses the A* algorithm to perform path search, marks the distance of the starting point as 0, marks other points as infinity, and then starts from the starting point to search for the point adjacent to the current point with the smallest distance, and updates its distance value, and repeats the above process until the target node is found;

[0072] The path optimization unit optimizes the original path obtained by the path search unit, and the optimization contents are path length, smoothness and obstacle avoidance. For the path length, the path length is obtained by accumulating the Euclidean geometric distances between two adjacent points on the path, and the shortest path is evaluated; for the smoothness optimization of the path, a curve fitting algorithm is used, and the key points on the original path are used as control points of the fitting curve, and a smooth curve is generated by calculation as the optimized path; for the obstacle collision optimization, the collision point between the path and the obstacle in the environment model is checked, and then the position of the path point is adjusted to re-plan the local path to avoid the obstacle.

[0073] The deep learning module includes a data preprocessing unit, a neural network architecture unit, a model training unit and a prediction output unit;

[0074] The data preprocessing unit cleans and normalizes the data input by the path optimization module, and performs random rotation, translation, scaling and other transformation operations on the original path data to increase the diversity of the training data;

[0075] The neural network architecture unit is used to construct a neural network structure suitable for AFM processing positioning trajectory planning tasks, extract data features and make predictions, adopt a recurrent neural network, take the historical trajectory and the original path as time series input, learn the dynamic relationship between them through the recurrent structure, and capture the time-dependent information in path planning;

[0076] The model training unit trains the neural network model using the preprocessed data and adjusts the model parameters to minimize the prediction error. First, the mean square error loss function is defined, and the formula is as follows:

[0077]

[0078] where y i is the actual trajectory coordinate value, is the trajectory coordinate value predicted by the model, and n is the number of samples; then the Adam optimization algorithm is selected. During the training process, the data is divided into a training set, a validation set, and a test set. The training set is used to learn and update the model parameters, and the validation set is used to monitor the model performance during the training process;

[0079] After the model training is completed, the prediction output unit inputs the new original path data into the trained neural network model, outputs the predicted positioning trajectory through the forward propagation calculation of the model, and converts the coordinate value of the prediction result into a format that can be recognized by the AFM device.

[0080] The trajectory comparison module includes an attention weight calculation unit, a weighted difference calculation unit and a visualization unit;

[0081] The attention weight calculation unit compares the original path and the predicted trajectory according to the predicted trajectory and the original path data output by the deep learning model, and calculates the attention weight of each path point. The attention weight calculation method based on distance and curvature is used. For each point i on the path, the average distance d between it and the adjacent points is calculated. i And the local curvature ki, and then calculate the attention weight, the formula is as follows:

[0082] w i =α·exp(-β·d i )+γ·exp(-δ·|k i |)

[0083] Among them, α, β, γ, and δ are learnable parameters;

[0084] The weighted difference unit calculates the difference indicators in spatial position, path length, and path curvature between the original path and the predicted trajectory based on the predicted trajectory and original path data output by the deep learning model, and uses the attention weight to weight the difference of each path point to highlight the difference of key points; for the spatial position difference MSE w , calculated using the mean square error formula, the formula is modified as follows:

[0085]

[0086] where w i is the attention weight of point i, (x i ,y i )and They are the corresponding coordinate points on the original path and the predicted trajectory;

[0087] For the path length difference, first measure the length of the original path and the predicted trajectory path respectively, and for discrete path points, use the Euclidean distance accumulation method to calculate, and finally calculate the path length difference index;

[0088] For the difference in path curvature, the original path and the predicted trajectory are first fitted using a curve fitting algorithm, and then the curvatures of the two curves at each point are calculated using the curvature calculation formula. Finally, the curvature difference index is calculated using the root mean square error formula.

[0089] The visualization unit uses a drawing library to draw the original path and the predicted trajectory in the same coordinate system, and distinguishes the two, while marking key path points and areas with large differences, and analyzes the model performance and path planning results based on the comparison results.

[0090] The positioning control module includes an error analysis and compensation unit, a motion control unit and a calibration and adaptation unit;

[0091] The error analysis and compensation unit receives the difference information between the original path and the predicted trajectory obtained by the trajectory comparison module, determines the current positioning error state of the AFM scanning head by quantifying the deviation of the position, direction and speed; uses the space vector calculation method to analyze the deviation vector between the actual position of the scanning head and the target position in the three-dimensional space, as well as the angle deviation between the actual movement direction and the tangent direction of the ideal path; then, generates a compensation control signal based on these analysis results to correct the movement of the scanning head; in the compensation process, considering the dynamic characteristics and mechanical precision limitations of the AFM system, an algorithm based on the proportional-integral-differential control principle is used to adjust the error in real time;

[0092] The motion control unit drives the motor drive device of the AFM scanning head according to the control signal provided by the error analysis and compensation unit, converts the digital control signal into actual mechanical motion, and controls the displacement and rotation of the scanning head in three-dimensional space; controls the speed and direction of the motor through pulse width modulation technology, and at the same time, the unit also monitors the motion state of the scanning head, including the current position, speed and acceleration, and feeds back this information to the error analysis and compensation unit to form a closed-loop control;

[0093] During the positioning process, the calibration and adaptive unit measures and calibrates the parameters of the AFM system; performs calibration measurements using standard samples, compares the measurement results with known standard values, calculates the system's deviation parameters, and modifies the positioning control model accordingly; at the same time, the unit uses an adaptive control algorithm to automatically adjust the control parameters according to the real-time operating status of the AFM system and environmental changes.

[0094] The mechanical detection module includes a detection mode switching unit, a mechanical data acquisition unit and a data sharing and feedback unit;

[0095] The detection mode switching unit, when the positioning control module accurately positions the AFM scanning head to the target point through the feedback control mechanism according to the difference information of the trajectory comparison module, sends a trigger signal to the mechanical detection module to inform the mechanical detection module that it can start the subsequent mechanical detection operation; after the mechanical detection module receives the trigger signal, the detection mode switching unit selects the mechanical detection mode according to the preset information of the sample;

[0096] The mechanical data acquisition unit starts working in the selected detection mode, collects mechanical data related to the sample surface, and records and stores the positioning information at the time of collection;

[0097] The data sharing and feedback unit, after completing data collection and preliminary processing, the mechanical detection module will feed back some key data to the positioning control module; the positioning control module optimizes subsequent positioning operations based on these feedback information; at the same time, the positioning control module will also inform the mechanical detection module of the information during the positioning process, providing analysis factors for the mechanical detection module in subsequent data processing and analysis.

[0098] The AFM target point positioning trajectory planning method based on deep learning is characterized by comprising the following steps:

[0099] S1, using the probe driving unit in the AFM scanning module, controlling the probe of the atomic force microscope through a motor driving device to scan the sample surface point by point according to a preset scanning path and speed;

[0100] S2, preprocessing the original image transmitted by the scanning module, extracting features from the preprocessed image, and determining the sample area using a segmentation algorithm;

[0101] S3, generating an original path by a path planning module according to the obtained sample data;

[0102] S4, training the generated original path data with a deep model to predict the positioning trajectory;

[0103] S5, input the trained path data, compare the differences in the trajectory comparison module, and then locate the position in the positioning control module;

[0104] S6. Transmit the located data to the mechanical detection module to detect the mechanical data of the target point.

[0105] Assume that the detection of a specific target point on the surface of a nanomaterial sample is taken as an example. First, the probe drive unit of the AFM scanning module controls the probe to scan the sample according to the set path and speed. The signal detection and conversion unit converts the signal of the probe and the sample into an electrical signal. The scanning parameter setting and control unit adjust the parameters to obtain the morphological image. In the image segmentation module, the image preprocessing unit uses the median filter to remove noise, the feature extraction unit extracts features, and the segmentation algorithm execution unit segments the sample area according to the grayscale features. The environmental modeling unit of the path planning module records the three-dimensional coordinates of the sample sampling point to build a model and mark obstacles. The path search unit uses the A algorithm to find the path from the starting point to the target point. The path optimization unit optimizes the path length, smoothness and obstacle avoidance. After the data preprocessing unit of the deep learning module processes the data, the neural network architecture unit constructs a recurrent neural network training, the model training unit uses the mean square error loss function and the Adam algorithm for training, and the prediction output unit outputs the positioning trajectory. The trajectory comparison module calculates the attention weight and difference index and visualizes them. The positioning control module analyzes the error according to the difference information, the error analysis and compensation unit analyzes the error, the motion control unit drives the scanning head to position, and the calibration and adaptation unit calibrates the parameters. Finally, after the mechanical detection module is positioned, the detection mode switching unit selects the mode, the mechanical data acquisition unit collects data and records the positioning information, and the data sharing and feedback unit exchanges data with the positioning control module.

[0106] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. AFM target point positioning trajectory planning system based on deep learning, characterized by: It comprises an AFM scanning module, an image segmentation module, a path planning module, a deep learning model module, a trajectory comparison module, a positioning control module and a mechanical detection module; the AFM scanning module is used to scan the original topographic image; the image segmentation module is used to process the image acquired by the AFM scanning module; the path planning module is used to generate a number of original paths from the starting point to the target point based on the target point determined by the image segmentation module; the deep learning model module is used to predict the positioning trajectory according to the input original path; the trajectory comparison module is used to compare the original path with the predicted trajectory output by the deep learning model module; the positioning control module accurately locates the target point through a feedback control mechanism according to the difference information obtained by the trajectory comparison module; the mechanical detection module is used to obtain the mechanical related data of the target point and process and analyze it after the target point is located.

2. The AFM target point positioning trajectory planning system based on deep learning according to claim 1, characterized in that: The AFM scanning module includes a probe driving unit, a signal detection and conversion unit, and a scanning parameter setting and control unit; The probe driving unit controls the probe of the atomic force microscope through a motor driving device to scan the sample surface point by point according to a preset scanning path and speed to obtain complete morphological information of the sample; The signal detection and conversion unit is responsible for detecting the physical signal generated by the interaction between the probe and the sample surface, and converting it into an electrical signal; The scanning parameter setting and control unit is used to set and adjust various parameters of the AFM.

3. The AFM target point positioning trajectory planning system based on deep learning according to claim 1, characterized in that: The image segmentation module includes an image preprocessing unit, a feature extraction unit and a segmentation algorithm execution unit; The image preprocessing unit performs preliminary processing on the original image obtained by AFM scanning, and uses a median filtering algorithm to remove noise in the image to improve the image quality; The feature extraction unit draws a grayscale histogram according to the grayscale distribution and texture features of the image processed by the image preprocessing unit, extracts image features by combining a grayscale co-occurrence matrix method, and separates the image features from the background; The segmentation algorithm execution unit calculates the grayscale features of the image through the grayscale co-occurrence matrix and analyzes the grayscale distribution and texture information; when the image is within a preset grayscale interval, the grayscale difference between the sample and the background exceeds a preset difference threshold and presents a concentrated distribution trend, at this time, the segmentation algorithm execution unit sets a threshold according to the grayscale information analyzed above, and then judges each pixel in the image, and determines the pixels with grayscale values ​​greater than the threshold as sample pixels, and the pixels with grayscale values ​​less than the threshold as background pixels; then the threshold segmentation algorithm is used to segment the image to separate the sample area from the background.

4. The AFM target point positioning trajectory planning system based on deep learning according to claim 1, characterized in that: The path planning module includes an environment modeling unit, a path search unit and a path optimization unit; The environment modeling unit receives image data and measurement information from the AFM module, records the three-dimensional coordinates of each sampling point on the sample surface, identifies the contour and features of the sample through a point cloud data processing algorithm, converts this information into a geometric shape representation in the environment model, and marks the position of obstacles that affect the movement of the probe, thereby constructing a virtual environment model containing complete information on the sample surface; The path search unit determines the starting point and the target point based on the environment model constructed by the environment modeling unit, uses the A* algorithm to perform path search, marks the distance of the starting point as 0, marks other points as infinity, and then starts from the starting point to search for the point adjacent to the current point with the smallest distance, and updates its distance value, and repeats the above process until the target node is found; The path optimization unit optimizes the original path obtained by the path search unit, and the optimization contents are path length, smoothness and obstacle avoidance. For the path length, the path length is obtained by accumulating the Euclidean geometric distances between two adjacent points on the path, and the shortest path is evaluated; for the smoothness optimization of the path, a curve fitting algorithm is used, and the key points on the original path are used as control points of the fitting curve, and a smooth curve is generated by calculation as the optimized path; for the obstacle collision optimization, the collision point between the path and the obstacle in the environment model is checked, and then the position of the path point is adjusted to re-plan the local path to avoid the obstacle.

5. The AFM target point positioning trajectory planning system based on deep learning according to claim 1, characterized in that: The deep learning module includes a data preprocessing unit, a neural network architecture unit, a model training unit and a prediction output unit; The data preprocessing unit cleans and normalizes the data input by the path optimization module, and performs random rotation, translation, scaling and other transformation operations on the original path data to increase the diversity of the training data; The neural network architecture unit is used to construct a neural network structure suitable for AFM processing positioning trajectory planning tasks, extract data features and make predictions, adopt a recurrent neural network, take the historical trajectory and the original path as time series input, learn the dynamic relationship between them through the recurrent structure, and capture the time-dependent information in path planning; The model training unit trains the neural network model using the preprocessed data and adjusts the model parameters to minimize the prediction error. First, the mean square error loss function is defined, and the formula is as follows: where y i is the actual trajectory coordinate value, is the trajectory coordinate value predicted by the model, and n is the number of samples; then the Adam optimization algorithm is selected. During the training process, the data is divided into a training set, a validation set, and a test set. The training set is used to learn and update the model parameters, and the validation set is used to monitor the model performance during the training process; After the model training is completed, the prediction output unit inputs the new original path data into the trained neural network model, outputs the predicted positioning trajectory through the forward propagation calculation of the model, and converts the coordinate value of the prediction result into a format that can be recognized by the AFM device.

6. The AFM target point positioning trajectory planning system based on deep learning according to claim 1, characterized in that: The trajectory comparison module includes an attention weight calculation unit, a weighted difference calculation unit and a visualization unit; The attention weight calculation unit compares the original path and the predicted trajectory according to the predicted trajectory and the original path data output by the deep learning model, and calculates the attention weight of each path point. The attention weight calculation method based on distance and curvature is used. For each point i on the path, the average distance d between it and the adjacent points is calculated. i and the local curvature k i , and then calculate the attention weight, the formula is as follows: w i =α·exp(-β·d i )+γ·exp(-δ|k i |) Among them, α, β, γ ,δ is a learnable parameter; The weighted difference unit calculates the difference indicators in spatial position, path length, and path curvature between the original path and the predicted trajectory based on the predicted trajectory and original path data output by the deep learning model, and uses the attention weight to weight the difference of each path point to highlight the difference of key points; for the spatial position difference MSE w , calculated using the mean square error formula, the formula is modified as follows: where w i is the attention weight of point i, (x i ,y i )and They are the corresponding coordinate points on the original path and the predicted trajectory; For the path length difference, first measure the length of the original path and the predicted trajectory path respectively, and for discrete path points, use the Euclidean distance accumulation method to calculate, and finally calculate the path length difference index; For the difference in path curvature, the original path and the predicted trajectory are first fitted using a curve fitting algorithm, and then the curvatures of the two curves at each point are calculated using the curvature calculation formula. Finally, the curvature difference index is calculated using the root mean square error formula. The visualization unit uses a drawing library to draw the original path and the predicted trajectory in the same coordinate system, and distinguishes the two, while marking key path points and areas with large differences, and analyzes the model performance and path planning results based on the comparison results.

7. The AFM target point positioning trajectory planning system based on deep learning according to claim 1, characterized in that: The positioning control module includes an error analysis and compensation unit, a motion control unit and a calibration and adaptation unit; The error analysis and compensation unit receives the difference information between the original path and the predicted trajectory obtained by the trajectory comparison module, and determines the current positioning error state of the AFM scanning head by quantifying the deviation of the position, direction and speed; Using the space vector calculation method, the deviation vector between the actual position of the scanning head and the target position in three-dimensional space, as well as the angular deviation between the actual motion direction and the tangent direction of the ideal path are analyzed; Then, based on these analysis results, compensation control signals are generated to correct the movement of the scanning head. During the compensation process, the dynamic characteristics and mechanical precision limitations of the AFM system are considered, and an algorithm based on the proportional-integral-differential control principle is used to adjust the error in real time. The motion control unit drives the motor drive device of the AFM scanning head according to the control signal provided by the error analysis and compensation unit, converts the digital control signal into actual mechanical motion, and controls the displacement and rotation of the scanning head in three-dimensional space; The speed and direction of the motor are controlled by pulse width modulation technology. At the same time, the unit also monitors the motion state of the scanning head, including the current position, speed and acceleration, and feeds this information back to the error analysis and compensation unit to form a closed-loop control; The calibration and adaptation unit measures and calibrates the parameters of the AFM system during the positioning process; Calibration measurements are performed using standard samples, and the measurement results are compared with known standard values ​​to calculate the system deviation parameters, and the positioning control model is corrected accordingly. At the same time, the unit uses an adaptive control algorithm to automatically adjust the control parameters according to the real-time operating status of the AFM system and environmental changes.

8. The AFM target point positioning trajectory planning system based on deep learning according to claim 1, characterized in that: The mechanical detection module includes a detection mode switching unit, a mechanical data acquisition unit and a data sharing and feedback unit; The detection mode switching unit, when the positioning control module accurately positions the AFM scanning head to the target point through the feedback control mechanism according to the difference information of the trajectory comparison module, sends a trigger signal to the mechanical detection module to inform the mechanical detection module that it can start the subsequent mechanical detection operation; after the mechanical detection module receives the trigger signal, the detection mode switching unit selects the mechanical detection mode according to the preset information of the sample; The mechanical data acquisition unit starts working in the selected detection mode, collects mechanical data related to the sample surface, and records and stores the positioning information at the time of collection; The data sharing and feedback unit, after completing data collection and preliminary processing, the mechanical detection module will feed back some key data to the positioning control module; the positioning control module optimizes subsequent positioning operations based on these feedback information; at the same time, the positioning control module will also inform the mechanical detection module of the information during the positioning process, providing analysis factors for the mechanical detection module in subsequent data processing and analysis.

9. A deep learning-based AFM target point positioning trajectory planning method, applied to the deep learning-based AFM target point positioning trajectory planning system according to any one of claims 1 to 8, characterized in that: The following steps are involved: S1, using the probe driving unit in the AFM scanning module, controlling the probe of the atomic force microscope through a motor driving device to scan the sample surface point by point according to a preset scanning path and speed; S2, preprocessing the original image transmitted by the scanning module, extracting features from the preprocessed image, and determining the sample area using a segmentation algorithm; S3, generating an original path by a path planning module according to the obtained sample data; S4, training the generated original path data with a deep model to predict the positioning trajectory; S5, input the trained path data, compare the differences in the trajectory comparison module, and then locate the position in the positioning control module; S6. Transmit the located data to the mechanical detection module to detect the mechanical data of the target point.

Images