A column chromatography automated control and analysis method and system based on DS-Transformer image segmentation

Abstract

The present invention discloses a column chromatography automated control and analysis method and system based on DS-Transformer image segmentation, which relates to the field of computer vision technology, and includes the steps of obtaining a column chromatography real-time process image; preprocessing the column chromatography real-time process image to obtain a preprocessed column chromatography real-time process image; using the preprocessed column chromatography real-time process image as input, and obtaining a segmentation result map using a DS-Transformer image segmentation model output; calculating the color band position according to the segmentation result map; generating a control instruction according to the color band position, and completing corresponding control operations according to the control instruction. The present invention introduces mechanical arm control and computer vision technology into the column chromatography system, which greatly reduces human errors, improves the experimental efficiency and analysis accuracy of the column chromatography, and effectively reduces the labor cost and consumables use of long-term operations, thereby reducing the analysis cost of each sample.

Description

Technical Field

[0001] The invention relates to a column chromatography automatic control and analysis method and system based on DS-Transformer image segmentation, belonging to the technical field of computer vision. Background Art

[0002] The importance of column chromatography in chromatographic analysis is reflected in its efficient separation ability, wide range of applications, high sensitivity and precision. It can effectively separate the components in a mixture by selecting appropriate stationary phase and mobile phase. It is suitable for the analysis of small molecules to large molecules, including volatile and non-volatile compounds, making it play a key role in environmental monitoring, food safety, drug development and life sciences.

[0003] Although column chromatography technology is extremely important in the field of chemistry, its complex operating procedures, efficiency bottlenecks and safety risks are still urgent problems to be solved. From sample pretreatment to data analysis, each step of column chromatography requires delicate operation, which makes the overall process time-consuming and error-prone. For users, the most critical and time-consuming operation is to replace the receiving container at the right time. This requires personnel to observe the column chromatography instrument at all times, otherwise the purified material will be lost or contaminated, resulting in experimental failure. With the growing demand for high-throughput analysis, the time-consuming steps of traditional chromatography technology limit its analytical efficiency. In addition, the use of high-pressure gas and organic solvents during the operation increases the safety risks of experimental operations. To this end, developing simplified operating procedures, improving the level of automation, and using safer and more effective analytical techniques are key directions for the development of this field, which will help improve the performance and practicality of column chromatography in various applications.

[0004] So far, the research and application of automation of column chromatography have yielded results. Modern column chromatography systems are equipped with automatic sample injectors that can automatically load samples and perform pretreatment steps such as cleaning and dilution, improving the efficiency and accuracy of sample processing. Equipment such as high performance liquid chromatography (HPLC) and gas chromatography (GC) have achieved a high degree of integration, including automatic solvent gradient generation, temperature control, flow control, etc., to simplify the operating process. Although the above-mentioned automated column chromatography equipment is extremely valuable in the field of analysis, it faces problems such as high cost, difficulty in popularization, challenges in large-scale application, and environmental impact. Expensive equipment and consumables, and the complexity of technical operation and maintenance limit its popularity, and it is inefficient when processing a large number of samples. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a column chromatography automated control and analysis method and system based on DS-Transformer image segmentation, introduce robotic arm control and computer vision technology into the column chromatography system, reduce the technical threshold of complex operations through automated and high-precision operations, enhance the safety and flexibility of the experimental process through remote monitoring, greatly reduce human errors, improve the experimental efficiency and analysis accuracy of column chromatography, and effectively reduce the labor cost and consumables used in long-term operations, thereby reducing the analysis cost of each sample.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0007] In one aspect, the present invention provides a column chromatography automated control and analysis method based on DS-Transformer image segmentation, comprising:

[0008] Get real-time process images of column chromatography;

[0009] Preprocessing the column chromatography real-time process image to obtain a preprocessed column chromatography real-time process image;

[0010] The pre-processed column chromatography real-time process image is used as input, and the segmentation result image is obtained based on the output of the DS-Transformer image segmentation model;

[0011] The color band position coordinates are calculated based on the segmentation result image;

[0012] Compare the ordinate of the ribbon position with a preset judgment threshold, and if the ordinate of the ribbon position is less than the preset judgment threshold, generate a control instruction to replace the receiving container, otherwise generate a control instruction not to replace the receiving container;

[0013] The control instructions are transmitted to the robotic arm, and the robotic arm completes the corresponding control operations according to the control instructions.

[0014] Furthermore, the preprocessing of the column chromatography real-time process image to obtain the preprocessed column chromatography real-time process image includes:

[0015] The column chromatography real-time process image is bilaterally filtered and denoised, and the calculation expression is:

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] in, represents the bilateral filter position coordinates, Represents the pixel position coordinates of the field, Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The Euclidean distance of represents the standard deviation of the spatial distance of locations within the neighborhood, Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The absolute value of the difference, Represents the bilateral filter position coordinates The gray value of Represents the neighborhood pixel position coordinates The gray value of represents the standard deviation of the gray value in the neighborhood. Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The weight coefficient of Represents the bilateral filter position coordinates The filtering result.

[0021] Furthermore, the DS-Transformer image segmentation model includes a dynamic feature extractor, a feature pyramid network module and a hierarchical object token connected in sequence, and the output ends of the feature pyramid network module and the hierarchical object token are both connected to the input end of the multi-stage deformation decoder;

[0022] The dynamic feature extractor is used to extract features from the input image to obtain a high-level scale feature map and a low-level scale feature map;

[0023] The feature pyramid network module is used to fuse the high-level scale feature map and the low-level scale feature map to obtain a fused feature map;

[0024] The hierarchical object token is used to represent the local area features in the fused feature map;

[0025] The multi-stage deformation decoder is used to gradually optimize the hierarchical object tokens and their corresponding local area features, and convert the hierarchical object tokens into a segmentation result map.

[0026] Furthermore, the dynamic feature extractor includes two processing branches, wherein one processing branch is used to output a high-level scale feature map, which includes an upsampling module, a gating module and a dynamic convolution module connected in sequence; the other processing branch is used to output a low-level scale feature map, which includes a gating module and a dynamic convolution module connected in sequence;

[0027] The upsampling module is used to perform an upsampling operation on the input image to obtain an upsampled image. The gating module includes a global pooling layer, a fully connected layer, an activation layer and a fully connected layer connected in sequence. The gating module is used to process the input image to generate a dynamic control signal. The dynamic convolution module is used to select a convolution kernel according to the dynamic control signal to perform feature extraction on the input image to obtain a multi-scale dynamic feature map, and the multi-scale dynamic feature map includes a high-level scale feature map and a low-level scale feature map.

[0028] Furthermore, the data processing process of the feature pyramid network module includes:

[0029] Expand the high-level scale feature map and the low-level scale feature map to obtain a high-level feature sequence and a low-level feature sequence respectively;

[0030] The relationship weight matrix between high-level feature sequences and low-level feature sequences is calculated based on the attention mechanism, and its expression is:

[0031] ;

[0032] in, represents the relationship weight matrix, represents the query matrix, which takes the low-level feature sequence, represents the key matrix, represents the value matrix, , All high-level feature sequences are taken;

[0033] The relationship weight matrix is ​​multiplied by the low-level feature sequence to obtain the weighted low-level feature sequence, and the weighted low-level feature sequence and the high-level feature sequence are concatenated and deformed to obtain the fused feature map, which is expressed as:

[0034] ;

[0035] in, represents the fused feature map, Represents a deformation operation, represents the relationship weight matrix, represents a low-level feature sequence, Represents a high-level feature sequence.

[0036] Furthermore, the data processing process of the hierarchical object token includes:

[0037] Randomly generate multiple learnable query vectors;

[0038] The query vector is matched with the local area features of the fused feature map respectively to generate an object token corresponding to the query vector, wherein the matching is achieved by calculating the similarity between the query vector and the local area features of the fused feature map, and its expression is:

[0039] ;

[0040] ;

[0041] in, represents the query vector, Represents the local area features in the fusion feature map, represents the L2 norm of the query vector, represents the L2 norm of the local area features of the fused feature map, n represents the number of query vectors, represents the i-th query vector, represents the i-th local area feature in the fusion feature map, m represents the number of local area features, Indicates similarity;

[0042] The object token is initialized and then outputted. The initialization is to perform an average linear combination of the weighted features of the local area of ​​the fusion feature map, and its expression is:

[0043] ;

[0044] in, represents the j-th object token, Represents the average gray value of the i-th local area feature in the fusion feature map, represents the weighting coefficient, and m represents the number of local area features.

[0045] Furthermore, the multi-stage deformable decoder includes a Flatten and position encoding layer, a cross attention layer, a feedforward neural network layer, a residual connection and a normalization layer and a recovery layer connected in sequence, the Flatten and position encoding layer is used to flatten the fused feature map into a two-dimensional sequence, the cross attention layer is used to obtain the dependency in the two-dimensional sequence according to the attention mechanism, and combine the object token with the two-dimensional sequence to generate a feature map, the feedforward neural network layer is used to enhance the nonlinear mapping capability, the residual connection and normalization layer is used to ensure the stability of the information flow to avoid gradient disappearance, the recovery layer is used to upsample the feature map using bilinear difference and convert it into a single-channel segmentation mask, and perform a pixel-by-pixel AND operation on the model input image and the single-channel segmentation mask to obtain a segmentation result map.

[0046] Furthermore, the calculating of the color band position coordinates according to the segmentation result graph includes:

[0047] The coordinates of the centroid of the color band are calculated based on the segmentation result map, and the expression is:

[0048] ;

[0049] in, The horizontal coordinate representing the centroid position of the color band, The ordinate represents the centroid position of the ribbon. The coordinates on the segmentation result map are The pixel value at , W represents the width of the segmentation result image, and H represents the height of the segmentation result image;

[0050] The color band centroid position coordinates are the color band position coordinates.

[0051] On the other hand, the present invention also provides a column chromatography automated control and analysis system based on DS-Transformer image segmentation, which is used to implement any of the above-mentioned column chromatography automated control and analysis methods based on DS-Transformer image segmentation, and specifically comprises a controller, a camera, a column chromatograph and a robotic arm, wherein the camera and the robotic arm are respectively arranged on both sides of the column chromatograph;

[0052] The column chromatograph is used to perform column chromatography experiments, the camera is used to photograph the column chromatography to obtain a real-time process image of the column chromatography, and transmit it to the controller, the controller is used to obtain the real-time process image of the column chromatography and generate a control instruction on whether to replace the receiving container, and transmit the control instruction to the robotic arm, and the robotic arm is used to complete the corresponding control operation according to the control instruction;

[0053] Wherein, the controller comprises:

[0054] An image acquisition module is configured to acquire a real-time process image of the column chromatography;

[0055] An image preprocessing module is configured to preprocess the column chromatography real-time process image to obtain a preprocessed column chromatography real-time process image;

[0056] An image analysis module is configured to take the pre-processed column chromatography real-time process image as input and obtain a segmentation result image using the DS-Transformer image segmentation model output;

[0057] A color band position determination module is configured to calculate the color band position coordinates according to the segmentation result image;

[0058] The instruction generation module is configured to generate control instructions according to the position of the ribbon and transmit the control instructions to the robot arm.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] The present invention combines the DS-Transformer image segmentation model with a robotic arm, obtains a real-time process image of the column chromatography by video, processes the image segmentation model and outputs the segmentation result, obtains the ribbon position according to the segmentation result, generates a control instruction, and finally uses the robotic arm to complete the beaker replacement operation, which greatly improves the accuracy and stability of the robotic arm in classifying and identifying objects and grasping them through the camera, and can well solve the problem of unstable center of gravity caused by the deviation in the calculation of the object coordinates when the robotic arm performs grasping, and solves the problem that the camera cannot be widely used when the robotic arm is cross-functionally dispatched;

[0061] The present invention introduces robotic arm control and computer vision technology into the column chromatography system, lowers the technical threshold for complex operations through automated and high-precision operations, enhances the safety and flexibility of the experimental process through remote monitoring, greatly reduces human errors, improves the experimental efficiency and analysis accuracy of column chromatography, and effectively reduces the labor cost and consumables used in long-term operations, thereby reducing the analysis cost of each sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic flow chart of a column chromatography automated control and analysis method based on DS-Transformer image segmentation in one embodiment of the present invention;

[0063] Figure 2 It is a structural schematic diagram of a DS-Transformer image segmentation model of a column chromatography automated control and analysis method based on DS-Transformer image segmentation in one embodiment of the present invention;

[0064] Figure 3 A schematic structural diagram of a dynamic feature extractor (DFE) of a DS-Transformer image segmentation model of a column chromatography automated control and analysis method based on DS-Transformer image segmentation in one embodiment of the present invention;

[0065] Figure 4 A schematic diagram of the structure of a scale-aware feature pyramid (SFP) of a DS-Transformer image segmentation model of a column chromatography automated control and analysis method based on DS-Transformer image segmentation in one embodiment of the present invention;

[0066] Figure 5 Schematic diagram of the structure of the column chromatography automation control and analysis system based on DS-Transformer image segmentation in Example 1 of the present invention. DETAILED DESCRIPTION

[0067] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0068] Embodiment 1:

[0069] like Figure 1 and Figure 5 As shown, an embodiment of the present invention provides a column chromatography automated control and analysis method based on DS-Transformer image segmentation, comprising the following steps:

[0070] Acquire the real-time process image of the column chromatography, and start the image acquisition device to capture the real-time process image of the column chromatography. In this embodiment, the image acquisition device adopts an IMX camera.

[0071] The column chromatography real-time process image is preprocessed to obtain a preprocessed column chromatography real-time process image. The preprocessing is to perform bilateral filtering and denoising on the column chromatography real-time process image. The calculation expression is as follows:

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] in, represents the bilateral filter position coordinates, Represents the pixel position coordinates of the field, Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The Euclidean distance of represents the standard deviation of the spatial distance of locations within the neighborhood, Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The absolute value of the difference, Represents the bilateral filter position coordinates The gray value of Represents the neighborhood pixel position coordinates The gray value of represents the standard deviation of the gray value in the neighborhood. Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The weight coefficient of Represents the bilateral filter position coordinates The filtering result.

[0077] like Figure 2 As shown in the figure, a DS-Transformer image segmentation model is constructed, and its structure includes a dynamic feature extractor, a feature pyramid network module, and a hierarchical object token connected in sequence. The output ends of the feature pyramid network module and the hierarchical object token are connected to the input end of the multi-level deformation decoder. Specifically:

[0078] The input image of the dynamic feature extractor is the preprocessed column chromatography real-time process image, which is an RGB image in this embodiment with a size of 256×256×3. The dynamic feature extractor extracts features from the input image to obtain a high-level scale feature map and a low-level scale feature map, and the sizes of the high-level scale feature map and the low-level scale feature map are both 128×128×6.

[0079] Combination Figure 3 The core idea of ​​the dynamic feature extractor (DFE) is to extract the most relevant features by dynamically adjusting the structure of the convolution kernel based on the adaptive characteristics of the input image content. The traditional convolution layer uses a fixed convolution kernel to extract image features, while the DFE adjusts the shape of the convolution kernel according to the different contents of the input image, making feature extraction more flexible and targeted.

[0080] The dynamic feature extractor includes two processing branches, one of which is used to output a high-level scale feature map, which includes an upsampling module, a gating module and a dynamic convolution module connected in sequence, and the other processing branch is used to output a low-level scale feature map, which includes a gating module and a dynamic convolution module connected in sequence. The upsampling module performs an upsampling operation on the input image to obtain an upsampled image. The gating module is used to process the input image to generate a dynamic control signal, which includes a global pooling layer, a fully connected layer, an activation layer and a fully connected layer connected in sequence. The dynamic convolution module is used to select a convolution kernel according to the dynamic control signal to extract features from the input image to obtain a corresponding high-level scale feature map or a low-level scale feature map. The size of the high-level scale feature map and the low-level scale feature map is 128×128×6.

[0081] Combination Figure 4 , the feature pyramid network module (SFP) is used to fuse the high-level scale feature map and the low-level scale feature map to obtain a fused feature map. Its data processing process specifically includes:

[0082] First, the high-level scale feature map and the low-level scale feature map are expanded into 128×128×6 feature sequences respectively, and then the attention mechanism is used to calculate the relationship weight matrix between the high-level scale feature map and the low-level scale feature map to generate a fused feature map.

[0083] The attention mechanism calculation is specifically as follows:

[0084] The query matrix and the key matrix are multiplied to obtain the attention matrix, and then the attention matrix is ​​normalized by the softmax function. The normalized attention matrix is ​​multiplied by the value matrix V to obtain the relationship weight matrix. In this embodiment, the low-level scale feature map is used as the query matrix, and the high-level scale feature map is used as the key matrix and the value matrix during calculation. The expression is:

[0085]

[0086] in, represents the relationship weight matrix, represents the query matrix, which takes the low-level feature sequence, represents the key matrix, represents the value matrix, , All high-level feature sequences are taken.

[0087] Then, the relationship weight matrix is ​​multiplied by the low-level feature sequence to obtain the weighted low-level feature sequence, and the weighted low-level feature sequence and the high-level feature sequence are concatenated and deformed to obtain a fused feature map. The size of the fused feature map is also 128×128×6, and its expression is:

[0088] ;

[0089] in, represents the fused feature map, Represents a deformation operation, represents the relationship weight matrix, represents a low-level feature sequence, Represents a high-level feature sequence.

[0090] Hierarchical Object Token (HOT) is used to represent the local area features in the fusion feature map. The data processing process includes:

[0091] A plurality of learnable query vectors are randomly generated. The query vector is a set of vectors of fixed dimension. In this embodiment, the dimension of the query vector is 16, and the number is 8 in total.

[0092] The query vector is matched with the local area features of the fused feature map to generate the object token corresponding to the query vector. The matching process is realized by calculating the similarity between the query vector and the local area features of the fused feature map. The calculation expression is as follows:

[0093] ;

[0094] ;

[0095] in, represents the query vector, Represents the local area features in the fusion feature map, represents the L2 norm of the query vector, represents the L2 norm of the local area features of the fused feature map, n represents the number of query vectors, in this embodiment n=8, represents the i-th query vector, represents the i-th local area feature in the fusion feature map, m represents the number of local area features, in this embodiment, m=16, Represents similarity, and its value range is The closer the value is to 1, the more similar the two are, and the closer the value is to -1, the less similar the two are.

[0096] Initialize the object token, that is, perform an average linear combination of the weighted features of the local area of ​​the fused feature map, and its expression is:

[0097] ;

[0098] in, represents the j-th object token, Represents the average gray value of the i-th local area feature in the fusion feature map, Represents the weighting coefficient, which represents right In this embodiment, the weighting coefficient is 1 / 16, and m represents the number of local area features.

[0099] The initialized object token is the output of hierarchical object tokenization.

[0100] A multi-level deformation decoder (MTD) is used to gradually optimize the hierarchical object tokens and their corresponding features, and convert the hierarchical object tokens into segmentation result maps and ribbon position information.

[0101] The multi-stage deformation decoder includes a Flatten and position encoding layer, a cross attention layer, a feedforward neural network layer, a residual connection and a normalization layer and a recovery layer connected in sequence. The data processing process specifically includes:

[0102] The input is the fused feature map and object token. The Flatten and position encoding layers flatten the fused feature map into a 128×128 two-dimensional sequence. Next, the cross-attention layer is used to capture the dependencies in the time series, where the object token is used as the query matrix and the fused feature map is used as the key matrix and value matrix. The calculation principle is the same as the attention mechanism calculation above, so I will not go into details here. The cross-attention layer combines the object token information and the fused feature map to generate a new feature map. The feedforward neural network layer is used to enhance the nonlinear mapping ability of the model, and then the residual connection and normalization layer are used to ensure the stability of the information flow and avoid the gradient disappearance problem. The cross-attention layer, feedforward neural network layer, residual connection and normalization layer are repeated three times, and the feature map representation is further optimized through multi-layer stacking. Finally, the new feature map is upsampled using bisexual interpolation through the recovery layer to restore it to the size of the original image, that is, 256×256×3, and the upsampled feature map is converted into a single-channel segmentation mask. The model input image and the segmentation mask are operated pixel by pixel, and the segmentation result map can be obtained, which is the final output of the DS-Transformer image segmentation model.

[0103] The constructed DS-Transformer image segmentation model needs to be pre-trained. The pre-training methods include:

[0104] A column chromatography real-time process image dataset is obtained. To increase the diversity of the data and help the model generalize better, the image data can be randomly translated, rotated, and flipped.

[0105] The column chromatography real-time process image dataset is divided into training set, validation set and test set according to 7:2:1.

[0106] After initializing the model parameters, the training set data is used as input, forward propagated, and the segmentation result image is output by the DS-Transformer image segmentation model. During the training process, the hybrid loss function is calculated, and its expression is:

[0107] ;

[0108] in, represents the mixing loss, represents the cross entropy loss, represents the Dice loss, , To adjust the hyperparameters of the loss weights of each part, , The initialization values ​​are 0.7 and 0.3 respectively.

[0109] Among them, the cross entropy loss function is used to measure the classification accuracy of each pixel, and the calculation formula is as follows:

[0110] ;

[0111] in, represents the cross entropy loss, Represents the number of image pixels, represents the true value, Represents the model prediction value.

[0112] The Dice loss function is used to emphasize the data with imbalanced classification, especially the segmentation of small objects. The calculation formula is as follows:

[0113] ;

[0114] in, represents the Dice loss, Represents the number of image pixels, represents the true value, Represents the model prediction value.

[0115] Then the validation set data is taken as input and the DS-Transformer image segmentation model is used for back propagation and gradient update.

[0116] Finally, the test set data is used as input to evaluate the model performance. The evaluation indicators include:

[0117] Accuracy refers to the ratio of correctly classified pixels to the total number of pixels, and its expression is:

[0118] ;

[0119] in, Indicates the accuracy, represents the true number of pixels of the i-th category, represents the number of false positive pixels of the i-th class, represents the number of false negative pixels of the i-th category, Represents the total number of categories.

[0120] The average intersection-over-union ratio is an indicator used to measure the degree of overlap between the predicted results and the true results. Its expression is:

[0121] ;

[0122] ;

[0123] in, represents the average intersection-over-union ratio, represents the prediction area of ​​the i-th category, represents the true area of ​​the i-th class, Represents the total number of categories.

[0124] The average precision is expressed as:

[0125] ;

[0126] in, represents the average precision, represents the average precision of the i-th class, Represents the total number of categories.

[0127] The calculated evaluation index is compared with the preset threshold. If the calculated evaluation index is less than the preset threshold, the training is repeated until the calculated evaluation index reaches the preset threshold, and a pre-trained DS-Transformer image segmentation model is obtained.

[0128] In the implementation stage, the preprocessed column chromatography real-time process image is input into the pre-trained DS-Transformer image segmentation model, and the segmentation result map is output after being processed by the model.

[0129] The centroid position of the color band is calculated based on the segmentation result image. The centroid position of the color band is the color band position, and its calculation expression is:

[0130] ;

[0131] in, The horizontal coordinate representing the centroid position of the color band, The ordinate represents the centroid position of the ribbon. The coordinates on the segmentation result map are The pixel value at , the pixel value is 0 or 1, W represents the width of the segmentation result image, and H represents the height of the segmentation result image.

[0132] The vertical coordinate of the ribbon position is compared with the preset judgment threshold. If the vertical coordinate of the ribbon position is less than the preset judgment threshold, a control instruction for replacing the beaker is generated, otherwise a control instruction for not replacing the beaker is generated. The robotic arm receives the control instruction, parses it, and completes the corresponding control operation. In this embodiment, the control instruction is sent to the robotic arm through the TCP / IP communication interface.

[0133] This example focuses on the realization of the automatic preparation process of spirofluorene xanthene (SFX), which will be used as part of the SFX preparation automation solution to replace human scientists in terms of vision and touch. SFX plays an important role in the fields of organic electroluminescence, photovoltaic cells, organic electrical storage, organic nonlinear optics, chemical and biological sensing, and organic lasers. The realization of its automated preparation process can free chemists from highly repetitive chemical operations, reduce yield losses caused by relying on experience, and greatly reduce the risks of chemical experiments.

[0134] Embodiment 2:

[0135] Based on Example 1, this example provides a column chromatography automated control and analysis system based on DS-Transformer image segmentation, including a controller, a camera, a column chromatograph and a robotic arm. For ease of operation, the camera and the robotic arm are respectively arranged on both sides of the column chromatograph.

[0136] The column chromatograph is used to perform column chromatography experiments, and the camera is used to photograph the column chromatography to obtain a real-time process image of the column chromatography and transmit it to the controller. In this embodiment, the camera is composed of a high-resolution camera and supporting optical equipment, and is installed in a position where the chromatographic column can be fully monitored to ensure that every detail of the chromatographic process can be clearly captured.

[0137] The controller is used to obtain the real-time process image of the column chromatography and generate a control instruction on whether to replace the receiving container, and transmit the control instruction to the robotic arm, which is used to complete the corresponding control operation according to the control instruction.

[0138] The controller specifically includes:

[0139] An image acquisition module is configured to acquire a real-time process image of the column chromatography;

[0140] An image preprocessing module is configured to preprocess the column chromatography real-time process image to obtain a preprocessed column chromatography real-time process image;

[0141] An image analysis module is configured to take the pre-processed column chromatography real-time process image as input and obtain a segmentation result image using the DS-Transformer image segmentation model output;

[0142] A color band position determination module is configured to calculate the color band position coordinates according to the segmentation result image;

[0143] The instruction generation module is configured to generate control instructions according to the position of the ribbon and transmit the control instructions to the robot arm.

[0144] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A column chromatography automated control and analysis method based on DS-Transformer image segmentation, characterized in that: include: Get real-time process images of column chromatography; Preprocessing the column chromatography real-time process image to obtain a preprocessed column chromatography real-time process image; The pre-processed column chromatography real-time process image is used as input, and a segmentation result image is obtained based on the output of a DS-Transformer image segmentation model, wherein the DS-Transformer image segmentation model includes a dynamic feature extractor, a feature pyramid network module, and a hierarchical object token connected in sequence, and the output ends of the feature pyramid network module and the hierarchical object token are both connected to the input end of a multi-level deformation decoder; The dynamic feature extractor is used to extract features from the input image to obtain a high-level scale feature map and a low-level scale feature map; The feature pyramid network module is used to fuse the high-level scale feature map and the low-level scale feature map to obtain a fused feature map; The hierarchical object token is used to represent the local area features in the fused feature map; The multi-stage deformation decoder is used to gradually optimize the hierarchical object tokens and their corresponding local area features, and convert the hierarchical object tokens into a segmentation result map; The color band position coordinates are calculated based on the segmentation result map, including: The coordinates of the centroid of the color band are calculated based on the segmentation result map, and the expression is: ； in, The horizontal coordinate representing the centroid position of the color band, The ordinate represents the centroid position of the ribbon. The coordinates on the segmentation result map are The pixel value at , W represents the width of the segmentation result image, and H represents the height of the segmentation result image; The color ribbon centroid position coordinates are the color ribbon position coordinates; The ordinate of the ribbon position is compared with a preset judgment threshold, and if the ordinate of the ribbon position is less than the preset judgment threshold, a beaker replacement control instruction is generated, otherwise a beaker replacement control instruction is generated; The control instructions are transmitted to the robotic arm, and the robotic arm completes the corresponding control operations according to the control instructions.

2. The column chromatography automated control and analysis method based on DS-Transformer image segmentation according to claim 1, characterized in that: The preprocessing of the column chromatography real-time process image to obtain the preprocessed column chromatography real-time process image comprises: The column chromatography real-time process image is bilaterally filtered and denoised, and the calculation expression is: ； in, represents the bilateral filtering position coordinates, Represents the pixel position coordinates of the field, Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The Euclidean distance of represents the standard deviation of the spatial distance of locations within the neighborhood, Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The absolute value of the difference, Represents the bilateral filter position coordinates The gray value of Represents the neighborhood pixel position coordinates The gray value of represents the standard deviation of the gray value in the neighborhood. Represents the neighborhood pixel position coordinates With bilateral filtering position coordinates The weight coefficient of Represents the bilateral filter position coordinates The filtering result.

3. The column chromatography automated control and analysis method based on DS-Transformer image segmentation according to claim 1, characterized in that: The dynamic feature extractor includes two processing branches, wherein one processing branch is used to output a high-level scale feature map, which includes an upsampling module, a gating module and a dynamic convolution module connected in sequence; the other processing branch is used to output a low-level scale feature map, which includes a gating module and a dynamic convolution module connected in sequence; The upsampling module is used to perform an upsampling operation on the input image to obtain an upsampled image. The gating module includes a global pooling layer, a fully connected layer, an activation layer and a fully connected layer connected in sequence. The gating module is used to process the input image to generate a dynamic control signal. The dynamic convolution module is used to select a convolution kernel according to the dynamic control signal to perform feature extraction on the input image to obtain a multi-scale dynamic feature map, and the multi-scale dynamic feature map includes a high-level scale feature map and a low-level scale feature map.

4. The column chromatography automated control and analysis method based on DS-Transformer image segmentation according to claim 1, characterized in that: The data processing process of the feature pyramid network module includes: Expand the high-level scale feature map and the low-level scale feature map to obtain a high-level feature sequence and a low-level feature sequence respectively; The relationship weight matrix between high-level feature sequences and low-level feature sequences is calculated based on the attention mechanism, and its expression is: ； in, represents the relationship weight matrix, represents the query matrix, which uses a low-level feature sequence, represents the key matrix, represents the value matrix, , All use advanced feature sequences; The relationship weight matrix is ​​multiplied by the low-level feature sequence to obtain the weighted low-level feature sequence, and the weighted low-level feature sequence and the high-level feature sequence are concatenated and deformed to obtain the fused feature map, which is expressed as: ； in, represents the fused feature map, Represents a deformation operation, represents the relationship weight matrix, represents a low-level feature sequence, Represents a high-level feature sequence.

5. The column chromatography automated control and analysis method based on DS-Transformer image segmentation according to claim 1, characterized in that: The data processing process of the hierarchical object token includes: Randomly generate multiple learnable query vectors; The query vector is matched with the local area features of the fused feature map respectively to generate an object token corresponding to the query vector, wherein the matching is achieved by calculating the similarity between the query vector and the local area features of the fused feature map, and its expression is: ； in, represents the query vector, Represents the local area features in the fusion feature map, represents the L2 norm of the query vector, represents the L2 norm of the local area features of the fused feature map, n represents the number of query vectors, represents the i-th query vector, represents the i-th local area feature in the fusion feature map, m represents the number of local area features, Indicates similarity; The object token is initialized and then outputted. The initialization is to perform an average linear combination of the weighted features of the local area of ​​the fusion feature map, and its expression is: ； in, represents the j-th object token, Represents the average gray value of the i-th local area feature in the fusion feature map, represents the weighting coefficient, and m represents the number of local area features.

6. The column chromatography automated control and analysis method based on DS-Transformer image segmentation according to claim 1, characterized in that: The multi-stage deformation decoder includes a Flatten and position encoding layer, a cross attention layer, a feedforward neural network layer, a residual connection and a normalization layer and a recovery layer connected in sequence. The Flatten and position encoding layer is used to flatten the fused feature map into a two-dimensional sequence. The cross attention layer is used to obtain the dependency relationship in the two-dimensional sequence according to the attention mechanism, and combine the object token with the two-dimensional sequence to generate a feature map. The feedforward neural network layer is used to enhance the nonlinear mapping capability. The residual connection and normalization layer are used to ensure the stability of the information flow to avoid gradient disappearance. The recovery layer is used to upsample the feature map using a bilinear difference and convert it into a single-channel segmentation mask, and perform a pixel-by-pixel AND operation on the model input image and the single-channel segmentation mask to obtain a segmentation result map.

7. A column chromatography automated control and analysis system based on DS-Transformer image segmentation, characterized in that: It is used to implement the column chromatography automated control and analysis method based on DS-Transformer image segmentation as described in any one of claims 1 to 6, and specifically comprises a controller, a camera, a column chromatograph and a robotic arm, wherein the camera and the robotic arm are respectively arranged on both sides of the column chromatograph; The column chromatograph is used to perform column chromatography experiments, the camera is used to photograph the column chromatography to obtain a real-time process image of the column chromatography, and transmit it to the controller, the controller is used to obtain the real-time process image of the column chromatography and generate a control instruction on whether to replace the receiving container, and transmit the control instruction to the robotic arm, and the robotic arm is used to complete the corresponding control operation according to the control instruction; Wherein, the controller comprises: An image acquisition module is configured to acquire a real-time process image of the column chromatography; An image preprocessing module is configured to preprocess the column chromatography real-time process image to obtain a preprocessed column chromatography real-time process image; An image analysis module is configured to take the pre-processed column chromatography real-time process image as input and obtain a segmentation result image using the DS-Transformer image segmentation model output; A color band position determination module is configured to calculate the color band position coordinates according to the segmentation result image; The instruction generation module is configured to generate control instructions according to the position of the ribbon and transmit the control instructions to the robot arm.

Images