Molecular property prediction method and system based on multi-scale image information extraction

Abstract

The application discloses a kind of molecular attribute prediction method and system based on multiscale image information extraction, comprising: constructing U type model architecture, encoder and decoder include multiple alternating cascaded convolution module and multiscale selective scanning module;The convolution module of both one-to-one correspondence, multiscale selective scanning module one-to-one correspondence, and the jump connection of corresponding module is realized through multiscale information fusion module, and the trained model is obtained after training.This scheme model takes multiscale selective scanning module as core, combines multiscale information fusion module, solves the long-range dependence modeling deficiency and global topological structure capture bottleneck problem caused by the local convolution of traditional CNN model;Multiscale selective scanning module is through 2D scanning operator and multi-branch feature interaction fusion mechanism, fully excavates the cross-space, cross-scale correlation information in molecular image, forms global feature expression while effectively avoids excessive smoothing of local structure information.

Description

Technical Field

[0001] This invention belongs to the field of drug screening technology, and in particular relates to a method and system for predicting molecular attributes based on multi-scale image information extraction. Background Technology

[0002] In the process of drug development, continuous innovation in science and technology has always been a crucial driving force for the discovery and optimization of novel therapeutic compounds. Among these advancements, the accurate prediction of molecular properties is at the heart of this process, determining whether researchers can quickly screen molecular structures with potential pharmacological efficacy and physicochemical stability from a vast pool of candidate molecules. Efficient and accurate assessment of molecular properties can not only significantly shorten the drug development cycle but also play a key role in reducing experimental costs and improving the success rate of preclinical studies.

[0003] Traditional methods for predicting molecular properties mostly rely on machine learning models. These methods typically train the models by manually designing and extracting features, such as counting the number of functional groups in molecules and calculating hydrophobicity parameters. Although this approach achieved some success in the early days, its heavy reliance on human experience and feature engineering makes it significantly limited in large-scale, complex molecular prediction scenarios, and it is difficult to meet today's demands for high accuracy and automation.

[0004] With the rapid development of deep learning, computer vision technology has gradually penetrated the fields of molecular modeling and drug discovery. Researchers typically employ graph-based molecular modeling, mapping molecular structures onto neural networks for feature extraction. Currently, common molecular image learning methods primarily rely on convolutional neural networks (CNNs). CNN models can effectively capture local neighborhood structural information of molecules and have a strong ability to model local interactions between atoms. However, due to the locality of convolution operations, these models have significant limitations in representing long-range dependencies and global topological structures within molecules. Even by increasing the convolution kernel size or adding more network layers to expand the receptive field, the model still faces significant bottlenecks in capturing long-distance interactions across atoms. Furthermore, existing CNN models also have significant shortcomings in multi-scale feature extraction. While the hierarchical structure of traditional CNNs can achieve a certain degree of aggregation from low-level local features to high-level abstract features, its inter-scale feature fusion capability is limited, easily leading to over-smoothing of local structural information or loss of global features. Summary of the Invention

[0005] The purpose of this invention is to propose a molecular attribute prediction method and system based on multi-scale image information extraction to address the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions:

[0007] A molecular attribute prediction method based on multi-scale image information extraction, the method comprising:

[0008] A U-shaped model architecture is constructed, which includes an encoder and a decoder. The encoder includes multiple alternating convolutional modules and a multi-scale selective scanning module, and the decoder includes multiple alternating convolutional modules and a multi-scale selective scanning module.

[0009] The convolution module and multi-scale selective scanning module of the decoder correspond one-to-one with the convolution module and multi-scale selective scanning module of the encoder, respectively, and the corresponding modules of the encoder and decoder are connected by a multi-scale information fusion module.

[0010] The constructed U-shaped model architecture is trained to obtain the trained model, which is used to predict the output attributes of the molecular graph to be predicted.

[0011] In the above-mentioned molecular attribute prediction method based on multi-scale image information extraction, the encoder, from input to output, starts with a convolution module and forms a structure in which convolution modules and multi-scale selective scanning modules are alternately cascaded.

[0012] The decoder, from input to output, starts with a multi-scale selective scanning module and consists of alternating cascaded multi-scale selective scanning modules and convolution modules.

[0013] In the above-mentioned molecular attribute prediction method based on multi-scale image information extraction, the output of the last multi-scale selective scanning module of the encoder is connected to a convolution module, and after being processed by the convolution module, it is output to the first multi-scale selective scanning module of the decoder.

[0014] The decoder's convolutional modules from output to input correspond one-to-one with the encoder's convolutional modules from input to output, and skip connections are achieved through a multi-scale information fusion module.

[0015] The decoder's multi-scale selective scanning modules from output to input correspond one-to-one with the encoder's multi-scale selective scanning modules from input to output, and are connected by a multi-scale information fusion module.

[0016] In the aforementioned molecular attribute prediction method based on multi-scale image information extraction, the multi-scale selective scanning module is used to input features into the multi-scale selective scanning module. Mapped to a feature representation with twice the number of channels ;

[0017] Features with half the number of channels A 2D scanning operator is fed in for directional expansion and feature extraction, while the other half of the channel numbers are used for feature extraction. The fused feature representation is obtained by performing a Hadamard product operation on the output of the 2D scanning operator, and finally output through a normalization layer and a linear layer.

[0018] In the aforementioned molecular attribute prediction method based on multi-scale image information extraction, the multi-scale selective scanning module includes the following components from input to output:

[0019] The bottom normalization layer and the first linear layer are input to the features of the multi-scale selective scanning module. After processing by the bottom normalization layer and the first linear layer, it is mapped to a feature representation with twice the number of channels. ;

[0020] The first 2D scan operator and the second 2D scan operator are located on two parallel branches, and the second 2D scan operator is input features. ,feature After performing a Hadamard product operation with the output of the second 2D scan operator, the result is input into the first 2D scan operator;

[0021] The third 2D scan operator takes the output of the second 2D scan operator as input;

[0022] The first normalization layer and the second linear layer are used to process the output of the first 2D scan operator in sequence.

[0023] The fourth 2D scan operator is used to fuse the features of the two branches. The input is the result of performing a Hadamard product operation on the output of the third 2D scan operator and the output of the second linear layer.

[0024] The second normalization layer and the third linear layer are used to process the output of the fourth 2D scan operator in sequence.

[0025] In the above-mentioned molecular attribute prediction method based on multi-scale image information extraction, the execution process of each 2D scanning operator includes:

[0026] First, a scan extension is performed, which includes the scan sequence from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right.

[0027] After the scan expansion is completed, the sequence expanded in each direction is input into the Long Short-Term Memory network for feature extraction.

[0028] After feature extraction is completed, a scan merging operation is performed to align and fuse the feature sequences output in the four directions, and restore them to the initial size consistent with the input image through rearrangement.

[0029] In the above-mentioned molecular attribute prediction method based on multi-scale image information extraction, the multi-scale information fusion module includes multiple pooling modules, each pooling module corresponding to a set of convolution modules / multi-scale selective scanning modules of encoder and decoder.

[0030] Each pooling module includes three parallel processing channels: max pooling, average pooling, and summation pooling.

[0031] Each pooling module takes the output of the corresponding encoder convolution module / multi-scale selective scanning module as input, and outputs three-channel features after three-channel pooling processing.

[0032] The three-channel features are concatenated and then input into a two-dimensional shared convolution module to generate spatial attention maps corresponding to the convolution module / multi-scale selective scanning module of the encoder.

[0033] The output of the convolution module / multi-scale selective scanning module of the decoder is summed with the corresponding spatial attention map and then input to the next level.

[0034] In the above-mentioned molecular attribute prediction method based on multi-scale image information extraction, the encoder consists of a convolution module, a multi-scale selective scanning module, a convolution module, a multi-scale selective scanning module, and a convolution module in sequence from input to output.

[0035] The decoder consists of a multi-scale selective scanning module, a convolution module, another multi-scale selective scanning module, and another convolution module, sequentially from input to output.

[0036] Furthermore, the convolution module is a two-dimensional convolution.

[0037] In the above-mentioned molecular attribute prediction method based on multi-scale image information extraction, the constructed U-shaped model architecture is trained in two stages, including a pre-training stage and an attribute prediction stage.

[0038] The loss function in the pre-training phase The expression is as follows:

[0039]

[0040] in, Indicates input features, Indicates the encoding process, Indicates the decoding process. Represents the L2 norm;

[0041] The loss function in the attribute prediction stage The expression is as follows:

[0042]

[0043] in, This represents the total number of molecules in the training set. Indicates the first The true fraction of each molecule, Indicates the first The predicted score of each molecule.

[0044] A molecular attribute prediction system based on multi-scale image information extraction includes a controller and a memory. The memory stores a trained model constructed and trained by the method described above. The controller is used to execute the trained model to output attribute prediction results for the molecular image to be predicted.

[0045] The advantages of this invention are:

[0046] The U-shaped model architecture constructed in this solution, with a multi-scale selective scanning module as the core and combined with a multi-scale information fusion module, solves the problems of insufficient long-range dependency modeling and global topology capture bottleneck caused by convolutional locality in traditional CNN models.

[0047] The multi-scale selective scanning module, through 2D scanning operators and multi-branch feature interaction fusion mechanism, can fully explore cross-spatial and cross-scale correlation information in molecular images, forming a more robust global feature representation, while effectively avoiding excessive smoothing of local structural information.

[0048] The multi-scale information fusion module achieves efficient fusion of multi-scale features through a combination of parallel processing using various pooling methods and shared convolution. Combined with a one-to-one skip connection mechanism, it not only fully recovers spatial feature details during the decoding stage but also accelerates the convergence speed of model training and enhances the model's sensitivity to molecular structural features at different scales. Attached Figure Description

[0049] Figure 1 This is a diagram of the U-shaped model architecture in the molecular attribute prediction method based on multi-scale image information extraction according to an embodiment of the present invention.

[0050] Figure 2 This is an architecture diagram of the multi-scale selective scanning module in the molecular attribute prediction method based on multi-scale image information extraction according to an embodiment of the present invention;

[0051] Figure 3 This is a model architecture diagram of the final trained model in the molecular attribute prediction method based on multi-scale image information extraction in an embodiment of the present invention.

[0052] Figure 4 This is a comparison result diagram of an embodiment of the present invention.

[0053] Figure labels: Encoder 1; Decoder 2; Multi-scale information fusion module 3; Multi-scale selective scanning module 4; Two-dimensional shared convolution module 5. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] like Figure 1 As shown, this embodiment of the molecular attribute prediction method based on multi-scale image information extraction includes the following steps:

[0056] S1: Construct a U-shaped model architecture, including encoder 1, decoder 2, and the jump connection path between encoder 1 and decoder 2. The jump connection path between encoder 1 and decoder 2 is output after passing through the multi-scale information fusion module 3.

[0057] In this embodiment, the encoder 1 includes two multi-scale selective scanning modules 4 connected in sequence, and a convolution module is provided in front of each multi-scale selective scanning module 4—two-dimensional (2D) convolution constitutes an alternating cascaded structure of the two.

[0058] In another embodiment, more convolutional modules and multi-scale selective scanning modules 4 can be cascaded in alternating stages.

[0059] The output of the last multi-scale selective scanning module 4 in encoder 1 is subjected to a 2D convolution and then input into decoder 2.

[0060] The decoder 2 includes two multi-scale selective scanning modules 4 connected in sequence, and a convolution module—2D convolution—is provided after each multi-scale selective scanning module 4. Similarly, in another embodiment, more convolution modules and multi-scale selective scanning modules 4 can be provided in alternating cascaded stages.

[0061] In another embodiment, the convolution module may also use a one-dimensional or three-dimensional convolution structure.

[0062] The multi-scale selective scanning module 4 and 2D convolution in decoder 2 correspond one-to-one with the multi-scale selective scanning module 4 and 2D convolution in encoder 1. Specifically, as follows... Figure 1 As shown:

[0063] The multi-scale selective scanning module 4 on the left side of encoder 1 (i.e., the first multi-scale selective scanning module 4 from input to output) corresponds to the multi-scale selective scanning module 4 on the left side of decoder 2 (i.e., the last multi-scale selective scanning module 4 from input to output);

[0064] The multi-scale selective scanning module 4 on the right side of encoder 1 (i.e., the last multi-scale selective scanning module 4 from input to output) corresponds to the multi-scale selective scanning module 4 on the right side of decoder 2 (i.e., the first multi-scale selective scanning module 4 from input to output);

[0065] The encoder 1 is set to a 2D convolution before the left multi-scale selective scanning module 4, and the decoder 2 is set to a 2D convolution after the left multi-scale selective scanning module 4.

[0066] The 2D convolution set in encoder 1 before the right multi-scale selective scanning module 4 corresponds to the 2D convolution set in decoder 2 after the right multi-scale selective scanning module 4.

[0067] Furthermore, the multi-scale selective scanning modules 4 / 2D convolutions corresponding to encoder 1 and decoder 2 are respectively implemented through the multi-scale information fusion module 3 to achieve skip connections.

[0068] The model consists of encoder 1, decoder 2, and skip connection paths, with a total depth of four layers. Encoder 1 and decoder 2 are equipped with multi-scale selective scanning modules 4 and standard convolution operations with a kernel size of 3 in each layer.

[0069] like Figure 2 As shown, the multi-scale selective scanning module 4 includes multiple normalization layers, linear layers, and two-dimensional (2D) scanning operators, forming a multi-branch feature processing and feature fusion structure. Specifically, its structure from input to output includes:

[0070] The bottom-level preprocessing includes a bottom-level normalization layer and a first linear layer. The output of the bottom-level normalization layer is connected to the input of the first linear layer to receive external input features, complete feature normalization, and map the input features to a representation with twice the number of channels.

[0071] Parallel feature processing, connected to the output of the first linear layer, includes two parallel branches:

[0072] The first branch consists of, in sequence, the first 2D scan operator, the first normalization layer, and the second linear layer;

[0073] The second branch contains a second 2D scan operator and a third 2D scan operator connected in sequence. The input of the second 2D scan operator is connected to the output of the first linear layer. The second 2D pooling module scan operator has two outputs: one is input to the third 2D scan operator, and the other is multiplied with the output of the first linear layer and then input to the first 2D scan operator of the first branch.

[0074] Feature fusion is achieved by multiplying the output of the second linear layer of the first branch with the output of the third 2D scan operator of the second branch element by element to achieve feature fusion between the two branches.

[0075] The top-level output consists of the fourth 2D scanning operator, the second normalization layer, and the third linear layer, which receive the features after feature fusion and process them sequentially. Finally, the processed features are output by the third linear layer.

[0076] The multi-scale selective scanning module 4 employs multiple 2D scanning operators to avoid losing global information before spatial interaction operations, and further introduces a linear layer at the output to enhance the learning ability of complex feature patterns. Specifically, in the selective scanning of each layer, the input features are first processed through a linear projection layer. Mapped to a representation with twice the number of channels. Subsequently, half the number of channels were used as features. The second 2D scanning operator is fed in for directional expansion and feature extraction, and the features of the other half of the channels are extracted. The fused feature representation is obtained by performing a Hadamard product operation with the output of the second 2D scan operator. Finally, the output is completed through a normalization layer and a linear layer.

[0077] The operation process is represented by the following formula:

[0078]

[0079]

[0080] in, Indicates the normalization layer. Indicates a linear layer. This represents the Hadamard product operation. This represents the 2D scan operator. Represents input features;

[0081] The operation modes of each 2D scanning operator in the multi-scale selective scanning module 4 are as follows:

[0082] First, the input molecular image is unfolded in a two-dimensional plane along different directions. Specifically, the scanning unfolding includes the following four directions in sequence: scanning from top left to bottom right, scanning from bottom right to top left, scanning from top right to bottom left, and scanning from bottom left to top right. Through these four different directional unfoldings, feature information in the input image under different spatial distributions can be fully captured.

[0083] After the scan expansion is completed, the sequence expanded in each direction is input into a Long Short-Term Memory (LSTM) network for feature extraction, so as to achieve multi-scale and multi-channel feature capture in the local spatial range, thereby improving the overall representation capability. By extracting features independently in the four scan directions, complementary directional information can be obtained.

[0084] After feature extraction, a scan-merge operation is performed. This operation aligns and fuses the feature sequences output from the four directions, and restores them to their initial size consistent with the input image through rearrangement. This scan-merge mechanism can fuse features from multiple directions while maintaining spatial consistency, thereby forming a more robust global representation.

[0085] The multi-scale information fusion module 3 is used to transfer multi-level feature information between encoder 1 and decoder 2. It achieves the step-by-step fusion of multi-scale features by introducing a pooling module and a two-dimensional shared convolution module 5. By utilizing the above-mentioned skip connection mechanism, not only can spatial features be fully recovered in the decoding stage, but the convergence speed of the overall model can also be accelerated, and the sensitivity to features at different scales can be improved, thereby enhancing the model's performance in molecular image attribute prediction.

[0086] Specifically, the multi-scale information fusion module 3 uses max pooling, average pooling, and summative pooling to perform parallel downsampling of the feature map, and then stitches together the features from the three channels of max pooling, average pooling, and summative pooling. Subsequently, a spatial attention map is generated using a two-dimensional convolution-two-dimensional shared convolution module 5 with shared weights to capture the importance distribution of features at different spatial locations.

[0087] Specifically, the multi-scale information fusion module 3 downsamples and stitches together the outputs of the two multi-scale selective scanning modules 4 of encoder 1 and the 2D convolution in front of them to obtain four stitched results. The four stitched results are input to the two-dimensional shared convolution module 5. The two-dimensional shared convolution module 5 processes the four stitched results to obtain four fusion features—spatial attention maps. The four fusion features are summed with the outputs of the corresponding 2D convolution or multi-scale selective scanning modules 4 of decoder 2 and then input to the next level.

[0088] Specifically, this embodiment Figure 1In the encoder 1, the output of the multi-scale selective scanning module 4 on the left is processed by downsampling, stitching, and 2D shared convolution module 5 to obtain the fused feature. This fused feature is then summed with the output of the corresponding multi-scale selective scanning module 4 on the left of the decoder 2 and input to the next level, i.e., input to the 2D convolution on the left of the decoder 2. Similarly, the output of the multi-scale selective scanning module 4 on the right of the encoder 1 is processed by downsampling, stitching, and 2D shared convolution module 5 to obtain the fused feature. This fused feature is then summed with the output of the corresponding multi-scale selective scanning module 4 on the right of the decoder 2 and input to the next level, i.e., input to the 2D convolution on the right of the decoder 2. Finally, the output of the 2D convolution in the middle of the encoder 1 is processed by downsampling, stitching, and 2D shared convolution module 5 to obtain the fused feature. This fused feature is then summed with the output of the corresponding 2D convolution on the right of the decoder 2 and input to the next level, i.e., input to the multi-scale selective scanning module 4 on the left of the decoder 2. The fused features obtained by downsampling, stitching and splicing and two-dimensional shared convolution module 5 of the leftmost 2D convolution of encoder 1 are summed with the output of the corresponding leftmost 2D convolution of decoder 2 to output a molecular map.

[0089] During the fusion process of decoder 2, the design of the multi-scale information fusion module 3 in this scheme can effectively improve the richness and diversity of information at different scales, and enhance the model's discrimination ability and generalization performance.

[0090] S2: Perform two-stage training on the U-shaped model architecture constructed in S1 to obtain the trained model.

[0091] Model training: The model training method proposed in this invention includes two stages: a pre-training stage and an attribute prediction stage. The two stages are combined to achieve efficient feature modeling of molecular image data and accurate prediction of molecular attributes.

[0092] In the pre-training phase, the training objective is to enable the U-shaped model architecture to reconstruct molecular graph features. This embodiment randomly selects approximately one million molecules from the PubChem database as raw samples to construct the training dataset. This dataset is not an ideal synthetic image; it contains noise and various rendering styles, thus more closely resembling the data distribution required by the model in real-world applications, which helps improve the model's generalization ability. In this phase, the input image is first mapped to the latent feature space by encoder 1, and then restored to an approximate molecular graph reconstruction by decoder 2. The model's training objective is to minimize the difference between the input image—input molecular graph—and the decoded result—output molecular graph, with the loss function defined as follows:

[0093]

[0094] in, Indicates input features, Indicates the encoding process, Indicates the decoding process. Represents the L2 norm;

[0095] In the attribute prediction stage, the model is further trained using the ESOL, FreeSolv, and Lipophilicity molecular attribute prediction datasets to evaluate and improve its performance on the molecular attribute prediction task. This stage retains only the encoder 1 part, and a multilayer perceptron (MLP) is connected after the feature vector output by encoder 1 as the prediction module. The flattened features from encoder 1 are input to the MLP, ultimately outputting the predicted values ​​of the molecular attributes. Mean squared error (MSE) is used as the optimization objective during training, and its loss function is defined as follows:

[0096]

[0097] in, This represents the total number of molecules in the training set. Indicates the first The true fraction of each molecule, Indicates the first The predicted score of each molecule, and the true score, which is the label in the training set, are known to those in the art.

[0098] Finally, the trained model is obtained, such as Figure 3 As shown, the trained model can be used to predict the output of the molecular map under test.

[0099] The trained model is deployed to a prediction system, whose controller executes the trained model stored in memory to predict the output attributes of the molecular graph to be predicted.

[0100] During prediction, the molecular graph to be predicted is input into the trained model to obtain the predicted values ​​of the molecular properties of the molecular graph.

[0101] This embodiment selects D-MPNN, MolCLR-GCN and MolCLR-GIN, which are representative in the field of molecular representation, as comparison methods, and uses root mean square error (RMSE) as the evaluation index. Figure 4 Experimental results of various methods on the ESOL, FreeSolv, and Lipophilicity datasets are presented. The results show that although the performance of the proposed MolPropImg method on the Lipophilicity dataset is slightly lower than existing methods, it achieves the best performance on both the ESOL and FreeSolv datasets, demonstrating a significant advantage. These experimental results validate the effectiveness and superiority of the proposed scheme in molecular property prediction tasks.

[0102] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A molecular attribute prediction method based on multi-scale image information extraction, characterized in that, The method includes: A U-shaped model architecture is constructed, comprising an encoder (1) and a decoder (2). The encoder (1) includes multiple alternating convolutional modules and a multi-scale selective scanning module (4), and the decoder (2) includes multiple alternating convolutional modules and a multi-scale selective scanning module (4). The convolution module and multi-scale selective scanning module (4) of the decoder (2) correspond one-to-one with the convolution module and multi-scale selective scanning module (4) of the encoder (1), and the corresponding modules of the encoder (1) and decoder (2) are connected by the multi-scale information fusion module (3). The constructed U-shaped model architecture is trained to obtain the trained model, which is used to predict the output attributes of the molecular graph to be predicted.

2. The molecular attribute prediction method based on multi-scale image information extraction according to claim 1, characterized in that, The encoder (1) described above, from input to output, consists of a structure in which convolution modules and multi-scale selective scanning modules (4) are cascaded alternately, starting with the convolution module; The decoder (2) described above, from input to output, starts with the multi-scale selective scanning module (4) and forms a structure in which the multi-scale selective scanning module (4) and the convolution module are cascaded alternately.

3. The molecular attribute prediction method based on multi-scale image information extraction according to claim 2, characterized in that, The output of the last multi-scale selective scanning module (4) of the encoder (1) is connected to a convolution module, and after being processed by the convolution module, it is output to the first multi-scale selective scanning module (4) of the decoder (2). The decoder (2) has several convolutional modules from output to input that correspond one-to-one with several convolutional modules from input to output of the encoder (1), and jump connections are achieved through the multi-scale information fusion module (3). The decoder (2) has several multi-scale selective scanning modules (4) from output to input, which correspond one-to-one with the encoder (1) from input to output, and the multi-scale information fusion module (3) enables jump connections.

4. The molecular attribute prediction method based on multi-scale image information extraction according to claim 1, characterized in that, The multi-scale selective scanning module (4) is used to input features into the multi-scale selective scanning module (4). Mapped to a feature representation with twice the number of channels ; Features with half the number of channels A 2D scanning operator is fed in for directional expansion and feature extraction, while the other half of the channel numbers are used for feature extraction. The fused feature representation is obtained by performing a Hadamard product operation on the output of the 2D scanning operator, and finally output through a normalization layer and a linear layer.

5. The molecular attribute prediction method based on multi-scale image information extraction according to claim 4, characterized in that, The multi-scale selective scanning module (4) includes the following from input to output: The bottom normalization layer and the first linear layer are input to the features of the multi-scale selective scanning module (4). After processing by the bottom normalization layer and the first linear layer, it is mapped to a feature representation with twice the number of channels. ; The first 2D scan operator and the second 2D scan operator are located on two parallel branches, and the second 2D scan operator is input features. ,feature After performing a Hadamard product operation with the output of the second 2D scan operator, the result is input into the first 2D scan operator; The third 2D scan operator takes the output of the second 2D scan operator as input; The first normalization layer and the second linear layer are used to process the output of the first 2D scan operator in sequence. The fourth 2D scan operator is used to fuse the features of the two branches. The input is the result of performing a Hadamard product operation on the output of the third 2D scan operator and the output of the second linear layer. The second normalization layer and the third linear layer are used to process the output of the fourth 2D scan operator in sequence.

6. The molecular attribute prediction method based on multi-scale image information extraction according to claim 5, characterized in that, The execution process of each 2D scan operator includes: First, a scan extension is performed, which includes the scan sequence from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right. After the scan expansion is completed, the sequence expanded in each direction is input into the Long Short-Term Memory network for feature extraction. After feature extraction is completed, a scan merging operation is performed to align and fuse the feature sequences output in the four directions, and restore them to the initial size consistent with the input image through rearrangement.

7. The molecular attribute prediction method based on multi-scale image information extraction according to any one of claims 1 to 3, characterized in that, The multi-scale information fusion module (3) includes multiple pooling modules, each pooling module corresponding to a set of convolution modules / multi-scale selective scanning modules (4) of the encoder (1) and decoder (2). Each pooling module includes three parallel processing channels: max pooling, average pooling, and summation pooling. Each pooling module takes the output of the corresponding encoder (1) convolution module / multi-scale selective scanning module (4) as input, and outputs three-channel features after three-channel pooling processing; The three-channel features are spliced ​​and then input into the two-dimensional shared convolution module (5) to generate the spatial attention map corresponding to the convolution module / multi-scale selective scanning module (4) of the corresponding encoder (1); The output of the convolution module / multi-scale selective scanning module (4) of the decoder (2) is summed with the corresponding spatial attention map and then input to the next level.

8. The molecular attribute prediction method based on multi-scale image information extraction according to claim 2 or 3, characterized in that, The encoder (1) consists of a convolution module, a multi-scale selective scanning module (4), a convolution module, a multi-scale selective scanning module (4), and a convolution module in sequence from input to output; The decoder (2) consists of a multi-scale selective scanning module (4), a convolution module, a multi-scale selective scanning module (4), and a convolution module, sequentially from input to output. Furthermore, the convolution module is a two-dimensional convolution.

9. The molecular attribute prediction method based on multi-scale image information extraction according to claim 1, characterized in that, The constructed U-shaped model architecture is trained in two stages, including a pre-training stage and an attribute prediction stage. The loss function in the pre-training phase The expression is as follows: ,in, Indicates input features, Indicates the encoding process, Indicates the decoding process. Represents the L2 norm; The loss function in the attribute prediction stage The expression is as follows: ,in, This represents the total number of molecules in the training set. Indicates the first The true fraction of each molecule, Indicates the first The predicted score of each molecule.

10. A molecular attribute prediction system based on multi-scale image information extraction, comprising a controller and a memory, characterized in that, The memory stores a trained model constructed and trained by the method described in any one of claims 1 to 9, and the controller is used to execute the trained model to output attribute prediction results of the molecular graph to be predicted.

Images