IGV genetic structure variation image recognition method, device and medium based on multi-task learning

Abstract

The present application discloses a multi-task learning IGV (Integrative Genomics Viewer) genetic structure variation image recognition method, device and medium, including: obtaining an IGV image set of gene fragments according to the original structure variation detection (VCF, Variant Call Format) file; the IGV image set includes the variation length and variation type label corresponding to each gene fragment; identifying the number in the IGV image with the variation type label inserted into the variation by optical character recognition (OCR, Optical Character Recognition) technology, marking the target number; taking each IGV image as input and the variation type label corresponding to the IGV image as output, training a pre-built multi-task network model; predicting the IGV image to be classified based on the trained multi-task network model, and the IGV image to be classified is obtained according to the gene fragment in the processed VCF file. In this way, the classification accuracy of the structural variation IGV image can be improved, the true and false positive problems of the software in identifying the structural variation can be effectively solved, and the subjectivity of human screening can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of bioinformatics, and in particular to a multi-task learning IGV genetic structure variation image recognition method, device and medium. Background Art

[0002] With the development of third-generation sequencing technology, the detection level of structural variation has been continuously improved, which has provided an important foundation for analyzing the genetic background of complex diseases. At present, structural variation software focuses on the type and size of variation, and there are many false positive reports under high sensitivity. Therefore, further filtering to improve accuracy is still an important issue of current concern.

[0003] VCF-based filtering is affected by the representation method of the detection software. In real research, it is usually combined with the alignment of sequencing reads in the IGV visualized variant segment to screen for highly reliable structural variants. However, it is difficult to unify the screening criteria for manual review, and the judgment efficiency and accuracy are limited by the amount of data.

[0004] In general, current filtering methods have high limitations, and it is difficult to achieve accurate screening from targeted to whole genome, resulting in low classification accuracy of structural variation IGV images. Summary of the invention

[0005] The purpose of this application is to provide a multi-task learning IGV genetic structural variation image recognition method, device and medium, by using OCR technology to recognize the numbers in the IGV image and mark the required numbers, and classify and predict the IGV image through a multi-task network model, it can improve the classification accuracy of structural variation IGV images, effectively solve the true and false positive problems of existing software for structural variation recognition, and reduce the subjectivity of human screening.

[0006] To achieve the above objectives:

[0007] In a first aspect, the present application embodiment provides a multi-task learning IGV genetic structure variation image recognition method, comprising:

[0008] Obtain corresponding IGV image sets according to multiple gene fragments in the original VCF file; the IGV image set includes IGV images corresponding to each gene fragment and variant type labels corresponding to each IGV image;

[0009] Use OCR technology to identify the numbers in the IGV image with the mutation type label as the insertion mutation, and mark the target numbers in the IGV image;

[0010] Each IGV image is used as input, and the mutation type label corresponding to the IGV image is used as output to train the pre-built multi-task network model;

[0011] The IGV images to be classified are predicted based on the trained multi-task network model, and the IGV images to be classified are obtained according to the gene fragments in the processed VCF files.

[0012] In one embodiment, obtaining corresponding IGV image sets according to multiple gene fragments in the original VCF file includes:

[0013] The gene start position and the gene end position corresponding to the target gene fragment in the original VCF file are respectively extended by a preset percentage of the variant length of the target gene fragment to obtain a new gene start position and a new gene end position; the target gene fragment is any gene fragment in the original VCF file;

[0014] Based on the new gene start position, the new gene end position and the variation type corresponding to the target gene fragment, the IGV image corresponding to the target gene fragment is generated using the IGV software, and the variation type corresponding to the target gene fragment is determined as the variation type label corresponding to the IGV image.

[0015] In one embodiment, the method of obtaining corresponding IGV image sets according to the plurality of gene fragments in the original VCF file further includes:

[0016] After using IGV software to generate the IGV image corresponding to the target gene fragment, use vector graphics editing software to convert the IGV image format into PNG format.

[0017] In one embodiment, before identifying the variation type label as a number in the IGV image with an inserted variation by OCR technology, the method further includes:

[0018] The white pixel values ​​in the IGV image with the mutation type label of insertion mutation are converted into black pixel values, purple pixel values ​​are converted into white pixel values, and gray pixel values ​​are converted into white pixel values ​​in sequence; among which, the purple pixel value represents the insertion mutation part.

[0019] In one embodiment, marking the target number in the IGV image includes:

[0020] For any number in the IGV image, comparing whether the number is greater than or equal to a first preset value and less than or equal to a second preset value; the first preset value is less than the second preset value, and the first preset value and the second preset value are determined based on the variant length of the gene segment corresponding to the IGV image;

[0021] If so, the number is determined as a target number, and the target number area of ​​the IGV map is marked with a preset color.

[0022] In one embodiment, the multi-task network model includes a backbone network and a first task network for homozygous true and false positive classification using a hard sharing mode, a second task network for homozygous variant type classification, a third task network for heterozygous true and false positive classification, and a fourth task network for heterozygous variant type classification, the backbone network is ResNet-18, and the task-specific layers in each task network include two linear layers, three convolutional layers, three ReLU layers, and three Dropouts.

[0023] In one embodiment, the method of taking each IGV image as input and the variant type label corresponding to the IGV image as output, and training the pre-built multi-task network model includes:

[0024] Each IGV image is input into the backbone network to obtain the cross entropy loss function corresponding to each task network; the cross entropy loss function corresponding to each task network is determined based on the actual output of the task network and the variation type label corresponding to the IGV image;

[0025] According to the cross entropy loss function corresponding to each task network and the preset weights corresponding to each task network, the loss value of the multi-task network model is obtained;

[0026] When the loss value of the multi-task network model is less than a preset threshold, the training of the multi-task network model is stopped.

[0027] In one embodiment, the prediction of the IGV image to be classified based on the trained multi-task network model includes:

[0028] After pixel conversion of the IGV image to be classified, the processed IGV image to be classified is input into the trained multi-task network model to obtain the prediction result output by the trained multi-task network model.

[0029] In a second aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory storing a computer program; when the processor executes the computer program, the steps of the IGV genetic structure variation image recognition method based on multi-task learning are implemented.

[0030] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the IGV genetic structure variation image recognition method based on the above-mentioned multi-task learning is implemented.

[0031] The multi-task learning IGV genetic structure variation image recognition method, device and medium provided in the embodiments of the present application include: obtaining a corresponding IGV image set according to multiple gene fragments in an original VCF file; the IGV image set includes an IGV image corresponding to each gene fragment and a variation type label corresponding to each IGV image; identifying a number in an IGV image with a variation type label as an inserted variation through OCR technology, and marking the target number in the IGV image; taking each IGV image as input and the variation type label corresponding to the IGV image as output, training a pre-constructed multi-task network model; predicting an IGV image to be classified based on the trained multi-task network model, where the IGV image to be classified is obtained according to the gene fragments in the processed VCF file. To ensure that the IGV images truly reflect the variations in the VCF files, OCR technology is used to identify the numbers in the IGV images. These numbers can truly reflect the alignment characteristics of the sequencing reads in the region, and can infer the variation type and fragment size. At the same time, the numbers consistent with the VCF variation labels are marked, and the IGV images are classified and predicted through a multi-task network model, which can improve the classification accuracy of structural variation IGV images, effectively solve the true and false positive problems of existing software for structural variation identification, and reduce the subjectivity of manual screening.

[0032] At the same time, this application uses a deep multi-task learning method for the first time to identify IGV images in the field of bioinformatics, and obtains IGV images on a large scale from VCF files through script files and other methods, and then performs subsequent image processing, which can improve the accuracy of image classification. Secondly, this application also proposes for the first time to use OCR technology to digitally identify IGV images, and before digital recognition, pixel conversion is performed according to image features to improve the accuracy of digital recognition. After the image is digitally recognized and marked, the main focus of the image is highlighted, the feature distinction is increased, and the classification accuracy of the multi-task network model for the image is improved. In addition, parallel multi-task learning is also designed in this application for image classification, that is, a backbone network such as ResNet-18 is used to extract the common features of each task network, and then each task network outputs the results on each branch. At the same time, according to the importance of each task network, loss functions with different weights are designed to highlight the main learning tasks and further improve the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of IGV images of inserted mutations;

[0034] Figure 2 Schematic diagram of IGV image of deletion variant;

[0035] Figure 3A schematic diagram of a flow chart of an IGV genetic structure variation image recognition method based on multi-task learning provided by an embodiment of the present invention;

[0036] Figure 4 A schematic diagram of digitally identifying and marking an IGV image in an embodiment of the present invention;

[0037] Figure 5 Schematic diagram of the architecture of a multi-task network model in an embodiment of the present invention;

[0038] Figure 6 Schematic diagram of the architecture of a task network in an embodiment of the present invention;

[0039] Figure 7 Schematic diagram of the architecture of a flat multi-task network model in an embodiment of the present invention;

[0040] Figure 8 Schematic diagram of the architecture of a progressive multi-task network model in an embodiment of the present invention;

[0041] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0043] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0044] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0045] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0046] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0047] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0049] In order to facilitate understanding of the IGV genetic variation image recognition method based on multi-task learning provided in the embodiment of the present application, the true and false positives of the IGV image are briefly described below:

[0050] See also Figure 1 and Figure 2 , the IGV image is homozygous (gt11) with a mutation and an insertion mutation ( Figure 1 The black part in the middle, the variant length is the number in the black part) and the missing variant ( Figure 2 There are black straight lines on both sides of the number, and the variation length is the number in the middle of the straight line), and the missing length is len. Figure 1 The number of black entries is m, the number of variant entries is n, the number of variant entries that meet the conditions is a, and the number of variant entries that meet the conditions is calculated as follows: the number of gray reads in the figure is m, the number of reads containing variants is n, the number of reads describing variants is a, and the reads with variant lengths in the interval [len*0.9,len*1.1] are used as reads describing variants.

[0051] Among them, the principles for judging true and false positives of homozygotes are:

[0052] True positive: m>8 and a / m>0.8

[0053] False positive: m<=8 or a / m<0.8.

[0054] The principles for judging true and false positives of heterozygotes are:

[0055] True positive: m>8 and a / m>0.2

[0056] False positive: m<=8 and a / m<0.2.

[0057] See also Figure 3 , is a multi-task learning IGV genetic structure variation image recognition method provided in an embodiment of the present application. The multi-task learning IGV genetic structure variation image recognition method can be performed by a multi-task learning IGV genetic structure variation image recognition device provided in an embodiment of the present application. The multi-task learning IGV genetic structure variation image recognition device can be implemented in software and / or hardware. In this embodiment, the multi-task learning IGV genetic structure variation image recognition method is applied to electronic devices such as computers as an example. The multi-task learning IGV genetic structure variation image recognition method provided in this embodiment includes:

[0058] Step S1, obtaining a corresponding IGV image set according to multiple gene fragments in the original VCF file; the IGV image set includes an IGV image corresponding to each gene fragment and a variation type label corresponding to each IGV image.

[0059] Among them, the original VCF file may include multiple VCF sub-files, and different VCF sub-files may be based on preset gene data, corresponding to cuteSV, PBSV, Svision and other methods through genetic variation and other means. After obtaining the original VCF file, the IGV software can be used to generate IGV images corresponding to each gene fragment in the original VCF file. Among them, the original VCF file and the VCF sub-file can be specifically an excel file, and may contain fields such as chromosome number, variation position, start position, end position, variation length, quality and genotype, as shown in Table 1. In this embodiment, 20,000 IGV images generated based on the VCF sub-file generated by the cuteSV method are used as the training set for training the multi-task network model, and 1,500 IGV images generated by the PBSV and Svision methods are used as the verification set for verifying the multi-task network model.

[0060] Table 1

[0061] Chromosome number Mutation position Starting position End position Variant Length Mutation Type quality genotype chr1 2209072 2209072 2211308 -2236 DEL 916.1 1 / 1 chr1 191116172 191116172 191116240 -68 DEL 429.4 1 / 1 chr1 182065032 182064852 182065032 180 INS 353.1 0 / 1 Chr1 230329881 230329575 230329881 306 INS 334 1 / 1 chr1 235399295 235399295 235399376 -81 DEL 334 0 / 1 chr1 65321914 65321644 65321914 270 INS 324.4 1 / 1 chr1 162382275 162382184 162382275 91 INS 324.4 1 / 1 chr1 4146686 4146686 4146819 -133 DEL 314.9 1 / 1 chr1 48071774 48071597 48071774 177 INS 314.9 0 / 1 chr1 163409203 163409203 163409272 -69 DEL 314.9 1 / 1 chr1 8611633 8611633 8611784 -151 DEL 305.4 1 / 1 chr1 53165169 53164741 53165169 428 INS 305.4 1 / 1 chr1 110667528 110667358 110667528 170 1NS 305.4 1 / 1 chr1 119029335 119029335 119029650 -315 DEL 305.4 1 / 1 chr1 1542155 1541618 1542155 537 INS 295.8 1 / 1 chr1 7908379 7908323 7908379 56 INS 295.8 1 / 1 chr1 21251535 21251535 21251861 -326 DEL 295.8 1 / 1

[0062] Specifically, the gene start position and gene end position corresponding to the target gene fragment in the original VCF file are respectively extended by a preset percentage of the variation length of the target gene fragment to obtain a new gene start position and a new gene end position; the target gene fragment is any gene fragment in the original VCF file; based on the new gene start position, the new gene end position and the variation type corresponding to the target gene fragment, the IGV image corresponding to the target gene fragment is generated using the IGV software, and the variation type corresponding to the target gene fragment is determined as the variation type label corresponding to the IGV image.

[0063] Here, each line in the original VCF file can be regarded as a gene fragment. After the gene start position and gene end position corresponding to the target gene fragment in the original VCF file are respectively extended by the preset percentage of the variant length of the target gene fragment, a new gene start position and a new gene end position are obtained. For example, assuming that the variant length is len, the gene start position is start, and the gene end position is end, when using the IGV software to generate an image, the gene start position and the gene end position need to be extended, that is, the new gene start position newstart is: newstart = start-0.1*len, and the new gene end position newend is newend = end+0.1*len.

[0064] Among them, after determining the new gene start position and the new gene end position, the IGV software can be used to generate the IGV image corresponding to the target gene fragment, and the variant type corresponding to the target gene fragment is determined as the variant type label corresponding to the IGV image, and then the IGV image and the variant type label corresponding to the IGV image are added to the IGV image set. In this way, the IGV image can be obtained on a large scale from the VCF file through the script file, which can further improve the accuracy of image classification.

[0065] In one embodiment, the method of obtaining corresponding IGV image sets according to the plurality of gene fragments in the original VCF file further includes:

[0066] After using IGV software to generate the IGV image corresponding to the target gene fragment, use vector graphics editing software to convert the IGV image format into PNG format.

[0067] It can be understood that when using IGV software to generate IGV images, since the generated IGV images are in SVG format and have low resolution, the subsequent digital recognition effect through optical character recognition technology is poor. Therefore, after using IGV software to generate the IGV image corresponding to the target gene fragment, vector graphics editing software is used to convert the format of the IGV image from SVG format to PNG format, thereby improving the clarity of the image and further improving the accuracy of subsequent digital recognition.

[0068] Step S2: Recognize the numbers in the IGV image by using OCR technology, and mark the target numbers in the IGV image.

[0069] Here, OCR technology is used to identify all the numbers in the IGV image with the mutation type label of insertion mutation, and then the target numbers in the IGV image are marked to highlight the main focus of the image and increase the feature distinction. Among them, the target number can be regarded as a number useful for image classification. Among them, marking the target number in the IGV image is marking the target number in the IGV image with the mutation type label of insertion mutation.

[0070] In one embodiment, before identifying the variation type label as a number in the IGV image with an inserted variation by OCR technology, the method further includes:

[0071] The white pixel values ​​in the IGV image with the mutation type label of insertion mutation are converted into black pixel values, purple pixel values ​​are converted into white pixel values, and gray pixel values ​​are converted into white pixel values ​​in sequence; among which, the purple pixel value represents the insertion mutation part.

[0072] Here, pixel conversion can be performed on the IGV image with the mutation type label of insertion mutation to further improve the accuracy of recognizing numbers in the IGV image through OCR technology. First, the white pixel values ​​in the IGV image with the mutation type label of insertion mutation need to be converted to black pixel values, then the purple pixel values ​​need to be converted to white pixel values, and finally the gray pixel values ​​need to be converted to white pixel values. The specific BGR value conversion process is as follows:

[0073] Convert the white pixel values ​​belonging to [(220,0,150)-(255,255,255)] to black pixel values: (0,0,0);

[0074] Convert the purple pixel values ​​belonging to [(220,0,100)-(255,200,200)] to white pixel values: (255,255,255);

[0075] Convert the purple pixel values ​​that belong to [(200,200,200)-(205,205,205)] to white pixel values: (255,255,255).

[0076] In one embodiment, marking the target number in the IGV image includes:

[0077] For any number in the IGV image whose mutation type label is insertion mutation, compare whether the number is greater than or equal to a first preset value and less than or equal to a second preset value; the first preset value is less than the second preset value, and the first preset value and the second preset value are determined based on the mutation length of the gene segment corresponding to the IGV image;

[0078] If so, the number is determined as the target number, and the target number area in the IGV image whose mutation type label is insertion mutation is marked with a preset color.

[0079] Here, when using optical OCR technology to identify numbers in IGV images with variant type labels as insertion variants, the identified numbers need to be processed because numbers greater than three digits may have punctuation interference and other phenomena, so the numbers need to be further filtered, such as Figure 4 As shown. If the mutation length corresponding to the gene fragment is len, the corresponding first preset value can be set to (1-n)×len, and the second preset value can be set to (1+n)×len, that is, the number num that meets the formula (1-n)×len≤num≤(1+n)×len is selected as the target number, and the value of n is 0.1. The preset color can be set according to the actual needs, such as green, red, etc. Continue to refer to Figure 4 , Figure 4 Figure (a) shows the Figure 1The image shown is the image after pixel conversion, Figure 4 Figure (b) shows the image after the target digital area in Figure (a) is marked, and the gray part represents the marked digital area.

[0080] In this way, by selecting the target digit and marking the target digit area in the IGV image with a preset color, the main focus of the image can be highlighted, the feature distinction can be increased, and the accuracy of image classification can be further improved.

[0081] Step S3: Take each IGV image as input and the variation type label corresponding to the IGV image as output, and train the pre-built multi-task network model.

[0082] Here, after determining the structure of the multi-task network model and designing the loss function, each IGV image (including IGV images with mutation type labels as deletion mutations and IGV images with processed mutation type labels as insertion mutations, etc.) is taken as input, and the mutation type label corresponding to the IGV image is taken as output to train the pre-built multi-task network model.

[0083] Among them, the multi-task network model includes a backbone network and four task networks using a hard sharing mode (i.e., Share-Bottom mode), namely, a first task network for homozygous true and false positive classification (i.e., Task 1), a second task network for homozygous variant type classification (i.e., Task 2), a third task network for heterozygous true and false positive classification (i.e., Task 3), and a fourth task network for heterozygous variant type classification (i.e., Task 4). The backbone network can be implemented by a shared module, which can be specifically a residual neural network-18 (ResNet-18). The task-specific layers in each task network include two linear layers, three convolutional layers, three ReLU layers, and three Dropout layers, such as Figure 5 and Figure 6 As shown in the figure, the advantage of ResNet-18 is that the use of residual connections can retain the original features, making the network learning smoother and more stable.

[0084] In one embodiment, considering that the classification of variant types of the same genotype is highly correlated with the identification of true and false positive variants, this embodiment designs a multi-task network model with two structures, including: Figure 7 The tiled multi-task network model shown in Figure 8The progressive multi-task network model shown in the figure. Among them, the parameter sharing part uses ResNet-18 as the backbone network, and the last layer needs to be removed when using it. The task network is different in the two multi-task network models with different structures. In the flat multi-task network model, the task-specific layer in each task network has 5 consecutive convolutional layers, each convolutional layer is followed by a ReLU layer and Dropout, and finally there is a maximum pooling layer connecting two linear layers; in the progressive multi-task network model, every two task networks share two convolutional layers and a ReLU layer, and then each task network has two separate convolutional layers, pooling layers, and finally two linear layers.

[0085] In one embodiment, the method of taking each IGV image as input and the variant type label corresponding to the IGV image as output, and training the pre-built multi-task network model includes:

[0086] Input each IGV image into the backbone network to obtain a cross entropy loss function corresponding to each task network; the cross entropy loss function corresponding to each task network is determined based on the actual output of the task network and the variation type label corresponding to the IGV image;

[0087] According to the cross entropy loss function corresponding to each task network and the preset weights corresponding to each task network, the loss value of the multi-task network model is obtained;

[0088] When the loss value of the multi-task network model is less than a preset threshold, the training of the multi-task network model is stopped.

[0089] Here, after multiple IGV images are input into the backbone network, the backbone network extracts features from each IGV image, and each task network predicts the variation type of each IGV image based on the extracted features, and then determines the cross entropy loss function corresponding to each task network according to the actual output of the task network and the variation type label corresponding to each IGV image. Next, according to the cross entropy loss function corresponding to each task network and the preset weight corresponding to each task network, the loss value of the multi-task network model is obtained, that is, the weighted sum of the cross entropy loss function corresponding to each task network and the preset weight corresponding to each task network. Finally, it is determined whether the loss value of the multi-task network model is less than the preset threshold. If the loss value of the multi-task network model is less than the preset threshold, the training of the multi-task network model is stopped. If the loss value of the multi-task network model is greater than or equal to the preset threshold, the parameters of the backbone network and each task network are adjusted, and the adjusted multi-task network model is continued to be trained based on the IGV image.

[0090] Among them, the expression of the cross entropy loss function Loss is:

[0091]

[0092] Here, y is the expected value label output, y' is the actual label output, and n is the number of samples. Then, according to the different weights set for different task networks based on the importance of each task network, the loss value expression of the multi-task network model is obtained as follows:

[0093] Loss all =a1Loss1+β1Loss2+a2Loss3+β2Loss4

[0094] Among them, α1 and Loss1 are the weight and cross entropy loss function value of the first task network, β1 and Loss2 are the weight and cross entropy loss function value of the second task network, α2 and Loss3 are the weight and cross entropy loss function value of the third task network, and β2 and Loss4 are the weight and cross entropy loss function value of the fourth task network. Optionally, the first task network and the third task network are unified type task networks, and the second task network and the fourth task network are the same type of task networks. Here, using different weights to express the importance of the task network is more conducive to learning more important image features. In this way, by designing a parallel multi-task network for image classification, the backbone network ResNet-18 is used to extract the common features of each task network, and then each task network outputs the prediction results in each branch. At the same time, according to the importance of each task network, loss functions with different weights are designed to highlight the main learning task network and improve the classification accuracy.

[0095] Step S4: predicting the IGV image to be classified based on the trained multi-task network model, where the IGV image to be classified is obtained based on the gene fragments in the processed VCF file.

[0096] Here, in order to verify the effectiveness and generalization ability of the multi-task network model, the IGV images to be classified can be predicted based on the trained multi-task network model. Among them, the processed VCF file may include multiple VCF sub-files, and different VCF sub-files may be based on preset genetic data, corresponding to the use of PBSV, Svision and other methods. After obtaining the processed VCF file, the Integrative Genomics Viewer software can be used to generate the IGV images corresponding to each gene fragment in the processed VCF file, that is, the IGV images to be classified. In this embodiment, 1500 IGV images each generated by the PBSV and Svision methods are taken as an example to form the IGV images to be classified, that is, the verification set.

[0097] Specifically, after pixel conversion is performed on the IGV image to be classified, the processed IGV image to be classified is input into the trained multi-task network model to obtain the prediction result output by the trained multi-task network model.

[0098] Among them, pixel conversion of the IGV image to be classified includes: converting the white pixel values ​​in the IGV image to be classified into black pixel values, converting the purple pixel values ​​into white pixel values, and converting the gray pixel values ​​into white pixel values. Experiments have proved that the multi-task network model still has a certain generalization ability under the premise of ensuring the classification accuracy. At the same time, the multi-task network model is compared and verified with the single-task model, and the effectiveness of the multi-task network model is also better.

[0099] In summary, in the IGV genetic structural variation image recognition method based on multi-task learning provided in the above embodiment, considering that the numbers in the IGV image will affect the judgment of the classification result, by using optical character recognition technology to recognize the numbers in the IGV image and marking the required numbers, and classifying and predicting the IGV image through a multi-task network model, the classification accuracy of the structural variation IGV image can be improved, and the true and false positive problems of the existing software for structural variation recognition are effectively solved, and the subjectivity of human screening is reduced. At the same time, the deep learning method is used for the first time in the embodiment of the present application to identify IGV images in the field of bioinformatics, and IGV images are obtained on a large scale from VCF files through script files and the like, and then subsequent image processing is performed, which can improve the accuracy of image classification. Secondly, the embodiment of the present application also proposes for the first time to use OCR technology to digitally recognize IGV images, and before digital recognition, pixel conversion is performed according to image features, which improves the accuracy of digital recognition, and after the image is digitally recognized and marked, the main focus of the image is highlighted, the feature distinction is increased, and the classification accuracy of the multi-task network model for the image is improved. In addition, in the embodiment of the present application, parallel multi-task learning is also designed for image classification, that is, a backbone network such as ResNet-18 is used to extract the common features of each task network, and then each task network outputs the result on each branch. At the same time, according to the importance of each task network, loss functions with different weights are designed to highlight the main learning tasks and further improve the classification accuracy. Furthermore, the images obtained by multiple methods are used to verify the effectiveness and generalization ability of the above-mentioned model. The results show that the model provided in this embodiment still has a certain generalization ability while ensuring the accuracy; at the same time, the multi-task network model is compared and verified with the single-task network model, and the results prove the effectiveness of the multi-task network model.

[0100] Based on the same inventive concept as the above embodiments, the embodiments of the present invention further provide an electronic device, such as Fig. 9 As shown, the device includes: a processor 310 and a memory 311 storing a computer program; wherein, Fig. 9The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Fig. 9 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the above-mentioned multi-task learning IGV genetic structure variation image recognition method is implemented.

[0101] The device may also include: at least one network interface 312. The various components in the device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig. 9 Various buses are labeled as bus system 313.

[0102] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 311 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0103] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the device. Examples of these data include: any computer program used to operate on the device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0104] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer-readable storage medium, in which a computer program is stored. The computer-readable storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); it may also be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer-readable storage medium is executed by the processor, the above-mentioned multi-task learning IGV genetic structure variation image recognition method is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 3 The description of the illustrated embodiment will not be repeated here.

[0105] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0107] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-task learning IGV genetic structural variation image recognition method, characterized in that: include: Obtain corresponding IGV image sets according to multiple gene fragments in the original VCF file; the IGV image set includes IGV images corresponding to each gene fragment and variant type labels corresponding to each IGV image; Identify the number in the IGV image with the mutation type label as the insertion mutation by using the OCR technology, and mark the target number in the IGV image; the target number is a number greater than or equal to a first preset value and less than or equal to a second preset value; the first preset value is less than the second preset value, and the first preset value and the second preset value are determined based on the mutation length of the gene fragment corresponding to the IGV image; Each IGV image is used as input, and the mutation type label corresponding to the IGV image is used as output to train the pre-built multi-task network model; The IGV images to be classified are predicted based on the trained multi-task network model, and the IGV images to be classified are obtained according to the gene fragments in the processed VCF files.

2. The method according to claim 1, characterized in that The step of obtaining corresponding IGV image sets according to the multiple gene fragments in the original VCF file includes: The gene start position and the gene end position corresponding to the target gene fragment in the original VCF file are respectively extended by a preset percentage of the variant length of the target gene fragment to obtain a new gene start position and a new gene end position; the target gene fragment is any gene fragment in the original VCF file; Based on the new gene start position, the new gene end position and the variation type corresponding to the target gene fragment, the IGV image corresponding to the target gene fragment is generated using the IGV software, and the variation type corresponding to the target gene fragment is determined as the variation type label corresponding to the IGV image.

3. The method according to claim 2, characterized in that The step of obtaining a corresponding IGV image set according to the multiple gene fragments in the original VCF file also includes: After using IGV software to generate the IGV image corresponding to the target gene fragment, use vector graphics editing software to convert the IGV image format into PNG format.

4. The method according to claim 1, characterized in that: Before identifying the number in the IGV image with the variant type label inserted as a variant by OCR technology, the method further includes: The white pixel values ​​in the IGV image with the mutation type label of insertion mutation are converted into black pixel values, purple pixel values ​​are converted into white pixel values, and gray pixel values ​​are converted into white pixel values ​​in sequence; among which, the purple pixel value represents the insertion mutation part.

5. The method according to claim 1, characterized in that Mark the target numbers in the IGV image, including: For any number in the IGV image whose mutation type label is insertion mutation, comparing whether the number is greater than or equal to a first preset value and less than or equal to a second preset value; If so, the number is determined as the target number, and the target number area in the IGV image whose mutation type label is insertion mutation is marked with a preset color.

6. The method according to claim 1, characterized in that The multi-task network model includes a backbone network and a first task network for homozygous true and false positive classification using a hard sharing mode, a second task network for homozygous variant type classification, a third task network for heterozygous true and false positive classification, and a fourth task network for heterozygous variant type classification. The backbone network is a residual neural network-18, and the task-specific layers in each task network include two linear layers, three convolutional layers, three ReLU layers, and three Dropout layers.

7. The method according to claim 6, characterized in that The method uses each IGV image as input and the variant type label corresponding to the IGV image as output to train the pre-built multi-task network model, including: Each IGV image is input into the backbone network to obtain the cross entropy loss function corresponding to each task network; the cross entropy loss function corresponding to each task network is determined based on the actual output of each task network and the variation type label corresponding to the IGV image; According to the cross entropy loss function corresponding to each task network and the preset weights corresponding to each task network, the loss value of the multi-task network model is obtained; When the loss value of the multi-task network model is less than a preset threshold, the training of the multi-task network model is stopped.

8. The method according to claim 1, characterized in that The multi-task network model after training is used to predict the IGV image to be classified, including: After pixel conversion of the IGV image to be classified, the processed IGV image to be classified is input into the trained multi-task network model to obtain the prediction result output by the trained multi-task network model.

9. An electronic device, characterized in that: It comprises a processor and a memory storing a computer program; when the processor executes the computer program, the IGV genetic structure variation image recognition method of multi-task learning described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the IGV genetic structure variation image recognition method of multi-task learning described in any one of claims 1 to 8 is implemented.

Images