Chromosome image generation method and device for deep learning

Abstract

The invention discloses a chromosome image generation method and device for deep learning. Performing adaptive multi-scale image enhancement processing on the chromosome mid-term image to obtain an enhanced chromosome mid-term image; obtaining a chromosome karyotype arrangement diagram based on the enhanced chromosome metaphase image; generating a chromosome set based on the chromosome karyotype arrangement diagram; sequentially merging the chromosomes in the chromosome set into one image as a chromosome foreground image; fusing the chromosome foreground image with a randomly selected background image, and carrying out edge smoothing processing on the fused image to generate a random background chromosome middle-term image; wherein the random background chromosome metaphase image is used for training a target detection model established based on deep learning. According to the method, high-quality and diversified chromosome metaphase images can be generated, and when the method is used for training the deep learning model, the generalization ability and precision of the deep learning model in a target detection task are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical image processing, in particular to a chromosome image generation method and device for deep learning. BACKGROUND

[0002] Chromosome karyotype analysis is a core technology in genetic research, prenatal screening and cancer diagnosis. Deep learning methods are used to analyze chromosome metaphase images to achieve chromosome positioning, segmentation and abnormal classification, which is a common means at present.

[0003] However, the existing chromosome image processing and data generation method has many limitations. Traditional chromosome image data generation relies on manual annotation or simple copying, resulting in insufficient diversity and low quality of the data set, which limits the generalization ability and accuracy of the deep learning model. SUMMARY

[0004] In view of the above, the present application provides a chromosome image generation method and device for deep learning, which can generate high-quality and diversified chromosome metaphase images, thereby improving the generalization ability and accuracy of the deep learning model in the target detection task when training the deep learning model.

[0005] To solve the above technical problems, the technical scheme of the present application is as follows:

[0006] In one embodiment, a chromosome image generation method for deep learning is provided, the method comprising:

[0007] obtaining a chromosome metaphase image;

[0008] performing adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain an enhanced chromosome metaphase image and corresponding first annotation information;

[0009] obtaining a chromosome karyotype arrangement based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generating corresponding second annotation information;

[0010] generating a chromosome set based on the chromosome karyotype arrangement and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes;

[0011] merging the chromosomes in the chromosome set into an image one by one as a chromosome foreground image, and generating third annotation information corresponding to the chromosome foreground image;

[0012] fusing the chromosome foreground image with a randomly selected background image, and performing edge smoothing processing on the fused image to generate a random background chromosome metaphase image;

[0013] According to the random background chromosome metaphase image, and the third annotation information, the fourth annotation information corresponding to the random background chromosome metaphase image is regenerated, and is stored; wherein the random background chromosome metaphase image and the corresponding fourth annotation information are used for training a target detection model based on deep learning.

[0014] Wherein, the chromosome set generated based on the chromosome karyotype arrangement and the corresponding second annotation information includes:

[0015] According to the boundary box aspect ratio of the chromosome in the chromosome karyotype arrangement, the corresponding chromosome is sorted in ascending order, and the sorted chromosome is uniformly divided into a first half group and a second half group; the boundary box aspect ratio is determined according to the second annotation information corresponding to the chromosome;

[0016] Two or three chromosomes in the first half group are merged by a crossover operation to obtain a first type of merged chromosome;

[0017] The chromosomes in the first half group that failed to merge are divided into the second half group, and two or three chromosomes in the updated second half group are merged by a connection operation to obtain a second type of merged chromosome;

[0018] The single chromosome in the updated second half group that failed to merge is subjected to a clipping operation to obtain an independent chromosome;

[0019] Random rotation is applied to the first type of merged chromosome, the second type of merged chromosome, and the independent chromosome to generate a chromosome set, and annotation information corresponding to the chromosomes in the chromosome set is generated.

[0020] Wherein, the merging of the chromosomes in the chromosome set into a picture as a chromosome foreground image includes:

[0021] The first chromosome in the chromosome set is taken as the basis of merging, and the placement position of the chromosome is determined under the premise of controlling the overlapping proportion of the chromosomes;

[0022] The chromosome at the determined position is offset, and the image size is aligned by filling operation;

[0023] After placing each offset chromosome on the canvas, the canvas is subjected to phased rotation, and all sequentially placed chromosomes are merged into a picture as a chromosome foreground image.

[0024] Wherein, the method further includes:

[0025] extract a single chromosome image in the enhanced metaphase chromosome image based on the first annotation information;

[0026] extract a single chromosome image in the random background metaphase chromosome image based on the fourth annotation information;

[0027] generate a single chromosome image set and annotation information corresponding to each chromosome in the single chromosome image set; the single chromosome image set includes single chromosome images extracted from the enhanced metaphase chromosome image and the random background metaphase chromosome image.

[0028] The method further comprises:

[0029] randomly select two chromosome images in the single chromosome image set, denoted as a first chromosome image and a second chromosome image;

[0030] determine a placement area of the chromosome corresponding to the second chromosome image in the first chromosome image under the premise of meeting the overlap ratio requirement; the overlap ratio is less than a preset ratio and greater than 0;

[0031] cover the chromosome pixels corresponding to the second chromosome image to the placement area through bit operation, update the first chromosome image to generate a cross-overlapping chromosome image, and corresponding annotation information;

[0032] The cross-overlapping chromosome image and the corresponding annotation information are used to train an image segmentation model established based on deep learning, and the cross-overlapping is that two chromosomes partially overlap.

[0033] The method further comprises:

[0034] select a chromosome image in the single chromosome image set, and extract a contour point set of the chromosome in the chromosome image according to mask data in the annotation information of the chromosome image;

[0035] extract a chromosome skeleton of the chromosome according to the mask data and the contour point set;

[0036] straighten the chromosome based on the chromosome skeleton and the mask data, obtain a straightened chromosome image, and generate annotation information corresponding to the chromosome image;

[0037] bend the chromosome in the straightened chromosome image, obtain a bent chromosome image, and generate annotation information corresponding to the bent chromosome image; wherein the straightened chromosome image and the bent chromosome image, and the corresponding annotation information are used to train an anomaly classification model established based on deep learning.

[0038] The bending processing of the chromosomes in the straightened chromosome image comprises:

[0039] The coordinate space of the straightened chromosome image is defined as a set of planar points;

[0040] The rotation center is determined according to the annotation information of the straightened chromosome image, and for the points located after the preset bending ratio, a two-dimensional rotation matrix adding a preset bending angle is applied for coordinate transformation to generate a target point set after bending;

[0041] The coordinate extreme values of all points in the target point set are calculated, and a size-adaptive output canvas is dynamically created based on the coordinate extreme values of all points;

[0042] An inverse mapping relationship from the output canvas coordinates to the straightened image coordinates is established;

[0043] Based on the inverse mapping relationship, each coordinate point in the output canvas is inversely rotated back to the coordinate system of the straightened chromosome image;

[0044] If the coordinates after inverse transformation are located within the range of the straightened chromosome image, pixel sampling is performed and the sampling value is assigned to the corresponding coordinate point on the output canvas; and the simulation image after pixel sampling is taken as the bent chromosome image.

[0045] In another embodiment, a chromosome image generation device for deep learning is provided, and the device comprises:

[0046] A first acquisition unit is configured to acquire a chromosome metaphase image;

[0047] An enhancement processing unit is configured to perform adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain an enhanced chromosome metaphase image and corresponding first annotation information;

[0048] A second acquisition unit is configured to acquire a chromosome karyotype arrangement based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generate corresponding second annotation information;

[0049] A generation unit is configured to generate a chromosome set based on the chromosome karyotype arrangement map and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes; the chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and a third annotation information corresponding to the chromosome foreground image is generated; the chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome mid-phase image; a fourth annotation information corresponding to the random background chromosome mid-phase image is regenerated based on the random background chromosome mid-phase image and the third annotation information, and stored; wherein, the random background chromosome mid-phase image and the corresponding fourth annotation information are used to train a target detection model based on deep learning.

[0050] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements a chromosome image generation method for deep learning.

[0051] In another embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements a chromosome image generation method for deep learning.

[0052] As can be seen from the above technical solution, in the above embodiments, the chromosome metaphase image is first enhanced using a chromosome adaptive multi-scale image enhancement method, and then a karyotype arrangement map is generated based on the enhanced chromosome metaphase image. Next, a chromosome set containing merged chromosomes and independent chromosomes is generated based on the chromosome karyotype arrangement map. Finally, the chromosomes in this chromosome set are merged as a foreground image and fused with a randomly selected background image to generate a random background chromosome metaphase image. This random background chromosome metaphase image is used to train a deep learning-based object detection model. This method can generate high-quality and diverse chromosome metaphase images, which, when used to train a deep learning model, improve the generalization ability and accuracy of the deep learning model in object detection tasks. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the chromosome image generation process for deep learning in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of a chromosome metaphase image;

[0055] Figure 3 This is a schematic diagram of an enhanced metaphase chromosome image in an embodiment of this application;

[0056] Figure 4A chromosome karyotype arrangement diagram based on the enhanced chromosome metaphase image obtained in the embodiment of the present application is shown in the figure.

[0057] Figure 5 A random background chromosome metaphase diagram in the embodiment of the present application is shown in the figure.

[0058] Figure 6 A chromosome set generation process diagram in the embodiment of the present application is shown in the figure.

[0059] Figure 7 A chromosome foreground image generation process diagram in the embodiment of the present application is shown in the figure.

[0060] Figure 8 A single chromosome image set generation process diagram in the embodiment of the present application is shown in the figure.

[0061] Figure 9 A cross-overlapping chromosome image generation process diagram in the embodiment of the present application is shown in the figure.

[0062] Figure 10 A cross-overlapping chromosome image diagram in the embodiment of the present application is shown in the figure.

[0063] Figure 11 A straightened and curved chromosome image generation process diagram in the embodiment of the present application is shown in the figure.

[0064] Figure 12 A straightened and curved image diagram in the embodiment of the present application is shown in the figure.

[0065] Figure 13 A chromosome image generation device structure diagram for deep learning in the embodiment of the present application is shown in the figure.

[0066] Figure 14 An electronic device physical structure diagram provided in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0068] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-mentioned drawings, if any, are used as identifiers to distinguish between similar objects, and are not necessarily intended to denote a sequential or chronological order of the objects. It is to be understood that data thus designated can be interchanged, where appropriate, so that an embodiment of the present application described herein could be carried out in a different order than the one illustrated or described herein. Furthermore, the terms "comprising" and "including" and any of their derivatives, are intended to be construed as encompassing not only the listed items, but also others that are similar or equivalent in function, appearance and / or performance. For example, a process, method, object, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements that are expressly listed, but can include additional steps or elements that are not expressly listed or inherent to such process, method, object, or apparatus.

[0069] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and for the same or similar concepts or processes, some examples may not be described again.

[0070] Chromosome karyotype analysis is a core technology in genetic research, prenatal screening and cancer diagnosis. Through deep learning method to analyze chromosome metaphase image realizes chromosome positioning, segmentation and abnormal classification, is the current commonly used means.

[0071] However, the existing chromosome image generation method has many limitations. First, the traditional chromosome image data generation relies on manual annotation or simple copying, resulting in insufficient diversity of the data set, which cannot fully simulate the complex morphology of chromosomes in real scenarios, background interference and overlapping conditions, thereby limiting the generalization ability and accuracy of the deep learning model. Secondly, in the chromosome segmentation task, the existing method is difficult to accurately control the chromosome overlapping ratio, and the generated image data lacks authenticity, affecting the generalization ability and accuracy of the segmentation algorithm. Finally, chromosome abnormal classification needs to extract accurate features from the image, but the chromosome is often curved or complex in shape, resulting in discontinuous skeleton extraction, boundary distortion or image distortion, reducing the classification effect.

[0072] In order to solve the problem that the traditional chromosome image data generation relies on manual annotation or simple copying, resulting in insufficient diversity of the data set and low image quality, an embodiment of the present application provides a chromosome image generation method for deep learning. First, the chromosome metaphase image is enhanced by a chromosome self-adaptive multi-scale image enhancement method, and then a karyotype arrangement map is generated based on the enhanced chromosome metaphase image. Then, a chromosome set containing merged chromosomes and independent chromosomes is generated based on the chromosome karyotype arrangement map. Finally, the chromosomes in the chromosome set are merged as a foreground image and fused with a randomly selected background image to generate a random background chromosome metaphase image, which is used to train a target detection model established based on deep learning. This method can generate high-quality and diversified chromosome metaphase images, which can be used to train a deep learning model, thereby improving the generalization ability and precision of the deep learning model in the target detection task.

[0073] In order to solve the problem that the segmentation algorithm has poor segmentation effect on overlapping chromosomes and unclear segmentation of main chromosomes, an embodiment of the present application also generates a cross-overlapping chromosome image for training an image segmentation model established based on deep learning. This scheme can generate diversified chromosome images, thereby improving the generalization ability and precision of the deep learning model in the image segmentation task.

[0074] In order to generate standardized chromosome images for feature extraction and abnormal classification, an embodiment of the present application also generates a curved chromosome image for training an image classification model established based on deep learning. This scheme can generate diversified and high-quality chromosome images, thereby improving the generalization ability and precision of the deep learning model in the image classification task.

[0075] The following will describe in detail the chromosome image generation process for deep learning in the embodiments of the present application with reference to the accompanying drawings.

[0076] Referring to Figure 1 , Figure 1 is a chromosome image generation flowchart for deep learning in the embodiments of the present application. The specific steps are as follows:

[0077] Step 101, obtain a chromosome metaphase image.

[0078] The chromosome metaphase image obtained in the present application can be one chromosome metaphase image or multiple chromosome metaphase images, and the source of the chromosome metaphase image is not limited, which can be a pre-stored chromosome metaphase image or a directly scanned chromosome metaphase image.

[0079] Referring to Figure 2 , Figure 2A schematic diagram of a chromosome metaphase image. The chromosome metaphase image is a common chromosome metaphase image. The chromosome metaphase image has problems such as low contrast and unclearness.

[0080] Step 102, performing adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain an enhanced chromosome metaphase image and corresponding first annotation information.

[0081] In this step, the chromosome metaphase image is processed by adaptive multi-scale image enhancement, which includes the following steps:

[0082] Firstly, the chromosome metaphase image is processed by grayscale processing.

[0083] The chromosome metaphase image is converted into a grayscale image to simplify the processing. For the image size, the maximum and minimum threshold values are set, and the oversized image is scaled down, for example, the default scaling factor can be set to 0.75, and the undersized image is scaled up, for example, the default scaling factor can be set to 1.5, to ensure that the image size is suitable for subsequent processing; but it is not limited to such size.

[0084] Secondly, the chromosome foreground and background of the grayscale-processed chromosome metaphase image are separated.

[0085] The karyotype of the grayscale-processed chromosome metaphase image is processed to separate the chromosome foreground and background, and the specific implementation can be:

[0086] The grayscale histogram of the image is calculated, and Gaussian smoothing is applied to reduce the noise effect. Based on the smoothed histogram, the highest peak value is identified as the background grayscale value, and the threshold value is dynamically adjusted according to the background value;

[0087] Morphological closing operation and dilation operation are applied to optimize the binary image, remove noise and enhance the connectivity of the chromosome region.

[0088] Thirdly, the external contour is extracted, the area reasonable contour is screened, and the dilation and median blur are applied to generate the foreground mask.

[0089] In this step, the external contour is extracted based on the binary image, the external contour extraction mode is adopted, and the contour points are simplified. The contour is screened based on the dynamic threshold of the image area 1 / 15000 and the median area, the contour with reasonable area is retained, and the noise contour with too small or too large area is removed. The dilation and median blur are applied to smooth the contour boundary, and the foreground mask is generated, which is used for region isolation in subsequent enhancement operation.

[0090] Fourthly, the dark and light backgrounds are distinguished according to the background brightness threshold value, the high and low frequency separation is adopted for the dark background, the low frequency component is extracted by mean blur, the high frequency component is calculated and weighted enhanced, and the contour mask and CLAHE are used for processing the light background image.

[0091] This step divides the image into two cases of dark background and light background according to the background average gray value, and applies different enhancement strategies respectively. For dark background image, first, separate the high frequency and low frequency components, extract the low frequency component by mean blur, calculate the high frequency component and weighted enhancement. Then, apply the multi-scale Retinex algorithm, use multiple Gaussian blur scales to calculate the logarithmic difference, enhance the details of low contrast area. Then, use the contour mask to limit the enhancement result in the chromosome region, and fill the background with white. Finally, apply adaptive histogram equalization (CLAHE, clipping limit 3, grid size 10x10) to further enhance the contrast. The calculation formula is as follows:

[0092] ;

[0093] ;

[0094] wherein, represents the intensity value of the input gray image at pixel point . represents the result of blurring the input image using a Gaussian blur kernel, wherein the standard deviation . represents a small constant, used to prevent division by zero or invalid values in logarithmic operation. represents the number of scales, corresponding to ; represents the logarithmic difference of the Retinex enhancement result; represents the pixel value of the final enhanced image.

[0095] For light background image, directly apply contour mask and CLAHE, simplify the processing flow to preserve the original details.

[0096] Step 5, adaptive contrast adjustment is performed on each contour region to realize enhancement processing of metaphase chromosome image.

[0097] Adaptive contrast adjustment is performed on each contour region. First, calculate the black pixel threshold based on the contour area, count the gray histogram in the contour region, identify the cumulative and threshold gray value as the lower limit, enhance the contrast of the contour region through linear transformation and brightness offset, and apply the result to the corresponding region of the original image. The whole image is then subjected to gamma correction and S-shaped contrast adjustment to further optimize the visual effect and feature saliency. The calculation formula of gamma correction is as follows:

[0098] ;

[0099] wherein, represents the input pixel value, γ represents the gamma value, The pixel value after gamma correction is represented. The final image is adjusted back to the original size, and the enhanced image is saved.

[0100] Referring to Figure 3 , Figure 3 The figure is a schematic diagram of the enhanced metaphase chromosome image in the embodiment of the present application. It can be seen that the enhanced metaphase chromosome image enhances the contrast and details, significantly improves the visual clarity of the chromosome region, highlights the chromosome edges and content structure, and facilitates feature extraction. Through background adaptive enhancement, the chromosomes in the dark and light background are effectively separated, and the background interference is reduced.

[0101] In step 103, the chromosome karyotype arrangement is obtained based on the enhanced metaphase chromosome image and the corresponding first annotation information, and the corresponding second annotation information is generated.

[0102] The specific implementation of generating the chromosome karyotype arrangement based on the enhanced metaphase chromosome image in the embodiment of the present application is not limited. Generally, it is based on the metaphase chromosome image, and in the embodiment of the present application, it is generated for the enhanced metaphase chromosome image.

[0103] Referring to Figure 4 , Figure 4 The figure is a schematic diagram of the chromosome karyotype arrangement obtained based on the enhanced metaphase chromosome image in the embodiment of the present application. Figure 4 In the middle, the chromosomes are arranged in order according to the classification number, and the classification number is marked, indicating the X chromosome and the Y chromosome.

[0104] In step 104, the chromosome set is generated based on the chromosome karyotype arrangement and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes.

[0105] In step 105, the chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and the third annotation information corresponding to the chromosome foreground image is generated.

[0106] In step 106, the chromosome foreground image is fused with a randomly selected background image, and the fused image is subjected to edge smoothing processing to generate a random background metaphase chromosome image.

[0107] Here, the randomly selected background image is any image in the image set, and the image set contains multiple images without chromosomes and cells.

[0108] The randomly selected background image is adjusted to the same size as the chromosome foreground image, and the chromosome foreground image is superimposed on the background image through a mask. Gaussian blur is applied to smooth the image edges, and weighted averaging is used to fuse the foreground and background to generate a random background metaphase chromosome image.

[0109] Referring to Figure 5 , Figure 5 is a schematic diagram of a random background chromosome metaphase in the embodiment of the application. Figure 5 is a random background chromosome metaphase diagram generated by merging the enhanced chromosomes and fusing with the random background diagram. The random background chromosome metaphase diagram data generated in this way is diverse and of high quality, and when used to train a target detection model established based on deep learning, it can improve the generalization ability and precision of the deep learning model in the target detection task.

[0110] Step 107, according to the random background chromosome metaphase image and the third annotation information, re-generate the fourth annotation information corresponding to the random background chromosome metaphase image, and store; wherein the random background chromosome metaphase image and the corresponding fourth annotation information are used to train a target detection model established based on deep learning.

[0111] After the related processing of the chromosomes, RLE encoding needs to be generated according to the mask data of each chromosome, the bounding box and area information are calculated, and the annotation file is constructed;

[0112] Each annotation information in the embodiment of the application usually includes: class number, segmentation mask, area, bounding box field, etc. In specific use, the content included in the annotation information of the chromosome can be increased or decreased according to the specific actual needs, and no limitation is made to this.

[0113] After generating the random background chromosome metaphase image and the corresponding annotation information, it is saved correspondingly, and the random background chromosome metaphase image here can be one or more, and a storage path is specified, and the random background chromosome metaphase image can be used to train the target detection model established based on deep learning subsequently.

[0114] In the embodiment of the application, first, the chromosome metaphase image is enhanced by the chromosome adaptive multi-scale image enhancement method, and then the karyotype arrangement diagram is generated based on the enhanced chromosome metaphase image; then the chromosome set containing the merged chromosomes and the independent chromosomes is generated based on the chromosome karyotype arrangement diagram; finally, the chromosomes in the chromosome set are merged as a foreground image and fused with a randomly selected background image to generate a random background chromosome metaphase image, which is used to train a target detection model established based on deep learning. This method can generate high-quality and diverse chromosome metaphase images, which can improve the generalization ability and precision of the deep learning model in the target detection task when used to train the deep learning model.

[0115] In another example,

[0116] Referring to Figure 6 , Figure 6A flowchart for generating a chromosome set in an embodiment of the present application is shown. The specific steps are as follows:

[0117] In step 601, the chromosomes corresponding to the boundary box aspect ratios of the chromosomes in the chromosome karyotype arrangement chart are sorted in ascending order, and the sorted chromosomes are evenly divided into a first half group and a second half group.

[0118] The boundary box aspect ratio of the chromosome here is determined according to the second annotation information corresponding to the chromosome.

[0119] Here, when the chromosomes cannot be evenly divided, i.e., the number of chromosomes is odd, the middle chromosomes can be divided into the first half group. The first half group here is used for crossover processing and can be referred to as a crossable group. The second half group is used for connection operation and can be referred to as a connectable group. For example, there are five chromosomes, namely chromosome 1, chromosome 2, chromosome 3, chromosome 4, and chromosome 5. Assuming that the five chromosomes are sorted in ascending order of the aspect ratio as follows: chromosome 5, chromosome 3, chromosome 2, chromosome 4, and chromosome 1. Then, chromosome 5, chromosome 3, and chromosome 2 are divided into the first half group, and chromosome 4 and chromosome 1 are divided into the second half group.

[0120] The connection operation here is to simulate the contact or adhesion of two or more chromosomes at the edge of the chromosome. It can form a V-shaped or head-to-tail connected chromosome;

[0121] The crossover operation is to simulate the overlapping or stacking phenomenon of two or more chromosomes in the physical space.

[0122] In step 602, two or three chromosomes in the first half group are merged by the crossover operation to obtain a first type of merged chromosome.

[0123] In an embodiment of the present application, a first preset number of chromosomes are randomly extracted from the first half group to perform the crossover operation. The first preset number can be any value from 2 to 5, such as 2, 3, 4, or 5. The crossover operation is performed to merge two or three chromosome images, wherein the probabilities of two-chromosome merging and three-chromosome merging are 40% and 60%, respectively.

[0124] In step 603, the chromosomes in the first half group that cannot be merged are divided into the second half group, and two or three chromosomes in the updated second half group are merged by the connection operation to obtain a second type of merged chromosome.

[0125] The single chromosome that is not combined in the first half group is combined into the second half group. If all chromosomes in the first half group are successfully combined, there is no single chromosome, and the connection operation of the second half group is directly performed. A second preset number of chromosomes are randomly extracted from the updated second half group, and the second preset number can be any one of 3 to 7. The extracted chromosomes are subjected to a connection operation to combine two or three chromosomes, wherein the probabilities of two-chromosome combination and three-chromosome combination are each 50%.

[0126] In step 604, a cutting operation is performed on the single chromosome that fails to be combined in the updated second half group to obtain an independent chromosome.

[0127] In step 605, random rotation is applied to the first type of combined chromosome, the second type of combined chromosome, and the independent chromosome to generate a chromosome set and generate annotation information corresponding to the chromosomes in the chromosome set.

[0128] Thus, the generation of the chromosome set is completed.

[0129] Referring to Figure 7 , Figure 7 is a schematic diagram of a chromosome foreground image generation process in the embodiment of the present application. The specific steps are as follows:

[0130] In step 701, the first chromosome in the chromosome set is taken as a combined basic object, and the placement position of the chromosome is determined under the premise of controlling the overlapping proportion of the chromosomes.

[0131] The process of specifically determining the placement position of the chromosome is as follows:

[0132] A chromosome is randomly selected from the chromosome set, such as the first chromosome, as a combined basic object, and then all chromosomes are sequentially combined, but the overlapping proportion of the chromosomes is considered during the combination.

[0133] During the combination process, the horizontal distance between two chromosomes is calculated based on the mask to control the overlapping proportion, such as controlling the overlapping proportion of each two chromosomes to be no more than 50%.

[0134] The overlapping proportion calculation here can be: the ratio of the area of the overlapping part of the two chromosomes to the area of the smaller chromosome of the two chromosomes.

[0135] In step 702, the chromosome at the determined position is offset, and the image size is aligned through a filling operation.

[0136] In this step, the chromosome at the determined position is offset, which specifically includes:

[0137] A positive offset within a first preset offset range is applied to each chromosome with a first preset probability, and a negative offset within a second preset offset range is applied to each chromosome with a second preset probability; wherein the first preset probability is greater than the second preset probability, and the sum of the first preset probability and the second preset probability is 1; the first preset offset range is smaller than the second preset offset range.

[0138] In a specific implementation, the first preset probability can be set as 80%, the second preset probability can be set as 20%, the first preset range can be regarded as a small offset range, and the second preset range can be regarded as a larger offset range. The effect of the implementation is that a small offset range is applied with a probability of 80%, and a larger negative offset is applied with a probability of 20%.

[0139] The image size is aligned by a padding operation, and the image and its mask are merged.

[0140] In step 703, after placing each offset chromosome on the canvas, a phased rotation is applied to the canvas, and all sequentially placed chromosomes are merged into an image as a chromosome foreground image.

[0141] When the phased rotation is applied to the canvas, the rotation angle of the initial rotation stage, the rotation stage length of the initial rotation stage, and the rotation angle reduction rule and the rotation stage length increase rule are set; the rotation stage length represents the number of chromosomes corresponding to each rotation stage.

[0142] In a specific embodiment of the present application, a relatively short rotation stage length is initially set, such as a length of 3, which means that the rotation stage length corresponds to 3 chromosomes, i.e., the first to third chromosomes are added to determine that the rotation stage is the initial rotation stage. An initial, relatively large rotation angle is set in the initial rotation stage, such as 90 degrees, i.e., the rotation angle determined when the first to third chromosomes are added is 90 degrees.

[0143] When the number of objects added to the canvas reaches the length of this rotation stage, for example, 3 objects are added, a rotation stage switching is performed. When switching, first, the rotation amplitude is reduced, and the rotation angle is reduced, for example, from 90 degrees to half to 45 degrees; second, the rotation stage length required to trigger the next switching is doubled, for example, now 7 new objects need to be added instead of 3; the chromosome is rotated using this new, smaller angle, such as 45 degrees; after 7 new objects are added and 7 times of 45-degree rotation are completed, the system will trigger switching again, and the rotation angle will be halved again, such as about 23 degrees, and the length of the next rotation stage will be doubled again, such as 15 new objects, and so on, until all chromosomes are merged.

[0144] The offset all chromosomes are applied to the phased rotation, and the specific steps include:

[0145] The first step, when any current chromosome is added to the main canvas, according to the number of chromosomes added to the canvas, and the rotation stage length of the initial rotation stage, and the length adjustment rule of the rotation stage, determine the rotation stage of the current chromosome.

[0146] If the chromosome to be currently added to the main canvas is the fourth chromosome, it is determined that the rotation stage of the current chromosome is the second rotation stage.

[0147] The second step, according to the rotation stage of the current chromosome, and the rotation angle adjustment rule, determine the rotation angle.

[0148] When it is determined that the rotation stage of the current chromosome is the second rotation stage, the rotation angle is determined to be 45 degrees.

[0149] The third step, using the determined rotation angle to rotate the canvas.

[0150] All the chromosomes placed in sequence are merged into an image as a chromosome foreground image. After merging, the image is integrated into a complete image, the target size is randomly expanded to 1 to 2 times the original size, and a random shift is applied to all chromosomes in the complete image (based on the size of the filling area).

[0151] Referring to Figure 8 , Figure 8 The figure is a schematic diagram of the generation process of a single chromosome image set in the embodiment of the present application. The specific steps are as follows:

[0152] Step 801, based on the first annotation information, extract a single chromosome image in the enhanced metaphase chromosome image. Perform step 803.

[0153] Step 802, based on the fourth annotation information, extract a single chromosome image in the random background metaphase chromosome image.

[0154] In specific implementation, a set of effective image-mask pairs are randomly selected as target images and their segmentation masks according to the annotation information, and a single chromosome is extracted according to the segmentation mask position to generate a single chromosome image. In specific extraction, the mask is binarized into foreground and background by adaptive thresholding based on mask value distribution, and the mask boundary is reduced by applying erosion operation to avoid edge pixel interference. The chromosome region is extracted from the source image using the eroded mask, and the pixels in the mask coverage area are retained by bit operation.

[0155] Step 803, generate a single chromosome image set and the annotation information corresponding to each chromosome in the single chromosome image set; the single chromosome image set includes single chromosome images extracted from the enhanced metaphase chromosome image and the random background metaphase chromosome image.

[0156] The images in the single-chromosome image set generated in the embodiments of the present application are of two types: one is from the enhanced chromosome metaphase image, and the other is from the random background chromosome metaphase image; such chromosomes are more diversified.

[0157] To solve the problem that the segmentation algorithm has poor segmentation effect on overlapping chromosomes and unclear segmentation of main chromosomes, the embodiments of the present application also generate a cross-overlapping chromosome image, as shown in Figure 9 , Figure 9 The flowchart of the generation of the cross-overlapping chromosome image in the embodiments of the present application is shown in FIG. 9. The specific steps are as follows:

[0158] Step 901: Randomly select two chromosome images from the single-chromosome image set, denoted as a first chromosome image and a second chromosome image.

[0159] Step 902: Determine the placement region of the second chromosome corresponding to the second chromosome image in the first chromosome image under the premise of meeting the requirement of the overlapping ratio; the overlapping ratio is less than a preset ratio value and greater than 0.

[0160] The determination process of the placement region in this step includes:

[0161] Randomly select a candidate region for placing the second chromosome corresponding to the second chromosome image in the first chromosome image;

[0162] Extract the mask of the position region corresponding to the first chromosome in the first chromosome image, and compare it with the eroded source mask to calculate the overlapping area.

[0163] The ratio of the overlapping area to the smaller mask area is determined as the overlapping ratio.

[0164] If the overlapping ratio is less than the preset ratio value and greater than 0, the candidate region is determined as the placement region.

[0165] If the overlapping ratio is not less than the preset ratio value or is 0, the candidate region for placing the second chromosome corresponding to the second chromosome image is randomly selected again until a candidate region with an overlapping ratio less than the preset ratio value and greater than 0 is selected, which is determined as the placement region.

[0166] In the specific implementation, if the candidate region is not successfully determined for multiple times, the generation of the image can also be ended, and two single-chromosome images are randomly selected again for the generation of the cross-overlapping chromosome image. For example, if the number of times of determining the candidate region reaches 20, the generation of the image is ended.

[0167] Step 903, the second chromosome image corresponding to the chromosome pixel is covered to the placement area by bit operation, the first chromosome image is updated to generate a cross-overlapping chromosome image, and the corresponding label information; wherein the cross-overlapping chromosome image and the corresponding label information are used for training an image segmentation model established based on deep learning, and the cross-overlapping is that two chromosomes partially overlap.

[0168] The label information herein at least includes a segmentation mask.

[0169] Referring to Figure 10 , Figure 10 This is a schematic diagram of the cross-overlapping chromosome image in the embodiments of the present application. Figure 10 The cross-overlapping chromosome image in the above is one of the cross-overlapping chromosome images generated in the embodiments. The chromosome segmentation of the cross-overlapping chromosome image is clear and has good effect.

[0170] The generation method of the cross-overlapping chromosome image provided in the embodiments can improve the clarity of chromosome segmentation, and the random cross-over of different chromosomes generates diversified cross-overlapping chromosome images. The generated cross-overlapping chromosome images are used for training an image segmentation model established based on deep learning, thereby improving the generalization ability and precision of the deep learning model in the image segmentation task.

[0171] In specific implementation, the mask region can also be superimposed on the target image in red transparent form to generate a visual image; the visual image can be used for debugging and verification.

[0172] In order to generate standardized chromosome images for feature extraction and abnormal classification, the present embodiments also propose a straightened chromosome and a straightened and curved chromosome image. Referring to Figure 11 , Figure 11 The present embodiments propose a straightened and curved chromosome image generation process diagram. The specific steps are as follows:

[0173] Step 1101, a chromosome image is randomly selected from a single chromosome image set, and an outline point set is extracted according to mask data in the label information of the chromosome image.

[0174] The extracted outline point set can be used for subsequent skeleton extension.

[0175] Step 1102, a chromosome skeleton is extracted according to the mask data and the outline point set.

[0176] The specific implementation of extracting the chromosome skeleton in this step includes the following steps:

[0177] Firstly, the mask data is subjected to morphological smoothing processing.

[0178] To eliminate the sawtooth and noise of the segmentation edge, morphological smoothing is performed on the mask. Specifically, a strategy of first closing operation and then opening operation can be adopted, a convolution kernel with a preset size (such as 3x3) is used, and small holes inside the mask are filled and edge burrs are smoothed to provide high-quality input for subsequent skeleton extraction.

[0179] In the second step, the center skeleton line corresponding to the chromosome is extracted by performing a medial axis transformation on the smoothed mask data.

[0180] The center skeleton line of the target is extracted by applying the medial axis transformation to the smoothed mask. The original skeleton usually contains many short branches caused by contour noise points. By using an iterative algorithm, the end branches with a length less than a preset threshold, such as 10 pixels, are detected and removed, and the main structure is retained.

[0181] In order to ensure the uniqueness of the skeleton, a graph structure of the skeleton pixels is constructed, the path distance between all endpoint pairs can be calculated using breadth-first search, only the longest path is retained as the main skeleton, and all remaining loops or secondary branches are removed.

[0182] In the third step, the center skeleton line is converted into an ordered geometric curve to obtain the skeleton center line.

[0183] The extracted skeleton is an unordered pixel set and needs to be converted into an ordered geometric curve. An 8-neighborhood depth-first search algorithm can be used to start from one end of the skeleton, track the entire path, and convert the discrete pixel coordinates into an ordered point list. In order to solve the discontinuity and sawtooth problem of the pixel-level skeleton, B-spline is used to fit the ordered point list. Then, one-dimensional Gaussian filtering is performed on the interpolated coordinates to remove local jitter and generate a smooth, continuous and point density uniform skeleton curve.

[0184] In the fourth step, the skeleton is extended from the inside to the physical edge of the corresponding chromosome to obtain the chromosome skeleton of the chromosome; wherein the physical edge is determined according to the contour point set.

[0185] In order to prevent the loss of the beginning and end of the straightened image, the skeleton needs to be extended from the inside to the physical edge of the target, a certain number of neighborhood points such as a window size of 10 are selected at the head and tail of the skeleton, the average tangent direction vector is calculated and normalized. Compared with using only the last two points, using a window neighborhood can effectively resist noise and make the direction more stable.

[0186] Along the calculated tangent direction, new skeleton points are iteratively added at a preset step size, such as 1 pixel. At each extension step, it is detected whether the new point is located inside the target mask. Once the new point touches or exceeds the mask boundary, the extension is immediately stopped. This step ensures that the skeleton can penetrate the entire geometric length of the target.

[0187] Step 1103, straighten the chromosome based on the chromosome skeleton and the mask data, obtain the straightened chromosome image, and generate the annotation information corresponding to the chromosome image.

[0188] The specific implementation of straightening the chromosome based on the chromosome skeleton and the mask data in this step is not limited, and the following implementation manner is given in the embodiment of the present application:

[0189] Firstly, the total arc length of the skeleton is determined according to the chromosome skeleton data, and the size of the output image after straightening is defined based on the total arc length of the skeleton and a preset target width.

[0190] The image is converted from the Cartesian coordinate system to the coordinate system after straightening, the tangent vector of each point on the extended skeleton curve is calculated, and the normal vector corresponding to the tangent vector is obtained by rotating 90 degrees. The width of the output image is defined as the target width, and the height is the arc length of the skeleton.

[0191] Secondly, the chromosome skeleton is parameterized based on the arc length, and the unit normal vector of each point on the skeleton is calculated.

[0192] For each pixel point (u, v) in the output image, u represents the length along the skeleton direction, and v represents the distance along the normal direction.

[0193] Thirdly, based on the coordinate mapping relationship, the corresponding source coordinates of each pixel point in the output image in the original image are determined.

[0194] The corresponding coordinates Psrc of each pixel point in the original image are calculated using the formula Psrc=Pskel(u) + v N(u), wherein Pskel(u) is the point at the corresponding arc length position on the skeleton, v N(u) is the unit normal vector of the point.

[0195] Fourthly, according to the coordinate mapping relationship, the original image is resampled to generate an intermediate image after straightening.

[0196] Using the calculated coordinate mapping table (Map_x, Map_y), the original image pixels can be resampled by bilinear interpolation.

[0197] Fifthly, the regions in the intermediate image corresponding to the regions outside the mask data are filled with background color to generate the chromosome image after straightening.

[0198] For the regions outside the boundary of the original image, white background (255, 255, 255) is filled, and finally the straightened long strip image is generated.

[0199] At step 1104, the chromosomes in the straightened chromosome image are subjected to bending processing to obtain a bent chromosome image, and labeled information corresponding to the bent chromosome image is generated; wherein the straightened chromosome image and the bent chromosome image, and the corresponding labeled information are used to train an abnormality classification model established based on deep learning.

[0200] In this step, the chromosomes in the straightened chromosome image are subjected to bending processing, which specifically includes:

[0201] The deformation parameters need to be defined first: a preset bending ratio and a preset bending angle are set; here, the bending is from the midpoint of the skeleton.

[0202] Firstly, the coordinate space of the straightened chromosome image is defined as a set of planar points.

[0203] Secondly, the rotation center is determined according to the labeled information of the straightened chromosome image, and for the points after the preset bending ratio, a two-dimensional rotation matrix adding the preset bending angle is applied for coordinate transformation to generate a set of target points after bending.

[0204] The preset bending ratio is represented by the value corresponding to the Y coordinate in the coordinate system, and then it is determined which points are the points after the preset bending ratio.

[0205] Thirdly, the coordinate extreme values of all points in the target point set are calculated, and a size-adaptive output canvas is dynamically created based on the coordinate extreme values of all points.

[0206] The implementation of this step ensures that the target after bending is not truncated.

[0207] Fourthly, an inverse mapping relationship from the output canvas coordinates to the straightened image coordinates is established.

[0208] Fifthly, based on the inverse mapping relationship, each coordinate point in the output canvas is inversely rotated back to the coordinate system of the straightened chromosome image based on the inverse mapping relationship.

[0209] Sixthly, if the coordinates after inverse transformation are within the range of the straightened chromosome image, pixel sampling is performed and the sampling value is assigned to the corresponding coordinate point on the output canvas; and the simulation image after pixel sampling is taken as the bent chromosome image.

[0210] Referring to Figure 12 , Figure 12 The straightened and bent image schematic diagram in the embodiments of the present application. Figure 12The leftmost image in the middle is a single chromosome image in the single chromosome image set, the image in the middle is a straightened chromosome image after straightening processing on the single chromosome image on the left, and the rightmost image is a curved chromosome image after bending processing on the straightened chromosome image in the middle.

[0211] In this embodiment, not only the straightened chromosome image is generated, but also the curved chromosome image is generated on the basis of the straightened chromosome image. In this embodiment, the straightened chromosome image and the straightened and curved chromosome image are generated on the basis of the enhanced single chromosome, which can increase the sample diversity, improve the image quality, and thus improve the generalization ability and precision of the deep learning model in the image classification task when the deep learning model is used to train the abnormal classification model.

[0212] The image generated in the embodiment of the present application significantly improves the image contrast and detail clarity, that is, improves the image quality: through the multi-scale Retinex algorithm, CLAHE adaptive histogram equalization and adaptive contrast adjustment, the edge and internal structure details of the chromosome region are significantly enhanced, the image quality is optimized, and the feature extraction of the subsequent deep learning algorithm is facilitated.

[0213] Improve data diversity and authenticity: through random cross, connection and background fusion operations, a diversified chromosome metaphase image dataset is generated, which effectively simulates the overlap, morphological change and background interference in the real scene, overcomes the defects of single data and unrealistic simulation of traditional methods, and significantly enhances the generalization ability of the deep learning model.

[0214] Improve the robustness of the segmentation task: by precisely controlling the chromosome overlap ratio to generate image data with authenticity and divisibility, the performance of the chromosome segmentation algorithm is optimized, which can make the model better cope with complex overlapping scenes and improve the segmentation accuracy.

[0215] Enhance the accuracy of feature extraction: adopt continuous skeleton extraction, extension to contour boundary and geometric straightening technology to generate standardized chromosome images, simplify the interference of complex morphology on feature extraction, and effectively improve the accuracy and reliability of the abnormal classification task.

[0216] Support diversified data augmentation: generate chromosome images with diversified morphology through various transformations, further enrich the training dataset, and enhance the adaptability of the deep learning model to different chromosome morphologies.

[0217] Improve the efficiency of automated processing: integrate various format annotation generation, batch image processing and visualization functions to realize efficient automated data generation process, reduce manual intervention, improve data preparation efficiency, and provide reliable support for genetics research, prenatal screening and cancer diagnosis.

[0218] The generated enhanced data and standardized images are suitable for various deep learning tasks, including target detection, image segmentation, and anomaly classification, and significantly improve the application effect in the fields of bioinformatics and medical diagnosis.

[0219] All the optional technical solutions described above can be combined to form optional embodiments of the present disclosure, which will not be described one by one here.

[0220] Based on the same inventive concept, the present application also provides a chromosome image generation device for deep learning. Referring to Figure 13 , Figure 13 The chromosome image generation device for deep learning in the embodiments of the present application is a structural schematic diagram. The device comprises:

[0221] The first acquisition unit 1301 is configured to acquire a chromosome metaphase image.

[0222] The enhancement processing unit 1302 is configured to perform adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain an enhanced chromosome metaphase image and corresponding first annotation information.

[0223] The second acquisition unit 1303 is configured to acquire a chromosome karyotype arrangement map based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generate corresponding second annotation information.

[0224] The first generation unit 1304 is configured to generate a chromosome set based on the chromosome karyotype arrangement map and the corresponding second annotation information; the chromosome set comprises merged chromosomes and independent chromosomes; the chromosomes in the chromosome set are sequentially merged into a chromosome foreground image, and corresponding third annotation information of the chromosome foreground image is generated; the chromosome foreground image is fused with a randomly selected background image, and edge smoothing processing is performed on the fused image to generate a random background chromosome metaphase image; fourth annotation information corresponding to the random background chromosome metaphase image is regenerated according to the random background chromosome metaphase image and the third annotation information, and is stored; wherein the random background chromosome metaphase image and the corresponding fourth annotation information are used to train a target detection model established based on deep learning.

[0225] In another example,

[0226] The first generation unit 1304 is specifically configured to sort the chromosomes corresponding to the boundary box height-width ratios of the chromosomes in the chromosome karyotype arrangement in ascending order, and uniformly divide the sorted chromosomes into a first half group and a second half group; perform two or three chromosome merging on the chromosomes in the first half group through a crossover operation to obtain first-type merged chromosomes; divide the chromosomes in the first half group that fail to be merged into the second half group, and perform two or three chromosome merging on the chromosomes in the updated second half group through a connection operation to obtain second-type merged chromosomes; perform a clipping operation on the single chromosomes in the updated second half group that fail to be merged to obtain independent chromosomes; apply random rotation to the first-type merged chromosomes, the second-type merged chromosomes, and the independent chromosomes, generate a chromosome set, and generate annotation information corresponding to the chromosomes in the chromosome set.

[0227] In another example,

[0228] The first generation unit 1304 is specifically configured to take the first chromosome in the chromosome set as a merged basic object, determine a chromosome placement position under the premise of controlling a chromosome overlap ratio, offset the chromosome at the determined position, and align an image size through a filling operation; place each chromosome after the offset on a canvas, apply phased rotation to the canvas, and merge all the chromosomes placed in sequence into an image as a chromosome foreground image.

[0229] In another example, the apparatus further includes:

[0230] The second generation unit 1305 is configured to extract single-chromosome images in the enhanced chromosome metaphase image based on the first annotation information, extract single-chromosome images in a random background chromosome metaphase image based on the fourth annotation information, generate a single-chromosome image set and annotation information corresponding to each chromosome in the single-chromosome image set, and the single-chromosome image set includes the single-chromosome images extracted from the enhanced chromosome metaphase image and the random background chromosome metaphase image.

[0231] In another example, the apparatus further includes:

[0232] The third generation unit 1306 is configured to randomly select two chromosome images from the single-chromosome-image set, and record the two chromosome images as a first chromosome image and a second chromosome image; determine a placement area of a chromosome corresponding to the second chromosome image in the first chromosome image under the premise that an overlap ratio requirement is met; the overlap ratio is less than a preset ratio and greater than 0; cover the chromosome pixels corresponding to the second chromosome image to the placement area through bit operation, update the first chromosome image to generate a cross-overlapping chromosome image and corresponding label information; the cross-overlapping chromosome image and the corresponding label information are used to train an image segmentation model established based on deep learning, and the cross-overlapping is that two chromosomes partially overlap.

[0233] In another example, the apparatus further includes:

[0234] The fourth generation unit 1307 is configured to select a chromosome image from the single-chromosome-image set, extract a contour point set of a chromosome in the chromosome image according to mask data in label information of the chromosome image; extract a chromosome skeleton of the chromosome according to the mask data and the contour point set; straighten the chromosome based on the chromosome skeleton and the mask data, obtain a straightened chromosome image, and generate label information corresponding to the straightened chromosome image; perform bending processing on the chromosome in the straightened chromosome image, obtain a bent chromosome image, and generate label information corresponding to the bent chromosome image; the straightened chromosome image and the bent chromosome image, and the corresponding label information are used to train an anomaly classification model established based on deep learning.

[0235] The fourth generation unit 1307 is specifically configured to define a coordinate space of the straightened chromosome image as a plane point set; determine a rotation center according to the label information of the straightened chromosome image, and for points located after a preset bending ratio, apply a two-dimensional rotation matrix adding a preset bending angle to perform coordinate transformation to generate a target point set after bending; calculate coordinate extreme values of all points in the target point set, and dynamically create an output canvas with adaptive size based on the coordinate extreme values of all points; establish an inverse mapping relationship from the output canvas coordinates to the straightened image coordinates; based on the inverse mapping relationship, reversely rotate each coordinate point in the output canvas back to the coordinate system of the straightened chromosome image; if the coordinates after inverse transformation are located within the range of the straightened chromosome image, perform pixel sampling and assign the sampling value to the corresponding coordinate point on the output canvas; and output the simulation image after pixel sampling as the bent chromosome image.

[0236] The units in the above embodiments can be integrated or separated; can be combined into one unit, or can be further split into multiple sub-units.

[0237] In another embodiment, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a chromosome image generation method for deep learning.

[0238] In another embodiment, a computer-readable storage medium is also provided, on which computer instructions are stored, which, when executed by a processor, implement a chromosome image generation method for deep learning.

[0239] Figure 14 This is a schematic diagram of the physical structure of the electronic device provided in an embodiment of this application. Figure 14 As shown, the electronic device may include: a processor 1410, a communications interface 1420, a memory 1430, and a communication bus 1440, wherein the processor 1410, the communications interface 1420, and the memory 1430 communicate with each other through the communication bus 1440. The processor 1410 can call logical instructions in the memory 1430 to execute the following methods:

[0240] Obtain images of chromosome metaphase;

[0241] Adaptive multi-scale image enhancement processing is performed on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information;

[0242] Based on the enhanced metaphase image of the chromosome and the corresponding first annotation information, a chromosome karyotype arrangement map is obtained, and the corresponding second annotation information is generated;

[0243] A chromosome set is generated based on the chromosome karyotype arrangement diagram and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes;

[0244] The chromosomes in the chromosome set are sequentially merged into one image as the chromosome foreground image, and the third annotation information corresponding to the chromosome foreground image is generated.

[0245] The chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome metaphase image.

[0246] The fourth annotation information corresponding to the random background chromosome mid-phase image is regenerated based on the random background chromosome mid-phase image and the third annotation information, and then stored; wherein, the random background chromosome mid-phase image and the corresponding fourth annotation information are used to train the target detection model based on deep learning.

[0247] In addition, the logic instructions in the memory 1430 described above can be implemented in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0248] The device embodiments described above are only schematic, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0249] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus necessary universal hardware platforms, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some part of the embodiment.

[0250] The computer program product of the first aspect can include a computer readable storage medium. The computer readable storage medium can be a tangible computer readable storage medium storing the program code. The computer readable storage medium can include, but is not limited to, magnetic storage, optical storage, and the like. The computer readable storage medium can be non-transitory. The computer readable storage medium can be a non-transitory computer readable medium. The above definitions of the computer program product of the first aspect and the computer readable storage medium of the first aspect apply, mutatis mutandis, to the computer program product of the second aspect and the computer readable storage medium of the second aspect.

[0251] It is to be understood that the features mentioned hereinbefore and / or shown in the drawings can be further combined with features of the various embodiments and / or claims, even if not, as a matter of fact, mentioned in connection with the embodiment or claim in question. In particular, the features of the various embodiments and / or claims can be combined with each other, even if not, as a matter of fact, mentioned in connection with the embodiment or claim in question, without departing from the spirit and the scope of the present application. In particular, the features of the various embodiments and / or claims can be combined with each other, even if not, as a matter of fact, mentioned in connection with the embodiment or claim in question, without departing from the spirit and the scope of the present application.

[0252] The principles and implementations of the present application are described in detail herein with specific embodiments. The above description is merely for the purpose of illustration of the method of the present application and its core idea, and is not intended to limit the present application. For those skilled in the art, the specific embodiments and application scope of the present application can be changed according to the idea, spirit and principle of the present application. Any modification, equivalent replacement, improvement, etc. made by them shall be included in the scope of protection of the present application.

Claims

1. A chromosome image generation method for deep learning, characterized by, The method comprises: acquiring a chromosome metaphase image; performing adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain an enhanced chromosome metaphase image and corresponding first annotation information; obtaining a chromosome karyotype arrangement based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generating corresponding second annotation information; generating a chromosome set based on the chromosome karyotype arrangement and the corresponding second annotation information; the chromosome set comprises merged chromosomes and independent chromosomes; merging the chromosomes in the chromosome set into an image in sequence as a chromosome foreground image, and generating third annotation information corresponding to the chromosome foreground image; fusing the chromosome foreground image with a randomly selected background image, and performing edge smoothing processing on the fused image to generate a random background chromosome metaphase image; regenerating fourth annotation information corresponding to the random background chromosome metaphase image according to the random background chromosome metaphase image and the third annotation information, and storing; wherein the random background chromosome metaphase image and the corresponding fourth annotation information are used to train a target detection model established based on deep learning.

2. The method of claim 1, wherein, The method further comprises: extracting a single chromosome image in the enhanced chromosome metaphase image based on the first annotation information; sorting the corresponding chromosomes in the chromosome karyotype arrangement according to the aspect ratio of the bounding box of the chromosomes in the chromosome karyotype arrangement from small to large, and uniformly dividing the sorted chromosomes into a first half group and a second half group; the aspect ratio of the bounding box is determined according to the second annotation information corresponding to the chromosomes; merging two or three chromosomes in the first half group by a crossover operation to obtain first-type merged chromosomes; dividing the chromosomes in the first half group that cannot be merged into the second half group, and merging two or three chromosomes in the updated second half group by a connection operation to obtain second-type merged chromosomes; carrying out a clipping operation on the single chromosomes in the updated second half group that cannot be merged to obtain independent chromosomes; 3. The method of claim 1, wherein, applying random rotation to the first-type merged chromosomes, the second-type merged chromosomes, and the independent chromosomes to generate a chromosome set, and generating annotation information corresponding to the chromosomes in the chromosome set. The method further comprises: taking the first chromosome in the chromosome set as a merged base object, determining a chromosome placement position under the premise of controlling the overlapping proportion of chromosomes; offsetting the chromosome at the determined position and aligning the image size by a filling operation; 4. The method of claim 1, wherein, after placing each offset chromosome on a canvas, applying a phased rotation to the canvas, and merging all sequentially placed chromosomes into an image as a chromosome foreground image. The method further comprises: extracting a single chromosome image in the enhanced chromosome metaphase image based on the first annotation information; extract a single chromosome image in the random background chromosome metaphase image based on the fourth annotation information; generate a single chromosome image set and annotation information corresponding to each chromosome in the single chromosome image set; the single chromosome image set includes single chromosome images extracted from the enhanced chromosome metaphase image and the random background chromosome metaphase image.

5. The method of claim 4, wherein, The method further comprises: randomly selecting two chromosome images from the single chromosome image set, denoted as a first chromosome image and a second chromosome image; determining a placement area of the chromosome corresponding to the second chromosome image in the first chromosome image under the premise of meeting the overlap ratio requirement; the overlap ratio is less than a preset ratio and greater than 0; covering the chromosome pixel corresponding to the second chromosome image to the placement area through bit operation, updating the first chromosome image to generate a cross-overlapping chromosome image, and corresponding annotation information; wherein the cross-overlapping chromosome image and the corresponding annotation information are used to train an image segmentation model established based on deep learning, and the cross-overlapping is that two chromosomes partially overlap.

6. The method of claim 4, wherein, The method further comprises: selecting a chromosome image from the single chromosome image set, and extracting a contour point set of the chromosome in the chromosome image according to mask data in the annotation information of the chromosome image; extracting a chromosome skeleton of the chromosome according to the mask data and the contour point set; straightening the chromosome based on the chromosome skeleton and the mask data, obtaining a straightened chromosome image, and generating annotation information corresponding to the chromosome image; bending the chromosome in the straightened chromosome image to obtain a bent chromosome image, and generating annotation information corresponding to the bent chromosome image; wherein the straightened chromosome image and the bent chromosome image, and the corresponding annotation information are used to train an anomaly classification model established based on deep learning.

7. The method of claim 6, wherein, The bending of the chromosome in the straightened chromosome image comprises: defining the coordinate space of the straightened chromosome image as a plane point set; determining a rotation center according to the annotation information of the straightened chromosome image, and applying a two-dimensional rotation matrix adding a preset bending angle to perform coordinate transformation on the points located after the preset bending ratio to generate a target point set after bending; calculating the coordinate extreme values of all points in the target point set, and dynamically creating a size-adaptive output canvas based on the coordinate extreme values of all points; establishing an inverse mapping relationship from the output canvas coordinates to the straightened image coordinates; based on the inverse mapping relationship, reversely rotating each coordinate point in the output canvas back to the coordinate system of the straightened chromosome image; if the coordinates after inverse transformation are within the range of the straightened chromosome image, performing pixel sampling and assigning the sampling value to the corresponding coordinate point on the output canvas; and outputting the simulation image after pixel sampling as the bent chromosome image.

8. A chromosome image generation apparatus for deep learning, characterized by comprising: The device comprises: a first acquisition unit configured to acquire a chromosome metaphase image; An enhancement processing unit is configured to perform adaptive multi-scale image enhancement processing on the metaphase chromosome image to obtain an enhanced metaphase chromosome image and corresponding first annotation information; A second obtaining unit is configured to obtain a chromosome karyotype arrangement based on the enhanced metaphase chromosome image and the corresponding first annotation information, and generate corresponding second annotation information; A generating unit is configured to generate a chromosome set based on the chromosome karyotype arrangement and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes; the chromosomes in the chromosome set are sequentially merged into one image as a chromosome foreground image, and corresponding third annotation information of the chromosome foreground image is generated; the chromosome foreground image is fused with a randomly selected background image, and edge smoothing processing is performed on the fused image to generate a random background metaphase chromosome image; fourth annotation information corresponding to the random background metaphase chromosome image is regenerated according to the random background metaphase chromosome image and the third annotation information, and is stored; wherein the random background metaphase chromosome image and the corresponding fourth annotation information are used to train a target detection model established based on deep learning.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-7. The program is executed by the processor to implement the method of any one of claims 1-7.

Images