A chromosome image generation method and apparatus for deep learning

Abstract

The application discloses a chromosome image generation method and device for deep learning. Adaptive multi-scale image enhancement processing is performed on a chromosome metaphase image to obtain an enhanced chromosome metaphase image; a chromosome karyotype arrangement map is obtained based on the enhanced chromosome metaphase image; a chromosome set is generated based on the chromosome karyotype arrangement map; chromosomes in the chromosome set are sequentially merged into one image as a chromosome foreground image; the chromosome foreground image is fused with a randomly selected background image, and edge smoothing processing is performed on the fused image to generate a random background chromosome metaphase image; wherein the random background chromosome metaphase image is used for training a target detection model established based on deep learning. The method can generate high-quality and diversified chromosome metaphase images, and thus is used for training a deep learning model, and improves the generalization ability and precision of the deep learning model in a target detection task.

Description

Technical Field

[0001] This application relates to the field of biomedical image processing technology, specifically to a method and apparatus for generating chromosome images using deep learning. Background Technology

[0002] Chromosome karyotype analysis is a core technology in genetic research, prenatal screening, and cancer diagnosis. Using deep learning methods to analyze metaphase chromosome images to achieve chromosome localization, segmentation, and abnormality classification is a commonly used approach.

[0003] However, existing methods for chromosome image processing and data generation have many limitations. Traditional chromosome image data generation relies on manual annotation or simple copying, resulting in insufficient dataset diversity and low quality, which in turn limits the generalization ability and accuracy of deep learning models. Summary of the Invention

[0004] In view of the above, this application provides a chromosome image generation method and apparatus for deep learning, which can generate high-quality and diverse chromosome metaphase images, thereby improving the generalization ability and accuracy of deep learning models in target detection tasks when used for training deep learning models.

[0005] To solve the above-mentioned technical problems, the technical solution of this application is implemented as follows:

[0006] In one embodiment, a chromosome image generation method for deep learning is provided, the method comprising:

[0007] Obtain images of chromosome metaphase;

[0008] Adaptive multi-scale image enhancement processing is performed on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information;

[0009] Based on the enhanced metaphase chromosome image and the corresponding first annotation information, a chromosome karyotype arrangement map is obtained, and corresponding second annotation information is generated;

[0010] A chromosome set is generated based on the chromosome karyotype arrangement diagram and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes;

[0011] The chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and the third annotation information corresponding to the chromosome foreground image is generated.

[0012] The chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome metaphase image.

[0013] Based on the random background chromosome midphase image and the third annotation information, the fourth annotation information corresponding to the random background chromosome midphase image is regenerated and stored; wherein, the random background chromosome midphase image and the corresponding fourth annotation information are used to train a target detection model based on deep learning.

[0014] The step of generating a chromosome set based on the chromosome karyotype arrangement map and the corresponding second annotation information includes:

[0015] Based on the aspect ratio of the bounding boxes of the chromosomes in the chromosome karyotype arrangement diagram, the corresponding chromosomes are sorted in ascending order, and the sorted chromosomes are evenly divided into the first half group and the second half group; the aspect ratio of the bounding boxes is determined according to the second annotation information corresponding to the chromosomes;

[0016] Two or three chromosomes in the first half of the group are merged through a crossover operation to obtain the first type of merged chromosomes;

[0017] Chromosomes that failed to merge in the first half group are assigned to the second half group, and two or three chromosomes in the updated second half group are merged by a joining operation to obtain the second type of merged chromosomes.

[0018] The individual chromosomes that failed to merge in the updated second half of the group were pruned to obtain independent chromosomes.

[0019] Random rotations are applied to the first type of merged chromosomes, the second type of merged chromosomes, and the independent chromosomes to generate a chromosome set, and labeling information corresponding to the chromosomes in the chromosome set is generated.

[0020] The step of sequentially merging the chromosomes in the chromosome set into a single image as a chromosome foreground image includes:

[0021] Using the first chromosome in the chromosome set as the base object for merging, the placement position of the chromosome is determined while controlling the chromosome overlap ratio;

[0022] The chromosome at the determined location is offset, and the image size is aligned by a padding operation;

[0023] After each offset chromosome is placed on the canvas, the canvas is rotated in stages, and all the chromosomes placed in sequence are merged into one image as the chromosome foreground image.

[0024] The method further includes:

[0025] Extract single chromosome images from the enhanced metaphase chromosome image based on the first annotation information;

[0026] Based on the fourth annotation information, extract a single chromosome image from the random background chromosome metaphase image;

[0027] Generate a set of single chromosome images, and annotation information corresponding to each chromosome in the set of single chromosome images; the set of single chromosome images includes single chromosome images extracted from enhanced chromosome metaphase images and random background chromosome metaphase images.

[0028] The method further includes:

[0029] Two chromosome images are randomly selected from the set of single chromosome images and denoted as the first chromosome image and the second chromosome image.

[0030] Provided that the overlap ratio requirement is met, a placement area for the chromosome corresponding to the second chromosome image is determined in the first chromosome image; the overlap ratio is less than a preset ratio and greater than 0.

[0031] Bitwise operations are used to overlay the chromosome pixels corresponding to the second chromosome image onto the placement area, and the first chromosome image is updated to generate a cross-overlapping chromosome image and corresponding annotation information;

[0032] The overlapping chromosome images and corresponding annotation information are used to train an image segmentation model based on deep learning. The overlapping refers to the partial overlap of two chromosomes.

[0033] The method further includes:

[0034] Select one chromosome image from the set of single chromosome images, and extract the set of chromosome contour points in the chromosome image based on the mask data in the annotation information of the chromosome image;

[0035] The chromosome skeleton of the chromosome is extracted based on the mask data and the contour point set;

[0036] Based on the chromosome skeleton and the mask data, the chromosome is straightened to obtain a straightened chromosome image, and annotation information corresponding to the chromosome image is generated.

[0037] The chromosomes in the straightened chromosome image are bent to obtain a bent chromosome image, and annotation information corresponding to the bent chromosome image is generated; wherein, the straightened chromosome image, the bent chromosome image, and the corresponding annotation information are used to train an anomaly classification model based on deep learning.

[0038] The step of bending the chromosomes in the straightened chromosome image includes:

[0039] The coordinate space of the straightened chromosome image is defined as a set of planar points;

[0040] The rotation center is determined based on the annotation information of the straightened chromosome image, and for points located after the preset bending ratio, a two-dimensional rotation matrix with a preset bending angle is applied to perform coordinate transformation to generate the target point set after bending.

[0041] Calculate the extreme values ​​of the coordinates of all points in the target point set, and dynamically create an output canvas with adaptive size based on the extreme values ​​of the coordinates of all points;

[0042] Establish an inverse mapping relationship between the output canvas coordinates and the straightened image coordinates;

[0043] Based on the inverse mapping relationship, the coordinate system of the straightened chromosome image is rotated back in reverse for each coordinate point in the output canvas.

[0044] If the coordinates after the inverse transformation are within the range of the straightened chromosome image, then pixel sampling is performed and the sampled values ​​are assigned to the corresponding coordinate points on the output canvas; and the output simulated image after pixel sampling is used as the bent chromosome image.

[0045] In another embodiment, a chromosome image generation apparatus for deep learning is provided, the apparatus comprising:

[0046] The first acquisition unit is used to acquire images of chromosome metaphase.

[0047] An enhancement processing unit is used to perform adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information.

[0048] The second acquisition unit is used to acquire a chromosome karyotype arrangement map based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generate corresponding second annotation information;

[0049] A generation unit is configured to generate a chromosome set based on the chromosome karyotype arrangement map and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes; the chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and a third annotation information corresponding to the chromosome foreground image is generated; the chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome mid-phase image; a fourth annotation information corresponding to the random background chromosome mid-phase image is regenerated based on the random background chromosome mid-phase image and the third annotation information, and stored; wherein, the random background chromosome mid-phase image and the corresponding fourth annotation information are used to train a target detection model based on deep learning.

[0050] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements a chromosome image generation method for deep learning.

[0051] In another embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements a chromosome image generation method for deep learning.

[0052] As can be seen from the above technical solution, in the above embodiments, the chromosome metaphase image is first enhanced using a chromosome adaptive multi-scale image enhancement method, and then a karyotype arrangement map is generated based on the enhanced chromosome metaphase image. Next, a chromosome set containing merged chromosomes and independent chromosomes is generated based on the chromosome karyotype arrangement map. Finally, the chromosomes in this chromosome set are merged as a foreground image and fused with a randomly selected background image to generate a random background chromosome metaphase image. This random background chromosome metaphase image is used to train a deep learning-based object detection model. This method can generate high-quality and diverse chromosome metaphase images, which, when used to train a deep learning model, improve the generalization ability and accuracy of the deep learning model in object detection tasks. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the chromosome image generation process for deep learning in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of a chromosome metaphase image;

[0055] Figure 3 This is a schematic diagram of an enhanced metaphase chromosome image in an embodiment of this application;

[0056] Figure 4This is a schematic diagram of chromosome karyotype arrangement obtained from an enhanced metaphase image of chromosomes in an embodiment of this application;

[0057] Figure 5 This is a schematic diagram of a random background chromosome metaphase map in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the chromosome set generation process in an embodiment of this application;

[0059] Figure 7 This is a schematic diagram of the chromosome foreground image generation process in an embodiment of this application;

[0060] Figure 8 This is a schematic diagram illustrating the process of generating a single chromosome image set in an embodiment of this application;

[0061] Figure 9 A schematic diagram of the process for generating crossover chromosome images in this application embodiment;

[0062] Figure 10 This is a schematic diagram of crossed and overlapping chromosome images in an embodiment of this application;

[0063] Figure 11 This application embodiment illustrates the process of generating images of straightened, bent chromosomes.

[0064] Figure 12 These are schematic diagrams of straightening and bending images in the embodiments of this application;

[0065] Figure 13 This is a schematic diagram of the chromosome image generation device for deep learning in an embodiment of this application.

[0066] Figure 14 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0067] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe the order or sequence of objects. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0069] The technical solution of the present invention will be described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0070] Chromosome karyotype analysis is a core technology in genetic research, prenatal screening, and cancer diagnosis. Using deep learning methods to analyze metaphase chromosome images to achieve chromosome localization, segmentation, and abnormality classification is a commonly used approach.

[0071] However, existing chromosome image generation methods have several limitations. First, traditional chromosome image data generation relies on manual annotation or simple copying, resulting in insufficient dataset diversity. This makes it impossible to fully simulate the complex morphology, background interference, and overlap of chromosomes in real-world scenes, thus limiting the generalization ability and accuracy of deep learning models. Second, in chromosome segmentation tasks, existing methods struggle to precisely control the chromosome overlap ratio, leading to a lack of realism in the generated image data and affecting the generalization ability and accuracy of segmentation algorithms. Finally, chromosome anomaly classification requires the extraction of accurate features from images, but chromosomes often suffer from discontinuous skeleton extraction, distorted boundary processing, or image distortion due to curvature or complex shapes, reducing classification performance.

[0072] To address the issues of insufficient dataset diversity and low image quality resulting from traditional chromosome image data generation relying on manual annotation or simple copying, this application provides a chromosome image generation method for deep learning. First, a chromosome metaphase image is enhanced using an adaptive multi-scale image enhancement method. Then, a karyotype arrangement map is generated based on the enhanced chromosome metaphase image. Next, a chromosome set containing merged and independent chromosomes is generated based on the chromosome karyotype arrangement map. Finally, the chromosomes in this set are merged and used as a foreground image, which is then fused with a randomly selected background image to generate a random background chromosome metaphase image. This random background chromosome metaphase image is used to train a deep learning-based object detection model. This method can generate high-quality and diverse chromosome metaphase images, thereby improving the generalization ability and accuracy of deep learning models in object detection tasks when used for training.

[0073] To address the issue of poor segmentation performance of overlapping chromosomes and unclear segmentation of main chromosomes in segmentation algorithms, this application embodiment also generates a cross-overlapping chromosome image for training a deep learning-based image segmentation model. This approach can generate diverse chromosome images, thereby improving the generalization ability and accuracy of the deep learning model in image segmentation tasks.

[0074] To generate standardized chromosome images for easier feature extraction and anomaly classification, this embodiment also generates a bent chromosome image for training a deep learning-based image classification model. This approach can generate diverse and high-quality chromosome images, thereby improving the generalization ability and accuracy of deep learning models in image classification tasks.

[0075] The chromosome image generation process for deep learning in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0076] See Figure 1 , Figure 1 This is a schematic diagram of the chromosome image generation process for deep learning in an embodiment of this application. The specific steps are as follows:

[0077] Step 101: Obtain a chromosome metaphase image.

[0078] The chromosome metaphase images obtained in this application can be a single chromosome metaphase image or multiple chromosome metaphase images; and there are no restrictions on the source of the chromosome metaphase images, which can be pre-stored chromosome metaphase images or directly obtained by scanning.

[0079] See Figure 2 , Figure 2This is a schematic diagram of a chromosome metaphase image. This is a common type of chromosome metaphase image. However, this chromosome metaphase image suffers from low contrast and is not clear.

[0080] Step 102: Perform adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information.

[0081] This step involves adaptive multi-scale image enhancement processing of the chromosome metaphase image, specifically including the following steps:

[0082] The first step is to process the chromosome metaphase image into grayscale.

[0083] Convert the chromosome metaphase image to grayscale to simplify processing. Set maximum and minimum thresholds for image size; scale down excessively large images proportionally (e.g., a default scaling factor of 0.75 can be set), and scale up excessively small images proportionally (e.g., a default scaling factor of 1.5 can be set), ensuring the image size is suitable for subsequent processing; however, such sizes are not limited.

[0084] The second step is to separate the chromosome foreground and background in the grayscale processed chromosome metaphase image.

[0085] Karyotype processing is performed on the grayscale processed chromosome metaphase image to separate the chromosome foreground and background. Specifically, this can be implemented as follows:

[0086] Calculate the gray-level histogram of the image and apply Gaussian smoothing to reduce noise. Based on the smoothed histogram, identify the highest peak value as the background gray-level value and dynamically adjust the threshold according to the background value.

[0087] Morphological closing and dilation operations are applied to optimize binary images, remove noise, and enhance the connectivity of chromosome regions.

[0088] The third step is to extract the outer contour, select contours with reasonable areas, and apply dilation and median blur to generate a foreground mask.

[0089] In this step, the outer contour is extracted based on the binary image. An outer contour extraction mode is used, and contour points are simplified. Contours are filtered based on a dynamic threshold of 1 / 15000 of the image area and the median area, retaining contours with areas within a reasonable range and removing noisy contours that are too small or too large. Dilation and median blur are applied to smooth the contour boundaries, generating a foreground mask for region isolation in subsequent enhancement operations.

[0090] Step 4: Differentiate between dark and light backgrounds based on the background brightness threshold. For dark backgrounds, use high-frequency and low-frequency separation, extract low-frequency components through mean blurring, calculate high-frequency components and weight them for enhancement. For light background images, use contour masking and CLAHE processing.

[0091] This step categorizes the image into two cases—dark background and light background—based on the average background grayscale value, and applies different enhancement strategies to each. For the dark background image, firstly, high-frequency and low-frequency components are separated. The low-frequency component is extracted using mean blurring, and the high-frequency component is calculated and weighted for enhancement. Subsequently, the multi-scale Retinex algorithm is applied, using multiple Gaussian blur scales to calculate logarithmic differences, enhancing details in low-contrast areas. Then, contour masks are used to confine the enhancement results to chromosome regions, and the background is filled with white. Finally, adaptive histogram equalization (CLAHE, clipping limit 3, grid size 10x10) is applied to further enhance contrast. The calculation formula is as follows:

[0092] ;

[0093] ;

[0094] in, This indicates the number of pixels in the input grayscale image. The intensity value. This represents the result of blurring the input image using a Gaussian blur kernel, where the standard deviation is... . This represents a small constant, used to prevent invalid values ​​in division by zero or logarithmic operations. Indicates the number of scales, corresponding to ; Represents the logarithmic difference of the Retinex enhancement results; This represents the pixel values ​​of the final enhanced image.

[0095] For images with light-colored backgrounds, contour masks and CLAHE are applied directly to simplify the processing and preserve the original details.

[0096] Step 5: Perform adaptive contrast adjustment on each contour region to enhance the chromosome metaphase image.

[0097] Adaptive contrast adjustment is performed on each contour region. First, a black pixel threshold is calculated based on the contour area. A gray-level histogram is plotted within the contour region, and the accumulated gray values ​​reaching the threshold are identified as lower limits. Linear transformation and brightness shift are used to enhance the contrast of the contour region, and the results are applied to the corresponding regions in the original image. The entire image then undergoes gamma correction and S-shaped contrast adjustment to further optimize visual effects and feature saliency. The gamma correction calculation formula is as follows:

[0098] ;

[0099] in, Indicates the input pixel value, and γ represents the gamma value. This represents the pixel value after gamma correction. The final image is resized back to its original size, and the enhanced image is saved.

[0100] See Figure 3 , Figure 3 This is a schematic diagram of an enhanced metaphase chromosome image in an embodiment of this application. It can be seen that the enhanced metaphase chromosome image improves contrast and detail, significantly enhances the visual clarity of chromosome regions, highlights chromosome edges and internal structure, and facilitates feature extraction. Through adaptive background enhancement, chromosomes against dark and light backgrounds are effectively separated, reducing background interference.

[0101] Step 103: Obtain the chromosome karyotype arrangement map based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generate the corresponding second annotation information.

[0102] The specific implementation of generating chromosome karyotype arrangement maps based on enhanced metaphase images in this application is not limited. However, while generation is typically based on metaphase images, this application focuses on generating enhanced metaphase images.

[0103] See Figure 4 , Figure 4 This is a schematic diagram of chromosome karyotype arrangement obtained from an enhanced metaphase image of chromosomes in an embodiment of this application. Figure 4 The chromosomes are arranged sequentially according to their classification numbers, and the classification numbers are marked, indicating the X chromosome and the Y chromosome.

[0104] Step 104: Generate a chromosome set based on the chromosome karyotype arrangement diagram and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes.

[0105] Step 105: The chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and the third annotation information corresponding to the chromosome foreground image is generated.

[0106] Step 106: The chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome metaphase image.

[0107] The background image randomly selected here is any image from the image set, which contains multiple images that do not contain chromosomes or cells.

[0108] A randomly selected background image is resized to match the chromosome foreground image. The chromosome foreground image is then overlaid onto the background image using a mask. Gaussian blur is applied to smooth the image edges, and a weighted average is used to fuse the foreground and background, generating a random background chromosome metaphase image.

[0109] See Figure 5 , Figure 5 This is a schematic diagram of a random background chromosome metaphase map in an embodiment of this application. Figure 5 To generate random background chromosome mid-maps by merging enhanced chromosomes and fusing them with random background maps, the generated random background chromosome mid-maps are diverse and of high quality. When used to train deep learning-based object detection models, they can improve the generalization ability and accuracy of deep learning models in object detection tasks.

[0110] Step 107: Regenerate the fourth annotation information corresponding to the random background chromosome mid-phase image based on the random background chromosome mid-phase image and the third annotation information, and store it; wherein, the random background chromosome mid-phase image and the corresponding fourth annotation information are used to train the target detection model based on deep learning.

[0111] After processing the chromosomes, RLE codes need to be generated based on the mask data of each chromosome, their bounding boxes and area information need to be calculated, and annotation files need to be constructed.

[0112] Each annotation information in the embodiments of this application typically includes: category number, segmentation mask, area, bounding box field, etc. In actual use, the content of the chromosome annotation information can be increased or decreased according to specific actual needs, and there is no limitation on this.

[0113] After generating random background chromosome mid-phase images and corresponding annotation information, they are saved accordingly. These random background chromosome mid-phase images can be one or more, and a storage path is specified. These random background chromosome mid-phase images can be used to train a target detection model based on deep learning.

[0114] In this embodiment, the chromosome metaphase image is first enhanced using a chromosome adaptive multi-scale image enhancement method. Then, a karyotype arrangement map is generated based on the enhanced chromosome metaphase image. Next, a chromosome set containing merged and independent chromosomes is generated based on the chromosome karyotype arrangement map. Finally, the chromosomes in this set are merged and used as a foreground image, which is then fused with a randomly selected background image to generate a random background chromosome metaphase image. This random background chromosome metaphase image is used to train a deep learning-based object detection model. This method can generate high-quality and diverse chromosome metaphase images, which, when used to train a deep learning model, can improve the generalization ability and accuracy of the deep learning model in object detection tasks.

[0115] In another example,

[0116] See Figure 6 , Figure 6This is a schematic diagram of the chromosome set generation process in an embodiment of this application. The specific steps are as follows:

[0117] Step 601: Sort the corresponding chromosomes according to the aspect ratio of the bounding boxes of the chromosomes in the chromosome karyotype arrangement diagram in ascending order, and divide the sorted chromosomes evenly into the first half group and the second half group.

[0118] The aspect ratio of the bounding box of the chromosome is determined based on the second annotation information corresponding to that chromosome.

[0119] When dividing chromosomes equally, if they cannot be perfectly divided (i.e., the number of chromosomes is odd), the middle chromosomes can be assigned to the first half of the group. This first half is used for crossover operations and can be called the crossoverable group; the second half is used for joining operations and can be called the joinable group. For example, if there are 5 chromosomes: chromosome 1, chromosome 2, chromosome 3, chromosome 4, and chromosome 5; and if these five chromosomes are ordered from smallest to largest aspect ratio as: chromosome 5, chromosome 3, chromosome 2, chromosome 4, and chromosome 1; then chromosomes 5, 3, and 2 are assigned to the first half of the group; and chromosome 4 and 1 are assigned to the second half of the group.

[0120] The connection operation here simulates the contact or adhesion of two or more chromosomes at the chromosome edge. This may result in a V-shape or chromosomes joining head to tail.

[0121] Crossover is a process that simulates the physical overlap or stacking of two or more chromosomes.

[0122] Step 602: Two or three chromosomes in the first half of the group are merged through a crossover operation to obtain the first type of merged chromosomes.

[0123] In this embodiment, a first preset number of chromosomes are randomly selected from the first half of the group and a crossover operation is performed. The first preset number of chromosomes can be any value from 2 to 5, such as 2, 3, 4 or 5. The crossover operation is performed to merge two or three chromosome images, wherein the probability of merging two chromosomes and merging three chromosomes is 40% and 60%, respectively.

[0124] Step 603: The chromosomes that failed to merge in the first half group are assigned to the second half group, and two or three chromosomes in the updated second half group are merged by a joining operation to obtain the second type of merged chromosomes.

[0125] Unmerged single chromosomes in the first half of the group are merged into the second half of the group. If all chromosomes in the first half of the group are successfully merged and there are no single chromosomes, the connection operation of the second half of the group is performed directly. A second preset number of chromosomes is randomly selected from the updated second half of the group. The second preset number of chromosomes can be any number from 3 to 7. The connection operation is performed on the selected chromosomes to merge two or three chromosomes. The probability of merging two chromosomes and merging three chromosomes is 50% each.

[0126] Step 604: Perform a trimming operation on the individual chromosomes that failed to merge in the updated second half group to obtain independent chromosomes.

[0127] Step 605: Apply random rotations to the first type of merged chromosomes, the second type of merged chromosomes, and the independent chromosomes to generate a chromosome set, and generate the labeling information corresponding to the chromosomes in the chromosome set.

[0128] This completes the generation of the chromosome set.

[0129] See Figure 7 , Figure 7 This is a schematic diagram illustrating the chromosome foreground image generation process in an embodiment of this application. The specific steps are as follows:

[0130] Step 701: Using the first chromosome in the chromosome set as the basis for merging, determine the placement position of the chromosome while controlling the chromosome overlap ratio.

[0131] The specific process for determining the placement of chromosomes is as follows:

[0132] Randomly select one chromosome from the set of chromosomes, such as the first chromosome, as the basis for merging, and then merge all chromosomes in sequence, but the chromosome overlap ratio must be considered when merging.

[0133] During the merging process, the horizontal distance between the two chromosomes is calculated based on the mask to control the overlap ratio, such as controlling the overlap ratio between any two chromosomes to be no more than 50%.

[0134] The overlap ratio here can be calculated as the ratio of the area of ​​the overlapping portion of the two chromosomes to the area of ​​the chromosome with the smaller area.

[0135] Step 702: Offset the chromosome at the determined location and align the image size using a padding operation.

[0136] This step involves shifting the chromosome at the determined location, specifically including:

[0137] For each chromosome, a positive offset within a first preset offset range is applied with a first preset probability, and a negative offset within a second preset offset range is applied with a second preset probability; wherein, the first preset probability is greater than the second preset probability, and the sum of the first preset probability and the second preset probability is 1; the first preset offset range is less than the second preset offset range.

[0138] In practical implementation, the first preset probability can be set to 80% and the second preset probability to 20%. The first preset range can be regarded as a small range offset and the second preset range can be regarded as a large range. The effect is to apply a small range offset with an 80% probability and apply a large negative offset with a 20% probability.

[0139] Align image dimensions using padding operations, and merge images and their masks.

[0140] Step 703: After placing each offset chromosome onto the canvas, apply a phased rotation to the canvas and merge all the chromosomes placed in sequence into one image as the chromosome foreground image.

[0141] When applying phased rotations to the canvas, set the rotation angle of the initial rotation phase, the rotation phase length of the initial rotation phase, and the rules for decreasing the rotation angle and increasing the rotation phase length; the rotation phase length represents the number of chromosomes corresponding to each rotation phase.

[0142] In a specific embodiment of this application, a relatively short rotation stage length is initially set, such as 3, indicating that this rotation stage length corresponds to 3 chromosomes. That is, the rotation stage at which the first to third chromosomes are added is determined as the initial rotation stage. An initial, relatively large rotation angle is set in the initial rotation stage, such as 90 degrees. That is, the rotation angle determined when adding the first to third chromosomes is 90 degrees.

[0143] When the number of objects added to the canvas reaches the length of this rotation phase, for example, after adding 3 objects, a rotation phase switch occurs. During the switch, first, the rotation amplitude is reduced, decreasing the rotation angle, for example, from 50 degrees to 45 degrees; second, the length of the rotation phase required to trigger the next switch is doubled, for example, now requiring 7 more new objects instead of 3; the chromosome uses this new, smaller angle, such as 45 degrees, to rotate; after adding 7 new objects and completing 7 45-degree rotations, the system triggers another switch, halving the rotation angle again, for example, to around 23 degrees, and doubling the length of the next rotation phase again, for example, requiring 15 new objects, and so on, until all chromosomes are merged.

[0144] The process of applying a phased rotation to all shifted chromosomes includes the following steps:

[0145] Step 1: When any current chromosome is added to the main canvas, the rotation phase of the current chromosome is determined based on the number of chromosomes added to the canvas, the length of the initial rotation phase, and the lengthening rules of the rotation phase.

[0146] If the chromosome to be added to the main canvas is the 4th chromosome, then the current rotation phase of the chromosome is determined to be the second rotation phase.

[0147] The second step is to determine the rotation angle based on the current rotation stage of the chromosome and the rules for reducing the rotation angle.

[0148] If the current rotation stage of the chromosome is determined to be the second rotation stage, then the rotation angle is determined to be 45 degrees.

[0149] Step 3: Rotate the canvas using the determined rotation angle.

[0150] All chromosomes placed sequentially are merged into a single image as the chromosome foreground image. After merging, the images are integrated into a complete image, the target size is randomly expanded to 1 to 2 times the original size, and all chromosomes in the complete image are simultaneously subjected to random shifts (based on the size of the fill region).

[0151] See Figure 8 , Figure 8 This is a schematic diagram illustrating the process of generating a single chromosome image set in an embodiment of this application. The specific steps are as follows:

[0152] Step 801: Extract single chromosome images from the enhanced metaphase chromosome image based on the first annotation information. Proceed to step 803.

[0153] Step 802: Extract single chromosome images from random background chromosome metaphase images based on the fourth annotation information.

[0154] In the specific implementation, a set of valid image-mask pairs is randomly selected as the target image and its segmentation mask according to the annotation information. Based on the position of the segmentation mask, individual chromosomes are extracted, generating a single chromosome image. Specifically, during extraction, the mask is binarized into foreground and background using an adaptive threshold based on the mask value distribution. An erosion operation is applied to reduce the mask boundary to avoid interference from edge pixels. The chromosome region is extracted from the source image using the eroded mask, and bitwise operations are used to retain the pixels in the mask-covered area.

[0155] Step 803: Generate a single chromosome image set, and the annotation information corresponding to each chromosome in the single chromosome image set; the single chromosome image set includes single chromosome images extracted from enhanced chromosome metaphase images and random background chromosome metaphase images.

[0156] In the embodiments of this application, the generated single chromosome image set contains two types of images: one type consists entirely of enhanced chromosome metaphase images, and the other type consists of random background chromosome metaphase images; this results in a greater diversity of chromosomes.

[0157] To address the issue of poor segmentation performance of overlapping chromosomes and unclear segmentation of the main chromosomes in segmentation algorithms, this application also generates a cross-overlapping chromosome image, see [link to relevant documentation]. Figure 9 , Figure 9 This application's embodiment illustrates the process for generating crossed-overlap chromosome images. The specific steps are as follows:

[0158] Step 901: Randomly select two chromosome images from the single chromosome image set, and denot them as the first chromosome image and the second chromosome image.

[0159] Step 902: Under the premise of meeting the overlap ratio requirement, determine the placement area of ​​the chromosome corresponding to the second chromosome image in the first chromosome image; the overlap ratio is less than the preset ratio value and greater than 0.

[0160] This step involves determining the placement area, including:

[0161] In the first chromosome image, a candidate region for placing the second chromosome corresponding to the second chromosome image is randomly selected;

[0162] Extract the mask of the location region corresponding to the first chromosome in the first chromosome image, and compare it with the eroded source mask to calculate the overlapping area;

[0163] The ratio of the overlapping area to the smaller mask area is determined as the overlap ratio;

[0164] If the overlap ratio is less than the preset ratio value but greater than 0, then the candidate area is determined as the placement area;

[0165] If the overlap ratio is not less than the preset ratio value, or is 0, then the candidate region for placing the second chromosome corresponding to the second chromosome image is randomly selected again, until a candidate region with an overlap ratio less than the preset ratio value and greater than 0 is selected and determined as the placement region.

[0166] In practice, if candidate regions are not successfully identified multiple times, the generation of the current image can be terminated, and two single chromosome images can be randomly selected again to generate a cross-over chromosome image. If the number of candidate regions identified reaches 20, the generation of the current image can be terminated.

[0167] Step 903: The chromosome pixels corresponding to the second chromosome image are covered to the placement area through bit operations, and the first chromosome image is updated to generate a cross-overlapping chromosome image and the corresponding annotation information; wherein, the cross-overlapping chromosome image and the corresponding annotation information are used to train the image segmentation model based on deep learning, and the cross-overlapping means that the two chromosomes partially overlap.

[0168] The annotation information here includes at least the segmentation mask.

[0169] See Figure 10 , Figure 10 This is a schematic diagram of crossed and overlapping chromosomes in an embodiment of this application. Figure 10 The crossed-overlap chromosome image in this example is one of the crossed-overlap chromosome images generated according to this embodiment. This crossed-overlap chromosome image shows clear chromosome segmentation and good results.

[0170] The method for generating cross-overlapping chromosome images provided in this embodiment can improve the clarity of chromosome segmentation, and generate diverse cross-overlapping chromosome images by randomly crossing different chromosomes. The generated cross-overlapping chromosome images can be used to train an image segmentation model based on deep learning, thereby improving the generalization ability and accuracy of the deep learning model in image segmentation tasks.

[0171] In practical implementation, the masked area can also be overlaid on the target image in a red transparent form to generate a visual image; this visual image can be used for debugging and verification.

[0172] To generate standardized chromosome images for easier feature extraction and anomaly classification, this application also proposes a method for straightening chromosomes, as well as a straightened and then bent chromosome image. See [link to relevant documentation]. Figure 11 , Figure 11 This application's embodiment illustrates the process for generating images of straightened, bent chromosomes. The specific steps are as follows:

[0173] Step 1101: Randomly select a chromosome image from the single chromosome image set, and extract the contour point set based on the mask data in the annotation information of the chromosome image.

[0174] The extracted contour point set can be used for subsequent skeleton extension.

[0175] Step 1102: Extract the chromosome skeleton based on the mask data and the set of contour points.

[0176] The specific steps for extracting the chromosome backbone in this process include the following:

[0177] The first step is to perform morphological smoothing on the mask data.

[0178] To eliminate jagged edges and noise at the segmentation edges, the mask is morphologically smoothed. Specifically, a strategy of closing operations followed by opening operations can be adopted, using a convolution kernel of a preset size (e.g., 3x3) to fill in the tiny holes inside the mask and smooth the edge burrs, providing high-quality input for subsequent skeleton extraction.

[0179] The second step is to perform a median transformation on the smoothed mask data to extract the central skeleton line of the corresponding chromosome.

[0180] Apply an axial transformation to the smoothed mask to extract the central skeleton line of the target. The original skeleton usually contains many short branches caused by contour noise. Use an iterative algorithm to detect and remove terminal branches with a length less than a preset threshold, such as 10 pixels, while retaining the main structure.

[0181] To ensure the uniqueness of the skeleton, a graph structure of the skeleton pixels is constructed. Breadth-first search can be used to calculate the path distance between all endpoint pairs, retain only the path with the longest distance as the main skeleton, and eliminate all remaining loops or minor branches.

[0182] The third step is to transform the central skeleton line into an ordered geometric curve to obtain the skeleton center line.

[0183] The extracted skeleton is an unordered set of pixels and needs to be transformed into an ordered geometric curve. An 8-neighborhood depth-first search algorithm can be used to trace the entire path starting from one endpoint of the skeleton, converting discrete pixel coordinates into an ordered list of points. To address the discontinuities and jagged edges in the pixel-level skeleton, B-splines are used to fit the ordered point list. Then, a one-dimensional Gaussian filter is applied to the interpolated coordinates to remove local jitter, generating a smooth, continuous skeleton curve with uniform point density.

[0184] Step 4: Extend the skeleton from the inside to the physical edge of the corresponding chromosome to obtain the chromosome skeleton; where the physical edge is determined based on the contour point set.

[0185] To prevent the straightened image from losing its beginning and end points, the skeleton needs to be extended from the inside to the physical edge of the target. A certain number of neighborhood points are selected at the head and tail of the skeleton, for example, with a window size of 10. The average tangent direction vector is calculated and normalized. Compared to using only the last two points, using the window neighborhood effectively reduces noise and makes the orientation more stable.

[0186] New skeleton points are iteratively added along the calculated tangent direction, with a preset step size, such as 1 pixel. At each step, it is checked whether the new point is within the target mask. Extension stops immediately once a new point touches or exceeds the mask boundary. This step ensures that the skeleton extends across the entire geometric length of the target.

[0187] Step 1103: Based on the chromosome skeleton and mask data, straighten the chromosome, obtain the straightened chromosome image, and generate the corresponding annotation information of the chromosome image.

[0188] The specific implementation of straightening the chromosome based on the chromosome skeleton and mask data in this step is not limited. The following implementation method is given in the embodiments of this application:

[0189] The first step is to determine the total arc length of the chromosomal skeleton based on the chromosomal skeleton data, and then define the size of the straightened output image based on the total arc length of the chromosomal skeleton and the preset target width.

[0190] The image is transformed from Cartesian coordinates to the straightened coordinate system. The tangent vector at each point on the extended skeleton curve is calculated, and the corresponding normal vector is obtained by rotating it 90 degrees. The width of the output image is defined as the target width, and the height as the arc length of the skeleton.

[0191] The second step is to parameterize the chromosome skeleton using arc length as a parameter and calculate the unit normal vector at each point on the skeleton.

[0192] For each pixel (u, v) in the output image, u represents the length along the skeleton direction and v represents the distance along the normal direction.

[0193] The third step is to determine the corresponding source coordinates of each pixel in the output image in the original image based on the coordinate mapping relationship.

[0194] Use the formula Psrc=Pskel(u) + v N(u) calculates its corresponding coordinates Psrc in the original image. Here, Pskel(u) is the point on the skeleton corresponding to the arc length, and v... N(u) is the unit normal vector at that point.

[0195] The fourth step is to resample the original image based on the coordinate mapping relationship to generate a straightened intermediate image.

[0196] Using the calculated coordinate mapping table (Map_x, Map_y), bilinear interpolation can be used to resample the pixels of the original image.

[0197] Step 5: Fill the area in the intermediate image that corresponds to the area outside the mask data with the background color to generate the straightened chromosome image.

[0198] For areas that extend beyond the boundaries of the original image, fill them with a white background (255, 255, 255), ultimately generating a straightened, elongated image.

[0199] Step 1104: Bend the chromosomes in the straightened chromosome image to obtain the bent chromosome image, and generate the corresponding annotation information for the bent chromosome image; wherein, the straightened chromosome image and the bent chromosome image, as well as the corresponding annotation information, are used to train the anomaly classification model based on deep learning.

[0200] This step involves bending the chromosomes in the straightened chromosome image, specifically including:

[0201] First, you need to define the deformation parameters: set the preset bending ratio and preset bending angle; the bending here starts from the midpoint of the skeleton.

[0202] The first step is to define the coordinate space of the straightened chromosome image as a set of planar points.

[0203] The second step is to determine the rotation center based on the annotation information of the straightened chromosome image. For points located after the preset bending ratio, a two-dimensional rotation matrix with the preset bending angle is applied to perform coordinate transformation to generate the target point set after bending.

[0204] The preset bending ratio is represented by the value of the Y-coordinate in the coordinate system, thereby determining which points are points after the preset bending ratio.

[0205] The third step is to calculate the extreme values ​​of the coordinates of all points in the target point set, and then dynamically create an output canvas with an adaptive size based on the extreme values ​​of the coordinates of all points.

[0206] This step ensures that the bent target is not truncated.

[0207] Step 4: Establish the inverse mapping relationship between the output canvas coordinates and the straightened image coordinates.

[0208] Step 5: Based on this inverse mapping relationship, rotate each coordinate point in the output canvas inversely back to the coordinate system of the straightened chromosome image.

[0209] Step 6: If the coordinates after the inverse transformation are within the range of the straightened chromosome image, then perform pixel sampling and assign the sampled values ​​to the corresponding coordinate points on the output canvas; and output the simulated image after pixel sampling as the bent chromosome image.

[0210] See Figure 12 , Figure 12 This is a schematic diagram of straightening and bending images in the embodiments of this application. Figure 12The leftmost image is a single chromosome image from the single chromosome image set, the middle image is a straightened chromosome image after the single chromosome image on the left has been straightened, and the rightmost image is a curved chromosome image after the straightened chromosome image in the middle has been bent.

[0211] This embodiment not only generates straightened chromosome images but also curved chromosome images based on the straightened chromosome images. Generating both straightened and curved chromosome images based on enhanced single chromosomes in this embodiment increases sample diversity and improves image quality. This, in turn, enhances the generalization ability and accuracy of deep learning models in image classification tasks when used to train deep learning-based anomaly classification models.

[0212] The images generated in this embodiment significantly improve image contrast and detail clarity, i.e., improve image quality: through multi-scale Retinex algorithm, CLAHE adaptive histogram equalization and adaptive contrast adjustment, the edge and internal structural details of chromosome regions are significantly enhanced, image quality is optimized, and feature extraction by subsequent deep learning algorithms is facilitated.

[0213] Enhancing Data Diversity and Realism: By performing random crossover, concatenation, and background fusion operations, a diverse dataset of chromosome metaphase images is generated, effectively simulating overlap, morphological changes, and background interference in real-world scenes. This overcomes the shortcomings of traditional methods, such as limited data and unrealistic simulations, and significantly enhances the generalization ability of deep learning models.

[0214] Improving the robustness of segmentation tasks: By precisely controlling the chromosome overlap ratio to generate image data that is both realistic and segmentable, the performance of the chromosome segmentation algorithm can be optimized, enabling the model to better cope with complex overlapping scenes and improve segmentation accuracy.

[0215] Enhanced accuracy of feature extraction: By employing continuous skeleton extraction, extension to contour boundaries, and geometric straightening techniques, standardized chromosome images are generated, simplifying the interference of complex morphology on feature extraction and effectively improving the accuracy and reliability of anomaly classification tasks.

[0216] Supports diverse data augmentation: Generates chromosome images with diverse morphologies through various transformations, further enriching the training dataset and enhancing the deep learning model's adaptability to different chromosome morphologies.

[0217] Improve automated processing efficiency: Integrate multiple format annotation generation, batch image processing and visualization functions to achieve an efficient automated data generation process, reduce manual intervention, improve data preparation efficiency, and provide reliable support for genetic research, prenatal screening and cancer diagnosis.

[0218] The generated augmented data and normalized images are suitable for a variety of deep learning tasks, including object detection, image segmentation, and anomaly classification, significantly improving their application in bioinformatics and medical diagnostics.

[0219] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0220] Based on the same inventive concept, this application also provides a chromosome image generation apparatus for deep learning. See also Figure 13 , Figure 13 This is a schematic diagram of a chromosome image generation device for deep learning, as described in an embodiment of this application. The device includes:

[0221] The first acquisition unit 1301 is used to acquire chromosome metaphase images;

[0222] The enhancement processing unit 1302 is used to perform adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information.

[0223] The second acquisition unit 1303 is used to acquire a chromosome karyotype arrangement map based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generate the corresponding second annotation information;

[0224] The first generation unit 1304 is used to generate a chromosome set based on the chromosome karyotype arrangement map and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes; the chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and the third annotation information corresponding to the chromosome foreground image is generated; the chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome mid-phase image; the fourth annotation information corresponding to the random background chromosome mid-phase image is regenerated based on the random background chromosome mid-phase image and the third annotation information, and stored; wherein, the random background chromosome mid-phase image and the corresponding fourth annotation information are used to train a target detection model based on deep learning.

[0225] In another example,

[0226] The first generation unit 1304 is specifically used to sort the corresponding chromosomes in ascending order according to the aspect ratio of the bounding boxes of the chromosomes in the chromosome karyotype arrangement diagram, and to evenly divide the sorted chromosomes into the first half group and the second half group; to merge two or three chromosomes in the first half group through a crossover operation to obtain the first type of merged chromosomes; to divide the chromosomes in the first half group that could not be merged into the second half group, and to merge two or three chromosomes in the updated second half group through a connection operation to obtain the second type of merged chromosomes; to perform a pruning operation on the single chromosomes in the updated second half group that could not be merged to obtain independent chromosomes; to apply random rotation to the first type of merged chromosomes, the second type of merged chromosomes, and the independent chromosomes to generate a chromosome set, and to generate the labeling information corresponding to the chromosomes in the chromosome set.

[0227] In another example,

[0228] The first generation unit 1304 is specifically used to take the first chromosome in the chromosome set as the base object for merging, determine the chromosome placement position under the premise of controlling the chromosome overlap ratio; offset the chromosome at the determined position and align the image size through a filling operation; after placing each offset chromosome on the canvas, apply a staged rotation to the canvas, and merge all the chromosomes placed in sequence into one image as the chromosome foreground image.

[0229] In another example, the device further includes:

[0230] The second generation unit 1305 is used to extract single chromosome images from the enhanced chromosome metaphase image based on the first annotation information; extract single chromosome images from the random background chromosome metaphase image based on the fourth annotation information; generate a single chromosome image set, and annotation information corresponding to each chromosome in the single chromosome image set; the single chromosome image set includes single chromosome images extracted from the enhanced chromosome metaphase image and the random background chromosome metaphase image.

[0231] In another example, the device further includes:

[0232] The third generation unit 1306 is used to randomly select two chromosome images from a single chromosome image set, denoted as the first chromosome image and the second chromosome image; under the premise of meeting the overlap ratio requirement, it determines the placement area of ​​the chromosome corresponding to the second chromosome image in the first chromosome image; the overlap ratio is less than a preset ratio and greater than 0; through bit operations, the chromosome pixels corresponding to the second chromosome image are covered to the placement area, and the first chromosome image is updated to generate a cross-overlapping chromosome image and the corresponding annotation information; wherein, the cross-overlapping chromosome image and the corresponding annotation information are used to train the image segmentation model based on deep learning, and the cross-overlapping means that the two chromosomes partially overlap.

[0233] In another example, the device further includes:

[0234] The fourth generation unit 1307 is used to select a chromosome image from a single chromosome image set, extract the contour point set of the chromosome in the chromosome image based on the mask data in the annotation information of the chromosome image; extract the chromosome skeleton based on the mask data and the contour point set; straighten the chromosome based on the chromosome skeleton and the mask data to obtain a straightened chromosome image and generate the corresponding annotation information of the chromosome image; bend the chromosome in the straightened chromosome image to obtain a bent chromosome image and generate the corresponding annotation information of the bent chromosome image; wherein, the straightened chromosome image and the bent chromosome image, as well as the corresponding annotation information, are used to train an anomaly classification model based on deep learning.

[0235] The fourth generation unit 1307 is specifically used to define the coordinate space of the straightened chromosome image as a planar point set; determine the rotation center based on the annotation information of the straightened chromosome image, and for points located after a preset bending ratio, apply a two-dimensional rotation matrix with a preset bending angle to perform coordinate transformation to generate a bent target point set; calculate the coordinate extreme values ​​of all points in the target point set, and dynamically create an output canvas with adaptive size based on the coordinate extreme values ​​of all points; establish an inverse mapping relationship from the output canvas coordinates to the straightened image coordinates; based on the inverse mapping relationship, rotate each coordinate point in the output canvas back to the coordinate system of the straightened chromosome image; if the coordinates after the inverse transformation are within the range of the straightened chromosome image, perform pixel sampling and assign the sampled values ​​to the corresponding coordinate points on the output canvas; and output the simulated image after pixel sampling as the bent chromosome image.

[0236] The units in the above embodiments can be integrated into one unit or deployed separately; they can be merged into one unit or further divided into multiple sub-units.

[0237] In another embodiment, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a chromosome image generation method for deep learning.

[0238] In another embodiment, a computer-readable storage medium is also provided, on which computer instructions are stored, which, when executed by a processor, implement a chromosome image generation method for deep learning.

[0239] Figure 14 This is a schematic diagram of the physical structure of the electronic device provided in an embodiment of this application. Figure 14 As shown, the electronic device may include: a processor 1410, a communications interface 1420, a memory 1430, and a communication bus 1440, wherein the processor 1410, the communications interface 1420, and the memory 1430 communicate with each other through the communication bus 1440. The processor 1410 can call logical instructions in the memory 1430 to execute the following methods:

[0240] Obtain images of chromosome metaphase;

[0241] Adaptive multi-scale image enhancement processing is performed on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information;

[0242] Based on the enhanced metaphase image of the chromosome and the corresponding first annotation information, a chromosome karyotype arrangement map is obtained, and the corresponding second annotation information is generated;

[0243] A chromosome set is generated based on the chromosome karyotype arrangement diagram and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes;

[0244] The chromosomes in the chromosome set are sequentially merged into one image as the chromosome foreground image, and the third annotation information corresponding to the chromosome foreground image is generated.

[0245] The chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome metaphase image.

[0246] The fourth annotation information corresponding to the random background chromosome mid-phase image is regenerated based on the random background chromosome mid-phase image and the third annotation information, and then stored; wherein, the random background chromosome mid-phase image and the corresponding fourth annotation information are used to train the target detection model based on deep learning.

[0247] Furthermore, the logical instructions in the aforementioned memory 1430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0248] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0250] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments disclosed in this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings. For example, two blocks shown connectedly may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0251] Those skilled in the art will understand that the features described in the various embodiments and / or claims disclosed in this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, and all such combinations and / or combinations fall within the scope of this application.

[0252] This document uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of these embodiments are merely illustrative of the method and core concepts of the present invention and are not intended to limit this application. Those skilled in the art can make changes to the specific implementation methods and application scope based on the ideas, spirit, and principles of the present invention. Any modifications, equivalent substitutions, or improvements made should be included within the scope of protection of this application.

Claims

1. A method for generating chromosome images using deep learning, characterized in that, The method includes: Obtain images of chromosome metaphase; Adaptive multi-scale image enhancement processing is performed on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information; Based on the enhanced metaphase image of the chromosome and the corresponding first annotation information, a chromosome karyotype arrangement map is obtained, and corresponding second annotation information is generated; A chromosome set is generated based on the chromosome karyotype arrangement diagram and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes; The chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and the third annotation information corresponding to the chromosome foreground image is generated. The chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome metaphase image. Based on the random background chromosome midphase image and the third annotation information, the fourth annotation information corresponding to the random background chromosome midphase image is regenerated and stored; wherein, the random background chromosome midphase image and the corresponding fourth annotation information are used to train a target detection model based on deep learning; The first annotation information, the second annotation information, the third annotation information, and the fourth annotation information all contain the corresponding chromosome category number, segmentation mask, and bounding box; The generation of a chromosome set based on the chromosome karyotype arrangement map and the corresponding second annotation information includes: Based on the aspect ratio of the bounding boxes of the chromosomes in the chromosome karyotype arrangement diagram, the corresponding chromosomes are sorted in ascending order, and the sorted chromosomes are evenly divided into the first half group and the second half group; the aspect ratio of the bounding boxes is determined according to the second annotation information corresponding to the chromosomes; Two or three chromosomes in the first half of the group are merged through a crossover operation to obtain the first type of merged chromosomes; Chromosomes that failed to merge in the first half group are assigned to the second half group, and two or three chromosomes in the updated second half group are merged by a joining operation to obtain the second type of merged chromosomes. The individual chromosomes that failed to merge in the updated second half of the group were pruned to obtain independent chromosomes. Random rotations are applied to the first type of merged chromosomes, the second type of merged chromosomes, and the independent chromosomes to generate a chromosome set, and labeling information corresponding to the chromosomes in the chromosome set is generated.

2. The method according to claim 1, characterized in that, The step of sequentially merging the chromosomes in the chromosome set into a single image as a chromosome foreground image includes: Using the first chromosome in the chromosome set as the base object for merging, the placement position of the chromosome is determined while controlling the chromosome overlap ratio; The chromosome at the determined location is offset, and the image size is aligned by a padding operation; After each offset chromosome is placed on the canvas, the canvas is rotated in stages, and all the chromosomes placed in sequence are merged into one image as the chromosome foreground image.

3. The method according to claim 1, characterized in that, The method further includes: Extract single chromosome images from the enhanced metaphase chromosome image based on the first annotation information; Based on the fourth annotation information, extract a single chromosome image from the random background chromosome metaphase image; Generate a set of single chromosome images, and annotation information corresponding to each chromosome in the set of single chromosome images; the set of single chromosome images includes single chromosome images extracted from enhanced chromosome metaphase images and random background chromosome metaphase images.

4. The method according to claim 3, characterized in that, The method further includes: Two chromosome images are randomly selected from the set of single chromosome images and denoted as the first chromosome image and the second chromosome image. Provided that the overlap ratio requirement is met, a placement area for the chromosome corresponding to the second chromosome image is determined in the first chromosome image; the overlap ratio is less than a preset ratio and greater than 0. Bitwise operations are used to overlay the chromosome pixels corresponding to the second chromosome image onto the placement area, and the first chromosome image is updated to generate a cross-overlapping chromosome image and corresponding annotation information; The overlapping chromosome images and corresponding annotation information are used to train an image segmentation model based on deep learning. The overlapping refers to the partial overlap of two chromosomes.

5. The method according to claim 3, characterized in that, The method further includes: Select one chromosome image from the set of single chromosome images, and extract the set of chromosome contour points in the chromosome image based on the mask data in the annotation information of the chromosome image; The chromosome skeleton of the chromosome is extracted based on the mask data and the contour point set; Based on the chromosome skeleton and the mask data, the chromosome is straightened to obtain a straightened chromosome image, and annotation information corresponding to the chromosome image is generated. The chromosomes in the straightened chromosome image are bent to obtain a bent chromosome image, and annotation information corresponding to the bent chromosome image is generated; wherein, the straightened chromosome image, the bent chromosome image, and the corresponding annotation information are used to train an anomaly classification model based on deep learning.

6. The method according to claim 5, characterized in that, The bending process of chromosomes in the straightened chromosome image includes: The coordinate space of the straightened chromosome image is defined as a set of planar points; The rotation center is determined based on the annotation information of the straightened chromosome image, and for points located after the preset bending ratio, a two-dimensional rotation matrix with a preset bending angle is applied to perform coordinate transformation to generate the target point set after bending. Calculate the extreme values ​​of the coordinates of all points in the target point set, and dynamically create an output canvas with adaptive size based on the extreme values ​​of the coordinates of all points; Establish an inverse mapping relationship between the output canvas coordinates and the straightened image coordinates; Based on the inverse mapping relationship, the coordinate system of the straightened chromosome image is rotated back in reverse for each coordinate point in the output canvas. If the coordinates after the inverse transformation are within the range of the straightened chromosome image, then pixel sampling is performed and the sampled values ​​are assigned to the corresponding coordinate points on the output canvas; and the output simulated image after pixel sampling is used as the bent chromosome image.

7. A chromosome image generation apparatus for deep learning, characterized in that, The device includes: The first acquisition unit is used to acquire images of chromosome metaphase. An enhancement processing unit is used to perform adaptive multi-scale image enhancement processing on the chromosome metaphase image to obtain the enhanced chromosome metaphase image and the corresponding first annotation information; The second acquisition unit is used to acquire a chromosome karyotype arrangement map based on the enhanced chromosome metaphase image and the corresponding first annotation information, and generate corresponding second annotation information; A generation unit is configured to generate a chromosome set based on the chromosome karyotype arrangement map and the corresponding second annotation information; the chromosome set includes merged chromosomes and independent chromosomes; the chromosomes in the chromosome set are sequentially merged into an image as a chromosome foreground image, and a third annotation information corresponding to the chromosome foreground image is generated; the chromosome foreground image is fused with a randomly selected background image, and the edges of the fused image are smoothed to generate a random background chromosome mid-phase image; a fourth annotation information corresponding to the random background chromosome mid-phase image is regenerated based on the random background chromosome mid-phase image and the third annotation information, and stored; wherein, the random background chromosome mid-phase image and the corresponding fourth annotation information are used to train a target detection model based on deep learning; The first annotation information, the second annotation information, the third annotation information, and the fourth annotation information all contain the corresponding chromosome category number, segmentation mask, and bounding box; The generation unit is specifically used to generate a chromosome set based on the chromosome karyotype arrangement diagram and the corresponding second annotation information. Specifically, it sorts the chromosomes according to the aspect ratio of the bounding boxes in the chromosome karyotype arrangement diagram in ascending order, and evenly divides the sorted chromosomes into a first half group and a second half group. The aspect ratio of the bounding boxes is determined according to the second annotation information corresponding to the chromosomes. It merges two or three chromosomes in the first half group through a crossover operation to obtain a first type of merged chromosome. Chromosomes that cannot be merged in the first half group are assigned to the second half group, and two or three chromosomes in the updated second half group are merged through a join operation to obtain a second type of merged chromosome. It performs a pruning operation on the single chromosomes that cannot be merged in the updated second half group to obtain independent chromosomes. It applies random rotations to the first type of merged chromosomes, the second type of merged chromosomes, and the independent chromosomes to generate a chromosome set, and generates the annotation information corresponding to the chromosomes in the chromosome set.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1-6.

Images