A highly controllable chromosome image generation method and system
Through structured mask composition and diffusion generation model, combined with chromosomal structured data, the efficient and controllable chromosome image generation from scratch, solving the problem of not flexible and controlling the in existing technology in the existing technology in the existing technology It improves the number of generics and analysis of the number of abnormal chromosome images.
Patent Information
- Application Number
- CN202411621279.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-01-28
- Estimated Expiration
- 2044-11-14
AI Technical Summary
Existing chromosome generation methods cannot generate chromosome images from scratch, and cannot control chromosome morphology, resulting in a sparse number of some abnormal chromosome images, making it difficult to perform effective analysis.
The chromosomal case is obtained through a structured mask compositioner, and the structured data of chromosomal is combined, and the chromosomal structured mask is generated in the order of the tape and the position of the silk grains. Use the diffusion generation model and control network to achieve efficient and controllable chromosome images from nothing.
The efficient and controllable chromosome image generation from scratch, solve the scarce problem of abnormal chromosomal images, and improve the reliability and efficiency of analysis.
Smart Images

Figure CN119131184B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a highly controllable chromosome image generation method and system. Background Art
[0002] Chromosome karyotype analysis is an important genetic testing technology used to observe and analyze the number and structure of chromosomes in cells. It is widely used in genetic research, prenatal diagnosis, infertility diagnosis and oncology research. This analysis is especially important in diagnosing diseases related to chromosomal abnormalities, such as Down syndrome, Turner syndrome and Klinefelter syndrome.
[0003] For the detection and analysis of abnormal chromosomes, usually after taking high-power images with a microscope, the chromosome karyotype analysis system is used for analysis. In the actual analysis process, some abnormal types will appear with low frequency, small amount of data, and the abnormal morphology is difficult to distinguish. Usually in this case, there is a lack of abnormal templates for comparison. Therefore, for such abnormal chromosomes, artificially synthesizing chromosomes of the corresponding abnormal type as templates for analysts to conduct comparative analysis is an effective solution.
[0004] The existing chromosome generation methods include splicing chromosome images. The advantages of this method are that it is simple and direct, and it uses real data for generation, so the data reliability is high. The disadvantages are that it requires the use of real chromosome images, cannot be created from scratch, and cannot control the chromosome morphology. Summary of the invention
[0005] The present invention provides a highly controllable chromosome image generation method and system, which can realize highly controllable chromosome image generation from scratch in a variety of scenarios, and solve the problem of the scarcity of some abnormal chromosome images.
[0006] In a first aspect, the present invention provides a highly controllable chromosome image generation method, the method comprising:
[0007] Step 1: The structured mask constructor in the chromosome generation module obtains the chromosome skeleton line; the method for obtaining the chromosome skeleton line is to directly extract the chromosome skeleton line from the chromosome image or to generate the chromosome skeleton line using a line constructor;
[0008] Directly extracting the chromosome skeleton line using the chromosome image includes: performing binarization processing on the chromosome image to obtain a shape mask, and extracting the chromosome skeleton line according to the shape mask;
[0009] Generating a chromosome skeleton line by using a line constructor includes: obtaining a chromosome skeleton line manually drawn by a user through a drawing interface; or randomly generating a chromosome skeleton line by specifying a skeleton line length L;
[0010] Step 2, obtaining chromosome structured data from a chromosome structured database or a chromosome structured module, wherein the chromosome structured data includes a band sequence list and a centromere position;
[0011] Step 3, draw the position of the stripes on the skeleton line according to the position in the stripe sequence list;
[0012] Step 4: Draw the chromosome centromere according to the centromere position to obtain a chromosome structured mask map.
[0013] Furthermore, in step 1, by specifying the skeleton line length L, a chromosome skeleton line is randomly generated, including:
[0014] Step 2.1, specify the skeleton line length L, set the number of line segment sampling points N, and calculate the average distance D between each sampling point mean =L / N;
[0015] Step 2.2, randomly generate the starting point P0 and randomly generate the vector angle a, with the average distance D mean is the vector modulus, calculate the position of the next point P1;
[0016] Step 2.3, randomly generate an angle offset a1 in the range of -10 degrees to 10 degrees, take point P1 as the starting point, angle a+a1 as the vector angle, and average distance D mean As the vector length, calculate the next point P2;
[0017] Step 2.4, repeat step 2.3 until the sampling of N points is completed and a skeleton line of length L is obtained.
[0018] Furthermore, in step 2, the chromosome structuring module working steps include:
[0019] Step 3.1, select a chromosome mask image from the chromosome image database, and split the chromosome mask image into three single-channel images, where channel R is the chromosome band mask, channel G is the chromosome skeleton mask, and channel B is the chromosome centromere mask;
[0020] Step 3.2, use the chromosome contour extraction method in OpenCV to extract the chromosome band mask contour, and according to all the extracted contours, find the contour centers in turn to obtain the band center list of all chromosome bands;
[0021] Step 3.3, determine whether the pixel grayscale of the point is 0. If the pixel grayscale is 0, the pixel is considered to belong to the background. If the pixel grayscale is not 0, the pixel is considered to be a pixel on the skeleton line. Extract the coordinates of all points on the skeleton line, record the upper and lower endpoints of the skeleton line, and obtain the coordinates of the skeleton line endpoints and the coordinates of all points on the skeleton line.
[0022] Step 3.4, select the upper endpoint in step 3.3 as the starting coordinate, traverse the coordinates of the skeleton line points around the upper endpoint, record the coordinate position and traversal order of the coordinate, and use the coordinate as the new starting coordinate, and repeat this process until the lower endpoint is traversed to obtain a position order list of all points on the skeleton line;
[0023] Step 3.5, calculating the length of the skeleton line, the calculation method is to count the number of position order list points extracted in step 3.4;
[0024] Step 3.6, traverse the list of the centers of all the bands in step 3.2, and calculate the Euclidean distances with all the points in the position sequence list extracted in step 3.4, select the order of the skeleton line points closest to the center point of the band as the sequence number of the band, record its sequence number, and obtain the band sequence list;
[0025] Step 3.7, using the same method as step 3.3, extract the centromere position in the chromosome centromere mask, and use the method in step 3.5 to calculate the sequential number of the centromere on the skeleton line, and record the centromere position number;
[0026] Step 3.8, structured preservation of the band sequence list and centromere position number as a record, and storage in the chromosome structured database.
[0027] Furthermore, in step 2, the method for constructing the chromosome image database is as follows:
[0028] Step 4.1, select the single chromosome image with category information as basic data;
[0029] Step 4.2, manually annotate the selected basic data; mark the upper and lower endpoints of the chromosome, the position of the band pattern, the chromosome skeleton line and the centromere position as the chromosome mask map, and store it in the chromosome image database as a group with the corresponding chromosome image; use different channels to annotate different information, channel R records the position of the band pattern, channel G records the position of the skeleton line, where the upper endpoint pixel value is marked as 64, the lower endpoint is marked as 196, the skeleton line is marked as 128, and channel B records the centromere position.
[0030] Furthermore, the method further comprises: training a diffusion generation model in a chromosome generation module:
[0031] Step 5.1, read a record from a chromosome image database, wherein the specific contents of the record include: a chromosome image and a chromosome mask image;
[0032] Step 5.2, build a diffusion generation model network, use stable diffusion+controlnet as the basic network, and perform diffusion generation tasks;
[0033] Step 5.3, use the chromosome mask image as the control input of ControlNet, the chromosome image as the generation target of StableDifference, and train the chromosome image diffusion generation model;
[0034] Step 5.4, train all records in the chromosome image database according to the above process, and perform multiple rounds of training to obtain the chromosome image diffusion generation model chrSD;
[0035] Step 5.5, use the chromosome image as the control input of controlnet, the chromosome mask image as the generation target of stable diffsion, train the chromosome mask image diffusion generation model, and obtain the chromosome mask image diffusion model chr2maskSD;
[0036] In step 5.6, the chromosome structured mask map is used as the control input of controlnet, the chromosome mask image is used as the generation target of stable diffsion, the chromosome mask image diffusion generation model is trained, and the chromosome mask image diffusion model struct2maskSD is obtained.
[0037] Furthermore, the method further comprises:
[0038] Step 6.1, input the chromosome into the chromosome mask image diffusion model chr2maskSD to generate a chromosome mask image, which contains the chromosome banding, centromere and skeleton line information;
[0039] Step 6.2, using the chromosome structuring module to extract the banding sequence list and centromere position of the chromosome mask map in step 6.1;
[0040] Step 6.3, based on the band sequence list and centromere position in step 6.2, reconstruct the abnormal structured data according to the actual abnormal logical structure, such as exchanging the band position;
[0041] Step 6.4, constructing an abnormal chromosome structured mask map using a structured mask constructor;
[0042] Step 6.5, input the abnormal chromosome structured mask image into the chromosome mask image diffusion model struct2maskSD to generate the abnormal chromosome mask image;
[0043] Step 6.6, input the abnormal chromosome mask map into the chromosome diffusion generation model chrSD to generate abnormal chromosomes.
[0044] Furthermore, the method further comprises:
[0045] Step 7.1, generate chromosome skeleton lines through line generator;
[0046] Step 7.2, obtaining a chromosome structured data from a chromosome structured database;
[0047] Step 7.3, reconstructing the structured data according to the abnormal logic, the abnormal logic includes exchanging the band position and constructing abnormal chromosome structured data;
[0048] Step 7.4, using a structured mask constructor to construct the abnormal chromosome structured data in step 7.3 into an abnormal chromosome structured mask graph;
[0049] Step 7.5, input the abnormal chromosome structured mask image into the chromosome mask image diffusion model struct2maskSD to generate the abnormal chromosome mask image;
[0050] Step 7.6, input the abnormal chromosome mask map into the chromosome diffusion generation model chrSD to generate abnormal chromosomes;
[0051] Repeat steps 7.1 to 7.6 to generate abnormal chromosomes in batches.
[0052] In a second aspect, the present invention provides a highly controllable chromosome image generation system for implementing the highly controllable chromosome image generation method described above, comprising:
[0053] A chromosome image database, used to provide chromosome images and their corresponding chromosome mask images;
[0054] A chromosome generation module is used to obtain a chromosome skeleton line through a structured mask constructor in the chromosome generation module; a method for obtaining a chromosome skeleton line is to directly extract the chromosome skeleton line from a chromosome image or to generate the chromosome skeleton line using a line constructor; using a chromosome image to directly extract the chromosome skeleton line includes: binarizing the chromosome image to obtain a shape mask, and extracting the chromosome skeleton line according to the shape mask; using a line constructor to generate a chromosome skeleton line includes: obtaining a chromosome skeleton line manually drawn by a user through a drawing interface; or randomly generating a chromosome skeleton line by specifying a skeleton line length L; obtaining chromosome structured data from a chromosome structured database or a chromosome structured module, the chromosome structured data including a band sequence list and a centromere position; drawing a band position on the skeleton line according to the position in the band sequence list; drawing a chromosome centromere according to the centromere position to obtain a chromosome structured mask map;
[0055] The chromosome image structuring module is used to structure the chromosome mask image corresponding to the chromosome image, and provide abnormal processing and structured data storage and retrieval functions;
[0056] The chromosome structure database is used to store the structured chromosome mask image information.
[0057] The present invention has the following beneficial effects: A highly controllable chromosome image generation method and system of the present invention can realize the generation of chromosomes with specified morphology and curvature, and can also generate chromosomes in a variety of scenarios. Experiments have shown that this method can effectively generate chromosome images, and the generation is highly controllable. The core innovation is that the morphology and banding of chromosomes are structured. On this basis, a multi-level generation method is used to separate the chromosome image generation and the chromosome structure construction, so that the generated chromosome image is highly controllable and the banding structure is complete. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0059] Figure 1 A schematic diagram of a highly controllable chromosome image generation system of the present invention;
[0060] Figure 2 Generate module schematics for chromosomes;
[0061] Figure 3 is a schematic diagram of chromosome images;
[0062] Figure 4 Schematic diagram of chromosome mask image. DETAILED DESCRIPTION
[0063] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The technical solutions provided by the embodiments of the present invention are described in detail below in conjunction with the drawings.
[0064] See also Figure 1 and Figure 2The present invention provides a highly controllable chromosome image generation system. The present invention performs structured processing on chromosome images and uses diffusion neural networks as the basic model to construct a multi-level chromosome generation technology architecture. The specific design modules are as follows:
[0065] A chromosome image database, used to provide chromosome images and their corresponding chromosome mask images;
[0066] A chromosome generation module is used to obtain a chromosome skeleton line through a structured mask constructor in the chromosome generation module; a method for obtaining a chromosome skeleton line is to directly extract the chromosome skeleton line from a chromosome image or to generate the chromosome skeleton line using a line constructor; using a chromosome image to directly extract the chromosome skeleton line includes: binarizing the chromosome image to obtain a shape mask, and extracting the chromosome skeleton line according to the shape mask; using a line constructor to generate a chromosome skeleton line includes: obtaining a chromosome skeleton line manually drawn by a user through a drawing interface; or randomly generating a chromosome skeleton line by specifying a skeleton line length L; obtaining chromosome structured data from a chromosome structured database or a chromosome structured module, the chromosome structured data including a band sequence list and a centromere position; drawing a band position on the skeleton line according to the position in the band sequence list; drawing a chromosome centromere according to the centromere position to obtain a chromosome structured mask map;
[0067] The chromosome image structuring module is used to structure the chromosome mask image corresponding to the chromosome image, and provide abnormal processing and structured data storage and retrieval functions;
[0068] The chromosome structure database is used to store the structured chromosome mask image information.
[0069] The highly controllable chromosome image generation system of the present invention can realize the following chromosome image generation method with highly controllable sight line, including:
[0070] Step 1: The structured mask constructor in the chromosome generation module obtains the chromosome skeleton line; the method for obtaining the chromosome skeleton line is to directly extract the chromosome skeleton line from the chromosome image or to generate the chromosome skeleton line using a line constructor;
[0071] Directly extracting the chromosome skeleton line using the chromosome image includes: performing binarization processing on the chromosome image to obtain a shape mask, and extracting the chromosome skeleton line according to the shape mask;
[0072] The use of a line constructor to generate a chromosome skeleton line includes: obtaining a chromosome skeleton line manually drawn by a user through a drawing interface; or, randomly generating a chromosome skeleton line by specifying a skeleton line length L.
[0073] Specifically, in step 1, by specifying the skeleton line length L, a chromosome skeleton line is randomly generated, including:
[0074] Step 2.1, specify the skeleton line length L, set the number of line segment sampling points N, and calculate the average distance D between each sampling point mean =L / N;
[0075] Step 2.2, randomly generate the starting point P0 and randomly generate the vector angle a, with the average distance D mean is the vector modulus, calculate the position of the next point P1;
[0076] Step 2.3, randomly generate an angle offset a1 in the range of -10 degrees to 10 degrees, take point P1 as the starting point, angle a+a1 as the vector angle, and average distance D mean As the vector length, calculate the next point P2;
[0077] Step 2.4, repeat step 2.3 until the sampling of N points is completed and a skeleton line of length L is obtained.
[0078] Step 2: Acquire chromosome structured data from a chromosome structured database or a chromosome structured module, wherein the chromosome structured data includes a band sequence list and a centromere position.
[0079] Specifically, in step 2, the chromosome structuring module working steps include:
[0080] Step 3.1, select a chromosome mask image from the chromosome image database, and split the chromosome mask image into three single-channel images, where channel R is the chromosome band mask, channel G is the chromosome skeleton mask, and channel B is the chromosome centromere mask;
[0081] Step 3.2, use the chromosome contour extraction method in OpenCV to extract the chromosome band mask contour, and according to all the extracted contours, find the contour centers in turn to obtain the band center list of all chromosome bands;
[0082] Step 3.3, determine whether the pixel grayscale of the point is 0. If the pixel grayscale is 0, the pixel is considered to belong to the background. If the pixel grayscale is not 0, the pixel is considered to be a pixel on the skeleton line. Extract the coordinates of all points on the skeleton line, record the upper and lower endpoints of the skeleton line, and obtain the coordinates of the skeleton line endpoints and the coordinates of all points on the skeleton line.
[0083] Step 3.4, select the upper endpoint in step 3.3 as the starting coordinate, traverse the coordinates of the skeleton line points around the upper endpoint, record the coordinate position and traversal order of the coordinate, and use the coordinate as the new starting coordinate, and repeat this process until the lower endpoint is traversed to obtain a position order list of all points on the skeleton line;
[0084] Step 3.5, calculating the length of the skeleton line, the calculation method is to count the number of position order list points extracted in step 3.4;
[0085] Step 3.6, traverse the list of the centers of all the bands in step 3.2, and calculate the Euclidean distances with all the points in the position sequence list extracted in step 3.4, select the order of the skeleton line points closest to the center point of the band as the sequence number of the band, record its sequence number, and obtain the band sequence list;
[0086] Step 3.7, using the same method as step 3.3, extract the centromere position in the chromosome centromere mask, and use the method in step 3.5 to calculate the sequential number of the centromere on the skeleton line, and record the centromere position number;
[0087] Step 3.8, structured preservation of the band sequence list and centromere position number as a record, and storage in the chromosome structured database.
[0088] Step 3: Draw the stripe positions on the skeleton line according to the positions in the stripe sequence list.
[0089] Step 4: Draw the chromosome centromere according to the centromere position to obtain a chromosome structured mask map.
[0090] Specifically, the method for constructing the chromosome image database is as follows:
[0091] Step 4.1, select the single chromosome image with category information as the basic data; the chromosome image example can be seen Figure 3 .
[0092] Step 4.2, manually annotate the selected basic data; mark the upper and lower endpoints of the chromosome, the position of the band pattern, the chromosome skeleton line and the centromere position as a chromosome mask map, and store it in the chromosome image database as a group with the corresponding chromosome image; use different channels to annotate different information, channel R records the position of the band pattern, channel G records the position of the skeleton line, where the upper endpoint pixel value is marked as 64, the lower endpoint is marked as 196, the skeleton line is marked as 128, and channel B records the centromere position. Annotated image examples can be seen Figure 4 .
[0093] Furthermore, the method further comprises: training a diffusion generation model in a chromosome generation module:
[0094] Step 5.1, read a record from a chromosome image database, wherein the specific contents of the record include: a chromosome image and a chromosome mask image;
[0095] Step 5.2, build a diffusion generation model network, use stable diffusion+controlnet as the basic network, and perform diffusion generation tasks;
[0096] Step 5.3, use the chromosome mask image as the control input of ControlNet, the chromosome image as the generation target of StableDifference, and train the chromosome image diffusion generation model;
[0097] Step 5.4: train all records in the chromosome image database according to the above process, and perform multiple rounds of training to obtain the chromosome image diffusion generation model chrSD; in this example, 400 rounds of training are performed.
[0098] Step 5.5, use the chromosome image as the control input of controlnet, the chromosome mask image as the generation target of stable diffsion, train the chromosome mask image diffusion generation model, and obtain the chromosome mask image diffusion model chr2maskSD;
[0099] In step 5.6, the chromosome structured mask map is used as the control input of controlnet, the chromosome mask image is used as the generation target of stable diffsion, the chromosome mask image diffusion generation model is trained, and the chromosome mask image diffusion model struct2maskSD is obtained.
[0100] Furthermore, the method further comprises:
[0101] Step 6.1, input the chromosome into the chromosome mask image diffusion model chr2maskSD to generate a chromosome mask image, which contains the chromosome banding, centromere and skeleton line information;
[0102] Step 6.2, using the chromosome structuring module to extract the banding sequence list and centromere position of the chromosome mask map in step 6.1;
[0103] Step 6.3, based on the band sequence list and centromere position in step 6.2, reconstruct the abnormal structured data according to the actual abnormal logical structure, such as exchanging the band position;
[0104] Step 6.4, constructing an abnormal chromosome structured mask map using a structured mask constructor;
[0105] Step 6.5, input the abnormal chromosome structured mask image into the chromosome mask image diffusion model struct2maskSD to generate the abnormal chromosome mask image;
[0106] Specifically, this step is completed by the mask generator in the chromosome generation module.
[0107] Step 6.6, input the abnormal chromosome mask map into the chromosome diffusion generation model chrSD to generate abnormal chromosomes.
[0108] Specifically, this step is completed by the chromosome generator in the chromosome generation module.
[0109] Furthermore, the method further comprises:
[0110] Step 7.1, generate chromosome skeleton lines through line generator;
[0111] Step 7.2, obtaining a chromosome structured data from a chromosome structured database;
[0112] Step 7.3, reconstructing the structured data according to the abnormal logic, the abnormal logic includes exchanging the band position and constructing abnormal chromosome structured data;
[0113] Step 7.4, using a structured mask constructor to construct the abnormal chromosome structured data in step 7.3 into an abnormal chromosome structured mask graph;
[0114] Step 7.5, input the abnormal chromosome structured mask image into the chromosome mask image diffusion model struct2maskSD to generate the abnormal chromosome mask image;
[0115] Specifically, this step is completed by the mask generator in the chromosome generation module.
[0116] Step 7.6, input the abnormal chromosome mask map into the chromosome diffusion generation model chrSD to generate abnormal chromosomes;
[0117] Specifically, this step is completed by the chromosome generator in the chromosome generation module.
[0118] Repeat steps 7.1 to 7.6 to generate abnormal chromosomes in batches.
[0119] A highly controllable chromosome image generation method provided by the present invention can be mainly applied to the field of chromosome image generation. In chromosome karyotype analysis, since the amount of data of some abnormal chromosomes is small and there are situations where the abnormal chromosomes have complex structures, it is difficult to directly analyze and identify them. The use of a chromosome generation method to generate abnormal chromosomes can increase the number of such abnormal chromosome images, and can obtain existing structured data and generate abnormal chromosomes according to the abnormal real logic, which can be used as a reference template for abnormal chromosomes to assist karyotype analysts in making abnormal judgments. The method proposed in the present invention has high flexibility and good controllability. It can refer to existing chromosomes for generation, or it can generate chromosomes through hand-drawn lines or line generators. In addition, it uses the structured data of real chromosomes when generating chromosomes, ensuring the reliability of the generated chromosomes.
[0120] The above-described embodiments of the present invention do not limit the protection scope of the present invention.
Claims
1. A highly controllable chromosome image generation method, characterized in that: The method comprises: Step 1: The structured mask constructor in the chromosome generation module obtains the chromosome skeleton line; the method for obtaining the chromosome skeleton line is to directly extract the chromosome skeleton line from the chromosome image or to generate the chromosome skeleton line using a line constructor; Directly extracting the chromosome skeleton line using the chromosome image includes: performing binarization processing on the chromosome image to obtain a shape mask, and extracting the chromosome skeleton line according to the shape mask; Generating a chromosome skeleton line by using a line constructor includes: obtaining a chromosome skeleton line manually drawn by a user through a drawing interface; or randomly generating a chromosome skeleton line by specifying a skeleton line length L; Step 2, obtaining chromosome structured data from a chromosome structured database or a chromosome structured module, wherein the chromosome structured data includes a band sequence list and a centromere position; Step 3, draw the position of the stripes on the skeleton line according to the position in the stripe sequence list; Step 4: Draw the chromosome centromere according to the centromere position to obtain a chromosome structured mask map.
2. A highly controllable chromosome image generation method as claimed in claim 1, characterized in that: In step 1, by specifying the skeleton line length L, the chromosome skeleton line is randomly generated, including: Step 2.1, specify the skeleton line length L, set the number of line segment sampling points N, and calculate the average distance D between each sampling point mean =L / N; Step 2.2, randomly generate the starting point P0 and randomly generate the vector angle a, with the average distance D mean is the vector modulus, calculate the position of the next point P1; Step 2.3, randomly generate an angle offset a1 in the range of -10 degrees to 10 degrees, take point P1 as the starting point, angle a+a1 as the vector angle, and average distance D mean As the vector length, calculate the next point P2; Step 2.4, repeat step 2.3 until the sampling of N points is completed and a skeleton line of length L is obtained.
3. A highly controllable chromosome image generation method as claimed in claim 2, characterized in that: In step 2, the chromosome structuring module working steps include: Step 3.1, select a chromosome mask image from the chromosome image database, and split the chromosome mask image into three single-channel images, where channel R is the chromosome band mask, channel G is the chromosome skeleton mask, and channel B is the chromosome centromere mask; Step 3.2, use the chromosome contour extraction method in OpenCV to extract the chromosome band mask contour, and according to all the extracted contours, find the contour centers in turn to obtain the band center list of all chromosome bands; Step 3.3, by traversing the coordinates of all points of the chromosome skeleton line mask, determine whether the pixel grayscale of the point is 0, if the pixel grayscale is 0, then the pixel is considered to belong to the background, and if the pixel grayscale is not 0, then the pixel is considered to be a pixel on the skeleton line; extract the coordinates of all points on the skeleton line, record the upper and lower endpoints of the skeleton line, and obtain the coordinates of the skeleton line endpoints and the coordinates of all points on the skeleton line; Step 3.4, select the upper endpoint in step 3.3 as the starting coordinate, traverse the coordinates of the skeleton line points around the upper endpoint, record the coordinate position and traversal order of the coordinate, and use the coordinate as the new starting coordinate, and repeat this process until the lower endpoint is traversed to obtain a position order list of all points on the skeleton line; Step 3.5, calculating the length of the skeleton line, the calculation method is to count the number of position order list points extracted in step 3.4; Step 3.6, traverse the list of the centers of all the bands in step 3.2, and calculate the Euclidean distances with all the points in the position sequence list extracted in step 3.4, select the order of the skeleton line points closest to the center point of the band as the sequence number of the band, record its sequence number, and obtain the band sequence list; Step 3.7, using the same method as step 3.3, extract the centromere position in the chromosome centromere mask, and use the method in step 3.5 to calculate the sequential number of the centromere on the skeleton line, and record the centromere position number; Step 3.8, structured preservation of the band sequence list and centromere position number as a record, and storage in the chromosome structured database.
4. A highly controllable chromosome image generation method as claimed in claim 3, characterized in that: In step 2, the method for constructing the chromosome image database is as follows: Step 4.1, select the single chromosome image with category information as the basic data; Step 4.2, manually annotate the selected basic data; mark the upper and lower endpoints of the chromosome, the position of the band pattern, the chromosome skeleton line and the centromere position as the chromosome mask map, and store it in the chromosome image database as a group with the corresponding chromosome image; use different channels to annotate different information, channel R records the position of the band pattern, channel G records the position of the skeleton line, where the upper endpoint pixel value is marked as 64, the lower endpoint is marked as 196, the skeleton line is marked as 128, and channel B records the centromere position.
5. A highly controllable chromosome image generation method as claimed in claim 4, characterized in that: The method further comprises: training a diffusion generation model in a chromosome generation module: Step 5.1, read a record from a chromosome image database, wherein the specific contents of the record include: a chromosome image and a chromosome mask image; Step 5.2, build a diffusion generation model network, use stable diffusion+controlnet as the basic network, and perform diffusion generation tasks; Step 5.3, use the chromosome mask image as the control input of ControlNet, the chromosome image as the generation target of StableDifference, and train the chromosome image diffusion generation model; Step 5.4, train all records in the chromosome image database according to the above process, and perform multiple rounds of training to obtain the chromosome image diffusion generation model chrSD; Step 5.5, use the chromosome image as the control input of controlnet, the chromosome mask image as the generation target of stablediffsion, train the chromosome mask image diffusion generation model, and obtain the chromosome mask image diffusion model chr2maskSD; In step 5.6, the chromosome structured mask map is used as the control input of controlnet, the chromosome mask image is used as the generation target of stable diffsion, the chromosome mask image diffusion generation model is trained, and the chromosome mask image diffusion model struct2maskSD is obtained.
6. A highly controllable chromosome image generation method as claimed in claim 5, characterized in that: The method further comprises: Step 6.1, input the chromosome into the chromosome mask image diffusion model chr2maskSD to generate a chromosome mask image, which contains the chromosome banding, centromere and skeleton line information; Step 6.2, using the chromosome structuring module to extract the banding sequence list and centromere position of the chromosome mask map in step 6.1; Step 6.3, based on the band sequence list and centromere position in step 6.2, reconstruct the abnormal structured data according to the actual abnormal logical structure; Step 6.4, constructing an abnormal chromosome structured mask map using a structured mask constructor; Step 6.5, input the abnormal chromosome structured mask image into the chromosome mask image diffusion model struct2maskSD to generate the abnormal chromosome mask image; Step 6.6, input the abnormal chromosome mask map into the chromosome diffusion generation model chrSD to generate abnormal chromosomes.
7. A highly controllable chromosome image generation method as claimed in claim 6, characterized in that: The method further comprises: Step 7.1, generate chromosome skeleton lines through line generator; Step 7.2, obtaining a chromosome structured data from a chromosome structured database; Step 7.3, reconstructing the structured data according to the abnormal logic, the abnormal logic includes exchanging the band position and constructing abnormal chromosome structured data; Step 7.4, using a structured mask constructor to construct the abnormal chromosome structured data in step 7.3 into an abnormal chromosome structured mask graph; Step 7.5, input the abnormal chromosome structured mask image into the chromosome mask image diffusion model struct2maskSD to generate the abnormal chromosome mask image; Step 7.6, input the abnormal chromosome mask map into the chromosome diffusion generation model chrSD to generate abnormal chromosomes; Repeat steps 7.1 to 7.6 to generate abnormal chromosomes in batches.
8. A highly controllable chromosome image generation system, used to implement the highly controllable chromosome image generation method according to any one of claims 1 to 7, characterized in that: include: A chromosome image database, used to provide chromosome images and their corresponding chromosome mask images; A chromosome generation module is used to obtain a chromosome skeleton line through a structured mask constructor in the chromosome generation module; a method for obtaining a chromosome skeleton line is to directly extract the chromosome skeleton line from a chromosome image or to generate the chromosome skeleton line using a line constructor; using a chromosome image to directly extract the chromosome skeleton line includes: binarizing the chromosome image to obtain a shape mask, and extracting the chromosome skeleton line according to the shape mask; using a line constructor to generate a chromosome skeleton line includes: obtaining a chromosome skeleton line manually drawn by a user through a drawing interface; or randomly generating a chromosome skeleton line by specifying a skeleton line length L; obtaining chromosome structured data from a chromosome structured database or a chromosome structured module, the chromosome structured data including a band sequence list and a centromere position; drawing a band position on the skeleton line according to the position in the band sequence list; drawing a chromosome centromere according to the centromere position to obtain a chromosome structured mask map; The chromosome image structuring module is used to structure the chromosome mask image corresponding to the chromosome image, and provide abnormal processing and structured data storage and retrieval functions; The chromosome structure database is used to store the structured chromosome mask image information.
Citation Information
Patent Citations
Adhesion chromosome separation method and device based on skeleton segmentation and key point detection
CN112200809A
Online chromosome collaborative analysis method and system and medium
CN117152147A