A method for determining hematopoietic tissue proportion based on bone marrow biopsy images

Abstract

The application discloses a kind of hematopoietic tissue proportion determination method based on bone marrow biopsy image, comprising the following steps: step S1, obtain a group of bone marrow biopsy image sequences of target object, and select out conventional type bone marrow biopsy image and special type bone marrow biopsy image in a group of bone marrow biopsy image sequences;Step S2, obtain a group of bone marrow biopsy image samples in medical big data, mark hematopoietic tissue area in the bone marrow biopsy image sample, and obtain the detection frame optimization model based on hematopoietic tissue detection frame and bone marrow biopsy image sample at multiple rotation angles.Model training.The detection frame optimization model is constructed, and the hematopoietic tissue in the bone marrow biopsy image can be accurately positioned in the rotation detection of the bone marrow biopsy image, and the boundary box hardly contains useless background area in the image, so as to reduce the influence of background on hematopoietic tissue identification, and rotation detection will make the prediction detection frame hardly overlap.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hematopoietic tissue proportion determination, and particularly relates to a hematopoietic tissue proportion determination method based on bone marrow biopsy images. BACKGROUND

[0002] A person skilled in the relevant art can study the bone marrow tissue structure by observing information such as the type and distribution of cells in the bone marrow biopsy image, wherein the proportion of hematopoietic tissue in the bone marrow biopsy image has important reference value for the study of the bone marrow tissue structure.

[0003] Currently, a person skilled in the relevant art mainly observes the bone marrow biopsy image subjectively and estimates the proportion of hematopoietic tissue in the bone marrow biopsy image, however, the result of the estimated proportion of hematopoietic tissue in the bone marrow biopsy image has poor stability, and different people or time sequences may obtain inconsistent results. SUMMARY

[0004] The present application aims to provide a hematopoietic tissue proportion determination method based on bone marrow biopsy images, so as to solve the technical problem of poor stability of the result of the estimated proportion of hematopoietic tissue in the bone marrow biopsy image in the prior art.

[0005] To solve the above technical problem, the present application specifically provides the following technical scheme:

[0006] A hematopoietic tissue proportion determination method based on bone marrow biopsy images, comprising the following steps:

[0007] Step S1, obtaining a sequence of bone marrow biopsy images of a target object, and selecting a regular type bone marrow biopsy image representing the general state of hematopoietic tissue of the target object and a specific type bone marrow biopsy image representing the specific state of hematopoietic tissue of the target object from the sequence of bone marrow biopsy images;

[0008] Step S2, obtaining a sequence of bone marrow biopsy image samples in medical big data, marking hematopoietic tissue regions in the bone marrow biopsy image samples, and marking hematopoietic tissue detection frames along the boundaries of the hematopoietic tissue regions, and performing model training of multiple rotation angles based on the hematopoietic tissue detection frames and the bone marrow biopsy image samples to obtain a detection frame optimization model, so that the obtained hematopoietic tissue detection frames realize rotation matching of hematopoietic tissue regions in any direction in the bone marrow biopsy image;

[0009] Step S3, performing detection frame optimization on the conventional bone marrow biopsy image at multiple rotation angles by using the detection frame optimization model to obtain a group of conventional hemopoietic tissue detection frames of the conventional bone marrow biopsy image, and performing detection frame optimization on the specific bone marrow biopsy image at multiple rotation angles by using the detection frame optimization model to obtain a group of specific hemopoietic tissue detection frames of the specific bone marrow biopsy image;

[0010] Step S4, performing area mean processing on the group of conventional hemopoietic tissue detection frames and the group of specific hemopoietic tissue detection frames to obtain a conventional hemopoietic tissue detection frame area mean value and a specific hemopoietic tissue detection frame area mean value, and performing error correction on the conventional hemopoietic tissue detection frame area mean value and the specific hemopoietic tissue detection frame area mean value based on the optimization error of the detection frame optimization model to obtain a hemopoietic tissue area of the conventional bone marrow biopsy image and a hemopoietic tissue area of the specific bone marrow biopsy image, and then calculating the proportion of the hemopoietic tissue area of the conventional bone marrow biopsy image and the hemopoietic tissue area of the specific bone marrow biopsy image in the conventional bone marrow biopsy image and the specific bone marrow biopsy image to obtain the hemopoietic tissue proportion interval of the target object.

[0011] As a preferred scheme of the present application, the conventional bone marrow biopsy image representing the general state of the hemopoietic tissue of the target object and the specific bone marrow biopsy image representing the specific state of the hemopoietic tissue of the target object are selected from the group of bone marrow biopsy image sequences, comprising:

[0012] Kmeans++ clustering is performed on all bone marrow biopsy images in the group of bone marrow biopsy image sequences to obtain a plurality of bone marrow biopsy image clusters, and the total number of bone marrow biopsy images contained in each bone marrow biopsy image cluster is counted;

[0013] A distinction threshold is set, and the total number of bone marrow biopsy images contained in each bone marrow biopsy image cluster is compared with the distinction threshold, wherein,

[0014] If the total number of bone marrow biopsy images is lower than the distinction threshold, the bone marrow biopsy images in the corresponding bone marrow biopsy image cluster are taken as specific bone marrow biopsy images;

[0015] If the total number of bone marrow biopsy images is higher than or equal to the distinction threshold, the bone marrow biopsy images in the corresponding bone marrow biopsy image cluster are taken as conventional bone marrow biopsy images.

[0016] As a preferred scheme of the present application, the model training at multiple rotation angles based on the hemopoietic tissue detection frame and the bone marrow biopsy image sample to obtain the detection frame optimization model comprises:

[0017] A set of multi-angle images of each bone marrow biopsy image sample is obtained by rotating each bone marrow biopsy image sample by 90 degrees, 180 degrees and 270 degrees clockwise in sequence, and a set of multi-angle detection boxes matched with the set of multi-angle images is obtained by rotating the hematopoietic tissue detection box in each bone marrow biopsy image sample by 90 degrees, 180 degrees and 270 degrees clockwise in sequence;

[0018] The multi-angle images and the multi-angle detection boxes are matched to form training samples for model training;

[0019] Based on the RetinaNet framework, deep convolutional neural networks are used to extract image features from the multi-angle images, and the NAS-FPN algorithm is used to fuse the extracted image features to obtain a set of feature maps representing hematopoietic tissues with different scales for each multi-angle image;

[0020] The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and based on the feature map, a binary coding labeling technique is used to obtain an angle classification model for regressing the multi-angle detection box to the feature map matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is:

[0021] U predict =Model(E);

[0022] In the formula, Model represents the detection box optimization model identifier, U predict represents the predicted detection box identifier, and E represents the feature map identifier;

[0023] The loss function of the angle classification model is:

[0024] Loss=smooth L1(U predict -U taget );

[0025] In the formula, Loss represents the loss function of the angle classification model, smooth L1 represents the smooth L1 loss function operator, U predict represents the predicted detection box, and U taget represents the multi-angle detection box.

[0026] As a preferred scheme of the present application, the use of the five-parameter definition method in the OpenCV definition method to represent the predicted detection box comprises:

[0027] The five representation parameters U predict of the predicted detection box are set as: x predict , y predict , w predict , and hpredict , θ predict ], wherein x predict and y predict are the center coordinates of the predicted bounding box, θ predict is the acute angle between the predicted bounding box and the x-axis, w predict is the width of the predicted bounding box, and h predict is the height of the predicted bounding box.

[0028] As a preferred scheme of the present application, the binary coding label technique based on the feature map is used to obtain an angle classification model for regressing multi-angle bounding boxes to angles matching the feature map, comprising:

[0029] An angle range AR and an angle discrete granularity d are set, an encoding list L is initialized, an encoding length n is set as n = log2(AR / d), and a loop coefficient i is set;

[0030] The i is stepped by the discrete granularity d from the initial value of the range of the AR to the terminal value of the range of the AR, the i is sequentially binary coded according to the encoding length n to obtain binary(i, n) = binary(-round((i-90°) / d), and the binary(i, n) is sequentially stored in the encoding list L to obtain the encoding list L containing all binary coding labels in the AR;

[0031] The evaluator in the RetinaNet framework is used to obtain the prediction probability p at each encoding position in the encoding list L, all prediction probabilities p are collected as a probability list B, and a binary list C is initialized;

[0032] The j is stepped from the initial value of the probability list B to the terminal value of the probability list B, the j value is sequentially converted to obtain round(sigmoid(log(j / (1-j))*d, the round(sigmoid(log(j / (1-j))*d is sequentially stored in the binary list C, the encoding values at all encoding positions in the binary list C are combined and converted to a decimal number D, and θ predict = 90°-D*d is calculated based on the decimal number D;

[0033] The θ predict in the loss function of the angle classification model is fixed, and the angle classification model is trained based on the loss function after the θ predict is fixed to obtain the bounding box optimization model;

[0034] In the formula, round is a rounding operator, sigmoid is a sigmoid algorithm operator, and binary is a binary coding operator.

[0035] As a preferred scheme of the present application, the optimization error calculation process of the detection box optimization model comprises:

[0036] The hematopoietic tissue area in each multi-angle detection box is counted, and the hematopoietic tissue area in each multi-angle image is counted.

[0037] The optimization error is calculated based on the hematopoietic tissue area in the multi-angle detection box and the hematopoietic tissue area in the multi-angle image, and the calculation formula of the optimization error is:

[0038]

[0039] In the formula, r represents the optimization error, S k represents the hematopoietic tissue area in the kth multi-angle detection box, Z k represents the hematopoietic tissue area in the multi-angle image, M represents the total number of multi-angle detection boxes or multi-angle images, and k is a constant.

[0040] As a preferred scheme of the present application, the multi-rotation angle detection box optimization of the conventional bone marrow biopsy image by using the detection box optimization model obtains a group of conventional hematopoietic tissue detection boxes of the conventional bone marrow biopsy image, which comprises:

[0041] The conventional bone marrow biopsy image is sequentially rotated by 90 degrees, 180 degrees and 270 degrees clockwise to obtain a group of conventional multi-angle images of each conventional bone marrow biopsy image.

[0042] A group of conventional multi-angle images are input into the detection box optimization model to obtain a group of conventional multi-angle detection boxes.

[0043] As a preferred scheme of the present application, the multi-rotation angle detection box optimization of the specific bone marrow biopsy image by using the detection box optimization model obtains a group of specific hematopoietic tissue detection boxes of the specific bone marrow biopsy image, which comprises:

[0044] The specific bone marrow biopsy image is sequentially rotated by 90 degrees, 180 degrees and 270 degrees clockwise to obtain a group of specific multi-angle images of each specific bone marrow biopsy image.

[0045] A group of specific multi-angle images are input into the detection box optimization model to obtain a group of specific multi-angle detection boxes.

[0046] As a preferred scheme of the present application, the error correction of the mean area of the conventional hematopoietic tissue detection box and the mean area of the specific hematopoietic tissue detection box based on the optimization error of the detection box optimization model obtains the hematopoietic tissue area of the conventional bone marrow biopsy image and the hematopoietic tissue area of the specific bone marrow biopsy image, which comprises:

[0047] The average of the area of the detection frame of the conventional type hematopoietic tissue and the average of the area of the detection frame of the specific type hematopoietic tissue are brought into the calculation formula of the optimization error to calculate the hematopoietic tissue area of the conventional type bone marrow biopsy image and the hematopoietic tissue area of the specific type bone marrow biopsy image.

[0048] As a preferred scheme of the present application, the proportion of the hematopoietic tissue area of the conventional type bone marrow biopsy image and the hematopoietic tissue area of the specific type bone marrow biopsy image in the conventional type bone marrow biopsy image and the specific type bone marrow biopsy image is calculated to obtain the hematopoietic tissue proportion interval of the target object, which comprises:

[0049] The proportion of the hematopoietic tissue area of the conventional type bone marrow biopsy image in the area of the conventional type bone marrow biopsy image is taken as the hematopoietic tissue conventional type proportion, and the proportion of the hematopoietic tissue area of the specific type bone marrow biopsy image in the area of the specific type bone marrow biopsy image is taken as the hematopoietic tissue specific type proportion.

[0050] The maximum hematopoietic tissue conventional type proportion and the minimum hematopoietic tissue specific type proportion are selected as the interval boundaries to constitute the hematopoietic tissue proportion interval of the target object.

[0051] Compared with the prior art, the present application has the following beneficial effects:

[0052] The detection frame optimization model is constructed to accurately position the hematopoietic tissue in the bone marrow biopsy image through the rotation detection, and the boundary frame almost does not contain the useless background area in the image, so that the influence of the background on the hematopoietic tissue recognition is reduced. The rotation detection makes the prediction detection frames almost have no overlap, so that the hematopoietic tissue contained in the frame can be more clearly recognized. The bone marrow biopsy image is divided into the conventional type and the specific type, the time sequence analysis of the bone marrow biopsy image can be realized, the overall characteristics of the hematopoietic tissue proportion of the target object are obtained, the contingency caused by the single time sequence is avoided, and the certainty of the hematopoietic tissue proportion is improved. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical schemes in the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other drawings can also be obtained from the provided drawings without creative labor.

[0054] Figure 1 The hematopoietic tissue proportion determination method flowchart provided for the embodiments of the present application;

[0055] Figure 2 The horizontal detection frame and the prediction detection frame structure schematic diagram provided for the embodiments of the present application;

[0056] Figure 3 An angle classification model operation result graph provided by an embodiment of the present application.

[0057] The reference signs in the drawings represent the following:

[0058] 1-hematopoietic tissue; 2-horizontal detection frame; 3-predicted detection frame. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0060] As shown in Figure 1 the present application provides a method for determining the proportion of hematopoietic tissue based on bone marrow biopsy images, comprising the following steps:

[0061] Step S1, obtaining a group of bone marrow biopsy image sequences of a target object, and selecting a conventional bone marrow biopsy image representing the general state of hematopoietic tissue of the target object and a specific bone marrow biopsy image representing the specific state of hematopoietic tissue of the target object from the group of bone marrow biopsy image sequences;

[0062] selecting a conventional bone marrow biopsy image representing the general state of hematopoietic tissue of the target object and a specific bone marrow biopsy image representing the specific state of hematopoietic tissue of the target object from the group of bone marrow biopsy image sequences, comprising:

[0063] Kmeans++ clustering all bone marrow biopsy images in the group of bone marrow biopsy image sequences to obtain a plurality of bone marrow biopsy image clusters, and counting the total number of bone marrow biopsy images contained in each bone marrow biopsy image cluster;

[0064] setting a distinction threshold, comparing the total number of bone marrow biopsy images contained in each bone marrow biopsy image cluster with the distinction threshold, wherein,

[0065] if the total number of bone marrow biopsy images is lower than the distinction threshold, the bone marrow biopsy images in the corresponding bone marrow biopsy image cluster are taken as specific bone marrow biopsy images;

[0066] if the total number of bone marrow biopsy images is higher than or equal to the distinction threshold, the bone marrow biopsy images in the corresponding bone marrow biopsy image cluster are taken as conventional bone marrow biopsy images.

[0067] Using the determination of the hematopoietic tissue proportion of a bone marrow biopsy image at a time sequence as the determination result of the hematopoietic tissue proportion of the target object, there is an accident if the bone marrow biopsy image at the time sequence cannot be used as the bone marrow biopsy image for determining the hematopoietic tissue proportion due to the shooting mode of the bone marrow biopsy image at the time sequence or the physiological factors of the target object, so the hematopoietic tissue proportion of the target object determined by the bone marrow biopsy image at the time sequence is an invalid result. Therefore, the embodiment uses a group of bone marrow biopsy image sequences to determine the hematopoietic tissue proportion of the target object. The group of bone marrow biopsy image sequences can also reflect the change of the hematopoietic tissue of the target object, that is, the regular value and the specific value of the hematopoietic tissue in the bone marrow of the target object can be comprehensively obtained, the interval range of the hematopoietic tissue proportion is obtained, and a better data basis is provided for diagnosis and treatment.

[0068] In step S2, a group of bone marrow biopsy image samples are obtained from the medical big data, the hematopoietic tissue region is marked in the bone marrow biopsy image sample, and the hematopoietic tissue detection frame is obtained along the boundary of the hematopoietic tissue region. The detection frame optimization model is obtained by model training based on the hematopoietic tissue detection frame and the bone marrow biopsy image sample, so that the obtained hematopoietic tissue detection frame realizes the rotation matching of the hematopoietic tissue region in any direction in the bone marrow biopsy image.

[0069] The model training based on the hematopoietic tissue detection frame and the bone marrow biopsy image sample to obtain the detection frame optimization model includes:

[0070] Each bone marrow biopsy image sample is sequentially rotated by 90 degrees clockwise, 180 degrees and 270 degrees to obtain a group of multi-angle images of each bone marrow biopsy image sample, and the hematopoietic tissue detection frame in each bone marrow biopsy image sample is sequentially rotated by 90 degrees clockwise, 180 degrees and 270 degrees to obtain a group of multi-angle detection frames matched with the group of multi-angle images.

[0071] Multi-angle rotation can further expand the training sample data volume of the model, and on the other hand, multi-angle rotation can enable the model obtained by training to detect the frame in multiple angles, thereby improving the model accuracy.

[0072] The multi-angle images and the multi-angle detection frames are correspondingly constituted as training samples for model training;

[0073] Based on the RetinaNet framework, the deep convolutional neural network is used to extract image features from the multi-angle images, and the NAS-FPN algorithm is used to fuse the extracted image features to obtain a group of feature maps representing hematopoietic tissue with different scales for each multi-angle image.

[0074] The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection frame, and a binary coding label technology is used based on the feature map to obtain an angle classification model for regressing the multi-angle detection frame to an angle matched with the feature map, and then the angle classification model is trained based on training samples to obtain a detection frame optimization model, and the function expression of the detection frame optimization model is:

[0075] U predict = Model (E) ;

[0076] In the formula, Model represents a detection frame optimization model identifier, U predict represents a predicted detection frame identifier, and E represents a feature map identifier.

[0077] The loss function of the angle classification model is:

[0078] Loss = smooth L1 (U predict -U taget ) ;

[0079] In the formula, Loss represents a loss function of the angle classification model, smooth L1 represents a smooth L1 loss function operator, U predict represents a predicted detection frame, and U taget represents a multi-angle detection frame.

[0080] The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection frame, including:

[0081] Five representation parameters U predict = [x predict , y predict , w predict , h predict , θ predict ] of the predicted detection frame are set, wherein x predict and y predict are center coordinates of the predicted detection frame, θ predict is an acute angle between the predicted detection frame and the x-axis, w predict is a width of the predicted detection frame, and h predict is a height of the predicted detection frame.

[0082] U taget = [x taget , y taget , w taget , h taget , θ taget ], wherein x taget and y taget are center coordinates of the multi-angle detection frame, θ taget is an acute angle between the multi-angle detection frame and the x-axis, w tagetWidth of the multi-angle bounding box taget Height of the multi-angle bounding box

[0083] An angle classification model is obtained by using a binary coding label technique based on the feature map to regress the multi-angle bounding box to an angle category matching the feature map, comprising:

[0084] Setting an angle range AR, an angle discrete granularity d, initializing an encoding list L, an encoding length n = log2(AR / d), and a loop coefficient i;

[0085] Step i from the initial value of the range of AR to the terminal value of the range of AR by the discrete granularity d, and sequentially encode i by the encoding length n to obtain binary(i, n) = binary(-round((i-90°) / d), and sequentially store binary(i, n) in the encoding list L to obtain the encoding list L containing all binary coding labels in AR;

[0086] Using the evaluator in the RetinaNet framework to obtain the prediction probability p at each encoding position in the encoding list L, and collecting all prediction probabilities p into a probability list B, and initializing a binary list C;

[0087] Step j from the initial value of the probability list B to the terminal value of the probability list B, and sequentially convert j values to obtain round(sigmoid(log(j / (1-j))*d, and sequentially store round(sigmoid(log(j / (1-j))*d in the binary list C, and convert the encoding values in all encoding positions in the binary list C to decimal numbers D, and then calculate θ predict = 90°-D*d based on the decimal numbers D;

[0088] Fix θ predict in the loss function of the angle classification model, and then train the angle classification model based on the fixed θ predict to obtain a bounding box optimization model;

[0089] In the formula, round is a rounding operator, sigmoid is a sigmoid algorithm operator, and binary is a binary coding operator.

[0090] As Figure 2As shown, the hematopoietic tissue in the bone marrow biopsy image is in any direction, and the use of a horizontal detection box to detect the target (hematopoietic tissue) will cause several problems. First, such a horizontal detection box usually contains a lot of background area, and the presence of too much background area in the detection box not only increases the difficulty of the target positioning task, but also leads to inaccurate representation of the target range. Second, the horizontal bounding box will cause strong overlap between the detection boxes, reducing detection accuracy.

[0091] Therefore, the embodiment provides a detection box optimization model, which can match the predicted detection box with the direction of the hematopoietic tissue, i.e. the direction of the hematopoietic tissue region, so that the predicted detection box is at the boundary of the hematopoietic tissue region, so that the predicted detection box contains the hematopoietic tissue region of the bone marrow biopsy image and contains less background area of the bone marrow biopsy image. Specifically, the detection box optimization model combines a neural architecture search framework and a feature pyramid network (NAS-FPN) module into a dense detector (RetinaNet) and uses a binary coding method in angle classification. This method reduces the background influence, so that there is almost no overlap between the detection boxes.

[0092] As Figure 3 shown, the embodiment provides a detection box angle optimization instance of the angle classification model, assuming that the hematopoietic tissue region of a bone marrow biopsy image has a x-axis θ taget = 88°, and setting d = 180° / 256, AR = 180°, then the calculation gives the encoding list L = [00000011], since B = [0.41, 0.12, 0.28, 0.32, 0.22, 0.28, 0.98, 0.99], then after calculation: C = [00000011], D = 3, θ predict = 90°-D*d = 87.89°, so the predicted detection box θ predict = 87.89°, is in the same direction as the hematopoietic tissue, so the detection box optimization model has high accuracy for detection optimization.

[0093] The optimization error calculation process of the detection box optimization model includes:

[0094] Statistical hematopoietic tissue area in each multi-angle detection box, and statistical hematopoietic tissue area in each multi-angle image;

[0095] Based on the hematopoietic tissue area in the multi-angle detection box and the hematopoietic tissue area in the multi-angle image, the optimization error is calculated, and the calculation formula of the optimization error is:

[0096]

[0097] In the formula, r represents the optimization error, Sk characterized as the hematopoietic tissue area in the kth multi-angle detection frame, Z k characterized as the hematopoietic tissue area in the multi-angle image, M is characterized as the total number of multi-angle detection frames or multi-angle images, and k is a constant of measurement.

[0098] The error caused by removing the background area in the prediction detection frame is removed, and the error of the background area is quantified as an optimization error.

[0099] Step S3, using the detection frame optimization model to optimize the detection frame of the conventional type bone marrow biopsy image at multiple rotation angles to obtain a group of conventional type hematopoietic tissue detection frames of the conventional type bone marrow biopsy image, and using the detection frame optimization model to optimize the detection frame of the specific type bone marrow biopsy image at multiple rotation angles to obtain a group of specific type hematopoietic tissue detection frames of the specific type bone marrow biopsy image.

[0100] Using the detection frame optimization model to optimize the detection frame of the conventional type bone marrow biopsy image at multiple rotation angles to obtain a group of conventional type hematopoietic tissue detection frames of the conventional type bone marrow biopsy image, comprising:

[0101] Rotating the conventional type bone marrow biopsy image by 90 degrees, 180 degrees and 270 degrees clockwise in turn to obtain a group of conventional type multi-angle images of each conventional type bone marrow biopsy image.

[0102] Inputting the group of conventional type multi-angle images into the detection frame optimization model to obtain a group of conventional type multi-angle detection frames.

[0103] Using the detection frame optimization model to optimize the detection frame of the specific type bone marrow biopsy image at multiple rotation angles to obtain a group of specific type hematopoietic tissue detection frames of the specific type bone marrow biopsy image, comprising:

[0104] Rotating the specific type bone marrow biopsy image by 90 degrees, 180 degrees and 270 degrees clockwise in turn to obtain a group of specific type multi-angle images of each specific type bone marrow biopsy image.

[0105] Inputting the group of specific type multi-angle images into the detection frame optimization model to obtain a group of specific type multi-angle detection frames, which can obtain detection frames from various angles, so that hematopoietic tissue areas are obtained at various angles, and the results are more reliable.

[0106] Step S4, the area mean value of the set of conventional type hematopoietic tissue detection frames and the area mean value of the set of specific type hematopoietic tissue detection frames are obtained by area mean processing, and the area mean value of the conventional type hematopoietic tissue detection frame and the area mean value of the specific type hematopoietic tissue detection frame are error corrected based on the optimization error of the detection frame optimization model to obtain the hematopoietic tissue area of the conventional type bone marrow biopsy image and the hematopoietic tissue area of the specific type bone marrow biopsy image, and the hematopoietic tissue area ratio interval of the target object is obtained by calculating the proportion of the hematopoietic tissue area of the conventional type bone marrow biopsy image and the hematopoietic tissue area of the specific type bone marrow biopsy image in the conventional type bone marrow biopsy image and the specific type bone marrow biopsy image.

[0107] The area mean value of the conventional type hematopoietic tissue detection frame and the area mean value of the specific type hematopoietic tissue detection frame are error corrected based on the optimization error of the detection frame optimization model to obtain the hematopoietic tissue area of the conventional type bone marrow biopsy image and the hematopoietic tissue area of the specific type bone marrow biopsy image, including:

[0108] The area mean value of the conventional type hematopoietic tissue detection frame and the area mean value of the specific type hematopoietic tissue detection frame are error corrected based on the optimization error of the detection frame optimization model to obtain the hematopoietic tissue area of the conventional type bone marrow biopsy image and the hematopoietic tissue area of the specific type bone marrow biopsy image, including:

[0109] The area mean value of the conventional type hematopoietic tissue detection frame and the area mean value of the specific type hematopoietic tissue detection frame are error corrected based on the optimization error of the detection frame optimization model to obtain the hematopoietic tissue area of the conventional type bone marrow biopsy image and the hematopoietic tissue area of the specific type bone marrow biopsy image, including:

[0110] The proportion of the hematopoietic tissue area of the conventional type bone marrow biopsy image in the area of the conventional type bone marrow biopsy image is taken as the hematopoietic tissue conventional type ratio, and the proportion of the hematopoietic tissue area of the specific type bone marrow biopsy image in the area of the specific type bone marrow biopsy image is taken as the hematopoietic tissue specific type ratio.

[0111] The maximum hematopoietic tissue conventional type ratio and the minimum hematopoietic tissue specific type ratio are selected as interval boundaries to constitute the hematopoietic tissue ratio interval of the target object.

[0112] The detection frame optimization model is constructed, which can accurately position the hematopoietic tissue in the bone marrow biopsy image in the rotation detection, and the boundary frame almost does not contain the useless background area in the image, thereby reducing the influence of the background on the hematopoietic tissue recognition, the rotation detection can make the prediction detection frames almost have no overlap, thereby the hematopoietic tissue contained in the frame can be more clearly recognized, and the bone marrow biopsy image is divided into conventional type and specific type, which can realize the time sequence analysis of the bone marrow biopsy image, obtain the overall characteristics of the hematopoietic tissue ratio of the target object, avoid the contingency caused by single time sequence, and improve the certainty of the hematopoietic tissue ratio.

[0113] The above embodiments are only exemplary embodiments of the present application, and are not intended to limit the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the present application within the spirit and protection scope of the present application, and such modifications or equivalent replacements should also be considered to fall within the protection scope of the present application.

Claims

1. A method for determining a hematopoietic tissue proportion based on a bone marrow biopsy image, characterized by, The method comprises the following steps: Step S1, obtaining a group of bone marrow biopsy image sequences of a target object, and selecting a conventional bone marrow biopsy image representing the general state of hematopoietic tissue of the target object and a specific bone marrow biopsy image representing the specific state of hematopoietic tissue of the target object from the group of bone marrow biopsy image sequences; Step S2, obtaining a group of bone marrow biopsy image samples in medical big data, marking the hematopoietic tissue region in the bone marrow biopsy image samples, and marking the hematopoietic tissue detection frame along the boundary of the hematopoietic tissue region, and performing model training of multiple rotation angles based on the hematopoietic tissue detection frame and the bone marrow biopsy image sample to obtain a detection frame optimization model, so that the obtained hematopoietic tissue detection frame realizes rotation matching of the hematopoietic tissue region in the bone marrow biopsy image in any direction; Step S3, performing detection frame optimization of multiple rotation angles on the conventional bone marrow biopsy image by using the detection frame optimization model to obtain a group of conventional hematopoietic tissue detection frames of the conventional bone marrow biopsy image, and performing detection frame optimization of multiple rotation angles on the specific bone marrow biopsy image by using the detection frame optimization model to obtain a group of specific hematopoietic tissue detection frames of the specific bone marrow biopsy image; Step S4, performing area mean processing on the group of conventional hematopoietic tissue detection frames and the group of specific hematopoietic tissue detection frames to obtain a conventional hematopoietic tissue detection frame area mean and a specific hematopoietic tissue detection frame area mean, and performing error correction on the conventional hematopoietic tissue detection frame area mean and the specific hematopoietic tissue detection frame area mean based on the optimization error of the detection frame optimization model to obtain the hematopoietic tissue area of the conventional bone marrow biopsy image and the hematopoietic tissue area of the specific bone marrow biopsy image, and then calculating the proportion of the hematopoietic tissue area of the conventional bone marrow biopsy image and the hematopoietic tissue area of the specific bone marrow biopsy image in the conventional bone marrow biopsy image and the specific bone marrow biopsy image to obtain a hematopoietic tissue proportion interval of the target object. 2.The method of claim 1, wherein: The step of selecting a conventional bone marrow biopsy image representing the general state of hematopoietic tissue of the target object and a specific bone marrow biopsy image representing the specific state of hematopoietic tissue of the target object from the group of bone marrow biopsy image sequences comprises: Kmeans++ clustering is performed on all bone marrow biopsy images in the group of bone marrow biopsy image sequences to obtain a plurality of bone marrow biopsy image clusters, and the total number of bone marrow biopsy images included in each bone marrow biopsy image cluster is counted; A threshold is set, and the total number of bone marrow biopsy images included in each bone marrow biopsy image cluster is compared with the threshold, wherein, If the total number of bone marrow biopsy images is lower than the threshold, the bone marrow biopsy images in the corresponding bone marrow biopsy image cluster are taken as specific bone marrow biopsy images; If the total number of bone marrow biopsy images is higher than or equal to the threshold, the bone marrow biopsy images in the corresponding bone marrow biopsy image cluster are taken as conventional bone marrow biopsy images.

3. The method of claim 2, wherein: The step of performing model training of multiple rotation angles based on the hematopoietic tissue detection frame and the bone marrow biopsy image sample to obtain a detection frame optimization model comprises: A set of multi-angle images of each bone marrow biopsy image sample is obtained by sequentially rotating each bone marrow biopsy image sample by 90 degrees, 180 degrees and 270 degrees clockwise, and a set of multi-angle detection boxes matched with the set of multi-angle images is obtained by sequentially rotating the hematopoietic tissue detection box in each bone marrow biopsy image sample by 90 degrees, 180 degrees and 270 degrees clockwise; The multi-angle images and multi-angle detection boxes are matched to form training samples for model training; Based on the RetinaNet framework, deep convolutional neural networks are used to extract image features from the multi-angle images, and the NAS-FPN algorithm is used to fuse the extracted image features to obtain a set of feature maps representing hematopoietic tissue with different scales for each multi-angle image; The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: ； In the formula, characterized as a detection box optimization model identifier, characterized as a predicted detection box identifier, characterized as a feature map identifier; The loss function of the angle classification model is: ； In the formula, a loss function characterized as an angle classification model, characterized as a loss function operator, characterized as a predicted detection box, characterized as a multi-angle detection box.

4. The method of claim 3, wherein: The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: Setting five representation parameters of a prediction bounding box = wherein, and are the center coordinates of the prediction bounding box, is the acute angle between the prediction bounding box and the x-axis, is the width of the prediction bounding box, is the height of the prediction bounding box.

5. The method of claim 4, wherein: The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: Setting an angle range AR, an angle discrete granularity d, initializing an encoding list as L, an encoding length as , a loop coefficient i; The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: The j is started from the initial value of the probability list B to the termination value of the probability list B step by step, and the j value is converted to round (sigmoid (log2 (j / (1-j)))) *d in turn, and round (sigmoid (log2 (j / (1-j)))) *d is stored in the binary list C in turn, and all encoding value combinations at the encoding positions in the binary list C are converted to the decimal number D, and the decimal number D is calculated based on the decimal number D ; The loss function of the angle classification model Fixed, then based on The optimized detection box model is obtained by training the model using the loss function of the fixed angle classification model. The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is:

6. The method of claim 5, wherein: The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: The five-parameter definition method in the OpenCV definition method is used to represent the predicted detection box, and a binary coding label technique is used based on the feature maps to obtain an angle classification model that regresses the multi-angle detection box to the angle matched with the feature map, and the angle classification model is trained based on the training samples to obtain the detection box optimization model, and the function expression of the detection box optimization model is: ； wherein r represents an optimization error, represents the area of hematopoietic tissue in the kth multi-angle detection frame, represents the area of hematopoietic tissue in the multi-angle image, M represents the total number of multi-angle detection frames or multi-angle images, and k is a constant of measurement.

7. The method of claim 6, wherein the method further comprises: ​ ​ Input a set of conventional multi-angle images into the detection box optimization model to obtain a set of conventional multi-angle detection boxes. 8.The method of claim 7, wherein, The specific bone marrow biopsy image is rotated by 90 degrees clockwise, 180 degrees, and 270 degrees in sequence to obtain a set of specific multi-angle images for each specific bone marrow biopsy image. Input a set of specific multi-angle images into the detection box optimization model to obtain a set of specific multi-angle detection boxes. Based on the optimization error of the detection box optimization model, the mean area of the conventional hematopoietic tissue detection box and the mean area of the specific hematopoietic tissue detection box are error corrected to obtain the hematopoietic tissue area of the conventional bone marrow biopsy image and the hematopoietic tissue area of the specific bone marrow biopsy image, including: 9.The method of claim 8, wherein, The mean area of the conventional hematopoietic tissue detection box and the mean area of the specific hematopoietic tissue detection box are brought into the calculation formula of the optimization error for calculation to obtain the hematopoietic tissue area of the conventional bone marrow biopsy image and the hematopoietic tissue area of the specific bone marrow biopsy image, respectively. The proportion of the hematopoietic tissue area of the conventional bone marrow biopsy image in the area of the conventional bone marrow biopsy image is taken as the conventional proportion of hematopoietic tissue, and the proportion of the hematopoietic tissue area of the specific bone marrow biopsy image in the area of the specific bone marrow biopsy image is taken as the specific proportion of hematopoietic tissue. 10.The method of claim 9, wherein, The maximum conventional proportion of hematopoietic tissue and the minimum specific proportion of hematopoietic tissue are selected as interval boundaries to constitute the hematopoietic tissue proportion interval of the target object. The proportion of the hematopoietic tissue area of the conventional bone marrow biopsy image in the area of the conventional bone marrow biopsy image is taken as the conventional proportion of hematopoietic tissue, and the proportion of the hematopoietic tissue area of the specific bone marrow biopsy image in the area of the specific bone marrow biopsy image is taken as the specific proportion of hematopoietic tissue. The maximum conventional proportion of hematopoietic tissue and the minimum specific proportion of hematopoietic tissue are selected as interval boundaries to constitute the hematopoietic tissue proportion interval of the target object.

Images