Stem cell detection method and related equipment

Abstract

The invention relates to the technical field of stem cell preparation, and provides a stem cell detection method and related equipment, and the method comprises the steps: obtaining a plurality of target stem cell images; performing stem cell detection on each target stem cell image by using the target detection model to obtain a stem cell detection result corresponding to each target stem cell image; constructing a dual-task loss function based on all the stem cell detection results, and training the target detection model by using the dual-task loss function to obtain a trained target detection model; the double-task loss function is used for describing the accuracy of all stem cell detection results; and performing stem cell detection on the to-be-detected stem cell image by using the trained target detection model to obtain a final stem cell detection result of the to-be-detected stem cell image. The method can improve the accuracy and universality of stem cell detection.

Description

Technical Field

[0001] This application relates to the field of stem cell preparation technology, and in particular to a stem cell detection method and related equipment. Background Technology

[0002] Against the backdrop of rapid development in regenerative medicine and cell therapy technologies, stem cell preparation, as a core link in the research and clinical translation of cell therapy products, directly determines the safety and efficacy of treatment through its quality control and automation level. For example, in the clinical preparation of mesenchymal stem cells, it is necessary to accurately separate live cell populations and remove differentiated cells and impurities; in the directed differentiation process of induced pluripotent stem cells (iPSCs), it is necessary to monitor changes in cell clone morphology and differentiation status in real time to provide a basis for decision-making in subsequent cell sorting and expansion. However, the microscopic image analysis task in stem cell preparation scenarios faces technical challenges that are significantly different from traditional biological imaging. Traditional target detection algorithms have obvious shortcomings in addressing these challenges and cannot meet the dual requirements of precision and efficiency in cell preparation.

[0003] Traditional object detection algorithms (such as Faster Region-based Convolutional Neural Network (Faster R-CNN) and Single Shot MultiBox Detector (SSD)) are primarily designed for natural images or routine biological tissue slices, using horizontal bounding boxes (HBB) for target localization. However, in stem cell preparation microscopic images, core detection targets (such as stem cell clones, single live cells, and differentiation impurities) are affected by the culture environment and microscopic imaging angle, exhibiting arbitrary orientation distribution characteristics—for example, mesenchymal stem cells in suspension culture may grow at any angle from 0° to 360°, and the aspect ratio of elongated hematopoietic stem cells can reach 8:1 or higher. In this case, the horizontal bounding box will inevitably select a large amount of redundant background (such as scratches on the edge of the culture dish, tiny air bubbles in the culture medium), which not only interferes with the accuracy of target classification (such as misidentifying air bubbles as live cells), but also causes severe overlap of the horizontal bounding boxes of dense targets (such as high-concentration seeded stem cell populations), leading to missed detection problems (such as missing small-volume clones).

[0004] Meanwhile, the high proportion of small targets in stem cell preparation images further exacerbates the detection difficulty. Targets such as early stem cell clones (often less than 20 μm in diameter, corresponding to image pixel sizes less than 25×25) and single suspended cells (mostly 10×10-15×15 pixel sizes) are prone to losing their shallow features (such as clone edge texture and cell nucleus outline) during network downsampling. The fixed receptive field of traditional Convolutional Neural Networks (CNNs) is ill-suited to these small targets—an overly large receptive field causes the small target to be overwhelmed by background noise (such as culture medium particles), while an overly small receptive field fails to capture the target's complete morphological information (such as the circular outline of a clone).

[0005] More importantly, the complex background interference and significant scale differences in stem cell preparation images amplify the shortcomings of traditional algorithms. For example, in stem cell differentiation monitoring scenarios, target cells are not only small in size and randomly oriented, but also obscured by clusters of undifferentiated cells. In cell viability detection scenarios, the grayscale features of live and dead cells differ only slightly and are easily affected by uneven staining distribution, causing the model to misclassify dead cells as live cells. The fixed feature extraction mechanism of traditional algorithms lacks adaptability to such complex scenarios, and their feature fusion modules (such as the common feature pyramid network (FPN)) cannot effectively filter key features (such as the translucent edges of live cells), resulting in insufficient ability of the model to suppress interference information and large fluctuations in detection accuracy (e.g., the detection accuracy of cell images from different batches of culture differs by more than 15%).

[0006] This shows that current stem cell detection methods suffer from poor accuracy and low versatility. Summary of the Invention

[0007] This application provides a stem cell detection method and related equipment, which can solve the problems of poor accuracy and low versatility in stem cell detection.

[0008] In a first aspect, this application provides a stem cell detection method, which includes:

[0009] Acquire multiple target stem cell images;

[0010] A target detection model is used to detect stem cells in each target stem cell image, and the stem cell detection result corresponding to each target stem cell image is obtained. The stem cell detection result is a small target detection box of the stem cell image in the target stem cell image. The target detection model includes a multi-scale feature extraction network, a cross-scale feature extraction network, and a detection head network. The multi-scale feature extraction network is used to extract features from the target stem cell image at multiple scales. The cross-scale feature extraction network is used to perform cross-scale feature extraction based on the multi-scale feature extraction result. The detection head network is used to perform stem cell detection based on the cross-scale feature extraction result and outputs the stem cell detection result.

[0011] A dual-task loss function is constructed based on all stem cell detection results, and the target detection model is trained using the dual-task loss function to obtain the trained target detection model; the dual-task loss function is used to describe the accuracy of all stem cell detection results.

[0012] The trained target detection model is used to perform stem cell detection on the stem cell images to be detected, and the final stem cell detection results of the stem cell images are obtained.

[0013] Optionally, the multi-scale feature extraction network includes shallow feature extraction models, mid-level feature extraction models, and deep feature extraction models;

[0014] The input terminals of the shallow feature extraction model, the mid-level feature extraction model, and the deep feature extraction model are all input terminals of the multi-scale feature extraction network, and the output terminals of the shallow feature extraction model, the mid-level feature extraction model, and the deep feature extraction model are all output terminals of the multi-scale feature extraction network.

[0015] Optionally, the shallow feature extraction model includes a shallow feature aggregation module and a first rotational convolution module connected in sequence.

[0016] The input of the shallow feature aggregation module is the input of the shallow feature extraction model, and the output of the first rotation convolution module is the output of the shallow feature extraction model.

[0017] The mid-level feature extraction model includes a mid-level feature aggregation module and a second rotational convolution module connected in sequence;

[0018] The input of the mid-level feature aggregation module is the input of the mid-level feature extraction model, and the output of the second rotational convolution module is the output of the mid-level feature extraction model.

[0019] The deep feature extraction model includes a deep feature aggregation module and a third rotational convolution module connected in sequence;

[0020] The input of the deep feature aggregation module is the input of the deep feature extraction model, and the output of the third rotational convolution module is the output of the deep feature extraction model.

[0021] Optionally, the first rotational convolution module, the second rotational convolution module, and the third rotational convolution module are all rotational convolution modules;

[0022] The rotation convolution module includes a small target branch convolution unit, a rotation target branch convolution unit, and a feature fusion unit;

[0023] The input terminals of the small target branch convolutional unit and the rotating target branch convolutional unit are both input terminals of the rotating convolutional module, and the output terminal of the feature fusion unit is the output terminal of the rotating convolutional module.

[0024] The outputs of both the small target branch convolutional unit and the rotated target branch convolutional unit are connected to the input of the feature fusion unit.

[0025] Optionally, the small target branch convolutional unit includes a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a channel attention layer, and a pixel attention layer connected in sequence.

[0026] The input of the first depthwise separable convolutional layer is the input of the small target branch convolutional unit, and the output of the pixel attention layer is the output of the small target branch convolutional unit.

[0027] The rotating target branch convolutional unit includes an angle prediction layer, an angle adaptation convolutional layer, and a dilated convolutional layer connected in sequence.

[0028] The input of the angle prediction layer is the input of the rotation target branch convolutional unit, and the output of the dilated convolutional layer is the output of the rotation target branch convolutional unit.

[0029] The feature fusion unit consists of a feature contribution coefficient calculation layer and a fusion layer connected in sequence;

[0030] The input of the feature contribution coefficient calculation layer is the input of the feature fusion unit, and the output of the fusion layer is the output of the feature fusion unit.

[0031] Optionally, the cross-scale feature extraction network includes: a first feature extraction model, a second feature extraction model, a third feature extraction model, a fourth feature extraction model, and a fifth feature extraction model;

[0032] The input terminals of the first feature extraction model, the second feature extraction model, and the third feature extraction model are all input terminals of a cross-scale feature extraction network.

[0033] The outputs of the first feature extraction model, the second feature extraction model, the third feature extraction model, the fourth feature extraction model, and the fifth feature extraction model are all outputs of a cross-scale feature extraction network.

[0034] The output of the second feature extraction model is connected to the input of the first feature extraction model. The output of the third feature extraction model is connected to the inputs of the second and fourth feature extraction models. The output of the fourth feature extraction model is connected to the inputs of the third and fifth feature extraction models. The output of the fifth feature extraction model is connected to the input of the fourth feature extraction model.

[0035] Optionally, the dual-task loss function is:

[0036]

[0037] in, This represents the value of the dual-task loss function. Indicates dynamic weighting coefficients. Indicates small target loss. Indicates the loss of the target rotation angle:

[0038]

[0039]

[0040]

[0041] in, This indicates the number of small target detection boxes in all stem cell test results. Represents scale-aware weights. Indicates the first The classification prediction probability of a small object detection box. Indicates the focus factor. Indicates the average pixel size of the target. Indicates the first The pixel size of a small target detection box. Indicates the angle penalty coefficient. This indicates the center point location of the small target detection bounding box. This indicates the center point position of the actual detection box. Indicates the prediction angle. Representing the true perspective, This indicates Euclidean distance.

[0042] Secondly, this application provides a stem cell detection device, comprising:

[0043] The acquisition module is used to acquire multiple target stem cell images;

[0044] The first detection module is used to perform stem cell detection on each target stem cell image using a target detection model, and obtain the stem cell detection result corresponding to each target stem cell image. The stem cell detection result is a small target detection box of the stem cell image in the target stem cell image. The target detection model includes a multi-scale feature extraction network, a cross-scale feature extraction network, and a detection head network. The multi-scale feature extraction network is used to perform multi-scale feature extraction on the target stem cell image. The cross-scale feature extraction network is used to perform cross-scale feature extraction based on the multi-scale feature extraction result. The detection head network is used to perform stem cell detection based on the cross-scale feature extraction result and output the stem cell detection result.

[0045] The training module is used to construct a dual-task loss function based on all stem cell detection results, and to train the target detection model using the dual-task loss function to obtain the trained target detection model; the dual-task loss function is used to describe the accuracy of all stem cell detection results.

[0046] The second detection module is used to perform stem cell detection on the stem cell image to be detected using the trained target detection model, and obtain the final stem cell detection result of the stem cell image to be detected.

[0047] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the aforementioned computer program to perform the aforementioned stem cell detection.

[0048] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the aforementioned stem cell detection.

[0049] The above-mentioned solution in this application has the following beneficial effects:

[0050] In the embodiments of this application, multiple target stem cell images are acquired, and then a target detection model is used to perform stem cell detection on each target stem cell image to obtain the stem cell detection result corresponding to each target stem cell image. A dual-task loss function is then constructed based on all stem cell detection results, and the target detection model is trained using the dual-task loss function to obtain the trained target detection model. Finally, the trained target detection model is used to perform stem cell detection on the target stem cell image to obtain the final stem cell detection result for the target stem cell image. The use of the target detection model to perform multi-scale and cross-scale feature extraction on the target stem cell images can effectively capture the features of stem cell images at multiple scales, as well as the feature relationships between different scales, improving the understanding of stem cell images and effectively improving the accuracy of stem cell detection. Using multiple target stem cell images as training data to train the target detection model can improve the adaptability of the target detection model to different stem cell images, thereby improving the versatility of stem cell detection.

[0051] Other beneficial effects of this application will be described in detail in the following detailed description section. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart of a stem cell detection method provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the structure of a target detection model provided in an embodiment of this application;

[0055] Figure 3 This is a schematic diagram of the structure of a rotational convolution module provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of the structure of a detail completion module provided in one embodiment of this application;

[0057] Figure 5 This is a schematic diagram of the structure of a stem cell detection device provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0059] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0060] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0061] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0062] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0063] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0064] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0065] To address the issues of poor accuracy and low versatility in existing stem cell detection methods, this application provides a stem cell detection method. This method acquires multiple target stem cell images, then uses a target detection model to perform stem cell detection on each image, obtaining the corresponding stem cell detection result for each image. A dual-task loss function is then constructed based on all the detection results, and the target detection model is trained using this function. Finally, the trained model is used to perform stem cell detection on the target stem cell image, yielding the final stem cell detection result. The method utilizes the target detection model for multi-scale and cross-scale feature extraction of the target stem cell images, effectively capturing features at multiple scales and the feature relationships between different scales, improving the understanding of stem cell images and thus enhancing the accuracy of stem cell detection. Using multiple target stem cell images as training data improves the model's adaptability to different stem cell images, thereby increasing the versatility of stem cell detection.

[0066] The stem cell detection method provided in this application will be described exemplarily below.

[0067] like Figure 1 As shown, the stem cell detection method provided in this application includes the following steps:

[0068] Step 11: Obtain multiple target stem cell images.

[0069] The target stem cell images described above are stem cell images with corresponding real stem cell detection bounding boxes. The stem cell detection bounding boxes are used to delineate the image regions where stem cells are located within the target stem cell image.

[0070] In some embodiments of this application, the target stem cell image can be obtained by using equipment such as a microscope, and then the actual stem cell detection frame of the target stem cell image can be obtained by using a model such as the YOLO model for detection and then manually verifying it.

[0071] Step 12: Use the target detection model to perform stem cell detection on each target stem cell image to obtain the stem cell detection result corresponding to each target stem cell image.

[0072] The above stem cell detection results are small target detection boxes within the target stem cell image. These small target detection boxes are used to delineate the image region where the stem cells are located within the target stem cell image. For example... Figure 2As shown, the target detection model includes a multi-scale feature extraction network, a cross-scale feature extraction network, and a detection head network. The multi-scale feature extraction network is used to extract features from the target stem cell image at multiple scales. The cross-scale feature extraction network is used to extract features across scales based on the multi-scale feature extraction results. The detection head network is used to detect stem cells based on the cross-scale feature extraction results and output the stem cell detection results.

[0073] The aforementioned multi-scale feature extraction network includes shallow feature extraction models, mid-level feature extraction models, and deep feature extraction models.

[0074] The input terminals of the shallow feature extraction model, the mid-level feature extraction model, and the deep feature extraction model are all input terminals of the multi-scale feature extraction network, and the output terminals of the shallow feature extraction model, the mid-level feature extraction model, and the deep feature extraction model are all output terminals of the multi-scale feature extraction network.

[0075] It should be noted that the above shallow feature extraction model is used to extract shallow features of the target stem cell image, the above mid-layer feature extraction model is used to extract mid-layer features of the target stem cell image, and the above deep feature extraction model is used to extract deep features of the target stem cell image.

[0076] The shallow feature extraction model described above includes a shallow feature aggregation module and a first rotational convolution module connected in sequence.

[0077] The input of the shallow feature aggregation module is the input of the shallow feature extraction model, and the output of the first rotational convolution module is the output of the shallow feature extraction model.

[0078] The mid-level feature extraction model consists of a mid-level feature aggregation module and a second rotational convolution module connected in sequence.

[0079] The input of the mid-level feature aggregation module is the input of the mid-level feature extraction model, and the output of the second rotational convolution module is the output of the mid-level feature extraction model.

[0080] The deep feature extraction model consists of a deep feature aggregation module and a third rotational convolution module connected in sequence.

[0081] The input of the deep feature aggregation module is the input of the deep feature extraction model, and the output of the third rotational convolution module is the output of the deep feature extraction model.

[0082] It should be noted that the first, second, and third rotational convolution modules mentioned above are all used to perform rotational convolution operations on their respective input data. The shallow, medium, and deep feature aggregation modules mentioned above are all used to perform feature aggregation on their respective input data. They can all be the C2f module used for feature aggregation in the YOLO model. The difference lies in the different parameters such as the convolution kernel size and receptive field. For example, the output data size of the shallow feature aggregation module is 160×160×64, the output data size of the medium feature aggregation module is 80×80×128, and the output data size of the deep feature aggregation module is 40×40×256.

[0083] The first, second, and third rotational convolution modules mentioned above are all rotational convolution modules.

[0084] like Figure 3 As shown, the rotation convolution module includes a small target branch convolution unit, a rotation target branch convolution unit, and a feature fusion unit.

[0085] The input terminals of the small target branch convolutional unit and the rotating target branch convolutional unit are both input terminals of the rotating convolutional module, and the output terminal of the feature fusion unit is the output terminal of the rotating convolutional module.

[0086] The outputs of both the small target branch convolutional unit and the rotated target branch convolutional unit are connected to the input of the feature fusion unit.

[0087] It should be noted that the small target branch convolutional unit mentioned above is used to extract small target features from the input data, and the rotated target branch convolutional unit mentioned above is used to extract rotated target features from the input data.

[0088] The small target branch convolutional unit includes a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a channel attention layer, and a pixel attention layer connected in sequence.

[0089] The input of the first depthwise separable convolutional layer is the input of the small target branch convolutional unit, and the output of the pixel attention layer is the output of the small target branch convolutional unit.

[0090] The rotating target branch convolutional unit consists of an angle prediction layer, an angle adaptation convolutional layer, and a dilated convolutional layer connected in sequence.

[0091] The input of the angle prediction layer is the input of the rotation target branch convolutional unit, and the output of the dilated convolutional layer is the output of the rotation target branch convolutional unit.

[0092] The feature fusion unit consists of a feature contribution coefficient calculation layer and a fusion layer connected in sequence.

[0093] The input of the feature contribution coefficient calculation layer is the input of the feature fusion unit, and the output of the fusion layer is the output of the feature fusion unit.

[0094] It should be noted that the first and second depthwise separable convolutional layers are used to perform depthwise separable convolution and ReLU function operations on the input data. The channel attention layer is used to perform channel attention operations on the input data. The pixel attention layer is used to perform pixel attention and Sigmoid function operations on the input data. The angle prediction layer is used to perform convolution and Softmax function operations on the input data to achieve angle prediction of the stem cell image region. The angle adaptation layer is used to perform rotational convolution kernel operations on the input data to achieve angle adjustment and alignment of the stem cell image region. The dilated convolutional layer is used to perform dilated convolution operations on the input data.

[0095] In the calculation of the aforementioned feature contribution coefficient calculation layer, the feature contribution coefficient is introduced. , by target scale With rotation angle Dynamic calculation, the calculation formula is as follows:

[0096]

[0097] Among them, when (Small goals) It is 0.7, when (When rotating the target at a large angle) It is 0.3.

[0098] The fusion formula for the fusion layer is:

[0099]

[0100] in For small target branch features, To achieve adaptive fusion of dual-task features, the target branch features are rotated.

[0101] like Figure 2 As shown, the above-mentioned cross-scale feature extraction network includes: a first feature extraction model, a second feature extraction model, a third feature extraction model, a fourth feature extraction model, and a fifth feature extraction model.

[0102] The input terminals of the first feature extraction model, the second feature extraction model, and the third feature extraction model are all input terminals of a cross-scale feature extraction network.

[0103] The outputs of the first, second, third, fourth, and fifth feature extraction models are all outputs of a cross-scale feature extraction network.

[0104] The output of the second feature extraction model is connected to the input of the first feature extraction model. The output of the third feature extraction model is connected to the inputs of the second and fourth feature extraction models. The output of the fourth feature extraction model is connected to the inputs of the third and fifth feature extraction models. The output of the fifth feature extraction model is connected to the input of the fourth feature extraction model.

[0105] It should be noted that, as Figure 2 As shown, the input of the first feature extraction model is connected to the output of the shallow feature extraction model, the input of the second feature extraction model is connected to the output of the mid-level feature extraction model, and the input of the third feature extraction model is connected to the output of the deep feature extraction model. These first, second, third, fourth, and fifth feature extraction models are all used to perform convolution and Gaussian operations on their respective input data to achieve feature extraction. The first feature extraction model also includes a detail completion module, which is connected between the first feature extraction model and the detection head network to perform detail completion on the extracted features.

[0106] like Figure 2 As shown, the detection head network includes multiple detection head models (i.e., the first detection head model, the second detection head model, the third detection head model, the fourth detection head model, and the fifth detection head model). The input of the first detection head model is connected to the output of the first feature extraction model, the input of the second detection head model is connected to the output of the second feature extraction model, the input of the third detection head model is connected to the output of the third feature extraction model, the input of the fourth detection head model is connected to the output of the fourth feature extraction model, and the input of the fifth detection head model is connected to the output of the fifth feature extraction model.

[0107] Each detection head model is used to perform operations such as convolution and fully connected layers on the input data to predict small target detection boxes in stem cell images based on input features.

[0108] like Figure 4As shown, the above detail completion module includes a first convolutional layer, a second convolutional layer, and a residual connection layer. The input of the first convolutional layer is the output data of the shallow feature extraction model, and the input of the second convolutional layer is the data obtained by convolution and Gaussian operation in the first feature extraction model. The residual connection layer is used to perform residual connection on the input data of the first convolutional layer and the second convolutional layer to realize the detail completion of the features.

[0109] For example, in the small target branch convolutional unit mentioned above, a cascaded structure of "1×1 depthwise separable convolution + 3×3 depthwise separable convolution" is adopted: the 1×1 convolution reduces the number of input channels from C to C / 2, reducing the computational load; the 3×3 depthwise separable convolution extracts shallow spatial details of small targets (such as ship masts and radar), preserving local features; a channel-pixel dual attention mechanism is added: the importance weights of each channel are first calculated through the channel attention module, as follows:

[0110]

[0111] in, For branch output features, For the Sigmoid function, The ReLU function is used, and then a pixel weight map is generated through a pixel-level attention module (Conv2d + Sigmoid) to suppress background noise and enhance the features of the target region.

[0112] In the rotation branch convolutional unit, the rotation angle of the current target is predicted using the feature map of the previous frame. (Fully connected layer + Softmax outputs the angle classification probability, argmax is obtained) Discrete values), according to Adjustment Convolution kernel rotation direction (e.g.) At that time, the convolution kernel rotates synchronously. ), adapts to any orientation of the target; Dilated convolution: After dynamic angle-adaptive convolution, the cascaded dilation rate is 2. Dilated convolution expands the receptive field from [previous stage] without increasing the number of parameters. Expand to This process captures the global contour of slender rotating targets (such as the drive axis in high-speed coronary rotational atherectomy). Then, a two-branch feature fusion is performed: a feature contribution coefficient is introduced. , by target scale With rotation angle Dynamic calculation, the calculation formula is as follows:

[0113]

[0114] Among them, when (Small goals) ,when (When rotating the target at a large angle) The fusion formula is:

[0115]

[0116] in For small target branch features, To achieve adaptive fusion of dual-task features, the target branch features are rotated.

[0117] In the aforementioned cross-scale feature extraction network, a new P2 feature layer (i.e., the first feature extraction model) is first added. Based on the original P2 layer (80×80) of YOLOv11n-obb, the resolution of the P2 layer feature map is upsampled from 80×80 to 160×160 through a transposed convolution layer (3×3 kernel, 2 stride, 1 padding, 64 output channels) to obtain the basic features of the P2 layer, which is suitable for the feature extraction requirements of small targets (pixel size 10×30).

[0118] Feature layer channel configuration: P2 layer 64 channels, P3 layer (i.e., the second feature extraction model) 128, P4 layer (i.e., the third feature extraction model) 256, P5 layer (i.e., the fourth feature extraction model) 512, P6 layer (i.e., the fifth feature extraction model) 1024, to ensure that the feature dimension of each layer matches the target scale (small targets correspond to low number of channels, and large targets correspond to high number of channels).

[0119] Then, a two-dimensional Gaussian distribution is constructed: for each feature layer from P2 to P6, the target angle is rotated. The encoding is a two-dimensional Gaussian distribution, which is then superimposed on the feature map as an "angle attention weight" to enable the feature layer to carry angle information. The specific steps are as follows:

[0120] First, for each region in the feature map, with the target center... The mean of the Gaussian distribution The covariance matrix is ​​calculated based on the angle, and its matrix is ​​as follows:

[0121]

[0122] in, , (w, h are normalized width and height). (Adjust the tilt direction of the ellipse according to the angle), and then generate a Gaussian weight map G with the same size as the feature map according to the two-dimensional Gaussian distribution formula. The calculation formula is as follows:

[0123]

[0124] The weight values ​​range from [0,1], with the target region having a weight close to 1 and the non-target region having a weight close to 0. After obtaining the weight map, the Gaussian weight map is then... With feature map Pixel-wise multiplication yields a feature map with angular information. This allows for the suppression of the background area and the enhancement of the target area.

[0125] Finally, cross-scale attention transfer and small target feature enhancement are performed. Gaussian similarity matching is used instead of traditional feature concatenation to prioritize the transfer of features with consistent angles. Taking the fusion of layer P2 (small target) and layer P3 (medium target) as an example, the Bhattacharyya distance (measuring the similarity between two Gaussian distributions) of the Gaussian distributions in the features of layers P2 and P3 is first calculated. The formula is as follows:

[0126]

[0127] in , The parameters are Gaussian distribution parameters for layer P2. , The parameters of the Gaussian distribution for layer P3 are given, and then the average value is calculated using the following formula:

[0128]

[0129] Then the obtained Bhattacharyya distance is converted into similarity weights. The smaller the distance, the higher the weight; however, a distance close to 1 also indicates a higher similarity. Then, their features are fused, calculated using the following formula:

[0130]

[0131] Upsample is a bilinear sampling operation that adjusts the resolution of layer P3 to be consistent with that of layer P2, and prioritizes the transmission of medium target features with consistent angles to the small target feature layer to supplement the semantic information of the small target.

[0132] A small target detail completion submodule is designed for the P2 layer. This submodule is added to the P2 layer to mitigate the loss of small target features caused by downsampling. The submodule structure is as follows: Convolution + Residual Linking

[0133] Convolution: Input the basic features of P2 layer, which are generally 64 channels, and output the features with 64 channels. The stride is 1, and "same" padding is used to integrate the feature dimensions.

[0134] Residual connections: obtained by downsampling the original YOLOv11n-OBB input layer by a factor of 1, with 32 channels. After adjusting the convolution channels, then... The formula for summing convolutional feature outputs is as follows:

[0135]

[0136] This allows you to add details to the edges of smaller targets.

[0137] Step 13: Construct a dual-task loss function based on all stem cell detection results, and use the dual-task loss function to train the target detection model to obtain the trained target detection model.

[0138] The aforementioned dual-task loss function is used to describe the accuracy of all stem cell detection results, that is, the difference between the small target detection box in the stem cell detection result and the corresponding real stem cell detection box.

[0139] For example, all stem cell detection results are input into a dual-task loss function for calculation to obtain the value of the loss function. If the value of the loss function is less than the preset loss function value, the training is completed and the trained target detection model is obtained. If the value of the loss function is greater than or equal to the preset loss value, the parameters in the target detection model are adjusted, and the process of using the target detection model to perform stem cell detection on each target stem cell image is returned to obtain the stem cell detection result corresponding to each target stem cell image.

[0140] The above dual-task loss function is:

[0141]

[0142] in, This represents the value of the dual-task loss function. Indicates dynamic weighting coefficients. Indicates small target loss. Indicates the loss of the target rotation angle:

[0143]

[0144]

[0145]

[0146] in, This indicates the number of small target detection boxes in all stem cell test results. Represents scale-aware weights. Indicates the first The classification prediction probability of a small object detection box. Indicates the focus factor. Indicates the average pixel size of the target. Indicates the first The pixel size of a small target detection box. Indicates the angle penalty coefficient. This indicates the center point location of the small target detection bounding box. This indicates the center point position of the actual detection box. Indicates the prediction angle. Representing the true perspective, This indicates Euclidean distance.

[0147] Step 14: Use the trained target detection model to perform stem cell detection on the stem cell image to be detected, and obtain the final stem cell detection result of the stem cell image to be detected.

[0148] The above images of stem cells to be detected are those that require stem cell detection. The final stem cell detection results are used to represent the small target detection boxes corresponding to the stem cell image regions in the images to be detected.

[0149] Specifically, the image of the stem cell to be detected is input into the trained target detection model for stem cell detection. The multi-scale feature extraction network in the target detection model performs multi-scale feature extraction on the image of the stem cell to be detected, and then the cross-scale feature extraction network is used to perform cross-scale feature extraction. Finally, the detection head network is used to detect the feature data and output the final stem cell detection result.

[0150] It is worth mentioning that using the target detection model to extract features from target stem cell images at multiple scales and across scales can effectively capture the features of stem cell images at multiple scales, as well as the feature relationships between different scales, thereby improving the understanding of stem cell images and effectively improving the accuracy of stem cell detection. Using multiple target stem cell images as training data to train the target detection model can improve the adaptability of the target detection model to different stem cell images, thereby improving the versatility of stem cell detection.

[0151] Furthermore, the target detection model in this application employs a 1×1+3×3 depthwise separable convolution combined with "channel-pixel dual attention" by inserting a small target branch into the C2f layer of the backbone network. This preserves the shallow details (such as cell nucleus outlines and cell edge textures) of early stem cell clones and individual suspended cells while suppressing background noise such as culture medium particles and bubbles. The rotation target branch adjusts the convolution kernel direction through dynamic angle-adaptive convolution and expands the receptive field with 5×5 dilated convolution to adapt to stem cells growing in any orientation. Simultaneously, a feature contribution coefficient is introduced to achieve adaptive fusion of features from both branches, avoiding feature competition between small and rotated target detection tasks. A cross-scale Gaussian attention feature pyramid is constructed, and a new P2 small target detection layer is added. The target angle is encoded as a two-dimensional Gaussian distribution and embedded as attention weights in each feature layer from P2 to P6. Gaussian similarity matching is used to prioritize the transfer of consistent features across scales. A detail completion submodule is designed in the P2 layer to alleviate feature loss in small targets such as early stem cell clones, improving the detection accuracy of both small and rotated targets while removing redundant computations. A dynamic non-monotonic focusing dual-task loss function is proposed to replace the traditional loss function. It optimizes the localization error of small targets (such as a single suspended cell) and the angle regression accuracy of rotated targets (such as adherent stem cells) respectively, solving the problems of imbalance between live and dead cell categories and inaccurate cell bounding box localization in target detection of stem cell preparation images.

[0152] The stem cell detection device provided in this application is described below as an example.

[0153] like Figure 5 As shown, this application embodiment provides a stem cell detection device, the stem cell detection device 500 including:

[0154] Acquisition module 501 is used to acquire multiple target stem cell images;

[0155] The first detection module 502 is used to perform stem cell detection on each target stem cell image using a target detection model, and obtain the stem cell detection result corresponding to each target stem cell image. The stem cell detection result is a small target detection box of the stem cell image in the target stem cell image. The target detection model includes a multi-scale feature extraction network, a cross-scale feature extraction network, and a detection head network. The multi-scale feature extraction network is used to perform multi-scale feature extraction on the target stem cell image. The cross-scale feature extraction network is used to perform cross-scale feature extraction based on the multi-scale feature extraction result. The detection head network is used to perform stem cell detection based on the cross-scale feature extraction result and output the stem cell detection result.

[0156] Training module 503 is used to construct a dual-task loss function based on all stem cell detection results, and to train the target detection model using the dual-task loss function to obtain the trained target detection model; the dual-task loss function is used to describe the accuracy of all stem cell detection results;

[0157] The second detection module 504 is used to perform stem cell detection on the stem cell image to be detected using the trained target detection model, and obtain the final stem cell detection result of the stem cell image to be detected.

[0158] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0160] like Figure 6 As shown, an embodiment of this application provides a terminal device, wherein the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 6 The diagram shows only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 executes the computer program D102 to implement the steps in any of the above method embodiments.

[0161] Specifically, when the processor D100 executes the computer program D102, it acquires multiple target stem cell images, then uses a target detection model to perform stem cell detection on each target stem cell image, obtaining the stem cell detection result corresponding to each target stem cell image. Based on all stem cell detection results, a dual-task loss function is constructed, and the target detection model is trained using this function to obtain a trained target detection model. Finally, the trained target detection model is used to perform stem cell detection on the target stem cell image, obtaining the final stem cell detection result for the target stem cell image. The use of the target detection model for multi-scale and cross-scale feature extraction of the target stem cell images effectively captures features at multiple scales and the feature relationships between different scales, improving the understanding of stem cell images and effectively improving the accuracy of stem cell detection. Using multiple target stem cell images as training data to train the target detection model improves its adaptability to different stem cell images, thereby enhancing the versatility of stem cell detection.

[0162] The processor D100 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0163] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may be an external storage device of the terminal device D10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device D10. Furthermore, the memory D101 may include both internal and external storage units of the terminal device D10. The memory D101 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory D101 can also be used to temporarily store data that has been output or will be output.

[0164] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0165] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0166] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the stem cell detection method device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0167] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0168] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0169] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention.

Claims

1. A method for detecting stem cells, characterized in that, include: Acquire multiple target stem cell images; A target detection model is used to perform stem cell detection on each target stem cell image to obtain a stem cell detection result corresponding to each target stem cell image. The stem cell detection result is a small target detection box of the stem cell image in the target stem cell image. The target detection model includes a multi-scale feature extraction network, a cross-scale feature extraction network, and a detection head network. The multi-scale feature extraction network is used to perform multi-scale feature extraction on the target stem cell image. The cross-scale feature extraction network is used to perform cross-scale feature extraction based on the multi-scale feature extraction result. The detection head network is used to perform stem cell detection based on the cross-scale feature extraction result and output the stem cell detection result. A dual-task loss function is constructed based on all stem cell detection results, and the target detection model is trained using the dual-task loss function to obtain the trained target detection model; the dual-task loss function is used to describe the accuracy of all stem cell detection results. The trained target detection model is used to perform stem cell detection on the image of the stem cell to be detected, and the final stem cell detection result of the image of the stem cell to be detected is obtained.

2. The stem cell detection method according to claim 1, characterized in that, The multi-scale feature extraction network includes a shallow feature extraction model, a medium-level feature extraction model, and a deep feature extraction model; The input terminals of the shallow feature extraction model, the mid-level feature extraction model, and the deep feature extraction model are all input terminals of the multi-scale feature extraction network, and the output terminals of the shallow feature extraction model, the mid-level feature extraction model, and the deep feature extraction model are all output terminals of the multi-scale feature extraction network.

3. The stem cell detection method according to claim 2, characterized in that, The shallow feature extraction model includes a shallow feature aggregation module and a first rotational convolution module connected in sequence. The input of the shallow feature aggregation module is the input of the shallow feature extraction model, and the output of the first rotational convolution module is the output of the shallow feature extraction model. The mid-level feature extraction model includes a mid-level feature aggregation module and a second rotational convolution module connected in sequence. The input of the mid-layer feature aggregation module is the input of the mid-layer feature extraction model, and the output of the second rotation convolution module is the output of the mid-layer feature extraction model. The deep feature extraction model includes a deep feature aggregation module and a third rotational convolution module connected in sequence; The input of the deep feature aggregation module is the input of the deep feature extraction model, and the output of the third rotational convolution module is the output of the deep feature extraction model.

4. The stem cell detection method according to claim 3, characterized in that, The first, second, and third rotational convolution modules are all rotational convolution modules; The rotational convolution module includes a small target branch convolutional unit, a rotational target branch convolutional unit, and a feature fusion unit; The input terminals of the small target branch convolutional unit and the rotating target branch convolutional unit are both input terminals of the rotating convolutional module, and the output terminal of the feature fusion unit is the output terminal of the rotating convolutional module. The outputs of the small target branch convolutional unit and the rotated target branch convolutional unit are both connected to the input of the feature fusion unit.

5. The stem cell detection method according to claim 4, characterized in that, The small target branch convolutional unit includes a first depthwise separable convolutional layer, a second depthwise separable convolutional layer, a channel attention layer, and a pixel attention layer connected in sequence. The input end of the first depth-separable convolutional layer is the input end of the small target branch convolutional unit, and the output end of the pixel attention layer is the output end of the small target branch convolutional unit; The rotating target branch convolutional unit includes an angle prediction layer, an angle adaptation convolutional layer, and a dilated convolutional layer connected in sequence. The input of the angle prediction layer is the input of the rotation target branch convolutional unit, and the output of the dilated convolutional layer is the output of the rotation target branch convolutional unit. The feature fusion unit includes a feature contribution coefficient calculation layer and a fusion layer connected in sequence. The input terminal of the feature contribution coefficient calculation layer is the input terminal of the feature fusion unit, and the output terminal of the fusion layer is the output terminal of the feature fusion unit.

6. The stem cell detection method according to claim 1, characterized in that, The cross-scale feature extraction network includes: a first feature extraction model, a second feature extraction model, a third feature extraction model, a fourth feature extraction model, and a fifth feature extraction model; The input terminals of the first feature extraction model, the second feature extraction model, and the third feature extraction model are all input terminals of the cross-scale feature extraction network; The output terminals of the first feature extraction model, the second feature extraction model, the third feature extraction model, the fourth feature extraction model, and the fifth feature extraction model are all output terminals of the cross-scale feature extraction network. The output of the second feature extraction model is connected to the input of the first feature extraction model. The output of the third feature extraction model is connected to the input of the second feature extraction model and the input of the fourth feature extraction model. The output of the fourth feature extraction model is connected to the input of the third feature extraction model and the input of the fifth feature extraction model. The output of the fifth feature extraction model is connected to the input of the fourth feature extraction model.

7. The stem cell detection method according to claim 1, characterized in that, The dual-task loss function is: in, This represents the value of the dual-task loss function. Indicates dynamic weighting coefficients. Indicates small target loss. Indicates the loss of the target rotation angle: in, This indicates the number of small target detection boxes in all stem cell test results. Represents scale-aware weights. Indicates the first The classification prediction probability of a small object detection box. Indicates the focus factor. Indicates the average pixel size of the target. Indicates the first The pixel size of a small target detection box. Indicates the angle penalty coefficient. This indicates the center point location of the small target detection bounding box. This indicates the center point position of the actual detection box. Indicates the prediction angle. Representing the true perspective, This indicates Euclidean distance.

8. A stem cell detection device, characterized in that, include: The acquisition module is used to acquire multiple target stem cell images; The first detection module is used to perform stem cell detection on each of the target stem cell images using a target detection model, and obtain the stem cell detection result corresponding to each target stem cell image; the stem cell detection result is a small target detection box of the stem cell image in the target stem cell image; the target detection model includes a multi-scale feature extraction network, a cross-scale feature extraction network, and a detection head network; the multi-scale feature extraction network is used to perform multi-scale feature extraction on the target stem cell image; the cross-scale feature extraction network is used to perform cross-scale feature extraction based on the multi-scale feature extraction result; and the detection head network is used to perform stem cell detection based on the cross-scale feature extraction result and output the stem cell detection result. The training module is used to construct a dual-task loss function based on all stem cell detection results, and to train the target detection model using the dual-task loss function to obtain the trained target detection model; the dual-task loss function is used to describe the accuracy of all stem cell detection results. The second detection module is used to perform stem cell detection on the stem cell image to be detected using the trained target detection model, and obtain the final stem cell detection result of the stem cell image to be detected.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the stem cell detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the stem cell detection method as described in any one of claims 1 to 7.

Images