Biological sample image detection method, device, computer equipment and storage medium

Abstract

This application relates to a method, apparatus, computer device, and storage medium for detecting biological sample images. The method includes: obtaining each image to be detected, where each image to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope; using the pre-trained image detection models corresponding to each focal length to perform image detection on the images to be detected corresponding to the focal lengths respectively, to obtain the initial image detection results of each image to be detected; and screening each initial image detection result to obtain the image detection result corresponding to the biological sample to be detected. By using the method of the embodiments of this application to detect images of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope, the influence of the blurriness of a single image can be reduced, missed detection can be avoided, and the final image detection result can be obtained by screening among the initial image detection results, thereby improving the detection accuracy and thus the detection efficiency of biological samples.

Description

Technical Field

[0001] This application relates to the technical field of medical image detection, and particularly to a method, device, computer device, and storage medium for detecting biological sample images. Background Art

[0002] In medical clinics, biological samples include samples of human organ tissues, whole blood, plasma, serum, biological body fluids, etc. Taking urine sediment biological samples as an example, urine sediment is the formed components in urine, which is the sediment formed after urine is centrifuged. Urine sediment mainly includes various formed components such as cells, casts, crystals, bacteria, sperm, etc. Urine sediment examination refers to examining the above-mentioned formed components under a microscope. Urine sediment examination is very important for the diagnosis of urinary system diseases and cannot be replaced by urine dry chemical analyzers.

[0003] In medical clinics, the biological samples are generally detected by the artificial microscopy method. However, the artificial microscopy method is cumbersome, time-consuming, inefficient, and easily affected by subjective factors. Traditional image detection algorithms are also easily affected by the image quality of biological samples, resulting in low detection efficiency of biological samples. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, and storage medium for detecting biological sample images that can effectively improve the detection efficiency of biological samples.

[0005] A method for detecting biological sample images, the method includes:

[0006] Obtain each image to be detected, and each image to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope;

[0007] Use the pre-trained image detection models corresponding to each of the focal lengths to perform image detection on the images to be detected corresponding to the focal lengths respectively, and obtain the initial image detection results of each of the images to be detected;

[0008] Screen each of the initial image detection results to obtain the image detection result corresponding to the biological sample to be detected.

[0009] In one of the embodiments, the training process of the image detection models corresponding to each of the focal lengths includes:

[0010] Obtain the sample sets corresponding to each of the focal lengths, and the sample sets include sample images of biological samples at each of the focal lengths in the same field of view of a microscope;

[0011] According to the sample sets corresponding to the respective focal lengths, train the network models to be trained corresponding to the respective focal lengths, and obtain the image detection models corresponding to the respective focal lengths.

[0012] In one embodiment, the obtaining the sample sets corresponding to the respective focal lengths includes:

[0013] Obtain the original sample images of the biological sample at the respective focal lengths in the same field of view of the microscope;

[0014] Perform image preprocessing on the original sample images to obtain the respective sample images of the biological sample.

[0015] In one embodiment, the performing image preprocessing on the original sample images to obtain the respective sample images of the biological sample includes:

[0016] Perform image denoising processing on the original sample images to obtain the respective sample images of the biological sample; the image denoising processing includes at least one of spatial domain denoising processing, transform domain denoising processing, and collaborative denoising processing of spatial domain and transform domain of the image.

[0017] In one embodiment, the performing image preprocessing on the original sample images to obtain the respective sample images of the biological sample includes:

[0018] Perform image enhancement processing on the original sample images to obtain the respective enhanced sample images of the biological sample; the sample images include the respective original sample images and the respective enhanced sample images;

[0019] The performing image enhancement processing on the original sample images includes at least one of the following:

[0020] Translate the pixel coordinates of the original sample image by a random pixel unit along the horizontal or vertical direction, the random pixel unit is less than a preset pixel unit, and the random pixel unit includes at least one;

[0021] Scale the original sample image based on a random multiple, the random multiple is greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one;

[0022] Rotate the original sample image by a random angle in a random direction, the random angle is less than a preset angle, and the random angle includes at least one;

[0023] Randomly flip the original sample image horizontally or vertically;

[0024] Randomly increase or decrease the pixel coordinates of the original sample image by a random pixel value, where the random pixel value is less than a preset pixel value and the random pixel value includes at least one;

[0025] Process the contrast of the original sample image by a random multiple, where the random multiple is greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one;

[0026] Process the brightness of the original sample image by a random multiple, where the random multiple is greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one.

[0027] In one embodiment, the step of training the network models to be trained corresponding to each focal length according to the sample sets corresponding to each focal length to obtain the image detection models corresponding to each focal length includes:

[0028] For the sample set and the network model to be trained corresponding to any one focal length, perform the following processing:

[0029] Adopt a cross-stage local network structure to extract the sample image features of each sample image in the sample set;

[0030] Based on at least one of a path aggregation network structure and a spatial pyramid pooling network structure, perform convolution on the sample image features to obtain enhanced sample image features;

[0031] According to the object detection function and the enhanced sample image features, classify and locate each sample image to obtain an image training result;

[0032] When it is determined based on the image training result that the training end condition is not satisfied, adjust the parameters of the path aggregation network structure or the spatial pyramid pooling network structure, and return to the step of performing convolution on the sample image features based on at least one of the path aggregation network structure and the spatial pyramid pooling network structure until the training end condition is satisfied.

[0033] In one embodiment, the method for determining whether the training end condition is satisfied based on the image training result includes:

[0034] Calculate the test confidence between the image training result of each sample image and the image calibration result corresponding to the sample image;

[0035] Compare the test confidence level with a preset threshold. When the test confidence level is greater than the preset threshold, it is determined that the training end condition is satisfied; otherwise, it is determined that the training end condition is not satisfied.

[0036] In one embodiment, screening each of the initial image detection results to obtain the image detection result corresponding to the biological sample to be detected includes:

[0037] Obtain the probability scores corresponding to each of the initial image detection results, where the probability scores are calculated based on the corresponding activation function when each image detection model performs image detection on the image to be detected.

[0038] Determine the initial image detection result with the highest probability score as the image detection result corresponding to the biological sample to be detected.

[0039] A biological sample image detection device, the device includes:

[0040] An image acquisition module, configured to acquire each image to be detected, where each image to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope;

[0041] An image detection module, configured to use the image detection models corresponding to each of the focal lengths that have been pre-trained to perform image detection on the image to be detected corresponding to the focal length, respectively, to obtain the initial image detection results of each image to be detected;

[0042] A result determination module, configured to screen each of the initial image detection results to obtain the image detection result corresponding to the biological sample to be detected.

[0043] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned biological sample image detection method are implemented.

[0044] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned biological sample image detection method are implemented.

[0045] The above-mentioned biological sample image detection method, device, computer equipment and storage medium obtain each image to be detected, where each image to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope; adopt the image detection models corresponding to each focal length that have been pre-trained to perform image detection on the images to be detected corresponding to the focal lengths respectively, and obtain the initial image detection results of each image to be detected; screen each initial image detection result to obtain the image detection result corresponding to the biological sample to be detected. By using the method of the above embodiment to detect images of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope, the influence of the blurriness of a single image can be reduced, missed detection can be avoided, and the final image detection result is obtained by screening from each initial image detection result, thereby improving the detection accuracy and thus improving the detection efficiency of the biological sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is an application environment diagram of the biological sample image detection method in one embodiment;

[0047] Figure 2 It is a flowchart of the biological sample image detection method in one embodiment;

[0048] Figure 3 It is a schematic diagram of the image detection model in one specific embodiment;

[0049] Figure 4 It is a schematic diagram of the biological sample image detection method in one specific embodiment;

[0050] Figure 5 It is a structural block diagram of the biological sample image detection device in one embodiment;

[0051] Figure 6 It is an internal structure diagram of the computer equipment in one embodiment;

[0052] Figure 7 It is an internal structure diagram of the computer equipment in another embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] In one of the embodiments, the application environment of the biological sample image detection method provided by the present application may involve both the terminal 102 and the server 104 at the same time, as Figure 1As shown in the figure. Among them, the terminal 102 can communicate with the server 104 through networks, protocols, or other means. In the server 104, image detection models corresponding to each focal length can be pre-trained. Specifically, the server 104 obtains each image to be detected through the terminal 102. Each image to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope. The image detection models corresponding to each focal length that are pre-trained are used to perform image detection on the images to be detected corresponding to the focal lengths respectively, so as to obtain the initial image detection results of each image to be detected. The initial image detection results are screened to obtain the image detection result corresponding to the biological sample to be detected.

[0055] In one embodiment, the biological sample image detection method provided in this application may have an application environment that only involves the server 104. In the server 104, image detection models corresponding to each focal length can be pre-trained. Specifically, the server 104 directly obtains each image to be detected and, based on the image detection models corresponding to each focal length that are pre-trained, obtains the image detection result corresponding to the biological sample to be detected.

[0056] In one embodiment, the biological sample image detection method provided in this application may have an application environment that only involves the terminal 102. In the terminal 102, image detection models corresponding to each focal length can be pre-trained. Specifically, the terminal 102 directly obtains each image to be detected and, based on the image detection models corresponding to each focal length that are pre-trained, obtains the image detection result corresponding to the biological sample to be detected.

[0057] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0058] In one embodiment, as Figure 2 shown, a biological sample image detection method is provided. Taking the case where this method is applied to the terminal 102 and / or the server 104 in Figure 1 as an example, the method includes the following steps:

[0059] Step S202: Obtain each image to be detected. Each image to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope.

[0060] Among them, biological samples include samples of human organ tissues, whole blood, plasma, serum, biological body fluids, etc. Urinary sediment is the formed components in urine, which is the sediment formed after urine centrifugation. Urinary sediment mainly includes various formed components such as cells, casts, crystals, bacteria, sperm, etc. When the biological sample is a urinary sediment biological sample, after obtaining the patient's urinary sediment biological sample, it is necessary to prepare the urinary sediment biological sample into a urinary sediment biological sample that can be photographed or detected under a microscope. The biological sample to be detected is called the biological sample to be detected, and the microscopic image of the biological sample to be detected taken under the microscope is called the image to be detected. Specifically, the microscope can be but is not limited to various optical microscopes, electron microscopes, fluorescence microscopes, transmission electron microscopes, scanning electron microscopes, reflection electron microscopes, and emission electron microscopes, etc.

[0061] In one embodiment, the image to be detected is at least one, and each image to be detected is an image of the biological sample to be detected at at least two focal lengths in the same field of view of the microscope. Moreover, in order to ensure the image quality, the image at the same focal length is at least one. Among them, when the microscope is an optical microscope, the focal length refers to the distance from the optical center of the lens to the focus where the parallel light converges. The microscope moves along its Z-axis to reach different focal lengths, so as to take images at each focal length. Moreover, the images taken by the microscope at each focal length are in the same field of view. Among them, the field of view refers to the spatial range that can be seen in the microscope, so as to ensure that the difference between each image to be detected only includes different focal lengths, so as to improve the detection accuracy of the image.

[0062] Step S204, using the pre-trained image detection models corresponding to each focal length, respectively perform image detection on the images to be detected corresponding to the focal lengths, and obtain the initial image detection results of each image to be detected.

[0063] In one embodiment, the neural network model for performing image detection on the image to be detected is called an image detection model, and the image detection model is obtained through pre-training. The image detection model corresponds to the focal length, that is, different focal lengths correspond to different image detection models. Since the focal length is at least two, the image detection models are also at least two. Specifically, using the pre-trained image detection models corresponding to each focal length, respectively perform image detection on each image to be detected, locate and classify the formed components in the image to be detected, obtain the initial positioning results and the corresponding classification results, and call the initial positioning results and the corresponding classification results the initial image detection results, and obtain the initial image detection results of each image to be detected.

[0064] Step S206, screen each initial image detection result to obtain the image detection result corresponding to the biological sample to be detected.

[0065] In one embodiment, after obtaining the initial image detection results of each image, it is necessary to screen the initial image detection results to determine the final localization result and the corresponding classification result of the formed components in the image to be detected, and use the final localization result and the corresponding classification result as the image detection result corresponding to the biological sample to be detected.

[0066] In the above biological sample image detection method, by obtaining each image to be detected, each image to be detected is an image of the biological sample to be detected at at least two focal lengths in the same field of view of the microscope; using the pre-trained image detection models corresponding to each focal length, respectively perform image detection on the images to be detected corresponding to the focal lengths to obtain the initial image detection results of each image to be detected; screen each initial image detection result to obtain the image detection result corresponding to the biological sample to be detected. By using the method of the above embodiment, by detecting the images of the biological sample to be detected at at least two focal lengths in the same field of view of the microscope, the influence of the blurriness of a single image can be reduced, missing detections can be avoided, and the final image detection result is obtained by screening among the initial image detection results, improving the detection accuracy, thereby improving the detection efficiency of the biological sample.

[0067] In one embodiment, the training process of the image detection models corresponding to each focal length includes:

[0068] Step S302, obtain the sample sets corresponding to each focal length, and the sample sets include the sample images of the biological sample at each focal length in the same field of view of the microscope.

[0069] In one embodiment, different focal lengths correspond to different image detection models. When training the image detection models, they can be trained separately according to different sample sets. Specifically, obtain the sample sets corresponding to each focal length. Among them, the sample sets include the sample images of the biological sample at each focal length in the same field of view of the microscope.

[0070] In one embodiment, after obtaining the sample sets corresponding to each focal length, the sample sets can be divided into a training sample set and a test sample set according to a preset ratio, and the training sample images in the training sample set do not overlap with the test sample images in the test sample set. Among them, the preset ratio can be set to 7:3, that is, the training sample images in the training sample set account for 70% of the total sample images in the sample set, and the test sample images in the test sample set account for 30% of the total sample images in the sample set.

[0071] Step S304, according to the sample sets corresponding to each focal length, respectively train the network models to be trained corresponding to each focal length to obtain the image detection models corresponding to each focal length.

[0072] In one embodiment, the image detection models corresponding to each focal length can be obtained by separately training the network models to be trained. That is, according to the sample sets corresponding to each focal length, the network models to be trained corresponding to each focal length are separately trained. Among them, the network structures of the network models to be trained are the same. Therefore, the network structures of the image detection models corresponding to each focal length obtained are also the same.

[0073] In one embodiment, step S302 of obtaining the sample sets corresponding to each focal length includes:

[0074] Step S402 of obtaining the original sample images of the biological sample at each focal length in the same field of view of the microscope.

[0075] In one embodiment, the microscopic images of the biological sample at each focal length in the same field of view of the captured microscope are referred to as the original sample images. After image preprocessing of the original sample images, the sample images of the biological sample are obtained to improve the detection accuracy of the image detection model trained for the biological sample.

[0076] Step S404 of performing image preprocessing on the original sample images to obtain the sample images of the biological sample.

[0077] Among them, image preprocessing refers to the processing performed before feature extraction, image segmentation, image matching, and image recognition of an image. The main purpose is to eliminate irrelevant information in the image, restore useful real information, enhance the detectability of information, simplify the data, and improve the reliability of feature extraction, image segmentation, image matching, and image recognition.

[0078] In one embodiment, step S404 of performing image preprocessing on the original sample images to obtain the sample images of the biological sample includes: performing image denoising processing on the original sample images to obtain the sample images of the biological sample. Among them, image denoising processing refers to removing noise in the image, mainly including at least one of spatial domain denoising processing, transform domain denoising processing, and collaborative denoising processing of the spatial domain and the transform domain. Specifically, at least one of median filtering, mean filtering, Gaussian filtering, and building an image pyramid can be used for image denoising processing.

[0079] In one embodiment, step S404 of performing image preprocessing on the original sample images to obtain the sample images of the biological sample includes: performing image enhancement processing on the original sample images to obtain the enhanced sample images of the biological sample. Specifically, the sample images of the biological sample include the original sample images and the enhanced sample images. Among them, image enhancement processing refers to enhancing the useful information in the image, mainly used to emphasize the overall or local characteristics of the image, expand the differences between the features of different objects in the image, and improve the image quality and enrich the information content.

[0080] In one of the embodiments, the image enhancement processing of the original sample image includes at least one of the following:

[0081] First, along the horizontal or vertical direction, randomly shift the pixel coordinates of the original sample image by a random pixel unit, where the random pixel unit is less than the preset pixel unit, and the random pixel unit includes at least one. Among them, the value range of the preset pixel unit can be set to 5 - 10.

[0082] Second, scale the original sample image based on a random multiple, where the random multiple is greater than or equal to the first preset multiple and less than or equal to the second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one. Among them, the first preset multiple is set to a positive number less than 1 and greater than 0.5, specifically it can be set to 0.8, and the second preset multiple is set to a positive number greater than 1 and less than 1.5, specifically it can be set to 1.2.

[0083] Third, rotate the original sample image by a random angle in a random direction, where the random angle is less than the preset angle, and the random angle includes at least one. Among them, the random direction can be the clockwise or counterclockwise direction, and the value range of the preset angle can be set to 30° - 90°.

[0084] Fourth, randomly flip the original sample image horizontally or vertically. Among them, flipping means transforming the pixel positions of the entire original sample image without changing the color of the original sample image.

[0085] Fifth, randomly increase or decrease the pixel coordinates of the original sample image by a random pixel value, where the random pixel value is less than the preset pixel value, and the random pixel value includes at least one. Among them, the value range of the preset pixel value can be set to 5 - 10.

[0086] Sixth, process the contrast of the original sample image by a random multiple, where the random multiple is greater than or equal to the first preset multiple and less than or equal to the second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one. Among them, the first preset multiple is set to a positive number less than 1 and greater than 0.5, specifically it can be set to 0.8, and the second preset multiple is set to a positive number greater than 1 and less than 1.5, specifically it can be set to 1.2.

[0087] Seventh, process the brightness of the original sample image by a random multiple, where the random multiple is greater than or equal to the first preset multiple and less than or equal to the second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one. Among them, the first preset multiple is set to a positive number less than 1 and greater than 0, specifically it can be set to 0.8, and the second preset multiple is set to a positive number greater than 1, specifically it can be set to 1.2.

[0088] In one embodiment, according to the sample sets corresponding to each focal length, the network models to be trained corresponding to each focal length are respectively trained to obtain the image detection models corresponding to each focal length, including:

[0089] Since the image detection models are respectively trained according to each sample set, for any sample set and the network model to be trained corresponding to a focal length, the following processes are respectively executed:

[0090] Step S502: Using a cross-stage local network structure, extract the sample image features of each sample image in the sample set.

[0091] In one embodiment, the network model to be trained includes a feature extraction (Backbone) network module.

[0092] Among them, the cross-stage local network (CSPNet) structure can achieve a richer gradient combination of the features (Feature) of the image, and through the cross-stage hierarchical structure, reduce the computational amount at the same time. Specifically, the Backbone network module can adopt the cross-stage local network structure CSPDarknet53 to extract the sample image features of each sample image in the sample set to obtain a feature map Feature Map.

[0093] Step S504: Based on at least one of the path aggregation network structure and the spatial pyramid pooling network structure, perform convolution on the sample image features to obtain enhanced sample image features.

[0094] In one embodiment, the network model to be trained includes a feature enhancement (Neck) network module. Among them, the Neck network module is mainly used to perform convolution on the sample image features to obtain enhanced sample image features, and is composed of a path aggregation network (PANet) structure and a spatial pyramid pooling network (SPP) structure. Specifically, the path aggregation network (PANet) structure can accurately retain the spatial information of the image, which is beneficial to accurately locating pixel points, and the spatial pyramid pooling network (SPP) structure is beneficial to improving the scale invariance of the image and can also reduce overfitting.

[0095] Step S506: According to the object detection function and the enhanced sample image features, classify and locate each sample image to obtain the image training result.

[0096] In one embodiment, the network model to be trained includes a classification and localization (Head) network module. Among them, the Head network module is mainly used to classify and locate each sample image according to the object detection function and the enhanced sample image features, and use the classification and localization results of the formed components in each sample image as the image training results to obtain the image training results. Specifically, the object detection function can be the Head detection of the YOLOV4 algorithm.

[0097] Step S508, when it is determined that the training end condition is not satisfied based on the image training results, adjust the parameters of the path aggregation network structure or the spatial pyramid pooling network structure, and return to the step of performing convolution on the sample image features based on at least one of the path aggregation network structure and the spatial pyramid pooling network structure until the training end condition is satisfied.

[0098] In one embodiment, the training end condition is a preset end condition. Among them, the training end condition can be set as the detection accuracy of the model, or can also be set as the number of iterations of the model. Specifically, when the training end condition is the detection accuracy of the model, determine the detection accuracy based on the image detection results, and compare the detection accuracy with the preset accuracy threshold. When the detection accuracy is less than the preset accuracy threshold, it is determined that the training end condition is not satisfied. When the training end condition is the number of iterations of the model, when the number of iterations of the model has not reached the preset number of iterations, it is determined that the training end condition is not satisfied.

[0099] In one embodiment, when it is determined that the training end condition is not satisfied based on the image training results, the parameters of the path aggregation network structure or the spatial pyramid pooling network structure can be adjusted based on the backpropagation algorithm. Among them, the parameters specifically include input parameters, weights, output parameters, activation functions, etc., and return to step S504 until the training end condition is satisfied.

[0100] In one embodiment, the method for determining whether the training end condition is satisfied based on the image training results includes:

[0101] Step S602, calculate the test confidence between the image training results of each sample image and the image calibration results corresponding to the sample image.

[0102] In one embodiment, the training end condition can be set as the confidence of the image training results. Among them, the confidence of the image training results refers to the intersection over union (IoU) corresponding to each sample image. Specifically, based on the image calibration results pre-calibrated in each sample image, calculate the intersection over union between the image training results of each sample image and the image calibration results corresponding to the sample image, which is also called the test confidence.

[0103] Step S604: Compare the test confidence with a preset threshold. When the test confidence is greater than the preset threshold, it is determined that the training end condition is met; otherwise, it is determined that the training end condition is not met.

[0104] In one embodiment, the test confidence is compared with a preset threshold, and whether the training end condition is met is determined based on the comparison result. The preset threshold can be set to 0.5. Specifically, when using a test sample set to test the trained model, the model will predict a series of candidate boxes, that is, the image training results. At this time, the non-maximum suppression (NMS) algorithm can be used to initially remove redundant candidate boxes, and whether the detection is correct is determined according to the IoU value among the remaining candidate boxes. When the IoU value is greater than 0.5, it is considered that the detection is correct. That is, when the test confidence is greater than the preset threshold, it is determined that the training end condition is met; otherwise, it is determined that the training end condition is not met.

[0105] In one embodiment, step S206 screens each initial image detection result to obtain the image detection result corresponding to the biological sample to be detected, including:

[0106] Step S702: Obtain the probability scores corresponding to each initial image detection result. The probability scores are calculated based on the corresponding activation function when each image detection model performs image detection on the image to be detected.

[0107] In one embodiment, when each image detection model performs image detection on the image to be detected, it also outputs the probability scores corresponding to each initial image detection result. Among them, for each localization result of the formed components in the image to be detected, the probability scores of each corresponding classification result will be obtained. Specifically, for the same localization result, there may be multiple classification results, and each classification result has a probability score.

[0108] In one embodiment, the probability scores are calculated based on the corresponding activation function when each image detection model performs image detection on the image to be detected. Specifically, the activation function of the image detection model can be one of the sigmoid function, hyperbolic tangent (Tanh) function, and rectified linear unit (ReLU) function.

[0109] Step S704: Determine the initial image detection result with the highest probability score as the image detection result corresponding to the biological sample to be detected.

[0110] In one embodiment, the classification result corresponding to the positioning result can be determined by comparing the probability scores of multiple classification results of the same positioning result. Among them, the classification result with the highest probability score is used as the classification result corresponding to the positioning result. That is, the initial image detection result with the highest probability score is determined as the image detection result corresponding to the biological sample to be detected.

[0111] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and one specific embodiment. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0112] In one specific embodiment, taking the urine sediment biological sample as an example of the biological sample, the biological sample image detection method includes the training process and the running process of the image detection model. The specific steps are as follows:

[0113] I. Training process

[0114] Place the urine sediment biological sample under the microscope, move the microscope along the Z-axis, and obtain the original sample images of the urine sediment biological sample at focal lengths A, B, and C in the same field of view of the microscope;

[0115] Perform image denoising processing on the original sample images by using the median filtering method to obtain the sample images of the urine sediment biological sample;

[0116] Perform image enhancement processing on the original sample images to obtain the enhanced sample images of the urine sediment biological sample. The image enhancement processing includes:

[0117] (1) Translate the pixel coordinates of the original sample image 5-10 pixel units along the horizontal or vertical direction;

[0118] (2) Scale the original sample image based on a random multiple, where the random multiple is greater than or equal to 0.8 and less than or equal to 1.2;

[0119] (3) Rotate the original sample image 30°-90° in the clockwise or counterclockwise direction;

[0120] (4) Randomly flip the original sample image horizontally or vertically;

[0121] (5) Randomly increase or decrease the pixel coordinates of the original sample image by 5-10 pixel values;

[0122]

[0122] (6) Process the contrast of the original sample image with a random multiple, where the random multiple is greater than or equal to 0.8 and less than or equal to 1.2; [[ID=(37]]

[0123] (7) Process the brightness of the original sample image by a random multiple, where the random multiple is greater than or equal to 0.8 and less than or equal to 1.2;

[0124] Obtain the sample set A corresponding to the focal length A, the sample set B corresponding to the focal length B, and the sample set C corresponding to the focal length C of the urinary sediment biological sample respectively. The sample images in the sample set A, the sample set B, and the sample set C include each original sample image and each enhanced sample image;

[0125] Divide the sample set A into a training set A and a test set A, divide the sample set B into a training set B and a test set B, and divide the sample set C into a training set C and a test set C respectively. The division ratio is 7:3 and non-overlapping;

[0126] According to the sample sets corresponding to each focal length, train the network models to be trained corresponding to each focal length respectively, and obtain the image detection model A corresponding to the focal length A, the image detection model B corresponding to the focal length B, and the image detection model C corresponding to the focal length C. As Figure 3 shown in the schematic diagram of the image detection model. The structures of each image detection model are the same, specifically including a Backbone module, a Neck module, and a Head module. Among them, the Backbone module uses CSPDarknet53 for feature extraction to obtain the Feature Map; the Feature Map is enhanced in the Neck module combined with PANet and SPP; the Head module is based on the Head detection of YOLOV4 and outputs the image detection result.

[0127] II. Running Process

[0128] As Figure 4 shown in the schematic diagram of the biological sample image detection method, place the urinary sediment biological sample to be detected under the microscope, move the microscope along the Z-axis, and obtain the images A, B, and C of the urinary sediment biological sample to be detected under the same field of view of the microscope at the focal lengths A, B, and C;

[0129] Use the image detection model A to detect the image A, use the image detection model B to detect the image B, and use the image detection model C to detect the image C respectively, and obtain the localization and classification result A corresponding to the image A, the localization and classification result B corresponding to the image B, and the localization and classification result C corresponding to the image C respectively;

[0130] Compare the probability scores score corresponding to each localization and classification result, and use the localization and classification result with the highest score as the image detection result corresponding to the urinary sediment biological sample to be detected.

[0131] It should be understood that although Figure 2The steps in the flowchart are shown in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least a part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0132] In one embodiment, as Figure 5 shown, a biological sample image detection device is provided, including: an image acquisition module 510, an image detection module 520, and a result determination module 530, where:

[0133] The image acquisition module 510 is used to acquire each image to be detected, and each of the images to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope.

[0134] The image detection module 520 is used to respectively perform image detection on the images to be detected corresponding to the focal lengths by using the image detection models corresponding to the respective focal lengths that have been pre-trained, and obtain the initial image detection results of the images to be detected.

[0135] The result determination module 530 is used to screen each of the initial image detection results to obtain the image detection result corresponding to the biological sample to be detected.

[0136] In one embodiment, the biological sample image detection device further includes:

[0137] An image detection model training module, which is used to train and obtain the image detection models corresponding to the respective focal lengths.

[0138] In one embodiment, the image detection model training module includes the following units:

[0139] A sample set acquisition unit, which is used to acquire the sample sets corresponding to the respective focal lengths, and the sample set includes sample images of a biological sample at the respective focal lengths in the same field of view of a microscope.

[0140] An image detection model training unit, which is used to respectively train the network models to be trained corresponding to the respective focal lengths according to the sample sets corresponding to the respective focal lengths, and obtain the image detection models corresponding to the respective focal lengths.

[0141] In one embodiment, the sample set acquisition unit includes the following units:

[0142] An original sample image acquisition unit, configured to acquire original sample images of the biological sample at each of the focal lengths in the same field of view of the microscope.

[0143] An image preprocessing unit, configured to perform image preprocessing on the original sample images to obtain the sample images of the biological sample.

[0144] In one embodiment, the image preprocessing unit includes the following units:

[0145] An image denoising processing unit, configured to perform image denoising processing on the original sample images to obtain the sample images of the biological sample; the image denoising processing includes at least one of spatial domain denoising processing, transform domain denoising processing, and collaborative denoising processing of the spatial domain and the transform domain of the image.

[0146] An image enhancement processing unit, configured to perform image enhancement processing on the original sample images to obtain enhanced sample images of the biological sample; the sample images include the original sample images and the enhanced sample images; the performing image enhancement processing on the original sample images includes at least one of the following: translating the pixel coordinates of the original sample image by a random pixel unit along the horizontal or vertical direction, the random pixel unit being less than a preset pixel unit and the random pixel unit including at least one; scaling the original sample image based on a random multiple, the random multiple being greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple being less than 1, the second preset multiple being greater than 1, and the random multiple including at least one; rotating the original sample image by a random angle in a random direction, the random angle being less than a preset angle and the random angle including at least one; randomly flipping the original sample image horizontally or vertically; randomly increasing or decreasing the pixel coordinates of the original sample image by a random pixel value, the random pixel value being less than a preset pixel value and the random pixel value including at least one; processing the contrast of the original sample image by a random multiple, the random multiple being greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple being less than 1, the second preset multiple being greater than 1, and the random multiple including at least one; processing the brightness of the original sample image by a random multiple, the random multiple being greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple being less than 1, the second preset multiple being greater than 1, and the random multiple including at least one.

[0147] In one embodiment, the image detection model training unit includes the following units:

[0148] A feature extraction unit, configured to extract sample image features of each sample image in the sample set by using a cross-stage partial network structure.

[0149] A feature enhancement unit, configured to perform convolution on the sample image features based on at least one of a path aggregation network structure and a spatial pyramid pooling network structure, or the path aggregation network structure and the spatial pyramid pooling network structure adjusted by a parameter adjustment unit, to obtain enhanced sample image features.

[0150] A training result acquisition unit, configured to classify and locate each of the sample images according to an object detection function and the enhanced sample image features, to obtain an image training result.

[0151] A training end determination unit, configured to determine whether a training end condition is satisfied based on the image training result.

[0152] A parameter adjustment unit, configured to adjust parameters of the path aggregation network structure or the spatial pyramid pooling network structure when the training end determination unit determines, based on the image training result, that the training end condition is not satisfied.

[0153] In one embodiment, the training end determination unit includes the following units:

[0154] A test confidence calculation unit, configured to calculate a test confidence between the image training result of each sample image and the image calibration result corresponding to the sample image.

[0155] A test confidence comparison unit, configured to compare the test confidence with a preset threshold, and when the test confidence is greater than the preset threshold, determine that the training end condition is satisfied; otherwise, determine that the training end condition is not satisfied.

[0156] In one embodiment, the result determination module 530 includes the following units:

[0157] A probability score acquisition unit, configured to acquire a probability score corresponding to each of the initial image detection results, where the probability score is calculated based on a corresponding activation function when each of the image detection models performs image detection on the image to be detected.

[0158] A result determination unit, configured to determine the initial image detection result with the highest probability score as the image detection result corresponding to the biological sample to be detected.

[0159] For the specific limitations of the biological sample image detection device, reference can be made to the limitations of the biological sample image detection method in the foregoing text, which will not be elaborated here. Each module in the above biological sample image detection device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0160] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store biological sample image detection data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a biological sample image detection method.

[0161] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a biological sample image detection method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0162] Those skilled in the art can understand that Figure 6 and Figure 7The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps of the above-mentioned biological sample image detection method are implemented.

[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned biological sample image detection method are implemented.

[0165] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0166] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0167] The above-described embodiments only represent several implementation manners of this application, and their descriptions are relatively specific and detailed, but they should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application patent should be subject to the appended claims.

Claims

1. A method for detecting biological sample images, the method comprising: Obtaining each image to be detected, where each image to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope; Using the image detection models corresponding to each of the focal lengths that have been pre-trained to perform image detection on the images to be detected corresponding to the focal lengths respectively, to obtain the initial image detection results of each of the images to be detected; Screening each of the initial image detection results to obtain the image detection result corresponding to the biological sample to be detected, where the training process of the image detection models corresponding to each of the focal lengths includes: Obtaining the sample sets corresponding to each of the focal lengths, where the sample set includes sample images of the biological sample at each of the focal lengths in the same field of view of the microscope; training the network models to be trained corresponding to each of the focal lengths respectively according to the sample sets corresponding to each of the focal lengths, to obtain the image detection models corresponding to each of the focal lengths, where the training the network models to be trained corresponding to each of the focal lengths respectively according to the sample sets corresponding to each of the focal lengths, to obtain the image detection models corresponding to each of the focal lengths includes: For any sample set and network model to be trained corresponding to a focal length, perform the following processing: Using a cross-stage partial network structure to extract the sample image features of each sample image in the sample set; performing convolution on the sample image features based on at least one of a path aggregation network structure and a spatial pyramid pooling network structure to obtain enhanced sample image features; classifying and localizing each of the sample images according to an object detection function and the enhanced sample image features to obtain an image training result; when it is determined based on the image training result that the training end condition is not satisfied, adjusting the parameters of the path aggregation network structure or the spatial pyramid pooling network structure, and returning to the step of performing convolution on the sample image features based on at least one of the path aggregation network structure and the spatial pyramid pooling network structure until the training end condition is satisfied, where screening each of the initial image detection results to obtain the image detection result corresponding to the biological sample to be detected includes: Obtaining the probability scores corresponding to each of the initial image detection results, where the probability scores are calculated based on the corresponding activation functions when each of the image detection models performs image detection on the images to be detected; determining the initial image detection result with the highest probability score as the image detection result corresponding to the biological sample to be detected.

2. The biological sample image detection method according to claim 1, wherein The obtaining the sample sets corresponding to each of the focal lengths includes: Obtaining the original sample images of the biological sample at each of the focal lengths in the same field of view of the microscope; Performing image preprocessing on the original sample images to obtain the sample images of the biological sample.

3. The biological sample image detection method according to claim 2, wherein, The performing image preprocessing on the original sample images to obtain the sample images of the biological sample includes at least one of the following: The first item: Perform image denoising on the original sample image to obtain each of the sample images of the biological sample; the image denoising includes at least one of spatial domain denoising, transform domain denoising, and collaborative denoising of spatial domain and transform domain of the image; Second item: Perform image enhancement on the original sample image to obtain each enhanced sample image of the biological sample; the sample images include each of the original sample images and each of the enhanced sample images; The performing image enhancement on the original sample image includes at least one of the following: Translate the pixel coordinates of the original sample image by a random pixel unit along the horizontal or vertical direction, the random pixel unit is less than a preset pixel unit, and the random pixel unit includes at least one; Scale the original sample image based on a random multiple, the random multiple is greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one; Rotate the original sample image by a random angle in a random direction, the random angle is less than a preset angle, and the random angle includes at least one; Randomly flip the original sample image horizontally or vertically; Randomly increase or decrease the pixel coordinates of the original sample image by a random pixel value, the random pixel value is less than a preset pixel value, and the random pixel value includes at least one; Process the contrast of the original sample image by a random multiple, the random multiple is greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one; Process the brightness of the original sample image by a random multiple, the random multiple is greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple is less than 1, the second preset multiple is greater than 1, and the random multiple includes at least one.

4. The biological sample image detection method according to claim 1, characterized in that The method for determining whether the training end condition is satisfied based on the image training result includes: Calculate the test confidence between the image training result of each sample image and the image calibration result corresponding to the sample image; Compare the test confidence with a preset threshold, and when the test confidence is greater than the preset threshold, determine that the training end condition is satisfied, otherwise, determine that the training end condition is not satisfied.

5. A biological sample image detection device, characterized in that, The device includes: An image acquisition module, configured to acquire each image to be detected, and each of the images to be detected is an image of a biological sample to be detected at at least two focal lengths in the same field of view of a microscope; An image detection module, configured to respectively perform image detection on the images to be detected corresponding to the focal lengths by using the pre-trained image detection models corresponding to the focal lengths, to obtain the initial image detection results of the images to be detected; A result determination module, configured to screen each of the initial image detection results to obtain the image detection result corresponding to the biological sample to be detected. Wherein, the biological sample image detection device further includes an image detection model training module, and the image detection model training module includes the following units: A sample set acquisition unit, configured to acquire a sample set corresponding to each of the focal lengths, where the sample set includes sample images of the biological sample at each of the focal lengths in the same field of view of the microscope; an image detection model training unit, configured to train the network model to be trained corresponding to each of the focal lengths respectively according to the sample sets corresponding to each of the focal lengths, to obtain an image detection model corresponding to each of the focal lengths. Wherein, the image detection model training unit includes the following units: A feature extraction unit, configured to extract sample image features of each sample image in the sample set by using a cross-stage local network structure; a feature enhancement unit, configured to perform convolution on the sample image features based on at least one of a path aggregation network structure and a spatial pyramid pooling network structure, or a path aggregation network structure and a spatial pyramid pooling network structure adjusted by a parameter adjustment unit, to obtain enhanced sample image feature training result acquisition units, configured to classify and locate each of the sample images according to a target detection function and the enhanced sample image features, to obtain an image training result; a training end determination unit, configured to determine whether a training end condition is satisfied based on the image training result; a parameter adjustment unit, configured to adjust the parameters of the path aggregation network structure or the spatial pyramid pooling network structure when the training end determination unit determines that the training end condition is not satisfied based on the image training result. Wherein, the result determination module includes the following units: A probability score acquisition unit, configured to acquire a probability score corresponding to each of the initial image detection results, where the probability score is calculated based on a corresponding activation function when each of the image detection models performs image detection on the image to be detected; a result determination unit, configured to determine the initial image detection result with the highest probability score as the image detection result corresponding to the biological sample to be detected.

6. The biological sample image detection device according to claim 5, characterized in that, The sample set acquisition unit includes the following units: An original sample image acquisition unit, configured to acquire the original sample images of the biological sample at each of the focal lengths in the same field of view of the microscope; An image preprocessing unit, configured to perform image preprocessing on the original sample images to obtain the sample images of the biological sample.

7. The biological sample image detection device according to claim 6, wherein The image preprocessing unit includes the following units: An image denoising processing unit, configured to perform image denoising processing on the original sample images to obtain the sample images of the biological sample. The image denoising processing includes at least one of spatial domain denoising processing, transform domain denoising processing, and collaborative denoising processing of the spatial domain and the transform domain; An image enhancement processing unit for performing image enhancement processing on the original sample image to obtain enhanced sample images of the biological sample; the sample images include the original sample images and the enhanced sample images; the performing image enhancement processing on the original sample image includes at least one of the following: translating the pixel coordinates of the original sample image by a random pixel unit along the horizontal or vertical direction, the random pixel unit being less than a preset pixel unit and the random pixel unit including at least one; scaling the original sample image based on a random multiple, the random multiple being greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple being less than 1, the second preset multiple being greater than 1, and the random multiple including at least one; rotating the original sample image by a random angle in a random direction, the random angle being less than a preset angle and the random angle including at least one; randomly flipping the original sample image horizontally or vertically; randomly increasing or decreasing the pixel coordinates of the original sample image by a random pixel value, the random pixel value being less than a preset pixel value and the random pixel value including at least one; processing the contrast of the original sample image by a random multiple, the random multiple being greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple being less than 1, the second preset multiple being greater than 1, and the random multiple including at least one; processing the brightness of the original sample image by a random multiple, the random multiple being greater than or equal to a first preset multiple and less than or equal to a second preset multiple, the first preset multiple being less than 1, the second preset multiple being greater than 1, and the random multiple including at least one.

8. The biological sample image detection device according to claim 5, wherein The training end determination unit includes the following units: A test confidence calculation unit for calculating the test confidence between the image training result of each sample image and the corresponding image calibration result of the sample image; A test confidence comparison unit for comparing the test confidence with a preset threshold, and when the test confidence is greater than the preset threshold, determining that the training end condition is satisfied, otherwise, determining that the training end condition is not satisfied.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the biological sample image detection method according to any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the biological sample image detection method according to any one of claims 1 to 4.

Images