Root hair segmentation method, system and device based on deep learning

Abstract

The application discloses a root hair segmentation method, system and device based on deep learning, and the method comprises the steps of: recognizing root hair stem information in an image; performing image preprocessing on a plurality of independent sub-regions patch; inputting the patch into a CAE module to learn the appearance characteristics of the root hair; inputting the image processed by the CAE module into a root hair instance segmentation model to detect and segment the root hair; predicting the depth of the root hair by using a preset depth estimation module, generating an accurate depth map by analyzing the depth information in the image, so that the overlapping root hairs before and after the image segmentation can be accurately distinguished and processed; retaining the detection and segmentation results within the set range in the boundary box; and recording the root hair segmentation results and the density of the root hairs in the region. The depth of the overlapping root hairs is extracted by using the depth estimation method, the obtained depth information is added as a feature into an encoder of the MP-Former, the model sufficiently learns the characteristics of the root hairs during segmentation, and the accuracy of the segmentation of the overlapping root hairs is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a root hair segmentation method, system and device based on deep learning. BACKGROUND

[0002] It is essential to obtain the number and density of plant root hairs in a timely and accurate manner to understand the phenotypic characteristics of plants. However, due to the fact that plant roots are usually buried in opaque soil, direct observation and study of their growth state presents many challenges, especially without damaging the roots and their surrounding environment. Traditional evaluation methods such as manual counting, X-ray scanning and nuclear magnetic resonance imaging, while feasible, not only consume a large amount of resources, but also are insufficient in providing detailed root structure information.

[0003] In recent years, with the development of deep learning technology, especially the successful application of instance segmentation technology in multiple fields, new possibilities have been provided to solve the above problems. However, the differences between different application scenarios make it difficult for a single technical solution to be universally applicable. In view of this, the present application aims to combine the knowledge of instance segmentation technology, image enhancement and denoising processing, and develop a new method that can effectively segment root hairs in a variety of complex environments, and will elaborate the design and implementation of the method. SUMMARY

[0004] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a root hair segmentation method, system and device based on deep learning, which improves the accuracy of segmentation through image denoising, image enhancement, depth information, etc., and calculates the root hair density per unit area using the segmentation results.

[0005] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a root hair segmentation method based on deep learning, comprising the following steps:

[0007] Processing the input image to identify the root hair stem information in the image, taking the geometric center of the identified root hair stem as the center point, and dividing the entire image into a plurality of independent sub-regions patch based on the center point;

[0008] Image preprocessing is performed on the plurality of independent sub-regions patch;

[0009] The preprocessed image is input into a root hair instance segmentation model to detect and segment the root hair; the root hair instance segmentation model adopts an improved MP-Former model, and the improved MP-Former model includes a CAE module and a depth estimation module; when detecting and segmenting the root hair, first, the preprocessed patch is input into the preset CAE module to learn the appearance features of the root hair; in the CAE module, the input patch is segmented into a visible patch and a mask patch, the visible patch is processed by an encoder to generate a latent representation, the latent representation of the mask patch is predicted by a regressor, and finally the occluded patch is reconstructed by a decoder; then a Depth-Anything model is applied to extract the depth information of all root hairs to identify the front-back relationship of the root hairs; the improved MP-Former model focuses on the depth information in the current frame to improve the segmentation success rate of the interlaced root hairs; the depth estimation module uses ConvNeXt-T as a shared backbone network to extract multi-scale features of the root hair image, and the root hair image is divided into a single-view branch and a multi-view branch to generate depth information of the root hair image respectively;

[0010] After the improved MP-Former model outputs the segmentation result, a low threshold L and a high threshold H are set, the detection result with a confidence higher than the high threshold H is retained, and for the detection result with a confidence between the high threshold H and the low threshold L, only the detection segmentation result within the set range in the bounding box is retained.

[0011] The root hair segmentation result is recorded, and after the segmentation of the entire region is completed, the number of root hairs of each patch is recorded, and the density of the root hairs in the region is calculated.

[0012] As a preferred technical solution, the input image is processed, the root hair stem information in the image is identified, the geometric center of the identified root hair stem is taken as a center point, and the entire image is divided into a plurality of independent sub-regions patches based on the center point, specifically:

[0013] Each image is input into a preset SAM model to generate image embedding, a prompt encoder is used to input prompt information, and a lightweight mask decoder is used to combine the image embedding and the prompt information to predict the segmentation mask of the image.

[0014] The number of segmentation in the vertical direction of the image, the overlap ratio of the sliding window, the annotation information of the root hair stem, the mask, and the outline of the mask are set, the height of each segmentation region is calculated according to the height of the image and the number of segmentation, the center point of each segmentation region is calculated based on the height of each segmentation region, the size of the window is calculated according to the predefined aspect ratio, the coordinates of the window are generated according to the outline points, and finally the window image is cropped, and the information of each patch is recorded.

[0015] As a preferred technical solution, the several independent sub-regions patch are image pre-processed, specifically:

[0016] The current input patch is converted into a gray image, the edges in the gray image are identified using a Canny edge detection algorithm, an edge intensity map is obtained, and then the average intensity value of the edge intensity map is calculated. The average intensity value is compared with the set threshold value. If it is less than the threshold value, the current patch needs to be image sharpened;

[0017] The gray image is judged, the signal-to-noise ratio is calculated based on the mean and standard deviation of the image, and the signal-to-noise ratio is compared with the set threshold value. If it is less than the threshold value, the current patch is image denoised to obtain the final pre-processed image;

[0018] The pre-processed image is expanded using the "copy and paste" method, and the root hair boundary box region in a certain original image in the data set is copied to another random original image to generate a new sample.

[0019] As a preferred technical solution, the pre-processed patch is first input into a preset CAE module to learn the appearance characteristics of the root hair, specifically:

[0020] The image is divided into several patches, and the size of the patch is set;

[0021] According to the random block occlusion strategy in the context information, the occlusion rate is set to obtain the visible patch mask patch;

[0022] The encoder first receives the input visible image patch, embeds each image patch into a patch embedding through linear projection, and adds a position embedding to generate an initial embedding vector. Then, the initial embedding vector is input into a series of Transformer blocks based on the self-attention mechanism to gradually generate latent representations;

[0023] The regressor uses the representation of the visible patch and the position information of the mask patch to process through a series of Transformer blocks based on cross-attention to generate the latent representation of the mask patch. Through the prediction alignment mechanism, it is ensured that the mask patch representation Z m The representation calculated by the encoder is aligned in the same representation space;

[0024] The decoder accepts the mask patch representation from the regressor and the position information of the mask patch, gradually extracts and refines the root hair features through multiple Transformer blocks, and then predicts the final mask image patch through a linear layer.

[0025] As a preferred technical solution, it further comprises predicting the depth of root hairs by using a preset depth estimation module, specifically:

[0026] In the depth hypothesis sampling stage, a certain number of depth hypotheses are uniformly sampled in the logarithmic space, and then a 4D cost volume is constructed, and the channel number is reduced to 1 through a 3D convolution layer;

[0027] Subsequently, the cost volume is regularized by using a stacked 2D hourglass network, the depth feature map is obtained in combination with the backbone feature, and the depth probability volume and the confidence map are generated through the Softmax operation; the depth estimation results of single view and multi-view are fused through the soft-weighted sum method to calculate the final single view depth;

[0028] Finally, the depth map is refined by using the CRF technology, and the model performance is improved through end-to-end training and parameter adjustment.

[0029] As a preferred technical solution, the depth estimation module adopts a single-target depth estimation model depthanything, generates an accurate depth map by analyzing the depth information in the image, and inputs the depth map as feature information into the MP-Former, so that the overlapping root hairs before and after are accurately distinguished and processed when the image is segmented.

[0030] As a preferred technical solution, after recording the root hair segmentation result and completing the segmentation of the entire region, the number of root hairs of each patch is recorded, and the density of root hairs in the region is calculated, specifically:

[0031] After the root hair segmentation of each patch is completed, the previously saved patch information is read, each patch is spliced back into the original image, the consistency of the same instance in the overlapping area is ensured in combination with the context information in the restoration process, and finally the total number n of root hairs is recorded. The density of root hairs per unit area is calculated based on the area of the original image and the total number of root hairs.

[0032] In a second aspect, the application provides a root hair segmentation system based on deep learning, which is applied to the root hair segmentation method based on deep learning, and comprises an image division module, an image preprocessing module, a root hair detection and segmentation module, a detection result screening module, and a root hair density calculation module.

[0033] The image division module is used for processing the input image and identifying the root hair stem information in the image, taking the geometric center of the identified root hair stem as a center point, and dividing the whole image into a plurality of independent sub-regions patch based on the center point.

[0034] The image preprocessing module is used for image preprocessing of the plurality of independent sub-regions patch.

[0035] The root hair detection and segmentation module is configured to input the preprocessed image into a root hair instance segmentation model to detect and segment the root hair; the root hair instance segmentation model adopts an improved MP-Former model, the improved MP-Former model comprises a CAE module and a depth estimation module; when detecting and segmenting the root hair, first, the preprocessed patch is input into a preset CAE module to learn the appearance features of the root hair; in the CAE module, the input patch is segmented into a visible patch and a mask patch, the encoder is used to process the visible patch to generate a latent representation, the regressor is used to predict the latent representation of the mask patch, and finally the decoder is used to reconstruct the occluded patch; then, a Depth-Anything model is applied to extract the depth information of all root hairs to identify the front-back relationship of the root hairs; the improved MP-Former model focuses on the depth information in the current frame to improve the segmentation success rate of the interlaced root hairs; the depth estimation module uses a ConvNeXt-T as a shared backbone network to extract the multi-scale features of the root hair image, and the root hair image is divided into a single-view branch and a multi-view branch to generate the depth information of the root hair image respectively;

[0036] The detection result selection module is configured to, after the improved MP-Former model outputs the segmentation result, set a low threshold L and a high threshold H, retain the detection result with a confidence higher than the high threshold H, and for the detection result with a confidence between the high threshold H and the low threshold L, only retain the detection segmentation result within a set range in the bounding box.

[0037] The root hair density calculation module is configured to record the root hair segmentation result, record the number of root hairs of each patch after completing the segmentation of the entire region, and calculate the density of the root hairs in the region.

[0038] In a third aspect, the present application provides an electronic device, which comprises:

[0039] at least one processor; and

[0040] a memory connected in communication with the at least one processor; wherein

[0041] The memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the deep learning-based root hair segmentation method.

[0042] In a fourth aspect, the present application provides a computer readable storage medium storing a program, and the program is executed by a processor to implement the deep learning-based root hair segmentation method.

[0043] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0044] 1、The image pre-processing module is adopted, the current patch is judged through the image diagnosis module, the current patch needing picture sharpening is subjected to sharpening operation, the current patch needing noise removal is subjected to noise removal, and the image is transmitted into the next module after enhancement. The diversified processing target makes the system more flexible, the adaptation range is wider, and the image processing demand in different scenes can be better met.

[0045] 2、The present application utilizes the CAE module to pre-train the unlabeled image, the CAE module predicts the representation of the hidden part, and ensures that the representation is in the same encoding representation space, so that the feature representation is more rich and efficient. This is particularly important for processing the segmentation task of root hair, because the model needs to capture the details and texture features of root hair.

[0046] 3、The present application adds depth information to the MP-Former, the depth information can provide more three-dimensional structure details, so that the segmentation result is more accurate and fine. Secondly, by combining the depth information, the model can better distinguish root hairs of different depth levels, thereby improving the overall accuracy of segmentation. In addition, depth information can also reduce errors caused by changes in illumination and background noise, thereby enhancing the robustness and generalization ability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0048] Figure 1 is a flow chart of the root hair instance segmentation method based on deep learning of the embodiment of the present application;

[0049] Figure 2 is an algorithm flow chart of the image pre-processing module of the embodiment of the present application;

[0050] Figure 3 is an unsegmented root hair image of the embodiment of the present application;

[0051] Figure 4 is a segmented root hair image after SAM model processing of the embodiment of the present application;

[0052] Figure 5 is an image segment obtained by sliding window processing of the embodiment of the present application;

[0053] Figure 6 is an improved MP-Former model diagram of an embodiment of the present application;

[0054] Figure 7 is a result diagram of root hair instance segmentation of an embodiment of the present application;

[0055] Figure 8 is a block diagram of a deep learning-based root hair instance segmentation system of an embodiment of the present application;

[0056] Figure 9 is a structural diagram of an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to enable persons skilled in the art to better understand the schemes of the present application, the technical schemes in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor are within the scope of protection of the present application.

[0058] Please refer to Figure 1 In an embodiment of the present application, a deep learning-based root hair instance segmentation method is provided, comprising the following steps:

[0059] (1), please refer to Figure 3 , Figure 4 , Figure 5 , the input image is processed by a preset SAM model, the main stem information in the image is automatically recognized, the geometric center of the recognized main stem is taken as a center point, and the whole image is divided into a plurality of independent sub-regions (patches) based on the center point.

[0060] It can be understood that this division method can effectively enhance the extraction ability of local features of the image, and ensure accurate capture of details in complex scenes; further, step (1) is specifically;

[0061] (1-1), input each image into a Segment Anything Model (SAM), and use the powerful image encoder of the model to generate image embedding. Then, use the prompt encoder to process the input prompt (such as foreground points, background points, frames, etc.), and combine the image embedding and the prompt information through a lightweight mask decoder to predict the segmentation mask of the image. Specifically, for each image, the model will segment the main stem part and calculate the center point position of the main stem, denoted as point A, and then write the information of the main stem into a json file.

[0062] (1-2), set the number of image segmentation in the vertical direction as num_split = 8, the overlap ratio of the sliding window as 0.5, read the annotation information of the backbone to generate a mask, and find the contour of the mask. The height of each segmentation region is calculated according to the height of the image height and the number of segmentation, and the specific formula is as follows:

[0063]

[0064] Then the center point of each segmentation region is calculated, and the specific formula is as follows:

[0065]

[0066] According to the predefined aspect ratio, the size of the window is calculated, and the coordinates of the window are generated according to the contour points, and finally the window image is cropped, and the information of each patch is recorded.

[0067] (2), please refer to Figure 2 Each patch is input into the image preprocessing module, and the image preprocessing module includes an image diagnosis module, a sharpening enhancement module, a noise suppression module and a data expansion module, specifically:

[0068] (2-1), the image diagnosis module first calculates the average value of the current input patch f k Convert it into a gray image, use Canny edge detection algorithm to identify the edge of the image, get the edge intensity map, then calculate the average intensity value of the edge intensity map, and compare it with the set threshold value, here the threshold value is set to 50; if it is less than the threshold value, the current patch f k Image sharpening needs to be performed.

[0069] (2-2), the image diagnosis module judges the gray image, first calculates the mean μ and standard deviation σ of the image, then calculates the signal-to-noise ratio, and the specific formula is as follows:

[0070]

[0071] Compare it with the set threshold value, here the threshold value is set to 15; if it is less than the threshold value, the current patch f k Image denoising needs to be performed to get the final preprocessed image a.

[0072] (2-3), use the "copy and paste" method to expand the image a, copy the boundary box region where the root hair is located in a certain original image in the data set to another random original image to generate a new sample.

[0073] (3), please refer to Figure 6The preprocessed image is input into the root hair instance segmentation model to detect and segment the root hair. The root hair instance segmentation model uses an improved MP-Former model, which includes a CAE module and a depth estimation module. When detecting and segmenting the root hair, the preprocessed patch is first input into the preset CAE module to learn the appearance features of the root hair. The image processed by the CAE module is input into the root hair instance segmentation model to detect and segment the root hair. In the CAE module, the input patch is divided into visible patches and mask patches. The encoder is used to process the visible patches to generate latent representations. The regressor is used to predict the latent representations of the mask patches. Finally, the decoder is used to reconstruct the occluded patches. The root hair instance segmentation model uses an improved MP-Former. To address the problem of low recognition rate caused by root hair interlacing and the problem of insufficient utilization of a large number of unlabeled images, the embodiment uses a convolutional autoencoder (CAE) model to further extract root hair features by pre-training on unlabeled images. In addition, the Depth-Anything model is applied to extract the depth information of all root hairs to accurately identify the front-back relationship of the root hairs. Through this method, we can capture the spatial relationship between root hairs more carefully, improve the model's ability to analyze complex root hair structures, and thus improve the overall recognition accuracy and robustness. The improved MP-Former focuses on the depth information in the current frame to improve the success rate of segmenting interlaced root hairs.

[0074] The depth processing module is configured to predict the depth of the root hair using a preset depth estimation module, extract features of four scales using ConvNeXt-T as a shared backbone network, and divide the features into single-view and multi-view branches to generate initial depth estimates. In the depth hypothesis sampling stage, a certain number of depth hypotheses (e.g., 128) are uniformly sampled in logarithmic space, and then a 4D cost volume is constructed. The number of channels is reduced to 1 through a 3D convolution layer. Subsequently, a stacked 2D hourglass network is used to regularize the cost volume, obtain depth feature maps in combination with backbone features, and generate a depth probability volume and a confidence map through a Softmax operation. The depth estimation results of the single-view and multi-view branches are fused through a soft-weighted sum method to calculate the final single-view depth, thereby alleviating errors caused by pose noise. Finally, techniques such as CRF are used to refine the depth map, and the model performance is improved through end-to-end training and parameter adjustment. The depth estimation module analyzes the depth information in the image to generate an accurate depth map, which enables accurate differentiation and processing of overlapping root hairs during image segmentation.

[0075] It can be understood that the embodiment utilizes the improved MP-Former instance segmentation model to detect and segment root hairs, because root hairs are fine and interlaced, and the improved MP-Former instance segmentation model is more likely to focus on the depth information in the current frame, thereby improving the success rate of segmentation of interlaced root hairs. Specifically:

[0076] (3-1), the image resolution is 2000x1000. In order to perform effective image processing and analysis, we divide the image into a number of small blocks (patches), each patch has a size of p x p, where p is set to 100 pixels. Therefore, each patch has a size of 100x100 pixels. This means that the entire image will be segmented into 20 rows and 10 columns of patches, for a total of 200 patches.

[0077] (3-2), according to the random block occlusion strategy in the context information, the occlusion rate a = 60% is set, the number of visible patches X v is 120, and the number of masked patches X m is 80.

[0078] (3-3), the encoder first receives the input visible image patch X v , embeds each image patch into a patch embedding through linear projection, adds a position embedding P v , and generates an initial embedding vector u v . Then, the initial embedding vector u v is input into a series of Transformer blocks based on self-attention mechanism, and the latent representation Z v is gradually generated.

[0079] (3-4), the regressor utilizes the representation Z v of the visible patch and the position information of the masked patch, and processes through a series of Transformer blocks based on cross-attention, to generate the latent representation Z m of the masked patch. Through the prediction alignment mechanism, it is ensured that the masked patch representation Z m output by the regressor is aligned with the representation calculated directly by the encoder in the same representation space.

[0080] (3-5), the decoder accepts the masked patch representation Z m from the regressor and the position information of the masked patch, and gradually extracts and refines the root hair features through multiple Transformer blocks, and then predicts the final masked image patch Y m through a linear layer.

[0081] (4) The depth of the root hair is predicted by using a depth estimation module. In this embodiment, the most advanced single-target depth estimation model depth anything is used. By analyzing the depth information in the image, an accurate depth map is generated as feature information input into the MP-Former, so that when image segmentation is performed, the front and rear overlapping root hairs can be accurately distinguished and processed. The segmentation error caused by the intersection of root hairs is reduced, and the reliability and accuracy of the overall segmentation effect are improved.

[0082] (5) Please refer to Figure 7 , a low threshold L and a high threshold H are set, the detection results with a confidence higher than the high threshold H are retained, and for the detection results with a confidence between the high threshold H and the low threshold L, only the detection segmentation results with 100 or more pixels existing in the bounding box are retained.

[0083] (6) After the root hair segmentation of each patch is completed, the previously saved patch information is read, and each patch is spliced back into the original image. In the process of restoration, the context information is combined to ensure the consistency of the same instance in the overlapping area, and the density of the root hair per unit area is calculated, laying a foundation for subsequent plant phenotype research.

[0084] It should be noted that, for the foregoing method embodiments, in order to facilitate description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the order of the described actions, because according to the present application, certain steps can be performed in other order or simultaneously.

[0085] Based on the same idea as the root hair segmentation method based on deep learning in the above embodiment, the present application also provides a root hair segmentation system based on deep learning, which can be used to execute the root hair segmentation method based on deep learning described above. For the convenience of description, in the structural schematic diagram of the root hair segmentation system based on deep learning embodiment, only the part related to the embodiment of the present application is shown, and those skilled in the art can understand that the structure shown in the diagram does not constitute a limitation on the device, and more or fewer components than those shown in the diagram can be included, or some components can be combined, or different component arrangements can be included.

[0086] Please refer to Figure 8 In another embodiment of the present application, a root hair segmentation system based on deep learning 100 is provided, which includes an image division module 101, an image preprocessing module 102, a root hair detection and segmentation module 103, a detection result screening module 104, and a root hair density calculation module 105.

[0087] The image division module 101 is configured to process an input image, identify root hair stem information in the image, take a geometric center of the identified root hair stem as a center point, and divide the entire image into a plurality of independent sub-regions patch based on the center point.

[0088] The image preprocessing module 102 is configured to perform image preprocessing on the plurality of independent sub-regions patch.

[0089] The root hair detection and segmentation module 103 is configured to input the preprocessed patch into a preset CAE module to learn the appearance features of the root hair, and input the image processed by the CAE module into a root hair instance segmentation model to detect and segment the root hair. The CAE module divides the input patch into a visible patch and a mask patch, uses an encoder to process the visible patch to generate a latent representation, uses a regressor to predict the latent representation of the mask patch, and finally uses a decoder to reconstruct the occluded patch. The root hair instance segmentation model uses an improved MP-Former. The Depth-Anything model is used to extract the depth information of all root hairs to accurately identify the front-back relationship of the root hairs. Through this method, we can more carefully capture the spatial relationship between the root hairs, improve the model's ability to analyze complex root hair structures, and thus improve the overall recognition accuracy and robustness. The improved MP-Former focuses on the depth information in the current frame to improve the success rate of segmenting interlaced root hairs.

[0090] The detection result selection module 104 is configured to set a low threshold L and a high threshold H after the modified MP-Former model outputs the segmentation result, retain the detection results with a confidence higher than the high threshold H, and only retain the detection and segmentation results within a certain range in the bounding box for the detection results with a confidence between the high threshold H and the low threshold L.

[0091] The root hair density calculation module 106 is configured to record the root hair segmentation result and calculate the density of the root hair in the region after completing the segmentation of the entire region.

[0092] It should be noted that the deep learning-based root hair segmentation system of the present application corresponds one-to-one to the deep learning-based root hair segmentation method of the present application. The technical features and advantages described in the above embodiment of the deep learning-based root hair segmentation method are applicable to the embodiment of the deep learning-based root hair segmentation.

[0093] In addition, in the implementation of the root hair segmentation system based on deep learning in the above embodiments, the logical division of each program module is only illustrative. In actual applications, the above functions can be completed by different program modules according to needs, for example, for the configuration requirements of corresponding hardware or the convenience of software implementation. That is, the internal structure of the root hair segmentation system based on deep learning is divided into different program modules to complete all or part of the above-described functions.

[0094] Please refer to Figure 9 In one embodiment, an electronic device implementing a root hair segmentation method based on deep learning is provided. The electronic device 200 can include a first processor 201, a first memory 202, and a bus. It can also include a computer program, such as a root hair segmentation program based on deep learning 203, stored in the first memory 202 and executable on the first processor 201.

[0095] The first memory 202 includes at least one type of readable storage medium, including flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the first memory 202 can include both an internal storage unit and an external storage device of the electronic device 200. The first memory 202 can be used not only to store application software and various data installed on the electronic device 200, such as the code of the root hair segmentation program based on deep learning 203, but also to temporarily store data that has been or will be output.

[0096] The first processor 201 may, in some embodiments, be composed of integrated circuits, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple packaged integrated circuits of the same function or different functions, including one or more combinations of central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 201 is the control core of the electronic device, connects various components of the entire electronic device through various interfaces and lines, executes programs or modules stored in the first memory 202, and calls data stored in the first memory 202 to perform various functions and process data of the electronic device 200.

[0097] Figure 9 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 9 The structure shown does not constitute a limitation on the electronic device 200, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0098] The deep learning-based root hair segmentation program 203 stored in the first memory 202 in the electronic device 200 is a combination of multiple instructions, which, when running in the first processor 201, can achieve:

[0099] Processing the input image to identify the root hair stem information in the image, taking the geometric center of the identified root hair stem as a center point, and dividing the entire image into a plurality of independent sub-regions patches based on the center point;

[0100] Image preprocessing is performed on the plurality of independent sub-regions patches;

[0101] The preprocessed image is input into the root hair instance segmentation model to detect and segment the root hair; the root hair instance segmentation model adopts an improved MP-Former model, and the improved MP-Former model includes a CAE module and a depth estimation module; when detecting and segmenting the root hair, first, the preprocessed patch is input into the preset CAE module to learn the appearance features of the root hair; in the CAE module, the input patch is segmented into a visible patch and a mask patch, the visible patch is processed by an encoder to generate a latent representation, the latent representation of the mask patch is predicted by a regressor, and finally the occluded patch is reconstructed by a decoder; then, a Depth-Anything model is applied to extract the depth information of all root hairs to identify the front-back relationship of the root hairs; the improved MP-Former model focuses on the depth information in the current frame to improve the segmentation success rate of the interlaced root hairs; the depth estimation module uses ConvNeXt-T as a shared backbone network to extract multi-scale features of the root hair image, and the root hair image is divided into a single-view branch and a multi-view branch to generate depth information of the root hair image;

[0102] After the improved MP-Former model outputs the segmentation result, a low threshold L and a high threshold H are set, the detection result with a confidence higher than the high threshold H is retained, and for the detection result with a confidence between the high threshold H and the low threshold L, only the detection segmentation result within a set range in the bounding box is retained.

[0103] The root hair segmentation result is recorded, and after the segmentation of the entire region is completed, the number of root hairs of each patch is recorded, and the density of the root hairs in the region is calculated.

[0104] Further, the modules / units of the electronic device 200 are implemented in the form of software function units and sold or used as independent products, which can be stored in a nonvolatile computer readable storage medium. The computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM).

[0105] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0106] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0107] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods and shall be within the scope of protection of the present application.

Claims

1. A deep learning-based root-hair segmentation method, characterized in that, Includes the following steps: The input image is processed to identify the root hair trunk information in the image. The geometric center of the identified root hair trunk is used as the center point, and the entire image is divided into several independent sub-region patches based on the center point. Perform image preprocessing on several independent sub-region patches; The preprocessed image is input into a root hair instance segmentation model for root hair detection and segmentation. The root hair instance segmentation model employs an improved MP-Former model, which includes a CAE module and a depth estimation module. During root hair detection and segmentation, the preprocessed patch is first input into a pre-defined CAE module to learn the appearance features of the root hair. In the CAE module, the input patch is segmented into visible patches and mask patches. An encoder processes the visible patches to generate latent representations, a regressor predicts the latent representations of the mask patches, and finally, a decoder reconstructs the occluded patches. Then, a Depth-Anything model is applied to extract the depth information of all root hairs to identify their relative positions. The improved MP-Former model focuses on the depth information in the current frame to improve the segmentation success rate of interleaved root hairs. The depth estimation module uses ConvNeXt-T as a shared backbone network to extract multi-scale features from the root hair image. The root hair image is divided into single-view and multi-view branches, which generate depth information for the root hair image respectively. After the improved MP-Former model outputs segmentation results, a low threshold L and a high threshold H are set. Detection results with a confidence level higher than the high threshold H are retained. For detection results with a confidence level between the high threshold H and the low threshold L, only the detection segmentation results within the set range within the bounding box are retained. Record the root hair segmentation results. After completing the segmentation of the entire region, record the number of root hairs in each patch and calculate the root hair density in that region.

2. The deep learning-based hair segmentation method according to claim 1, characterized in that, The process of processing the input image involves identifying the root hair trunk information within the image. Using the geometric center of the identified root hair trunk as the center point, the entire image is divided into several independent sub-region patches based on this center point. Specifically: Each image is input into a pre-defined SAM model to generate an image embedding. A cue encoder is used to input cue information, and a lightweight mask decoder combines the image embedding and cue information to predict the segmentation mask of the image. Set the number of segments in the vertical direction of the image, the overlap ratio of the sliding window, read the annotation information of the root hair trunk to generate a mask, find the outline of the mask, calculate the height of each segmented region based on the image height and the number of segments, calculate the center point of each segmented region based on the height of each segmented region, calculate the size of the window according to the predefined aspect ratio, generate the coordinates of the window based on the outline points, finally crop the window image, and record the information of each patch.

3. The deep learning-based hair segmentation method according to claim 1, characterized in that, The image preprocessing of several independent sub-region patches specifically includes: The current input patch is converted into a grayscale image. The Canny edge detection algorithm is used to identify the edges in the grayscale image to obtain an edge intensity map. Then, the average intensity value of the edge intensity map is calculated and compared with a set threshold. If it is less than the threshold, the current patch needs to be image sharpened. The grayscale image is evaluated, and the signal-to-noise ratio (SNR) is calculated based on the image's mean and standard deviation. The SNR is then compared with a set threshold. If the SNR is less than the threshold, the current patch is denoised to obtain the final preprocessed image. The "copy and paste" method is used to augment the preprocessed image by copying the bounding box region of the root hair in one of the original images in the dataset to another random original image to generate a new sample.

4. The deep learning-based hair segmentation method according to claim 1, characterized in that, The process begins by inputting the preprocessed patch into a preset CAE module to learn the appearance features of the root hairs. Specifically: Divide the image into several patches and set the size of each patch; Based on the random block occlusion strategy in the context information, set the occlusion rate to obtain the visible patch mask patch; The encoder first receives the visible patches of the input, embeds each image patch into a patch embedding through linear projection, and adds position embeddings to generate an initial embedding vector; Then, the initial embedding vector is input into a series of Transformer blocks based on the self-attention mechanism to gradually generate latent representations; The regressor utilizes the representation of the visible patch and the positional information of the mask patch, processing them through a series of cross-attention-based Transformer blocks to generate a latent representation of the mask patch. A prediction alignment mechanism ensures that the mask patch representation output by the regressor is accurate. Z m The representations directly computed by the encoder are aligned in the same representation space; The decoder receives the mask patch representation and the location information of the mask patch from the regressor, extracts and refines the root hair features step by step through multiple Transformer blocks, and then predicts the final mask image patch through a linear layer.

5. The deep learning-based hair segmentation method according to claim 1, characterized in that, It also includes using a preset depth estimation module to predict the depth of root hairs, specifically: In the depth hypothesis sampling stage, a set number of depth hypotheses are uniformly sampled in the logarithmic space, and then a 4D cost volume is constructed. The number of channels is reduced to 1 through a 3D convolutional layer. Subsequently, a stacked 2D hourglass network is used to regularize the cost volume, and a depth feature map is obtained by combining the backbone features. The depth probability volume and confidence map are generated by the Softmax operation. The depth estimation results of single view and multi-view are fused by the soft-weighted sum method to calculate the final single view depth. Finally, CRF technology was used to refine the depth map, and model performance was improved through end-to-end training and parameter tuning.

6. The deep learning-based hair segmentation method according to claim 1 or 5, characterized in that, The depth estimation module uses a single-target depth estimation model called "depth anything". By analyzing the depth information in the image, it generates an accurate depth map, which is then input into MP-Former as feature information. This enables the accurate differentiation and processing of overlapping hairs during image segmentation.

7. The deep learning-based hair segmentation method according to claim 1, characterized in that, The process of recording root hair segmentation results involves recording the number of root hairs in each patch after segmenting the entire region, and calculating the root hair density in that region. Specifically: After the root hair segmentation of each patch is completed, the previously saved patch information is read and each patch is pieced back into the original image. During the restoration process, the context information is combined to ensure the consistency of the same instance in the overlapping area. Finally, the total number of root hairs n is recorded, and the density of root hairs per unit area is calculated based on the area of ​​the original image and the total number of root hairs.

8. A deep learning-based hair segmentation system, characterized in that, The deep learning-based hair segmentation method applied to any one of claims 1-7 includes an image segmentation module, an image preprocessing module, a hair detection and segmentation module, a detection result selection and filtering module, and a hair density calculation module. The image segmentation module is used to process the input image, identify the root hair trunk information in the image, and use the geometric center of the identified root hair trunk as the center point to divide the entire image into several independent sub-region patches based on the center point. The image preprocessing module is used to preprocess several independent sub-region patches. The root hair detection and segmentation module is used to input the preprocessed image into the root hair instance segmentation model for root hair detection and segmentation. The root hair instance segmentation model adopts an improved MP-Former model, which includes a CAE module and a depth estimation module. When detecting and segmenting root hairs, the preprocessed patch is first input into the preset CAE module to learn the appearance features of the root hair. In the CAE module, the input patch is segmented into visible patches and mask patches. The encoder processes the visible patches to generate latent representations, the regressor predicts the latent representations of the mask patches, and finally the decoder reconstructs the occluded patches. Then, the Depth-Anything model is applied to extract the depth information of all root hairs to identify the front-to-back relationship of the root hairs. The improved MP-Former model focuses on the depth information in the current frame to improve the segmentation success rate of interlaced root hairs. The depth estimation module uses ConvNeXt-T as a shared backbone network to extract multi-scale features of the root hair image. The root hair image is divided into single-view and multi-view branches, which generate depth information of the root hair image respectively. The detection result selection and filtering module is used to set a low threshold L and a high threshold H after the improved MP-Former model outputs the segmentation results. It retains detection results with a confidence level higher than the high threshold H, and for detection results with a confidence level between the high threshold H and the low threshold L, it only retains the detection segmentation results that exist within the set range in the bounding box. The root hair density calculation module is used to record the root hair segmentation results. After completing the segmentation of the entire region, it records the number of root hairs in each patch and calculates the root hair density in that region.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor to enable the at least one processor to perform the deep learning-based root hair segmentation method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the deep learning-based hair segmentation method according to any one of claims 1-7.

Images