Organ-like cell microcarrier detection method and organ-like cell microcarrier model building method

Abstract

The invention provides an organoid cell microcarrier detection method and a model building method thereof, and belongs to the field of organoid cell detection based on computer vision. The method comprises the following steps: deploying a high-precision camera in a micro-fluidic chip sorting area, collecting an organoid cell microcarrier image, and carrying out a series of preprocessing to obtain a standard data set; and constructing a cell microcarrier detection model comprising a multi-layer feature extraction module, a feature fusion module and a target detection module. After model training, channel pruning and lightweight operation are further adopted to optimize the model so as to adapt to embedded equipment with a limited memory. According to the method, automatic identification of the cell microcarrier in the chip is realized, and the detection efficiency and the intelligent level are remarkably improved; meanwhile, a multi-level feature extraction and fusion module is utilized, so that the model can still keep high accuracy and stability under the condition of coping with dense or sparse cell distribution; in addition, the lightweight design ensures that the model can be smoothly deployed on microfluidic equipment with limited computing power.

Description

Technical Field

[0001] The present invention belongs to the technical field of organoid cell detection based on computer vision, and in particular relates to an organoid cell microcarrier detection method and a model building method thereof. Background Art

[0002] Cell microcarrier detection technology is the key to improving the accuracy and efficiency of cell detection, and it is also the cornerstone to meet the urgent needs of modern biomedicine, genetic engineering and other fields. This technology achieves high-precision detection through micron-level control, and combined with automated operation, it can significantly reduce human errors and improve detection efficiency. In biomedical research, cell microcarrier detection technology is widely used in cell culture, drug screening and disease diagnosis. At the same time, in the field of genetic engineering and gene therapy, this technology ensures the quality and safety of microcarriers and escorts the safe implementation of gene therapy. In addition, the development of cell microcarrier detection technology has also promoted the innovation of related technologies and the upgrading of the industrial chain. It not only integrates cutting-edge technologies such as micro-nano processing, image processing, and machine learning, but also promotes the development of the entire industrial chain from microcarrier production to detection. Therefore, with its advantages of intelligent micro, simple operation and low cost, cell microcarrier detection technology provides new solutions for disease diagnosis and treatment, and is expected to improve the treatment status of many diseases. Especially in individualized diagnosis and treatment, this technology provides a powerful tool for single-cell analysis and improves the treatment effect.

[0003] The existing cell microcarrier detection methods mainly include the following two methods: microscopic observation method, fluorescent staining method and automated detection method.

[0004] Microscope observation method: Microscope observation is one of the most direct and commonly used methods for detecting cell microcarriers. By taking a uniformly suspended culture sample, dropping it on a glass slide, and observing the cell growth under a microscope. This method can intuitively see the morphology, distribution and growth status of cells on microcarriers. At the same time, by counting the number of cells on the microcarriers, the cell growth can be quantitatively evaluated. However, this method is relatively time-consuming and depends on the experience and skills of the operator.

[0005] Fluorescent staining: Fluorescence staining is a method of staining cell nuclei with fluorescent dyes and then observing and counting cells under a fluorescent microscope. For example, the NucleoCounting method uses fluorescent staining of cell nuclei and labels cells with the fluorescent dye DAPI, which binds to DNA with high specificity and can accurately detect cell nuclei even in the presence of cell debris. This method can significantly improve the accuracy and efficiency of cell counting, and is particularly suitable for large-scale cell culture and high-throughput screening.

[0006] Automated detection methods: The image acquisition device is used to acquire images of the distribution area of ​​the cell microcarriers, and the acquired images are sent to the image processing algorithm to detect the distribution of the cell microcarriers.

[0007] However, microscopic observation and fluorescent staining methods require a lot of manpower and time, and detection is highly dependent on subjective judgment, with a relatively low degree of automation. Some existing automatic detection methods use detection models that are too large, and consume too much power and computing resources in actual use, limiting the actual deployment scenarios. Summary of the invention

[0008] In view of the above problems, the first aspect of the present invention provides a method for constructing an organoid cell microcarrier detection model, comprising the following steps: S1, collect images of organoid cell microcarriers to build the initial dataset; S2, preprocessing the initial data set images to construct a standard data set; the preprocessing includes image filtering, image normalization, image standardization and data calibration; S3, constructing a cell microcarrier detection model, including a multi-layer feature extraction module, a feature fusion module and a target detection module; the multi-layer feature extraction module is a cell microcarrier feature extraction module based on a separable convolution operation, which performs multi-level feature extraction on the image to include the global feature information and local feature information of the cell microcarrier image; the feature fusion module effectively fuses the extracted multi-level features to fully consider the global feature information and local feature information of the cell microcarrier image; the target detection module detects the output features of the feature fusion module and outputs the detection results; S4, train the cell microcarrier detection model based on the standard dataset, and optimize the model based on the designed channel pruning and quantization operations to improve the detection accuracy of the model in memory-constrained embedded devices and obtain the final model.

[0009] Preferably, the specific process of pre-processing in S2 includes: S21, image filtering: Gaussian filter is used to filter the collected cell microcarrier images. Perform Gaussian filtering to remove the noise in the image and obtain the denoised image ;

[0010] in, Represents Gaussian filter processing function; S22, image normalization: Use the maximum and minimum normalization method to normalize the denoised image Perform normalization operation, the normalization operation is as follows:

[0011] Normalize all pixel values ​​in the image to To get the normalized image ;in, Representing images Middle Line The pixel value of the pixel point in the column, Represents the image after normalization Middle Line The pixel value of the pixel point in the column, Respectively represent images The minimum pixel value and the maximum pixel value among all pixels; S23, image normalization: normalize the image The standard image dataset has been obtained by standardization operation , the standardized formula is as follows:

[0012] in, Represents the image after normalization operation Middle Line The pixel value of the pixel point in the column, Respectively represent the image The pixel average value of all pixels and the standard deviation calculated for all pixels; S24, data calibration: based on the obtained standard image data , use the image annotation software LabelImg to annotate the image The cell microcarriers in the image are marked with rectangular frames to obtain the position information of the cell microcarriers in the image, including the specific x, y coordinates and the length s and width h of the cell microcarriers.

[0013] Preferably, the preprocessing in S2 also includes enhancing and expanding the image. The specific process includes: Image geometric transformation processing: standard image dataset The image with a resolution of 800*400 is scaled to 400*200 to obtain a scaled image dataset. After the image scaling operation, the images in the scaled image dataset are randomly flipped horizontally and vertically to obtain the flipped image dataset. ; Image color transformation processing: scaling the dataset The images in the image are randomly adjusted in brightness, contrast and color saturation to obtain the color transformation image dataset ; Image stitching operation: from scaled image dataset , flipped image dataset And the color transformation dataset Four different images are randomly selected from the three datasets for image stitching. The four images with a resolution of 400*200 are re-stitched into an image with a resolution of 800*400, and the re-stitched images are added to the original standard dataset. middle.

[0014] Preferably, the multi-layer feature extraction module includes a channel convolution operation and a point convolution operation; The channel convolution operation first uses three 3*3 convolution kernels to perform convolution operations on the RGB channels of the input data, where each 3*3 convolution kernel can obtain a corresponding output feature, and the three different features obtained are defined in turn as ; The point convolution operation is performed on three different channel features. Perform point convolution again, that is, use the 1*1 convolution kernel to further extract the features of the image and obtain the features ; Then, a fully connected layer with three neurons as input and one neuron as output is used to transform the features As the input of this fully connected layer, the output of the fully connected layer is used as the final feature extracted :

[0015] in, Represents the fully connected layer function.

[0016] Preferably, the feature fusion module is designed as two feature fusion operation modules, one from deep to shallow and the other from shallow to deep, to fully consider the information of features at different levels; The shallow-to-deep feature fusion module is a feature transfer module that transfers features from shallow layers to deep layers step by step. The shallow features are fused with the deeper features after passing through the convolution layer. The shallow features are added with the features of the next layer after passing through the convolution layer as an outward output feature. This output feature is processed by a convolution layer as the input feature of the deep-to-shallow feature fusion module. This fusion operation starts from the shallowest features and ends with the deepest features. The deep-to-shallow feature fusion module performs feature fusion operation on the deep features after passing through the pooling layer and the shallower features. This fusion operation starts from the deepest features and ends with the shallowest features to obtain the output of this multi-scale feature fusion module. .

[0017] Preferably, the target detection module is composed of three 3*3 convolutional layers plus a Relu activation function to form the front part of this module, and two maximum pooling layers, a Dropout layer and a Softmax activation function to form the back part of this target detection module.

[0018] Preferably, the model is optimized based on the channel pruning and quantization operations obtained by design in S4, and the specific process is as follows: S41, using the standard image data set to train the cell microcarrier detection model. After the training is completed, the output feature vectors of q feature extraction modules in the multi-layer feature extraction module are defined as , ;

[0019] in, Represents the output feature vector The first element values, ; S42, calculate each output feature vector in turn The mean of all elements in

[0020]

[0021] S43, the model finally outputs the feature vector The mean calculation of :

[0022] in, ;

[0023] S44, feature correlation value calculation: Calculate the output feature vectors of q feature extraction models in sequence , and the model's final feature output vector The characteristic correlation value of :

[0024] The feature correlation value , represents the output feature vector of each feature extraction module And the final model output The larger the feature correlation value, the higher the correlation between this feature and the model output, that is, the more important this feature is. The feature extraction modules with the highest feature correlation value are retained, and the other feature extraction modules are eliminated. S45, after obtaining the cell microcarrier detection model after channel pruning, further quantize the weight values ​​in the model, and quantize all floating-point data in the model into fixed-point data to further reduce the size of the model.

[0025] The second aspect of the present invention provides an organoid cell microcarrier detection method, which deploys the cell microcarrier detection model constructed by the construction method described in the first aspect in a Raspberry Pi, and includes the following process: S1, based on the high-precision image sensor deployed above the sorting area of ​​the microfluidic chip, collects images of the sorting area of ​​the microfluidic chip in real time; S2, performing image filtering, image normalization and standardization processing on the collected initial image; S3, input the standardized image into the cell-microcarrier detection model; S4, output the detection image results of the cell microcarriers in the sorting area of ​​the microfluidic chip.

[0026] The third aspect of the present invention provides an organoid cell microcarrier detection device, the device comprising at least one processor and at least one memory, the processor and the memory being coupled; the memory stores a computer execution program of the cell microcarrier detection model constructed by the construction method described in the first aspect; when the processor executes the computer execution program stored in the memory, the processor executes an organoid cell microcarrier detection method.

[0027] The fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program or instruction of a cell microcarrier detection model constructed by the construction method described in the first aspect, and when the program or instruction is executed by a processor, the processor executes an organoid cell microcarrier detection method.

[0028] Compared with the prior art, the present invention has the following beneficial effects: (1) Improved intelligence and practicality: This method uses a neural network to automatically identify cell microcarriers in the chip, significantly improving detection efficiency and intelligence. At the same time, the multi-level feature extraction and fusion modules enable the model to maintain high accuracy and stability when dealing with dense or sparse cell distribution. In addition, the final lightweight design ensures that the model can be successfully deployed on microfluidic devices with limited computing power, thereby greatly enhancing the practicality of this technology in microfluidic chip applications. (2) Improving the lightweight degree of the model: In the constructed cell microcarrier detection model, a feature extraction module based on deep separable convolution is used. Compared with conventional convolution operations, the amount of computation consumed in feature extraction is reduced; at the same time, a model lightweight module based on channel pruning and quantization operations is constructed to further improve the lightweight degree of the model, and the final lightweight cell microcarrier detection model is obtained; when conducting long-term, multi-batch cell microcarrier detection, the use of a lightweight cell microcarrier detection model can significantly reduce the consumption of power resources and reduce the cost of using the model; (3) Improving detection accuracy: In the cell microcarrier detection model constructed by the present invention, a multi-layer feature extraction module is designed to extract image features at different levels, so as to improve the detection rate of cell microcarriers during actual cell microcarrier detection, and ultimately enhance the detection accuracy; (4) Improving the robustness of target detection: In the present invention, a multi-level feature fusion module including fusion from deep layer to shallow layer and from shallow layer to deep layer is designed, which can further improve the robustness of target detection. Regardless of whether the density of cell microcarriers in the cell sorting area of ​​the microfluidic chip is high or low, this method can ensure a high detection rate of cell microcarriers; (5) Strong generalization ability: A data enhancement module is designed to improve the diversity and differentiation of model training data, so as to enhance the generalization ability of the cell microcarrier detection model. When dealing with the influence of external conditions (light, cell stacking, cell fluid flow rate) on the morphology of cell microcarriers in actual cell microcarrier detection, the detection ability can still be maintained under the influence of external conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the overall scheme of the present invention; Figure 2 This is the overall structure diagram of the cell microcarrier detection model of the present invention; Figure 3 It is a structural block diagram of the multi-layer feature extraction module of the present invention; Figure 4 This is a structural block diagram of the feature fusion module of the present invention; Figure 5 It is a schematic diagram of the optimization process of lightweighting the model of the present invention; Figure 6 This is a flow chart of the detection after the actual deployment of the cell microcarrier detection model of the present invention; Figure 7 This is a simplified diagram of the structure of the organoid cell microcarrier detection device. DETAILED DESCRIPTION

[0030] The present invention is further described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] This invention aims to improve the accuracy and real-time performance of microfluidic chips in the cell sorting process, and achieves full process optimization from data collection to model deployment through a series of innovative steps. Figure 1 As shown, first, a high-precision camera is deployed in the sorting area of ​​the microfluidic chip to collect high-quality organoid cell microcarrier images, and after noise reduction and normalization processing, the image quality is ensured, laying a solid foundation for subsequent model training. Afterwards, in order to enhance the generalization ability of the model, the present invention designs a data enhancement module, which significantly expands the data set using multi-sample data enhancement technology to improve the model detection accuracy. Next, a cell microcarrier detection model including a multi-layer feature extraction module, a feature fusion module, and a target detection module is constructed. After the model training is completed, the cell microcarrier detection model is further compressed by channel pruning and quantization operations to obtain a lightweight cell microcarrier detection model to adapt to memory-constrained embedded devices and ensure that efficient detection can be achieved on the terminal platform. Finally, the compressed lightweight model is deployed in the actual microfluidic chip sorting area, and the image data transmitted back by the image acquisition device is received in real time, and the flow of the cell microcarrier is quickly analyzed and fed back, providing strong technical support for cell sorting.

[0032] 1. Dataset Collection and Preprocessing 1. Initial cell microcarrier image acquisition: As a data-driven technology, the present invention needs to obtain the real cell microcarrier activity image in the sorting area of ​​the microfluidic chip. The present invention completes the cell microcarrier image acquisition by deploying a high-precision camera above the sorting area of ​​the microfluidic chip. The image resolution is 800*400; 2. Image filtering: During the cell microcarrier image acquisition process, the cell microcarrier image may be There is interference from noise factors in the image, and the presence of noise will be detrimental to the accurate identification of cell microcarriers in the image. The present invention proposes an image filtering processing module, in which a Gaussian filter is used to filter the collected cell microcarrier image. Perform Gaussian filtering to remove the noise in the image and obtain the denoised image ;

[0033] in, Represents Gaussian filter processing function; 3. Image normalization: Image processed by Gaussian filtering The noise information contained in the image will be greatly reduced, but The range of pixel values ​​in is very large, which causes the model to consume more resources during training and actual use. Therefore, in order to speed up the convergence of the model, reduce the memory consumed by the model, and improve the lightweight degree of the model, the present invention uses the maximum and minimum normalization method to denoise the image. Perform normalization operation, as shown in the following formula, to normalize all pixel values ​​in the image to The image after normalization operation has been obtained ;

[0034] in, Representing images Middle Line The pixel value of the pixel point in the column, Represents the image after normalization Middle Line The pixel value of the pixel point in the column, Respectively represent images The minimum pixel value and the maximum pixel value among all pixels; 4. Image normalization processing method: In order to further improve the generalization effect after model training, the present invention performs normalization on the image after normalization operation. The standard image dataset has been obtained by standardization operation , the standardized formula is as follows:

[0035] in, Represents the image after normalization operation Middle Line The pixel value of the pixel point in the column, Respectively represent the image The pixel average value of all pixels and the standard deviation calculated for all pixels; 5. Data calibration: based on the obtained standard image data , use the image annotation software LabelImg to annotate the image The cell microcarriers in the image are marked with rectangular frames to obtain the position information of the cell microcarriers in the image, including the specific x, y coordinates and the length s and width h of the cell microcarriers.

[0036] 6. Data Augmentation: In order to avoid overfitting in subsequent model training, improve the robustness of the model in cell microcarrier detection, and enhance the generalization ability of the model, the present invention proposes a data enhancement module based on a multi-sample data enhancement method, which specifically includes the following steps: Image geometric transformation processing: standard image dataset The images with a resolution of 800*400 are scaled to 400*200. After the image scaling operation, the images in the scaled image dataset are randomly flipped horizontally and vertically to obtain the flipped image dataset. .

[0037] Image color transformation processing: scaling the dataset The images in the image are randomly adjusted in brightness, contrast and color saturation to obtain the color transformation image dataset .

[0038] Image stitching operation: from scaled image dataset , flipped image dataset And the color transformation dataset Four different images are randomly selected from the three datasets for image stitching, and the four images with a resolution of 400*200 are stitched together into an image with a resolution of 800*400. The stitched images are then added to the original dataset. middle.

[0039] 2. Construction of cell microcarrier detection model The cell microcarrier detection model constructed in the present invention is mainly composed of three modules: a multi-layer feature extraction module, a feature fusion module, and a target detection module. The multi-layer feature extraction module performs multi-level feature extraction on the image to include the global feature information and local feature information of the cell microcarrier image to ensure the comprehensiveness of the feature information. The feature fusion module effectively fuses the extracted multi-level features to fully consider the global feature information and local feature information of the cell microcarrier image and improve the accuracy of model detection. Finally, the target detection module is used to further process the output features of the feature fusion module to obtain the final detection result. The cell microcarrier detection model is as follows: Figure 2 shown.

[0040] 1. Multi-layer feature extraction module design: Since the features of cell microcarriers are relatively simple, too many feature extraction operations will not only increase the memory occupied by the model, but also introduce more redundant information. Therefore, in the present invention, a cell microcarrier feature extraction module based on separable convolution operation is designed. Compared with the feature extraction module using the traditional convolution operation, the feature extraction module using this separable convolution operation can reduce the network parameters included in the feature extraction module, reduce the computing resources required for the model, and improve the overall lightweight level of the model. The specific structure is as follows: Figure 3 As shown, the specific process includes: 1) Channel convolution operation: In the feature extraction module proposed in the present invention, three 3*3 convolution kernels are first used to perform convolution operation on the RGB channels of the input data, where each 3*3 convolution kernel can obtain a corresponding output feature. In the present invention, the three different features are defined as .

[0041] 2) Point convolution operation: three different channel features respectively Perform point convolution again, that is, use the 1*1 convolution kernel to further extract the features of the image and obtain the features ;

[0042]

[0043]

[0044] Then, a fully connected layer with three neurons as input and one neuron as output is used to transform the features As the input of this fully connected layer, the output of the fully connected layer is used as the final feature extracted .

[0045]

[0046] in, Represents the fully connected layer function.

[0047] 2. Multi-scale feature fusion module design: In the cell microcarrier detection model designed by the present invention, multiple feature extraction modules are designed to extract the features of cell microcarriers at different levels from multiple levels and different receptive fields. In order to effectively fuse the extracted features at different levels, the present invention designs two feature fusion operation modules, from deep to shallow and from shallow to deep, which fully consider the information of features at different levels, taking into account both global features and local features, thereby improving the detection accuracy under different cell carrier distribution densities, such as Figure 4 shown.

[0048] Shallow to deep feature fusion module design: This structure is a feature transfer module that transfers features from shallow layers to deep layers step by step. After passing through the convolution layer, the shallow features are fused with the features of the deeper layers. The shallow features are added to the features of the next layer after passing through the convolution layer as an outward output feature. This output feature is processed by a convolution layer and used as the input feature of the deep to shallow feature fusion module. This fusion operation starts from the shallowest layer features and ends with the deepest layer features. Deep to shallow feature fusion module design: When performing feature fusion, in addition to performing feature fusion from shallow to deep layers, in order to further improve the feature fusion effect, the present invention designs a feature fusion module from deep to shallow layers, which performs feature fusion operation on deep features after passing through the pooling layer and shallower features. This fusion operation starts from the deepest layer features and ends with the shallowest layer features to obtain the output of this multi-scale feature fusion module. .

[0049] 3. Object detection module design: based on the output of multi-scale feature fusion module , design a target detection module to establish a connection with the final cell microcarrier detection results of the model to achieve the final detection of cell microcarriers. This target detection module consists of three 3*3 convolutional layers plus a Relu activation function to form the front part of this module, and two maximum pooling layers, a Dropout layer and a Softmax activation function to form the back part of this target detection module.

[0050] 3. Model Optimization For the trained cell microcarrier detection model, in order to further improve the model's lightweightness and reduce the model's required memory, the present invention proposes a model lightweight module based on channel pruning and quantization operations; its overall flow chart is as follows: Figure 5 As shown, the specific steps include: S1, the cell microcarrier detection model is trained using a standard image dataset. After the training is completed, the output feature vectors of the nine feature extraction modules in the constructed multi-layer feature extraction module are defined as , .

[0051] in, Represents the output feature vector The first element values, ; S2, calculate each output feature vector in turn The mean of all elements in

[0052]

[0053] S3, the model finally outputs the feature vector The mean calculation of :

[0054] in, ,

[0055] S4, feature correlation value calculation: calculate the output feature vectors of the nine feature extraction models in sequence , and the model's final feature output vector The characteristic correlation value of .

[0056]

[0057] Among them, the feature correlation value , represents the output feature vector of each feature extraction module And the final model output The larger the feature correlation value, the higher the correlation between this feature and the model output, that is, the more important this feature is. Generally speaking, selecting the top 40% of features will not reduce the prediction effect of the model. 9*40% Therefore, the present invention only retains the feature extraction modules with feature correlation values ​​in the top 4, and the remaining 5 feature extraction modules will be eliminated, thereby reducing the model memory and improving the lightweight degree of the model.

[0058] S5, after obtaining the cell microcarrier detection model after channel pruning, further quantize the weight values ​​in the model, and quantize all floating-point data in the model into fixed-point data, so as to further reduce the size of the model, and reduce the memory usage and power consumption of the model; After the channel pruning operation and the quantization operation, the cell microcarrier detection model at this time is used as the final lightweight cell microcarrier detection model of the present invention.

[0059] 4. Model Deployment and Application Based on the finally obtained lightweight cell microcarrier detection model, the detection model is used to monitor the cell microcarriers in the sorting area of ​​the microfluidic chip in real time. The specific implementation steps are as follows: Figure 6 As shown: Image acquisition: A high-precision image sensor is deployed above the sorting area of ​​the microfluidic chip to collect images of the sorting area of ​​the microfluidic chip in real time; Image processing: The collected initial image is sent to the image processing module (including image filtering module, image normalization module, and image standardization module) for processing to enhance the image quality and improve the detection accuracy of the subsequent model for cell microcarriers; Deployment of lightweight cell microcarrier detection model: The lightweight cell microcarrier detection model finally obtained by the present invention is deployed into the Raspberry Pi, and the processed image is sent to the cell microcarrier detection model for detection to obtain the specific position of the cell microcarrier in the sorting area of ​​the microfluidic chip; Based on the detection results of the model, the staff was able to automatically detect the flow of cell microcarriers in the sorting area of ​​the microfluidic chip in order to further deploy subsequent work.

[0060] like Figure 7 As shown, the present invention also provides an organoid cell microcarrier detection device, the device includes at least one processor and at least one memory, and also includes a communication interface and an internal bus; the memory stores a computer execution program of the cell microcarrier detection model built by the construction method described in Example 1; when the processor executes the computer execution program stored in the memory, the processor can execute an organoid cell microcarrier detection method. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0061] The device may be provided as a terminal, a server or other forms of device. In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0062] The present invention also provides a computer-readable storage medium, which stores a computer execution program of the underwater obstacle detection model constructed by the construction method as described above. When the computer execution program is executed by a processor, the processor can execute an unmanned boat obstacle recognition method based on image analysis.

[0063] Specifically, a system, device or equipment equipped with a readable storage medium may be provided, on which a software program code for implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system, device or equipment reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the above-mentioned embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0064] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0065] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for constructing an organoid cell microcarrier detection model, characterized in that: The process includes: S1, collect images of organoid cell microcarriers to build the initial dataset; S2, preprocess the initial dataset images and construct a standard dataset; The preprocessing includes image filtering, image normalization, image standardization and data calibration; S3, constructing a cell microcarrier detection model, including a multi-layer feature extraction module, a feature fusion module and a target detection module; the multi-layer feature extraction module is a cell microcarrier feature extraction module based on a separable convolution operation, which performs multi-level feature extraction on the image to include the global feature information and local feature information of the cell microcarrier image; the feature fusion module effectively fuses the extracted multi-level features to fully consider the global feature information and local feature information of the cell microcarrier image; the target detection module detects the output features of the feature fusion module and outputs the detection results; S4, train the cell microcarrier detection model based on the standard dataset, and optimize the model based on the designed channel pruning and quantization operations to improve the detection accuracy of the model in memory-constrained embedded devices and obtain the final model.

2. The method for constructing an organoid cell microcarrier detection model according to claim 1, characterized in that: The specific process of preprocessing in S2 includes: S21, image filtering: Gaussian filter is used to filter the collected cell microcarrier images. Perform Gaussian filtering to remove the noise in the image and obtain the denoised image ; in, Represents Gaussian filter processing function; S22, image normalization: Use the maximum and minimum normalization method to normalize the denoised image Perform the normalization operation, which is as follows: Normalize all pixel values ​​in the image to To get the normalized image ;in, Representing images Middle Line The pixel value of the pixel point in the column, Represents the image after normalization Middle Line The pixel value of the pixel point in the column, Respectively represent images The minimum pixel value and the maximum pixel value among all pixels; S23, image normalization: normalize the image The standard image dataset has been obtained by standardization operation , the standardized formula is as follows: in, Represents the image after normalization operation Middle Line The pixel value of the pixel point in the column, Respectively represent the image The pixel average value of all pixels and the standard deviation calculated for all pixels; S24, data calibration: based on the obtained standard image data , use the image annotation software LabelImg to annotate the image The cell microcarriers in the image are marked with rectangular frames to obtain the position information of the cell microcarriers in the image, including the specific x, y coordinates and the length s and width h of the cell microcarriers.

3. The method for constructing an organoid cell microcarrier detection model according to claim 1, characterized in that: The preprocessing in S2 also includes enhancing and expanding the image. The specific process includes: Image geometric transformation processing: standard image dataset The image with a resolution of 800*400 is scaled to 400*200 to obtain a scaled image dataset. After the image scaling operation, the images in the scaled image dataset are randomly flipped horizontally and vertically to obtain the flipped image dataset. ; Image color transformation processing: scaling the dataset The images in the image are randomly adjusted in brightness, contrast and color saturation to obtain the color transformation image dataset ; Image stitching operation: from scaled image dataset , flipped image dataset And the color transformation dataset Four different images are randomly selected from the three datasets for image stitching. The four images with a resolution of 400*200 are re-stitched into an image with a resolution of 800*400, and the re-stitched images are added to the original standard dataset. middle.

4. The method for constructing an organoid cell microcarrier detection model according to claim 1, characterized in that: The multi-layer feature extraction module includes a channel convolution operation and a point convolution operation; The channel convolution operation first uses three 3*3 convolution kernels to perform convolution operations on the RGB channels of the input data, where each 3*3 convolution kernel can obtain a corresponding output feature, and the three different features obtained are defined in turn as ; The point convolution operation is performed on three different channel features. Perform point convolution again, that is, use the 1*1 convolution kernel to further extract the features of the image and obtain the features ; Then, a fully connected layer with three neurons as input and one neuron as output is used to transform the features As the input of this fully connected layer, the output of the fully connected layer is used as the final feature extracted : in, Represents the fully connected layer function.

5. The method for constructing an organoid cell microcarrier detection model according to claim 1, characterized in that: The feature fusion module is designed as two feature fusion operation modules, one from deep to shallow and the other from shallow to deep, which fully considers the information of features at different levels; The shallow-to-deep feature fusion module is a feature transfer module that transfers features from shallow layers to deep layers step by step. The shallow features are fused with the deeper features after passing through the convolution layer. The shallow features are added with the features of the next layer after passing through the convolution layer as an outward output feature. This output feature is processed by a convolution layer as the input feature of the deep-to-shallow feature fusion module. This fusion operation starts from the shallowest features and ends with the deepest features. The deep-to-shallow feature fusion module performs feature fusion operation on the deep features after passing through the pooling layer and the shallower features. This fusion operation starts from the deepest features and ends with the shallowest features to obtain the output of this multi-scale feature fusion module. .

6. The method for constructing an organoid cell microcarrier detection model according to claim 1, characterized in that: The target detection module consists of three 3*3 convolutional layers plus a Relu activation function, and the latter part of the target detection module consists of two maximum pooling layers, a Dropout layer and a Softmax activation function.

7. The method for constructing an organoid cell microcarrier detection model according to claim 1, characterized in that: In S4, the model is optimized based on the designed channel pruning and quantization operations. The specific process is as follows: S41, using the standard image data set to train the cell microcarrier detection model. After the training is completed, the output feature vectors of q feature extraction modules in the multi-layer feature extraction module are defined as , ; in, Represents the output feature vector The first element values, ; S42, calculate each output feature vector in turn The mean of all elements in S43, the model finally outputs the feature vector The mean calculation of : in, ; S44, feature correlation value calculation: Calculate the output feature vectors of q feature extraction models in sequence , and the model's final feature output vector The characteristic correlation value of : The feature correlation value , represents the output feature vector of each feature extraction module And the final model output The larger the feature correlation value, the higher the correlation between this feature and the model output, that is, the more important this feature is. The feature extraction modules with the highest feature correlation value are retained, and the other feature extraction modules are eliminated. S45, after obtaining the cell microcarrier detection model after channel pruning, further quantize the weight values ​​in the model, and quantize all floating-point data in the model into fixed-point data to further reduce the size of the model.

8. A method for detecting organoid cell microcarriers, characterized in that: The cell microcarrier detection model constructed by the construction method according to any one of claims 1 to 7 is deployed in a Raspberry Pi, and includes the following process: S1, based on the high-precision image sensor deployed above the sorting area of ​​the microfluidic chip, collects images of the sorting area of ​​the microfluidic chip in real time; S2, performing image filtering, image normalization and standardization processing on the collected initial image; S3, input the standardized image into the cell-microcarrier detection model; S4, output the detection image results of the cell microcarriers in the sorting area of ​​the microfluidic chip.

9. An organoid cell microcarrier detection device, characterized in that: The device comprises at least one processor and at least one memory, wherein the processor and the memory are coupled; the memory stores a computer execution program of a cell microcarrier detection model constructed by the construction method according to any one of claims 1 to 7; when the processor executes the computer execution program stored in the memory, the processor executes an organoid cell microcarrier detection method.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instruction of a cell microcarrier detection model constructed by the construction method according to any one of claims 1 to 7, and when the program or instruction is executed by a processor, the processor executes an organoid cell microcarrier detection method.

Images