A remote real-time analysis and monitoring system for cell growth

Abstract

The present application proposes a method for remotely and real-time analyzing and monitoring cell growth, including the following steps: obtaining at least two cell images of the area to be monitored collected at a preset acquisition interval, inputting them into a trained instance segmentation module to obtain first cell features, and fusing the first cell features and second cell features to obtain cell image features of the current cell image; arranging the cell image features of multiple cell images in the order of acquisition time to form a cell feature sequence, and inputting the cell feature sequence into a trained prediction module to obtain the predicted cell number and predicted fusion degree of the cells to be monitored. This solution constructs an instance segmentation module and a prediction module, and predicts the cell number and fusion degree by performing instance segmentation on cell images and combining the environmental information during the cell growth process.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and particularly to a remote real-time analysis and monitoring system for cell growth. Background Art

[0002] With the development of modern science, in vitro cell culture technology has become an essential research tool in multiple fields, such as being widely applied in genetics, immunology, oncology, virology, molecular biology, etc. In vitro cell culture refers to a technology that places cells collected from tissues in the body under sterile, appropriate temperature, humidity, and rich nutrient conditions to make them grow, reproduce, and maintain their structure and function. The most crucial aspect of in vitro cell culture is to maintain cell viability, so it requires experimental personnel to conduct long-term dynamic observation and detection of the cell morphology and growth state.

[0003] When observing cells, experimental personnel usually take the cells out of the incubator and place them under an inverted microscope for observation. However, the direct observation method exposes the cell culture dish to the air, increasing the risk of cell contamination. In addition, the cells may also be stressed due to changes in environmental temperature, which affects the cell growth state and reduces cell activity. Moreover, cell number and cell confluence are important indicators for evaluating cell viability and assessing different experimental conditions. They can not only help experimental personnel deeply understand the molecular mechanism at the cell level but also provide important guidance for predicting disease progression and formulating treatment plans. Currently, the main methods for cell counting include manual counting, cell counter counting, and flow cytometry, etc. However, all these three methods require trypsin or a cell scraper to detach the cells from the bottom of the container, make a cell suspension, and then dilute it to an appropriate multiple for observation. In the manual counting method, the cell suspension needs to be stained with trypan blue, and the blue-stained cells cannot be counted during counting. After counting, the cell counting plate also needs to be cleaned, with cumbersome operations and large human judgment errors; in the cell counter counting method, dead cells and live cells cannot be distinguished, and the counting accuracy is poor; when using flow cytometry for cell counting, it is efficient, fast, and the results are accurate, but the instrument is expensive, the maintenance cost is high, and professional operators are required. In addition, the analysis of cell confluence is usually manually judged based on the experience of experimental personnel, with large errors and low precision.

[0004] With the development of machine vision technology, although the prior art can calculate cell count and cell confluence through neural networks, its detection effect is only good for scattered cells, and the detection effect for connected cells is poor. In addition, the existing neural networks only consider the current cell state during detection and lack the analysis and early warning of the future development state of cells.

[0005] In summary, there is an urgent need for a method that can view the cell status in real time without taking the cells out of the incubator and can accurately analyze the future development status of the current cells. Summary of the Invention

[0006] An embodiment of the present application provides a method and device for remotely and real-time analyzing and monitoring cell growth. By constructing an instance segmentation module and a prediction module, the number and confluence of cells are predicted by performing instance segmentation on cell images and combining the environmental information during the cell growth process.

[0007] In a first aspect, an embodiment of the present application provides a method for remotely and real-time analyzing and monitoring cell growth, the method comprising:

[0008] Obtaining at least two cell images of a to-be-monitored area collected at a preset acquisition interval;

[0009] Inputting the cell images into a trained instance segmentation module to obtain first cell features corresponding to each cell image, and fusing the first cell features and second cell features to obtain cell image features of the current cell image, wherein the first cell features include cell category, cell number, and cell confluence, and the second cell features are environmental features of the growth environment corresponding to the current cell image;

[0010] Arranging the cell image features of multiple cell images in the order of acquisition time to form a cell feature sequence, and inputting the cell feature sequence into a trained prediction module to obtain the predicted cell number and predicted confluence of the to-be-monitored cells.

[0011] In a second aspect, an embodiment of the present application provides a device for remotely and real-time analyzing and monitoring cell growth, comprising a platform end, a device end, and a background database:

[0012] The platform end is configured to send operation instructions to the device end;

[0013] The device end includes a data acquisition module, a trained instance segmentation module, and a trained prediction module. The data acquisition module is configured to obtain at least two cell images of a to-be-monitored area collected at a preset acquisition interval, input the cell images into a trained instance segmentation module to obtain first cell features corresponding to each cell image, fuse the first cell features and second cell features to obtain cell image features of the current cell image, wherein the first cell features include cell category, cell number, and cell confluence, and the second cell features are environmental features of the growth environment corresponding to the current cell image, arrange the cell image features of multiple cell images in the order of acquisition time to form a cell feature sequence, and input the cell feature sequence into a trained prediction module to obtain the predicted cell number and predicted confluence of the to-be-monitored cells;

[0014] The background database is used to store all information generated by the platform side and the device side, and is called by the platform side and the device side.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute a method for remotely and real-time analyzing and monitoring cell growth.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium. A computer program is stored in the readable storage medium, and the computer program includes program codes for controlling a process to execute the process, and the process includes a method for remotely and real-time analyzing and monitoring cell growth.

[0017] The main contributions and innovations of the present invention are as follows:

[0018] In the embodiment of the present application, an instance segmentation module is constructed to obtain the cell categories and the number of cells in a cell image. And in the present solution, the instance segmentation module sets a dynamic segmentation threshold for each detection box based on the mask area of the detection box with the highest confidence to perform segmentation, so as to effectively avoid misidentifying multiple closely adjacent cells as a single large cell, or mis-segmenting a single cell into multiple small cells, and improve the accuracy of segmentation. When predicting, the prediction module in the present solution will comprehensively predict by combining the environmental information of each cell image during growth. And through the prediction of multiple cell images of the same area to be monitored, the prediction accuracy of the prediction module for the area to be monitored can be dynamically improved. Moreover, with the accumulation of experimental data, the prediction module can self-optimize and provide more and more accurate growth predictions.

[0019] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. Description of the Drawings

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0021] Figure 1 is a flowchart of a method for remotely and real-time analyzing and monitoring cell growth according to an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of a device for remotely and real-time analyzing and monitoring cell growth according to an embodiment of the present application;

[0023] Figure 3Schematic diagram of feature fusion using an attention mechanism according to an embodiment of the present application;

[0024] Figure 4 Schematic diagram of a prediction module according to an embodiment of the present application;

[0025] Figure 5 Schematic diagram of the training process of an instance segmentation module and a prediction module according to an embodiment of the present application;

[0026] Figure 6 Schematic diagram of the application process of an instance segmentation module and a prediction module according to an embodiment of the present application;

[0027] Figure 7 Schematic diagram of a main control unit according to an embodiment of the present application;

[0028] Figure 8 Schematic diagram of data interaction between a background database and the front end according to an embodiment of the present application;

[0029] Figure 9 Schematic diagram of data interaction between a device end and a background database according to an embodiment of the present application;

[0030] Figure 10 Schematic diagram of data interaction between a main control unit and a lower-level unit according to an embodiment of the present application;

[0031] Figure 11 Schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. Detailed implementation mode

[0032] Here, exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0033] It should be noted that: In other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0034] Embodiment 1

[0035] The embodiment of the present application provides a method for remotely and real-time analyzing and monitoring cell growth. By constructing an instance segmentation module and a prediction module, the method predicts the cell number and confluence by performing instance segmentation on cell images and combining the environmental information during the cell growth process. Specifically, referring to Figure 1 , the method for remotely and real-time analyzing and monitoring cell growth includes;

[0036] Obtaining at least two cell images of the area to be monitored collected at a preset acquisition interval;

[0037] Inputting the cell images into the trained instance segmentation module to obtain the first cell features corresponding to each cell image, and fusing the first cell features and the second cell features to obtain the cell image features of the current cell image, where the first cell features include cell category, cell number, and cell confluence, and the second cell features are environmental features of the growth environment corresponding to the current cell image;

[0038] Arranging the cell image features of multiple cell images in the order of acquisition time to form a cell feature sequence, and inputting the cell feature sequence into the trained prediction module to obtain the predicted cell number and predicted confluence of the cells to be monitored.

[0039] It should be noted that the method for remotely and real-time analyzing and monitoring cell growth in this solution is implemented relying on a device for remotely and real-time analyzing and monitoring cell growth. The device for remotely and real-time analyzing and monitoring cell growth is as Figure 2 shown and includes:

[0040] A platform terminal, a device terminal, and a background database;

[0041] The platform terminal is used to send operation instructions to the device terminal;

[0042] The device terminal includes a data acquisition module, a trained instance segmentation module, and a trained prediction module. The data acquisition module is used to obtain at least two cell images of the cells to be monitored collected at a preset acquisition interval, input the cell images into the trained instance segmentation module to obtain the first cell features corresponding to each cell image, fuse the first cell features and the second cell features to obtain the cell image features of the current cell image, where the first cell features include cell category, cell number, and cell confluence, and the second cell features are environmental features of the growth environment corresponding to the current cell image, arrange the cell image features of multiple cell images in the order of acquisition time to form a cell feature sequence, and input the cell feature sequence into the trained prediction module to obtain the predicted cell number and predicted confluence of the cells to be monitored;

[0043] The background database is used to store all information generated by the platform side and the device side, and is called by the platform side and the device side.

[0044] Regarding the data acquisition module:

[0045] This solution uses a data acquisition module to obtain at least two cell images of the cells to be monitored collected at a preset acquisition interval. Specifically, in the step of "obtaining at least two cell images of the cells to be monitored collected at a preset acquisition interval", any one or more than two preprocessing operations such as image brightness processing, image offset correction processing, image compression processing, and image recognition processing are performed on the cell images.

[0046] Specifically, in some embodiments, image brightness processing is performed on each cell image to improve the visibility and resolution of the cells. The means of image brightness processing is to perform light and dark optimization on each cell image to improve the contrast and brightness of the cell image until the requirements for segmentation by the instance segmentation module are met.

[0047] In some embodiments, image offset correction processing is performed on each cell image to ensure the accuracy and consistency of the image. The means of image offset correction is to correct the orientation of each cell image to a unified orientation and eliminate the offset caused by device movement or lens jitter during image acquisition.

[0048] In some embodiments, image compression processing is performed on each cell image to compress the volume while ensuring the image quality and reduce the storage volume.

[0049] In some embodiments, image recognition processing is performed on each cell image to facilitate cell recognition. The means of image recognition processing is to convert the pixel values of each cell image into RGB values and perform normalization processing on the R, G, and B values of each cell image. Specifically, normalization can be completed by dividing the R value, G value, and B value in the cell image by 255.

[0050] The data acquisition module of this solution is used to acquire cell images. In some embodiments, the data acquisition module includes a camera controlled by the device side. After receiving the acquisition instruction from the platform side, the device side controls the camera to acquire cell images of the cells to be monitored at a preset acquisition interval. After any one or more than two preprocessing operations such as image brightness processing, image offset correction processing, image compression processing, and image recognition processing are performed on the cell images at the device side, they are transmitted to the background database for storage.

[0051] In some embodiments, the preset acquisition interval in the device acquisition module is set manually. The preset acquisition interval in this solution is 1 hour. That is to say, the device acquisition module automatically acquires an image every 1 hour.

[0052] Regarding the instance segmentation module:

[0053] The instance segmentation module in this solution includes a detection unit, a segmentation unit, and a classification unit. The detection unit records the position of each cell in the cell image with a detection box. The segmentation unit is used to determine whether each pixel point in the detection box is a cell pixel to segment the cell contour. The classification unit is used to identify the category of each cell, where dead cells are regarded as a separate cell category.

[0054] In some embodiments, the instance segmentation module can adopt any intelligent model that can perform instance segmentation. In this solution, Yolov9 is used as the instance segmentation module.

[0055] Furthermore, a detection threshold is set to retain the detection boxes with a confidence level greater than the detection threshold. Then, an initial segmentation threshold is set, and the initial segmentation threshold is used to segment each pixel point in each detection box. The pixel points greater than the initial segmentation threshold are regarded as cell pixels, and the pixel points less than or equal to the initial segmentation threshold are regarded as non-cell pixels. The mask area composed of all cell pixels in each detection box is calculated. Finally, based on the mask area of the detection box with the highest confidence level, the segmentation threshold of each detection box is dynamically adjusted to obtain the dynamic threshold corresponding to each detection box, and each detection box is segmented based on the corresponding dynamic threshold to obtain the cell contour.

[0056] Specifically, the formula for dynamically adjusting the segmentation threshold of each detection box is as follows:

[0057] Tseg = Si / (Si + Smax)

[0058] Where Tseg is the dynamic threshold, Si is the mask area of the current detection box under the initial segmentation threshold, and Smax is the mask area of the detection box with the highest confidence level.

[0059] Specifically, by dynamically adjusting the segmentation thresholds of different detection boxes, it can effectively avoid misidentifying multiple closely adjacent cells as a single large cell, or mis-segmenting a single cell into multiple small cells, improving the accuracy of segmentation.

[0060] Specifically, when judging the category of cells in this solution, the dead cells are regarded as a separate cell category, and an alarm can be issued when the number of dead cells reaches a certain amount.

[0061] Specifically, category labels are used to represent cell categories, from 0 to C, where C is the number of categories. The numbers 0 to C - 1 correspond to different types of living cells respectively, and C is the number of all dead cells.

[0062] In some specific embodiments, the number of cells and the degree of fusion in the cell image can be obtained through the analysis of the cell segmentation results, so as to be used for the prediction of the subsequent prediction module.

[0063] In some specific embodiments, the training method of the instance segmentation module is as follows:

[0064] Obtain a plurality of cell images marked with cell categories and the corresponding mask image for each cell image as training samples. The contour information of each cell is marked and recorded in the mask image, and the pre-constructed instance segmentation framework is trained with the training samples to obtain a trained instance segmentation network.

[0065] During the training process, in order to ensure the generalization of the instance segmentation network, the training samples are augmented by scaling, rotating, and changing the contrast.

[0066] In this solution, the attention mechanism is used to assign weights to the cell category, the number of cells, the cell fusion degree, and the environmental features and sum them up after weighting to obtain the cell image features. The fusion process using the attention mechanism for fusion is as Figure 3 shown.

[0067] Specifically, using the attention mechanism to assign weights to and fuse various types of features can not only reduce the computational complexity of the prediction module, but also highlight the most powerful features for prediction, so as to enhance the accuracy of the prediction result.

[0068] In this solution, the growth environment features of the cells include temperature, humidity, carbon dioxide concentration, etc. In this solution, more accurate growth prediction is carried out by combining these multi-modal data during prediction, so as to identify the influence of environmental factors on cell growth, and the growth environment of the cells can also be optimized according to the prediction results.

[0069] Furthermore, in order to ensure the accuracy of the prediction, the first cell feature and the second cell feature are normalized.

[0070] In some embodiments, based on the proportion of the cell area in the cell image obtained by the instance segmentation module in the entire cell image, the upper limit of the number of cell images is estimated, or the upper limit of the number of each cell image is directly set manually. The number of cells in the first cell feature is divided by the upper limit of the number of cell images to complete the normalization.

[0071] Exemplarily, this solution adopts the manual setting method to set the upper limit of the number of each cell image, and the upper limit of the number of cells in this solution is set to 400.

[0072] In some specific embodiments, if the second cell feature includes temperature and humidity, then the temperature is divided by 100 and the relative humidity is used for calculation to complete the normalization.

[0073] Regarding the prediction module:

[0074] The schematic diagram of the prediction module in this solution is as shown in Figure 4 . The structure of the prediction module is three fully connected layers in series. Set the prediction time span, combine the cell feature sequence with the prediction time span, and input them into the prediction module to obtain the predicted cell number and predicted fusion degree within the prediction time span.

[0075] Exemplarily, when the set prediction time span is 5 hours, the prediction result of the prediction module is the predicted cell number and predicted fusion degree within 5 hours. That is to say, if the acquisition time set in this solution is to acquire a cell image every 1 hour, then the obtained result is 5 prediction results within 5 hours.

[0076] Furthermore, set the longest prediction time, normalize the prediction time span with the longest prediction time and then input it into the prediction module. The formula is as follows:

[0077] Prediction time span / Longest prediction time

[0078] Among them, the prediction time span is the future time minus the current time. That is to say, in this solution, when the time span is greater than the longest prediction time, that is, when time span / longest prediction time >= 1, it exceeds the prediction ability of the prediction module and cannot make a prediction.

[0079] In this solution, the predicted cell number will reach the upper limit, and the upper limit value is a manually set value. For example, if the set upper limit of the cell number in this solution is 400, then the predicted cell number by the prediction module will not be greater than 400.

[0080] In some specific embodiments, the schematic diagram of the training process of the instance segmentation module and the prediction module is as shown in Figure 5 . The schematic diagram of the application process of the instance segmentation module and the prediction module is as shown in Figure 6 .

[0081] The predicted cell number and predicted fusion degree obtained through the prediction module can issue a warning before an abnormal situation occurs, so as to adjust the experimental conditions in advance, such as the ratio of nutrient solution or the fine-tuning of environmental parameters. This prediction mode can significantly improve the success rate of the experiment and reduce the experimental period and cost.

[0082] In the cell growth remote real-time analysis and monitoring device, the platform side is built through a Web application. Users can access the platform side interface through a browser to operate. The platform side has user verification and permission management functions to ensure system security. Moreover, the platform side provides a graphical interface to display real-time data, analysis results, and operation buttons, which is convenient for users to operate.

[0083] In some specific embodiments, the platform end provides input boxes for username and password, as well as a submit button. The platform end provides a user authentication function to ensure that only authorized users can access the system. Users fill in the username and password and send them to the server via an https request. After receiving the request, the backend queries the corresponding username and password in the database for matching verification. If the verification is successful, a login success message is returned; otherwise, a failure message is returned.

[0084] Specifically, the operation instructions issued by the platform end include monitoring points, shooting time intervals, etc. A editable table or form is provided on the platform end and users are allowed to input or modify parameter values therein. The backend provides an API interface for the front end to call. The API interface is used to obtain the current parameter values or update the parameter values.

[0085] In this solution, the device end receives various operation instructions sent by the platform end through the main control unit. The schematic diagram of the main control unit is as Figure 7 shown, and an instance segmentation module and a prediction module are loaded in the main control unit, which is responsible for performing various preprocessing on the cell images collected by the image acquisition module. The device end drives the motor through the lower computer to control the camera to move to the specified location and collect images at a preset acquisition interval. At the same time, the lower computer also obtains environmental characteristics through various sensors such as temperature and humidity and returns them to the main control unit for various calculations and predictions.

[0086] Specifically, after the platform end issues a start monitoring instruction, the main control processing unit of the device end controls the motor module to move the camera to the specified position and perform image acquisition according to the instruction of the platform end. The main control processing unit receives the collected image data, applies image processing and analysis algorithms to optimize the image quality, count the number of cells and the degree of fusion, and generate quantitative data. For the existing cell models in the system, the system applies image segmentation technology to extract the cell regions in the image, and then automatically identifies and counts the number of cells and the degree of fusion.

[0087] In this solution, the predicted cell number and predicted fusion degree are called in the background database, and the predicted cell number and predicted fusion degree are used to generate images, scatter plots, line graphs or time-lapse videos and displayed on the platform end.

[0088] Specifically, by generating images, scatter plots, line graphs or time-lapse videos of the predicted cell number and predicted fusion degree, it is convenient for customers to intuitively see the experimental results. In the scientific research field, it provides visual data for scientific researchers in the fields of cell biology, drug screening, disease mechanism research, etc., which helps to promote the progress of related research; in the education field, this system can be used as a teaching aid to help students more intuitively understand the complex processes of cell biology.

[0089] In some specific embodiments, the data interaction between the background database and the front end is as follows Figure 8 shown, and its steps include:

[0090] User input: The user inputs data or performs operations through the platform, such as setting project parameters;

[0091] Sending a request: The front end generates an https request from the user's operation, and the request data is encrypted using the https protocol before being sent to protect the privacy and data security of the customer;

[0092] Server processing: The request is sent to the back-end server. After receiving the request, the server parses the data and parameters of the request;

[0093] Database access: The back end generates corresponding database query or operation requests according to the request content and business logic, such as executing a database query or calling other services;

[0094] Returning a response: After processing the request, the back end returns the result to the UI in JSON format;

[0095] Displaying the result: After receiving the response from the server, the UI updates the page display, such as displaying the query result or feedback information;

[0096] Real-time update: Using the WebSocket protocol and packing data in JSON format to achieve real-time update of the UI without reloading the page;

[0097] Error handling: The UI needs to be able to handle and display possible errors, such as network problems or server errors.

[0098] In some specific embodiments, the data interaction between the device side and the background database is as follows Figure 9 shown, and its steps include:

[0099] Data collection: The device side collects image data through devices such as cameras;

[0100] Data processing: Process and analyze the collected images to extract the required information.

[0101] Data preparation: Pack the processed data in JSON format;

[0102] Establishing a connection: The device side establishes a secure connection with the background database through https;

[0103] Sending data: Send the data to the background database through the established connection;

[0104] Receiving data: The background database receives and stores the data in the corresponding database table;

[0105] Confirm the upload. The background database sends a confirmation message to the device side, indicating that the data upload is successful;

[0106] Handle errors. If the upload fails, the device side will retry or perform other operations according to the error message;

[0107] Record logs: The background database records the data interaction process for subsequent monitoring and problem troubleshooting.

[0108] In some specific embodiments, the master control unit includes a lower-level unit, which is used to control the movement of the camera, image acquisition, and collection of environmental features. The data interaction between the master control unit and the lower-level unit is as Figure 10 shown, and its steps include:

[0109] Receive instructions. The master control receives instructions from the platform side;

[0110] Generate instructions. The master control generates motor control instructions, temperature and humidity acquisition instructions, etc. according to the image acquisition task and system requirements;

[0111] Execute tasks. The lower-level unit drives the motor to perform corresponding actions according to the instructions, adjusts the position and focus of the camera, etc., and completes tasks such as motor control tasks, camera acquisition tasks, and temperature and humidity acquisition; Receive data. The master control receives image data from the camera and processes it, receives data from the temperature and humidity acquisition module transmitted by the sensor, and uses it for system monitoring or adjustment;

[0112] Data processing and interaction. The master control inputs the acquired image data into the AI algorithm for analysis, obtains data such as the number of cells and the degree of fusion, and uploads the processing results, status information, etc. through data interaction with the platform; Heartbeat packet task. The master control regularly sends heartbeat packets to the platform to confirm the online status, which helps the platform detect whether the master control is online;

[0113] Receive feedback. The master control receives the status feedback from the lower-level unit to confirm the instruction execution result and device status;

[0114] Error handling. If the lower-level unit feedbacks an error or abnormal information, the master control will perform error handling or resend the instruction;

[0115] Feedback to the user: The master control displays the processing result or system status information on the user interface and feedbacks it to the user.

[0116] In some specific embodiments, the device-side host uses a single-chip microcomputer main control processing unit, which has high reliability and low power consumption. The camera acquisition module uses an industrial-grade camera, and the motor control module uses a stepper motor to ensure the precise movement and positioning of the camera. The communication module selects a stable Wi-Fi module, which can ensure the stability and real-time performance of data transmission. The background database uses an SQL database to store image data and analysis results, which has strong transaction support and rich data types. The data storage is designed to store image data and analysis results separately to ensure the clarity of the data structure and the efficiency of data access. The database server is hosted by a cloud service provider, which provides data backup and disaster recovery measures to ensure data security.

[0117] Embodiment III

[0118] This embodiment also provides an electronic device. Refer to Figure 11 , which includes a memory 404 and a processor 402. A computer program is stored in the memory 404, and the processor 402 is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0119] Specifically, the above-mentioned processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0120] Among them, the memory 404 may include a mass storage 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 404 may include removable or non-removable (or fixed) media. In a suitable case, the memory 404 may be internal or external to the data processing device. In a particular embodiment, the memory 404 is a non-volatile memory. In a particular embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. In a suitable case, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended date out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0121] The memory 404 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402.

[0122] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the cell growth remote real-time analysis and monitoring methods in the above embodiments.

[0123] Optionally, the above electronic device may further include a transmission device 406 and an input / output device 408. Among them, the transmission device 406 is connected to the above processor 402, and the input / output device 408 is connected to the above processor 402.

[0124] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above network may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0125] The input / output device 408 is used to input or output information. In this embodiment, the input information can be a cell image sequence, etc., and the output information can be the predicted cell number and the predicted fusion degree.

[0126] Optionally, in this embodiment, the above processor 402 can be set to execute the following steps through a computer program:

[0127] Obtain at least two cell images of the area to be monitored collected at a preset acquisition interval;

[0128] Input the cell images into the trained instance segmentation module to obtain the first cell features corresponding to each cell image, and fuse the first cell features and the second cell features to obtain the cell image features of the current cell image, where the first cell features include cell category, cell number, and cell fusion degree, and the second cell features are the environmental features of the growth environment corresponding to the current cell image;

[0129] Arrange the cell image features of multiple cell images in the order of acquisition time to form a cell feature sequence, and input the cell feature sequence into the trained prediction module to obtain the predicted cell number and the predicted fusion degree of the cells to be monitored.

[0130] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and alternative embodiments, and will not be elaborated herein.

[0131] In general, various embodiments can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the present invention can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. However, the present invention is not limited thereto. Although the various aspects of the present invention can be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, the blocks, devices, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or a controller or other computing device, or some combination thereof.

[0132] Embodiments of the present invention can be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. A computer software or program (also referred to as a program product), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer-executable components that are configured to perform the embodiments when the program runs. The one or more computer-executable components can be at least one software code or a part thereof. Additionally, in this regard, it should be noted that any block in the logical flow, as Figure 11 described herein, can represent a program step, or an interconnected logical circuit, block, and function, or a combination of a program step and a logical circuit, block, and function. The software can be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. The physical media is a non-transitory medium.

[0133] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as falling within the scope described in this specification.

[0134] The above embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for remote real-time analysis and monitoring of cell growth, characterized in that: The following steps are involved: Acquire at least two cell images of the area to be monitored acquired at a preset acquisition interval; Inputting the cell image into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, wherein the instance segmentation module includes a detection unit, a segmentation unit and a classification unit, wherein the detection unit records the position of each cell in the cell image with a detection frame, the segmentation unit is used to determine whether each pixel point in the detection frame is a cell pixel so as to segment the cell contour, and the classification unit is used to identify the category of each cell, wherein dead cells are regarded as separate cell categories, and the first cell feature and the second cell feature are fused to obtain the cell image feature of the current cell image, wherein the first cell feature includes the cell category, the number of cells and the degree of cell fusion, and the second cell feature is the environmental feature of the growth environment corresponding to the current cell image; The cell image features of multiple cell images are arranged in the order of acquisition time to form a cell feature sequence, and the cell feature sequence is input into the trained prediction module to obtain the predicted cell number and predicted fusion degree of the cells to be monitored.

2. A method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: In the step of "obtaining at least two cell images of the cells to be monitored acquired at a preset acquisition interval", the cell images are pre-processed by any one or more of image brightness processing, image offset correction processing, image compression processing and image recognition processing.

3. The method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: Set the detection threshold, retain the detection frames with confidence greater than the detection threshold, then set the initial segmentation threshold, use the initial segmentation threshold to segment each pixel in each detection frame, regard the pixels greater than the initial segmentation threshold as cell pixels, and the pixels less than or equal to the initial segmentation threshold as non-cell pixels. The mask area composed of all cell pixels in each detection frame, finally, dynamically adjust the segmentation threshold of each detection frame based on the mask area of ​​the detection frame with the highest confidence to obtain the dynamic threshold corresponding to each detection frame, and segment each detection frame based on the corresponding dynamic threshold to obtain the cell contour.

4. The method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: The attention mechanism is used to assign weights to cell categories, cell numbers, cell fusion, and environmental features, and the weighted sum is used to obtain cell image features.

5. The method for remote real-time analysis and monitoring of cell growth according to claim 1, characterized in that: The prediction time span is set, and the cell feature sequence is combined with the prediction time span and input into the prediction module to obtain the predicted cell number and predicted fusion degree within the prediction time span.

6. A remote real-time analysis and monitoring device for cell growth, characterized in that: Including platform side, device side and backend database: The platform end is used to send operation instructions to the device end; The device end includes a data acquisition module, a trained instance segmentation module and a trained prediction module, wherein the data acquisition module is used to obtain at least two cell images of cells to be monitored acquired at a preset acquisition interval, and the cell images are input into the trained instance segmentation module to obtain a first cell feature corresponding to each cell image, wherein the instance segmentation module includes a detection unit, a segmentation unit and a classification unit, wherein the detection unit records the position of each cell in the cell image with a detection frame, and the segmentation unit is used to determine whether each pixel point in the detection frame is a cell pixel so as to segment the cell contour, and the classification unit is used to identify the category of each cell, wherein dead cells are taken as separate cell categories, and the first cell feature and the second cell feature are fused to obtain the cell image feature of the current cell image, wherein the first cell feature includes the cell category, the number of cells and the degree of cell fusion, and the second cell feature is the environmental feature of the growth environment corresponding to the current cell image, and the cell image features of multiple cell images are arranged in the order of acquisition time to form a cell feature sequence, and the cell feature sequence is input into the trained prediction module to obtain the predicted cell number and predicted fusion degree of the cells to be monitored; The backend database is used to store all information generated by the platform side and the device side, and is used by the platform side and the device side for calling.

7. The device for remote real-time analysis and monitoring of cell growth according to claim 6, characterized in that: The predicted cell number and predicted fusion degree are called in the background database, and the predicted cell number and predicted fusion degree are generated into images, scatter plots, curve graphs or time-lapse videos and displayed on the platform.

8. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for remote real-time analysis and monitoring of cell growth according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes a method for remote real-time analysis and monitoring of cell growth according to any one of claims 1-6.

Images