Model online training method, cell recognition model training method, device and medium

Abstract

The application discloses a model online training method, a cell recognition model training method, device and medium. The model online training method comprises the following steps: obtaining a pre-built model, and starting a round of online training on the pre-built model. The online training comprises multiple training, and each training can obtain a corresponding loss value. According to the loss values corresponding to the multiple training respectively, a loss difference value is calculated. If the loss difference value is within a threshold range determined according to a preset rule, the round of online training is stopped, wherein the preset rule is associated with the loss value of the pre-built model in the round of online training. If the loss difference value is out of the threshold range, the next training is performed. The application sets the threshold range related to the loss value as a dynamic stopping training condition, so that the online training is not dependent on the unchanged parameters input by the user, the training time of the model can be shortened, and overfitting can be reduced.

Description

Technical Field

[0001] This application relates to the field of cell image analysis, and in particular to an online model training method, a cell recognition model training method, device, and medium. Background Technology

[0002] Currently, using deep learning methods for cell recognition, counting, and classification is a hot research topic in computer vision and a challenging area in medical image processing. However, in practical applications, the online training time for cell recognition models or cell counting models is often too long, requiring users to wait for extended periods. This is especially true on embedded devices, where model training is extremely time-consuming.

[0003] Current model training relies on user-defined hard stopping conditions, such as meeting a user-defined convergence condition or reaching a certain number of training iterations. Because current training models cannot flexibly and promptly stop online training, this not only leads to excessively long training times but also overfitting. Overfitting occurs when convergence conditions are set too strictly (e.g., setting the loss function threshold too low), causing the model to perform too well on training data but poorly on new data. For example, in label-free cell counting, the model not only has long online training times but also the manually set hard stopping conditions are almost impossible to trigger, resulting in overfitting due to too many training iterations or failure to achieve proper fitting.

[0004] The above background information is provided only to assist in understanding the inventive concept and technical solution of this invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical teaching. In the absence of clear evidence that the above information was disclosed before the filing date of this patent application, the above background information should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0005] The purpose of this invention is to provide an online model training method, a cell recognition model training method, an apparatus, and a medium. By setting a threshold range related to the loss value as a dynamic stopping condition, online training is not dependent on user-inputted invariant parameters, which can shorten the model training time and reduce overfitting.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An online model training method includes the following steps:

[0008] Obtain a pre-built model and start an online training on it. The online training includes multiple training sessions, and each training session can obtain a corresponding loss value.

[0009] Calculate the loss difference based on the loss values ​​corresponding to each of the multiple training iterations;

[0010] If the loss difference is within a threshold range determined according to a preset rule, then this round of online training is stopped, wherein the preset rule is associated with the loss value of the pre-built model in this round of online training; if the loss difference exceeds the threshold range, then the next round of training is executed.

[0011] Furthermore, based on any one or a combination of the aforementioned technical solutions, the threshold range is determined according to the following preset rules:

[0012] The threshold range is determined based on the loss value corresponding to the first training iteration of the online training.

[0013] Alternatively, the threshold range can be determined based on the loss values ​​corresponding to multiple training iterations of the online training.

[0014] Alternatively, the threshold range can be determined based on the loss value corresponding to each training iteration of the online training.

[0015] Furthermore, based on any one or a combination of the aforementioned technical solutions, the upper limit of the threshold range is determined according to a preset ratio:

[0016] The product of the loss value corresponding to the first training and the preset ratio value is used as the upper limit of the threshold range, wherein the preset ratio value is a constant or a value that changes with the number of training sessions in this round of online training.

[0017] Alternatively, calculate the average of the loss values ​​corresponding to multiple training iterations, and calculate the product of this average value and a preset ratio value as the upper limit of the threshold range;

[0018] Alternatively, the product of the loss value corresponding to the most recent training and a preset ratio value can be used as the upper limit of the threshold range, wherein the preset ratio value is a constant or increases with the number of training sessions in this round of online training.

[0019] Furthermore, based on any one or a combination of the aforementioned technical solutions, the preset ratio value ranges from 1% to 50%, or from 5% to 15%, or from 8% to 12%, or from 10%.

[0020] Furthermore, following any one or a combination of the aforementioned technical solutions, the step of calculating the loss difference based on the loss values ​​corresponding to the multiple training sessions includes: calculating the difference between the previous loss value and the latest loss value based on the loss values ​​corresponding to the two most recent training sessions.

[0021] Furthermore, following any of the aforementioned technical solutions or combinations thereof, if the difference between the previous loss value and the latest loss value repeatedly falls within the threshold range, then this round of online training is stopped.

[0022] Furthermore, following any one or a combination of the aforementioned technical solutions, the step of calculating the loss difference based on the loss values ​​corresponding to multiple training sessions includes: calculating the difference between the loss values ​​corresponding to each of two adjacent training sessions based on the loss values ​​corresponding to three or more adjacent training sessions.

[0023] Furthermore, following any of the aforementioned technical solutions or combinations thereof, if the difference between the loss values ​​corresponding to each of the two adjacent training sessions is within the threshold range, then this round of online training is stopped.

[0024] Alternatively, if the proportion of the difference between the loss values ​​corresponding to two consecutive training sessions that falls within the threshold range exceeds a preset proportion, then this round of online training will be stopped.

[0025] Furthermore, based on any one or a combination of the aforementioned technical solutions, a maximum training count is preset; if the current training count reaches the maximum training count, then this round of online training is stopped.

[0026] Alternatively, after stopping this round of online training, it may also include: updating the pre-built model using the currently trained model.

[0027] According to another aspect of the present invention, a method for training a cell recognition model is provided, comprising the following steps:

[0028] Establish a pre-built model for preliminary cell identification;

[0029] The pre-built model is trained online using any one or a combination of the above technical solutions to obtain a cell recognition model for cell identification.

[0030] Furthermore, based on any one or a combination of the aforementioned technical solutions, the pre-built model is established through the following steps:

[0031] A deep learning model is selected as the initial model, and deep learning is performed on the initial model using learning samples to obtain the base model;

[0032] The base model is retrained using a semi-supervised learning method to obtain the pre-built model. The semi-supervised learning method focuses on and learns only the regions of manually labeled cell images during the retraining process.

[0033] Furthermore, following any one or a combination of the aforementioned technical solutions, the step of retraining the base model using a semi-supervised learning method to obtain the pre-built model includes:

[0034] Obtain sample images for semi-supervised learning;

[0035] The sample image is input into the base model to obtain the intermediate image;

[0036] Identify regions in the intermediate image that contain defective features, including one or more of the following: cell adhesion, at least partial cell disappearance, and cell aberration;

[0037] The operation of manually marking at least one area with defective features;

[0038] The current model is retrained based on the manually labeled regions. During the retraining process, only the manually labeled regions are studied and learned to obtain the retrained model.

[0039] Based on the retrained model, the sample image is identified to obtain a new intermediate image; manual annotation is performed on at least one region in the new intermediate image that has defect features, and the retrained model is obtained.

[0040] The process continues until a preset number of repetitions is reached, or until no defective features are found in the intermediate images, at which point the current model is used as the pre-built model.

[0041] According to another aspect of the present invention, an online model training apparatus is provided, wherein one round of online training includes multiple training sessions, each with a corresponding loss value; the apparatus includes the following modules:

[0042] The pre-built model module is configured to acquire a pre-built model;

[0043] The loss difference module is configured to calculate the loss difference based on the loss values ​​corresponding to multiple training iterations.

[0044] A threshold module is configured to determine a threshold range according to a preset rule, wherein the preset rule is associated with the loss value of the pre-built model in this round of online training;

[0045] The stop module is configured to compare the loss difference with the threshold range, and if the loss difference is within the threshold range, then stop this round of online training.

[0046] According to another aspect of the present invention, a cell recognition model training device is provided, comprising:

[0047] A pre-built model creation module, configured to create pre-built models for initial cell identification; and

[0048] The online model training device described above is configured to train the pre-built model output by the pre-built model building module online to obtain a cell recognition model for recognizing cells.

[0049] According to another aspect of the present invention, an electronic device is provided, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is configured to execute the program instructions, which are executed to perform the steps of the method as described in any one of the above technical solutions or a combination of multiple technical solutions.

[0050] According to another aspect of the present invention, a computer-readable storage medium is provided for storing program instructions configured to be invoked to perform the steps of the method described in any one of the above technical solutions or a combination of multiple technical solutions.

[0051] According to another aspect of the present invention, a computer program product is provided, comprising a readable and storable computer program, the computer program including program instructions, which, when executed on a computer device, cause the computer device to perform the steps of the method as described in any one or a combination of the above technical solutions.

[0052] The beneficial effects of the technical solution provided by this invention are as follows:

[0053] a. This invention sets a threshold range related to the loss value as a dynamic stopping condition, so that online training does not depend on user-input invariant parameters, which can shorten the training time of the model and reduce overfitting;

[0054] b. The stopping condition for online training of the model is that the difference in loss is within a threshold range determined according to a preset rule. This stopping condition is not based on the result of a single training session, but on the difference between the loss values ​​of multiple training sessions. This makes the online training results of the model less affected by specific samples and can obtain a more reliable training model.

[0055] c. The online model training device provided by this invention can obtain a more reliable model more quickly and efficiently, and has a wide range of applications, including cell counting, cell identification, and cell classification.

[0056] d. The cell recognition model training device provided by this invention can quickly obtain a reliable cell recognition model, so that users do not need to spend a long time waiting for the online training process of the cell recognition model. Attached Figure Description

[0057] To more clearly illustrate the technical solutions or conventional technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or conventional technical solutions will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a basic flowchart of an online model training method provided in an exemplary embodiment of the present invention;

[0059] Figure 2 This is a schematic diagram illustrating the specific process of an online model training method provided in an exemplary embodiment of the present invention;

[0060] Figure 3 This is another specific flowchart illustrating the online model training method provided by an exemplary embodiment of the present invention;

[0061] Figure 4 This is another specific flowchart illustrating the online model training method provided by an exemplary embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of a system 100 that performs the methods described in the embodiments of the present invention. Detailed Implementation

[0063] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0064] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0065] In the field of neural networks, the loss value is defined as the difference between the predicted result and the true value. The loss value allows observation of the changes in online training results. It is typically used as a criterion for determining whether a model has converged. A common rule is to compare the loss value for each training iteration with a threshold; if it is less than the threshold, training stops. Therefore, setting the threshold is crucial for model training. If the threshold is set inappropriately, such as too high, the training time will be longer, and convergence may be delayed, resulting in low training efficiency. Conversely, if the threshold is set too low, although the model can complete training quickly, the convergence of the trained model will be low.

[0066] Thresholds are usually set based on empirical values. Compared to this approach of artificially setting the same loss threshold for different models or different training stages of the same model, this invention aims to propose an online model training scheme based on a more reasonable loss threshold.

[0067] In one embodiment of the present invention, see Figure 1 This paper provides an online model training method, which includes the following steps:

[0068] Obtain a pre-built model and start an online training on it. The online training includes multiple training sessions, and each training session can obtain a corresponding loss value.

[0069] Calculate the loss difference based on the loss values ​​corresponding to each of the multiple training iterations;

[0070] If the loss difference is within a threshold range determined according to a preset rule, then the current round of online training is stopped. The preset rule is associated with the loss value of the pre-built model in this round of online training. If the loss difference exceeds the threshold range, then online training continues, that is, the next training session of this round of online training is executed. After obtaining the loss value corresponding to the next training session, the step of calculating the loss difference is repeated until the loss difference is within the threshold range, at which point the current round of online training is stopped.

[0071] After stopping this round of online training, the pre-built model is updated using the currently trained model. That is, when starting the next round of online training, the pre-built model can adopt the model after completing the previous round of online training.

[0072] In this embodiment of the invention, the loss difference is set to fall within a threshold range determined according to a preset rule as the stopping condition for online model training. The preset rule is associated with the loss value of the pre-built model in this round of online training. Therefore, compared to manually set hard stopping conditions, this application uses the loss value corresponding to one or more training iterations of the model in this round of online training to determine a soft stopping condition according to a preset rule, which can shorten the online training time of the model while ensuring reliable training results. The hard stopping condition mentioned here is like manually setting an empirical threshold value, which does not refer to the model itself; while the soft stopping condition sets a relevant threshold value with reference to the model itself.

[0073] The threshold range described above, which is determined according to preset rules, that is, using the loss value corresponding to one or more training sessions of the model in this round of online training to determine a soft stopping condition according to preset rules, can be done in a variety of ways. Examples are given below.

[0074] In one embodiment of the present invention, the threshold range is determined based on the loss value corresponding to the first training in the online training.

[0075] The product of the loss value corresponding to the first training and the preset ratio value is used as the upper limit of the threshold range, thereby determining the threshold range. The preset ratio value is a constant or a value that changes with the number of training sessions in this round of online training.

[0076] In this embodiment, the preset ratio value ranges from 1% to 50%. Preferably, the preset ratio value is 10%. For example, if the loss value corresponding to the first training in this round of online training is 0.8, then the upper limit of the threshold range is determined to be 0.8 * 10% = 0.08. Usually, the lower limit of the threshold range can be set to 0.

[0077] In another embodiment of the present invention, the threshold range is determined based on the loss values ​​corresponding to multiple training iterations of the online training.

[0078] For example, the average of the loss values ​​corresponding to multiple training iterations can be calculated, and the product of this average and a preset proportion can be used as the upper limit of the threshold range, thereby determining the threshold range. In this embodiment, the preset proportion ranges from 5% to 15%, preferably 10%. For example, if the loss value corresponding to the first training iteration in this round of online training is 0.8, the loss value corresponding to the second training iteration is 0.75, and the loss value corresponding to the third training iteration is 0.7, then 10% of the average of the first three loss values ​​is taken as the upper limit of the threshold range, i.e., 0.075. This invention does not limit whether it is the average of the first three loss values ​​or the average of the first two loss values, nor does it limit it to the average of multiple loss values ​​starting from the first one, nor does it limit it to the average of adjacent consecutive or scattered multiple loss values, and it does not even limit the method of calculating the average. In addition to calculating the average, other methods can also be used, such as if the loss value corresponding to the first training iteration in this round of online training is 0.8, and the loss value corresponding to the second training iteration is 0.7, then three-quarters of the difference between these two losses, i.e., 0.075, is taken as the upper limit of the threshold range.

[0079] In another embodiment of the present invention, the threshold range is determined based on the loss value corresponding to each training iteration of the online training. Unlike the embodiments described above where the threshold range determined after one or more training iterations remains unchanged, the upper limit of the threshold range determined in this embodiment dynamically changes with the loss value of each training iteration.

[0080] The upper limit of the threshold range is calculated as the product of the loss value corresponding to the most recent training and a preset ratio value, wherein the preset ratio value is a constant or increases with the number of training iterations in this round of online training. In a specific embodiment, the preset ratio value increases with the number of training iterations in this round of online training. If the loss value corresponding to the first training iteration in this round of online training is 0.8 and the ratio value is 10%, then the upper limit of the threshold range is calculated to be 0.08. If the condition for stopping this round of online training is not met, the next training iteration is executed, resulting in a loss value of 0.75 for the second training iteration, at which point the ratio value increases to 10.8%, and so on. This invention does not limit the ratio value to increase at a constant rate each time.

[0081] As can be seen from the above embodiments of determining the threshold range according to preset rules, using the loss difference falling within the threshold range determined by preset rules as the stopping condition for online model training is a dynamic stopping condition (the loss threshold corresponding to different rounds of online training changes dynamically). That is, different rounds of online training will obtain different threshold ranges due to the differences in loss values ​​in different rounds of online training, even if they are based on the same preset rules. When the online training result meets this dynamic condition, training will automatically stop, which can not only shorten the training time but also reduce overfitting. In addition, the stopping condition is not based on the result of a single training session, but on the loss difference of multiple training sessions as the object of comparison with the threshold range, so that the online training result of the model is less affected by specific samples and can obtain more reliable training results.

[0082] In one specific embodiment of the present invention, see [link to relevant documentation]. Figure 2 A pre-built model is acquired, and an online training round is initiated. Based on the loss values ​​corresponding to the two most recent training iterations, the difference between the previous and latest loss values ​​is calculated. If the difference between the previous and latest loss values ​​repeatedly falls within the threshold range, the online training round is stopped. In this embodiment, the threshold range can be determined using any of the various methods described above for determining the threshold range according to preset rules, and will not be elaborated further.

[0083] In another specific embodiment of the present invention, see Figure 3 A pre-built model is acquired and an online training iteration is initiated. Based on the loss values ​​corresponding to three or more consecutive training iterations, the difference between the loss values ​​corresponding to each two consecutive training iterations is calculated. If the difference between the loss values ​​corresponding to each two consecutive training iterations is within the threshold range, the online training iteration is stopped. Figure 3 Taking the loss values ​​corresponding to the three most recent consecutive training iterations as an example, deltal1 is the difference between the loss value corresponding to the first training iteration and the loss value corresponding to the second training iteration, and deltal2 is the difference between the loss value corresponding to the second training iteration and the loss value corresponding to the third training iteration (the most recent training iteration). In this embodiment, the threshold range can be determined by any of the various methods described above for determining the threshold range according to preset rules, which will not be elaborated further.

[0084] For example, in one specific embodiment, during a training round, the threshold range is (0, 0.08). The stopping condition is that the difference between the loss values ​​corresponding to two adjacent training iterations in three consecutive training iterations is within the threshold range. The difference between the loss values ​​corresponding to two adjacent training iterations is the difference between the previous loss value and the next loss value. In this embodiment, the loss values ​​corresponding to the first four training iterations are 0.45, 0.35, 0.30, and 0.27, respectively. The differences between adjacent loss values ​​are deltal1 = 0.45 - 0.35 = 0.1, deltal2 = 0.35 - 0.30 = 0.05, and deltal3 = 0.30 - 0.27 = 0.03, respectively. It can be seen that deltal2 and deltal3 are within the threshold range, thus satisfying the stopping condition. If the loss values ​​corresponding to the first four training iterations are 0.45, 0.35, 0.38, and 0.41, then the stopping condition is not satisfied because the loss difference is less than 0. It should be noted that the online training method for the model provided by this invention requires the loss value to have a decreasing trend, but does not require the difference between adjacent loss values ​​to also have a decreasing trend. For example, the loss values ​​corresponding to the first four training iterations are 0.45, 0.35, 0.30, and 0.23, respectively, and the corresponding loss differences are 0.1, 0.05, and 0.07, respectively, which also satisfy the stopping condition.

[0085] Figure 2 and Figure 3 The difference between the corresponding embodiments is that, Figure 3 The corresponding implementation uses the condition that the difference between two adjacent losses obtained from three consecutive training iterations both satisfy a threshold range as the criterion for stopping the current round of online training; while Figure 2 The corresponding implementation does not limit the continuity. For example, the loss difference between the first and second training sessions is within the threshold range, the loss difference between the second and third training sessions is not within the threshold range, the loss difference between the third and fourth training sessions is within the threshold range, the loss difference between the fourth and fifth training sessions is not within the threshold range, and the loss difference between the fifth and sixth training sessions is within the threshold range. This results in a statistical analysis of cases where the loss difference is within the threshold range three times. Figure 2 If the "preset number of times" is 3, then the condition for stopping this round of online training is met.

[0086] In another specific embodiment of the present invention, see Figure 4 If the proportion of the difference between the loss values ​​corresponding to two consecutive training iterations falling within the threshold range exceeds a preset percentage, then this round of online training is stopped. In this embodiment, the threshold range can be determined using any of the various methods described above for determining the threshold range according to preset rules, which will not be elaborated further.

[0087] In one embodiment of the present invention, a preset upper limit for the number of training iterations can be established. If the current number of training iterations reaches the upper limit, the online training should be stopped even if the loss difference is not yet within the threshold range. During training, unstable factors inevitably exist, which may cause significant and unreasonable deviations in the threshold range determined according to the preset rules. In this embodiment, setting an upper limit for the number of training iterations can effectively avoid or reduce the risk and loss of low training efficiency caused by this situation.

[0088] In another embodiment of the present invention, a cell recognition model training method is provided, comprising the following steps:

[0089] Establish a pre-built model for preliminary cell identification;

[0090] The cell image to be identified is input into the pre-built model. If the output image is not satisfactory, the output image can be manually annotated. The purpose is to correct and supplement the recognition result of the pre-built model. The manually annotated image is used as a learning sample for online training.

[0091] The pre-built model is trained online using the online training method described in any of the above embodiments to obtain a cell recognition model for identifying cells. If the output image of the cell recognition model is still unsatisfactory, a new round of online learning is initiated for the current cell recognition model until the final cell recognition model can output a satisfactory result image.

[0092] In this embodiment, the pre-built model is established through the following steps:

[0093] A deep learning model is selected as the initial model, and deep learning is performed on the initial model using learning samples to obtain the base model;

[0094] The base model is retrained using a semi-supervised learning method to obtain the pre-built model. The semi-supervised learning method focuses on and learns only the regions of manually labeled cell images during the retraining process.

[0095] In this embodiment, retraining the base model using a semi-supervised learning method to obtain the pre-built model includes:

[0096] Acquire sample images for semi-supervised learning, wherein the number of sample images is multiple;

[0097] The sample image is input into the base model to obtain the intermediate image;

[0098] Identify regions in the intermediate image that contain defective features, including one or more of the following: cell adhesion, at least partial cell disappearance, and cell aberration;

[0099] The operation of manually marking at least one area with defective features;

[0100] The current model is retrained based on the manual annotations. Specifically, the sample image and at least the local region image with annotation (or the training result image after the annotation operation) are input into the current model to retrain and obtain a new model. During the retraining process, only the manually annotated regions are studied and learned to obtain the retrained model.

[0101] Based on the retrained model, the sample image is identified to obtain a new intermediate image; manual annotation is performed on at least one region in the new intermediate image that has defect features, and the retrained model is obtained.

[0102] The process continues until a preset number of repetitions is reached, or until no defective features are found in the intermediate images, at which point the current model is used as the pre-built model. In this embodiment, the pre-built model possesses preliminary recognition capabilities, and its level of recognition capability is not limited. Alternatively, the method for obtaining the pre-built model can be described by referencing the full text of Chinese invention patent application CN114972222A.

[0103] In another embodiment of the present invention, an online model training device is provided, wherein one round of online training includes multiple training sessions, each with a corresponding loss value; the device includes the following modules:

[0104] The pre-built model module is configured to acquire a pre-built model;

[0105] The loss difference module is configured to calculate the loss difference based on the loss values ​​corresponding to multiple training iterations.

[0106] A threshold module is configured to determine a threshold range according to a preset rule, wherein the preset rule is associated with the loss value of the pre-built model in this round of online training;

[0107] The stop module is configured to compare the loss difference with the threshold range, and if the loss difference is within the threshold range, then stop this round of online training.

[0108] This embodiment of the online model training device and the above-described embodiment of the online model training method belong to the same inventive concept. Therefore, the entire contents of the online model training method embodiment are incorporated into this embodiment of the device by means of full reference.

[0109] In another embodiment of the present invention, a cell recognition model training device is provided, including a pre-built model establishment module and the online model training device described in the above embodiment. The pre-built model establishment module is configured to establish a pre-built model for preliminary cell recognition. The online model training device is configured to perform online training on the pre-built model output by the pre-built model establishment module to obtain a cell recognition model for cell recognition.

[0110] In one embodiment of the present invention, an electronic device is provided, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is configured to run the program instructions, which are executed to perform the steps performed as described in the above method embodiments.

[0111] In one embodiment of the present invention, a computer-readable storage medium is provided for storing program instructions configured to be invoked to perform the steps performed as described in the above method embodiments.

[0112] In one embodiment of the present invention, a computer program product is provided, including a readable and storable computer program, the computer program including program instructions, which, when executed on a computer device, cause the computer device to perform the steps performed as described in the above method embodiments.

[0113] It should be noted that the above-mentioned embodiments of the online model training device, cell recognition model training device, electronic device, computer-readable storage medium, and computer program product belong to the same inventive concept as the embodiments of the online model training method. All contents of the embodiments of the online model training method are incorporated into the embodiments of the online training device, cell recognition model training device, electronic device, computer-readable storage medium, and computer program product by reference.

[0114] Some embodiments relate to a microscope including a device for cell recognition. Optionally, the microscope may be used to perform actions such as... Figures 1 to 4 One or more methods in the system are part of the system or connected to the system. Figure 5A schematic diagram of a system 100 configured to perform the methods described in this invention is shown. System 100 includes a microscope 110 and a computer system 120. The microscope 110 is configured to capture images and is connected to the computer system 120. The computer system 120 is configured to perform at least a portion of the methods described in this invention. The computer system 120 may be configured to execute machine learning algorithms. The computer system 120 and the microscope 110 may be separate entities, but may also be integrated into a common housing. The computer system 120 may be part of the central processing system of the microscope 110 and / or, the computer system 120 may be part of a sub-component of the microscope 110, such as a sensor, actuator, camera, or illumination unit of the microscope 110.

[0115] Computer system 120 may be a local computer device (e.g., a personal computer, laptop, tablet, or mobile phone) having one or more processors and one or more storage devices, or it may be a distributed computer system (e.g., having one or more processors and one or more storage devices distributed in various locations, such as local clients and / or one or more remote server sites and / or data centers). Computer system 120 may include any circuitry or combination of circuitry. In one embodiment, computer system 120 may include one or more processors that can be of any type. As used in this invention, a processor may refer to any type of computing circuitry, such as, but not limited to, a microprocessor, microcontroller, complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long long instruction word (VLIW) microprocessor, graphics processor, digital signal processor (DSP), multi-core processor, field-programmable gate array (FPGA) for example, a microscope or microscope component (e.g., a camera), or any other type of processor or processing circuitry. Other types of circuitry that may be included in computer system 120 may be custom circuitry, application-specific integrated circuits (AS1C), etc., such as one or more circuits (e.g., communication circuitry) used in wireless devices such as mobile phones, tablets, laptops, two-way radios, and similar electronic systems. Computer system 120 may include one or more storage devices, which may include one or more storage elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard disk drives, and / or one or more drives that process removable media, such as optical discs (CDs), flash memory cards, digital video discs (DVDs), etc. Computer system 120 may also include a display device, one or more speakers, and a keyboard and / or controller, which may include a mouse, trackball, touchscreen, voice recognition device, or any other device that allows a system user to input information into or receive information from computer system 120.

[0116] Some or all of the method steps can be performed by (or using) hardware devices (e.g., processors, microprocessors, programmable computers, or electronic circuits). In some embodiments, such devices can perform one or more of the most important method steps.

[0117] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software. This implementation can be carried out using a non-transitory storage medium (such as a digital storage medium, e.g., floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH) with electronically readable control signals stored thereon, which cooperate (or are capable of cooperating with) a programmable computer system to perform the corresponding methods. Therefore, the digital storage medium can be computer-readable.

[0118] Some embodiments of the present invention include a data carrier having electronically readable control signals that are capable of cooperating with a programmable computer system to perform one of the methods described in the present invention.

[0119] Typically, embodiments of the present invention can be implemented as a computer program product having program code that, when run on a computer, is executable to perform one of the methods. The program code may, for example, be stored on a machine-readable medium.

[0120] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described in this invention.

[0121] In other words, an embodiment of the present invention is therefore a computer program having program code for performing one of the methods described in the present invention when the computer program is run on a computer.

[0122] Therefore, another embodiment of the invention is a storage medium (or data carrier or computer-readable medium) comprising a computer program stored thereon for performing one of the methods described in the invention when executed by a processor. Data carriers, digital storage media, or recording media are typically tangible and / or non-transitory. Yet another embodiment of the invention is a device as described in the invention, comprising a processor and a storage medium.

[0123] Therefore, another embodiment of the present invention represents a data stream or signal sequence for performing one of the methods described in the present invention. The data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection, such as via the Internet.

[0124] Another embodiment includes a processing device, such as a computer or programmable logic device, configured or adapted to perform one of the methods described in this invention.

[0125] Another embodiment includes a computer on which a computer program for performing one of the methods described in this invention is installed.

[0126] Another embodiment of the invention includes an apparatus or system configured to transmit a computer program (e.g., electronically or optically) for performing one of the methods described in the invention to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0127] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described in this invention. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described in this invention. Generally, the method is preferably performed by any hardware device.

[0128] Implementation examples can be based on the use of machine learning models or machine learning algorithms. Machine learning can refer to algorithms and statistical models that allow computer systems to perform specific tasks without explicit instructions, relying instead on models and inference. For example, in machine learning, instead of rule-based data transformations, data transformations derived from the analysis of historical and / or training data can be used. For example, machine learning models or algorithms can be used to analyze the content of images. To enable a machine learning model to analyze the content of an image, it can be trained using training images as input and training content information as output. By training the machine learning model with a large number of training images and / or training sequences (e.g., words or sentences) and associated training content information (e.g., labels or annotations), the machine learning model “learns” to recognize the content of images, making it possible to identify image content not included in the training data. The same principle can also be applied to other types of sensor data: by training a machine learning model using training sensor data and a desired output, the machine learning model can “learn” the transformation between sensor data and output, which can be used to provide outputs based on non-training sensor data provided to the machine learning model. The provided data (e.g., sensor data, metadata, and / or image data) can be preprocessed to obtain feature vectors, which are used as inputs to the machine learning model.

[0129] Machine learning models can be trained using training input data. The example specified above uses a training method called "guided learning." In guided learning, a machine learning model is trained using multiple training samples, where each sample can include multiple input data values ​​and multiple desired output values; that is, each training sample is associated with a desired output value. By specifying training samples and desired output values, the machine learning model "learns" the output values ​​to be provided based on input samples similar to those provided during training. In addition to guided learning, semi-guided learning can also be used. In semi-guided learning, some training samples lack corresponding desired output values. Guided learning can be based on guided learning algorithms (such as classification algorithms, regression algorithms, or similarity learning algorithms). Classification algorithms can be used when the output is restricted to a set of limited values ​​(categorical variables), i.e., the input is classified into one of a set of limited values. Regression algorithms can be used when the output can have any numerical value (within a range). Similarity learning algorithms may be similar to classification and regression algorithms, but are based on learning from examples using a similarity function that measures the degree of similarity or correlation between two objects. In addition to guided or semi-guided learning, unguided learning can also be used to train machine learning models. In unsupervised learning, input data can be provided (only), and unsupervised learning algorithms can be used to find structures in the input data (e.g., by grouping or clustering the input data to find commonalities). Clustering is the process of assigning input data containing multiple input values ​​to subsets (clusters) such that input values ​​in the same cluster are similar according to one or more (predefined) similarity criteria, while being dissimilar to input values ​​contained in other clusters.

[0130] Reinforcement learning is the third group of machine learning algorithms. In other words, reinforcement learning can be used to train machine learning models. In reinforcement learning, one or more software actors (called "software agents") are trained to take actions in an environment. Rewards are calculated based on the actions taken. Reinforcement learning is based on training one or more software agents to make action choices so that the cumulative reward increases, thereby making the software agents better at performing tasks (proven by increasing rewards).

[0131] Furthermore, certain techniques can be applied to some machine learning algorithms. For example, feature learning can be used. In other words, feature learning can be used to at least partially train machine learning models, and / or machine learning algorithms can include feature learning components. Feature learning algorithms (which may be called representation learning algorithms) can retain information in their input, but can also transform it in ways that make it useful, often as a preprocessing step before performing classification or prediction. Feature learning can be based on, for example, principal component analysis or cluster analysis.

[0132] In some examples, anomaly detection (i.e., outlier detection) can be used, the purpose of which is to provide identification of suspicious input values ​​by making them significantly different from most of the input or training data. In other words, anomaly detection can be used to at least partially train machine learning models, and / or machine learning algorithms can include anomaly detection components.

[0133] In some examples, machine learning algorithms can use decision trees as predictive models. In other words, machine learning models can be based on decision trees. In a decision tree, observations about an item (e.g., a set of input values) can be represented by branches of the decision tree, and the output value corresponding to that item can be represented by the leaves of the decision tree. Decision trees can support both discrete and continuous values ​​as output values. If discrete values ​​are used, a decision tree can be represented as a classification tree; if continuous values ​​are used, a decision tree can be represented as a regression tree.

[0134] Association rules are another technique that can be used in machine learning algorithms. In other words, a machine learning model can be based on one or more association rules. Association rules are created by identifying relationships between variables in a large amount of data. Machine learning algorithms can identify and / or utilize one or more relational rules that represent knowledge derived from the data. These rules can be used, for example, to store, manipulate, or apply that knowledge.

[0135] Machine learning algorithms are typically based on machine learning models. In other words, the term "machine learning algorithm" can refer to a set of instructions that can be used to create, train, or use a machine learning model. The term "machine learning model" can refer to a data structure and / or set of rules representing the learned knowledge (e.g., based on training performed by a machine learning algorithm). In embodiments, the use of a machine learning algorithm can mean the use of a potential machine learning model (or multiple potential machine learning models). The use of a machine learning model can mean that the machine learning model and / or the data structure / rule set serving as the machine learning model is trained by a machine learning algorithm.

[0136] For example, a machine learning model can be an artificial neural network (ANN). An ANN is a system inspired by biological neural networks, such as those found in the retina or brain. An ANN consists of multiple interconnected nodes and multiple connections between nodes, called edges. There are typically three types of nodes: input nodes that receive input values, hidden nodes that are (only) connected to other nodes, and output nodes that provide output values. Each node can represent an artificial neuron. Each edge can transfer information from one node to another. The output of a node can be defined as a (non-linear) function of its inputs (e.g., the sum of its inputs). The node's inputs can be used in the function based on the "weights" of the edges or nodes that provide the input. The weights of nodes and / or edges can be adjusted during the learning process. In other words, training an artificial neural network can involve adjusting the weights of the nodes and / or edges of the artificial neural network to achieve the desired output for a given input.

[0137] Optionally, the machine learning model can be a Support Vector Machine (SVM), a Random Forest model, or a Gradient Boosting model. A Support Vector Machine (SVM) is a guided learning model with relevant learning algorithms that can be used to analyze data (e.g., in classification or regression analysis). An SVM can be trained by providing multiple training input values ​​belonging to one of two categories. An SVM can be trained to assign new input values ​​to one of the two categories. Optionally, the machine learning model can be a Bayesian network, which is a probabilistic directed acyclic graph model. A Bayesian network can use a directed acyclic graph to represent a set of random variables and their conditional dependencies. Optionally, the machine learning model can be based on a genetic algorithm, which is a search algorithm and exploration technique that mimics the process of natural selection.

[0138] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0139] As used in this invention, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0140] The above description is merely a specific embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An online training method for a cell recognition model, characterized in that, The following steps are included in training the cell recognition model: Establish a pre-built model for preliminary cell identification; Input a cell image to be identified into the pre-built model. If the output image is not satisfactory, the output image is manually annotated to correct and supplement the recognition result of the pre-built model. The manually annotated image is used as a learning sample for online training. The pre-built model is trained online once to obtain a cell recognition model for cell identification. The online training includes multiple training sessions, each of which yields a corresponding loss value. Calculate the loss difference based on the loss values ​​corresponding to each of the multiple training iterations; If the loss difference falls within a threshold range determined by a preset rule, the current round of online training is stopped. The preset rule calculates the upper limit of the threshold range by multiplying the loss value corresponding to the first training iteration by a preset ratio, where the preset ratio is a value that changes with the number of training iterations. Stopping the current round of online training requires that the loss value shows a decreasing trend, but does not require that the difference between adjacent loss values ​​show a decreasing trend. If the difference between the loss value corresponding to the previous training iteration and the loss value corresponding to the next training iteration is less than 0, the condition for stopping the current round of online training is not met. If the loss difference exceeds the threshold range, the next training iteration is executed. If the current number of training iterations reaches a preset upper limit, the current round of online training is stopped. If the output image of the cell recognition model is still unsatisfactory, a new round of online training will be started for the current cell recognition model until the final cell recognition model can output a satisfactory image.

2. The online model training method according to claim 1, characterized in that, The preset ratio value can be between 1% and 50%, or between 5% and 15%, or between 8% and 12%, or 10%.

3. The online model training method according to claim 1 or 2, characterized in that, The step of calculating the loss difference based on the loss values ​​corresponding to multiple training sessions includes: calculating the difference between the previous loss value and the latest loss value based on the loss values ​​corresponding to the two most recent training sessions.

4. The online model training method according to claim 3, characterized in that, If the difference between the previous loss value and the latest loss value falls within the threshold range multiple times, then this round of online training will be stopped.

5. The online model training method according to claim 1 or 2, characterized in that, The step of calculating the loss difference based on the loss values ​​corresponding to multiple training sessions includes: calculating the difference between the loss values ​​corresponding to each of two adjacent training sessions based on the loss values ​​corresponding to three or more adjacent training sessions.

6. The online model training method according to claim 5, characterized in that, If the difference between the loss values ​​corresponding to each of the two adjacent training iterations is within the threshold range, then stop this round of online training. Alternatively, if the proportion of the difference between the loss values ​​corresponding to two consecutive training sessions that falls within the threshold range exceeds a preset proportion, then this round of online training will be stopped.

7. The online model training method according to claim 1, characterized in that, The pre-built model is established through the following steps: A deep learning model is selected as the initial model, and deep learning is performed on the initial model using learning samples to obtain the base model; The base model is retrained using a semi-supervised learning method to obtain the pre-built model. The semi-supervised learning method focuses on and learns only the regions of manually labeled cell images during the retraining process.

8. The online model training method according to claim 7, characterized in that, The retraining of the base model using a semi-supervised learning method to obtain the pre-built model includes: Obtain sample images for semi-supervised learning; The sample image is input into the base model to obtain the intermediate image; Identify regions in the intermediate image that contain defective features, including one or more of the following: cell adhesion, at least partial cell disappearance, and cell aberration; The operation of manually marking at least one area with defective features; The current model is retrained based on the manually labeled regions. During the retraining process, only the manually labeled regions are studied and learned to obtain the retrained model. Based on the retrained model, the sample image is identified to obtain a new intermediate image; manual annotation is performed on at least one region in the new intermediate image that has defect features, and the retrained model is obtained. The process continues until a preset number of repetitions is reached, or until no defective features are found in the intermediate images, at which point the current model is used as the pre-built model.

9. An online training device for a cell recognition model, characterized in that, Used for online training to obtain a cell recognition model; one round of online training includes multiple training sessions, each with a corresponding loss value; the device includes the following modules: A pre-built model creation module is configured to create pre-built models for preliminary cell identification. The pre-built model module is configured to acquire a pre-built model; the cell image to be identified is input into the pre-built model; if the output image is not satisfactory, the output image is manually annotated to correct and supplement the recognition result of the pre-built model; the manually annotated image is used as a learning sample for online training. The loss difference module is configured to calculate the loss difference based on the loss values ​​corresponding to multiple training iterations. A threshold module is configured to determine a threshold range according to a preset rule, wherein the preset rule calculates the product of the loss value corresponding to the first training in the online training and a preset ratio value as the upper limit of the threshold range, wherein the preset ratio value is a value that changes as the number of training sessions in this round of online training increases. A stop module is configured to compare the loss difference with the threshold range. If the loss difference is within the threshold range, or the current training iterations reach a preset upper limit for the number of training iterations, then the current round of online training is stopped to obtain a cell recognition model for identifying cells. The condition for stopping the current round of online training requires that the loss value has a decreasing trend, but does not require that the difference between adjacent loss values ​​has a decreasing trend. If the difference between the loss value corresponding to the previous training iteration and the loss value corresponding to the next training iteration is less than 0, then the condition for stopping the current round of online training is not met. If the output image of the cell recognition model is still unsatisfactory, a new round of online training will be started for the current cell recognition model until the final cell recognition model can output a satisfactory image.

10. An electronic device comprising a processor and a memory, wherein, The memory is used to store program instructions, and the processor is configured to run the program instructions, characterized in that the program instructions are run to perform the steps of the method as claimed in any one of claims 1 to 8.

11. A computer-readable storage medium for storing program instructions, characterized in that, The program instructions are configured to be invoked to perform the steps of the method as described in any one of claims 1 to 8.

12. A computer program product comprising a readable and storable computer program, said computer program including program instructions, characterized in that, When the program instructions are run on a computer device, the computer device performs the steps of the method as described in any one of claims 1 to 8.

Images