Cell counting method, device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of image processing, and discloses a cell counting method, device and equipment, a storage medium and a program product, the method comprises the following steps: acquiring cell images and cell marking information corresponding to the cell images one by one; inputting the cell image into a pre-constructed cell prediction model to obtain cell prediction point information corresponding to the cell image; the cell prediction point information comprises cell prediction point position information and confidence values in one-to-one correspondence with cell prediction points; based on the cell marking information and the cell prediction point information, performing one-to-one matching on the reference points corresponding to the cell image and the cell prediction points to obtain a matching result; and adjusting a cell prediction model based on the matching result, wherein the cell prediction model is used for predicting the cell position and the cell number. The cell prediction model is obtained through training, counting and positioning of the cells can be achieved at the same time, and the positioning precision and counting performance of the cells are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a cell counting method, device, equipment, storage medium and program product. Background Art

[0002] Cell counting is an important issue in medical image research. Accurate cell counting can accurately detect potential cell diseases and related lesions.

[0003] At present, the existing cell counting methods mainly include cell counting methods based on density maps and cell counting methods based on pseudo-bounding box positioning. The counting method based on density maps uses density map regression to achieve cell counting, that is, pixel-level density map regression is used to directly predict the density value of each pixel, and then the total number of cells is obtained by accumulating the density value of the entire image. However, the cell counting method based on density maps cannot provide the exact location of individual cells, so subsequent cell analysis and processing tasks cannot be carried out. The cell counting method based on pseudo-bounding box positioning mainly counts by predicting the location of individual cells. This method relies on the generated pseudo-bounding box to achieve cell counting and positioning. When cells are highly crowded or overlapping, the use of cell counting methods based on pseudo-bounding box positioning will result in multiple cells being identified as a single cell.

[0004] In view of this, there is an urgent need for a method that can both accurately locate individual cells and accurately predict the number of cells. Summary of the invention

[0005] In view of this, the present invention provides a cell counting method, device, equipment, storage medium and program product to achieve accurate prediction of cell number while accurately locating individual cells.

[0006] In a first aspect, the present invention provides a method for counting cells, the method comprising:

[0007] Acquire cell images and cell marking information corresponding to the cell images, wherein the cell marking information includes pre-marked reference point position information and cell quantity;

[0008] Inputting the cell image into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image; the cell prediction point information includes cell prediction point position information and confidence values ​​corresponding to the cell prediction points one by one;

[0009] Based on the cell marker information and the cell prediction point information, the reference points corresponding to the cell image are matched with the cell prediction points one by one to obtain a matching result;

[0010] The cell prediction model is adjusted based on the matching results, and the cell prediction model is used to predict the cell position and the cell number.

[0011] In the present invention, a group of annotated point label images are received for training, and a group of cell prediction points are directly predicted in the cell image during the inference process, including the cell prediction point position information and the number of cells. A cell prediction model is obtained through training, which can simultaneously realize cell counting and positioning, and eliminate unnecessary intermediate representation steps, thereby improving cell positioning accuracy and counting performance.

[0012] In an optional implementation, based on the cell marker information and the cell prediction point information, the reference points corresponding to the cell image are matched one by one with the cell prediction points, including:

[0013] Based on the cell prediction point position information, the reference point position information, and the one-to-one corresponding confidence value of each cell prediction point, the offset between the cell prediction point and the reference point is calculated to obtain an offset matrix;

[0014] Based on the offset matrix, the Hungarian algorithm is used to minimize the offset between the reference point and the cell prediction point, and determine the reference point with one-to-one correspondence between the cell prediction points.

[0015] In this embodiment, the one-to-one matching method not only avoids the two unexpected situations of matching multiple reference points to the same prediction point and matching multiple prediction points with the same reference point, but also improves the matching accuracy between the cell prediction point and the reference point, making the cell prediction result more accurate.

[0016] In an optional embodiment, the offset between the cell prediction point and the reference point is calculated by the following formula:

[0017]

[0018] Where P represents the reference point, Represents the cell prediction point, p i Indicates the reference point position information corresponding to the i-th reference point, Represents the cell prediction point location information corresponding to the jth cell prediction point, represents the confidence value corresponding to the j-th cell prediction point, ||·||2 represents the distance between the ith reference point and the j-th cell prediction point, τ represents the weight, N represents the total number of reference points, and M represents the total number of cell prediction points.

[0019] In this embodiment, by calculating the offset between the cell prediction point and the reference point, the accuracy of the cell detection result can be effectively quantified to optimize the model performance. This method can not only help identify the error of the cell position, but also weight the high-quality prediction by combining the confidence value, thereby enhancing the stability and accuracy of the model in identifying high-concentration cells.

[0020] In an optional embodiment, the cell prediction model comprises:

[0021] The pre-trained convolutional neural network, feature pyramid network, and decoding network are connected in sequence;

[0022] The output end of the decoding network is also connected to the regression branch network and the classification branch network respectively.

[0023] In this embodiment, the use of a pre-trained convolutional neural network for feature extraction and the combination of the feature pyramid network and the decoding network structure can fully utilize the multi-scale features of the image, thereby improving the accuracy of cell detection, and further effectively improving the accuracy of cell counting and positioning.

[0024] In an optional embodiment, the cell image is input into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image, including:

[0025] Extracting first image features from the cell image by pre-training a convolutional neural network;

[0026] Based on the first image feature, feature upsampling and feature fusion are performed through a feature pyramid network to obtain a second image feature;

[0027] Based on the second image feature, feature decoding is performed through a decoding network to generate a feature map;

[0028] The feature map input is respectively input into the regression branch network and the classification branch network, wherein the regression branch network is used to output the location information of the cell prediction point; the classification branch network is used to output the one-to-one corresponding confidence value of each cell prediction point.

[0029] In this embodiment, by extracting image features through pre-training convolutional neural network, important information in cell images can be effectively captured; feature pyramid network performs feature upsampling and fusion, so that features of different scales can be fully utilized; the decoding network generates feature maps, which can make subsequent regression and classification results more accurate; the regression branch network and the classification branch network respectively output the cell prediction point position information and confidence value, which can ensure the accurate prediction of cell position and quantity.

[0030] In a second aspect, the present invention provides a cell counting method, the method comprising:

[0031] Acquire images of cells to be counted;

[0032] The image of cells to be counted is input into the cell prediction model in the cell counting method described in any of the above embodiments, and the number of cells in the image of cells to be counted and the corresponding cell position of each cell are output.

[0033] In a third aspect, the present invention provides a cell counting device, comprising:

[0034] An acquisition module, used to acquire cell images and cell marking information corresponding to the cell images, wherein the cell marking information includes pre-marked reference point position information and cell quantity;

[0035] A prediction module is used to input the cell image into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image; the cell prediction point information includes cell prediction point position information and confidence values ​​corresponding to the cell prediction points one by one;

[0036] A matching module is used to match the reference points corresponding to the cell image with the cell prediction points one by one based on the cell marking information and the cell prediction point information to obtain a matching result;

[0037] The optimization module is used to adjust the cell prediction model based on the matching results, and the cell prediction model is used to predict the cell position and the cell number.

[0038] In a fourth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the cell counting method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0039] In a fifth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the cell counting method of the first aspect or any corresponding embodiment thereof.

[0040] In a sixth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the cell counting method of the first aspect or any corresponding embodiment thereof.

[0041] It should be noted that the cell counting device, computer device, computer readable storage medium and computer program product provided by the present invention correspond to the above-mentioned cell counting method. Therefore, for the beneficial effects of the cell counting device, computer device, computer readable storage medium and computer program product, please refer to the description of the corresponding beneficial effects of the cell counting method above, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 is a schematic flow chart of a cell counting method according to an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of matching a reference point with a prediction point according to an embodiment of the present invention;

[0045] Figure 3 is a schematic diagram of the structure of a cell prediction model according to an embodiment of the present invention;

[0046] Figure 4 is a schematic diagram of a cell counting effect according to an embodiment of the present invention;

[0047] Figure 5 is a structural block diagram of a cell counting device according to an embodiment of the present invention;

[0048] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0050] According to an embodiment of the present invention, a cell counting method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] In this embodiment, a cell counting method is provided, which can be executed by a server, a terminal, a mobile terminal and other devices. Figure 1 is a flow chart of a cell counting method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0052] Step S101, obtaining a cell image and cell marking information corresponding to the cell image, wherein the cell marking information includes pre-marked reference point position information and cell quantity. The cell image can be acquired by a microscope or other equipment, and the reference point position information can be cell position information and cell quantity pre-marked by a user manually or automatically based on the cell image.

[0053] Step S102, inputting the cell image into the pre-built cell prediction model to obtain the cell prediction point information corresponding to the cell image; the cell prediction point information includes the cell prediction point position information and the confidence value corresponding to the cell prediction point one by one.

[0054] In this embodiment, the cell prediction model may include a pre-trained convolutional neural network, a feature pyramid network and a decoding network. Through the above networks, features can be extracted from cell images, and cell prediction point location information and confidence values ​​can be output through a regression branch network and a classification branch network.

[0055] Step S103, based on the cell marking information and the cell prediction point information, the reference points corresponding to the cell image are matched with the cell prediction points one by one to obtain a matching result.

[0056] In this embodiment, after the preliminary prediction is completed using the pre-built cell prediction model, the accuracy of the predicted prediction points is further verified based on the cell marker information and the cell prediction point information. Specifically, the Hungarian algorithm can be used for optimal one-to-one matching, the unmatched reference points are temporarily retained, and then the matched reference points and prediction points are dynamically updated, that is, the prediction points with potential and better performance will be pushed to the corresponding target, and finally gradually used as the final prediction points through one-to-one matching, avoiding the disadvantages of underestimating the count and overestimating the count, and can improve the normalized average precision index, so that it can more accurately infer the cell position and cell number.

[0057] Step S104, adjusting the cell prediction model based on the matching result, the cell prediction model is used to predict the cell position and the cell number.

[0058] The matching result includes a one-to-one matching relationship between the reference point and the cell prediction point and the number of matches. The cell prediction model is then optimized based on the matching result. Specifically, the loss between the prediction point and the reference point is calculated based on the matching result, including regression loss and classification loss. After the calculated losses are combined into a total loss, the gradient of the loss is calculated by the back propagation algorithm to update the weights and biases of the model to optimize the cell prediction model. After the model is updated, the same cell image can be used again for prediction to test the improvement effect of the model.

[0059] The loss function in this embodiment consists of two parts: classification loss and regression loss. Classification loss is used to train the classification of prediction points, while regression loss is used to guide the regression of point coordinates. The final loss function is is the classification loss With regression loss The sum of these two losses is shown in the following formula:

[0060]

[0061]

[0062]

[0063] Among them, N represents the total number of reference points, M represents the total number of cell prediction points, i and j are variables, λ1 and λ2 are weights, represents the confidence value of the cell prediction point that matches the i-th reference point, represents the position information of the cell prediction point matching the i-th reference point, p i Represents the reference point position information corresponding to the i-th reference point.

[0064] In this embodiment, by receiving a set of annotated point label images for training, and directly predicting a set of cell prediction points in the cell image during the inference process, including the cell prediction point position information and the number of cells, a cell prediction model is obtained through training, which can simultaneously realize cell counting and positioning, and eliminate unnecessary intermediate representation steps, thereby improving cell positioning accuracy and counting performance.

[0065] In some optional embodiments, based on the cell marker information and the cell prediction point information, the reference points corresponding to the cell image are matched one by one with the cell prediction points, including:

[0066] Based on the cell prediction point position information, the reference point position information, and the one-to-one corresponding confidence value of each cell prediction point, the offset between the cell prediction point and the reference point is calculated to obtain an offset matrix.

[0067] Based on the offset matrix, the Hungarian algorithm is used to minimize the offset between the reference point and the cell prediction point, and determine the reference point with one-to-one correspondence between the cell prediction points.

[0068] In this embodiment, the Hungarian algorithm can be used as a matching strategy to perform a one-to-one matching between the predicted points (red points) and the pre-marked reference points (green points). Figure 2 As shown in Figure 1, a one-to-one matching strategy is used to assign a reference point to each predicted point.

[0069] The offset matrix in this embodiment is a pairwise matching cost matrix of shape N×M, which is used to measure the distance between a pair of points. Based on the obtained offset matrix, the Hungarian algorithm is used for processing. The Hungarian algorithm is an optimization algorithm that can solve the bipartite graph matching problem in polynomial time. Through the Hungarian algorithm, the offset between all cell prediction points and reference points in the offset matrix can be minimized, thereby determining the reference point corresponding to each cell prediction point. This ensures the optimality of the matching result and makes the matching between the cell prediction point and the reference point more accurate.

[0070] In this embodiment, the one-to-one matching method not only avoids the two unexpected situations of matching multiple reference points to the same prediction point and matching multiple prediction points with the same reference point, but also improves the matching accuracy between the cell prediction point and the reference point, making the cell prediction result more accurate.

[0071] In some optional embodiments, the offset between the cell prediction point and the reference point is calculated by the following formula:

[0072]

[0073] Where P represents the reference point, Represents the cell prediction point, p i Indicates the reference point position information corresponding to the i-th reference point, Represents the cell prediction point location information corresponding to the jth cell prediction point, represents the confidence value corresponding to the j-th cell prediction point, ||·||2 represents the distance between the ith reference point and the j-th cell prediction point, τ represents the weight, N represents the total number of reference points, and M represents the total number of cell prediction points.

[0074] After obtaining the feature map through feature extraction, each pixel on the feature map should correspond to an area of ​​size s×s in the input image. In this area, a set of fixed reference points R = {R k |k∈{1,...,K}}, whose position is R k =(x k ,y k ), these reference points can be arranged in the region. Since there are K reference points at each position on the feature map, the regression branch generates a total of H×W×K prediction points. Assume that the reference point R k Its prediction point The offset between The predicted point The coordinates of are calculated as follows:

[0075]

[0076]

[0077] Here, γ is a normalization term used to scale the offset to correct for relatively small predictions.

[0078] In this embodiment, by calculating the offset between the cell prediction point and the reference point, the accuracy of the cell detection result can be effectively quantified to optimize the model performance. This method can not only help identify the error of the cell position, but also weight the high-quality prediction by combining the confidence value, thereby enhancing the stability and accuracy of the model in identifying high-concentration cells.

[0079] In some optional embodiments, the cell prediction model comprises:

[0080] The pre-trained convolutional neural network, feature pyramid network, and decoding network are connected in sequence;

[0081] The output end of the decoding network is also connected to the regression branch network and the classification branch network respectively.

[0082] Cell prediction model structure reference Figure 3 As shown in the figure, the pre-trained VGG-16 convolutional neural network can be used to extract image deep features, and then the feature upsampling and lateral connection are realized through the feature pyramid network (FPN) structure. Then, the feature map generated after feature decoding is input into the regression branch network and the classification branch network respectively, and finally the cell prediction point information is output.

[0083] In this embodiment, the use of a pre-trained convolutional neural network for feature extraction and the combination of a feature pyramid network and a decoding network structure can fully utilize the multi-scale features of the image, thereby improving the accuracy of cell detection, and further effectively improving the accuracy of cell counting and positioning.

[0084] In some optional embodiments, inputting a cell image into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image includes:

[0085] The first image features are extracted from the cell images by a pre-trained convolutional neural network.

[0086] Based on the first image features, feature upsampling and feature fusion are performed through a feature pyramid network to obtain the second image features.

[0087] Based on the second image feature, feature decoding is performed through a decoding network to generate a feature map.

[0088] The feature map input is respectively input into the regression branch network and the classification branch network, wherein the regression branch network is used to output the location information of the cell prediction point; the classification branch network is used to output the one-to-one corresponding confidence value of each cell prediction point.

[0089] In this embodiment, the pre-trained VGG-16 convolutional neural network can be used to extract image depth features, and then the feature pyramid network (FPN) structure is used to implement upsampling and lateral connection of features, and the decoding process of FPN is implemented through a decoding network, which includes three main convolutional layers and upsampling operations to fuse the detail information of the low-level (high-resolution) feature map and the semantic information of the high-level (low-resolution) feature map, and finally obtain a fine-grained feature map. The cell prediction model also includes two main parallel branches, namely a regression branch network and a classification branch network.

[0090] Specifically, the regression branch is used to predict the coordinates of the cell prediction points. It uses four convolutional layers to process the feature map output by FPN, and finally outputs the coordinates of each cell prediction point through a convolutional layer, that is, a tensor with a shape of (batch_size, num_anchors, 2), which represents the coordinates of each cell prediction point. The classification branch is used to predict the category confidence of the cell (confidence is used to determine which cell prediction points are valid and which cell prediction points can be ignored). It also uses four convolutional layers to process the feature map output by FPN, and finally uses the Softmax function to output a tensor with a shape of (batch_size, num_anchors, num_classes), which represents the category probability corresponding to each cell prediction point. The output of the entire model is a piece of information containing prediction confidence and coordinates, that is, two parallel branches are used to predict a set of cell prediction point location information and its confidence value.

[0091] In addition, in this embodiment, an evaluation standard nAP (density normalization) is defined based on the average precision (average precision is the area under the PR curve) to evaluate the positioning error and counting performance. The formula is as follows:

[0092]

[0093] in for represents the Euclidean distance, dkNN(p i ) indicates to p i The average distance of the k nearest neighbors, and the threshold δ controls the positioning accuracy. Then feature extraction is performed: downsampling feature extraction is performed on the basis of VGG16, and the first 13 convolutional layers of VGG are used to extract the deep features of the image. There are four stages in total, and the feature maps of sizes H / 2, W / 2, H / 4, W / 4, H / 8, W / 8, H / 16, W / 16 are obtained through convolution operations. These features are then upsampled by the nearest neighbor interpolation method to obtain a finer feature map.

[0094] Furthermore, Fs is used to represent the deep feature map output from the backbone network, where s is the downsampling stride, and the size of Fs is H×W. Then, based on Fs, two parallel branches (classification branch and regression branch) are used to perform point coordinate regression and prediction point classification. Both branches are composed of three stacked convolutional layers, and the ReLU activation function is introduced in the middle of the layer. For the classification branch, it outputs the confidence value after Softmax normalization to determine which prediction points are valid and which can be ignored; for the regression branch, the inherent translation invariance of the convolutional layer is used to predict the offset of the coordinates of the given reference point prediction point. The regression branch can predict a large number of prediction points, which are finally dynamically updated through one-to-one matching, and the best performing prediction point is used as the final prediction point.

[0095] In addition, when training the cell prediction model, the present invention can use a set of marked reference points as its learning target to provide the exact location of cells in the cell image and count the number. Specifically, given a cell image containing N cells, use p i =(x i ,y i ),i∈{1,...,N} represents the position of the i-th cell, that is, the reference point i is located at (x i ,y i ), the point set of all cells can be further expressed as P = {p i |i∈{1,...,N}}, N represents the total number of reference points. Further, this point set is used to predict the other two sets: and Among them, M represents the total number of predicted points, that is, the total number of predicted cells, represents the location of the prediction point, Indicates the confidence value of the predicted point. When predicting, you need to ensure The distance between and pi is as close as possible and has a high enough confidence value In addition, the predicted cell number M also needs to be close enough to the actual number N. After the prediction is completed, the predicted cell prediction point information is finally output.

[0096] In this embodiment, by extracting image features through pre-training convolutional neural network, important information in cell images can be effectively captured; feature upsampling and fusion are performed by feature pyramid network, so that features of different scales can be fully utilized; the decoding network generates feature maps, which can make subsequent regression and classification results more accurate; the regression branch network and the classification branch network respectively output the cell prediction point position information and confidence value, which can ensure the accurate prediction of cell position and quantity.

[0097] In this embodiment, a cell counting method is also provided, which can be executed by a server, a terminal, a mobile terminal and other devices. The process includes the following steps:

[0098] Step S201, obtaining an image of cells to be counted;

[0099] Step S202, input the cell image to be counted into the cell prediction model in the cell counting method described in any of the above embodiments, and output the number of cells in the cell image to be counted and the cell position corresponding to each cell. Figure 4 shown.

[0100] The further functional description of each of the above steps is the same as that of the above corresponding embodiments and will not be repeated here.

[0101] In this embodiment, point labels are directly used as learning targets. While accurately inferring the number of cells in the image, the precise position of the points can also be output to locate individual cells, which helps to achieve more comprehensive cell analysis.

[0102] In this embodiment, a cell counting device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0103] This embodiment provides a cell counting device, such as Figure 5 As shown, the device comprises:

[0104] An acquisition module 301 is used to acquire cell images and cell marking information corresponding to the cell images, where the cell marking information includes pre-marked reference point position information and the number of cells;

[0105] Prediction module 302, used to input the cell image into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image; the cell prediction point information includes cell prediction point position information and confidence values ​​corresponding to the cell prediction points one by one;

[0106] The matching module 303 is used to match the reference points corresponding to the cell image with the cell prediction points one by one based on the cell marking information and the cell prediction point information to obtain a matching result; wherein, it is used to calculate the offset between the cell prediction point and the reference point based on the cell prediction point position information, the reference point position information, and the one-to-one confidence value of each cell prediction point to obtain an offset matrix; based on the offset matrix, the Hungarian algorithm is used to minimize the offset between the reference point and the cell prediction point to determine the one-to-one reference point of the cell prediction point.

[0107] The optimization module 304 is used to adjust the cell prediction model based on the matching results, and the cell prediction model is used to predict the cell position and the cell number.

[0108] In some optional implementations, the prediction module 302 includes:

[0109] The feature extraction module is used to extract the first image feature from the cell image through a pre-trained convolutional neural network; based on the first image feature, feature upsampling and feature fusion are performed through a feature pyramid network to obtain the second image feature; based on the second image feature, feature decoding is performed through a decoding network to generate a feature map; the feature map input is respectively input into the regression branch network and the classification branch network, wherein the regression branch network is used to output the cell prediction point position information; the classification branch network is used to output the one-to-one corresponding confidence value of each cell prediction point.

[0110] The cell counting device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and a memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0111] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0112] The embodiment of the present invention also provides a computer device having the above Figure 5 The cell counting apparatus is shown.

[0113] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0114] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0115] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0116] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0117] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0118] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0119] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0120] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0121] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A cell counting method, characterized in that: The method comprises: Acquire cell images and cell marking information corresponding to the cell images, wherein the cell marking information includes pre-marked reference point position information and cell quantity; Inputting the cell image into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image; the cell prediction point information includes cell prediction point position information and confidence values ​​corresponding to the cell prediction points one by one; Based on the cell marking information and the cell prediction point information, one-to-one matching is performed between the reference point corresponding to the cell image and the cell prediction point to obtain a matching result; The cell prediction model is adjusted based on the matching result, and the cell prediction model is used to predict cell location and cell number.

2. The method according to claim 1, characterized in that The step of matching the reference points corresponding to the cell image with the cell prediction points one by one based on the cell marking information and the cell prediction point information comprises: Based on the cell prediction point position information, the reference point position information, and the one-to-one corresponding confidence value of each cell prediction point, the offset between the cell prediction point and the reference point is calculated to obtain an offset matrix; Based on the offset matrix, the Hungarian algorithm is used to minimize the offset between the reference point and the cell prediction point, and the reference point corresponding to the cell prediction point is determined.

3. The method according to claim 2, characterized in that The offset between the cell prediction point and the reference point is calculated by the following formula: Wherein, P represents the reference point, Indicates the cell prediction point, p i represents the reference point position information corresponding to the i-th reference point, represents the cell prediction point position information corresponding to the j-th cell prediction point, represents the confidence value corresponding to the j-th cell prediction point, ||·||2 represents the distance between the ith reference point and the j-th cell prediction point, τ represents the weight, N represents the total number of reference points, and M represents the total number of cell prediction points.

4. The method according to claim 1, characterized in that: The cell prediction model comprises: The pre-trained convolutional neural network, feature pyramid network, and decoding network are connected in sequence; The output end of the decoding network is also connected to the regression branch network and the classification branch network respectively.

5. The method according to claim 4, characterized in that The step of inputting the cell image into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image includes: Extracting a first image feature from the cell image by the pre-trained convolutional neural network; Based on the first image feature, performing feature upsampling and feature fusion through the feature pyramid network to obtain a second image feature; Based on the second image feature, feature decoding is performed through the decoding network to generate a feature map; The feature map input is respectively input into the regression branch network and the classification branch network, wherein the regression branch network is used to output the position information of the cell prediction point; and the classification branch network is used to output a one-to-one confidence value for each of the cell prediction points.

6. A cell counting method, characterized in that: The method comprises: Acquire images of cells to be counted; The image of cells to be counted is input into the cell prediction model in the cell counting method according to any one of claims 1 to 5, and the number of cells in the image of cells to be counted and the corresponding cell position of each cell are output.

7. A cell counting device, characterized in that: The device comprises: An acquisition module, used to acquire cell images and cell marking information corresponding to the cell images, wherein the cell marking information includes pre-marked reference point position information and cell quantity; A prediction module, used for inputting the cell image into a pre-built cell prediction model to obtain cell prediction point information corresponding to the cell image; the cell prediction point information includes cell prediction point position information and confidence values ​​corresponding to the cell prediction points one by one; A matching module, used for matching the reference points corresponding to the cell image with the cell prediction points one by one based on the cell marking information and the cell prediction point information to obtain a matching result; An optimization module is used to adjust the cell prediction model based on the matching result, and the cell prediction model is used to predict the cell position and the cell number.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the cell counting method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the cell counting method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the cell counting method according to any one of claims 1 to 6.

Images