Physiological image processing method, model training method, device, equipment and medium

Abstract

The present application discloses a physiological image processing method, model training method, device, equipment and medium, belonging to the field of artificial intelligence technology. The method includes: obtaining a physiological image that has been stained; determining the position information of at least one lesion object in the physiological image; performing color decomposition processing on the physiological image to obtain the staining intensity information of the physiological image; and statistically obtaining the staining count result of the lesion object based on the position information and staining intensity information of the lesion object. The present application obtains the staining count result of the lesion object through the position information and staining intensity information of the lesion object, thereby realizing the rapid statistics of the staining count result of the lesion object in the physiological image; improving the generalization ability of the physiological image processing method, and can obtain good counting results for physiological images with different staining treatment methods, thereby avoiding the influence of different staining treatment methods on the physiological image and ensuring the accuracy of the staining count result of the lesion object.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a physiological image processing method, model training method, device, equipment and medium. Background Art

[0002] Physiological images are images of physiological sections after staining. Different staining reagents have different staining effects on cells of different functional types.

[0003] In related technologies, in order to fully observe physiological slices, it is usually necessary to use multiple staining methods to stain the physiological slices separately; at the same time, the physiological images of physiological slices contain biological structure information, intuitively reflecting the microscopic morphology of cells, and cells are divided into diseased cells and normal cells.

[0004] However, the number of diseased cells in physiological images is often very large, and how to quickly count whether the diseased cells are stained is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a physiological image processing method, model training method, device, equipment and medium. The technical solutions are as follows:

[0006] According to one aspect of the present application, a physiological image processing method is provided, the method comprising:

[0007] acquiring the physiological image after dyeing;

[0008] determining position information of at least one pathological object in the physiological image;

[0009] performing color decomposition processing on the physiological image to obtain staining intensity information of the physiological image;

[0010] The staining count result of the lesion object is obtained by statistics according to the position information of the lesion object and the staining intensity information.

[0011] According to another aspect of the present application, a method for training a physiological image processing model is provided, wherein the physiological image processing model includes a detection network, and the method includes:

[0012] Acquiring a sample physiological image and marking information of the sample physiological image;

[0013] Calling the detection network in the physiological image processing model to perform detection processing on the sample physiological image to obtain a predicted detection result of the sample physiological image;

[0014] The physiological image processing model is trained according to the error between the predicted detection result and the label information to obtain a trained physiological image processing model.

[0015] According to another aspect of the present application, a physiological image processing device is provided, the device comprising:

[0016] an acquisition module, configured to acquire the physiological image after dyeing;

[0017] a determination module, configured to determine position information of at least one lesion object in the physiological image;

[0018] The determination module is further configured to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image;

[0019] The statistical module is used to obtain the staining counting result of the lesion object according to the position information of the lesion object and the staining intensity information.

[0020] In an optional design of the present application, the apparatus is executed based on a physiological image processing model, wherein the physiological image processing model includes a detection network and a decomposition network;

[0021] The determining module is further configured to:

[0022] calling the detection network to perform detection processing on the physiological image to obtain position information of at least one of the lesion objects in the physiological image;

[0023] The decomposition network is called to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image.

[0024] In an optional design of the present application, the detection network includes a position detection subnetwork and a region segmentation subnetwork;

[0025] The determining module is further configured to:

[0026] calling the position detection subnetwork to perform detection processing on the physiological image to obtain position information of the physiological object in the physiological image;

[0027] Calling the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain a lesion region in the physiological image, where the lesion region includes at least one lesion object;

[0028] According to the lesion area and the position information of the physiological object, the physiological object belonging to the lesion area is determined as the lesion object, and the position information of the lesion object is determined.

[0029] In an optional design of the present application, the determining module is further configured to:

[0030] calling the position detection subnetwork to perform detection processing on the physiological image to obtain a physiological object detection result of the physiological image;

[0031] Searching for a region of interest in the physiological object detection result, wherein a center point of the region of interest and an edge point of the region of interest show an increasing or decreasing trend;

[0032] The position corresponding to the maximum value of the region of interest in the physiological image is determined as the position information of the physiological object.

[0033] In an optional design of the present application, the determining module is further configured to:

[0034] Calling the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain a segmentation result of the physiological image, wherein the segmentation result is used to indicate probability information that multiple pixels of the physiological image belong to the lesion area;

[0035] The segmentation result of the physiological image is binarized to obtain the lesion area in the physiological image.

[0036] In an optional design of the present application, the determining module is further configured to:

[0037] Calling the decomposition network to perform color decomposition processing on the physiological image to obtain color information of the physiological image in at least two color channels;

[0038] The color information of the physiological image in the first color channel is determined as the staining intensity information.

[0039] In an optional design of the present application, the staining counting result includes staining counting information;

[0040] The statistics module is also used to:

[0041] Obtaining a first counting result of the lesion objects belonging to a first staining state according to the position information of the lesion objects and the staining intensity information, wherein the staining intensity information is used to indicate at least two staining states of the physiological object in the physiological image;

[0042] obtaining a second counting result of the lesion object according to the position information of the lesion object;

[0043] The ratio of the first counting result to the second counting result is determined as the staining counting information of the lesion object.

[0044] In an optional design of the present application, the statistical module is further configured to:

[0045] determining a region of the physiological image where the staining intensity information exceeds a staining threshold as a first staining region, wherein the first staining region corresponds to the physiological object in a first staining state;

[0046] determining the lesion object belonging to a first stained state according to the first stained area and the position information of the lesion object;

[0047] A first counting result of the lesion object belonging to the first staining state is obtained by statistics.

[0048] In an optional design of the present application, the staining counting result includes a staining counting image;

[0049] The statistics module is also used to:

[0050] At least one staining state of the lesion object is marked in the physiological image according to the position information of the lesion object and the staining intensity information to obtain the staining count image.

[0051] According to another aspect of the present application, a training device for a physiological image processing model is provided, wherein the physiological image processing model includes a detection network, and the device includes:

[0052] an acquisition module, configured to acquire a sample physiological image and marking information of the sample physiological image;

[0053] A prediction module, configured to call a detection network in the physiological image processing model to perform detection processing on the sample physiological image to obtain a predicted detection result of the sample physiological image;

[0054] The training module is used to train the physiological image processing model according to the error between the predicted detection result and the marking information to obtain a trained physiological image processing model.

[0055] In an optional design of the present application, the marking information of the sample physiological image includes: position marking information and / or region marking information;

[0056] The position mark information is used to indicate the position information of the physiological object in the sample physiological image, and the region mark information is used to indicate the lesion region in the sample physiological image.

[0057] In an optional design of the present application, the detection network includes a position detection subnetwork;

[0058] The prediction module is also used to:

[0059] In a case where the tag information includes the position tag information, calling the position detection subnetwork to perform detection processing on the sample physiological image to obtain predicted position information of the physiological object in the sample physiological image;

[0060] The training module is also used to:

[0061] The physiological image processing model is trained according to the error between the predicted position information and the position mark information to obtain the trained physiological image processing model.

[0062] In an optional design of the present application, the detection network includes a region segmentation subnetwork;

[0063] The prediction module is also used to:

[0064] In a case where the label information includes the region label information, calling the region segmentation subnetwork to perform segmentation processing on the sample physiological image to obtain a predicted lesion region in the sample physiological image, where the predicted lesion region includes at least one lesion object;

[0065] The training module is also used to:

[0066] The physiological image processing model is trained according to the error between the predicted lesion area and the area marking information to obtain the trained physiological image processing model.

[0067] According to another aspect of the present application, a computer device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the physiological image processing method described above, and / or the training method of the physiological image processing model.

[0068] According to another aspect of the present application, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the physiological image processing method described above, and / or the training method of the physiological image processing model.

[0069] According to another aspect of the present application, a computer program product is provided, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor reads and executes the computer instructions from the computer-readable storage medium to implement the physiological image processing method described above, and / or the training method of the physiological image processing model.

[0070] The beneficial effects of the technical solution provided by this application include at least:

[0071] The staining counting results of the lesion objects are obtained through the location information and staining intensity information of the lesion objects, realizing the rapid statistics of the staining counting results of the lesion objects in the physiological images; the location information and staining intensity information of the lesion objects are determined by performing detection processing and color decomposition processing on the physiological images respectively, thereby improving the generalization ability of the physiological image processing method, and obtaining good counting effects for physiological images with different staining processing methods, thus avoiding the influence of different staining processing methods on the physiological images and ensuring the accuracy of the staining counting results of the lesion objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0073] Figure 1 is a schematic diagram of a computer system provided by an exemplary embodiment of the present application;

[0074] Figure 2 is a schematic diagram of a physiological image processing model provided by an exemplary embodiment of the present application;

[0075] Figure 3 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0076] Figure 4 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0077] Figure 5 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0078] Figure 6 is a schematic diagram of a physiological image and a position-marked image of a physiological object provided by an exemplary embodiment of the present application;

[0079] Figure 7 is a schematic diagram of a segmented image of a physiological image and a lesion area marking image provided by an exemplary embodiment of the present application;

[0080] Figure 8 is a schematic diagram of a position-marked image of a diseased object provided by an exemplary embodiment of the present application;

[0081] Figure 9 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0082] Figure 10 is a schematic diagram of a physiological object detection result provided by an exemplary embodiment of the present application;

[0083] Figure 11 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0084] Figure 12 is a schematic diagram of a segmentation result image of a physiological image and a lesion area image provided by an exemplary embodiment of the present application;

[0085] Figure 13 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0086] Figure 14 is a schematic diagram of staining intensity information provided by an exemplary embodiment of the present application;

[0087] Figure 15 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0088] Figure 16 is a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application;

[0089] Figure 17 is a schematic diagram of a staining counting image provided by an exemplary embodiment of the present application;

[0090] Figure 18 is a schematic diagram of a physiological image and a staining count image provided by an exemplary embodiment of the present application;

[0091] Figure 19 is a flowchart of a training method for a physiological image processing model provided by an exemplary embodiment of the present application;

[0092] Figure 20 is a flowchart of a training method for a physiological image processing model provided by an exemplary embodiment of the present application;

[0093] Figure 21 is a flowchart of a training method for a physiological image processing model provided by an exemplary embodiment of the present application;

[0094] Figure 22 is a structural block diagram of a physiological image processing device provided by an exemplary embodiment of the present application;

[0095] Figure 23is a structural block diagram of a training device for a physiological image processing model provided by an exemplary embodiment of the present application;

[0096] Figure 24 This is a structural block diagram of a server provided by an exemplary embodiment of the present application.

[0097] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application. DETAILED DESCRIPTION

[0098] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0099] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0100] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0101] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the physiological images and other information involved in this application were obtained with full authorization.

[0102] It should be understood that although the terms first, second, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter without departing from the scope of this disclosure. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0103] Figure 1 A schematic diagram of a computer system provided by one embodiment of the present application is shown. The computer system can be implemented as a system architecture for a physiological image processing method and / or a model training method. The computer system may include: a terminal 100 and a server 200. The terminal 100 may be an electronic device such as a mobile phone, a tablet computer, a vehicle-mounted terminal (vehicle computer), a wearable device, a PC (Personal Computer), an unmanned reservation terminal, etc. A client for running a target application may be installed in the terminal 100. The target application may be an application for training and / or using an event representation prediction model, or other application that provides training and / or using functions for an event representation prediction model, which is not limited in this application. In addition, the present application does not limit the form of the target application, including but not limited to an App (Application) installed in the terminal 100, a mini-program, etc., or a web page. The server 200 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The server 200 may be the background server of the above-mentioned target application, used to provide background services for the client of the target application.

[0104] In the physiological image processing method and / or model training method provided in the embodiments of the present application, the execution subject of each step may be a computer device, which refers to an electronic device with data calculation, processing and storage capabilities. Figure 1 Taking the implementation environment of the scheme shown as an example, the physiological image processing method and / or the model training method can be executed by the terminal 100 (such as the client of the target application installed and running in the terminal 100 executes the physiological image processing method and / or the model training method), or the physiological image processing method and / or the model training method can be executed by the server 200, or the terminal 100 and the server 200 can interact and cooperate to execute them, and this application does not limit this.

[0105] Furthermore, the technical solution of this application can be combined with blockchain technology. For example, some of the data involved in the physiological image processing method and / or model training method disclosed in this application (physiological images, sample physiological images, and other data) can be stored on a blockchain. The terminal 100 and the server 200 can communicate via a network, such as a wired or wireless network.

[0106] Next, the physiological image processing in this application is introduced:

[0107] Figure 2 A schematic diagram of a physiological image processing model provided by an embodiment of the present application is shown.

[0108] The physiological image processing model 320 is a trained network model; the physiological image processing model 320 includes: a detection network 322 and a decomposition network 324;

[0109] A physiological image 310 is acquired. The physiological image 310 is a physiological image that has been stained. For example, the physiological image 310 is a physiological image obtained by observing a breast tissue section, and the breast tissue section is stained with nuclear antigen proliferation index (Ki67).

[0110] Calling the detection network 322 to perform detection processing on the physiological image; illustratively, the detection network 322 includes: a position detection subnetwork 322a and a region segmentation subnetwork 322b;

[0111] Specifically, the position detection subnetwork 322a is called to perform detection processing on the physiological image 310 to obtain the position information of the cells in the physiological image 310. The cell position map 310a shows the center point position of the cells in the physiological image 310. For example, in this embodiment, the physiological objects in the physiological image are cells.

[0112] Region segmentation subnetwork 322b is called to segment physiological image 310 to obtain a lesion region in the physiological image. Lesion region map 310b illustrates the lesion region in physiological image 310. For example, lesion region map 310b is a binary image. The white area in lesion region map 310b represents the lesion region, which includes at least one lesion cell. Exemplarily, a lesion cell is a cell in the physiological image that has undergone a lesion. Specifically, the lesion cell is a cancerous cell, also known as a cancer cell. In one example, the cause of the lesion in the cancerous cell is abnormal cell proliferation.

[0113] The position information of at least one diseased cell is determined based on the position information of the lesion area and the cells; the lesion position map 310d shows the center point position of the diseased cell in the lesion area.

[0114] Decomposition network 324 is called to perform color decomposition processing on physiological image 310 to obtain staining intensity information of the physiological image. Exemplarily, the staining intensity information indicates two staining states of cells in physiological image 310: a stained state and an unstained state. Stained cell image 310c shows an image of cells in the stained state. Exemplarily, cells in the stained state are brown, and cells in the unstained state are blue.

[0115] According to the center point position and staining intensity information of the diseased cells, the counting results of the diseased cells in the stained state are obtained. The stained diseased cell image 310e shows the diseased cells in the stained state and also shows the envelope of the lesion area.

[0116] Next, the physiological image processing method will be introduced through the following embodiments.

[0117] Figure 3 A flowchart of a physiological image processing method provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device. The method includes:

[0118] Step 510: Acquire a physiological image after dyeing;

[0119] For example, physiological images contain the microscopic morphology of physiological objects. The source of physiological images can be images of physiological sections or images obtained by direct observation of physiological tissue. This application does not impose any restrictions on the method of acquiring physiological images. Physiological objects include, but are not limited to, at least one of the following: cells, tissues, organelles, ribosomes, proteins, and antigen receptors.

[0120] Exemplarily, the physiological image is a physiological image that has been stained. The staining process can be performed directly on the physiological slice, or the staining process can be performed on the physiological image obtained by observing the physiological slice. In one implementation, the physiological slice is stained, and different staining methods have different staining effects on the object.

[0121] For example, a physiological section of breast tissue is used for staining. Immunohistochemistry (IHC) staining is used to stain the physiological section. Specifically, the nuclear staining method in the IHC staining method includes, but is not limited to, at least one of the following: estrogen receptor (ER) staining, progesterone receptor (PR) staining, nuclear antigen proliferation index (Ki67) staining, and human epidermal growth factor receptor 2 (HER2) staining. Exemplarily, the staining method also includes a membrane staining method.

[0122] Step 520: Determine location information of at least one lesion object in the physiological image;

[0123] For example, a pathological object is a physiological object that has undergone a lesion in a physiological image, and the microstructure of the pathological object is different from the microstructure of a non-lesional physiological object. The cause of the pathological object may be damage to the physiological object or abnormal proliferation of the physiological object. This application does not impose any restrictive provisions on the cause of the pathological object.

[0124] For example, the location information of the lesion object can be indicated by a location information table, or the location information of the lesion object can be indicated by marking in the physiological image; this application does not impose any restrictions on the display method of the location information of the lesion object. Similarly, the location information of the lesion object can determine the location information of multiple lesions one by one, or determine the location area where multiple lesions are located.

[0125] For example, the location information of the lesion object can be determined by statistical processing, or it can be determined by prediction using a neural network model; this application does not make any restrictive provisions on the method of determining the location information of the lesion object.

[0126] Step 530: performing color decomposition processing on the physiological image to obtain staining intensity information of the physiological image;

[0127] Exemplarily, the staining intensity information is used to indicate the staining intensity of a stained physiological image, indicating the staining intensity of at least one pixel in the physiological image that has been stained.

[0128] For example, the staining intensity information of the physiological image can be determined by statistical processing, or by prediction of a neural network model; this application does not make any restrictive provisions on the method of determining the staining intensity information of the physiological image.

[0129] For example, different staining methods generally have different staining effects on physiological images, and physiological images stained with different staining methods generally correspond to different staining intensity information.

[0130] Step 540: Obtain the staining count result of the lesion object based on the position information and staining intensity information of the lesion object.

[0131] The staining count result of the lesion object is used to indicate the count result of the lesion object after the staining process. For example, the staining count result of the lesion object can be indicated by staining count information, or by marking the staining count result of the lesion object in the physiological image. This application does not impose any restrictions on the presentation of the staining count result.

[0132] In summary, the method provided in this embodiment obtains the staining counting result of the lesion object through the position information and staining intensity information of the lesion object, thereby realizing the rapid statistical counting of the staining counting result of the lesion object in the physiological image; by performing detection processing and color decomposition processing on the physiological image respectively, the position information and staining intensity information of the lesion object are determined, thereby improving the generalization ability of the physiological image processing method, and obtaining good counting results for physiological images with different staining processing methods, thereby avoiding the influence of different staining processing methods on the physiological image, and ensuring the accuracy of the staining counting result of the lesion object.

[0133] Figure 4 FIG1 shows a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 3 In the illustrated embodiment, step 520 may be implemented as step 520a, and step 530 may be implemented as step 530a:

[0134] Step 520a: calling a detection network to perform detection processing on the physiological image to obtain location information of at least one lesion object in the physiological image;

[0135] Exemplarily, the physiological image processing method is performed based on a physiological image processing model, which includes a detection network and a decomposition network.

[0136] The detection network is called to detect and process the physiological image, and the detection network includes but is not limited to at least one of the following: Convolutional Neural Network (CNN), Long Short-Term Memory Network (LSTM), Recurrent Neural Network (RNN), Fully Convolution Networks (FCN), U-Net, Segmentation Convolutional Neural Network (SegNet), and LinkNet.

[0137] Step 530a: calling the decomposition network to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image;

[0138] Exemplarily, the color information carried in the physiological image is decomposed to obtain the staining intensity information of the physiological image; exemplary, the decomposition network is called to perform color decomposition processing on the physiological image according to the color system.

[0139] In summary, the method provided in this embodiment executes a physiological image processing method based on a physiological image processing model, performs detection processing and color decomposition processing on physiological images respectively, determines the position information and staining intensity information of the lesion object, improves the generalization ability of the physiological image processing method, and can obtain good counting effects for physiological images with different staining processing methods, avoiding the influence of different staining processing methods on physiological images, and ensuring the accuracy of the staining counting results of the lesion object.

[0140] Figure 5 FIG1 shows a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 3 In the illustrated embodiment, step 520a may be implemented as steps 522, 524, and 526:

[0141] Step 522: calling the position detection subnetwork to perform detection processing on the physiological image to obtain the position information of the physiological object in the physiological image;

[0142] The detection network includes a position detection subnetwork and a region segmentation subnetwork; wherein the position detection subnetwork includes but is not limited to at least one of the following: CNN, LSTM, RNN, FCN, U-Net, SegNet, LinkNet.

[0143] Exemplarily, the physiological image includes at least one physiological object, which includes a pathological object having a lesion, and may include other physiological objects other than the pathological object, such as a non-lesional object. This application does not impose any restrictions on the specific type of the physiological object. In one example, the physiological object includes at least one pathological object.

[0144] For example, the position information of the physiological object may be indicated by a position information table, or may be marked in the physiological image to indicate the position information of the physiological object. Figure 6 Schematic diagram showing a physiological image and an image of a position mark of a physiological object. Figure 6 In the embodiment, a physiological object position mark image 414 is obtained by marking the position information of the physiological object in the physiological image 412. For example, a mark point 414a in the physiological object position mark image 414 indicates the position of the physiological object; and the physiological object position mark image 414 may contain multiple mark points respectively indicating the positions of multiple physiological objects.

[0145] Step 524: calling the region segmentation sub-network to perform segmentation processing on the physiological image to obtain a lesion region in the physiological image, where the lesion region includes at least one lesion object;

[0146] Exemplarily, the region segmentation subnetwork includes but is not limited to at least one of the following: CNN, LSTM, RNN, FCN, U-Net, SegNet, LinkNet.

[0147] Exemplarily, the lesion area is used to indicate the position of at least one lesion object; the lesion area can be indicated by marking an envelope of the lesion area in a physiological image, or by a segmentation result of the physiological image.

[0148] Figure 7 The figure shows the segmentation image of the physiological image and the image of the lesion area marking. Figure 7 In the figure, segmented image 422 of the physiological image indicates the lesion area based on the segmentation result of the physiological image. First region 422a in segmented image 422 of the physiological image is the lesion area. First region 422a is white. For example, first region 422a is not a connected region, and all white regions in segmented image 422 of the physiological image are first region 422a. Lesion area marking image 424 is generated by marking an envelope line 424a of the lesion area in the physiological image. The area enclosed by envelope line 424a of the lesion area is the lesion area.

[0149] Step 526: According to the position information of the lesion area and the physiological object, the physiological object in the lesion area is determined as the lesion object, and the position information of the lesion object is determined.

[0150] Exemplarily, a physiological object in a lesion area is determined as a lesion object, and the position information of the lesion object is determined based on the position information of the physiological object. Figure 8 The schematic diagram of the position mark image of the lesion object is shown. The mark point 432b in the position mark image 432 of the lesion object indicates the position of the lesion object; the position mark image 432 of the lesion object contains multiple mark points respectively indicating the positions of multiple lesions.

[0151] The position marking image 432 of the lesion object also includes an envelope 432 a of the lesion area.

[0152] In summary, the method provided in this embodiment executes a physiological image processing method based on a physiological image processing model, and the detection network includes a position detection subnetwork and a region segmentation subnetwork; the position detection subnetwork and the region segmentation subnetwork are called to perform detection processing and segmentation processing on the physiological image respectively, and the position information and the lesion area of ​​the physiological object are determined respectively, thereby improving the generalization ability of the physiological image processing method, and obtaining good counting effects for physiological images with different staining processing methods, thereby avoiding the influence of different staining processing methods on the physiological image, and ensuring the accuracy of the staining counting results of the lesion object.

[0153] Figure 9 FIG1 shows a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 5 In the illustrated embodiment, step 522 may be implemented as steps 522a, 522b, and 522c:

[0154] Step 522a: calling the position detection sub-network to perform detection processing on the physiological image to obtain a physiological object detection result of the physiological image;

[0155] Exemplarily, the detection result of the physiological object is used to indicate the probability information that the pixel point in the physiological image belongs to the physiological object; exemplary, the detection result of the physiological object corresponding to the pixel point and the probability that the pixel point belongs to the physiological object are correlated, and the correlation can be positive or negative.

[0156] Step 522b: searching for a region of interest in the physiological object detection result, where the distance from the center point of the region of interest to the edge point of the region of interest presents an increasing or decreasing trend;

[0157] Exemplarily, the shape of the region of interest may be rectangular, circular, elliptical, or other irregular shapes; the changing trend from the center point of the region of interest to the edge point of the region of interest may be a changing trend in one direction or a changing trend in multiple directions. Figure 10 Schematic diagram showing physiological object detection results. Figure 10 The following example illustrates a positive correlation between the detection result of a physiological object corresponding to a pixel and the probability that the pixel belongs to the physiological object. First image 442 shows the physiological object detection result of the physiological image; to more clearly illustrate the physiological object detection result, second image 444 captures a portion of first image 442 for exemplary illustration; illustratively, second image 444 is located in a selected area of ​​first image 442, and the selected area is marked with a dotted line in first image 442.

[0158] In the second image 444, four regions of interest are found and marked with dotted lines in the second image 444; the center point of the region of interest is the first pixel point, and the edge point of the region of interest is the second pixel point. The physiological object detection results from the first pixel point to the second pixel point show an increasing or decreasing trend, and exemplarily, show a decreasing trend.

[0159] Step 522c: determining the position corresponding to the maximum value of the region of interest in the physiological image as the position information of the physiological object;

[0160] Exemplarily, the position of the pixel corresponding to the maximum or minimum value of the physiological object detection result in the region of interest is determined as the position information of the physiological object. In one example, the position information of the physiological object is the center position of the physiological object.

[0161] In summary, the method provided in this embodiment utilizes the position detection subnetwork to perform detection and processing on physiological images to determine the location information of physiological objects. This method fully exploits the position detection subnetwork's ability to segment physiological images, determining the location information of physiological objects within the physiological object detection results. This detection network improves the generalization of the physiological image processing method, enabling good counting results for physiological images with different staining treatments. This mitigates the impact of different staining treatments on physiological images and ensures the accuracy of staining counting results for pathological objects.

[0162] Figure 11 FIG1 shows a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 5 In the illustrated embodiment, step 524 may be implemented as steps 524a and 524b:

[0163] Step 524a: calling the region segmentation sub-network to perform segmentation processing on the physiological image to obtain a segmentation result of the physiological image;

[0164] Exemplarily, the segmentation result is used to indicate the probability information that multiple pixels of a physiological image belong to a lesion area; exemplary, the segmentation result of a physiological image is a grayscale image, and the grayscale value corresponding to the pixel in the grayscale image is positively correlated with the probability that the pixel belongs to the lesion area.

[0165] Step 524b: binarizing the segmentation result of the physiological image to obtain the lesion area in the physiological image;

[0166] Exemplarily, the segmentation result of the physiological image is binarized to distinguish the lesion area and the non-lesion area of ​​the physiological image; exemplary, the segmentation result of the physiological image is a grayscale image, and the grayscale image is binarized to obtain the lesion area in the physiological image. Figure 12 The diagram shows a segmentation result image of a physiological image and a lesion region image. The segmentation result image 452 of the physiological image is obtained by calling the region segmentation subnetwork to segment the physiological image; the segmentation result image 452 of the physiological image is binarized to obtain the lesion region image 454.

[0167] In summary, the method provided in this embodiment utilizes a region segmentation subnetwork to segment physiological images and identify lesion regions within them. This method fully exploits the segmentation capabilities of the region segmentation subnetwork to identify lesion regions within the segmented physiological images. The detection network improves the generalization of the physiological image processing method, enabling good counting results for physiological images with different staining treatments. This mitigates the impact of different staining treatments on physiological images and ensures the accuracy of staining counts for lesions.

[0168] Figure 13 FIG1 shows a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 4 In the illustrated embodiment, step 530a may be implemented as steps 532 and 534:

[0169] Step 532: calling the decomposition network to perform color decomposition processing on the physiological image to obtain color information of the physiological image in at least two color channels;

[0170] Exemplarily, a color channel is a channel for storing color information of a physiological image. The color information of a physiological image can be the color of the physiological image, such as red, blue, brown, yellow, etc., or can be the color attributes of the physiological image, such as saturation, hue, brightness, etc. Exemplarily, in the color information of at least two color channels of the physiological image obtained by color decomposition processing, at least two color channels belong to the same color system, such as the color system is at least one of a red-green-blue (RGB) color system, a hue-saturation-value (HSV) color system, and a hematoxylin-eosin staining-diaminobezidin 3 (HED) color system (also known as a hematoxylin, eosin, 3-diaminobenzidine color system).

[0171] Step 534: determining the color information of the physiological image in the first color channel as staining intensity information;

[0172] Exemplarily, among the color information in at least two color channels of the physiological image, the color information in the first color channel is determined as the staining intensity information. Exemplarily, the color information in the DAB color channel of the physiological image is determined as the staining intensity information. Exemplarily, the staining intensity information is used to indicate the staining condition of the physiological object in the physiological image. For example, different staining methods have different staining effects on the physiological object in the physiological image; the staining intensity information is used to indicate the staining effect of the physiological object in the physiological image.

[0173] Figure 14 A schematic diagram of staining intensity information is shown. For example, first image 462 is the color information of the DAB color channel of a physiological image in the hematoxylin-eosin-DAB color system; the color information of the physiological image in the DAB color channel is determined as the staining intensity information. In an optional implementation, the staining intensity information is binarized to obtain a second image 464, which is the processed staining intensity information.

[0174] In summary, the method provided in this embodiment implements a physiological image processing method based on a physiological image processing model, invokes a decomposition network to perform color decomposition processing on physiological images, and determines the staining intensity information of lesions. This method fully extracts the color information carried in the sample image and uses this color information to indicate the staining status of the physiological objects in the sample image. This improves the generalization capability of the physiological image processing method, enabling good counting results for physiological images with different staining treatments, thereby avoiding the impact of different staining treatments on physiological images and ensuring the accuracy of staining count results for lesions.

[0175] Figure 15 FIG1 shows a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 3 In the illustrated embodiment, step 540 may be implemented as steps 541, 542, and 543:

[0176] Step 541: Obtaining a first counting result of the lesion object belonging to a first staining state according to the position information and staining intensity information of the lesion object;

[0177] In one implementation, the stain count result includes stain count information; the stain count information is used to indicate a ratio between the number of lesion objects in the first stain state and the number of lesion objects in the physiological image. It should be noted that in another implementation, the stain count information may also be used to indicate a count result of lesion objects in the first stain state.

[0178] Staining intensity information is used to indicate the staining status of physiological objects in physiological images. For example, different staining methods may have different staining effects on physiological objects in physiological images; staining intensity information is used to indicate the staining effects of physiological objects in physiological images. Specifically, staining intensity information is used to indicate at least two staining states of physiological objects in physiological images.

[0179] In one implementation, an area of ​​the physiological image whose staining intensity information exceeds a staining threshold is determined as a first staining area, and the first staining area corresponds to a physiological object belonging to a first staining state; illustratively, the first staining state is used to indicate whether the physiological object belongs to a stained state or an unstained state.

[0180] According to the position information of the first staining area and the lesion object, the lesion object belonging to the first staining state is determined; illustratively, the younger object located in the first staining area is determined as the lesion object belonging to the first staining state.

[0181] A first counting result of the lesion objects belonging to the first staining state is obtained by counting the number of the lesion objects belonging to the first staining state as the first counting result.

[0182] Step 542: Obtain a second counting result of the lesion object according to the position information of the lesion object;

[0183] The lesion objects are statistically processed according to their position information to obtain a count result of the lesion objects.

[0184] Step 543: Determine the ratio of the first counting result to the second counting result as the staining counting information of the lesion object.

[0185] The staining count information of the lesion objects is used to indicate the ratio between the first counting result and the second counting result; illustratively, the staining count information is used to indicate the ratio of the number of lesion objects in the first staining state to the number of all lesion objects.

[0186] In summary, the method provided in this embodiment has a staining counting result including staining counting information. By counting the lesion objects, the staining counting information of the lesion objects is obtained; the staining counting results of the lesion objects in the physiological images are quickly counted; the staining count information is used to describe the staining status of the lesion objects; the generalization ability of the physiological image processing method is improved, and good counting effects can be obtained for physiological images with different staining processing methods, thereby avoiding the influence of different staining processing methods on the physiological images and ensuring the accuracy of the staining counting results of the lesion objects.

[0187] Figure 16 FIG1 shows a flow chart of a physiological image processing method provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 3 In the illustrated embodiment, step 540 may be implemented as step 544:

[0188] Step 544: Mark at least one staining state of the lesion object in the physiological image according to the position information and staining intensity information of the lesion object to obtain a staining count image;

[0189] In one implementation, the staining count result includes a staining count image; the staining count image is obtained by marking a lesion object of at least one staining state in the physiological image;

[0190] It should be noted that, in the staining count image, only lesion objects with one staining state may be marked; or lesion objects with multiple staining states may be marked. In an optional implementation, the staining count image also marks the envelope of the lesion area.

[0191] For example, Figure 17 A schematic diagram of a staining count image is shown. Exemplarily, in staining count image 472, lesions in a stained state are marked by a first marking point 472a, and lesions in an unstained state are marked by a second marking point 472b. Exemplarily, staining count image 472 also includes an envelope 472c of the lesion area and a ratio 472d of the number of lesions in a stained state to the total number of lesions. Ki-67 is used to indicate the staining method of the physiological image, and 31% is used to indicate the ratio of the number of lesions in a stained state to the total number of lesions.

[0192] For example, Figure 18 A schematic diagram of physiological images and staining count images is shown. A first physiological image 482 is a physiological image processed by PR staining, and a first staining count image 484 is a staining count image corresponding to the first physiological image 482; a second physiological image 486 is a physiological image processed by ER staining, and a second staining count image 488 is a staining count image corresponding to the second physiological image 486;

[0193] In summary, the method provided in this embodiment has a staining counting result including a staining counting image. By marking the staining status of the lesion object in the physiological image, a staining counting image of the lesion object is obtained; the staining counting result of the lesion object in the physiological image is quickly counted; the staining status of the lesion object is described using the staining counting image; the generalization ability of the physiological image processing method is improved, and good counting results can be obtained for physiological images with different staining treatment methods, thereby avoiding the influence of different staining treatment methods on the physiological image and ensuring the accuracy of the staining counting results of the lesion object.

[0194] Next, the training method of the physiological image processing model will be introduced through the following embodiments.

[0195] Figure 19 A flowchart of a method for training a physiological image processing model provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device. The method includes:

[0196] Step 610: Acquire a sample physiological image and label information of the sample physiological image;

[0197] Exemplarily, the sample physiological image is a marked physiological image; the marking information of the sample physiological image is used to indicate the marking result of the sample physiological image. It should be noted that in this embodiment, the marking information of the sample physiological image can be the location information of the lesion object in the sample physiological image, or other information related to the location information of the lesion object. This embodiment does not impose any restrictions on the specific content of the marking information. In one example, the marking information of the sample physiological image is information related to the location information of the lesion object.

[0198] In an optional implementation, the marking information of the sample physiological image includes: position marking information and / or region marking information; the position marking information is used to indicate the position information of the physiological object in the sample physiological image, and the region marking information is used to indicate the lesion area in the sample physiological image.

[0199] Step 620: calling the detection network in the physiological image processing model to perform detection processing on the sample physiological image to obtain a predicted detection result of the sample physiological image;

[0200] Exemplarily, the physiological image processing model includes a detection network configured to perform detection processing on a sample physiological image to obtain a predicted detection result of the sample physiological image. In one example, the predicted detection result of the sample physiological image is predicted location information of a lesion object.

[0201] In an optional implementation, the physiological image processing model further includes a decomposition network; the decomposition network is used to perform color decomposition processing on the sample physiological image to obtain predicted staining intensity information. It should be noted that, in one example, the decomposition network determines the predicted staining intensity information by statistical methods. There are no network parameters in the decomposition network that need to be trained; the decomposition network in this embodiment is the same as the decomposition network in the physiological image processing model obtained after training. In another optional implementation, only the detection network in the physiological image processing model is trained, and after obtaining the trained physiological image processing model, the decomposition network is added to realize the construction of the physiological image processing model in the embodiment of the physiological image processing method above.

[0202] Step 630: training the physiological image processing model according to the error between the predicted detection result and the label information to obtain a trained physiological image processing model;

[0203] Exemplarily, the prediction error is obtained by comparing the difference between the predicted detection result and the label information. The physiological image processing model is trained using the prediction error to obtain the physiological image processing model in any of the above embodiments.

[0204] Exemplarily, a backpropagation algorithm is used to update the parameters of the physiological image processing model, and multiple sets of sample physiological images and labeling information of the sample physiological images are used to compare errors multiple times and update the parameters of the physiological image processing model to improve the prediction accuracy of the physiological image processing model.

[0205] In summary, the method provided in this embodiment trains the physiological image processing model by predicting the error between the detection results and the labeling information, thereby improving the prediction accuracy of the physiological image processing model and laying the foundation for calling the physiological image processing model to perform physiological image processing. By detecting and processing the sample physiological images through the detection network, the predicted detection results of the sample physiological images are obtained, thereby improving the generalization ability of the physiological image processing model and ensuring the accuracy of the staining counting results of the diseased objects.

[0206] Figure 20 FIG1 shows a flow chart of a training method for a physiological image processing model provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 19 In the illustrated embodiment, step 620 may be implemented as step 622, and step 630 may be implemented as step 632:

[0207] Step 622: When the tag information includes position tag information, call the position detection subnetwork to perform detection processing on the sample physiological image to obtain predicted position information of the physiological object in the sample physiological image;

[0208] Exemplarily, the marking information includes position marking information, and the position marking information is used to indicate the position information of the physiological object in the sample physiological image. The detection network includes a position detection subnetwork; it should be noted that this embodiment does not impose any restrictions on whether the detection network includes other subnetworks. In one implementation, the detection network also includes a region segmentation subnetwork; the training processes of the region segmentation subnetwork and the position detection subnetwork in the detection network can be independent of each other. Exemplarily, the position marking information is obtained through labeling. In one implementation, the size of the sample physiological image is 2000*2000 pixels.

[0209] Step 632: training the physiological image processing model based on the error between the predicted position information and the position mark information to obtain a trained physiological image processing model;

[0210] Exemplarily, the initial physiological image processing model is a detection network trained on the public ImageNet dataset. The initial physiological image processing model is then trained to obtain a trained physiological image processing model. In one example, the ImageNet dataset has an image size of 512*512 pixels, a batch size of 8, a learning rate of 0.0001, and a maximum number of iterations of 200.

[0211] In summary, the method provided in this embodiment trains the physiological image processing model by predicting the error between position information and position mark information, thereby improving the prediction accuracy of the physiological image processing model, laying the foundation for calling the physiological image processing model to perform physiological image processing, and detecting and processing the sample physiological image through the detection network to obtain the predicted detection results of the sample physiological image, thereby improving the generalization ability of the physiological image processing model and ensuring the accuracy of the staining counting results of the diseased object.

[0212] Figure 21 FIG1 shows a flow chart of a training method for a physiological image processing model provided by an exemplary embodiment of the present application. The method can be executed by a computer device. Figure 19 In the illustrated embodiment, step 620 may be implemented as step 624, and step 630 may be implemented as step 634:

[0213] Step 624: When the label information includes region label information, the region segmentation sub-network is called to perform segmentation processing on the sample physiological image to obtain a predicted lesion region in the sample physiological image;

[0214] Exemplarily, the predicted lesion region includes at least one lesion object;

[0215] Exemplarily, the marking information includes region marking information, and the region marking information is used to indicate the diseased area in the sample physiological image. The detection network includes a region segmentation subnetwork; it should be noted that this embodiment does not impose any restrictions on whether the detection network includes other subnetworks. In one implementation, the detection network also includes a position detection subnetwork; the training processes of the region segmentation subnetwork and the position detection subnetwork in the detection network can be independent of each other. Exemplarily, the position marking information is obtained through labeling. In one implementation, the sample physiological image undergoes data enhancement processing, and the data enhancement processing methods include but are not limited to: at least one of random flipping, random cropping, and staining perturbation.

[0216] Step 634: training the physiological image processing model based on the error between the predicted lesion area and the area marking information to obtain a trained physiological image processing model;

[0217] Exemplarily, the initial physiological image processing model is a detection network obtained by training the public dataset Image Net dataset, and the trained physiological image processing model is obtained by training the initial physiological image processing model.

[0218] Exemplarily, the error between the predicted lesion region and the region label information includes: at least one of a mean square error (MSE) loss function, a cross entropy loss function, and a mean absolute error (MAE) loss function.

[0219] In summary, the method provided in this embodiment trains the physiological image processing model by predicting the error between the lesion area and the regional marking information, thereby improving the prediction accuracy of the physiological image processing model, laying the foundation for calling the physiological image processing model to perform physiological image processing, and detecting and processing the sample physiological image through the detection network to obtain the predicted detection results of the sample physiological image, thereby improving the generalization ability of the physiological image processing model and ensuring the accuracy of the staining counting results of the lesion object.

[0220] Those skilled in the art will appreciate that the above embodiments may be implemented independently, or the above embodiments may be freely combined to form new embodiments to implement the physiological image processing method and / or the training method of the physiological image processing model of the present application.

[0221] Figure 22 A block diagram of a physiological image processing device provided by an exemplary embodiment of the present application is shown. The device includes:

[0222] An acquisition module 810 is configured to acquire the physiological image after dyeing;

[0223] a determination module 820, configured to determine position information of at least one lesion object in the physiological image;

[0224] The determination module 820 is further configured to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image;

[0225] The statistics module 830 is configured to obtain a staining count result of the lesion object based on the position information of the lesion object and the staining intensity information.

[0226] In an optional design of the present application, the apparatus is executed based on a physiological image processing model, wherein the physiological image processing model includes a detection network and a decomposition network;

[0227] The determining module 820 is further configured to:

[0228] calling the detection network to perform detection processing on the physiological image to obtain position information of at least one of the lesion objects in the physiological image;

[0229] The decomposition network is called to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image.

[0230] In an optional design of the present application, the detection network includes a position detection subnetwork and a region segmentation subnetwork;

[0231] The determining module 820 is further configured to:

[0232] calling the position detection subnetwork to perform detection processing on the physiological image to obtain position information of the physiological object in the physiological image;

[0233] Calling the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain a lesion region in the physiological image, where the lesion region includes at least one lesion object;

[0234] According to the lesion area and the position information of the physiological object, the physiological object belonging to the lesion area is determined as the lesion object, and the position information of the lesion object is determined.

[0235] In an optional design of the present application, the determining module 820 is further configured to:

[0236] calling the position detection subnetwork to perform detection processing on the physiological image to obtain a physiological object detection result of the physiological image;

[0237] Searching for a region of interest in the physiological object detection result, wherein a center point of the region of interest and an edge point of the region of interest show an increasing or decreasing trend;

[0238] The position corresponding to the maximum value of the region of interest in the physiological image is determined as the position information of the physiological object.

[0239] In an optional design of the present application, the determining module 820 is further configured to:

[0240] Calling the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain a segmentation result of the physiological image, wherein the segmentation result is used to indicate probability information that multiple pixels of the physiological image belong to the lesion area;

[0241] The segmentation result of the physiological image is binarized to obtain the lesion area in the physiological image.

[0242] In an optional design of the present application, the determining module 820 is further configured to:

[0243] Calling the decomposition network to perform color decomposition processing on the physiological image to obtain color information of the physiological image in at least two color channels;

[0244] The color information of the physiological image in the first color channel is determined as the staining intensity information.

[0245] In an optional design of the present application, the staining counting result includes staining counting information;

[0246] The statistics module 830 is further configured to:

[0247] Obtaining a first counting result of the lesion objects belonging to a first staining state according to the position information of the lesion objects and the staining intensity information, wherein the staining intensity information is used to indicate at least two staining states of the physiological object in the physiological image;

[0248] obtaining a second counting result of the lesion object according to the position information of the lesion object;

[0249] The ratio of the first counting result to the second counting result is determined as the staining counting information of the lesion object.

[0250] In an optional design of the present application, the statistics module 830 is further configured to:

[0251] determining a region of the physiological image where the staining intensity information exceeds a staining threshold as a first staining region, wherein the first staining region corresponds to the physiological object in a first staining state;

[0252] determining the lesion object belonging to a first stained state according to the first stained area and the position information of the lesion object;

[0253] A first counting result of the lesion object belonging to the first staining state is obtained by statistics.

[0254] In an optional design of the present application, the staining counting result includes a staining counting image;

[0255] The statistics module 830 is further configured to:

[0256] At least one staining state of the lesion object is marked in the physiological image according to the position information of the lesion object and the staining intensity information to obtain the staining count image.

[0257] Figure 23 A block diagram of a training device for a physiological image processing model provided by an exemplary embodiment of the present application is shown. The device includes:

[0258] An acquisition module 840 is configured to acquire a sample physiological image and marking information of the sample physiological image;

[0259] The prediction module 850 is configured to call the detection network in the physiological image processing model to perform detection processing on the sample physiological image to obtain a predicted detection result of the sample physiological image;

[0260] The training module 860 is used to train the physiological image processing model according to the error between the predicted detection result and the label information to obtain a trained physiological image processing model.

[0261] In an optional design of the present application, the marking information of the sample physiological image includes: position marking information and / or region marking information;

[0262] The position mark information is used to indicate the position information of the physiological object in the sample physiological image, and the region mark information is used to indicate the lesion region in the sample physiological image.

[0263] In an optional design of the present application, the detection network includes a position detection subnetwork;

[0264] The prediction module 850 is further configured to:

[0265] In a case where the tag information includes the position tag information, calling the position detection subnetwork to perform detection processing on the sample physiological image to obtain predicted position information of the physiological object in the sample physiological image;

[0266] The training module 860 is further configured to:

[0267] The physiological image processing model is trained according to the error between the predicted position information and the position mark information to obtain the trained physiological image processing model.

[0268] In an optional design of the present application, the detection network includes a region segmentation subnetwork;

[0269] The prediction module 850 is used to:

[0270] In a case where the label information includes the region label information, calling the region segmentation subnetwork to perform segmentation processing on the sample physiological image to obtain a predicted lesion region in the sample physiological image, where the predicted lesion region includes at least one lesion object;

[0271] The training module 860 is further configured to:

[0272] The physiological image processing model is trained according to the error between the predicted lesion area and the area marking information to obtain the trained physiological image processing model.

[0273] It should be noted that the device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example to implement its functions. In actual applications, the above-mentioned functions can be assigned to different functional modules according to actual needs, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0274] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method; the technical effects achieved by each module performing operations are the same as the technical effects in the embodiment of the method, and will not be elaborated here.

[0275] An embodiment of the present application also provides a computer device, which includes: a processor and a memory, wherein a computer program is stored in the memory; the processor is used to execute the computer program in the memory to implement the physiological image processing method provided by the above-mentioned method embodiments, and / or the training method of the physiological image processing model.

[0276] Optionally, the computer device is a server. For example, Figure 24 This is a structural block diagram of a server provided by an exemplary embodiment of the present application.

[0277] Typically, the server 2300 includes a processor 2301 and a memory 2302 .

[0278] The processor 2301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 2301 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 2301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 2301 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 2301 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0279] The memory 2302 may include one or more computer-readable storage media, which may be non-transitory. The memory 2302 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 2302 is used to store at least one instruction, which is used to be executed by the processor 2301 to implement the physiological image processing method provided in the method embodiment of the present application, and / or the training method of the physiological image processing model.

[0280] In some embodiments, the server 2300 may further optionally include an input interface 2303 and an output interface 2304. The processor 2301, the memory 2302, and the input interface 2303 and the output interface 2304 may be connected via a bus or signal lines. Each peripheral device may be connected to the input interface 2303 and the output interface 2304 via a bus, a signal line, or a circuit board. The input interface 2303 and the output interface 2304 may be used to connect at least one peripheral device related to input / output (I / O) to the processor 2301 and the memory 2302. In some embodiments, the processor 2301, the memory 2302, and the input interface 2303 and the output interface 2304 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 2301, the memory 2302, the input interface 2303, and the output interface 2304 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0281] Those skilled in the art will understand that the structure shown above does not constitute a limitation on the server 2300, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0282] In an exemplary embodiment, a chip is also provided, which includes a programmable logic circuit and / or program instructions. When the chip is run on a computer device, it is used to implement the physiological image processing method described in the above aspects and / or the training method of the physiological image processing model.

[0283] In an exemplary embodiment, a computer program product is also provided. The computer program product includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to implement the physiological image processing method and / or the physiological image processing model training method provided in each of the above method embodiments.

[0284] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores a computer program. The computer program is loaded and executed by a processor to implement the physiological image processing method provided by the above-mentioned method embodiments, and / or the training method of the physiological image processing model.

[0285] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0286] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0287] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A physiological image processing method, characterized in that: The method is performed based on a physiological image processing model, wherein the physiological image processing model includes a detection network and a decomposition network; the detection network includes a position detection subnetwork and a region segmentation subnetwork; the method includes: acquiring the physiological image after dyeing; calling the position detection subnetwork to perform detection processing on the physiological image to obtain position information of the physiological object in the physiological image; Calling the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain a lesion region in the physiological image, wherein the lesion region includes at least one lesion object; According to the position information of the lesion area and the physiological object, determining the physiological object belonging to the lesion area as the lesion object, and determining the position information of the lesion object; Calling the decomposition network to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image; The staining count result of the lesion object is obtained by statistics according to the position information of the lesion object and the staining intensity information.

2. The method according to claim 1, characterized in that The calling of the position detection subnetwork to perform detection processing on the physiological image to obtain position information of the physiological object in the physiological image includes: calling the position detection subnetwork to perform detection processing on the physiological image to obtain a physiological object detection result of the physiological image; Searching for a region of interest in the physiological object detection result, wherein a center point of the region of interest and an edge point of the region of interest show an increasing or decreasing trend; The position corresponding to the maximum value of the region of interest in the physiological image is determined as the position information of the physiological object.

3. The method according to claim 1, characterized in that The calling of the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain the lesion region in the physiological image includes: Calling the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain a segmentation result of the physiological image, wherein the segmentation result is used to indicate probability information that multiple pixels of the physiological image belong to the lesion area; The segmentation result of the physiological image is binarized to obtain the lesion area in the physiological image.

4. The method according to claim 1, wherein The calling of the decomposition network to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image includes: Calling the decomposition network to perform color decomposition processing on the physiological image to obtain color information of the physiological image in at least two color channels; The color information of the physiological image in the first color channel is determined as the staining intensity information.

5. The method according to any one of claims 1 to 4, characterized in that: The staining counting result includes staining counting information; The step of obtaining a staining count result of the lesion object based on the position information of the lesion object and the staining intensity information includes: Obtaining a first counting result of the lesion objects belonging to a first staining state according to the position information of the lesion objects and the staining intensity information, wherein the staining intensity information is used to indicate at least two staining states of the physiological object in the physiological image; obtaining a second counting result of the lesion object according to the position information of the lesion object; The ratio of the first counting result to the second counting result is determined as the staining counting information of the lesion object.

6. The method according to claim 5, characterized in that Obtaining a first counting result of the lesion object belonging to a first staining state according to the position information of the lesion object and the staining intensity information includes: determining a region of the physiological image where the staining intensity information exceeds a staining threshold as a first staining region, wherein the first staining region corresponds to the physiological object in a first staining state; determining the lesion object belonging to a first stained state according to the first stained area and the position information of the lesion object; A first counting result of the lesion object belonging to the first staining state is obtained by statistics.

7. The method according to any one of claims 1 to 4, characterized in that: The staining counting result includes a staining counting image; The step of obtaining a staining count result of the lesion object based on the position information of the lesion object and the staining intensity information includes: At least one staining state of the lesion object is marked in the physiological image according to the position information of the lesion object and the staining intensity information to obtain the staining count image.

8. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquiring a sample physiological image and marking information of the sample physiological image; Calling a detection network in a physiological image processing model to perform detection processing on the sample physiological image to obtain a predicted detection result of the sample physiological image; Training the physiological image processing model according to an error between the predicted detection result and the label information to obtain a trained physiological image processing model; The staining intensity information of the physiological image is predicted by the trained physiological image processing model.

9. The method according to claim 8, characterized in that The marking information of the sample physiological image includes: position marking information and / or region marking information; The position mark information is used to indicate the position information of the physiological object in the sample physiological image, and the region mark information is used to indicate the lesion region in the sample physiological image.

10. The method according to claim 9, characterized in that The detection network includes a position detection subnetwork; The calling of the detection network in the physiological image processing model to perform detection processing on the sample physiological image to obtain a predicted detection result of the sample physiological image includes: In a case where the tag information includes the position tag information, calling the position detection subnetwork to perform detection processing on the sample physiological image to obtain predicted position information of the physiological object in the sample physiological image; The step of training the physiological image processing model according to the error between the predicted detection result and the label information to obtain a trained physiological image processing model comprises: The physiological image processing model is trained according to the error between the predicted position information and the position mark information to obtain the trained physiological image processing model.

11. The method according to claim 9, characterized in that The detection network includes a region segmentation subnetwork; The calling of the detection network in the physiological image processing model to perform detection processing on the sample physiological image to obtain a predicted detection result of the sample physiological image includes: In a case where the label information includes the region label information, calling the region segmentation subnetwork to perform segmentation processing on the sample physiological image to obtain a predicted lesion region in the sample physiological image, where the predicted lesion region includes at least one lesion object; The step of training the physiological image processing model according to the error between the predicted detection result and the label information to obtain a trained physiological image processing model comprises: The physiological image processing model is trained according to the error between the predicted lesion area and the area marking information to obtain the trained physiological image processing model.

12. A physiological image processing device, characterized in that: The device is executed based on a physiological image processing model, wherein the physiological image processing model includes a detection network and a decomposition network; the detection network includes a position detection subnetwork and a region segmentation subnetwork; the device includes: an acquisition module, configured to acquire the physiological image after dyeing; a determination module, configured to call the position detection subnetwork to perform detection processing on the physiological image to obtain position information of the physiological object in the physiological image; Calling the region segmentation subnetwork to perform segmentation processing on the physiological image to obtain a lesion region in the physiological image, wherein the lesion region includes at least one lesion object; According to the position information of the lesion area and the physiological object, determining the physiological object belonging to the lesion area as the lesion object, and determining the position information of the lesion object; The determination module is further configured to call the decomposition network to perform color decomposition processing on the physiological image to obtain staining intensity information of the physiological image; The statistical module is used to obtain the staining counting result of the lesion object according to the position information of the lesion object and the staining intensity information.

13. A computer device, characterized in that: The computer device includes: a processor and a memory, wherein the memory stores at least one program; the processor is configured to execute the at least one program in the memory to implement the physiological image processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The readable storage medium stores executable instructions, which are loaded and executed by a processor to implement the physiological image processing method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor reads and executes the computer instructions from the computer-readable storage medium to implement the physiological image processing method according to any one of claims 1 to 11.

Images