A region of interest determination method, device, storage medium and electronic equipment

Abstract

The specification discloses a region of interest determination method, device, storage medium and electronic equipment. In the region of interest determination method provided in the specification, a medical image containing a lesion is acquired, and the medical image is displayed to a user; in response to an initial designation operation of the user on the medical image, an initial position corresponding to the initial designation operation on the medical image is determined, and an initial region of interest is determined according to the initial position; according to the medical image and the initial region of interest, a to-be-determined region of interest is output by a pre-trained prediction model; in response to a subsequent designation operation of the user on the medical image, a subsequent position corresponding to the subsequent designation operation on the medical image is determined, and a point corresponding to the subsequent position on the medical image is determined as a feature point; the to-be-determined region of interest is adjusted according to the feature point by the prediction model, and the adjusted to-be-determined region of interest is determined as a predicted region of interest.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer, and particularly relates to a region of interest determination method and device, a storage medium and an electronic device. BACKGROUND

[0002] In the medical field, determining a lesion region in a medical image, that is, determining a region of interest in the medical image is a common auxiliary medical method. Generally, due to the high complexity of diseases, it is often necessary for technical personnel with professional knowledge to manually annotate the medical image.

[0003] It is not difficult to imagine that the existing manual annotation method consumes a large amount of time and effort, and the labor cost is high; at the same time, individual differences inevitably exist between different technical personnel, and it is difficult to ensure the consistency of annotations of different technical personnel.

[0004] Therefore, how to reduce manual operation when annotating the region of interest is a problem to be solved. SUMMARY

[0005] The present specification provides a region of interest determination method and device, a storage medium and an electronic device to partially solve the above problems existing in the prior art.

[0006] The present specification adopts the following technical solutions:

[0007] The present specification provides a region of interest determination method, comprising:

[0008] Obtaining a medical image containing a lesion, and displaying the medical image to a user;

[0009] In response to an initial designation operation of the user on the medical image, determining an initial position corresponding to the initial designation operation on the medical image, and determining an initial region of interest according to the initial position;

[0010] According to the medical image and the initial region of interest, outputting a to-be-determined region of interest by a pre-trained prediction model;

[0011] In response to a subsequent designation operation of the user on the medical image, determining a subsequent position corresponding to the subsequent designation operation on the medical image, and determining a feature point corresponding to the subsequent position on the medical image as a feature point;

[0012] Adjusting the to-be-determined region of interest according to the feature point by the prediction model, and determining the adjusted to-be-determined region of interest as a predicted region of interest.

[0013] Optionally, the feature points include positive example points or negative example points, wherein the positive example points are used to represent points determined by the user to be contained in the region of interest, and the negative example points are used to represent points determined by the user to be not contained in the region of interest.

[0014] Optionally, the point corresponding to the subsequent position on the medical image is determined as a feature point, and specifically includes:

[0015] The input interface is displayed to the user, the input information of the user is received, and the point corresponding to the subsequent position specified by the user on the medical image is determined as a positive example point or a negative example point according to the input information.

[0016] Optionally, before the to-be-determined region of interest is adjusted by the prediction model according to the feature points, the method further includes:

[0017] According to the positive example points determined on the medical image, an envelope box containing all the positive example points is determined.

[0018] When the point specified by the user on the medical image is a negative example point, if the negative example point is located in the envelope box, an error prompt is returned.

[0019] Optionally, the prediction model is pre-trained, and specifically includes:

[0020] A sample medical image containing a lesion and a labeled region of interest of the sample medical image are obtained.

[0021] An initial sample positive example point in the sample medical image is selected, and a sample to-be-determined region of interest containing the initial sample positive example point is determined; wherein the sample positive example point is a point contained in the labeled region of interest, and the sample negative example point is a point not contained in the labeled region of interest.

[0022] A plurality of subsequent sample positive example points are determined in the sample medical image, a total feature map of positive example points is generated according to all the subsequent sample positive example points, a plurality of subsequent sample negative example points are determined in the sample medical image, and a total feature map of negative example points is generated according to all the subsequent sample negative example points.

[0023] The sample to-be-determined region of interest, the total feature map of positive example points, and the total feature map of negative example points are input into a to-be-trained prediction model, to obtain a prediction region of interest output by the prediction model.

[0024] The prediction model is trained with the minimum difference between the prediction region of interest and the labeled region of interest as an optimization target.

[0025] Optionally, the sum of the number of subsequent sample positive example points and the number of subsequent sample negative example points determined in the medical image is a specified number.

[0026] Optionally, a positive example point total feature map is generated according to all the subsequent sample positive example points, specifically including:

[0027] For each subsequent sample positive example point, a feature map of the subsequent sample positive example point on the sample medical image is determined;

[0028] According to the feature maps of the subsequent sample positive example points, a positive example point total feature map is generated;

[0029] A negative example point total feature map is generated according to all the subsequent sample negative example points, specifically including:

[0030] For each subsequent sample negative example point, a feature map of the subsequent sample negative example point on the sample medical image is determined;

[0031] According to the feature maps of the subsequent sample negative example points, a negative example point total feature map is generated.

[0032] Optionally, the positive example point total feature map and the negative example point total feature map are represented in the form of a feature map;

[0033] For each pixel point in the positive example point total feature map, the pixel value of the pixel point is the maximum pixel value of the pixel point in the feature maps corresponding to each positive example point;

[0034] For each pixel point in the negative example point total feature map, the pixel value of the pixel point is the maximum pixel value of the pixel point in the feature maps corresponding to each negative example point.

[0035] The present specification provides a region of interest determination apparatus, comprising:

[0036] An acquisition module is configured to acquire a medical image containing a lesion and display the medical image to a user;

[0037] A first response module is configured to determine an initial position corresponding to an initial designation operation of the user on the medical image in response to the initial designation operation, and determine an initial region of interest according to the initial position;

[0038] An output module is configured to output a to-be-determined region of interest according to the medical image and the initial region of interest by using a pre-trained prediction model;

[0039] A second response module is configured to determine a subsequent position corresponding to a subsequent designation operation of the user on the medical image in response to the subsequent designation operation, and determine a feature point as a point corresponding to the subsequent position on the medical image;

[0040] an adjusting module configured to adjust the to-be-determined region of interest according to the feature point by using the prediction model, and determine a predicted region of interest according to the adjusted to-be-determined region of interest.

[0041] The specification provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned region of interest determination method.

[0042] The specification provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor implements the above-mentioned region of interest determination method when executing the program.

[0043] The above-mentioned at least one technical solution adopted by the specification can achieve the following beneficial effects:

[0044] In the region of interest determination method provided by the specification, a medical image containing a lesion is acquired, and the medical image is displayed to a user. In response to an initial designation operation of the user on the medical image, an initial position corresponding to the initial designation operation on the medical image is determined, and an initial region of interest is determined according to the initial position. According to the medical image and the initial region of interest, a to-be-determined region of interest is output by using a pre-trained prediction model. In response to a subsequent designation operation of the user on the medical image, a subsequent position corresponding to the subsequent designation operation on the medical image is determined, and a point corresponding to the subsequent position on the medical image is determined as a feature point. The to-be-determined region of interest is adjusted according to the feature point by using the prediction model, and a predicted region of interest is determined according to the adjusted to-be-determined region of interest.

[0045] When the region of interest determination method provided by the specification is used, an initial region of interest can be first determined according to an initial designation operation of a user, and a to-be-determined region of interest can be output according to the initial region of interest by using a prediction model. Then, a feature point can be determined according to a subsequent designation operation of the user, and a predicted region of interest can be obtained by adjusting the to-be-determined region of interest according to the feature point. By combining the user operation and the pre-trained prediction model, semi-automatic determination of the region of interest is realized. In the implementation process of the method, the user only needs to make simple operations, and the region of interest can be determined, which greatly reduces the workload of the user in the process of labeling the region of interest. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings, which are included to provide a further understanding of the specification, constitute a part of the specification, and the illustrative embodiments of the specification and their description serve to explain the specification, and do not constitute an improper limitation on the specification. In the drawings:

[0047] Figure 1 A flowchart of a region of interest determination method provided in the present specification;

[0048] Figure 2 A process diagram of expanding a region of interest to be determined according to positive example points provided in the present specification;

[0049] Figure 3 A process diagram of reducing a region of interest to be determined according to negative example points provided in the present specification;

[0050] Figure 4 A schematic diagram of a region of interest determination device provided in the present specification;

[0051] Figure 5 A schematic diagram of an electronic device corresponding to Figure 1 provided in the present specification. DETAILED DESCRIPTION

[0052] In order to make the purposes, technical solutions and advantages of the present specification clearer, the technical solutions of the present specification will be described in detail below with reference to the embodiments of the present specification and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present specification, not all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present specification.

[0053] The technical solutions provided by the embodiments of the present specification will be described in detail below with reference to the drawings.

[0054] Figure 1 A flowchart of a region of interest determination method provided in the present specification, comprising the following steps:

[0055] S100: Obtain a medical image containing a lesion, and display the medical image to a user.

[0056] In the present specification, the execution subject for implementing the region of interest determination method can refer to a server or other specified device provided in a business platform. In order to facilitate the description, the present specification only takes the server as the execution subject to describe a code region of interest prediction method provided in the present specification.

[0057] In the region of interest determination method provided in this application, the region of interest can be a target object and its surrounding area in an image that needs to be focused on. The target object can be any entity. For ease of understanding, the application scenario of determining the region of interest in a medical image in the medical field is taken as an example for description in this specification. In the embodiments provided in this specification, the target object can be a lesion, and the region of interest is the lesion and the area around the lesion, that is, the lesion box in the medical field.

[0058] Generally, the lesion box is a region of interest determined on the basis of a medical image on the medical image. Therefore, in this step, a medical image containing a lesion can be obtained first, and the medical image is displayed to the user. The medical image can be any image containing a lesion used in the medical field, including but not limited to X-ray imaging, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound image, radionuclide imaging, etc.; the user can be any person who has the ability to label the lesion box in the medical image, usually a doctor, professor, etc. with professional medical knowledge.

[0059] S102: In response to the initial designation operation of the user on the medical image, determining the initial position corresponding to the initial designation operation on the medical image, and determining an initial region of interest according to the initial position.

[0060] The region of interest determination method provided in this specification is mainly used to realize semi-automatic labeling of the region of interest, that is, to assist the prediction model to output the region of interest through a small amount of manual operation. Therefore, before the region of interest is determined by using the model, an initial region needs to be given on the medical image in response to the user operation, to assist the model to better determine the region of interest.

[0061] Based on this, in this step, first, in response to the initial designation operation of the user on the medical image, the initial position corresponding to the initial designation operation on the medical image is determined. The initial designation operation of the user can include but is not limited to single-click, double-click, long-press, etc. Any operation that can determine a position on the medical image. The position on the medical image corresponding to the initial designation operation is taken as the initial position, and the initial region of interest is determined according to the initial position.

[0062] There can be various methods for determining the initial region of interest based on the initial position, and the present specification provides an embodiment for reference. Specifically, an initial region of interest of a specified size and shape can be determined with the initial position as the center. The specified size and shape can be determined according to specific requirements, and the specified shape can be a regular shape or an irregular shape. It is not difficult to imagine that although the size and shape of each different lesion can not be the same, the size and shape of the lesion in each case can not differ much for many lesions corresponding to the same disease, and thus different specified sizes and shapes can be determined according to different types of diseases. In addition, the initial position can not be the center, but can only be included in the determined initial region of interest.

[0063] In this step, the initial region of interest determined based on the initial designation operation of the user helps the prediction model to determine the lesion region that needs to be focused on, that is, to artificially inform the prediction model of the position of the lesion. At present, it is also feasible to help the prediction model to determine the position of the lesion only by the initial position, but in order to improve the prediction effect of the prediction model, the method of predicting the region of interest containing the lesion by the initial region of interest in the method for determining the region of interest provided in the present specification is selected.

[0064] S104: outputting a to-be-determined region of interest from a prediction model pre-trained according to the medical image and the initial region of interest.

[0065] According to the medical image and the initial region of interest determined in step S102, a to-be-determined region of interest can be output in this step by a prediction model pre-trained.

[0066] The initial region of interest and the medical image are input into the prediction model, and the prediction model can output the predicted to-be-determined region of interest after processing and calculation. The size of the to-be-determined region of interest is related to the size of the lesion in the medical image, and in order to be able to intuitively give the to-be-determined region of interest, the shape of the to-be-determined region of interest is usually a common regular shape, and according to different requirements during training, the shape can be a two-dimensional shape such as a rectangle, a square, a circle, etc., or a three-dimensional shape such as a cuboid, a cube, a sphere, etc., which is not specifically limited in the present specification. In the case where the medical image is fixed, a set of data can be used to represent the to-be-determined region of interest, and the data can include the position coordinates of the center point of the to-be-determined region of interest in the medical image and the size of the to-be-determined region of interest. At the same time, the to-be-determined region of interest can also be represented in the medical image in the form of a highlighted region or a frame.

[0067] S106: In response to the subsequent designation operation of the user on the medical image, a subsequent position corresponding to the subsequent designation operation on the medical image is determined, and a point corresponding to the subsequent position on the medical image is determined as a feature point.

[0068] As mentioned in the background, in the medical field, since the lesion area is more difficult for the machine to identify, even with the assistance of professionals, it is difficult for the prediction model to directly and very accurately predict the region of interest according to the initial region of interest. Therefore, after the prediction model outputs the pending region of interest in step S104, the user still needs to further assist the prediction model to determine a more accurate region of interest.

[0069] First, the subsequent designation operation of the user on the medical image can be responded to, and the subsequent position corresponding to the subsequent designation operation on the medical image is determined. Similar to the determination of the initial position in step S102, in this step, the subsequent designation operation of the user on the medical image can also be a single click, double click, long press, etc. Any operation that can determine a position in the medical image. The position determined on the medical image according to the subsequent designation operation of the user is determined as the subsequent position.

[0070] For the determined subsequent position, the point corresponding to the subsequent position on the medical image can be determined as a feature point. The feature point can be used to help the prediction model to further determine a more accurate region of interest.

[0071] Specifically, the feature point can include a positive example point or a negative example point, wherein the positive example point is used to represent a point determined by the user to be included in the region of interest, and the negative example point is used to represent a point determined by the user to be not included in the region of interest.

[0072] Any one of the feature points determined according to the subsequent designation operation of the user can be one of a positive example point or a negative example point. Among them, the positive example point is a point determined by the user to be included in the region of interest, and the negative example point is a point determined by the user to be not included in the region of interest. In other words, if the user himself divides the region of interest, the positive example point must be in the region of interest divided by the user, and the negative example point must be outside the region of interest divided by the user.

[0073] In actual application, a feature point can be determined as a positive example point or a negative example point in various ways. This specification gives a specific embodiment for reference. Specifically, when the point corresponding to the subsequent position on the medical image is determined as a feature point, an input interface can be displayed to the user, the input information of the user is received, and the point corresponding to the position designated by the user on the medical image is determined as a positive example point or a negative example point according to the input information.

[0074] After determining a feature point, an input interface can be shown to the user and receive input information of the user, and determine the feature point as a positive example point or a negative example point according to the output information. Among them, the input method shown to the user in the input interface can exist in multiple forms, including but not limited to text input, voice input, or selection form key input and the like. Among them, the input method such as text or voice that can obtain the text content of the user input can adopt a pre-set input rule, for example, the user inputs "1" to represent a positive example point, inputs "0" to represent a negative example point, and other contents are invalid, and the user is notified to re-input; It can also be used to identify whether the input content of the user contains a keyword, for example, if the input content of the user contains "positive", it is determined as a positive example point, and if it contains "negative", it is determined as a negative example point, and if it contains or does not contain, it is invalid. The selection form key input can give two keys representing "positive example point" and "negative example point" in the input interface, and correspondingly determine the feature point as a positive example point or a negative example point according to the user's selection.

[0075] S108: Adjusting the to-be-determined region of interest according to the feature point through the prediction model, and determining the adjusted to-be-determined region of interest as a predicted region of interest.

[0076] In this step, the to-be-determined region of interest can be adjusted according to the feature point determined in step S106 through the prediction model to obtain the final predicted region of interest.

[0077] Further, in the case of determining the feature point as a positive example point or a negative example point, the to-be-determined region of interest can be adjusted according to the positive example point and the negative example point through the prediction model. Specifically, when the determined feature point is a positive example point, the prediction model can expand the to-be-determined region of interest according to the position of the positive example point in the medical image; when the determined feature point is a negative example point, the prediction model can reduce the to-be-determined region of interest according to the position of the negative example point in the medical image.

[0078] For example, Figure 2 is a schematic diagram of the process of expanding and adjusting the to-be-determined region of interest according to the positive example point by the prediction model. As Figure 2 shown, the smaller point at the intersection of the cross line in the figure represents the initial position determined according to the user's initial specification operation, the rectangular frame represents the frame of the region of interest, the inside of which is the region of interest, the larger solid circle represents the center point of the region of interest, and the hollow circle represents the feature point determined as a positive example point. In Figure 2 the medical image on the left side, the region inside the frame is the to-be-determined region of interest generated by the prediction model according to the initial region of interest, and the feature point outside the frame is the feature point determined as a positive example point according to the user's subsequent specification operation. According to the above information, the prediction model can expand and adjust the to-be-determined region of interest, and the result after adjustment isFigure 2 The rectangular border in the medical image on the right indicates the predicted region of interest.

[0079] For example, Figure 3 This is a schematic diagram illustrating the process by which the prediction model narrows down and adjusts the region of interest based on negative examples. For example... Figure 3 As shown in the figure, the smaller dot at the intersection of the crosshairs represents the initial position determined by the user's initial specified operation. The rectangular border represents the border of the region of interest, and the area inside the rectangle is the region of interest. The larger light-colored solid dots represent the center point of the region of interest, and the larger dark-colored solid dots represent feature points determined as negative examples. Figure 3 In the medical image on the left, the region within the bounding box is the region of interest to be determined, generated by the prediction model based on the initial region of interest. The feature points within the bounding box are feature points identified as negative examples based on subsequent user-specified operations. Based on this information, the prediction model can reduce and adjust the region of interest to be determined. The adjusted result is... Figure 3 The predicted region of interest is indicated by the rectangular border in the medical image on the right.

[0080] Based on the identified feature points and the region of interest to be determined, the prediction model can be adjusted to obtain the final predicted region of interest.

[0081] In fact, even after the prediction model adjusts the region of interest based on the feature points once, it still cannot guarantee that the obtained region of interest is the most accurate. Therefore, it is possible to further determine the feature points multiple times and make multiple adjustments during the adjustment process until the user confirms that the adjustment is complete.

[0082] Specifically, after adjusting the region of interest (ROI), a prompt interface can be displayed to the user, asking if they want to end the adjustment. If the user selects "Yes," the adjusted ROI can be determined as the final predicted ROI, and the prediction of the ROI ends. If the user selects "No," the adjusted ROI is displayed in the medical image, and the system responds again to subsequent user-specified operations on the medical image, determining feature points and adjusting the ROI based on these feature points. This involves re-executing steps S106 to S108 until the user selects "Yes," ending the adjustment of the ROI.

[0083] In the case of multiple adjustments of the to-be-determined region of interest, it is necessary to consider that each feature point given by the user should conform to the basic logic and cannot be wrong. Therefore, after each feature point is determined, before the to-be-determined region of interest is adjusted, an envelope box containing all positive example points can be determined according to the positive example points determined on the medical image; when the point specified by the user on the medical image is a negative example point, if the negative example point is located in the envelope box, an error prompt is returned. The envelope box containing all positive example points can be the edge of the smallest figure that can include all the determined positive example points.

[0084] It can be conceived that, under normal circumstances, a lesion should be a single complete and continuous target object and cannot be hollow or partially separated. When there are two completely separated parts of a lesion under a case, two lesions should be processed to determine two corresponding regions of interest. Therefore, there will be no separated or hollow region of interest in the region of interest corresponding to a lesion. In the above case, the area within the envelope box containing all the determined positive example points should be part of the region of interest, and if a negative example point appears in the area within the envelope box, it can be considered that the user has made a mistake, and an error prompt is returned to the user.

[0085] In the implementation of the region of interest determination method provided in the present specification, an initial region of interest can be first determined according to the initial specifying operation of the user, and a to-be-determined region of interest can be output from the initial region of interest by a prediction model; then, a feature point can be determined according to the subsequent specifying operation of the user, and the to-be-determined region of interest can be adjusted according to the feature point by prediction to obtain a predicted region of interest. By combining the user operation with the pre-trained prediction model, semi-automatic determination of the region of interest is realized; in the implementation process of the present method, the user only needs to make simple operations to determine the region of interest, which greatly reduces the workload of the user in the process of labeling the region of interest.

[0086] Additionally, the prediction model used in the region of interest determination method provided in the present specification can be pre-trained. The training process can specifically include: obtaining a sample medical image containing a lesion and an annotated region of interest of the sample medical image; selecting an initial sample positive point in the sample medical image and determining a sample pending region of interest containing the initial sample positive point; wherein the sample positive point is a point contained in the annotated region of interest, and the sample negative point is a point not contained in the annotated region of interest; determining a plurality of subsequent sample positive points in the sample medical image, generating a total positive point feature map according to all the subsequent sample positive points, and determining a plurality of subsequent sample negative points in the sample medical image, generating a total negative point feature map according to all the subsequent sample negative points; inputting the sample pending region of interest, the total positive point feature map, and the total negative point feature map into the prediction model to be trained to obtain a predicted region of interest output by the prediction model; and training the prediction model with the minimum difference between the predicted region of interest and the annotated region of interest as the optimization goal.

[0087] During training, a medical image in which the region of interest has been annotated in the history or a medical image in which the region of interest has been determined by a doctor in actual work can be used as the sample medical image containing the lesion and the corresponding annotated region of interest. During the training process, the sample positive point is a point contained in the annotated region of interest on the sample medical image, and the sample negative point is a point not contained in the annotated region of interest on the sample medical image.

[0088] The initial sample positive point can be any point in the sample pending region of interest. The size of the sample pending region of interest can be pre-set according to specific requirements, which is not specifically limited in the present specification. In order to ensure the training effect of the prediction model and prevent overfitting, the sum of the number of subsequent sample positive points and the number of subsequent sample negative points determined during training can be a specified number. According to different requirements for the prediction model, the specified number can be set arbitrarily, which is not specifically limited in the present specification.

[0089] According to all the subsequent sample positive points, a total feature map of the positive points can be determined, and similarly, according to all the subsequent sample negative points, a total feature map of the negative points can be determined. The total feature map of the positive points and the total feature map of the negative points respectively represent the feature map of all the subsequent sample positive points on the medical image and the feature map of all the subsequent sample negative points on the medical image. In generating the total feature map of the positive points according to all the subsequent sample positive points, specifically, for each subsequent sample positive point, a feature map of the subsequent sample positive point on the sample medical image is determined; and the total feature map of the positive points is generated according to the feature maps of the subsequent sample positive points. Similarly, in generating the total feature map of the negative points according to all the subsequent sample negative points, specifically, for each subsequent sample negative point, a feature map of the subsequent sample negative point on the sample medical image is determined; and the total feature map of the negative points is generated according to the feature maps of the subsequent sample negative points. The feature map generated by one positive point or negative point is determined by the position of the positive point or negative point and the size of the feature map. In the calculation process, the positive point or negative point is the center of the Gaussian kernel function, and the pixel value of each point on the feature map is calculated. The maximum value on the feature map is obtained at the position of the positive point or negative point on the medical image, and the value is close to 1. Such a feature map can reflect the position distribution of the positive point or negative point on the sample medical image, so the total feature map of the positive points can be the total distribution of the positive points on the medical image, and the total feature map of the negative points can be the total distribution of the negative points on the medical image.

[0090] In the above case, for each pixel point in the total distribution of the positive points, the pixel value of the pixel point is the maximum pixel value of the pixel point in the feature map corresponding to each positive point; and for each pixel point in the total distribution of the negative points, the pixel value of the pixel point is the maximum pixel value of the pixel point in the feature map corresponding to each negative point.

[0091] When the value range of the pixel point on the feature map is (0, 1), the feature map of the feature point can be regarded as a probability distribution. The total distribution of the positive points can be obtained according to the probability distribution of each individual positive point, and the total distribution of the negative points can be obtained according to the probability distribution of each individual negative point. The probability distribution can be a distribution obtained by a Gaussian kernel function, and the probability distribution of each feature point and the total distribution can be represented in the form of a feature map. In the feature map of one feature point, the pixel value of a pixel point on the feature map can be understood as the probability that the feature point is located at the pixel point. In the total feature map of the positive points or the negative points, the pixel value of a pixel point on the feature map can be understood as the probability that a feature point exists at the pixel point. Whether it is a feature map of one feature point or a total feature map of one type of feature point, it is generated on the basis of the sample medical image, that is, the positions of the pixel points in the feature maps are corresponding to each other, and the position of a pixel point in the feature maps is fixed and unchanged.

[0092] Therefore, when determining the total feature map of positive example points, the pixel value of each pixel point can be determined as the maximum pixel value in the feature map corresponding to the probability distribution of each positive example point, that is, the maximum probability of the presence of a positive example point on the pixel point. Similarly, when determining the total score of negative example points, the pixel value of each pixel point can be determined as the maximum pixel value in the feature map corresponding to the probability distribution of each negative example point, that is, the maximum probability of the presence of a negative example point on the pixel point.

[0093] It should be noted that when the value range of each pixel point in the feature map is not (0, 1), the pixel value of each pixel point in the feature map can no longer represent the probability, but only represent the pixel value of the point itself. At this time, the feature map can be regarded as a heat map. However, the pixel value of each pixel point in the feature map can still be calculated by the Gaussian kernel function, and the probability of each position in the feature map obtained by the Gaussian kernel function can be mapped to the maximum value of the pixel value range to obtain the pixel value of the pixel point at the position.

[0094] Based on the same idea, the present specification also provides a corresponding region of interest determination device, as shown in Figure 4 .

[0095] Figure 4 A schematic diagram of a region of interest determination device provided by the present specification includes:

[0096] The acquisition module 200 is configured to acquire a medical image containing a lesion and display the medical image to a user.

[0097] The first response module 202 is configured to determine an initial position corresponding to an initial designation operation of the user on the medical image, and determine an initial region of interest according to the initial position in response to the initial designation operation on the medical image.

[0098] The output module 204 is configured to output a to-be-determined region of interest according to the medical image and the initial region of interest by using a pre-trained prediction model.

[0099] The second response module 206 is configured to determine a subsequent position corresponding to a subsequent designation operation of the user on the medical image, and determine a feature point as a point corresponding to the subsequent position on the medical image in response to the subsequent designation operation on the medical image.

[0100] The adjustment module 208 is configured to adjust the to-be-determined region of interest according to the feature point by using the prediction model, and determine a predicted region of interest as the adjusted to-be-determined region of interest.

[0101] Optionally, the feature points include positive example points or negative example points, wherein positive example points are used to characterize points that the user determines are included in the region of interest, and negative example points are used to characterize points that the user determines are not included in the region of interest.

[0102] Optionally, the second response module 206 is specifically used to display an input interface to the user, receive the user's input information, and determine the point corresponding to the position subsequently specified by the user on the medical image as a positive or negative point based on the input information.

[0103] Optionally, the device further includes a prompting module 210, specifically used to determine an envelope containing all positive points based on the positive points identified on the medical image; when the point specified by the user on the medical image is a negative point, if the negative point is located within the envelope, an error prompt is returned.

[0104] Optionally, the device further includes a training module 212, specifically configured to acquire a sample medical image containing lesions and a labeled region of interest (ROI) of the sample medical image; select an initial positive sample point in the sample medical image and determine a sample region of interest to be determined containing the initial positive sample point; wherein, the positive sample point is a point contained in the labeled ROI, and the negative sample point is a point not contained in the labeled ROI; determine several subsequent positive sample points in the sample medical image, generate a total feature map of positive sample points based on all subsequent positive sample points, and determine several subsequent negative sample points in the sample medical image, generate a total feature map of negative sample points based on all subsequent negative sample points; input the sample region of interest to be determined, the total feature map of positive sample points, and the total feature map of negative sample points into a prediction model to be trained to obtain the predicted ROI output by the prediction model; and train the prediction model with the minimum difference between the predicted ROI and the labeled ROI as the optimization objective.

[0105] Optionally, the sum of the number of subsequent positive sample points and the number of subsequent negative sample points determined in the medical image is a specified number.

[0106] Optionally, the training module 212 is specifically configured to: for each subsequent positive sample point, determine the feature map of that subsequent positive sample point on the sample medical image; generate a total feature map of positive sample points based on the feature maps of each subsequent positive sample point; for each subsequent negative sample point, determine the feature map of that subsequent negative sample point on the sample medical image; and generate a total feature map of negative sample points based on the feature maps of each subsequent negative sample point.

[0107] Optionally, the feature map, the positive example point total feature map and the negative example point total feature map are represented in the form of feature maps; for each pixel point in the positive example point total feature map, a pixel value of the pixel point is a maximum pixel value of the pixel point in the feature maps corresponding to the positive example points; for each pixel point in the negative example point total feature map, a pixel value of the pixel point is a maximum pixel value of the pixel point in the feature maps corresponding to the negative example points.

[0108] The specification also provides a computer readable storage medium storing a computer program, the computer program being used to execute the above Figure 1 The specification provides a region of interest determination method.

[0109] The specification also provides a computer readable storage medium storing a computer program, the computer program being used to execute the above Figure 5 The specification provides an electronic device corresponding to the above Figure 1 The specification provides an electronic device corresponding to the above Figure 5 The specification provides an electronic device corresponding to the above Figure 1 The specification provides an electronic device corresponding to the above

[0110] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0111] The controller can be implemented in any suitable way, e.g. the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, e.g. software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91 SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to being implemented in pure computer readable program code form, the controller can perfectly well be implemented by means of logic programmed into logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. to perform the same functions. The controller can thus be considered as a hardware component, and the means comprised therein for performing various functions can be considered as structures within the hardware component. Alternatively, or even, the means for performing various functions can be considered as both a software module implementing a method and a structure within a hardware component.

[0112] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0113] For the sake of description, the above apparatuses are described in functional division and are described respectively. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware when implementing the present specification.

[0114] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0115] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product according to embodiments of the specification, it can be understood that each block in the flow diagrams and / or block diagrams and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product according to embodiments of the specification, it can be understood that each block in the flow diagrams and / or block diagrams and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.

[0116] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product according to embodiments of the specification, it can be understood that each block in the flow diagrams and / or block diagrams and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product according to embodiments of the specification, it can be understood that each block in the flow diagrams and / or block diagrams and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.

[0117] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product according to embodiments of the specification, it can be understood that each block in the flow diagrams and / or block diagrams and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. ​ The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product according to embodiments of the specification, it can be understood that each block in the flow diagrams and / or block diagrams and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks.

[0118] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0119] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) for example. Mass storage can include, in examples, EEPROM, flash memory or other non-volatile memory. The memory is an example of computer readable media.

[0120] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0121] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0122] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0124] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different aspects of the description. For each embodiment, the description focuses on the differences from the other embodiments. In particular, the description of the system embodiments is relatively brief, as the system embodiments are largely analogous to the method embodiments. The relevant parts of the description of the method embodiments are referred to.

[0125] The above description is embodied in the form of embodiments only and is not intended to limit the present specification. The present specification can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.

Claims

1. A region of interest determination method characterized by, The method comprises: obtaining a medical image containing a lesion, and displaying the medical image to a user; in response to an initial designation operation of the user on the medical image, determining an initial position corresponding to the initial designation operation on the medical image, and determining an initial region of interest according to the initial position; outputting a to-be-determined region of interest by a pre-trained prediction model according to the medical image and the initial region of interest; in response to a subsequent designation operation of the user on the medical image, determining a subsequent position corresponding to the subsequent designation operation on the medical image, and determining a point corresponding to the subsequent position on the medical image as a feature point; adjusting the to-be-determined region of interest according to the feature point by the prediction model, and determining an adjusted to-be-determined region of interest as a predicted region of interest; wherein the pre-trained prediction model comprises: obtaining a sample medical image containing a lesion, and a labeled region of interest of the sample medical image; selecting an initial sample positive example point in the sample medical image, and determining a sample to-be-determined region of interest containing the initial sample positive example point; wherein the sample positive example point is a point contained in the labeled region of interest, and a sample negative example point is a point not contained in the labeled region of interest; determining a plurality of subsequent sample positive example points in the sample medical image, generating a total feature map of positive example points according to all the subsequent sample positive example points, and determining a plurality of subsequent sample negative example points in the sample medical image, generating a total feature map of negative example points according to all the subsequent sample negative example points; inputting the sample to-be-determined region of interest, the total feature map of positive example points, and the total feature map of negative example points into a to-be-trained prediction model to obtain a predicted region of interest output by the prediction model; training the prediction model with the minimum difference between the predicted region of interest and the labeled region of interest as an optimization target.

2. The method of claim 1, wherein, The feature point comprises a positive example point or a negative example point, wherein the positive example point is used to represent a point determined by the user to be contained in the region of interest, and the negative example point is used to represent a point determined by the user to be not contained in the region of interest.

3. The method of claim 2, wherein, The point corresponding to the subsequent position on the medical image is determined as a feature point, specifically comprising: displaying an input interface to the user, receiving input information of the user, and determining the point corresponding to the position designated by the user on the medical image as a positive example point or a negative example point according to the input information.

4. The method of claim 2, wherein, Before adjusting the to-be-determined region of interest according to the feature point by the prediction model, the method further comprises: determining an envelope box containing all the positive example points according to the positive example points determined on the medical image; when the point designated by the user on the medical image is a negative example point, if the negative example point is located in the envelope box, an error prompt is returned.

5. The method of claim 1, wherein, The sum of the number of subsequent sample positive example points and the number of subsequent sample negative example points in the sample medical image is a specified number.

6. The method of claim 1, wherein, Generating a total feature map of positive example points according to all the subsequent sample positive example points comprises: determining a feature map of each subsequent sample positive example point on the sample medical image; generating a total feature map of positive points according to the feature maps of the positive points of each subsequent sample; generating a total feature map of negative points according to all the negative points of the subsequent samples, specifically comprising: for each negative point of the subsequent samples, determining a feature map of the negative point on the sample medical image; generating a total feature map of negative points according to the feature maps of the negative points of each subsequent sample.

7. The method of claim 6, wherein, for each pixel point in the total feature map of positive points, the pixel value of the pixel point is the maximum pixel value of the pixel point in the feature maps corresponding to each positive point; for each pixel point in the total feature map of negative points, the pixel value of the pixel point is the maximum pixel value of the pixel point in the feature maps corresponding to each negative point.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-7.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1-7.

Images