Lesion marking method and device, electronic equipment and readable storage medium

Abstract

Embodiments of the present application provide a lesion marking method and device, electronic equipment and readable storage medium. The method comprises: displaying each layer of image in order of image layer; in response to the user continuously marking the outline of the lesion on two layers of images, obtaining a first lesion outline and a second lesion outline respectively; performing feature extraction on the image area framed by the first lesion outline and the second lesion outline respectively, and obtaining a first lesion feature set and a second lesion feature set; determining whether there is a missing layer image between the two layers of images according to the image layer of each of the two layers of images; in the case that there is a missing layer image between the two layers of images, determining a third lesion feature set on the missing layer image according to the first lesion feature set and the second lesion feature set; and according to the third lesion feature set, outlining the lesion outline on the missing layer image and marking the lesion. Through the method, the user can effectively avoid missing layers during the lesion marking process, and improve the efficiency of lesion marking.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, in particular to a lesion marking method and device, electronic equipment and readable storage medium. BACKGROUND

[0002] Currently, after a patient has an image examination, a doctor needs to manually mark a lesion in a photographed image. For a series of tomographic images or cross-sectional images generated by a certain examination (for example, CT, nuclear magnetic resonance), the doctor needs to mark a lesion for each layer of image in the series of images. However, due to the large number of images in a series of images in one examination, the doctor is prone to miss a layer (for example, a certain layer or several layers of images in the series of images are not marked with a lesion) in the process of marking a lesion, which results in an incomplete three-dimensional profile of the lesion, inaccurate calculation of the volume and size of the lesion, and thus affects the formulation of a subsequent treatment plan. SUMMARY

[0003] In view of the above problems, embodiments of the present application are proposed to provide a lesion marking method, device, electronic equipment and readable storage medium which overcome the above problems or at least partially solve the above problems.

[0004] In a first aspect of the present application, a lesion marking method is first provided, and the method comprises:

[0005] displaying each layer of image in order of image layer number;

[0006] in response to a user continuously marking a lesion contour on two layers of image, obtaining a first lesion contour and a second lesion contour, respectively;

[0007] performing feature extraction on the image region framed by the first lesion contour and the second lesion contour, respectively, to obtain a first lesion feature set and a second lesion feature set;

[0008] determining whether there is a missing layer of image between the two layers of image according to the image layer number of each of the two layers of image;

[0009] in a case where there is a missing layer of image between the two layers of image, determining a third lesion feature set on the missing layer of image according to the first lesion feature set and the second lesion feature set;

[0010] drawing a lesion contour on the missing layer of image according to the third lesion feature set and marking a lesion.

[0011] Optionally, the method further comprises:

[0012] obtaining a fourth lesion feature set on the missing layer of image, the fourth lesion feature set being a lesion feature set obtained by automatic lesion identification on the missing layer of image.

[0013] the third lesion feature set, including:

[0014] determining whether the deviation between the third lesion feature set and the fourth lesion feature set is within a preset threshold;

[0015] in the case that the deviation is within the preset threshold, delineating a lesion contour and performing lesion labeling on the missing layer image according to the third lesion feature set.

[0016] Optionally, each lesion feature set includes multiple attribute features: a center point coordinate of a lesion, a contour coordinate of a lesion, an area of a lesion, and a color of a lesion.

[0017] the third lesion feature set on the missing layer image according to the first lesion feature set and the second lesion feature set, including:

[0018] respectively performing statistical calculation on the same attribute features in the first lesion feature set and the second lesion feature set to obtain statistical feature values of the same attribute features, to constitute the third lesion feature set.

[0019] Optionally, the determining whether the deviation between the third lesion feature set and the fourth lesion feature set is within a preset threshold includes:

[0020] respectively determining whether the deviation between the same attribute features in the third lesion feature set and the same attribute features in the fourth lesion feature set is within a preset threshold;

[0021] the third lesion feature set, including:

[0022] in the case that the errors of the same attribute features are all within the preset threshold, delineating a lesion contour and performing lesion labeling on the missing layer image according to the third lesion feature set.

[0023] Optionally, the method further includes:

[0024] automatically identifying a reference lesion contour of each layer image for each layer image;

[0025] performing feature extraction on an image region framed by the reference lesion contour to obtain a reference feature set; and according to the reference feature set, delineating a reference lesion contour on each layer image and performing lesion labeling for user reference.

[0026] Optionally, the method further includes:

[0027] In a case where the deviation is not within the preset threshold, a prompt information is displayed, the prompt information being used for prompting a user to manually draw a lesion contour on the missing layer image and perform lesion marking.

[0028] In a second aspect of the embodiment of the present application, a lesion marking device is further provided, the device comprising:

[0029] a display module, configured to sequentially display each layer image in order of image layer number;

[0030] a response module, configured to obtain a first lesion contour and a second lesion contour respectively in response to a user performing a contour marking operation on a lesion successively on two layer images;

[0031] a feature extraction module, configured to perform feature extraction on an image region framed by the first lesion contour and the second lesion contour respectively, to obtain a first lesion feature set and a second lesion feature set;

[0032] a first determination module, configured to determine whether there is a missing layer image between the two layer images according to image layer numbers of the two layer images respectively;

[0033] a feature determination module, configured to determine a third lesion feature set on the missing layer image according to the first lesion feature set and the second lesion feature set in a case where there is a missing layer image between the two layer images;

[0034] a first marking module, configured to draw a lesion contour and perform lesion marking on the missing layer image according to the third lesion feature set.

[0035] In still another aspect of the embodiment of the present application, an electronic device is further provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being executed by the processor to implement steps of the lesion marking method according to the first aspect of the embodiment of the present application.

[0036] In still another aspect of the embodiment of the present application, a computer readable storage medium is further provided, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement steps of the lesion marking method according to the first aspect of the embodiment of the present application.

[0037] The embodiment of the present application has the following advantages:

[0038] Through the lesion marking method of the embodiment of the present application, whether the user exists the situation of missing layer unmarked in the process of marking the lesions of a series of images is detected, if yes, the lesions on the missing layer image are marked according to the lesion feature sets of the upper and lower two layers marked by the user, so that the user missing layer in the lesion marking process is effectively avoided, and the lesion marking efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0040] Figure 1 is a step flow chart of a lesion marking method according to an embodiment of the present application;

[0041] Figure 2 is a structural block diagram of a lesion marking device provided by an embodiment of the present application;

[0042] Figure 3 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions of the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0044] The term "comprising" and variations thereof as used in this document mean "including but not limited to". Unless otherwise specified, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "one example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. can refer to different or same objects. Other explicit and implicit definitions can also be included below.

[0045] As mentioned above, in the process of relevant imaging diagnosis, doctors need to determine the size, area, volume and other attributes of the lesion in order to formulate the corresponding treatment plan. In the traditional scheme, doctors need to manually draw and mark the lesion on each layer of a series of tomographic images or cross-sectional images generated by CT, magnetic resonance imaging and other such examinations, and then calculate. Since the number of images in one examination is large, doctors are prone to miss layers, resulting in inaccurate calculation of the volume and size of the lesion, thereby affecting the formulation of the subsequent treatment plan. The lesion refers to the part of the body where the tissue or organ is affected by the pathogenic factor and causes pathological changes.

[0046] Therefore, in order to at least partially solve one or more of the above problems and other potential problems, embodiments of the present application propose a lesion marking method which can detect whether there is a missing layer in the process of marking the lesion of a series of images by the user, and if so, mark the lesion on the missing layer image according to the lesion feature sets of the upper and lower two layers marked by the user, thereby effectively avoiding the user from missing layers in the lesion marking process and improving the efficiency of lesion marking.

[0047] Reference Figure 1 , Figure 1 is a step flowchart of a lesion marking method according to an embodiment of the present application. The method is applied to a lesion marking system, and the lesion marking system of the present embodiment is a program that can be run on a computer (here, the computer refers to a general computer) and can display examination images, assist users in marking lesions on examination images, and calculate and store lesion data, as shown in Figure 1 The lesion marking method of the present embodiment can include the following steps:

[0048] Step S11: sequentially display each layer of images according to the image layer number.

[0049] In the present embodiment, after the lesion marking system obtains a series of examination images generated by the examination device, it sequentially displays each layer of images according to the image layer number of each image. The image layer number is a label given to each examination image by the examination device when generating a series of examination images, and the image layer number represents the position of each examination image (i.e. each layer of images) in the examination site. In the present embodiment, the lesion marking system will sequentially display each layer of images according to the image layer number, so that the user can view the examination images in order in the system and perform lesion marking on each layer of examination images.

[0050] Step S12: in response to the user's continuous contour marking operation on the lesion of two layers of images, respectively obtain a first lesion contour and a second lesion contour.

[0051] In this embodiment, a user (e.g., a doctor) performs lesion marking on each of the examination images in the order of the image layers by using the lesion marking system. The lesion marking system determines the first lesion contour and the second lesion contour in response to the user's continuous contour marking operations on the two images, i.e., in response to the user's contour marking operations on the two different images twice in succession.

[0052] For example, the lesion marking system can determine the first lesion contour on the first examination image and the second lesion contour on the second examination image in response to the user's contour marking operation on the first examination image (i.e., the examination image with the image layer number 1) and the user's contour marking operation on the second examination image (i.e., the examination image with the image layer number 2) in succession. For another example, the lesion marking system can determine the first lesion contour on the fifth examination image and the second lesion contour on the seventh examination image in response to the user's contour marking operation on the fifth examination image (i.e., the examination image with the image layer number 5) and the user's contour marking operation on the seventh examination image (i.e., the examination image with the image layer number 7) in succession. That is, the "continuous contour marking operations on the two images" in this embodiment represent the continuity of the user's marking operations, rather than the continuity of the image layers.

[0053] Step S13: performing feature extraction on the image regions framed by the first lesion contour and the second lesion contour, respectively, to obtain a first lesion feature set and a second lesion feature set.

[0054] In this embodiment, after obtaining the first lesion contour and the second lesion contour, the lesion marking system performs feature extraction on the image region framed by the first lesion contour to obtain a plurality of features corresponding to the first lesion contour, thereby forming the first lesion feature set. The lesion marking system also performs feature extraction on the image region framed by the second lesion contour to obtain a plurality of features corresponding to the second lesion contour, thereby forming the second lesion feature set. The first lesion feature set and the second lesion feature set have the same types and numbers of features.

[0055] Step S14: determining whether there is a missing layer image between the two images according to the image layer numbers of the two images.

[0056] In this embodiment, each of the examination images in the series of examination images obtained by the lesion marking system carries its corresponding image layer number. The lesion marking system determines the relationship between the two images according to the image layer numbers corresponding to the two images (i.e., the two different images in the "user's contour marking operations on the two different images twice in succession") to determine whether there is a missing layer between the two images.

[0057] Step S15: In the case that there is a missing layer image between the two layer images, determining a third set of lesion features on the missing layer image according to the first set of lesion features and the second set of lesion features.

[0058] In this embodiment, if the lesion marking system determines that the layer difference between the two layer images is 2, it is judged that there is a missing layer image between the two layer images, and the missing layer image is the intermediate layer image between the two layer images.

[0059] For example, if the user successively performs lesion marking on a first layer examination image (an examination image with an image layer number of 1) and a second layer examination image (an examination image with an image layer number of 2), the layer difference (the difference between the image layer numbers) between the two layer images (the first layer examination image and the second layer examination image) is 1, and the user sequentially performs lesion marking on the images, and there is no missing layer, so it is judged that there is no missing layer image between the two layer images. If the user successively performs lesion marking on a fifth layer examination image (an examination image with an image layer number of 5) and a seventh layer examination image (an examination image with an image layer number of 7), the layer difference (the difference between the image layer numbers) between the two layer images (the fifth layer examination image and the seventh layer examination image) is 2, and the user does not perform lesion marking on a sixth layer examination image (an examination image with an image layer number of 6), that is, the user misses one layer when performing lesion marking, so it is judged that there is a missing layer image between the two layer images.

[0060] In another case, for example, if the user successively performs lesion marking on a fifth layer examination image (an examination image with an image layer number of 5) and an eighth layer examination image (an examination image with an image layer number of 8), the layer difference (the difference between the image layer numbers) between the two layer images (the fifth layer examination image and the eighth layer examination image) is 3 (the same for the case where the layer difference is 3 or more), and the user does not perform lesion marking on a sixth layer examination image and a seventh layer examination image (examination images with image layer numbers of 6 and 7), that is, the user misses multiple layers successively when performing lesion marking, and the method of this embodiment cannot effectively solve the problem of missing layers in lesion marking, so in this embodiment, the lesion marking system still judges that there is no missing layer image between the two layer images.

[0061] In the embodiment, when the lesion marking system determines that there is a missing layer image between the two images, the lesion marking system can determine a third lesion feature set on the missing layer image according to the first lesion feature set and the second lesion feature set. In the embodiment, the missing layer image is an intermediate layer image between the two images on which the user continuously performs lesion marking. The third lesion feature set in the embodiment is a lesion feature set of the intermediate layer image determined according to the lesion feature sets of the two images before and after which the user performs manual lesion marking.

[0062] Step S16: according to the third lesion feature set, the lesion contour is outlined and lesion marking is performed on the missing layer image.

[0063] In the embodiment, after the lesion marking system determines the third lesion feature set on the missing layer image according to the first lesion feature set and the second lesion feature set, the lesion contour can be directly outlined on the missing layer image according to each feature in the third lesion feature set, and each feature information is displayed beside the outlined lesion contour, so as to complete the lesion marking on the missing layer image.

[0064] In the embodiment, the lesion marking system can monitor the lesion marking behavior of the user. When it is monitored that there is a missing layer image that is not marked in the process of the user performing lesion marking on a series of images, the lesion on the missing layer image is outlined and marked according to the lesion feature sets of the two images before and after which the user performs manual marking, so as to effectively avoid the user from missing a layer in the process of lesion marking, and improve the efficiency of lesion marking. In addition, since the feature set for automatically outlining the lesion on the missing layer in the embodiment is obtained according to the lesion feature sets of the two images before and after which the user (such as a doctor) performs manual marking, and the lesion contour manually marked by the user (such as a doctor) is obtained by the user according to years of experience, the lesion feature sets of the two images before and after which the user performs manual marking are accurate. Therefore, the feature set for automatically outlining the lesion on the missing layer in the embodiment is relatively accurate, so as to ensure the accuracy of automatically marking the lesion on the missing layer, and ensure that the volume and size of the lesion calculated after the series of images are marked are accurate, which will not affect the subsequent treatment plan.

[0065] In combination with the above embodiments, in an implementation manner, the embodiment of the present application further provides a lesion marking method. Specifically, the method further includes the following steps:

[0066] Step S21: obtaining a fourth lesion feature set on the missing layer image, the fourth lesion feature set being a lesion feature set obtained by automatically identifying the lesion on the missing layer image.

[0067] In this embodiment, the lesion marking system also performs lesion identification on the missing layer image to obtain a lesion contour identified on the missing layer image, and then performs feature extraction on an image region framed by the lesion contour identified on the missing layer image, so as to obtain a plurality of features corresponding to the lesion contour identified on the missing layer image to constitute a fourth lesion feature set on the missing layer image. The fourth lesion feature set in this embodiment is a lesion feature set obtained by the lesion marking system through automatic lesion identification on the missing layer image.

[0068] On this basis, step S16 includes step S22 and step S23.

[0069] Step S22: Determine whether the deviation between the third lesion feature set and the fourth lesion feature set is within a preset threshold.

[0070] In this embodiment, after the lesion marking system obtains the third lesion feature set and the fourth lesion feature set of the missing layer image, it is necessary to compare the two lesion feature sets of the missing layer image to determine whether the deviation between the two lesion feature sets (the third lesion feature set and the fourth lesion feature set) is within a preset threshold.

[0071] For example, the deviation between the two feature sets can be determined by formula (1):

[0072]

[0073] Step S23: In the case where the deviation is within the preset threshold, according to the third lesion feature set, a lesion contour is outlined and lesion marking is performed on the missing layer image.

[0074] In this embodiment, when the lesion marking system determines that the deviation between the two lesion feature sets (the third lesion feature set and the fourth lesion feature set) on the missing layer image is within the preset threshold (including the case where the deviation is the preset threshold), the lesion marking system outlines a lesion contour on the missing layer image according to each feature in the third lesion feature set, and displays each feature information beside the outlined lesion contour, so as to complete lesion annotation on the missing layer image.

[0075] The preset threshold in this embodiment is an error experience threshold obtained according to artificial experience, and the preset threshold represents the maximum allowable error under the premise of meeting the lesion outlining effect. For example, the preset threshold in this embodiment can be 5%. It should be noted that the specific value of the preset threshold is not limited in this embodiment.

[0076] In the embodiment, after the lesion marking system obtains the third lesion feature set of the missing layer image, the lesion marking on the missing layer image is not directly performed according to the third lesion feature set, but the fourth lesion feature set of the missing layer image is further obtained, and the two lesion feature sets (the third lesion feature set and the fourth lesion feature set) of the missing layer image are compared. Since the third lesion feature set is obtained according to the lesion features of the front and rear two layers manually marked by the user, and the fourth lesion feature set is obtained by the lesion recognition of the lesion marking system on the missing layer image, when the error of the two is within the allowable range, it is indicated that the accuracy of the third lesion feature set is high, and the third lesion feature set can be directly used for lesion marking on the missing layer image. Through the method of the embodiment, the accuracy of the lesion marking system in automatically marking the missing layer lesion can be further improved, so that the volume and size of the lesion calculated after a series of image lesion marking is completed are more accurate, and the subsequent treatment plan is not affected.

[0077] In combination with the above embodiments, in an implementation, the embodiment of the present application further provides a lesion marking method. In the method, each lesion feature set includes multiple attribute features: the center point coordinates of the lesion, the contour coordinates of the lesion, the area of the lesion, and the color of the lesion. Specifically, the method includes the following steps:

[0078] Step S31: statistical calculation is respectively performed on the same attribute features in the first lesion feature set and the second lesion feature set, to obtain statistical feature values of each same attribute feature, so as to constitute the third lesion feature set.

[0079] In the embodiment, each lesion feature set includes the following multiple attribute features: the center point coordinates of the lesion, the contour coordinates of the lesion, the area of the lesion, and the color of the lesion. For example, the first lesion feature set includes: the center point coordinates of the first lesion contour, the contour coordinates of the first lesion contour, the area of the first lesion contour, and the color of the lesion in the first lesion contour; and the second lesion feature set includes: the center point coordinates of the second lesion contour, the contour coordinates of the second lesion contour, the area of the second lesion contour, and the color of the lesion in the second lesion contour.

[0080] In implementation, the method for determining the third lesion feature set on the missing layer image according to the first lesion feature set and the second lesion feature set is that the lesion marking system respectively performs statistical calculation between each same attribute feature in the first lesion feature set and the second lesion feature set (for example, statistical calculation is performed between the center point coordinates of the first lesion contour in the first lesion feature set and the center point coordinates of the first lesion contour in the second lesion feature set, statistical calculation is performed between the contour coordinates of the first lesion contour in the first lesion feature set and the contour coordinates of the second lesion contour in the second lesion feature set, statistical calculation is performed between the area of the first lesion contour in the first lesion feature set and the area of the second lesion contour in the second lesion feature set, statistical calculation is performed between the color RGB value of the lesion in the first lesion contour in the first lesion feature set and the color RGB value of the lesion in the second lesion contour in the second lesion feature set, until the four attribute features in the lesion feature set are all calculated), to obtain the statistical feature values of each same attribute feature, so as to constitute the third lesion feature set.

[0081] In an optional embodiment, the statistical calculation in the present embodiment can be average value calculation. For example, average value calculation is performed between the center point coordinates of the first lesion contour in the first lesion feature set and the center point coordinates of the first lesion contour in the second lesion feature set, average value calculation is performed between the contour coordinates of the first lesion contour in the first lesion feature set and the contour coordinates of the second lesion contour in the second lesion feature set, average value calculation is performed between the area of the first lesion contour in the first lesion feature set and the area of the second lesion contour in the second lesion feature set, average value calculation is performed between the color RGB value of the lesion in the first lesion contour in the first lesion feature set and the color RGB value of the lesion in the second lesion contour in the second lesion feature set, until the four attribute features in the lesion feature set are all calculated, to respectively obtain the average center point coordinates of the lesion contour, the average contour coordinates of the lesion contour, the average area of the lesion contour and the average color RGB value of the lesion contour, so as to constitute the third lesion feature set.

[0082] Wherein, in the average profile coordinate calculation of the lesion profile, the four vertex coordinates (such as the left upper corner coordinate, the left lower corner coordinate, the right upper corner coordinate and the right lower corner coordinate) of the lesion profile are respectively taken to calculate the average value: the left upper corner coordinate of the first lesion profile and the left upper corner coordinate of the second lesion profile are taken to calculate the average value to obtain the average coordinate of the left upper corner of the profile; the right upper corner coordinate of the first lesion profile and the right upper corner coordinate of the second lesion profile are taken to calculate the average value to obtain the average coordinate of the right upper corner of the profile; the left lower corner coordinate of the first lesion profile and the left lower corner coordinate of the second lesion profile are taken to calculate the average value to obtain the average coordinate of the left lower corner of the profile; the right lower corner coordinate of the first lesion profile and the right lower corner coordinate of the second lesion profile are taken to calculate the average value to obtain the average coordinate of the right lower corner of the profile; thereby the four average coordinates (the average coordinate of the left upper corner of the profile, the average coordinate of the right upper corner of the profile, the average coordinate of the left lower corner of the profile and the average coordinate of the right lower corner of the profile) are obtained to constitute the average profile coordinate of the lesion profile. It should be noted that the coordinate system in the embodiment is the picture coordinate system under the unified screen resolution.

[0083] In the embodiment, the lesion marking system respectively performs statistical calculation on each same attribute feature in the first lesion profile and the second lesion profile to obtain the third lesion feature set, so that the lesion features of the intermediate missing layer image can be obtained according to the lesion features of the front and rear two layer images, and the user's marking operation is not required to obtain the relatively accurate lesion feature set of the missing layer image, which provides a basis for subsequent automatic delineation of the missing layer lesion.

[0084] In combination with the above embodiment, in an implementation mode, the embodiment of the present application further provides a lesion marking method. In the method, the step S22 comprises a step S41.

[0085] The step S41 comprises: respectively determining whether the deviation between each same attribute feature in the third lesion feature set and each same attribute feature in the fourth lesion feature set is within a preset threshold.

[0086] In the embodiment, the lesion marking system needs to calculate the deviation between each same attribute feature in the third lesion feature set and the fourth lesion feature set, and respectively determine whether the deviation between each same attribute feature in the third lesion feature set and the fourth lesion feature set is within a preset threshold.

[0087] In a specific implementation, for example, the lesion marking system calculates the deviation between the center point coordinates of the lesion contour in the third lesion feature set and the center point coordinates of the lesion contour in the fourth lesion feature set, determines whether the deviation between the two is within a preset threshold; calculates the deviation between the contour coordinates of the lesion contour in the third lesion feature set and the contour coordinates of the lesion contour in the fourth lesion feature set, determines whether the deviation between the two is within a preset threshold; calculates the deviation between the contour area of the lesion contour in the third lesion feature set and the contour area of the lesion contour in the fourth lesion feature set, determines whether the deviation between the two is within a preset threshold; and calculates the deviation between the color RGB value of the lesion contour in the third lesion feature set and the color RGB value of the lesion contour in the fourth lesion feature set, determines whether the deviation between the two is within a preset threshold.

[0088] In an example, the embodiment can determine the deviation between the same attribute features in the two feature sets through formula (2):

[0089]

[0090] When the same attribute feature in the third lesion feature set in formula (2) refers to the center point coordinates of the lesion contour in the third lesion feature set (i.e., the average coordinates of the left upper corner of the contour of the lesion contour in the aforementioned step S31), the same attribute feature in the fourth lesion feature set in formula (2) refers to the center point coordinates of the lesion contour in the fourth lesion feature set (i.e., the center point coordinates of the lesion contour automatically recognized by the system). When the same attribute feature in the third lesion feature set in formula (2) refers to the contour coordinates of the lesion contour in the third lesion feature set (i.e., the average contour coordinates of the lesion contour in the aforementioned step S31), the same attribute feature in the fourth lesion feature set in formula (2) refers to the contour coordinates of the lesion contour in the fourth lesion feature set (i.e., the contour coordinates of the lesion contour automatically recognized by the system). Similarly, the error between the lesion area and the color RGB value of the lesion between the third lesion feature set and the fourth lesion feature set is calculated and determined.

[0091] It should be noted that when calculating the error between the contour coordinates of the lesion contour between the third lesion feature set and the fourth lesion feature set, the four vertex coordinates (the coordinates of the left upper corner of the contour, the coordinates of the right upper corner of the contour, the coordinates of the left lower corner of the contour, and the coordinates of the right lower corner of the contour) of the two contours are calculated through formula (3) respectively,

[0092]

[0093] When the "coordinates of each point of the contour in the third lesion feature set" in formula (3) refers to the upper-left corner coordinates of the lesion contour in the third lesion feature set (i.e. the average upper-left corner coordinates of the lesion contour in the aforementioned step S31), the "coordinates of each point of the contour in the fourth lesion feature set" in formula (3) refers to the upper-left corner coordinates of the lesion contour in the fourth lesion feature set (i.e. the upper-left corner coordinates of the lesion contour automatically recognized by the system). When the "coordinates of each point of the contour in the third lesion feature set" in formula (3) refers to the lower-left corner coordinates of the lesion contour in the third lesion feature set (i.e. the average lower-left corner coordinates of the lesion contour in the aforementioned step S31), the "coordinates of each point of the contour in the fourth lesion feature set" in formula (3) refers to the lower-left corner coordinates of the lesion contour in the fourth lesion feature set (i.e. the lower-left corner coordinates of the lesion contour automatically recognized by the system). And so on, until the error between the upper-right corner coordinates and the lower-right corner coordinates of the contour between the third lesion feature set and the fourth lesion feature set is calculated to determine the error between the coordinates of the contour.

[0094] In the present method, step S23 comprises step S42.

[0095] Step S42: in the case that the errors of each same attribute feature between the two lesion feature sets (the third lesion feature set and the fourth lesion feature set) calculated in step S41 are all within the preset threshold value (including the error being equal to the preset threshold value), the lesion contour in the missing layer image is outlined and the lesion is marked according to the third lesion feature set.

[0096] In the present embodiment, in the case that the errors (i.e. the error between the center point coordinates, the error between the contour coordinates, the error between the lesion areas, and the error between the lesion color RGB values) of each same attribute feature between the two lesion feature sets (the third lesion feature set and the fourth lesion feature set) calculated in step S41 are all within the preset threshold value (including the error being equal to the preset threshold value), it is determined that the feature values of the third lesion feature set are accurate, and the third lesion feature set can be used to outline the lesion in the missing layer image.

[0097] In the present embodiment, in the case that the errors (i.e. the error between the center point coordinates, the error between the contour coordinates, the error between the lesion areas, and the error between the lesion color RGB values) of each same attribute feature between the two lesion feature sets (the third lesion feature set and the fourth lesion feature set) calculated in step S41 are all within the preset threshold value (including the error being equal to the preset threshold value), it is determined that the feature values of the third lesion feature set are accurate, and the third lesion feature set can be used to outline the lesion in the missing layer image.

[0098] In the embodiment, the lesion marking system respectively judges the errors between the same attribute features in the third lesion contour and the fourth lesion contour. If the errors between the same attribute features in the third lesion contour and the fourth lesion contour both satisfy the preset threshold, it indicates that the feature values of the third lesion feature set are accurate, and the third lesion feature set can be used to perform lesion delineation of the missing layer image. Through the method of the embodiment, a relatively accurate lesion feature set of the missing layer image can be obtained without the marking operation of the user, thereby providing a basis for subsequent automatic delineation of the missing layer lesion, so as to further improve the accuracy of automatic lesion marking of the missing layer by the lesion marking system, and make the volume and size of the lesion calculated after a series of image lesion marking more accurate, which does not affect the subsequent treatment plan.

[0099] In combination with the above embodiments, in an implementation, the embodiment of the present application further provides a lesion marking method. Specifically, the method further includes the following steps:

[0100] Step S51: automatically identifying a reference lesion contour of each layer image for each layer image.

[0101] In the embodiment, the lesion marking system performs automatic lesion identification operation on each layer image in a series of images, thereby identifying the reference lesion contour of each layer image.

[0102] Step S52: performing feature extraction on the image region framed by the reference lesion contour to obtain a reference feature set; and performing lesion marking for the user to reference according to the reference lesion contour delineated on each layer image based on the reference feature set.

[0103] In the embodiment, after the lesion marking system identifies the reference lesion contour of each layer image, the image region framed by the reference lesion contour is respectively subjected to feature extraction, a plurality of features corresponding to the reference lesion contour are obtained, and the reference feature set of the reference lesion contour on each layer image is constructed. The lesion marking system performs the delineation of the reference lesion contour on each layer image based on each feature in the obtained reference feature set, and displays the feature information beside the delineated reference lesion contour, so as to provide a reference for the user to select the lesion contour when performing the lesion contour delineation layer by layer, and also to obtain the fourth lesion feature set for the missing layer image.

[0104] In the embodiment, the lesion marking system automatically performs lesion identification and lesion contour delineation and display for each layer image, so as to provide a reference for the user (such as a doctor) to select the lesion contour when performing the lesion contour delineation layer by layer, thereby realizing fast and convenient lesion contour delineation layer by layer. It can be understood that, however, the effect of automatic lesion contour identification by the system is not as good as that of manual lesion contour delineation by the user, but the embodiment combines the automatic and manual double lesion contour delineation modes, thereby improving the lesion marking efficiency of the user and the accuracy of the lesion marking.

[0105] In combination with the above embodiments, in an implementation, the embodiments of the present application further provide a lesion marking method. Specifically, the method further comprises the following steps:

[0106] Step S61: in the case that the deviation is not within the preset threshold, displaying prompt information for prompting the user to manually outline the lesion contour on the missing layer image and perform lesion marking.

[0107] In the present embodiment, when the lesion marking system determines that the deviation between the third lesion feature set and the fourth lesion feature set is not within the preset threshold, it indicates that the error of the third lesion feature set obtained by the lesion marking system and the actual lesion contour on the missing layer image is large, and the lesion marking on the missing layer image according to the third lesion feature set cannot guarantee the accuracy of the automatic lesion marking, nor can it guarantee the accuracy of the volume and size of the lesion calculated after the lesion marking of the series of images is completed. Marking according to the third lesion feature set will still affect the subsequent treatment plan.

[0108] Therefore, in the case that the lesion marking system determines that the deviation between the third lesion feature set and the fourth lesion feature set is not within the preset threshold, the lesion marking system directly displays prompt information for prompting the user to manually outline the lesion contour on the missing layer image and perform lesion marking. For example, display the prompt information "Layer N image is not marked with lesion, please manually mark it". Wherein, the Nth layer image is the missing layer image determined by the system. And in a preferred embodiment, the lesion marking system can always display the prompt information until the user completes the manual lesion marking of the missing layer image (i.e. the image indicated by the prompt information), so as to ensure that the series of examination images will not have the situation of "missing layer, unmarked lesion".

[0109] In the present embodiment, when the third lesion feature set obtained by the system does not satisfy the condition, the system displays prompt information to the user to remind the user to perform manual lesion marking on the missing layer image, thereby not only avoiding the situation of missing layer in the lesion marking process of the user, but also further improving the accuracy of lesion marking.

[0110] It should be noted that for the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiments of the present application.

[0111] Based on the same inventive concept, an embodiment of the present application provides a lesion marking device 200. Referring to Figure 2 , Figure 2 is a structural block diagram of the lesion marking device provided by an embodiment of the present application. As shown in Figure 2 , the device 200 comprises:

[0112] a display module 201 configured to sequentially display each image layer according to the image layer order;

[0113] a response module 202 configured to obtain a first lesion contour and a second lesion contour respectively in response to a user's continuous contour marking operation on the two image layers;

[0114] a feature extraction module 203 configured to perform feature extraction on the image region framed by the first lesion contour and the second lesion contour respectively, and obtain a first lesion feature set and a second lesion feature set;

[0115] a first determination module 204 configured to determine whether there is a missing image layer between the two image layers according to the image layer of each of the two image layers;

[0116] a feature determination module 205 configured to determine a third lesion feature set on the missing image layer according to the first lesion feature set and the second lesion feature set in the case that there is a missing image layer between the two image layers;

[0117] a first marking module 206 configured to delineate a lesion contour and mark a lesion on the missing image layer according to the third lesion feature set.

[0118] Optionally, the device 200 further comprises:

[0119] an acquisition module configured to acquire a fourth lesion feature set on the missing image layer, the fourth lesion feature set being a lesion feature set obtained by automatic lesion identification on the missing image layer;

[0120] the first marking module 206 comprises:

[0121] a second determination module configured to determine whether the deviation between the third lesion feature set and the fourth lesion feature set is within a preset threshold;

[0122] a second marking module configured to delineate a lesion contour and mark a lesion on the missing image layer according to the third lesion feature set in the case that the deviation is within the preset threshold.

[0123] Optionally, the device 200 comprises: each lesion feature set comprises a plurality of attribute features: the center point coordinates of the lesion, the contour coordinates of the lesion, the area of the lesion, and the color of the lesion.

[0124] The feature determination module 205 includes:

[0125] The feature statistics module is used to perform statistical calculations on the same attribute features in the first lesion feature set and the second lesion feature set respectively, to obtain the statistical feature values ​​of each same attribute feature, so as to constitute the third lesion feature set.

[0126] Optionally, the second determination module includes:

[0127] The third determination module is used to determine whether the deviation between the same attribute features in the third lesion feature set and the same attribute features in the fourth lesion feature set is within a preset threshold.

[0128] The second marking module includes:

[0129] The third marking module is used to outline the lesion contour and mark the lesion on the missing layer image according to the third lesion feature set, provided that the errors of each identical attribute feature are within a preset threshold.

[0130] Optionally, the device 200 further includes:

[0131] An automatic identification module is used to automatically identify the reference lesion contour of each image layer;

[0132] The fourth marking module is used to outline the lesion contour and mark the lesion on the missing layer image according to the third lesion feature set, provided that the errors of each identical attribute feature are within a preset threshold.

[0133] Optionally, the device 200 further includes:

[0134] The information prompting module is used to display prompting information when the deviation is not within a preset threshold. The prompting information is used to prompt the user to manually outline the lesion and mark the lesion on the missing layer image.

[0135] Based on the same inventive concept, another embodiment of the present invention provides an electronic device 300, such as... Figure 3 As shown. Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a memory 302, a processor 301, and a computer program stored in the memory and executable on the processor. When executed by the processor, the computer program implements the lesion marking method described in any of the above embodiments of the present invention.

[0136] Based on the same inventive concept, another embodiment of the present application provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the lesion marking method in any of the above embodiments of the present application.

[0137] Each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0138] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. In addition, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] The embodiments of the present application are described with reference to flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flow(s) or block(s).

[0140] These computer program instructions can also be stored in a computer readable memory capable of directing the computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flow(s) or block(s).

[0141] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer implemented process, so that the instructions executed on the computer or other programmable terminal device provide a process for implementing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocksFigure 1 the functions specified in the one or more blocks.

[0142] While preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to the described embodiments without departing from the scope of the application. Accordingly, the appended claims are intended to encompass within their scope all such modifications and variations.

[0143] Finally, it should be noted that, in this document, the term "only" is used to set off at least one particular element from another; however, this term should not be interpreted in the broadest sense as it can be employed to create a set of elements that is limited only to the particular recited elements. Moreover, the terms "include," "have," and "exist" are intended to be interpreted as "comprising," "including," or "containing," respectively, such that a process, method, article, or apparatus that "includes," "has," or "exists" one step includes, has or exists not only that step but also other steps not expressly listed or inherent to such process, method, article, or apparatus. Stated another way, elements preceded by "comprising," "including," or "containing" together with another element are intended in the context to imply the inclusion of that other element in the process, method, article, or apparatus but not to those other elements that are inherent in the process, method, article, or apparatus or that can be added to the process, method, article, or apparatus without now having a basic and novel characteristic over processes, methods, articles, or apparatuses of that type. Stated another way, a process, method, article, or apparatus that "includes," "has," or "exists" a step is not limited as "a process, method, article, or apparatus that only includes," "only has," or "only exists" that step.

[0144] The above provides a lesion marking method and device, electronic equipment and readable storage medium. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method and core idea of the present application. For those skilled in the art, according to the idea of the present application, the specific implementation manners and application scope can be changed. The above description of the present application should not be understood as a limitation.

Claims

1. A method of lesion marking, characterized by, The method comprises: displaying each layer of images in order of image layer; in response to a user continuously marking the contours of a lesion on two layers of images, obtaining a first lesion contour and a second lesion contour, respectively; performing feature extraction on the image area framed by each of the first lesion contour and the second lesion contour, respectively, to obtain a first lesion feature set and a second lesion feature set; determining whether there is a missing layer of images between the two layers of images according to the image layer of each of the two layers of images; in the case where there is a missing layer of images between the two layers of images, determining a third lesion feature set on the missing layer of images according to the first lesion feature set and the second lesion feature set; according to the third lesion feature set, outlining a lesion contour and marking a lesion on the missing layer of images; The method further comprises: automatically identifying a reference lesion contour of each layer of images for each layer of images; performing feature extraction on the image area framed by the reference lesion contour to obtain a reference feature set; according to the reference feature set, outlining a display reference lesion contour on each layer of images and marking a lesion for user reference.

2. The method of claim 1, wherein, The method further comprises: obtaining a fourth lesion feature set on the missing layer of images, the fourth lesion feature set being a lesion feature set obtained by automatic lesion identification on the missing layer of images; the according to the third lesion feature set, outlining a lesion contour and marking a lesion on the missing layer of images, comprising: determining whether the deviation between the third lesion feature set and the fourth lesion feature set is within a preset threshold; in the case where the deviation is within the preset threshold, according to the third lesion feature set, outlining a lesion contour and marking a lesion on the missing layer of images.

3. The method of claim 2, wherein, Each lesion feature set comprises a plurality of attribute features: the center point coordinates of the lesion, the contour coordinates of the lesion, the area of the lesion, and the color of the lesion; the according to the first lesion feature set and the second lesion feature set, determining a third lesion feature set on the missing layer of images, comprising: respectively performing statistical calculation on the same attribute features in the first lesion feature set and the second lesion feature set to obtain statistical feature values of each same attribute feature to constitute the third lesion feature set.

4. The method of claim 3, wherein, the determining whether the deviation between the third lesion feature set and the fourth lesion feature set is within a preset threshold, comprising: respectively determining whether the deviation between the same attribute features in the third lesion feature set and the same attribute features in the fourth lesion feature set is within a preset threshold; the in the case where the deviation is within the preset threshold, according to the third lesion feature set, outlining a lesion contour and marking a lesion on the missing layer of images, comprising: in the case where the error of each same attribute feature is within the preset threshold, according to the third lesion feature set, outlining a lesion contour and marking a lesion on the missing layer of images.

5. The method according to any one of claims 2 to 4, characterized in that, The method further comprises: in the case where the deviation is not within the preset threshold, displaying prompt information, the prompt information being used to prompt a user to manually outline a lesion contour and mark a lesion on the missing layer of images.

6. A lesion marker device, comprising: The device comprises: a display module configured to display each image layer in sequence; a response module configured to obtain a first lesion contour and a second lesion contour in response to a user marking the contours of the lesion on two image layers in succession; a feature extraction module configured to extract features from image regions framed by the first lesion contour and the second lesion contour respectively to obtain a first lesion feature set and a second lesion feature set; a first determination module configured to determine whether there is a missing image layer between the two image layers according to image layer numbers of the two image layers; a feature determination module configured to determine a third lesion feature set on the missing image layer according to the first lesion feature set and the second lesion feature set when there is a missing image layer between the two image layers; a first marking module configured to delineate a lesion contour and mark a lesion on the missing image layer according to the third lesion feature set; the apparatus further comprises: an automatic identification module configured to automatically identify a reference lesion contour of each image layer from each image layer; a fourth marking module configured to delineate a lesion contour and mark a lesion on the missing image layer according to the third lesion feature set when errors of features of the same attribute are all within a preset threshold.

7. The apparatus of claim 6, wherein, the apparatus further comprises: an acquisition module configured to acquire a fourth lesion feature set on the missing image layer, the fourth lesion feature set being a lesion feature set obtained by automatically identifying a lesion from the missing image layer; the first marking module comprises: a second determination module configured to determine whether a deviation between the third lesion feature set and the fourth lesion feature set is within a preset threshold; a second marking module configured to delineate a lesion contour and mark a lesion on the missing image layer according to the third lesion feature set when the deviation is within the preset threshold. 8.An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is executed by the processor to implement the lesion marking method according to any one of claims 1 to 5. 9.A computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the lesion marking method according to any one of claims 1 to 5.

Images