A lesion detection method, readable storage medium and electronic device

Abstract

The specification discloses a lesion detection method, a readable storage medium and an electronic device. The method comprises the following steps: registering a breast region in a breast projection image and a breast X-ray image to obtain a coordinate conversion relationship between the breast projection image and the breast X-ray image; converting a first region in which a lesion is located in the breast projection image and a second region in which the lesion is located in the breast X-ray image into a same coordinate system according to the coordinate conversion relationship; and determining a lesion detection result according to the first region and the second region in the same coordinate system. The method determines a more accurate lesion detection result by using the breast three-dimensional image and the breast X-ray image together, and improves the accuracy of lesion detection.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of medical technology, and in particular to a lesion detection method, a readable storage medium and an electronic device. BACKGROUND

[0002] With the development of computer technology and the need for in-depth business integration, in the medical field, image collected by a medical image system such as a full-field digital mammography (FFDM) system or a digital breast tomosynthesis (DBT) system is used for lesion detection, which has become one of the commonly used medical detection methods.

[0003] Currently, a doctor usually needs to obtain a breast X-ray FFDM image of a patient or a breast three-dimensional DBT image of the patient, and performs target object recognition on the obtained image to determine a lesion in the patient's body and a position corresponding to the lesion according to a recognition result, so as to perform pathological sampling and other businesses.

[0004] However, due to the low accuracy of the existing technology in target object recognition, the accuracy of the lesion detection result obtained by the existing technology is also low. Based on this, the present specification provides a new lesion detection method. SUMMARY

[0005] The present specification provides a lesion detection method, a readable storage medium and an electronic device to partially solve the above problems existing in the prior art.

[0006] The present specification adopts the following technical solutions:

[0007] The present specification provides a lesion detection method, comprising:

[0008] obtaining a breast three-dimensional DBT image, and determining a breast projection image according to the breast three-dimensional image, and determining a first region corresponding to a lesion in the breast projection image;

[0009] obtaining a breast X-ray FFDM image, and determining a second region corresponding to the lesion in the breast X-ray image according to the breast X-ray image;

[0010] According to the breast projection image and the breast X-ray image, the breast region in the breast projection image and the breast X-ray image is registered as a target, a coordinate conversion relationship between the breast projection image and the breast X-ray image is determined, and the first region and the second region are converted into a target coordinate system according to the coordinate conversion relationship;

[0011] In the target coordinate system, the first region and the second region are matched, and a lesion detection result is determined according to a matching result.

[0012] The acquisition angle and the acquisition position of the breast three-dimensional image and the breast X-ray image are the same.

[0013] Optionally, before determining the coordinate conversion relationship between the breast projection image and the breast X-ray image, the method further comprises:

[0014] According to a target voxel spacing, the size of the breast X-ray image and the size of the breast projection image are adjusted, so that the voxel spacing of the breast X-ray image and the voxel spacing of the breast projection image are the same.

[0015] Optionally, the breast X-ray image and the breast projection image of the patient are acquired, and specifically comprising:

[0016] An initial breast X-ray image and an initial breast three-dimensional image are acquired, and an initial breast projection image is determined according to the initial breast three-dimensional image.

[0017] A breast region contained in the initial breast projection image is determined, and the initial breast projection image is cropped according to a minimum rectangle containing the breast region, and the cropped initial breast projection image is taken as the breast projection image of the patient.

[0018] A breast region contained in the initial breast X-ray image is determined, and the initial X-ray image is cropped according to a minimum rectangle containing the breast region, and the cropped initial breast X-ray image is taken as the breast X-ray image of the patient.

[0019] Optionally, in the target coordinate system, the first region and the second region are matched, and specifically comprising:

[0020] According to the coordinates of the first region and the coordinates of the second region in the target coordinate system, an intersection-over-union of the first region and the second region is determined.

[0021] It is judged whether the intersection-over-union is greater than a preset matching threshold.

[0022] If yes, it is determined that the lesions corresponding to the first region and the second region are the same lesion.

[0023] Optionally, the first region corresponding to the lesion in the breast projection image is determined, and specifically comprising:

[0024] The breast projection image is used as input to a pre-trained first recognition model to obtain the first region corresponding to each lesion output by the first recognition model, and the lesion type corresponding to each first region.

[0025] Determining the second region corresponding to the lesion in the mammogram specifically includes:

[0026] The mammogram image is used as input to a pre-trained second recognition model to obtain the second region corresponding to each lesion output by the second recognition model, and the lesion type corresponding to each second region.

[0027] Optionally, in the target coordinate system, the first region and the second region are matched, and the lesion detection result is determined based on the matching result, specifically including:

[0028] For each lesion type, determine the corresponding first region and second region;

[0029] In the target coordinate system, for each first region corresponding to the lesion type, it is determined whether there is a second region that matches the first region;

[0030] If so, then the second region that matches the first region and the lesion corresponding to the first region are determined to be the same lesion;

[0031] The first region and its corresponding lesion, as well as the second region and its corresponding lesion, are used as the lesion detection results.

[0032] Optionally, determining that the second region matching the first region and the lesion corresponding to the first region are the same lesion specifically includes:

[0033] When there are multiple second regions that match the first region, the matching degree between each second region and the first region is determined. The matching degree is determined based on at least one of the following: the intersection-union ratio of the second region and the first region, the similarity between the image of the second region and the image of the first region, and the confidence level of the second region.

[0034] From all the second regions that match the first region, select the second region with the highest matching degree as the target second region;

[0035] The lesions corresponding to the first region and the target second region are determined to be the same lesion.

[0036] Optionally, according to the breast projection image and the breast X-ray image, a coordinate conversion relationship between the breast projection image and the breast X-ray image is determined by taking the breast region in the breast projection image and the breast X-ray image as a registration target, and specifically includes:

[0037] The breast projection image and the breast X-ray image are taken as inputs to input a pre-trained registration model;

[0038] A coordinate conversion relationship between the breast projection image and the breast X-ray image output by the registration model is determined;

[0039] The registration model is trained by using the following method:

[0040] A breast three-dimensional image and a breast X-ray image collected at the same body position and the same angle are obtained;

[0041] According to the breast three-dimensional image, a breast projection image before registration is determined, first sample features of the breast projection image before registration are determined, and second sample features of the breast X-ray image are determined;

[0042] The breast projection image before registration, the breast X-ray image, the first sample features, and the second sample features are taken as inputs to input a registration model to be trained, so that a to-be-determined conversion relationship output by the registration model is obtained;

[0043] According to the to-be-determined conversion relationship and the breast projection image before registration, a breast projection image after registration is obtained, and third sample features of the breast projection image after registration are determined;

[0044] According to a difference between the breast projection image after registration and the breast X-ray image and a difference between the second sample features and the third sample features, a loss is determined, and the registration model is trained by taking a minimum loss as an optimization target.

[0045] The present specification provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned lesion detection method.

[0046] The present specification provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned lesion detection method when executing the program.

[0047] The above-mentioned at least one technical solution adopted by the present specification can achieve the following beneficial effects:

[0048] The coordinate conversion relationship between the breast projection image and the breast X-ray image is obtained by registering the breast region in the breast projection image and the breast X-ray image, and then the first region in which the lesion is located in the breast projection image and the second region in which the lesion is located in the breast X-ray image are converted into the same coordinate system according to the coordinate conversion relationship, and the lesion detection result is determined according to the first region and the second region in the same coordinate system.

[0049] From the above, it can be seen that the method determines more accurate lesion detection results by using the breast three-dimensional image and the breast X-ray image together, and improves the accuracy of lesion detection. BRIEF DESCRIPTION OF DRAWINGS

[0050] The accompanying drawings, which are included to provide a further understanding of the present description, constitute a part of the present description, and the illustrative embodiments of the present description and their description serve to explain the present description, and do not constitute improper limitations on the present description. In the drawings:

[0051] Figure 1 A flowchart of the lesion detection method provided in the present description is shown;

[0052] Figure 2 A flowchart of the lesion detection method provided in the present description is shown;

[0053] Figure 3 A flowchart of the lesion detection method provided in the present description is shown;

[0054] Figure 4 A flowchart of the lesion detection method provided in the present description is shown;

[0055] Figure 5 A flowchart of the lesion detection method provided in the present description is shown;

[0056] Figure 6 A flowchart of the lesion detection method provided in the present description is shown; Figure 1 A flowchart of the lesion detection method provided in the present description is shown; DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the present description clearer, the technical scheme of the present description will be described in detail below in combination with the specific embodiments of the present description and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present description, not all the embodiments. Based on the embodiments in the present description, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present description.

[0058] The technical scheme provided by each embodiment of the present description will be described in detail below in combination with the drawings.

[0059] Generally, in the field of lesion detection technology, during breast examination or breast treatment, lesion detection needs to be performed based on the obtained breast image, and based on the lesion detection result, pathological sampling and other operations are performed on the patient, so as to complete the breast examination or breast treatment of the patient. However, due to the pathological sampling and other operations, the patient will be damaged to a certain extent, and how to determine the accurate lesion detection result has become one of the problems to be solved in the field of lesion detection technology.

[0060] Therefore, unlike the current method of determining the lesion detection result based on only the three-dimensional image of the patient's breast or the X-ray image of the patient's breast to identify the target object, the precision is low. The present specification provides a new lesion detection method, which determines whether the lesion is a real lesion by determining the corresponding regions of the lesion in the three-dimensional image of the breast and the X-ray image of the breast. By using the three-dimensional image of the breast and the X-ray image of the breast together to determine a more accurate lesion detection result, the accuracy of lesion detection is improved.

[0061] Figure 1 A flowchart of a lesion detection method provided in the present specification is provided, which specifically includes the following steps:

[0062] S100: Obtain a three-dimensional DBT image of a breast, and determine a breast projection image according to the three-dimensional image of the breast, and determine a first region corresponding to a lesion in the breast projection image.

[0063] In one or more embodiments provided in the present specification, the lesion detection method can be executed by a terminal, for example, the terminal can be a personal computer, a server, etc., and of course can also be other electronic devices with computing and image processing capabilities, which are not limited in the present specification.

[0064] Since the three-dimensional image of the breast is a three-dimensional image, the region where the lesion is located determined based on the three-dimensional image of the breast is of course a three-dimensional region. The X-ray image of the breast is a two-dimensional image, so the region where the lesion is located determined based on the X-ray image of the breast is of course a two-dimensional region. If an accurate lesion matching result is to be determined, it is necessary to unify the dimensions of the region where the lesion is located. Therefore, in order to determine a more accurate lesion detection result, the terminal can also determine a breast projection image based on the three-dimensional image of the breast, and determine a lesion detection result based on the region where the lesion is located in the breast projection image.

[0065] Specifically, the terminal can first randomly obtain any breast three-dimensional image from the pre-stored breast three-dimensional images as the breast three-dimensional image of the patient. The breast three-dimensional image can be obtained by collecting the breast at a preset angle and a preset body position. The body position can be the body position of the patient when the medical image is taken, specifically the body position when the mammogram is taken. For example, inner-outer oblique lateral position, breast up-down axial position, breast 90° lateral position, etc. The angle is the angle of the patient when the medical image is taken, for example, left side shooting, right side shooting, etc.

[0066] Then, the terminal can determine the breast projection image according to the breast three-dimensional image of the patient. The breast projection image can be a maximal intensity projection (MIP) image, and can also be a minimum intensity projection image. Of course, the breast projection image can also be a projection image determined based on a light ray projection algorithm such as an accumulated density algorithm. That is, the breast three-dimensional image is projected according to a preset projection direction, and for the coordinate system in which the breast projection image is located, for each coordinate, the pixel value corresponding to the coordinate is accumulated to obtain the breast projection image. The specific type of the breast projection image and how to determine it can be set as needed, and the present specification does not limit this.

[0067] Finally, the terminal can identify the breast projection image according to the determined breast projection image, and determine the first region where the lesion is located in the breast projection image. The shape of the first region can be a regular shape such as a circle or a rectangle, or an irregular shape. The shape and size of the first region can be set as needed, and the present specification does not limit this.

[0068] Of course, the step of determining the breast three-dimensional image of the patient by the terminal can also be that the terminal sends an acquisition instruction to the acquisition device that takes the breast three-dimensional image. The acquisition device that takes the breast three-dimensional image returns the breast three-dimensional image of the patient to the terminal according to the received acquisition instruction. The specific determination of the breast three-dimensional image can be set as needed, and the present specification does not limit this.

[0069] S102: Obtain a breast X-ray FFDM image, and determine a second region corresponding to the lesion in the breast X-ray image according to the breast X-ray image.

[0070] In one or more embodiments provided in the present specification, in order to accurately detect the lesion based on the breast X-ray image and the identification result of the breast projection image, the lesions respectively present in the breast X-ray image and the breast projection image need to be determined first. Therefore, the terminal can obtain the breast X-ray image of the patient and determine the lesion in the breast X-ray image.

[0071] Specifically, the terminal can acquire a breast X-ray image acquired at the same angle and the same body position as the breast three-dimensional image in step S100.

[0072] Then, the terminal can identify the breast X-ray image to determine a second region in which the lesion is located in the breast X-ray image. Similarly, the shape and size of the second region and how to determine the breast X-ray image can be set as needed, and the present specification does not limit this.

[0073] S104: According to the breast projection image and the breast X-ray image, the coordinate conversion relationship between the breast projection image and the breast X-ray image is determined by registering the breast regions in the breast projection image and the breast X-ray image as the target, and the first region and the second region are converted into a target coordinate system according to the conversion relationship.

[0074] In one or more embodiments provided in the present specification, the terminal can determine the lesion detection result based on the fusion of the first region in which the lesion is located in the breast projection image and the second region in which the lesion is located in the breast X-ray image, so as to improve the accuracy of lesion detection. However, the position of the same lesion in the breast projection image and the position of the same lesion in the breast X-ray image may be different, resulting in the inability to fuse according to the identification result. Therefore, before determining the lesion detection result, the first lesion and the second lesion also need to be converted into the same target coordinate system.

[0075] Therefore, the coordinate conversion relationship between the breast projection image and the breast X-ray image can be determined based on the breast regions in the breast projection image and the breast X-ray image.

[0076] Specifically, the terminal can determine the initial conversion relationship between the breast X-ray image and the breast projection image according to the acquisition posture of the breast X-ray image and the breast projection image.

[0077] Then, the terminal can convert the breast projection image into the first coordinate system in which the breast X-ray image is located according to the initial conversion relationship, and determine the initial projection result of the breast projection image in the first coordinate system.

[0078] Finally, the terminal can update the conversion relationship according to the difference between the initial projection result and the breast X-ray image. And according to the updated conversion relationship, the projection result of the breast projection image in the first coordinate system is determined again. And according to the projection result, the conversion relationship is continuously updated until the difference between the determined projection result and the breast X-ray image is less than a preset first threshold.

[0079] In the process of updating the conversion relationship according to the gap, the conversion relationship can be adjusted according to the gap. If the gap becomes smaller, the conversion relationship is updated. If the gap does not change or becomes larger, the conversion relationship is updated.

[0080] Of course, the above-mentioned coordinate conversion relationship can also be determined based on the gap between the projection results of the breast X-ray image and the breast projection image in the second coordinate system.

[0081] As mentioned above, the lesion detection result needs to be determined based on the first region and the second region in the same coordinate system in the present specification. Therefore, after the coordinate conversion relationship is determined, the terminal can convert the first region and the second region into a target coordinate system according to the coordinate conversion relationship. The target coordinate system can be the first coordinate system in which the breast X-ray image is located, or the second coordinate system in which the breast projection image is located, or other coordinate systems other than the above-mentioned two coordinate systems. The specific type of coordinate system corresponding to the target coordinate system can be set as needed, and the present specification does not limit this.

[0082] S106: In the target coordinate system, the first region and the second region are matched, and a lesion detection result is determined according to the matching result.

[0083] In one or more embodiments provided in the present specification, generally, if a lesion is detected in the same region in the breast X-ray image and the breast projection image, it is very likely that there is a real lesion in the region. Similarly, if the lesion coincides in the first region in the breast X-ray image and the second region in the breast projection image, the probability that the lesion is a real lesion is greater.

[0084] Therefore, the terminal can match the first region and the second region after converting them into a coordinate system, and determine a lesion detection result according to the matching result.

[0085] Specifically, the terminal can determine whether the first region and the second region in the target coordinate system coincide.

[0086] If yes, the terminal can determine that the first region and the lesion corresponding to the first region are real lesions.

[0087] If no, the terminal can determine that the lesion corresponding to the first region and the second region is not a real lesion.

[0088] Therefore, the terminal can take the aforementioned determination result of whether the lesion is a real lesion as a lesion detection result.

[0089] Further, due to the differences in size, breast shape, and other factors between the mammogram and the breast projection image, the first region and the second region of the lesion in the same coordinate system based on the determined coordinate conversion relationship often cannot completely coincide. Based on this, the terminal can also be provided with a second threshold.

[0090] Therefore, the terminal can determine the center positions of the first region and the second region according to the coordinates of the first region and the coordinates of the second region, and determine the matching degree of the first region and the second region.

[0091] After determining the matching degree, the terminal can compare the matching degree with the second threshold. In an embodiment provided in the present specification, if the matching degree is higher than the preset second threshold, the terminal can regard the lesion as a real lesion. If the matching degree is lower than the preset second threshold, the terminal can regard the lesion as a non-real lesion. The specific value of the threshold can be set as needed, and the present specification does not limit this.

[0092] Further, since the first region and the second region can be irregular shapes, in this case, the step of determining the region centers of the first region and the second region has a higher requirement for computing resources. In order to ensure that the lesion detection result can still be quickly and accurately determined in the case where the first region and the second region are irregular shapes, the terminal can directly determine the intersection-over-union of the first region and the second region, and determine the lesion detection result according to the intersection-over-union.

[0093] Specifically, the terminal can also directly determine the intersection-over-union of the first region and the second region in the target coordinate system according to the coordinates of the first region and the coordinates of the second region.

[0094] Secondly, the terminal can determine whether the intersection-over-union is greater than a preset matching threshold.

[0095] In an embodiment provided in the present specification, if yes, the terminal can determine that the lesion is a real lesion, and regard the first region and the second region as the lesion region. If no, the terminal can determine that the lesion is a non-real lesion, and the terminal can not regard the first region and the second region as the lesion region.

[0096] In addition, in the present specification, for each first region and each second region, the first region and the second region are both the determined region where the lesion is located. Therefore, if the first region and the second region are not matched, and the first region and / or the second region are not taken as the lesion region, some lesions may be missed. In order to avoid the above situation, in another embodiment provided in the present specification, the terminal can also take the first region and the second region matched with each other as the lesion region corresponding to the same lesion, and for the first region and the second region that are not matched, the terminal can not process them.

[0097] Specifically, for each first region, when it is determined that there is a second region matched with the first region, and the intersection-over-union of the first region and the second region is greater than a preset matching threshold, or it is determined that there is a second region matched with the first region, and the matching degree of the first region and the second region is greater than a preset second threshold, the terminal can take the first region and the second region as the lesion region corresponding to the same lesion.

[0098] Therefore, after determining each first region and the lesion corresponding to each first region, and each second region and the lesion corresponding to each second region, the terminal can directly take each first region and its lesion, and each second region and its corresponding lesion as the lesion detection result.

[0099] Of course, the terminal can also determine, for each lesion, the lesion region corresponding to the lesion, i.e., the first region and / or the second region. Then, the terminal can take each lesion and the lesion region corresponding to each lesion as the lesion detection result.

[0100] It should be noted that the lesion detection result not only contains the first region and the second region matched with each other, but also contains the second region not matched with each first region, and / or the first region not matched with each second region.

[0101] Further, for each lesion, the more lesion regions corresponding to the lesion, the greater the possibility that the lesion is a real lesion region. Therefore, the terminal can set a confidence level for each first region and each second region, and determine the lesion detection result based on the confidence level.

[0102] Therefore, for each first region, if there is a second region matched with the first region, the terminal can increase the confidence level that the first region and the second region belong to the lesion region. If there is no second region matched with the first region, the terminal can decrease the confidence level that the first region belongs to the lesion region.

[0103] Of course, the terminal only increases the confidence that the first region and the second region matching the first region belong to the lesion region, or only decreases the confidence that the first region belongs to the lesion region. How to adjust the confidence based on the matching result can be set as needed, and the present specification does not limit this.

[0104] After adjusting the confidence, the terminal can store each first region and the confidence corresponding to each first region, and each second region and the confidence corresponding to each second region, as a lesion detection result.

[0105] When adjusting the confidence of each first region and each second region, the terminal can preset three levels of low confidence, medium confidence, and high confidence. The initial confidence of each first region and each second region is set to the medium confidence level. For each first region and / or each second region, when the confidence of the first region and / or the second region needs to be increased, the confidence of the first region and / or the second region is adjusted to high confidence. When the confidence of the first region and / or the second region needs to be decreased, the confidence of the first region and / or the second region is adjusted to low confidence. The initial confidence is the confidence set for the first region and / or the second region when the first region and / or the second region is determined.

[0106] Of course, the terminal can also preset the numerical value corresponding to each initial confidence. For example, the initial confidence of each first region and each second region is set to 50%. For each first region and / or each second region, when the confidence of the first region and / or the second region needs to be increased, the confidence of the first region and / or the second region is increased by a preset confidence, such as an increase of 5% confidence. When the confidence of the first region and / or the second region needs to be decreased, the confidence of the first region and / or the second region is decreased by a preset confidence, such as a decrease of 5% confidence. Alternatively, for each first region, when there is a second region matching the first region, the terminal can determine the matching degree of the first region and the second region, and increase the confidence that the first region and the second region belong to the lesion region according to the matching degree. For example, the matching degree is 86%, and according to a preset matching degree threshold of 80%, the terminal can increase the confidence that the first region and the second region belong to the lesion region by 6%. Of course, the terminal can also only increase the confidence that the first region and the second region belong to the lesion region by 8.6% according to 0.1 times the matching degree of 86%. How the terminal adjusts the confidence of each first region and second region, the numerical value of the adjustment or the level of the adjustment, the number of preset confidence levels, the initial confidence level, and the confidence value corresponding to the initial confidence, etc. can be set as needed, and the present specification does not limit this.

[0107] It should be noted that in the present specification, the lesion detection result is only used to indicate the pathological information of the breast part of the patient. The lesion detection result can be provided to an evaluator (e.g., a doctor) evaluating the health condition of the patient as reference information for evaluating the health condition of the breast part of the patient, so that the evaluator can have a biological index for reference when evaluating the health condition of the breast part of the patient, and the evaluator can obtain an evaluation result with higher accuracy. Moreover, the present specification does not limit the manner in which the evaluator evaluates the health condition of the breast part of the patient based on the lesion detection result.

[0108] Based on Figure 1 As shown in the lesion detection method, the breast region in the breast projection image and the breast X-ray image is registered to obtain the coordinate conversion relationship between the breast projection image and the breast X-ray image, and then the first region where the lesion is located in the breast projection image and the second region where the lesion is located in the breast X-ray image are converted into the same coordinate system according to the coordinate conversion relationship, and the lesion detection result is determined according to the first region and the second region in the same coordinate system. The present method determines a more accurate lesion detection result by using the breast three-dimensional image and the breast X-ray image together, thereby improving the accuracy of lesion detection.

[0109] In addition, in general, even if the acquisition angle and the acquisition position are the same, the compression degree of the breast of the patient may also be different when the breast three-dimensional image and the breast X-ray image of the patient are acquired. That is, in general, there is a certain difference between the breast region in the breast three-dimensional image and the breast region in the breast X-ray image. This undoubtedly brings challenges to the registration of the breast region in the breast projection image and the breast X-ray image. Based on this, before registration, the terminal can also adjust the voxel spacing of the breast projection image and the breast X-ray image in step S104.

[0110] Specifically, the terminal can determine the voxel spacing of the breast projection image and the voxel spacing of the breast X-ray image, respectively.

[0111] Then, the terminal can adjust the size of the breast X-ray image and the breast projection image, respectively, according to the target voxel spacing. The target voxel spacing can be the voxel spacing of the breast projection image, the voxel spacing of the breast X-ray image, or any other voxel spacing except the above two. The technical means adopted when adjusting the image size can be any sampling means such as downsampling or upsampling. How to determine the target voxel spacing and how to adjust the image size based on the target voxel spacing can be set as needed, and the present specification does not limit this.

[0112] Finally, the terminal can determine the voxel spacing of the breast X-ray image and the breast projection image after the size adjustment, respectively. And continue to adjust the size of the breast X-ray image and the breast projection image according to the difference between the determined voxel spacing and the target voxel spacing, until the voxel spacing of the breast X-ray image and the breast projection image is the same.

[0113] The breast X-ray image and the breast projection image adjusted to the same voxel spacing can ensure that the pixel spacing of the breast X-ray image and the breast projection image is the same. Therefore, based on the breast X-ray image and the breast projection image adjusted to the same voxel spacing, the breast X-ray image and the breast projection image can be registered more quickly, and the efficiency of lesion detection is improved.

[0114] Further, when two images with larger size are matched, the time required may be longer, and when two images with smaller size are matched, the time required may be shorter. Therefore, in order to further speed up the registration between the breast X-ray image and the breast projection image, after the breast X-ray image and the breast projection image adjusted to the same voxel spacing are obtained, the terminal can also perform down-sampling operation on the breast X-ray image after the size adjustment and the breast projection image after the size adjustment, so as to obtain a down-sampled breast X-ray image and a down-sampled breast projection image. Wherein, the size of the down-sampled breast X-ray image and the down-sampled breast projection image is smaller.

[0115] Therefore, based on the down-sampled breast X-ray image and the breast projection image with smaller size, the terminal can more quickly determine the coordinate conversion relationship between the breast X-ray image and the breast projection image.

[0116] Further, the breast projection image and the breast X-ray image usually contain a large number of background regions, and these background regions cannot help the registration of the breast region, but will increase the requirement of the computing resources for the registration process. Therefore, in step S104, the terminal can also crop the background regions in the breast projection image and the breast X-ray image before registration.

[0117] Specifically, the terminal can obtain an initial breast three-dimensional image and an initial breast X-ray image, and determine an initial breast projection image according to the initial breast three-dimensional image. Wherein, the initial three-dimensional image and the initial breast X-ray image are images collected by a collection device.

[0118] Then, the terminal can perform semantic segmentation on the initial breast projection image to determine the breast region contained in the initial breast projection image, and perform semantic segmentation on the initial breast X-ray image to determine the breast region contained in the initial breast X-ray image.

[0119] Finally, the terminal can determine a minimum rectangle containing the breast region in the initial breast projection image according to the breast region in the initial breast projection image. And the terminal can crop the initial breast projection image according to the determined minimum rectangle, and take the cropped initial breast projection image as the breast projection image of the patient. Meanwhile, the terminal can also determine a minimum rectangle containing the breast region in the initial breast X-ray image according to the breast region in the initial breast X-ray image. And the terminal can crop the initial breast X-ray image according to the determined minimum rectangle, and take the cropped initial breast X-ray image as the breast X-ray image of the patient.

[0120] Of course, the terminal determining the breast region contained in the initial breast projection image and the initial breast X-ray image respectively can also be a technical means of object recognition, and the breast region corresponding to the initial breast projection image and the initial breast X-ray image respectively can be recognized. How to determine the breast region can be set as needed, and the present specification does not limit this.

[0121] In addition, in the present specification, the number of lesions contained in the breast projection image and the breast X-ray image can be more than one. Therefore, in the present specification, the terminal can also judge whether each lesion is a real lesion.

[0122] Specifically, the terminal can judge whether there is a second region matched with the first region for each first region. If there is, the terminal can determine that the lesion corresponding to the first region is a real lesion, and then the terminal can take the first region and the second region matched with the first region as a lesion region. If there is not, the terminal can determine that the lesion corresponding to the first region is not a real lesion, and then the terminal can not take the first region as a lesion region. Alternatively, the terminal can also increase the confidence that the second region matched with the first region and the first region belong to the lesion region, or decrease the confidence that the first region belongs to the lesion region, to determine the lesion detection result. Of course, the terminal can also take the lesion corresponding to the second region matched with the first region and the first region as the same lesion to determine the lesion detection result. How to determine the lesion detection result based on the matching result of the first region and the second region can be set as needed, and the present specification does not limit this.

[0123] Further, in the present specification, the lesion type corresponding to the lesion can be various, such as calcification point type and lump type. For a lesion, if the lesion type of the lesion in the breast projection image and the lesion type of the lesion in the breast X-ray image are inconsistent, the lesion in the breast projection image and the lesion in the breast X-ray image have a high probability of being different lesions. Therefore, the terminal can also determine the lesion detection result according to the lesion type corresponding to the lesion.

[0124] Specifically, for each lesion type, the terminal can determine each first region and each second region corresponding to the lesion type.

[0125] After that, under the target coordinate system, the terminal can determine, for each first region corresponding to the lesion type, whether there is a second region matching the first region.

[0126] If there is, the terminal can take the second region matching the first region and the lesion corresponding to the first region as the same lesion. If there is not, the terminal can not process the first region and the lesion corresponding to it, and continue to determine, according to a preset order, whether there is a second region matching the first region corresponding to the lesion. Then, after determining the lesions corresponding to each first region and each second region, the terminal can determine the lesion detection result.

[0127] Of course, the terminal can also determine the confidence of each first region and each second region respectively by setting the confidence, and then determine the lesion detection result based on the confidence. That is, if there is, the terminal can increase the confidence that the first region and the second region matching the first region belong to the lesion region. If there is not, the terminal can decrease the confidence that the first region belongs to the lesion region. Of course, the terminal can only increase the confidence that the second region matching the first region and the first region belong to the lesion region. Or, only decrease the confidence that the first region belongs to the lesion region. Specifically, how to adjust the confidence based on the matching result can be set as needed, and the present specification does not limit this.

[0128] Further, multiple second regions matching the same first region can occur at the same time, in which case the terminal can determine the lesion detection result based on the matching degree of each second region corresponding to the first region.

[0129] Specifically, for each first region in the lesion type, when there are multiple second regions matching the first region, the terminal can determine the matching degree of each second region and the first region respectively.

[0130] The terminal can determine the matching degree of the second region and the first region based on the intersection-over-union of the second region and the first region. The intersection-over-union and the matching degree are positively correlated. Alternatively, the terminal can determine the matching degree of the second region and the first region according to the similarity of the image of the second region and the similarity of the image of the first region, where the similarity and the matching degree are positively correlated. Alternatively, the terminal can determine the matching degree of the second region based on the confidence of the second region, where the confidence and the matching degree are positively correlated. Of course, the terminal can also set different weights for the intersection-over-union, the matching degree, and the confidence, and determine the matching degree of the second region according to the intersection-over-union of the second region and the weight of the intersection-over-union, the matching degree and the weight of the matching degree, and the confidence and the weight of the confidence. That is, the terminal can determine the matching degree of the second region based on at least one of the intersection-over-union, the similarity, and the confidence. The specific way of determining the matching degree can be set as needed, and the present specification does not limit this.

[0131] The terminal can then determine, from the second regions, a second region with the highest matching degree as a target second region. The terminal can determine the lesion detection result by regarding the lesions corresponding to the first region and the target second region as the same lesion. Alternatively, the terminal can determine the lesion detection result by increasing the confidence that the first region and the target second region belong to the lesion region.

[0132] Of course, if the intersection-over-union of the target second region and the first region is still less than the preset matching threshold, the terminal can not process the first region and the target second region, or reduce the confidence that the first region and the target second region belong to the lesion region. The terminal can also determine that the lesions corresponding to the first region and the target second region are different lesions. The terminal can determine the lesion detection result after determining the lesions corresponding to the first region and the second regions, or after determining the confidence that the first regions and the second regions belong to the lesion region. The specific way of determining the lesion detection result based on the matching result of the first region and the target second region can be set as needed, and the present specification does not limit this.

[0133] In addition, the step of registering the breast projection image and the breast X-ray image can be performed in a supervised learning manner.

[0134] Specifically, the terminal can input the breast projection image and the breast X-ray image into a pre-trained registration model to obtain a coordinate conversion relationship between the breast projection image and the breast X-ray image output by the registration model.

[0135] The registration model can be trained in the following manner:

[0136] Firstly, the terminal can obtain a breast projection image that has been registered with a breast X-ray image, and take the breast projection image before registration and the breast X-ray image as training samples, and take the breast projection image after registration as a label.

[0137] Secondly, the terminal can input the training samples into a registration model to be trained, so that the registration model to be trained determines a to-be-determined conversion relationship, and determines a breast projection image after pre-registration according to the to-be-determined conversion relationship.

[0138] Then, the terminal can determine the to-be-determined conversion relationship output by the registration model to be trained and the breast projection image after pre-registration.

[0139] Finally, the terminal can take the difference between the breast projection image after pre-registration and the label of the training sample as an optimization target, and train the registration model.

[0140] Of course, in order to avoid the situation that the training cost of the registration model is too high due to the need for manual labeling of training samples, the registration model in the specification can also be trained in the following unlabeled manner:

[0141] Firstly, the terminal can obtain a breast three-dimensional image and a breast X-ray image collected at the same body position and the same angle. The breast three-dimensional image and the breast X-ray image are images of the same part of the same patient collected at the same body position and the same angle.

[0142] Therefore, the terminal can determine a breast projection image before registration according to the breast three-dimensional image, and perform feature extraction on the breast projection image before registration to determine a first sample feature of the breast projection image before registration. And perform feature extraction on the obtained breast X-ray image to determine a second sample feature of the breast X-ray image. Wherein the feature extraction method of the breast projection image and the feature extraction method of the breast X-ray image can be the same or different, and the specific way of feature extraction of the above two images can be set as needed, and the specification does not limit this.

[0143] Secondly, the terminal can take the breast projection image before registration, the breast X-ray image, the first sample feature and the second sample feature as training samples, and input the training samples into a registration model to be trained to obtain a to-be-determined conversion relationship output by the registration model. Wherein the breast X-ray image and the second sample feature can be taken as a label of the training sample.

[0144] Then, the terminal can obtain a registered breast projection image according to the to-be-determined conversion relationship and the breast projection image before registration, and perform feature extraction on the registered breast projection image to determine a third sample feature of the registered breast projection image. The feature extraction manner of the registered breast projection image is the same as the feature extraction manner used when the second sample feature is determined.

[0145] Finally, the terminal can determine a registration loss according to the difference between the registered breast projection image and the breast X-ray image, determine a feature loss according to the difference between the second sample feature and the third sample feature, combine the registration loss and the feature loss to determine a total loss, and adjust the model parameters of the registration model to complete the training of the registration model with the total loss as the optimization target.

[0146] The registration loss can be used to represent the loss between the registered breast projection image and the breast X-ray image, and the feature loss can be used to represent the loss between the image features of the registered breast projection image and the image features of the breast X-ray image. The image features can be the distance between the lesion and the reference object, the orientation of the vector from the lesion to the reference object, the type of the lesion, the feature point distribution of the breast projection image and the breast X-ray image, and the like. The reference object can be a nipple, a breast, or the like. The specific type of the image feature and the specific type of the reference object can be set as needed, and the present specification does not limit them.

[0147] Based on the total loss, not only the difference between the registered breast projection image and the breast X-ray image can be measured, but also the image features of the breast projection image can be extracted and compared with the image features of the breast X-ray image to quantify the image feature difference information of the breast projection image before registration. The training of the registration model can be more effectively and accurately guided, and the model obtained by the training can be more accurate.

[0148] The difference between the registered breast projection image and the breast X-ray image can be measured by mutual information method or other methods. Of course, the difference can also be determined in various forms such as the brightness difference between the breast projection image and the breast X-ray image. The specific manner of determining the difference between the breast projection image and the breast X-ray image can be set as needed, and the present specification does not limit it.

[0149] Further, in the process of training the registration model, in order to ensure that the topological structure of the registered breast projection image does not change, the smoothness of the coordinate conversion relationship based on the registration model needs to be quantified to obtain a smooth loss of the coordinate conversion relationship.

[0150] Therefore, the terminal can further determine a smooth loss based on the coordinate conversion relationship output by the registration model, and determine a total loss based on the smooth loss and the determined registration loss and feature loss, and train the registration model according to the total loss.

[0151] The total loss of the registration model includes three parts: the feature loss, the registration loss, and the smooth loss. The smooth loss is used to ensure the smoothness of the coordinate conversion relationship and ensure that the topological structure of the registered breast projection image and the breast X-ray image remains consistent. The feature loss and the registration loss work together to guide the training of the registration model and ensure the performance of the registration task.

[0152] Further, after determining the total loss, the terminal can iteratively optimize the registration model based on the total loss.

[0153] Specifically, the terminal can iteratively adjust the model parameters used for feature extraction of the breast projection image before registration and the model parameters used for feature extraction of the breast X-ray image according to the determined total loss. The iteration is stopped when the determined total loss meets a preset iteration termination condition. The iteration termination condition can be that the number of iterations reaches a preset threshold, the total loss of the model is less than a preset threshold, and the like. The specific content of the iteration termination condition can be set as needed, and the present specification does not limit it.

[0154] It should be noted that the terminal used for training the registration model and the terminal for performing the lesion detection can be the same terminal or different terminals. The specific terminal for training the registration model can be set as needed, and the present specification does not limit it.

[0155] Of course, the lesion detection method in the present specification can also obtain breast images of the same part of the same patient taken at different body positions and at the same angle. However, due to the large difference between the images taken at different body positions, the accuracy of the registration result is low, which further leads to an inaccurate lesion detection result based on the registration result. In the present specification, the breast X-ray image and the breast three-dimensional image are obtained at the same angle and at the same body position, and the lesion is detected based on the obtained images to quickly and accurately determine the true lesion.

[0156] Further, the terminal can also use a machine learning method to determine the corresponding lesions in the breast projection image and the breast X-ray image.

[0157] Specifically, the terminal can input the obtained breast projection image into a pre-trained first recognition model to obtain the lesion in the breast projection image and the first region corresponding to the lesion output by the first recognition model.

[0158] The terminal can also input the obtained breast X-ray image into a pre-trained second recognition model to obtain a lesion in the breast X-ray image and a second region corresponding to the lesion output by the second recognition model.

[0159] The first recognition model can be trained in the following manner.

[0160] First, a plurality of breast projection images are obtained as first training samples, and labels of the first training samples are determined. The label of the first training sample is the position of each lesion.

[0161] Then, each first training sample is input into the first recognition model to be trained to obtain a recognition result of the first training sample output by the first recognition model.

[0162] Finally, a first recognition loss is determined according to the recognition result and the label of each first training sample, and the first recognition model is trained based on the first recognition loss.

[0163] The second recognition model can be trained in the following manner.

[0164] First, a plurality of breast projection images are obtained as second training samples, and labels of the second training samples are determined. The label of the second training sample is the position of each lesion.

[0165] Then, each second training sample is input into the second recognition model to be trained to obtain a recognition result of the second training sample output by the second recognition model.

[0166] Finally, a second recognition loss is determined according to the recognition result and the label of each second training sample, and the second recognition model is trained based on the second recognition loss.

[0167] Of course, when determining the lesions in the breast projection image and the breast X-ray image, a threshold-based segmentation method such as a global threshold method, an adaptive threshold method, an edge detection-based segmentation method, a region growing method, a watershed method, and a graph theory-based segmentation method can be used to segment the lesions to determine each lesion contained in the image and the position corresponding to each lesion. Since the method of determining the lesions contained in the image based on segmentation is a relatively mature technology, this specification will not be described again.

[0168] Similarly, the terminal used for training the first and second identification models and the terminal performing the lesion detection can be the same terminal or different terminals. The terminal performing the training of the first and second identification models can be set as needed, and the present specification does not limit the terminal.

[0169] Based on Figure 1 The present specification also provides a flowchart of a lesion matching method, as shown in Figure 2

[0170] Figure 2 The present specification provides a flowchart of a lesion matching method. In the flowchart, the terminal can convert the first region corresponding to the lesion in the breast projection image to the first coordinate system of the breast X-ray image through the determined coordinate conversion relationship, to obtain a first region in the first coordinate system. The target coordinate system is the first coordinate system. The terminal can determine the intersection of union (IoU) based on the first region and the second region in the first coordinate system, and compare the IoU with a preset matching threshold R, to obtain a lesion detection result according to the comparison result.

[0171] Based on the same idea, when the IoU of the first region and multiple second regions is greater than the matching threshold R, the present specification also provides a flowchart of determining a lesion matching result, as shown in Figure 3

[0172] Figure 3 The present specification provides a flowchart of determining a lesion matching result. In the flowchart, the first region, the second region 1, and the second region 2 are all lesion regions in the target coordinate system. The terminal can determine a lesion detection result based on the matching degree of the second region 1 and the first region, and the matching degree of the second region 2 and the first region. For example, when the matching degree is determined based on the IoU, the terminal can determine that the second region 1 matches the first region, increase the confidence that the first region and the second region 1 belong to the lesion region, and take the first region, the second region 1, and the corresponding confidence as the lesion detection result.

[0173] The present specification also provides a flowchart of adjusting the voxel spacing, as shown in Figure 4

[0174] Figure 4 ​​​The flowchart for adjusting the voxel spacing provided by the present specification is shown in the figure. From left to right, the figure shows the breast contour in the breast X-ray image, the breast contour in the breast projection image, the breast contour of the breast projection image after adjusting the voxel spacing, and the contrast image of the breast contour in the breast X-ray image and the breast contour of the breast projection image after adjusting the voxel spacing. It can be seen that the voxel spacing of the breast X-ray image is taken as an example to illustrate the target voxel spacing. Then the terminal can adjust the voxel spacing of the breast projection image according to the target voxel spacing until the voxel spacing of the breast projection image and the voxel spacing of the breast X-ray image are consistent.

[0175] In addition, the above Figure 2 、 3 , 4, etc. are only exemplary descriptions of the embodiments in the present specification. Specifically, how to perform lesion matching, how to determine the lesion matching result, and how to adjust the voxel spacing, etc. can be set as needed, and the present specification does not limit this.

[0176] It should be noted that all actions of obtaining signals, information or data in the present specification are performed in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization of the corresponding device owner.

[0177] Based on the same idea, the present specification provides a structural schematic diagram of a lesion matching device, as shown in Figure 5 .

[0178] Figure 5 The structural schematic diagram of a lesion matching device provided by the present specification is shown in the figure. The device comprises:

[0179] The first obtaining module 200 is configured to obtain a breast three-dimensional DBT image, determine a breast projection image according to the breast three-dimensional image, and determine a first region corresponding to a lesion in the breast projection image.

[0180] The second obtaining module 202 is configured to obtain a breast X-ray FFDM image, and determine a second region corresponding to the lesion in the breast X-ray image according to the breast X-ray image.

[0181] The conversion module 204 is configured to determine a coordinate conversion relationship between the breast projection image and the breast X-ray image by taking the breast region in the breast projection image and the breast X-ray image as a target, and convert the first region and the second region into a target coordinate system according to the conversion relationship.

[0182] The matching module 206 is configured to match the first region and the second region in the target coordinate system, and determine a lesion detection result according to a matching result. The acquisition angles and the acquisition postures of the breast three-dimensional image and the breast X-ray image are the same.

[0183] Optionally, before determining the coordinate conversion relationship between the breast projection image and the breast X-ray image, the conversion module 204 is configured to adjust the size of the breast X-ray image and the size of the breast projection image according to a target voxel spacing, so that the voxel spacing of the breast X-ray image and the voxel spacing of the breast projection image are the same.

[0184] Optionally, the first acquisition module 200 is configured to acquire an initial breast X-ray image and an initial breast three-dimensional image, determine an initial breast projection image according to the initial breast three-dimensional image, determine a breast region contained in the initial breast projection image, and crop the initial breast projection image according to a minimum rectangle containing the breast region, so as to obtain the breast projection image of the patient. The first acquisition module 200 is further configured to determine a breast region contained in the initial breast X-ray image, and crop the initial breast X-ray image according to a minimum rectangle containing the breast region, so as to obtain the breast X-ray image of the patient.

[0185] Optionally, the matching module 206 is configured to determine an intersection-over-union of the first region and the second region according to the coordinates of the first region and the coordinates of the second region in the target coordinate system, determine whether the intersection-over-union is greater than a preset matching threshold, and if yes, determine that the lesions corresponding to the first region and the second region are the same lesion.

[0186] Optionally, the first acquisition module 200 is configured to input the breast projection image as an input into a first recognition model that is pre-trained, so as to obtain the first region corresponding to each lesion output by the first recognition model, and the lesion type corresponding to each first region.

[0187] Optionally, the second acquisition module 202 is configured to input the breast X-ray image as an input into a second recognition model that is pre-trained, so as to obtain the second region corresponding to each lesion output by the second recognition model, and the lesion type corresponding to each second region.

[0188] Optionally, the matching module 206 is configured to determine, for each lesion type, the first regions and the second regions corresponding to the lesion type, determine, for each first region corresponding to the lesion type, whether there is a second region matched with the first region in the target coordinate system, and if so, determine that the second region matched with the first region and the lesion corresponding to the first region are the same lesion, and take the first regions and the lesions corresponding thereto, and the second regions and the lesions corresponding thereto as the lesion detection result.

[0189] Optionally, the matching module 206 is configured to determine, when there are multiple second regions matched with the first region, the matching degrees of the second regions and the first region respectively, determine the matching degree according to at least one of the intersection-over-union of the second region and the first region, the similarity between the image of the second region and the image of the first region, and the confidence of the second region, select, from the second regions matched with the first region, the second region with the highest matching degree as a target second region, and determine that the lesion corresponding to the first region and the target second region are the same lesion.

[0190] The method further includes:

[0191] The training module 208 is configured to acquire a breast three-dimensional image and a breast X-ray image acquired at the same body position and the same angle, determine a breast projection image before registration according to the breast three-dimensional image, determine first sample features of the breast projection image before registration, and determine second sample features of the breast X-ray image, input the breast projection image before registration, the breast X-ray image, the first sample features, and the second sample features as input into a registration model to be trained, obtain a pending conversion relationship output by the registration model, obtain a breast projection image after registration according to the pending conversion relationship and the breast projection image before registration, and determine third sample features of the breast projection image after registration, determine a loss according to the difference between the breast projection image after registration and the breast X-ray image, and the difference between the second sample features and the third sample features, and train the registration model with the minimum loss as an optimization target.

[0192] The present specification also provides a computer-readable storage medium storing a computer program, and the computer program is configured to execute the above Figure 1 The provided lesion detection method.

[0193] The present specification also provides Figure 6 The schematic structural diagram of the electronic device is shown. As Figure 6At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and can also include other hardware required by a business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above Figure 1 The lesion detection method. Of course, in addition to the software implementation, the present specification does not exclude other implementation manners, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0194] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0195] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0196] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0197] For the sake of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of each unit can be implemented in one or more software and / or hardware in implementing the present specification.

[0198] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, the present specification can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0199] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0200] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0201] The computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more flow or multiple flows and / or blocks Figure 1 one or more flow or multiple flows and / or blocks

[0202] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0203] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer-readable media.

[0204] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0205] It should also be noted that the terms "comprising", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0206] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0207] The present specification can be described in the general context of computer-executable instructions, such as program modules, executed by computers. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media, including storage devices.

[0208] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different but related aspects of the description. Each of the various embodiments can stand on its own, and each can be combined with the subject matter of other embodiments to produce further embodiments. Where appropriate, therefore, the contents of the specification can be regarded as being incorporated by reference, including the description, drawings, claims, abstract and the like.

[0209] The above description is embodied in the form of embodiments only and is not intended to limit the present specification. The present specification can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the scope of the claims of the present specification.

Claims

1. A lesion detection method characterized by, The method comprises: acquiring a breast three-dimensional DBT image, and determining a breast projection image according to the breast three-dimensional image, and determining a first region corresponding to a lesion in the breast projection image; acquiring a breast X-ray FFDM image, and determining a second region corresponding to the lesion in the breast X-ray image according to the breast X-ray image; registering the breast regions in the breast projection image and the breast X-ray image as a target, determining a coordinate conversion relationship between the breast projection image and the breast X-ray image, and converting the first region and the second region into a target coordinate system according to the coordinate conversion relationship; matching the first region and the second region in the target coordinate system, and determining a lesion detection result according to the matching result; the matching comprises: determining an intersection-over-union of the first region and the second region according to the coordinates of the first region and the coordinates of the second region in the target coordinate system; determining whether the intersection-over-union is greater than a preset matching threshold; and if yes, determining that the lesions corresponding to the first region and the second region are the same lesion. The acquisition angle and the acquisition body position of the breast three-dimensional image and the breast X-ray image are the same.

2. The method of claim 1, wherein, Before determining the coordinate conversion relationship between the breast projection image and the breast X-ray image, the method further comprises: adjusting the size of the breast X-ray image and the size of the breast projection image according to a target voxel spacing, so that the voxel spacing of the breast X-ray image and the voxel spacing of the breast projection image are the same.

3. The method of claim 1, wherein, Acquiring a breast X-ray image and a breast projection image of a patient, specifically comprising: acquiring an initial breast X-ray image and an initial breast three-dimensional image, and determining an initial breast projection image according to the initial breast three-dimensional image; determining a breast region contained in the initial breast projection image, and cropping the initial breast projection image according to a minimum rectangle containing the breast region, and taking the cropped initial breast projection image as the breast projection image of the patient; determining a breast region contained in the initial breast X-ray image, and cropping the initial breast X-ray image according to a minimum rectangle containing the breast region, and taking the cropped initial breast X-ray image as the breast X-ray image of the patient.

4. The method of claim 1, wherein, Determining a first region corresponding to a lesion in the breast projection image, specifically comprising: inputting the breast projection image as an input into a first recognition model trained in advance, obtaining a first region corresponding to each lesion output by the first recognition model, and a lesion type corresponding to each first region; determining a second region corresponding to the lesion in the breast X-ray image, specifically comprising: inputting the breast X-ray image as an input into a second recognition model trained in advance, obtaining a second region corresponding to each lesion output by the second recognition model, and a lesion type corresponding to each second region.

5. The method of claim 4, wherein, In the target coordinate system, the first region and the second region are matched, and a lesion detection result is determined according to a matching result, specifically including: For each lesion type, determine the first region and the second region corresponding to the lesion type; In the target coordinate system, for each first region corresponding to the lesion type, determine whether there is a second region matched with the first region; If yes, determine that the second region matched with the first region and the first region correspond to the same lesion; According to the first region and the corresponding lesion, and the second region and the corresponding lesion, determine the lesion detection result.

6. The method of claim 5, wherein, Determine that the second region matched with the first region and the first region correspond to the same lesion, specifically including: When there are multiple second regions matched with the first region, respectively determine the matching degree of each second region and the first region, and the matching degree is determined according to at least one of the intersection over union of the second region and the first region, the similarity between the image of the second region and the image of the first region, and the confidence of the second region; From each second region matched with the first region, select the second region with the highest matching degree as the target second region; Determine that the first region and the target second region correspond to the same lesion.

7. The method of claim 1, wherein, According to the breast projection image and the breast X-ray image, the breast regions in the breast projection image and the breast X-ray image are registered as the target, and a coordinate conversion relationship between the breast projection image and the breast X-ray image is determined, specifically including: Input the breast projection image and the breast X-ray image as input into a pre-trained registration model; Determine the coordinate conversion relationship between the breast projection image and the breast X-ray image output by the registration model; The registration model is trained by the following method: Obtain a breast three-dimensional image and a breast X-ray image collected at the same body position and the same angle; According to the breast three-dimensional image, determine a breast projection image before registration, determine a first sample feature of the breast projection image before registration, and determine a second sample feature of the breast X-ray image; Input the breast projection image before registration, the breast X-ray image, the first sample feature, and the second sample feature as input into a registration model to be trained, and obtain a pending conversion relationship output by the registration model; According to the pending conversion relationship and the breast projection image before registration, obtain a breast projection image after registration, and determine a third sample feature of the breast projection image after registration; According to the difference between the breast projection image after registration and the breast X-ray image, and the difference between the second sample feature and the third sample feature, determine a loss, and take the minimum loss as the optimization target to train the registration model.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-7.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method of any one of claims 1-7.

Images