A breast symmetry evaluation method and system based on automatic positioning of mid-axis profile

Abstract

The application discloses a breast symmetry evaluation method and system based on automatic positioning of a central axis profile, which comprises the following steps: acquiring a chest picture, segmenting a left areola and a right areola in the chest picture based on a global optimization algorithm; obtaining the positions of the left areola and the right areola based on a bounding box regression positioning algorithm; calculating the horizontal coordinates of a breast symmetry axis according to the positions of the left areola and the right areola; calculating the symmetry rates of the left and right areolas according to the breast symmetry axis, calculating the areas and perimeters of the left and right areolas respectively, and calculating the area variance and the perimeter variance of the left and right areolas according to the areas and perimeters of the left and right areolas; and evaluating the symmetry grade of the breast according to the symmetry rates, the area variance and the perimeter variance of the left and right areolas as well as the areas and perimeters of the left and right areolas, and obtaining an evaluation result. Through non-contact measurement and symmetry evaluation of the breast, the symmetry of the breast of a patient during an operation is improved.

Description

Technical Field

[0001] This invention relates to the field of breast symmetry, and more particularly to a method and system for assessing breast symmetry based on automatic positioning of the central contour. Background Technology

[0002] Currently, breast cancer ranks first among female cancers. Due to increased health awareness and improved clinical detection methods, approximately 90% of breast cancers are detected early, with 5-year and 10-year survival rates reaching 80% and 90%, respectively. Therefore, in addition to surgically removing the tumor to cure patients, breast reshaping techniques are receiving increasing attention from the medical community and patients. Assessing breast symmetry during breast reshaping surgery is crucial.

[0003] Current methods for breast measurement and assessment of bilateral breast symmetry (or symmetry) mainly fall into two categories: subjective and objective methods. Subjective methods primarily rely on visual inspection by the surgeon or other personnel to assess breast symmetry or symmetry. Objective methods mainly include three categories: mechanically based measurements of scale and volume; magnetic resonance imaging-based measurements; and measurements based on 2D or 3D imaging of the external surface.

[0004] Among the methods described above, subjective visual assessment is difficult to accurately evaluate bilateral breast differences or symmetry due to visual errors and individual differences among surgeons. Mechanical measurement methods based on scale or volume are not suitable for measuring the non-rigid and curved shape of the breast, and being contact-based, they are not convenient for use in the surgical environment. Magnetic resonance imaging (MRI)-based measurement and evaluation methods require large MRI equipment and subsequent image reconstruction calculations, making them unsuitable for intraoperative measurement and evaluation in breast reconstruction surgery. Measurement and evaluation methods based on 2D or 3D surface imaging are mostly dependent on the subject's standing posture and are used for preoperative or postoperative measurement and evaluation.

[0005] In summary, current methods for measuring and evaluating bilateral breast differences (or symmetry) are either ineffective or unsuitable for intraoperative full-size measurement and analysis of the breast, especially in breast reconstruction surgery. Summary of the Invention

[0006] This invention provides a method and system for assessing breast symmetry based on automatic positioning of the central contour, in order to overcome the above-mentioned technical problems.

[0007] A method for assessing breast symmetry based on automatic midline contour localization, comprising:

[0008] Step 1: Obtain the breast image and segment the left and right areolas in the breast image using a global optimization algorithm.

[0009] Step 2: Obtain the positions of the left and right areolas respectively based on the bounding box regression localization algorithm.

[0010] Step 3: Calculate the x-coordinate of the breast's axis of symmetry based on the positions of the left and right areolas.

[0011] Step 4: Calculate the symmetry ratio of the left and right areolas based on the axis of symmetry of the breast. Calculate the area and perimeter of the left and right areolas respectively. Based on the area and perimeter of the left and right areolas, calculate the variance of the area and the variance of the perimeter of the left and right areolas.

[0012] Step 5: Assess the symmetry level of the breasts based on the symmetry ratio, area variance, and perimeter variance of the left and right areolas, as well as the area and perimeter of the left and right areolas, and obtain the assessment results.

[0013] Preferably, the segmentation of the left and right areolas in the chest image based on the global optimization algorithm includes designing the inter-class variance according to formula (1).

[0014] g = r fore *r back *(ave fore -ave back ) 2 (1)

[0015] Where, r fore Indicates the percentage of pixels in the areola, r back This indicates the percentage of pixels in the remaining part of the breast, ave fore This represents the average gray level of the areola. back The value represents the average gray level of the remaining portion of the breast, and g represents the inter-class variance.

[0016] Set the number of iterations for the global optimization algorithm, and iteratively train the chest image using the global optimization algorithm. Calculate the value of the inter-class variance at each iteration. When the inter-class variance is at its maximum, segment the left and right areolas of the chest image.

[0017] Preferably, step three includes calculating the abscissa of the breast's axis of symmetry according to formula (2).

[0018] Rect l =[x l ,y l ,width l height l ]

[0019] Rect r =[x r ,y r ,width r heightr ]

[0020]

[0021] Where Rect l Represents the rectangular bounding box data of the left areola. r Rectangular data of the right areola, s x The x-coordinate represents the axis of symmetry. l The x-coordinate of the bottom left point of the left areola rectangle is represented by the x-coordinate. r The x-coordinate of the bottom left point of the rectangle representing the right areola is y. l This represents the y-coordinate of the bottom left point of the left areola rectangle. r The width represents the ordinate of the bottom left point of the rectangle containing the right areola. l This represents the width of the rectangle containing the left areola. r The width and height of the rectangle representing the right areola are indicated. l This indicates the height of the rectangle representing the left areola. r This indicates the height of the rectangle representing the right areola.

[0022] Preferably, the step of calculating the symmetry ratio of the left and right areolas based on the axis of symmetry of the breast includes,

[0023] When the left and right areolas are on the same horizontal plane, the symmetry ratio of the left and right areolas can be calculated according to formula (3).

[0024]

[0025] Where c represents the total number of areolar edge points symmetrical about the axis of symmetry on both sides, t represents the total number of areolar edge points on both sides, and r represents the symmetry ratio.

[0026] When the left and right areolas are not on the same horizontal plane, the midpoint of the height of the left and right areola rectangles is obtained according to formula (4).

[0027]

[0028]

[0029] Where C l C r These are the midpoints of the heights of the left and right areola rectangles, respectively.

[0030] Iterate through the first pixel on the edge of the left areola, and calculate the cosine of the offset angle of the first pixel relative to the midpoint of the height of the left areola rectangle according to formula (5).

[0031]

[0032]

[0033]

[0034] Where A1 is the first pixel with coordinates (x1, y1), and A1B1 represents the first pixel along line C. l The diffusion length in the direction, with an offset angle of ∠A1C l B1 = θ1,

[0035] The second pixel in the rectangle of the right areola is obtained based on the cosine value of the offset angle and the diffusion length. If the second pixel is located at the edge of the right areola, it means that the first pixel and the second pixel are symmetrical pixels. Otherwise, it means that the first pixel does not have symmetrical pixels. The number of symmetrical pixels is counted and used as the total number of edge points of the left and right areolas that are symmetrical about the axis of symmetry. The symmetry rate of the left and right areolas is calculated according to formula (3).

[0036] Preferably, the calculation of the area variance and perimeter variance of the left and right areolas based on their areas and perimeters includes calculations according to formulas (6) and (7).

[0037]

[0038]

[0039]

[0040]

[0041] Where C1, S1, C2 and S2 are the area, perimeter, area and perimeter of the left areola, respectively, δ1 is the variance of the perimeter of the left and right areolas and δ2 is the variance of the area of ​​the left and right areolas.

[0042] A breast symmetry assessment system based on automatic midline contour localization includes a breast image acquisition module, an image segmentation module, a left and right areola localization module, a left and right areola symmetry axis calculation module, a left and right areola symmetry rate calculation module, a left and right areola area and perimeter calculation module, and a breast symmetry assessment module.

[0043] The chest image acquisition module is used to acquire chest images.

[0044] The image segmentation module is used to segment the left and right areolas in a chest image based on a global optimization algorithm.

[0045] The left and right areola positioning modules are used to obtain the positions of the left and right areolas respectively based on the bounding box regression positioning algorithm.

[0046] The module for calculating the left and right areola symmetry axes is used to calculate the abscissa of the breast symmetry axis based on the positions of the left and right areolas.

[0047] The left and right areola symmetry calculation module is used to calculate the symmetry ratio of the left and right areolas based on the breast's axis of symmetry.

[0048] The left and right areola area and perimeter calculation modules are used to calculate the area and perimeter of the left and right areolas respectively, and to calculate the area variance and perimeter variance of the left and right areolas based on the area and perimeter of the left and right areolas.

[0049] The breast symmetry assessment module is used to assess the symmetry level of the breast based on the symmetry rate, area variance, and perimeter variance of the left and right areolas, as well as the area and perimeter of the left and right areolas, and to obtain the assessment results.

[0050] This invention provides a method and system for breast symmetry assessment based on automatic midline contour positioning. By acquiring bilateral breast images of the patient in a supine position with different supine angles during surgery, the system accurately segments the left and right areolas in the images based on a global optimization algorithm and accurately locates the positions of the left and right areolas based on a bounding box regression positioning algorithm. Through non-contact measurement and symmetry assessment of the breast, this invention assists users in improving the symmetry of the patient's breast during surgery. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of the method of the present invention;

[0053] Figure 2 This is the areola image used for border regression localization in this invention;

[0054] Figure 3 This is a diagram illustrating the symmetry axis of the areola edge according to the present invention;

[0055] Figure 4 This is a calculation and construction diagram of the symmetry ratio of the left and right areola edges according to the present invention;

[0056] Figure 5 This is the overall architecture diagram of the APP of this invention;

[0057] Figure 6 This is a schematic diagram of the homepage of the APP of this invention;

[0058] Figure 7 This is a schematic diagram of patient information in the APP of this invention;

[0059] Figure 8 This is a diagram showing the results of the patient's breast symmetry evaluation in the APP of this invention;

[0060] Figure 9 This is a schematic diagram of user personal information in the APP of this invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Figure 1 This is a flowchart of the method of the present invention, as shown below. Figure 1 As shown, the method in this embodiment may include:

[0063] Step 1: Obtain the breast image and segment the left and right areolas in the breast image using a global optimization algorithm.

[0064] Step 2: Obtain the positions of the left and right areolas respectively based on the bounding box regression localization algorithm.

[0065] Step 3: Calculate the x-coordinate of the breast's axis of symmetry based on the positions of the left and right areolas.

[0066] Step 4: Calculate the symmetry ratio of the left and right areolas based on the axis of symmetry of the breast. Calculate the area and perimeter of the left and right areolas respectively. Based on the area and perimeter of the left and right areolas, calculate the variance of the area and the variance of the perimeter of the left and right areolas.

[0067] Step 5: Assess the symmetry level of the breasts based on the symmetry ratio, area variance, and perimeter variance of the left and right areolas, as well as the area and perimeter of the left and right areolas, and obtain the assessment results.

[0068] Based on the above scheme, by acquiring bilateral breast images of the patient in a supine position with different supine angles during the operation, the left and right areolas in the images are accurately segmented based on a global optimization algorithm, and the positions of the left and right areolas are accurately located based on a bounding box regression localization algorithm. By non-contact measurement of the breast and using different symmetry assessment methods based on whether the left and right areolas are located on the same horizontal plane, the symmetry of the patient's breast after intraoperative reshaping is improved.

[0069] Step 1: Obtain the breast image and segment the left and right areolas in the breast image using a global optimization algorithm.

[0070] Regarding the segmentation of the left and right areolas, this embodiment employs a global optimization algorithm. In a breast image with target and background regions, it is often desirable to separate the areola from the rest of the breast so that the areola region can be focused on for analysis. The global optimization algorithm uses machine learning to quickly find a suitable threshold to segment the pixels in the breast image, thereby clearly distinguishing the areola from the rest of the breast.

[0071] Global optimization algorithms are characterized by their simplicity, low pixel computation requirements, and good stability. Due to the unique characteristics of breast cancer patient image data, where the areola exhibits significant contrast with the rest of the body, global optimization algorithms can effectively segment the areola by finding a threshold. The core mathematical model is as follows:

[0072]

[0073] Where g(x,y) represents the image after thresholding, P(x,y) represents the original image, and threshold represents the threshold. If a1 is 1 and a0 is 0, the obtained g(x,y) can be transformed into a common binary image, thus obtaining the binary segmentation result of the areola.

[0074] The global optimization algorithm used in this embodiment is an adaptive thresholding segmentation algorithm based on the grayscale of the breast image. Its aim is to find an optimal threshold that can distinguish the areola from the rest of the breast through multiple iterations, maximizing the inter-class variance between the foreground and background when this threshold is chosen. The formula for the inter-class variance is shown below:

[0075]

[0076]

[0077] C fore +C back =Sum

[0078] r fore +rback =1

[0079] Ave = r fore *ave fore +r back *ave back

[0080] g = r fore *(Ave-ave fore ) 2 +r back *(Ave-ave back ) 2 (2)

[0081] Where Sum represents the total number of pixels in the breast image, r fore C represents the pixel percentage of the areola. fore r represents the number of pixels in the areola. back C represents the percentage of pixels in the remaining part of the breast. back Ave represents the number of pixels in the remaining part of the breast, and Ave represents the average gray level of the breast image. fore This represents the average gray level of the areola. back represents the average gray level of the rest of the breast, and g represents the inter-class variance.

[0082] The simplified formula above yields the overall formula for the inter-class variance, as shown below. Specifically, the segmentation of the left and right areolas in a breast image based on the global optimization algorithm includes designing the inter-class variance according to formula (3).

[0083] g = r fore *r back *(ave fore -ave back ) 2 (3)

[0084] Where, r fore Indicates the percentage of pixels in the areola, r back This indicates the percentage of pixels in the remaining part of the breast, ave fore This represents the average gray level of the areola. back The value represents the average gray level of the remaining portion of the breast, and g represents the inter-class variance.

[0085] The number of iterations for the global optimization algorithm is set, and the breast image is iteratively trained using the global optimization algorithm. The inter-class variance is calculated at each iteration. When the inter-class variance is at its maximum, the left and right areolas of the breast image are segmented.

[0086] By iterating through this formula, the threshold value at which the inter-class variance g reaches its maximum can be selected. This allows for further refinement of the segmentation of the left and right areolas.

[0087] Step 2: Obtain the positions of the left and right areolas respectively based on the bounding box regression localization algorithm.

[0088] Regarding the localization of the left and right areolas, this embodiment employs bounding box regression. Bounding box regression algorithms are widely used in object detection and localization. This algorithm makes the bounding box, after translation and scaling transformations, infinitely close to the Ground Truth, thereby accurately locating the left and right areola regions. The specific transformation relationship is as follows:

[0089] f(R x ,R y ,R w ,R h )=(R x ',R y ',R w ',R h ')

[0090] (R x ',R y ',R w ',R h ')≈(G x G y G w G h (4)

[0091] Among them (R) x ,R y ,R w ,R h ) represents the original border, (R x ',R y ',R w ',R h ') represents the transformed border, (G x G y G w G h ) represents Ground Truth, R x R represents the x-coordinate of the starting point of the original border. y R represents the ordinate of the starting point of the original border. w R represents the width of the original border. h This represents the height of the original border. Similarly, we can understand the meaning of the various parameters in the transformed border and Ground Truth.

[0092] Boundary regression algorithms, by inputting the feature vectors of the original bounding boxes and transforming them, can obtain bounding boxes with the smallest error relative to the ground truth, thereby achieving object detection and localization. The optimization objective of this algorithm is as follows:

[0093]

[0094] Where t * Represents Ground Truth Represents the bounding box for the predicted value, φ5(R) i ) represents the eigenvectors of R, w * λ represents the target function (transformation function) to be learned, and λ is a constant.

[0095] Based on the binary segmentation results of the left and right areolas obtained in the previous step, the positions of the left and right areolas can be accurately located using the bounding box regression operation described above. The specific areola localization results are as follows: Figure 2 As shown, the positioning rectangle (Rect) is represented by the following formula:

[0096] Rect = [x, y, width, height] (6)

[0097] Where Rect represents the specific data of the rectangle's position, x represents the x-coordinate of the bottom-left point of the rectangle, y represents the y-coordinate of the bottom-left point of the rectangle, width represents the width of the rectangle, and height represents the height of the rectangle.

[0098] Step 3: Calculate the abscissa of the breast symmetry axis based on the positions of the left and right areolas. Step 3 includes calculating the abscissa of the breast symmetry axis according to formula (7).

[0099] Rect l =[x l ,y l ,width l height l ]

[0100] Rect r =[x r ,y r ,width r height r ]

[0101]

[0102] Where Rect l Represents the rectangular bounding box data of the left areola. r Rectangular data of the right areola, s x The x-coordinate represents the axis of symmetry. l The x-coordinate of the bottom left point of the left areola rectangle is represented by the x-coordinate. r The x-coordinate of the bottom left point of the rectangle representing the right areola is y. lThis represents the y-coordinate of the bottom left point of the left areola rectangle. r The width represents the ordinate of the bottom left point of the rectangle containing the right areola. l This represents the width of the rectangle containing the left areola. r The width and height of the rectangle representing the right areola are indicated. l This indicates the height of the rectangle representing the left areola. r This indicates the height of the rectangle representing the right areola.

[0103] From the above formula, we can calculate that x = s x This is the desired axis of symmetry. The specific result for the axis of symmetry of the areola edge is as follows: Figure 3 As shown.

[0104] Step 4: Calculate the symmetry ratio of the left and right areolas based on the axis of symmetry of the breast. Regarding the determination of the symmetry ratio of the areola edges, this embodiment uses a mathematical method to calculate the number of symmetrical points by comparing the symmetrical distances of the left and right areolas about the axis of symmetry, thereby calculating the symmetry ratio. Based on the obtained axis of symmetry data, symmetrical areola edges are found by spreading the same distance outwards from the axis of symmetry, thus calculating the symmetry ratio. Angle transformation is incorporated to address the issue of non-horizontal areola edge symmetry. Figure 4 As shown, in this embodiment, point A1 on the edge is used for symmetry judgment; the other points on the edge can be judged in the same way.

[0105] When the left and right areolas are on the same horizontal plane, the symmetry ratio of the left and right areolas can be calculated according to formula (8).

[0106]

[0107] Where c represents the total number of areolar edge points symmetrical about the axis of symmetry on both sides, t represents the total number of areolar edge points on both sides, and r represents the symmetry ratio.

[0108] When the left and right areolas are not on the same horizontal plane, the midpoint of the height of the left and right areola rectangles can be obtained according to formula (9).

[0109]

[0110]

[0111] Where C l C r These are the midpoints of the heights of the left and right areola rectangles, respectively.

[0112] The first pixel on the edge of the left areola is traversed sequentially, and the cosine of the offset angle of the first pixel relative to the midpoint of the height of the left areola rectangle is calculated according to formula (10).

[0113]

[0114]

[0115]

[0116] Where A1 is the first pixel with coordinates (x1, y1), and A1B1 represents the first pixel along line C. l The diffusion length in the direction is the length between the first pixel and the midpoint of the height of the left areola rectangle, with an offset angle of ∠A1C. l B1 = θ1,

[0117] The second pixel in the rectangle of the right areola is obtained based on the cosine value of the offset angle and the diffusion length. If the second pixel is located at the edge of the right areola, it means that the first pixel and the second pixel are symmetrical pixels. Otherwise, it means that the first pixel does not have symmetrical pixels. The number of symmetrical pixels is counted and used as the total number of edge points of the left and right areolas that are symmetrical about the axis of symmetry. The symmetry rate of the left and right areolas is calculated according to formula (8).

[0118] Specifically, assuming that the parallel diffusion d along both sides of the symmetry axis is equivalent to the diffusion towards C... l C r Diffusion to the left and right respectively The distance. Taking the coordinates of point A1 on the edge as (x1, y1), then this is equivalent to moving towards C. l Let the length of the diffusion A1B1 be denoted as ∠A1C. l Given B1 = θ1, we can calculate cosθ1 corresponding to point A1 on the edge. If the point is A4 on the edge, the corresponding angle is obtuse, and its cosine value is negative. Based on the calculated cosθ1 value, we can find the point with C on the right. r Check if any points that extend equidistant to the right and have the same cosine value lie on the edge of the right areola. If so, increment the count. Repeat this process for all points on the edge of the left areola to arrive at the final count.

[0119] Calculate the area and perimeter of the left and right areolas respectively. Calculate the area variance and perimeter variance of the left and right areolas based on their areas and perimeters. This calculation includes using formulas (11) and (12).

[0120]

[0121]

[0122]

[0123]

[0124] Where C1, S1, C2, and S2 are the area, perimeter, area, and perimeter of the left areola, respectively; δ1 is the variance of the perimeter of the left and right areolas; and δ2 is the variance of the area of ​​the left and right areolas.

[0125] Considering the reality that the size of the areola on the left and right sides cannot be exactly the same, directly subtracting the perimeter and area would be too absolute and would not yield a satisfactory evaluation result. Therefore, we further measure the fluctuation of the size of the left and right areolas by calculating the variance δ1 of the perimeter and the variance δ2 of the area. The smaller the variance, the closer the sizes of the left and right areolas are.

[0126] Step 5: Assess the symmetry level of the breasts based on the symmetry ratio, area variance, and perimeter variance of the left and right areolas, as well as the area and perimeter of the left and right areolas, and obtain the assessment results.

[0127] Specifically, based on the symmetry ratio r, perimeter variance δ1, and area variance δ2, corresponding numerical relationships are designed to derive the final rating index. A rating index is designed for the symmetry of the left and right areolas, divided into four levels: A, B, C, and D. The determination of different levels is as follows:

[0128] If C k -C avg ≤1, S k -S avg When the ratio is ≤1.5, i.e. δ1≤1, δ2≤2.25, and the symmetry ratio r≥0.95, the left and right areolas are considered to be very close and are classified as Grade A.

[0129] If 1 < C k -C avg ≤1.5, 1.5<S k -S avg When the ratio is ≤2, i.e., 1<δ1≤2.25, 2.25<δ2≤4, and the symmetry ratio is 0.9≤r<0.95, the left and right areolas are considered to be close and are classified as grade B.

[0130] If 1.5 < C k -C avg ≤2、2<S k -S avg When the ratio is ≤2.5, i.e. 2.25<δ1≤4, 4<δ2≤6.25, and the symmetry ratio is 0.85≤r<0.9, the left and right areolas are considered to be relatively close and are classified as grade C.

[0131] If C k -Cavg >2 (δ1>4) or S k -S avg If the value is greater than 2.5 (δ2 > 6.25) or the symmetry ratio r < 0.85, this invention considers the difference between the left and right areolas to be too large and classifies it as grade D.

[0132] A breast symmetry assessment system based on automatic midline contour localization includes a breast image acquisition module, an image segmentation module, a left and right areola localization module, a left and right areola symmetry axis calculation module, a left and right areola symmetry rate calculation module, a left and right areola area and perimeter calculation module, and a breast symmetry assessment module.

[0133] The chest image acquisition module is used to acquire chest images.

[0134] The image segmentation module is used to segment the left and right areolas in a chest image based on a global optimization algorithm.

[0135] The left and right areola positioning modules are used to obtain the positions of the left and right areolas respectively based on the bounding box regression positioning algorithm.

[0136] The module for calculating the left and right areola symmetry axes is used to calculate the abscissa of the breast symmetry axis based on the positions of the left and right areolas.

[0137] The left and right areola symmetry calculation module is used to calculate the symmetry ratio of the left and right areolas based on the breast's axis of symmetry.

[0138] The left and right areola area and perimeter calculation modules are used to calculate the area and perimeter of the left and right areolas respectively, and to calculate the area variance and perimeter variance of the left and right areolas based on the area and perimeter of the left and right areolas.

[0139] The breast symmetry assessment module is used to assess the symmetry level of the breast based on the symmetry rate, area variance, and perimeter variance of the left and right areolas, as well as the area and perimeter of the left and right areolas, and to obtain the assessment results.

[0140] Furthermore, this embodiment also includes a dedicated APP formed by a breast symmetry assessment system that automatically locates breasts based on their central contour. The APP includes a breast image acquisition section, an image segmentation section, a left and right areola localization section, a left and right areola symmetry axis calculation section, a left and right areola edge symmetry rate calculation section, a left and right areola area and perimeter calculation section, a breast symmetry assessment section, and a system design section. The overall architecture diagram of the APP in this embodiment is shown below. Figure 5As shown, the page includes three sections: Home, Information, and My Profile. The Home page has functions for uploading photos and images. The Information section includes two subpages: All and Follow-up. The My Profile section has three functions: Account Security, Service Agreement, and Privacy Policy. The designs for the Home page, Patient Information, Breast Symmetry Evaluation Results, and User Personal Information are as follows: Figure 6 , Figure 7 , Figure 8 , Figure 9 As shown.

[0141] Overall beneficial effects:

[0142] This invention provides a method and system for breast symmetry assessment based on automatic midline contour positioning. By acquiring bilateral breast images of the patient in a supine position with different supine angles during surgery, the system accurately segments the left and right areolas in the images based on a global optimization algorithm and accurately locates the positions of the left and right areolas based on a bounding box regression positioning algorithm. Through non-contact measurement and symmetry assessment of the breast, this invention assists users in improving the symmetry of the patient's breast during surgery.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for assessing breast symmetry based on automatic midline contour localization, characterized in that, include, Step 1: Obtain the breast image and segment the left and right areolas in the breast image using a global optimization algorithm. Step 2: Obtain the positions of the left and right areolas respectively based on the bounding box regression localization algorithm. Step 3: Calculate the x-coordinate of the breast's axis of symmetry based on the positions of the left and right areolas. Step 4: Calculate the symmetry ratio of the left and right areolas based on the axis of symmetry of the breast. Calculate the area and perimeter of the left and right areolas respectively. Based on the area and perimeter of the left and right areolas, calculate the variance of the area and the variance of the perimeter of the left and right areolas. Step 5: Assess the symmetry level of the breasts based on the symmetry rate, area variance, and perimeter variance of the left and right areolas, as well as the area and perimeter of the left and right areolas, and obtain the assessment results. Step three includes calculating the abscissa of the breast symmetry axis according to formula (2). (2) in Represents the rectangular data of the left areola. The rectangular data of the right areola. The x-coordinate represents the axis of symmetry. This represents the x-coordinate of the bottom left point of the rectangle containing the left areola. This represents the x-coordinate of the bottom left point of the rectangle containing the right areola. This represents the ordinate of the bottom left point of the rectangle containing the left areola. This represents the ordinate of the bottom left point of the rectangle containing the right areola. This indicates the width of the rectangle representing the left areola. This indicates the width of the rectangle representing the right areola. This indicates the height of the rectangle representing the left areola. Indicates the height of the rectangle representing the right areola; The calculation of the symmetry ratio of the left and right areolas based on the axis of symmetry of the breast includes, When the left and right areolas are on the same horizontal plane, the symmetry ratio of the left and right areolas can be calculated according to formula (3). (3) in, This represents the total number of points on the left and right areola edges that are symmetrical about the axis of symmetry. This indicates the total number of points on the edges of the left and right areolas. Represents the symmetry ratio. When the left and right areolas are not on the same horizontal plane, the midpoint of the height of the left and right areola rectangles is obtained according to formula (4). (4) in , These are the midpoints of the heights of the left and right areola rectangles, respectively. Iterate through the first pixel on the edge of the left areola, and calculate the cosine of the offset angle of the first pixel relative to the midpoint of the height of the left areola rectangle according to formula (5). (5) in, Let be the first pixel, with coordinates as , Indicates the first pixel point along The diffusion length in the direction, with an offset angle of . , The second pixel in the rectangle of the right areola is obtained based on the cosine of the offset angle and the diffusion length. If the second pixel is located at the edge of the right areola, it indicates that the first pixel and the second pixel are symmetrical pixels. Otherwise, it indicates that the first pixel does not have a symmetrical pixel. The number of symmetrical pixels is counted and taken as the total number of edge points of the left and right areolas that are symmetrical about the axis of symmetry. The symmetry ratio of the left and right areolas is calculated according to formula (3). The calculation of the area variance and perimeter variance of the left and right areolas based on the area and perimeter of the left and right areolas includes calculation according to formulas (6) and (7). (6) (7) (8) (9) in, , , and These are the area and circumference of the left areola, the area and circumference of the right areola, respectively. The sum of the variances of the perimeters of the left and right areolas The variance of the area of ​​the left and right areolas.

2. The breast symmetry assessment method based on automatic midline contour positioning according to claim 1, characterized in that, The segmentation of the left and right areolas in the chest image based on the global optimization algorithm includes designing the inter-class variance according to formula (1). (1) in, Indicates the percentage of pixels in the areola. This indicates the percentage of pixels in the remaining part of the breast. This represents the average gray level of the areola. This represents the average gray level of the rest of the breast. Represents the variance between classes. Set the number of iterations for the global optimization algorithm, and iteratively train the chest image using the global optimization algorithm. Calculate the value of the inter-class variance at each iteration. When the inter-class variance is at its maximum, segment the left and right areolas of the chest image.

3. A breast symmetry assessment system based on automatic midline contour positioning, characterized in that, It includes modules for acquiring breast images, image segmentation, locating left and right areolas, calculating the symmetry axis of left and right areolas, calculating the symmetry rate of left and right areolas, calculating the area and perimeter of left and right areolas, and assessing breast symmetry. The chest image acquisition module is used to acquire chest images. The image segmentation module is used to segment the left and right areolas in a chest image based on a global optimization algorithm. The left and right areola positioning modules are used to obtain the positions of the left and right areolas respectively based on the bounding box regression positioning algorithm. The module for calculating the left and right areola symmetry axes is used to calculate the abscissa of the breast symmetry axis based on the positions of the left and right areolas. The left and right areola symmetry calculation module is used to calculate the symmetry ratio of the left and right areolas based on the breast's axis of symmetry. The left and right areola area and perimeter calculation modules are used to calculate the area and perimeter of the left and right areolas respectively, and to calculate the area variance and perimeter variance of the left and right areolas based on the area and perimeter of the left and right areolas. The breast symmetry assessment module is used to assess the symmetry level of the breast based on the symmetry rate, area variance, and perimeter variance of the left and right areolas, as well as the area and perimeter of the left and right areolas, and to obtain the assessment results. The calculation module for the left and right areola symmetry axes calculates the abscissa of the breast symmetry axis according to formula (2). (2) in Represents the rectangular data of the left areola. The rectangular data of the right areola. The x-coordinate represents the axis of symmetry. This represents the x-coordinate of the bottom left point of the rectangle containing the left areola. This represents the x-coordinate of the bottom left point of the rectangle containing the right areola. This represents the ordinate of the bottom left point of the rectangle containing the left areola. This represents the ordinate of the bottom left point of the rectangle containing the right areola. This indicates the width of the rectangle representing the left areola. This indicates the width of the rectangle representing the right areola. This indicates the height of the rectangle representing the left areola. Indicates the height of the rectangle representing the right areola; The calculation of the symmetry ratio of the left and right areolas based on the axis of symmetry of the breast includes, When the left and right areolas are on the same horizontal plane, the symmetry ratio of the left and right areolas can be calculated according to formula (3). (3) in, This represents the total number of points on the left and right areola edges that are symmetrical about the axis of symmetry. This indicates the total number of points on the edges of the left and right areolas. Represents the symmetry ratio. When the left and right areolas are not on the same horizontal plane, the midpoint of the height of the left and right areola rectangles is obtained according to formula (4). (4) in , These are the midpoints of the heights of the left and right areola rectangles, respectively. Iterate through the first pixel on the edge of the left areola, and calculate the cosine of the offset angle of the first pixel relative to the midpoint of the height of the left areola rectangle according to formula (5). (5) in, Let be the first pixel, with coordinates as , Indicates the first pixel point along The diffusion length in the direction, with an offset angle of . , The second pixel in the rectangle of the right areola is obtained based on the cosine of the offset angle and the diffusion length. If the second pixel is located at the edge of the right areola, it indicates that the first pixel and the second pixel are symmetrical pixels. Otherwise, it indicates that the first pixel does not have a symmetrical pixel. The number of symmetrical pixels is counted and taken as the total number of edge points of the left and right areolas that are symmetrical about the axis of symmetry. The symmetry ratio of the left and right areolas is calculated according to formula (3). The calculation of the area variance and perimeter variance of the left and right areolas based on the area and perimeter of the left and right areolas includes calculation according to formulas (6) and (7). (6) (7) (8) (9) in, , , and These are the area and circumference of the left areola, the area and circumference of the right areola, respectively. The sum of the variances of the perimeters of the left and right areolas The variance of the area of ​​the left and right areolas.

Images