Bra system for tumour detection

Abstract

There is disclosed a method and system for detection of a tumor in a breast and / or for calculating breast fibroglandular density. The method includes obtaining a sequence of images of an exterior surface of a breast supported by a bra having optical elements thereon, during movements of the breast, obtaining a three-dimensional model of the breast, isolating the movements of the breast relative to a thoracic reference, processing the images to identify surfacic vibrations of the breast of the exterior surface, as a function of the isolated movements, analyzing the surfacic vibrations and deformation as a function of the three-dimensional model of the breast, defining a three- dimensional distribution of internal stiffness of the breast using the analyzed surfacic vibrations and deformation with elastography and locating a rigid mass in the breast using the three-dimensional distribution.

Description

BRA SYSTEM FOR TUMOUR DETECTIONCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims the priority of United States Patent Application No. 63 / 665,741, filed on June 28, 2024, and the priority of United States Patent Application No. 63 / 812,514, filed on May 27, 2025.TECHNICAL FIELD

[0002] The application relates to tumour detection and screening in breasts and to a system and equipment for performing tumour detection and for calculating breast density (e.g., fibroglandular density).BACKGROUND

[0003] Breast cancer screening is typically performed using x-ray mammography or x- ray tomosynthesis (3D mammography). One issue for both of these technologies is imaging in women with elevated breast density. Mammography may be less effective in women with elevated breast density, and these women may have a higher risk of developing breast cancer. Recent studies have also found that breast density may be significantly higher based on some ethnical factors, leading to racial disparities in breast cancer detection and survival.

[0004] Stiffness contrast detected by elastography methods has been shown to be effective at diagnosing breast cancer even in women with dense breasts, but there are no current breast screening or breast monitoring systems on the market based on elastography methods. Currently available elastography systems use ultrasound or MRI, which are labor intensive and expensive to use for regular breast screening.SUMMARY

[0005] In a first aspect, there is provided a system for detection of a tumor in a breast, comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining a sequence of images of an exterior surface of a breast supported by a bra having optical elements thereon, during movements of the breast; obtaining a three-dimensional model of the breast; isolating the movements of the breast relative to a thoracic reference; processing the images to identify surfacic vibrations of the breast of the exterior surface, as a function of the isolated movements; analyzing the surfacic vibrations and deformation as a function of the three-dimensional model of the breast; defining a three-dimensional distribution of internal stiffness of the breast using the analyzed surfacic vibrations and deformation with elastography; and locating a rigid mass in the breast using the three- dimensional distribution.

[0006] In accordance with a second aspect, there is provided a system for calculating breast fibroglandular density, comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining a sequence of images of an exterior surface of a breast supported by a bra having optical elements thereon, during movements of the breast; obtaining a three- dimensional model of the breast; isolating the movements of the breast relative to a thoracic reference; processing the images to identify surfacic vibrations of the breast of the exterior surface, as a function of the isolated movements; analyzing the surfacic vibrations and deformation as a function of the three-dimensional model of the breast; defining a three-dimensional distribution of internal stiffness of the breast using the analyzed surfacic vibrations and deformation with elastography; and calculating and outputting breast density for the breast using the three-dimensional distribution.

[0007] Further in accordance with the first aspect and / or the second aspect, for example, the processing unit is configured to dynamically adjust illumination parameters of a lamp to optimize light contrasts and intensities in the obtained sequence of images.

[0008] Still further in accordance with the first aspect and / or the second aspect, for example, the lamp includes a plurality of lights placed at predefined positions and orientations with respect to the user, the intensity of each said lights being selectively adjustable by the processing unit.

[0009] Still further in accordance with the first aspect and / or the second aspect, for example, the three-dimensional model of the breast is obtained using a monocular depth estimation and / or depth camera information.

[0010] Still further in accordance with the first aspect and / or the second aspect, for example, the three-dimensional model is obtained using a triangulation technique connecting surfacic points detected in the sequence of images.

[0011] Still further in accordance with the first aspect and / or the second aspect, for example, the optical elements include dots with a contour density between 70% and 95%, the dots being distributed randomly or in a predefined pattern.

[0012] Still further in accordance with the first aspect and / or the second aspect, for example, the optical elements are arranged in a speckle pattern characterizing predefined regions in the bra using specific patterns, said obtaining the sequence of images is performed by obtaining images of the predefined regions.

[0013] Still further in accordance with the first aspect and / or the second aspect, for example, the specific patterns include a dot density respective to the corresponding predefined region.

[0014] Still further in accordance with the first aspect and / or the second aspect, for example, the processing unit is configured to control a camera to acquire the sequence of images by providing the camera with an external signal.

[0015] Still further in accordance with the first aspect and / or the second aspect, for example, the surfacic vibrations are computed by Fourier transformation of the isolatedmovements as a function of time and as a function of the external signal, said external signal including periods delimited between a respective rising edge and falling edge.

[0016] Still further in accordance with the first aspect and / or the second aspect, for example, the sequence of images includes images of a calibration object having calibration markers, further wherein the processing unit is configured to estimate camera parameters using the images of the calibration object.

[0017] Still further in accordance with the first aspect and / or the second aspect, for example, the three-dimensional distribution is obtained by combining stiffness values from sequences of different movements or different imaging selections.

[0018] Still further in accordance with the first aspect and / or the second aspect, for example, the combined stiffness values are aligned in a single combined coordinate system and averaged using mean and variance calculations.

[0019] Still further in accordance with the first aspect and / or the second aspect, for example, the processing unit is configured to target regions of interest within the sequence of images using a spatial attention mechanism or a pretrained convolutional neural network.

[0020] Still further in accordance with the first aspect and / or the second aspect, for example, the three-dimensional model obtained using at least one of a pretrained convolutional neural network (CNN) and a 3D mesh regression based on the targeted regions of interest.

[0021] Still further in accordance with the first aspect and / or the second aspect, for example, the thoracic reference includes thoracic markers placed on the chest of the user near the breast.

[0022] Still further in accordance with the first aspect and / or the second aspect, for example, the bra includes a layer of adhesive material to ensure adhesion between the bra and the breast surface.

[0023] Still further in accordance with the first aspect and / or the second aspect, for example, the processing unit is configured to detect a position of the user and provide feedback for guiding the user through necessary motions for acquiring the sequence of image.

[0024] Still further in accordance with the first aspect and / or the second aspect, for example, the breast density is calculated by classification of higher breast density with higher stiffness of the breast or a subset of the breast.DESCRIPTION OF THE DRAWINGS

[0025] Reference is now made to the accompanying figures in which:

[0026] Fig. 1 is a diagram of a system for calculating breast density and / or for detection of a tumor in a breast, in accordance with an embodiment;

[0027] Fig. 2 is a flowchart of a method for calculating breast density and / or detecting a tumor in a breast, in accordance with an embodiment;

[0028] Fig. 3 is a front view of a bra used in the system of Fig. 1, in accordance with an embodiment;

[0029] Fig. 4 is a perspective view of the bra of Fig. 3 worn by a user, in accordance with an embodiment;

[0030] Figs. 5A, 5B and 5C are respectively a front view of a 2D camera calibration object, a 3D coordinate calibration object and a 3D model thereof, respectively, in accordance with some embodiments;

[0031] Figs. 6A, 6B and 6C are respectively a meshed 3D model of a breast, a measured point cloud of the breast surface projected on the meshed 3D model and nodes of the mesh rear surface used for boundary conditions, in accordance with an embodiment;

[0032] Figs. 7A and 7B are 3D distribution of internal stiffness of the breast obtained using the system of Fig. 1 and / or the method of Fig. 2, in accordance with some embodiments; and

[0033] Fig. 8 is a schematic diagram of a computing device, in accordance with an embodiment.DETAILED DESCRIPTION

[0034] Referring to Fig. 1 , there is shown a diagram of a system 1 for calculating breast density and / or for detection of a tumor in a breast, in accordance with an embodiment. The breast density may include breast fibroglandular density, thus when reference is made to breast density herein, it includes breast fibroglandular density. The system 1 includes a bra 10 configured to be worn by a user. The bra 10 may have on the surface thereof optical elements 11. The bra 10 is one possible way to have optical elements 11 on the breasts. While the bra 10 shown herein is of the sports-type, other types of bras or breast supports may be used, including ones without shoulder straps, tank tops, bikini top, etc. Optionally, the user may also wear additional optical elements 11 and / or thoracic markers 12 that are placed directly on the skin of the user, on other parts of the bra 10, i.e., not the breast cups, or on chest clothes other than the bra 10. The system 1 also includes a processing device 20, which may be connected to a lighting system or a lamp 21 and to a camera or a plurality of cameras 22.

[0035] The lamp 21 is configured for illuminating at least the chest portion of the user. It will be appreciated that omission of the lamp 21 may be possible. Indeed, in some cases the user may be already illuminated by another light source, such as ambient light. In some cases, the processing device 20 may be configured for controlling the light intensity of the lamp 21 , turning on and / or off the lamp 21 , creating a light pattern with the lamp 21 and so on. Thus, based on its control, the lamp 21 may illuminate continuously, in a varying fashion, may create a stroboscopic effect, etc. The illumination may be altered to provide specific contrasts or shadows to captured images, and illumination parameters may be altered dynamically. The control of the lamp 21 and its illumination may enable a depth camera functionality, for example by varying the wavelength and capturingimages as a function of wavelength. The lamp 21 may include a plurality of lights, which may be placed at predefined positions and orientations with respect to the user. Each light may be adjusted, e.g., at 50%, 75%, 85% and 100% of the total intensity by the processing device 20.

[0036] The processing device 20 may also be connected to an image or video camera 22 for capturing images of the optical elements 11 and optionally the thoracic markers 12. In some embodiments, the system 1 includes a plurality of cameras 22, e.g., four cameras. In other cases, the camera 22 may not be directly connected to the processing device 20; in such case the images can be uploaded to the processing device 20 after being captured. In some cases, the lamp 21 is configured for illuminating the chest of the user to optimize light contrasts and intensities in the captured image. The processing device 20 may be a computing device similar to the one shown and described as 200 herein. In a variant, the processing device 20 may detect the experimental conditions and the image quality of the bra 10 and adjust automatically the light contrasts and intensities of the lamp 21 . In a variant, depending on the light intensity of the lamp 21 and the speckle pattern on the bra, the processing device 20 may adjust automatically the parameters of cameras 22 such as the exposure time and others to improve screening. In a variant, the lamp 21 and / or camera 22 may be part of a portable device, such as a portable phone, a tablet, a computer, a personal camera, for images to be captured locally to then be processed distally, in the manner described herein. The process of capturing images may be done via an application that can provide guidance and feedback to the user.

[0037] Referring now to Fig. 2, there is shown a flowchart of a method 50 for calculating and optionally classifying breast density and / or for detecting a tumor in a breast. Depending on the configuration, a single breast or both breasts of a user may be monitored when performing the method 50.

[0038] At step 51 , a sequence of images of an exterior surface of a breast supported by a bra 10 having optical elements thereon is obtained during the movements of the breast.

[0039] The sequence of images taken at step 51 may be obtained from one or more camera(s) 22. In operation, the person being screened may be asked to move the chestin order to take images of the breasts at multiple positions and in movement. In a variant, the movement is a natural movement of the person being screened, seated or standing upright, though other positions and induced movements are also possible. During the movement, the images of the breasts include the optical elements 11 and optionally the thoracic markers 12. In operation, the number of images captured by the camera(s) 22 may vary, and be, for example, 300 images having a resolution of 1900 x 1200 pixels. In operation, the time for the movement of the person being screened may be for a given duration (e.g., 10 seconds) for each acquisition and the number of acquired images per second may vary, such as to be 60 images / second or higher, as an example. In some cases, an image analysis from the processing device 20 may determine automatically the appropriate position of the person being screened in front of the camera(s) 22. Hence, it may be determined that the breast position or movement is not ideal for screening, and instructions for the adjustments to the position or movement of the breast may be communicated to the person being screened in any appropriate way, e.g., via interface, by a flickering of the light, and / or activation of a light of a given colour. It is also possible to enable movement of the camera(s) 22. In some instances, image analysis from the processing device, 20, may evaluate that the breast position or movement is not ideal for screening, and may reconfigure the operation of one or more of the cameras, 22, to improve screening (i.e. ensure that the full breast surface is captured, ensure that the surface is in focus, ensure that the image is sufficiently illuminated, ensure that the image acquisition speed is sufficient for the performed movements, etc). In a variant, image analysis by the processing device 20, may detect automatically if the bra speckle pattern is good for screening and if the person wears something else that comes down on the breast such as necklace. The processing device 20, may also detect the bra fit of the person being screened and give instructions for adjustment. In a variant, a spatial attention mechanism, can be used to automatically identify regions within the images containing the optical elements 11. In a variant, a pretrained convolutional neural network (CNN), can be used to automatically identify regions containing the optical elements 11. The CNN may use the spatial attention mechanism as additional input. In some cases these variants enable real-time identification of regions containing the optical elements 11. The following equation defines a spatial attention map S(x, y) derived from an input image l(x, y) using frequency domain manipulation:

[0041] First, F( / (x, y)) represents the Fourier Transform of the image, which separates the image into its frequency components. Then |F( / (X, y))| denotes the magnitude (or amplitude spectrum), while 6 y))} represents the phase spectrum. The termlog (|F( / (x, y))|) converts the magnitude to the logarithmic scale to emphasize subtle features. Then, h(u, v) is a low-pass filter applied to the log-magnitude spectrum to capture the average or slowly varying components. Subtracting h(u, v) ■ log (|F( / (x, y)) |) from the original log-magnitude isolates the high-frequency components (i.e., fine details or salient features). This result is combined with the phase spectrum y))} by forming a new complex representation using Euler’s formula, eie, andthen the inverse Fourier Transform F-1is applied to project the modified frequency components back into the spatial domain. Finally, the absolute value |-| gives the intensity of the resulting attention map S(x, y). This process enhances visually or structurally salient regions of the image, emphasizing areas that differ from their surroundings.

[0042] In some cases where multiple cameras 22 are used, the frame rate of each camera 22 may be synchronized. The processing device 20 may provide an external signal to each camera 22, causing images to be acquired in response to triggers in an external signal. In some embodiments, the triggers may correspond to the frequency of the external signal. In this case, the external signal may include a periodic square wave signal including periods delimited between a respective rising edge and falling edge. During these periods, the camera(s) 22 may be configured to acquire an image. The processing device 20 may include a function generator to generate the external signal.

[0043] The external signal may have a predefined duration in which images are acquired by the camera(s) 22. During the image acquisition, the processing device 20 may provide a signal to the light 21 to generate a light pattern indicative of images being captured. The light pattern may be, for instance, a flickering of the light, or activation of a light of a given colour, e.g., a green light. It will be understood that the light pattern is perceivable by the user, and that the user is usually tasked to move the upper body when the light pattern ispresent, in order to capture images of the breasts in movement. In some cases, the processing device 20 may provide other signals before and / or after the image acquisition to the light 21 in order to notify, using a yellow or red light, the user that the image acquisition is about to start and / or that the image acquisition has ended. In some embodiment, the duration of the period in which images are acquired may be of about twenty seconds, as another possibility. In some instances, the external signal may not have a predefined period during which images are acquired, and may be permanently active, to synchronize camera(s) 22. Image analysis from the processing device 20 may evaluate the breast motions and determine the start and end of image acquisition, filter out some of the footage, crop the footage, etc. Start and end points of image acquisition may be communicated by the lamp 21 , as an example.

[0044] In some cases, the number of images and / or the frequency of the external signal may be determined by an operator operating the processing device 20. Once the images have been acquired, the processing device 20 receives the images and creates a 3D model of the breasts based on a 3D definition of the breast obtained either by calibration or by the depth camera, as described further below.

[0045] At step 52, a 3D model of the breast is obtained using, e.g., Direct Linear Transformation (DLT), DLT reconstruction and Digital Image Correlation (DIC). The 3D model may be obtained by analyzing multiple images including the optical elements 11 present on the bra and, optionally, the thoracic markers 12 placed on the chest of the user. The 3D model may be generated prior to screening, or during screening, notably by image processing, such as during calibration. In a variant, the camera(s) 22 is(are) a depth camera that has(have) the capacity to contribute to the generation of the 3D model of the breast. In a variant, the 3D model of the breast is defined by the front and rear surfaces of the breast. The front surface of the breast may be defined from the 3D positions of points on the breast surface, called the point cloud, determined by DIC methods. The rear surface may be positioned at a specified distance from the point cloud described above. This rear surface may have a specified form, such as a circle, oval, ellipse, or other form determined by the 3D camera images, and may be flat or curved. In one embodiment, the rear surface of the breast is determined by placing an oval on a plane that best respects a specified distance from the rear edge of the front surface ofthe breast. In one embodiment, this specified distance between the rear edge of the front surface and the plane of the rear surface is 3mm. In one embodiment, the outer edge of the rear surface is an oval drawn on this plane, such that a straight line connecting the rear edge of the front surface to the outer edge of the rear surface is within 45 degrees of the orientation of the front surface at the rear edge. In these cases, the front surface is extended to include the surface between the rear edge of the point cloud and the outer edge of the rear surface, defined by a straight line. In some cases, the rear surface may be obtained by a suitable algorithm that defines a plane passing closest to all the free contour points of the front surface with a minimum distance of 3 mm and the projection of the free contour points of the front surface onto the rear surface may be used to create the 3D model. In some cases, the rear surface may be determined by the point cloud of the front surface determined by the DIG method using a pretrained neural network, for example a CNN or a ll-Net. In these cases, the neural network is trained to reproduce the rear surface as defined above, based on previous cases. The determination of the rear surface may also take into account other factors, such as the motion measured by the DIC calculation. In some cases, the 3D model may be obtained using robust monocular depth estimation methods to compute relative inverse depth from a single image (e.g. https: / / doi.org / 10.48550 / arXiv.2401 .10891 , https: / / pytorch.org / hub / intelisl midas v2 / ). In this case one image, or a series of images, may be used to create a 3D model of the breast. In some cases, the 3D model may be obtained using 3D Mesh Regression using pretrained Light Weight CNNs (e.g. https: / / developers.googleblog.com / en / mediapipe-3d-face-transform / and https: / / arxiv.org / abs / 1907.05047). This approach may be based on predefined Landmarks for the breast geometry. As another option, the 3D model of the breast may be obtained using SOLIDWORKS and the front and rear surfaces of the breast determined by the approaches described above. As yet another option, the 3D model of the breast may be obtained by applying alpha wrap techniques to point clouds representing the front and rear surfaces defined by the methods described above.

[0046] In some embodiments, movement of the optical elements 11 may be analyzed in one or more regions of interest selected by an operator of the processing device 20 or detected automatically by the processing device 20 based on the optical elements 11 and the thoracic markers 12. It will be appreciated that the one or more regions of interest aregenerally regions where the optical elements 11 are distinguishable from one another. In operation, the operator, or the processing device 20, may select seed points inside the one or more regions of interest. The seed points may thereafter be used for calculating the surfacic movements and vibrations of the breast by DIC.

[0047] The region(s) of interest may be detected automatically by the processing device 20. This selection can be based on any combination of the following elements: the optical elements 11 and the thoracic markers 12; an initial analysis of the movements of the breast including: the deformation of the breast, the relative movement of the breast region compared to the breast center, the relative movement of the breast surface compared to best estimates of the rigid body movement of the torso; the 3D profile of the breast determined from the 3D model of the breast (step 52). In some cases, the speckle of the bra may include predefined regions characterized by specific patterns that may be used to select automatically the regions of interest. In some cases, the regions of interest containing optical elements 11 and the thoracic markers 12 may be automatically selected using a pretrained neural network, for example a ll-Net. 2D displacements and deformations of thousands of points identified inside the regions of interest may be estimated using DIC with motion tracking algorithms and then the 3D movements of the breast are obtained. The number of points identified inside the regions of interest by DIC analysis can be adjusted by the operator of processing device 20 and may depend on the size of the regions of interest and the bra speckle pattern density.

[0048] At step 53 the movements of the breast are isolated relative to a thoracic reference. When acquiring images of the breast in movement, surface vibrations are measured on the surface of the bra 10 via the optical markers 11 and on the chest of the user, e.g., via the thoracic markers 12. The isolation of the surface vibrations may be computed by separating the movement of the breast with respect to the movement of the chest using a suited algorithm. In a variant, image processing may be performed from footage of the camera(s) 22 to isolate the movement of the optical markers 11 , i.e., without the need for a thoracic marker 12. In some cases, the movement of the surface of the breast may be determined by a sequence of 3D models of the breast, such as by a sequence of images treated with robust monocular depth estimation methods. In these cases, material points, or landmarks, are identified in two sequential 3D models of thebreast, such as models generated by monocular depth estimation performed on two sequential images from a series of images. The movement of the breast surface is then determined by calculating the 3D displacement of each individual material point or landmark from one model to the next, where the displacement of each landmark represents the movement of that point on the breast surface during the period between the two acquired images. In some instances, the algorithm to separate movement of breast with respect to the movement of the chest is an analysis of the best approximation for the rigid body movement of the chest / torso of the user, (for example, https: / / doi.ora / 10.1016 / 0021-9290(94)00116-L).

[0049] In other embodiments, the chest of the user may be restrained in order to minimize the movement of the chest when the user is in motion, so that the majority of the movement and surface vibrations are located in the breast. In such a case, the surface vibrations of the chest may be neglected in the reconstruction of the 3D model of the breast.

[0050] At step 54 the images are processed to identify surfacic vibrations of the breast of the exterior surface, as a function of the isolated movements. The processing device 20 computes the surfacic vibrations of the breasts by Fourier transformation of the isolated movements as a function of time and of the external signal. The processing device 20 also computes the projection of these surfacic vibrations of the breasts onto the 3D model of the breasts based on the 3D calibration or by the depth camera information as described above. If using monocular depth estimation methods, the surface vibrations may be converted to units of length via an image from a calibration object within the view of the camera 22. The projected points on the 3D mesh are illustrated in Fig. 6B and the 3D complex displacements of these points at different frequency may be used to evaluate the 3D stiffness distribution.

[0051] At step 55, the surfacic vibrations and deformation are analyzed as a function of the 3D model of the breast. The identified surfacic vibrations and deformation may be juxtaposed as a function of time in order to generate a time-variant, and more precisely, time-harmonic, model of the breast, including the identified surfacic vibrations.

[0052] Because the method described is dynamic, the thoracic reference is not static, such that the inertial forces associated with the thoracic reference must be taken into consideration. Hence, the breast surface may be isolated relative to the thoracic reference, the surface vibrations are processed, and the isolated surface vibrations are analyzed with the 3D model, as a function of the implicit inertial forces of the thoracic reference, with the thoracic movement set to zero, as the surfacic vibrations of the breast are relative to the thoracic reference, in this case. As another approach, the surfacic vibrations of the breasts and the vibrations of the thoracic reference are both processed from the images. The two resulting vibration fields are separated. The surfacic vibrations with the 3D model are analyzed, as a function of the non-zero, thoracic reference, within a static reference with no implicit inertial forces. In both cases, the thoracic movement may be applied as a boundary condition to the rear surface of the 3D model of the breast. The movements of the thoracic reference may be related to the surfacic movements of the breast by direct application of the thoracic reference movement to the rear surface of the 3D model of the breast, or by coupling via a rigid or non-rigid transformation as a function of the relative position of the thoracic reference and the rear surface of the 3D model of the breast.

[0053] At step 56, a 3D distribution of internal stiffness of the breast is defined using the analyzed surfacic vibrations and deformation with elastography. It will be appreciated that elastography generally refers to a class of imaging that maps the elastic properties and stiffness of a soft tissue in order to identify the presence or status of a disease. Elastography imaging may use the difference between a set of measured movements and movements calculated by a 3D model to establish the map of internal stiffness within a soft tissue. In some cases of elastography imaging, the map of internal stiffness of the soft tissue can be estimated from purely surfacic movements. In some cases, internal stiffness may be estimated without the use of a 3D model. In some cases, the vibrations and / or elastography results obtained from a sequences of different movements or different imaging selections may be combined to determine a final estimate of the internal stiffness. In the present case, the properties of the surfacic vibrations and, optionally, the properties of the thoracic movement of the breast, may be analyzed to extract the 3D distribution of internal stiffness of the breast. In some embodiments, the internal stiffness includes a dynamic stiffness representative of the stiffness and attenuation of a givenpoint in the 3D distribution as a function of vibration frequency. In some cases, this dynamic stiffness may be represented by a power-law model, where the stiffness is an exponential function of the vibration frequency. The boundary conditions applied to the 3D model for step 56 may be generated from the optical elements 11 and the thoracic markers 12. For the boundary conditions, the 3D complex displacements of each node on the mesh rear surface are determined as shown in Fig. 6C. It may include the analysis of the best approximation for the rigid body movement of those elements and markers, or some portion thereof, or of the chest and torso region as a whole (see point 13). These conditions may be applied uniformly to the back surface, or rear region of the 3D breast model, or as some distribution of conditions across these surfaces or regions. In some cases, this distribution of conditions is determined from the best approximation for the rigid body movement determined above. The distribution of internal stiffness may be determined using established elastography methods, including methods based on physics informed physics-informed machine learning (PIML), or other machine learning based methods using CNNs, ll-Nets, or Generative-Adversarial Networks (GANs).

[0054] At step 57, the breast density may be calculated, and / or a rigid mass in the breast is located using the three-dimensional distribution. It will be understood that the stiffness of a breast is generally low compared to the stiffness of a rigid mass. Therefore, a local variation of the stiffness unexpected for a healthy breast may be correlated to the presence of a rigid mass in this region. In some embodiments, the mass may include a tumor or other carcinogenic masses of the like. In some cases, the 3D stiffness distributions from different sequences of movements of the chest or torso can be combined in a single combined coordinate system, where the multiple stiffness values available at each internal location can be combined to estimate the probable stiffness at that point via mean and variance calculation. In some cases, these combined probabilities are calculated based on the stiffness distribution relative to a background value determined either from its physical position or based on a quantile of the stiffness distribution. In some cases, the distribution of viscoelastic and diffusive properties of the breast tissue are determined by the method above, and the rigid mass is located based on the combined contrast of these different mechanical properties. In some cases, the stiffness, viscoelastic or diffusive properties of the whole breast, or some subset of thewhole breast, are used to estimate the breast density by classification of higher breast density with higher stiffness of the whole breast or some subset of the whole breast.

[0055] As best seen in Figs. 3 and 4, there are shown examples of a bra 10 used during clinical studies. A pattern of optical elements 11 is applied on each bra 10, and may be made of small dots with a diameter of approximately 1.5 mm with a contour density between 70 and 95%, which may be distributed randomly or in a predefined pattern on the bra 10. The small dots on the bras 10 may be made using paint and may not affect the structure of the bra 10. In some cases, the patterns and the density of the small dots of the optical elements 11 may be controlled. In some embodiments, the pattern of the optical elements 11 may be based on random distribution of small dots that aims to identify thousands of points on the surface of the bra 10. The 3D motion of the thousands of points identified on the surface of the bra 10 may be determined and processed to obtain the 3D stiffness distribution within the breast of the user. The pattern of dots on the optical elements may include colors and combinations of different dots densities for region segmentations to identify those which are suitable for boundary conditions and surface breast data. The pattern on the bra 10 can be integrated into the bra 10 manufacturing process. It will be appreciated that the optical elements 11 and the thoracic markers 12 may have other forms than small dots, such as complex visual motifs, or a combination of small dots and other forms, given that the general guidelines of contour density and pattern are consistent for DIG. Different predefined regions can be made on the bra 10 using specific patterns of dots for each region and these predefined regions are used for automatic region of interest selection.

[0056] The fabric of the area which contains the breast could be chosen so as to create adhesion between the breast and the bra 10. To ensure adhesion between the bra 10 fabric and the breast surface, a thin layer of silicon gel, or other adhesive material, can be added to ensure that there is no relative motion between the breast surface and the bra 10 surface. The bra 10 cup, in direct contact with the breast, may be made from supple, elastic tissue to minimize impact on the dynamic and / or mechanical behavior of the breast as the wearer moves. This region may be flexible and extensible such that the movement of the breast surface itself is not modified by the presence of the bra 10. In addition, regions of the bra 10 external to the breast cup may be rigid, in order to focusbody motion within the breast itself and to constrain the torso to correspond to the boundary conditions applied for breast stiffness characterization.

[0057] In some cases, prior to step 51 in the method 50, a calibration may be performed to calibrate the camera. As shown in Fig. 5A, there is shown a calibration sample 100 having a body 101 and calibration markers 102 thereon. Fig. 5B shows a 3D coordinate calibration sample 103 having a body 104 and calibration markers 105 thereon. Fig. 5C shows a 3D model 106 of the calibration sample 103 including a plurality of virtual markers 107 corresponding to the calibration markers 105. The calibration of the camera(s) 22 may be performed by capturing a plurality of images using the calibration markers 102, which may have a checkerboard pattern, to estimate the parameters of a lens and the image sensor of the camera(s) 22, using multiple images of the checkerboard calibration pattern. These parameters may then be used to correct lens distortion. The camera parameters may include intrinsics, extrinsics, and distortion coefficients. An image of the calibration markers 102 may be captured by the camera(s) 22 during the calibration from multiple positions. While the body 101 of the calibration sample 100 has a rectangular shape and the calibration markers 102 are configured in a checkered pattern as depicted in Fig. 5a, it will be appreciated that various other types and shapes of calibration sample 100 may apply, depending on the implementation.

[0058] In some embodiments, the processing device 20 may be configured for estimating camera parameters. As such the 3D model is generated using acquired sets of 2D images of the calibration markers 102. Using a suited algorithm, the processing device 20 is configured for obtaining a camera matrix using the extrinsic and intrinsic parameters. The extrinsic parameters generally represent a rigid transformation from a 3D world coordinate system to the camera’s local coordinate system. The extrinsic parameters may include a rotation and a translation to define the plane of the calibration markers in the camera’s local coordinate system. The intrinsic parameters generally represent a projective transformation from the camera’s local coordinates into the 2D image coordinates, and may include the focal length, the optical centre or the principal point, and the skew coefficient. It will be appreciated that the calibration procedure may be configured to remove radial and / or tangential distortion. In some embodiments, a single image of a 3D coordinate calibration object 103 captured by the camera(s) 22 may beused to define a unique 3D world coordinate system and identify the transformation(s) from the camera(s) local coordinate system(s) to this world coordinate system. The 3D model 106 shown in Fig. 5C shows the calibration markers 105 from the 3D coordinate calibration object 103 in this 3D world coordinate system.

[0059] Now referring to Fig. 6A, there is shown an exemplary meshed 3D model 120 of a breast supported by a bra 10. The rendering method used to produce the meshed 3D model 120 uses a triangulation technique to connect surfacic points detected in 3D using the method 50. The triangles obtained using the triangulation technique are then subdivided into regions 121 each having a particular dimension. Using a suited software, the boundary regions 121 may be smoothed to create a continuous surface representative of the breast in motion. It will be appreciated that various types of 3D model rendering methods may be contemplated, and that current technology generated the meshed 3D model 120 depicted in Fig. 6A. In some cases, meshes using forms other than triangles may be used.

[0060] Referring to Fig. 6B, there is shown measurement points 122 of the breast surface projected onto the 3D mesh model 120. Fig. 6C shows the rear surface of the breast and the nodes 123 used for boundary condition application. The 3D complex displacements of these points 122 and 123 may be calculated at different frequencies and used to estimate the 3D stiffness of the breast.

[0061] Now referring to Figs. 7A and 7B, there are shown 3D distributions 150, 151 of internal stiffness of a breast obtained using the system 1 of Fig. 1 and / or the method 50 of Fig. 2. The 3D distribution 150 includes a volume 153 of a healthy breast, meaning that the breast does not include a mass. The volume 153 has a low stiffness distribution, which is typically the case for healthy breasts. The 3D distribution 151 includes a volume 154 of an unhealthy breast including a mass 155. As seen in Fig. 7B, the mass 155 has a higher stiffness than the rest of the volume 154. It will be appreciated that the 3D distributions 150, 151 are exemplary, and that other types of 3D distributions may apply.

[0062] Some of the steps of the method may be implemented remotely. The computer processing the data acquired may be remote from the data acquisition itself, and from theperson wearing the bra, the lighting, camera, etc. In some embodiments, the operation of the device may be automated with Artificial Intelligence implemented in device 20 or in computing device 200 described below, which automate the use of the system by the patient, by automatically detecting the position of the patient, guiding the patient through the necessary motions required to acquire the necessary image sequences, and providing feedback to the patient as to whether the acquired data is adequate or should be reacquired. In some embodiments, some or all computations required to compute intermediate results such as localizing the breast in the image, localizing the region of interest in the image, modeling the breast in 3D, accelerating some computations during the stiffness determination, comparing the 3D stiffness images to known prior images of other patients, determining automatically if a suspicious lesion might be present in the breast, might be assisted, complemented, replaced or accelerated with artificial intelligence.

[0063] In addition to detecting stiff breast abnormalities, such as cancerous tumors, this system and / or method may also be used to evaluate the fibroglandular density of the breasts, which may or may not contain breast abnormalities. In addition to detecting stiff breast abnormalities, such as cancerous tumors, the system and / or method may also be used to detect other breast abnormalities, which may be softer or stiffer than the normal breast tissue. In addition to detecting stiff breast abnormalities, such as cancerous tumors, the system and / or method may also be used to evaluate the condition and function of breast prosthesis, or other surgical implant within the breast. When stiffness is discussed herein with respect to the system and / or the method, it may be in terms of absolute stiffness or relative stiffness, and may include damping properties as well.

[0064] With reference to Fig. 8, an example of a computing device 200 is illustrated. For simplicity only one computing device 200 is shown but the system may include more computing devices 200 operable to exchange data. The computing devices 200 may be the same or different types of devices. The processing device 20 may be in communication and implemented with one or more computing devices 200.

[0065] The computing device 200 comprises a processing unit 201 and a memory 202 which has stored therein computer-executable instructions 203. The processing unit 201may comprise any suitable devices configured to implement the method described herein such that instructions 203, when executed by the computing device 200 or other programmable apparatus, may cause the functions / acts / steps performed as part of the method as described herein to be executed. The processing unit 201 may comprise, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, a central processing unit (CPU), a graphical processing unit (GPU), an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, other suitably programmed or programmable logic circuits, or any combination thereof.

[0066] The memory 202 may comprise any suitable known or other machine-readable storage medium. The memory 202 may comprise non-transitory computer readable storage medium, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. The memory 202 may include a suitable combination of any type of computer memory that is located either internally or externally to device, for example random-access memory (RAM), read-only memory (ROM), compact disk readonly memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically-erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like. Memory 202 may comprise any storage means (e.g., devices) suitable for retrievably storing machine- readable instructions 203 executable by processing unit 201 .

[0067] The methods and systems described herein may be implemented in a high level procedural or object oriented programming or scripting language, or a combination thereof, to communicate with or assist in the operation of a computer system, for example the computing device 200. Alternatively, the methods and systems described herein may be implemented in assembly or machine language. The language may be a compiled or interpreted language. Program code for implementing the methods and systems described herein may be stored on a storage media or a device, for example a ROM, a magnetic disk, an optical disk, a flash drive, or any other suitable storage media or device. The program code may be readable by a general or special-purpose programmable computer for configuring and operating the computer when the storage media or deviceis read by the computer to perform the procedures described herein. Embodiments of the methods and systems described herein may also be considered to be implemented by way of a non-transitory computer-readable storage medium having a computer program stored thereon. The computer program may comprise computer-readable instructions which cause a computer, or more specifically the processing unit 201 of the computing device 200, to operate in a specific and predefined manner to perform the functions described herein, for example those described in the method 50.

[0068] Computer-executable instructions may be in many forms, including program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Typically the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0069] The embodiments described herein are implemented by physical computer hardware, including computing devices, servers, receivers, transmitters, processors, memory, displays, and networks. The embodiments described herein provide useful physical machines and particularly configured computer hardware arrangements. The embodiments described herein are directed to electronic machines and methods implemented by electronic machines adapted for processing and transforming electromagnetic signals which represent various types of information. The embodiments described herein pervasively and integrally relate to machines, and their uses; and the embodiments described herein have no meaning or practical applicability outside their use with computer hardware, machines, and various hardware components. Substituting the physical hardware particularly configured to implement various acts for non-physical hardware, using mental steps for example, may substantially affect the way the embodiments work. Such computer hardware limitations are clearly essential elements of the embodiments described herein, and they cannot be omitted or substituted for mental means without having a material effect on the operation and structure of the embodiments described herein. The computer hardware is essential to implement the various embodiments described herein and is not merely used to perform steps expeditiously and in an efficient manner.

[0070] The technical solution of embodiments may be in the form of a software product. The software product may be stored in a non-volatile or non-transitory storage medium, which can be a compact disk read-only memory (CD-ROM), a USB flash disk, or a removable hard disk. The software product includes a number of instructions that enable a computer device (personal computer, server, or network device) to execute the methods provided by the embodiments.

[0071] The above description is meant to be exemplary only, and one skilled in the art will recognize that changes may be made to the embodiments described without departing from the scope of the invention disclosed. Still other modifications which fall within the scope of the present invention will be apparent to those skilled in the art, in light of a review of this disclosure, and such modifications are intended to fall within the appended claims.

Claims

CLAIMS1. A system for detection of a tumor in a breast, comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining a sequence of images of an exterior surface of a breast supported by a bra having optical elements thereon, during movements of the breast; obtaining a three-dimensional model of the breast; isolating the movements of the breast relative to a thoracic reference; processing the images to identify surfacic vibrations of the breast of the exterior surface, as a function of the isolated movements; analyzing the surfacic vibrations and deformation as a function of the three- dimensional model of the breast; defining a three-dimensional distribution of internal stiffness of the breast using the analyzed surfacic vibrations and deformation with elastography; and locating a rigid mass in the breast using the three-dimensional distribution.

2. A system for calculating breast fibroglandular density, comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining a sequence of images of an exterior surface of a breast supported by a bra having optical elements thereon, during movements of the breast; obtaining a three-dimensional model of the breast; isolating the movements of the breast relative to a thoracic reference; processing the images to identify surfacic vibrations of the breast of the exterior surface, as a function of the isolated movements; analyzing the surfacic vibrations and deformation as a function of the three- dimensional model of the breast;defining a three-dimensional distribution of internal stiffness of the breast using the analyzed surfacic vibrations and deformation with elastography; and calculating and outputting breast density for the breast using the three- dimensional distribution.

3. The system of claim 1 or 2, wherein the processing unit is configured to dynamically adjust illumination parameters of a lamp to optimize light contrasts and intensities in the obtained sequence of images.

4. The system of claim 3, wherein the lamp includes a plurality of lights placed at predefined positions and orientations with respect to the user, the intensity of each said lights being selectively adjustable by the processing unit.

5. The system of any one of claims 1 to 4, wherein the three-dimensional model of the breast is obtained using a monocular depth estimation and / or depth camera information.

6. The system of any one of claims 1 to 5, wherein the three-dimensional model is obtained using a triangulation technique connecting surfacic points detected in the sequence of images.

7. The system of any one of claims 1 to 6, wherein the optical elements include dots with a contour density between 70% and 95%, the dots being distributed randomly or in a predefined pattern.

8. The system of any one of claims 1 to 7, wherein the optical elements are arranged in a speckle pattern characterizing predefined regions in the bra using specific patterns, said obtaining the sequence of images is performed by obtaining images of the predefined regions.

9. The system of claim 8, wherein the specific patterns include a dot density respective to the corresponding predefined region.

10. The system of any one of claims 1 to 9, wherein the processing unit is configured to control a camera to acquire the sequence of images by providing the camera with an external signal.

11. The system of claim 10, wherein the surfacic vibrations are computed by Fourier transformation of the isolated movements as a function of time and as a function of the external signal, said external signal including periods delimited between a respective rising edge and falling edge.

12. The system of claim 10, wherein the sequence of images includes images of a calibration object having calibration markers, further wherein the processing unit is configured to estimate camera parameters using the images of the calibration object.

13. The system of any one of claims 1 to 12, wherein the three-dimensional distribution is obtained by combining stiffness values from sequences of different movements or different imaging selections.

14. The system of claim 13, wherein the combined stiffness values are aligned in a single combined coordinate system and averaged using mean and variance calculations.

15. The system of any one of claims 1 to 14, wherein the processing unit is configured to target regions of interest within the sequence of images using a spatial attention mechanism or a pretrained convolutional neural network.

16. The system of claim 15, wherein the three-dimensional model obtained using at least one of a pretrained convolutional neural network (CNN) and a 3D mesh regression based on the targeted regions of interest.

17. The system of any one of claims 1 to 16, wherein the thoracic reference includes thoracic markers placed on the chest of the user near the breast.

18. The system of any one of claims 1 to 17, wherein the bra includes a layer of adhesive material to ensure adhesion between the bra and the breast surface.

19. The system of any one of claims 1 to 18, wherein the processing unit is configured to detect a position of the user and provide feedback for guiding the user through necessary motions for acquiring the sequence of image.

20. The system of claim 2, wherein the breast density is calculated by classification of higher breast density with higher stiffness of the breast or a subset of the breast.

Images