System and method for training in sensor-based x-ray mammography - Patents.com

Abstract

To provide various methods and systems for breast positioning assistance during mammography and image guided interventional procedures.SOLUTION: A sensor detection system (100, 252) is utilized to evaluate one or more of a patient position, a breast position and a breast anatomy to determine if the patient and breast are adjusted to desired positions preferred for a desired view and imaging procedure prior to acquiring X-ray images, and real-time feedback is provided to show the breast and / or patient position to the user based on the evaluation. The evaluation is performed by an artificial intelligence (AI) model (262) stored on the X-ray mammography system (10).SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] Various methods and systems are provided to assist with breast positioning during mammography acquisition and image-guided interventional procedures. In one embodiment, a vision system is utilized in conjunction with an X-ray imaging system to assess one or more of patient position, breast position, and breast anatomy to determine if the patient and / or breast need to be adjusted to a desired position suitable for the desired view and imaging procedure. Additionally, this assessment may be performed by an on-site trained artificial intelligence employed within the imaging system to provide real-time feedback to assist the user in positioning the breast and / or patient. [Background technology]

[0002] FIELD OF THE DISCLOSURE Embodiments of the present invention relate generally to x-ray medical imaging, and more particularly to systems and methods for performing mammography examinations, i.e., breast digital tomosynthesis (DBT).

[0003] X-ray mammography (MG) is a radiographic modality used to scan the breast for screening, diagnostic, and / or interventional procedures. The effectiveness of X-ray mammography is influenced by many factors, one of which is the two-dimensional (2D) rendering of the images acquired.

[0004] In mammography, alternatives to 2D X-ray mammography are known, such as breast digital tomosynthesis (DBT) systems, which are dedicated mammography systems that acquire multiple (e.g., dozens) angularly offset projection X-ray images and use the resulting X-ray image data to reconstruct a three-dimensional (3D) image data set.

[0005] The 2D and / or 3D images obtained during the mammography procedure can detect one or more cancers in the breast. Producing high quality mammograms is necessary for accurate interpretation of mammography images (also called mammograms) and accurate detection of breast cancer. A key factor affecting the quality of a mammogram is breast positioning. Failure to properly position the breast can result in artifacts and tissue being removed from the mammogram, which can result in missed cancers. The training and experience level of the technician can greatly affect image quality. For example, technicians with little or only basic training or with little or only basic experience may not be able to properly position the breast, which can result in high follow-up rates and missed cancers.

[0006] Furthermore, the conventional method for evaluating the positioning of the breast in a mammography image includes a technician reviewing the already acquired X-ray image. As a result, the patient has already been exposed to a certain amount of radiation, but the breast was not properly positioned for the image, so that a further dose of radiation is required to obtain a proper mammography image after the breast is properly positioned. This overlap of radiation doses to the patient is highly undesirable. Furthermore, even while reviewing the X-ray image, some technicians may not evaluate the X-ray image correctly, which increases the recall rate and reduces the reliability of the diagnosis based on the mammography image.

[0007] To aid in the detection of breast positioning errors prior to obtaining an X-ray image of an inaccurately positioned breast, mammography imaging systems including one or more cameras have been developed, as disclosed, for example, in U.S. Patent Application Publication No. US2021 / 0307711 (titled Methods And Systems For User And / Or Patient Experience Improvement In Mammography), the entirety of which is expressly incorporated herein by reference for all purposes. The camera acquires real-time images of the position of the breast and the imaging system, which are compared with a user input of a desired position of the breast and an associated mammography image to be acquired. If the camera image shows a position of the breast that does not correspond to the desired position of the user input, the imaging system alerts the operator that the breast is improperly positioned, so that the operator can make the necessary corrections to prevent unnecessary radiation doses from being applied to the patient.

[0008] However, one drawback of this mammography imaging system is that the imaging system requires training to properly compare the camera images with the user inputs so that the actual position of the breast can be accurately evaluated relative to the desired and / or selected position based on the user inputs. In the prior art imaging systems, training the artificial intelligence included in the imaging system and used to make the comparison requires a training data set that includes a large number of data including mammography images and associated camera images so that the artificial intelligence can learn the appropriate relationship between the mammography images and the camera images and output the appropriate comparison results. To provide the artificial intelligence with the necessary training data set, there are significant concerns and regulations regarding the collection and use of patient data, and there are significant time and cost constraints on the process of obtaining a sufficient training data set and the requirements regarding the protection (e.g., anonymization) of the data to form the training data set used during the training of the artificial intelligence.

[0009] As a result, it is desirable to develop a system and method for performing artificial intelligence training for a mammography imaging system that can utilize image data continuously obtained from an on-site imaging system without removing the data from the clinical environment or changing the operation or workflow for utilizing the imaging system within the clinical environment. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] US Patent Application Publication No. 2021 / 0307711 Summary of the Invention

[0011] According to one aspect of an exemplary embodiment of the present disclosure, an X-ray mammography system operable in an imaging mode and an intervention / biopsy mode includes a gantry disposed on a support surface, the gantry including an X-ray source, an X-ray detector alignable with the X-ray source, and a compression plate movable relative to the detector, the compression plate securing a patient's breast between the detector and the compression plate; an image processing system operatively connected to the gantry for controlling operation of the X-ray source and the X-ray detector to generate X-ray image data in an imaging mode of the mammography system, the image processing system including a processor for processing the X-ray image data from the detector, a database operatively connected to the processor and storing instructions for operating a feature detection AI model and a training module, a display operatively connected to the image processing system for presenting information to a user, and a user interface operatively connected to the image processing system for allowing user input to the image processing system; and a sensor system disposed in a user interface and operatively connected to the image processing system, the sensor system including at least one sensor operable to generate sensor data of at least one of a user, the gantry, and the patient, the feature detection AI model including a sensor detection system operable to detect features of one or both of the gantry and the patient from the sensor data to evaluate positions of the gantry and the patient, the training module being configured to receive at least one of user input parameters from the user interface for an imaging procedure to be performed by the X-ray mammography system, input parameters from sensors of the mammography system, and outputs from algorithms used during the imaging procedure, as well as the sensor data from the sensor system to form a training dataset for use in training the feature detection AI model, the training dataset not being transmitted externally from the X-ray mammography system.

[0012] According to yet another aspect of an exemplary embodiment of the present disclosure, a method for training a camera-based feature detection artificial intelligence model for an X-ray mammography system includes the steps of providing an X-ray mammography system including a gantry disposed on a support surface, the gantry including an X-ray source, an X-ray detector alignable with the X-ray source, and a compression plate movable relative to the detector, the compression plate securing a patient's breast between the detector and the compression plate, an image processing system operatively connected to the gantry and configured to control operation of the X-ray source and the X-ray detector to generate X-ray image data in an imaging mode of the mammography system, the image processing system including a processor for processing the X-ray image data from the detector, a database operatively connected to the processor and storing instructions for operating a feature detection AI model and a training module, a display operatively connected to the image processing system for presenting information to a user, and a display operatively connected to the image processing system for allowing user input to the image processing system. the patient, and a sensor system disposed on the gantry and operatively connected to the image processing system, the sensor system including at least one sensor operable to generate sensor data of at least one of a user, the gantry, and the patient, and the feature detection AI model operable to detect features of one or both of the gantry and the patient in the sensor data to evaluate positions of the gantry, the patient, and the patient's breast. The method includes the steps of: preparing an X-ray mammography system including: an image processing system including a user interface for enabling a user to view a user's position on the gantry; and a sensor detection system disposed on the gantry and operatively connected to the image processing system, the sensor system including at least one sensor operable to generate sensor data of at least one of a user, the gantry, and the patient, and the feature detection AI model operable to detect features of one or both of the gantry and the patient in the sensor data to evaluate positions of the gantry, the patient, and the patient's breast; [Brief description of the drawings]

[0013] These and other exemplary aspects, features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the drawings, which illustrate the best embodiment presently contemplated for carrying out the invention. [Figure 1] 1 is a schematic diagram of a mammography system including a visual detection system according to one embodiment of the present disclosure. [Diagram 2] FIG. 1 is a block diagram of a mammography system including a medical vision system according to one embodiment of the present disclosure, showing multiple objects sensed by the medical vision system and multiple actuators controlled based on the sensed objects. [Diagram 3] FIG. 1 is a schematic diagram of a method for on-site training an artificial intelligence model for assessing one or more of breast position, patient position, patient and / or breast morphology, user position, and user morphology, according to one embodiment of the present disclosure. [Figure 4] 13 is a flowchart of a method for on-site training an artificial intelligence model for assessing one or more of breast position, patient position, patient and / or breast morphology, user position, and user morphology according to another embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] One or more specific embodiments are described below. In an effort to describe these embodiments concisely, all features of an actual implementation may not be described herein. It should be understood that the development of such an actual implementation, as with any engineering or design project, will require numerous implementation-specific decisions to be made in order to achieve the developer's particular goals (such as adhering to system-related and business-related constraints), which may vary from implementation to implementation. Moreover, it should be understood that such a development effort, while complex and time-consuming, would be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill in the art having the benefit of this disclosure.

[0015] When introducing elements of various embodiments of the invention, the articles "a," "an," "the," and "said" are intended to mean the presence of one or more of the element. The terms "including," "comprising," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. Moreover, the numerical examples in the following description are intended to be non-limiting, and therefore, additional numerical values, ranges, and percentages are within the scope of the disclosed embodiments.

[0016] The following description relates to various embodiments of an x-ray system for mammography and biopsy procedures. An exemplary embodiment of an x-ray system is shown in FIG. 1 and includes a sensor system. The sensor system includes one or more sensors (e.g., a time-of-flight (TOF) sensor, a lidar sensor, or a visual sensor (such as a camera)) to detect the position of one or more accessories associated with each procedure, a site involved in the procedure, and one or more objects in the environment surrounding the system, the patient, and the user. Based on the position detection by the sensor system, the x-ray system including the sensor system can evaluate one or more of breast morphology, breast position, patient morphology, patient position, user morphology, and user position. A block diagram of one or more objects and motions detected by the sensor system including a sensor such as a camera and one or more actuators of the x-ray system that are adjusted based on the detected one or more objects and motions is shown in FIG. 2. A controller of an X-ray mammography system can be configured to operate a training module that uses sensor data and / or position data from a sensor system in a clinical environment for the imaging system to train an artificial intelligence (AI) / neural network model that evaluates one or more of the mammography system configurations, including breast position and morphology, patient position and morphology, user position and morphology, and accessories (such as accessories for a biopsy procedure), as illustrated in the exemplary system and method of the high level flowchart in Figure 3. Further, another exemplary system and method for training the AI / neural network model used by the controller to perform breast position and / or patient position evaluation and for providing real-time feedback for breast position adjustment during use of the X-ray imaging system is illustrated and described with respect to Figure 4.

[0017] During radiographic procedures (such as mammography or DBT imaging) and during radiographic image-guided interventional procedures (such as DBT-guided biopsy, CESM biopsy, stereotactic biopsy), positioning the breast plays an important role in obtaining high-quality images showing various parts of the breast. Furthermore, some positioning landmarks may differ depending on the view. Typically, a determination of whether the breast is positioned to provide a high-quality mammogram cannot be made until the mammogram is obtained. The inventors herein identify the above-mentioned problems and provide a method and system for improving breast positioning before data acquisition begins. In particular, a method and system are provided for assessing breast position and patient positioning prior to imaging and providing real-time feedback for improving breast and patient positioning. In one embodiment, a first detection sensor is used to assess patient position and capture position data (e.g., images of the patient and the x-ray system), and a second detection sensor is used to assess breast position and capture position data and / or images of the compressed breast. The sensor position data (e.g., camera images) obtained from the first and second detection sensors are then input into an artificial intelligence (AI) / neural network based position data / image processing model that evaluates the input position data and / or images according to a desired breast positioning framework (i.e., including breast anatomical landmarks based on the view) and further evaluates the patient position and provides real-time feedback to the technician / user via a user interface to enable one or more of patient position corrections and breast position corrections. Evaluating one or more of the patient position and breast position to improve breast position for imaging with the mammography system is described in further detail below.

[0018] The AI-based image processing model or AI is trained or constructed to perform the above-mentioned evaluations utilizing data obtained during normal operation of the radiography system in a clinical setting or environment in addition to the evaluations provided by the AI / neural network-based image processing model. More specifically, the radiography system receives sensor position data and / or images for a particular imaging procedure performed by the radiography system, as well as user inputs (such as user console inputs regarding laterality (right or left), field of view, and / or patient position) and system outputs (such as breast position monitoring results) used by the radiography system to obtain mammography images. These inputs and / or outputs for each imaging procedure can be provided as training data in various combinations with the sensor position data and / or images to a training system or training manager operatively connected to the radiography system to train the AI / neural network using actual inputs to and outputs from the radiography system for the imaging procedure. Using data including inputs and / or outputs of imaging procedures performed by the X-ray imaging system, the training manager can operate to continuously train the AI ​​during normal operation of the X-ray imaging system without removing data obtained during the imaging procedures from the imaging system and without altering the normal workflow of the operation of the X-ray imaging system.

[0019] Referring to FIG. 1, a mammography system 100 is shown including an X-ray system 10 for performing an X-ray imaging procedure of a breast according to an exemplary embodiment. The X-ray system 10 may be a tomosynthesis system, such as a digital breast tomosynthesis ("DBT") system, as shown and disclosed, for example, in U.S. Patent Application Publication No. 2021 / 0703311 (Methods And Systems For User And / Or Patient Experience Improvement In Mammography), the entirety of which is expressly incorporated herein by reference for all purposes. The X-ray system 10 may be used to perform one or more procedures, including digital tomosynthesis imaging and DBT-guided breast biopsy. Additionally, the X-ray system 10 may be used to perform a mammography imaging procedure, obtaining one or more views, including craniocaudal (CC view) and medial-lateral oblique (MLO) views of the breast. Additionally, the x-ray system can be used to perform other x-ray screening and diagnostic imaging procedures, such as CESM, and contrast-enhanced DBT (CE-DBT) diagnostic imaging, as well as interventional procedures, such as CESM-guided biopsies and stereotactic procedures.

[0020] X-ray system 10 includes a support structure 42 on which a radiation source 16, a radiation detector 18, and a collimator 20 are mounted. The radiation source 16 is housed in a gantry 15 that is movably coupled to the support structure 42. In particular, the gantry 15 may be mounted to the support structure 42 such that the gantry 15, including the radiation source 16, can rotate about an axis 58 relative to the radiation detector 18. The rotational angular range of the gantry 15 housing the radiation source 16 refers to the range of rotation to a desired angle in either direction about a vertical axis perpendicular to the horizontal detection plane of the detector 18.

[0021] The radiation source 16 is configured to be directed toward a volume or object to be imaged and emit radiation at desired times to acquire one or more images. The radiation detector 18 is configured to receive radiation through a surface 24. The detector 18 may be any one of a variety of different detectors, such as an x-ray detector, a digital x-ray detector, or a flat panel detector. The collimator 20 is disposed adjacent to the radiation source 16 and configured to adjust the illuminated area of ​​the object to be imaged.

[0022] In some exemplary embodiments, the system 10 may further include a patient shield 36 attached to the radiation source 16 via a face shield rail 38 to protect the patient's body part (e.g., head) from direct radiation exposure. The system 10 may further include a compression plate 40, which may be movable up and down along a vertical axis 60 relative to a support structure 42. Thus, the compression plate 40 may be adjusted to be positioned closer to the radiation detector 18 by moving it downward toward the detector 18. The distance between the detector 18 and the compression plate 40 may be increased by moving the compression plate upward along the vertical axis 60 away from the detector. The movement of the compression plate 40 may be adjusted by a user via a compression plate actuator (not shown) included in the x-ray system 10. The compression plate 40 may hold a body part (such as a breast) in place against a surface 24 of the radiation detector 18. The compression plate 40 may compress the body part to hold the body part in place and may optionally provide an opening to allow a biopsy needle (such as a core needle or suction core needle) to be inserted. In this manner, the compression plate 40 can be used to compress the body part to minimize the thickness traversed by the x-rays and to help reduce movement of the body part due to patient movement. The x-ray system 10 can also include an object support (not shown) upon which the body part is positioned.

[0023] The mammography system 100 may further include a workstation 43 including a controller 44 including at least one processor and memory. The controller 44 may be communicatively coupled to one or more components of the X-ray system 10, including one or more of the radiation source 16, the radiation detector 18, the compression paddle 40, and the biopsy device. In an exemplary embodiment, communication between the controller and the X-ray system 10 may be by a wireless communication system. In other exemplary embodiments, the controller 44 may be in electrical communication with one or more components of the X-ray system through a cable 47. Furthermore, in an exemplary embodiment, the controller 44 is integrated into the workstation 43, as shown in FIG. 1. In other exemplary embodiments, the controller 44 may be integrated into one or more of the various components of the system 10 disclosed above. Furthermore, the controller 44 may include processing circuitry for executing stored program logic, and the controller 44 may be any one of different computers, processors, controllers, or combinations thereof available and compatible with the various types of apparatus and devices used in the X-ray system 10.

[0024] The workstation 43 may include a radiation shield 48 to protect an operator of the system 10 from radiation emitted by the radiation source 16. The workstation 43 may further include a display 56, a keyboard 52, a mouse 54, and / or other suitable user input devices to facilitate control of the system 10 through a user interface 50.

[0025] Additionally, the X-ray system 10 may include a second control station (not shown) to facilitate control of the system 10 and to view one or more images captured by one or more of the vision system and the X-ray system 10. The second control station may include a second user interface with a second display having appropriate input capabilities. The second control station may be located near the X-ray system and may be coupled (wired or wirelessly) to the X-ray system 10. In particular, the second control station may be positioned such that a user can view the second display and / or the second user interface while adjusting the position of the breast and / or patient. Thus, by positioning the second control station, a user may simultaneously view real-time sensor / camera feedback and adjust the position of the patient and / or breast.

[0026] The mammography system processor and controller allows the controller 44 to coordinate the operation and function of the X-ray system 10. As an example, the controller 44 can provide timing control of when the X-ray source 16 emits X-rays, and can also coordinate how the detector 18 reads and transmits information or signals after X-rays strike the detector 18, how the X-ray source 16 and the detector 18 move relative to each other, and how the X-ray source 16 and the detector 18 move relative to the body part. The controller 44 can also control how information, including images 42 and data acquired during operation, is processed, displayed, stored, and manipulated. The different processing steps performed by the controller 44, including receiving one or more signals from one or more sensors, receiving user input, evaluating the received signals / inputs, image processing, determining reconstruction errors, outputting operating parameters including error indications, and adjusting one or more actuators of the X-ray system to control operation of the X-ray system, can be provided by a set of instructions stored in the processor's non-transitory memory. The information may be stored in one or more non-transitory memories of the controller 44 for later retrieval and use.

[0027] Further, as described above, the radiation detector 18 can receive radiation emitted from the radiation source 16. In particular, during imaging with the x-ray system, a projection image of the imaging site can be obtained at the detector 18. In some exemplary embodiments, data received by the radiation detector 18 (e.g., projection image data) can be transmitted electrically and / or wirelessly from the radiation detector 18 to the controller 44. The controller 44 can then reconstruct one or more scan images based on the projection image data, for example, by implementing a reconstruction algorithm. The reconstructed images can be displayed to a user via a display screen 56 in the user interface 50.

[0028] The radiation source 16 together with the radiation detector 18 form part of the X-ray system 10 that provides X-ray images for one or more of the following purposes: screening for abnormalities, diagnosis, dynamic imaging, and image-guided biopsy. For example, the X-ray system 10 can be operated in a mammography mode to screen for abnormalities. During mammography, the patient's breast is positioned and compressed between the detector 18 and the compression plate 40. Thus, the volume of the X-ray system 10 between the compression plate 40 and the detector 18 is the imaging volume. The radiation source 16 then emits radiation rays into the compressed breast, and a projection image of the breast is formed on the detector 18. The projection image can then be reconstructed by the controller 44 and displayed on the interface 56 by the display 50.

[0029] During a mammogram, the gantry 15 can be adjusted to different angles to obtain images in different orientations, such as craniocaudal (CC) images and mediolateral oblique (MLO) views. Additionally, while obtaining the mammography views (e.g., CC and MLO views), the gantry 15, compression paddle 40, and detector 18 can rotate as a single unit along line 62 about axis 58. In other embodiments, the gantry 15 can rotate about axis 58 while the compression paddle 40 and detector 18 remain stationary.

[0030] Additionally, the x-ray system 10 can operate in a tomosynthesis mode to perform digital breast tomosynthesis (DBT). During tomosynthesis, the x-ray system 10 can operate to irradiate an imaging volume (between the compression paddle 40 and the detector 18) with low-dose radiation at various angles across the angular range of the x-ray system 10. During tomosynthesis, similar to mammography, the breast is compressed between the compression paddle 40 and the detector 18. The radiation source 16 then rotates from -θ to +θ, and multiple projection images of the compressed breast are obtained at regular angular intervals across the angular range. For example, if the x-ray system has an angular range of ±11 degrees, 22 projection images can be captured by the detector, approximately one every 1 degree, during the gantry angle movement. The multiple projection images are then processed by the controller 44 to generate multiple DBT image slices. This processing can include applying one or more reconstruction algorithms to reconstruct a three-dimensional DBT image of the breast.

[0031] Additionally, the x-ray system 10 may be configured to perform DBT-guided biopsy procedures. Accordingly, in some exemplary embodiments, the system 10 may further include a biopsy device (not shown) that includes a biopsy needle to extract a tissue sample for further analysis in an appropriate manner.

[0032] The mammography system 100 may further include one or more position / detection sensors for sensing and / or acquiring position data of one or more components and accessories of the mammography system 100. The one or more detection sensors may include a first sensor 101 and a workstation sensor 102, as shown in FIG. 1. The first sensor 101 and the second sensor 102 may each be configured as a visual sensor (e.g., a camera including a camera for obtaining RGB images), a depth sensor (e.g., a TOF sensor), a combination of visual and depth sensors, a force / pressure sensor and / or a velocity sensor, e.g., a sensor is used and the system 100 learns / trains on an optimal compression profile obtained from a motion analysis of the position data / position images, and the sensor may be configured to sense one or more components and accessories associated with the X-ray system 10. Furthermore, the first sensor 101 may be configured to sense one or more of the morphology and motion of the user and the patient, while the workstation sensor 102 may be utilized to monitor the position and / or motion of the user at the workstation. In this embodiment, the first sensor 101 and the second sensor 102 are shown configured as two cameras for visual sensing by the mammography system 100, however, it is understood that the sensor system may include additional or fewer sensors (e.g., cameras) as described further below. Additionally, the one or more sensors may include a second sensor 154 coupled to the X-ray system 10. The second sensor 154 may be configured similarly to the first sensor 101, such as a camera, and may be utilized to capture position data and / or camera images of the compressed breast within the imaging volume. In this manner, the position data and / or camera images obtained by the second sensor 154 may be utilized to assess the position of the breast prior to acquiring the X-ray projection images.

[0033] The second sensor 154 can be coupled to the gantry 15 such that the field of view of the second sensor 154 is positioned relative to the field of view of the x-ray system 10 (particularly relative to the radiation source 16 located within the gantry 15). The second sensor 154 can be used to monitor the patient's breast position and breast anatomical landmarks during a procedure (such as a mammogram, DBT, or image-guided biopsy) performed by the x-ray system 10. In one embodiment, the second sensor 154 can be configured as an RGB-D camera that combines depth information with RGB color information, similar to the other sensors 101, 102, etc. Additionally, the second sensor 154 can be communicatively coupled to the controller 44 of the x-ray system 10, and one or more position data / sensor images and / or sensor image sequences captured by the second sensor 154 can be stored in a non-transitory memory of the controller 44. Additionally, the second sensor 154 can be configured to capture movement and / or motion. For example, a sequence of sensor images over a period of time can be acquired and used to recognize motion.

[0034] It will be appreciated that in some embodiments, the second sensor 154 can be positioned such that the position of the imaging volume including the compression paddle 40, the detector 18, and the compressed breast is located and / or visualized by the second sensor 154. In some embodiments, the second sensor 154 can be positioned to monitor and evaluate a portion of the patient in addition to the position of the compressed breast. That is, the second sensor 154 can be used to capture a portion of the patient (e.g., the patient's shoulder) and thus obtain a view of the portion of the patient in addition to a view of the compressed breast.

[0035] In summary, the second sensor 154 can be adjusted (e.g., by adjusting the position of the second sensor 154) to obtain and / or capture views of one or more of the imaging volume (including the compression plate, detector, and compressed breast), the imaging volume and a margin volume, and the imaging volume and a portion of the patient.

[0036] Further, in one embodiment, the position data and / or sensor images captured by the second sensor 154 may be pre-processed to extract relevant image data, and the processed image data may be used as input for an artificial intelligence-based deep learning model (stored in the controller, an edge device connected to the controller, a cloud in communication with the controller, or a suitable combination thereof). The deep learning model includes a neural network (such as a convolutional neural network) for evaluating breast position based on breast anatomical landmarks detected during breast positioning in various imaging and image-guided procedures using the x-ray system 10. Some landmarks of the breast anatomical landmarks (also referred to as breast structures) may differ depending on the view (e.g., CC view or MLO view) and procedure (e.g., mammography or DBT), while there may be multiple common breast anatomical landmarks (i.e., breast structures) that are evaluated for breast position in each view.

[0037] Returning to Figure 1, the one or more visual sensors may be communicatively coupled to controller 44. Details of the various components of mammography system 100 that are sensed and monitored by first sensor 101, second sensor 154 and / or workstation sensor 102, as well as the various actuators of mammography system 100 that may be adjusted in response to the sensor sensing and monitoring, are described in further detail below in Figure 2.

[0038] Referring to Figure 2, a block diagram of a mammography system 200 is shown. The mammography system 200 may be a non-limiting example of the mammography system 100 in Figure 1. The mammography system 200 may be utilized to perform one or more of mammography procedures (such as routine mammograms, digital breast tomosynthesis, etc.) and biopsy procedures (such as stereotactic or DBT-guided biopsies, etc.). The mammography system 200 may include a medical vision system 250 having at least a sensor system 252 and a processing system 256 to perform one or more of sensing, monitoring, and analyzing one or more accessories and components associated with the mammography system 200. Additionally, the mammography system 200 may be automatically adjusted, controlled, and configured by the processing system 256 based on input from the sensor system 252 and analysis of the input by the processing system 256 to improve workflow, breast positioning, and patient positioning during one or more of a mammography procedure and a biopsy procedure performed using the mammography system 200. The processing system 256 may be a non-limiting example of the controller 44 in FIG. 1, and as discussed further below, the processing system 256 may be configured to receive signals from one or more sensor systems (e.g., sensor system 252) of the mammography system 200. The processor may be configured to analyze data received from the sensor systems, adjust operation of the mammography system 200 via one or more x-ray system actuators 276, and provide real-time feedback, including one or more alerts and instructions, to a user and / or a patient through a user interface 286 of the mammography system 200, as further discussed below.

[0039] The mammography system 200 can include an X-ray system 210 and a medical vision system 250. The X-ray system 210 can be an implementation of the X-ray system 10 described in FIG. 1. In one exemplary embodiment, the X-ray system 210 can be configured as a medical imaging modality to perform a mammography procedure to image and analyze a patient's body part (such as a breast). In another exemplary embodiment, the X-ray system 210 can be configured to perform a biopsy procedure (such as an X-ray guided biopsy) to obtain a tissue sample from the patient's body part. Furthermore, the X-ray system 210 can be transformed from a mammography system for obtaining medical scan images to a biopsy system for performing a biopsy procedure to extract tissue so that it can be evaluated. When the X-ray system 210 is utilized to perform a biopsy procedure, a biopsy device 212 can be coupled to the X-ray system 210. The biopsy device 212 may be an embodiment of the biopsy device described in Fig. 1, and may include a compression plate 222 for supporting and holding the body part such that movement during biopsy is reduced. The compression plate 222 is an embodiment of the compression plate 40 described with respect to Fig. 1, and the mammography system may be configured to monitor one or more of the components (such as the biopsy device 212, the different components of the biopsy device 212) and accessories (such as the compression plate 222) through the medical vision system 250. As an example, during a mammography or biopsy examination, a user may move the compression plate to adjust the position of the breast. The vision system 250 may detect the movement of the compression plate 222, and based on the changed position of the compression plate after compression, the controller may command an adjustment of the collimator so that the breast is within the field of view of the X-ray system.

[0040] Additionally, in some embodiments, the vision system 250 may be utilized to determine a final (locked) position of the compression paddle 222. The final position of the compression paddle may indicate that the breast is in position for imaging. In one example, once the final position of the compression paddle is confirmed, the breast position of the compressed breast may be evaluated, which may result in a breast position evaluation interface being activated to allow the vision system 250 to evaluate the breast position. Once the vision system 250 has confirmed that the compressed breast is in the desired position for imaging, the mammography system may be used to begin acquiring an x-ray image of the compressed breast. In another example, once the final position of the compression paddle is confirmed, the vision system 250 may be used to evaluate one or more of the breast position and the patient position. Once the breast position, the patient position, and / or the user position are confirmed to be in the desired position for the respective imaging, the mammography system may be used to begin acquiring an x-ray image.

[0041] The mammography system 200 may be further configured to monitor the environment 226 surrounding the x-ray system 210 with a medical vision system 250 .

[0042] As described above, the medical vision system 250 includes a sensor detection system 252 including one or more position detection sensors (e.g., a camera 254) and an image processing system 256 including a processor 258 and a non-transitory memory 260. The sensor system 252 can be communicatively coupled to the image processing system 256. In particular, the processing system 256 can receive one or more signals from the one or more cameras 254 of the system 252. The one or more sensors or cameras 254 of the system 252 can be similar to the sensors or cameras 101, 102 and 154 described with respect to FIG. 1, and the one or more sensors / cameras 254 can sense the mammography system 200 and its components, accessories, and environment. Data from the one or more sensors / cameras 254 can be transmitted to the processing system 256 for further analysis and storage.

[0043] The processing system 256 includes a processor 258 configured to execute machine-readable instructions stored in a non-transitory memory 260. The processor 258 may be a single-core processor or a multi-core processor, and programs executed by the processor may be configured for parallel or distributed processing. In some embodiments, the processor 258 may include individual components, optionally distributed across two or more devices, and may be remotely located and / or configured for cooperative processing. In some embodiments, one or more aspects of the processor 258 may be virtualized and realized by remotely accessible networked computing devices built in a cloud computing configuration. According to other embodiments, the processor 258 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field programmable gate array (FPGA), or a graphics board. According to other embodiments, the processor 258 may include multiple electronic components capable of performing processing functions. For example, the processor 258 may include two or more electronic components selected from a list of electronic components including a central processing unit, a digital signal processor, a field programmable gate array, and a graphics board. In yet further embodiments, the processor 258 may be configured as a graphics processing unit (GPU) that includes a parallel computing architecture and parallel processing capabilities. The non-transitory memory 260 may store an artificial intelligence (AI) based image processing / neural network model 262, positional and / or image data 264, an accessory monitoring module 266, a user input monitoring module 268, an environmental monitoring module 270, a workflow monitoring module 272, and a patient monitoring module 274.

[0044] In one embodiment, the AI / neural network model 262 may include a deep learning model including a number of parameters (e.g., weights, biases, activation functions) and instructions for implementing one or more deep neural networks to receive positional data and / or image data from the sensor system 252, identify one or more objects corresponding to one or more of the x-ray system components and accessories, further identify one or more environmental parameters, and further identify one or more processes and operations related to one or more of the mammography and biopsy. For example, the AI / neural network model 262 may store instructions for implementing a deep learning module including one or more neural networks (e.g., convolutional neural networks (CNNs)). The AI / neural network model 262 may include trained and / or untrained neural networks, and the AI / neural network model 262 may further include various data or metadata associated with the stored one or more neural networks.

[0045] Additionally, using input from sensor system 252, AI / neural network model 262 can determine breast location, breast anatomical landmarks, patient location, patient morphology, user location, user morphology, and system and / or accessory configuration. Additionally, using input from sensor system 252, AI / neural network model 262 can evaluate breast location as well as patient location and determine one or more errors based on the evaluation, including breast positioning errors, patient positioning errors, and user errors, and control the mammography system based on the errors (e.g., stop image acquisition in response to detecting the one or more errors) and provide real-time feedback based on the detected errors (e.g., once the breast is positioned, a visual sensor can acquire one or more images and utilize the images to analyze breast location and morphology and provide real-time feedback based on the analysis).

[0046] In one embodiment, the first AI / neural network model or part of the AI / neural network model 262 may include parameters for a desired patient position (including a patient position with respect to the whole body and a patient position with respect to the X-ray system), the desired patient position being based on an imaging mode of the X-ray system and a desired view to be acquired by the imaging system. Furthermore, the second AI / neural network model or part of the AI / neural network model 262 may include parameters for a desired breast position (including one or more features of a compressed breast included in an imaging volume of the X-ray system), the desired breast position being based on an imaging mode of the X-ray system and a desired view to be acquired by the imaging system. Before starting the acquisition of the X-ray image, the sensor system 252 may be used to evaluate the current patient position (with respect to the first module) and the current breast position (with respect to the second module), and based on the evaluation, real-time feedback may be provided to the user through the user interface. If the current patient position matches the first module and the current breast position matches the second module, the acquisition of the X-ray image may be started.

[0047] The non-transitory memory 260 may further store sensor / camera image data 264. The sensor / camera image data 264 may include position data / images captured by the sensor system 252. For example, the position data / images captured by the sensor system 252 may include position data / images of one or more mammography systems (including the X-ray system 210, including its components and accessories, the environment 226, and processes and / or operations associated with the X-ray system 210 and the environment 226). The sensor / camera image data 264 may further include patient monitoring images, user monitoring images, images of the compressed breast, images of the system and / or accessories.

[0048] The non-transitory memory 260 may further store an accessory monitoring module 266 that includes instructions for monitoring and analyzing whether one or more accessories 214 and biopsy device 212 are present, and for monitoring and analyzing the current location of one or more accessories 214 and biopsy device 212.

[0049] The non-transitory memory 260 may further store a user input monitoring module 268 that includes instructions for monitoring and analyzing user input from the user interface.

[0050] The non-transitory memory 260 may further store an environment monitoring module 270 that includes instructions for monitoring and analyzing the environment 226, and may store a workflow monitoring module 272 that includes instructions for monitoring and analyzing one or more processes and operations 238. Additionally, the non-transitory memory 260 may store a patient monitoring module 274 for monitoring and analyzing one or more of a patient presence or absence, a patient position, and a patient's movement into and out of the examination room.

[0051] The non-transitory memory 260 may further store medical image data 275. The medical image data 275 may include scanned images of a body part captured by the x-ray system 210.

[0052] Upon sensing and analyzing one or more of the X-ray system 210, the environment 226, and the processes and operations 238, the image processing system 256 can output commands to one or more X-ray system actuators 276 based on its sensing and analyzing steps. The X-ray system actuators 276 can include an image acquisition actuator 278 for controlling radiation source output from a radiation source (such as radiation source 16 in FIG. 1), a gantry movement actuator 280 for controlling a gantry position of the X-ray system 210, and a bed position actuator based, for example, on the presence or absence of an object in the environment 226. The gantry movement actuators 280 can include one or more actuators for adjusting one or more of gantry lift, gantry rotation, and gantry angle setting, where gantry lift movement includes moving the gantry up or down along a vertical axis of the X-ray system 210, gantry rotation is the rotation of the detector and the X-ray generating tube about the axis of rotation, and gantry angle setting is the rotation of the X-ray tube while the detector is stationary within a range of rotation angles.

[0053] The X-ray system actuators 276 may further include a biopsy device actuator for adjusting the operation of the biopsy device (such as firing a biopsy needle) based on, for example, sensing one or more discrepancies between user input and the actual X-ray system configuration, as described in further detail below. The X-ray system actuators 276 may further include a compression plate actuator 285 for adjusting the movement of the compression plate 222, among other things.

[0054] Furthermore, upon sensing and analyzing one or more of the X-ray system 210, breast position, patient position, environment 226, and process and operation 238, the image processing system 256 may output one or more alerts including real-time feedback through the user interface 286. The user interface 286 may be an example of the user interface 56 in FIG. 1. Furthermore, the processing system 256 may be configured to update the image acquisition input 292 on the user interface 286 and adjust one or more of the setup, configuration, and operation of the X-ray system accordingly. Furthermore, the processing system may be configured to update the accessory information input 294 on the user interface 286 and adjust the setup, configuration, and operation of the X-ray system accordingly.

[0055] 2 and 3, the non-transitory memory 260 may further store an AI / neural network training module 263 and / or instructions for the implementation and operation of the training module 263, including instructions for training one or more of the AI ​​and / or deep neural networks stored in the AI / neural network model 262. Alternatively, the training module 263 and the stored instructions for operating the training module 263 may be stored in a non-transitory memory (not shown) located remotely from the imaging system 200 but operably connected to the imaging system 200.

[0056] For one or more imaging procedures performed using the imaging system 200, training data may be provided by the image processing system 265 / processor 258 to the training module 263 to train the AI / neural network model 262. In particular, the training data may include various inputs to and / or outputs from the imaging system 200 for a particular imaging procedure performed by the imaging system 200.

[0057] To provide the training module 263 with ground truth values ​​to be used in training the AI / neural network model 262, the image processing system 256 / processor 258 can provide the training module 263 with known parameters for the particular imaging procedure to be performed by the imaging system 200. For example, the training data set provided to the training module 263 can include user input regarding various parameters for the particular imaging procedure, such as laterality (right or left), field of view, patient position, and other imaging system setting selections for the imaging procedure.

[0058] The training data set provided to the training module 263 may further include internal system outputs from various system algorithms (not shown) used by the image processing system 256 to inform the image processing system 256 (such as outputs from the second AI / neural network model or part of the AI / neural network model 262 regarding the detected current breast position, e.g., internal system outputs from AI / image processing algorithms run on the x-ray image to determine whether the breast is correctly positioned, etc.). This internal system output is obtained from a process already performed by the system 100 and serves as one of the one or more ground solutions provided to the training module 263. These user inputs and / or internal system algorithm outputs are used by the training module 263, alone or in combination with each other, as ground solutions or evaluation parameters for the AI / neural network model 262.

[0059] Additionally, the training data sent to the training module 263 by the image processing system 256 / processor 258 includes sensor / camera image data 264, which may include position data and / or camera images of the compressed breast, camera images of a phantom, each corresponding to an imaging sequence associated with a correct answer, i.e., user input and / or system algorithm output.

[0060] It should be understood that the image processing system 256 shown in Figure 2 is for purposes of illustration and not limitation, and other suitable image processing systems may have more or fewer different components.

[0061] 3, a high level flow chart is shown illustrating a method 300 used by the training module 263 to obtain a training data set from an imaging procedure and use the training data set to train the AI / neural network model 262. In particular, the method 300 may be performed during operation of an imaging system (such as the x-ray system 100 or 200 of FIG. 1 or FIG. 2) prior to commencing image acquisition to train the AI / neural network model 262 for evaluation of the position of the body part being imaged relative to the x-ray imaging system and / or the patient position relative to the x-ray system.

[0062] To train the AI / neural network model 262 according to the method 300, the image processing / medical vision system 250 first senses or determines a triggering event in step 302. For example, the vision system 250 determines a final (locked) position of the compression paddle 222 after the patient's breast is placed between the compression paddle 222 and the detector, and moves the compression paddle 222, such as using the compression paddle actuator 285, to place the compression paddle 222 in a desired position for breast compression. Other types of triggering events that the image processing / medical vision system 250 can sense relate to various operating conditions and / or settings of the x-ray system 10,210 that indicate that the x-ray system 10,210 is in a state to evaluate and / or determine the position and / or orientation of one or more portions of the x-ray system 10,210 and / or the patient's breast prior to the start of an imaging procedure. In response to the triggering event 302, the medical vision system 250 operates the sensor system 252 including the sensor / camera 254 in step 304 to acquire position data and / or patient monitoring images and / or images of a compressed breast, etc., which are received by the sensor system 252. These position data / images or frames 306 are then transmitted by the sensor system 252 to the medical vision system 250 in step 308 and stored as sensor / camera image data 264. The frames 306 / sensor / camera image data 264 may be pre- or post-processed in step 310 and forwarded by the image processing system 256 / processor 258 to the training module 263.

[0063] In addition to the frame / sensor / image data 306, the medical vision system 250 receives user input 312 from the user interface 286 and / or algorithm output 314 from the radiography system 10, 210 corresponding to selected imaging parameters and system configuration of the imaging procedure to be performed. The medical vision system 250 provides these system inputs and outputs to the training module 263 in step 316, which are used as ground truth input data 318 for the training module 263. In step 322, a training data set 320 is created using the frame / sensor / image data 264 and ground truth input data 318 obtained from the imaging procedure being performed on the imaging system 10, 210, and the training data set 320 is sent by the training module 263 to the AI / neural network model 262. The AI / neural network model 262 performs evaluations against the training data set 320 to determine various attributes of the frame / sensor / image data 264 to both train the AI / neural network model 262 and to test / verify the AI / neural network model 262's determination of whether the frame / sensor / image data 264 corresponds to parameters identified by the ground truth input data 318 for the imaging procedure from which the frame / sensor / image data 264 was obtained. In this manner, the AI / neural network model 262 is trained using data obtained directly from the radiography system 10, 210 incorporating the AI / neural network model 262 and training module 263, thereby eliminating issues or concerns regarding the collection of the training data set as with previous training data set collection methods.

[0064] With regard to the manner of operation of training the AI / neural network model 262 using the method 300, several options are available. First, the method 300 can be performed in a continuous learning process using data acquired in several imaging procedures performed utilizing the radiography system 10, 210. In the continuous learning process, once the output from the AI / neural network model 262 of the imaging procedure based method 300 reaches a certain level of accuracy (e.g., as verified by a validation data set (not shown) applied to the AI / neural network model 262), the configuration of the AI / neural network model 262 can be switched from continuous learning to an automatic mode that disables the learning module 263, and the frame / sensor / image data 264 is fed directly to the AI / neural network model 262 for feature detection and / or evaluation of laterality (right or left), field of view, breast position monitoring, sample presence, etc.

[0065] In a second alternative operation of the method 300, the method 300 may be operated in an active learning mode. The active learning mode may be used after a continuous learning process is utilized on the AI / neural network model 263, where the AI / neural network model 262 outputs an inference confidence score for each training data set 320 for a particular imaging procedure provided to the AI / neural network model 262 by the training module 263. If the inference confidence score is above some preset threshold determined by the user or the training module 263, the AI / neural network model 262 may be maintained in an automatic feature detection operating setting, as described above. However, if the inference confidence score from the AI / neural network model 262 falls below the threshold, the AI / neural network model 262 is returned to a training mode, where the training module 263 provides the AI / neural network model 262 with additional training data sets 320 from other imaging procedures performed by the radiography system 10, 210 until the output of the AI / neural network model 262 is verified and / or exceeds the threshold. Furthermore, if the user input changes, i.e., the user changes the model's answer, the AI / neural network model 262 is returned to training mode and the training module 263 provides the AI / neural network model 262 with an additional training data set 320 that corresponds to the newly specified user input shooting sequence.

[0066] The above-mentioned operation mode of the learning method 300 relies on a training data set formed using user input / system output and image data obtained from an imaging procedure performed by a single radiography system 10, 210 on which the AI / neural network model 262 and the training module 263 are stored, the patient data being stored on-site. In a third option, the method 300 operates in a federated learning model or a personalized federated learning model. In this embodiment, as shown diagrammatically in FIG. 4, initially, in step 402 of the method 400, the AI / neural network model 262 can be formed outside the radiography system 10, 210. The AI / neural network model 262 can be trained outside the radiography system 10, 210, for example on a central server 410, which can be located remotely from the imaging systems 10, 210, which can also be located remotely from each other. In step 404, the untrained model 262 is transmitted or installed in each individual radiography system 10, 210 in combination with the training module 263 from the central server 410. Next, in step 406, the AI / neural network model 262 is trained in each individual radiography system 10, 210 by one of the techniques described above in method 300 until the model 262 is validated and / or reaches a predefined threshold for accuracy. Next, in step 408, the trained model 262 is retrieved from each individual radiography system 10, 210 by the central server 410 without transmitting or storing data from the training data set utilized to train the model 262. The central server 410 then pools or aggregates the results of each of the trained models 262 to achieve a global trained model 414. The global trained model 414 may be utilized by each of the individual radiography systems 10,210 to provide feature detection functionality for the individual radiography systems 10,210.

[0067] In each iteration of the embodiments of the training method 300, 400 for training the AI / neural network model 262 in the training module 263 located within the respective radiography system 10, 210, the data 312, 314, 316, 264 used for the training data set 320 is obtained directly and continuously from imaging procedures performed using the radiography system 10, 210. In this manner, the data remains secure and is not transmitted outside the clinical environment of the radiography system 10, 210, preserving the privacy and confidentiality of the patient information contained in the training data set 320. Furthermore, in each embodiment of the method 300, 400, the information forming the training data set 320 is obtained in a seamless manner during normal operation of the radiography system 10, 210, and the operation or workflow of the radiography system 10, 210 is not altered for the technician as the method 300, 400 is performed, e.g., the method 300, 400 operates in the background of normal operation of the radiography system 10, 210 as radiography procedures are performed. Furthermore, the method 300, 400 for training the AI / neural network model 262 is completely transparent to the user and does not require any modifications to the way the user operates the system 10, 210, nor does it require the user to actively annotate the x-ray images and / or other input data used to train the AI / neural network model 262.

[0068] It is understood that the above compositions, devices and methods of the present disclosure are not limited to specific embodiments and methodologies, as these may vary, and it is also understood that the terminology used herein describes specific exemplary embodiments only, and is not intended to limit the scope of the present disclosure, which is limited only by the claims.

[0069] [Embodiment 1] 1. A method for training a sensor-based feature detection artificial intelligence (AI) model for an X-ray mammography system, the method comprising: a. providing an x-ray mammography system, said x-ray mammography system comprising: i. a gantry disposed on a support surface, the gantry including an x-ray source, an x-ray detector alignable with said x-ray source, and a compression plate movable relative to said detector, the compression plate securing a patient's breast between said detector and said compression plate; ii. an image processing system operatively connected to the gantry and controlling operation of the x-ray source and the x-ray detector to generate x-ray image data in an imaging mode of the mammography system, the image processing system including a processor for processing the x-ray image data from the detector, a database operatively connected to the processor for storing instructions for operating a feature detection AI model and a training module, a display operatively connected to the image processing system for presenting information to a user, and a user interface operatively connected to the image processing system for allowing user input to the image processing system; and iii. a sensor detection system operatively connected to the image processing system, the sensor detection system including at least one sensor operable to generate sensor data of the gantry and the patient, the feature detection AI model operable to detect features of one or both of the gantry and the patient in the sensor data to assess at least one of a user, the gantry, and the patient; Providing an X-ray mammography system, comprising: b. inputting parameters of an imaging procedure to be performed by said X-ray mammography system; c. positioning the patient's breast between the detector and the compression paddle; d. acquiring sensor data with said sensor detection system; e. providing at least one of the input parameters, one or more internal system outputs from a control algorithm stored in said database, and combinations thereof, along with the sensor data to a training module to form a training data set; e. feeding the training data set to the feature detection AI model. The method includes: [Embodiment 2] 2. The method of claim 1, comprising the step of feeding the training data set to the feature detection AI model while performing the imaging procedure. [Embodiment 3] 2. The method of embodiment 1, wherein the at least one sensor is a camera operable to acquire camera images of the gantry and the position of the patient as the sensor data. [Embodiment 4] a. receiving one or more internal system outputs from a control algorithm stored in said database in response to said entered parameters for said imaging procedure; and b. providing the system output, along with the input parameters and the sensor data, to the training module to form the training data set. 2. The method of embodiment 1, comprising: [Embodiment 5] 5. The method of embodiment 4, wherein the step of providing the input parameters, the internal system output, and the sensor data to a training module includes a step of using the input parameters and the system output as a ground truth for the training data set. [Embodiment 6] 2. The method of claim 1, wherein the training data set is fed to the feature detection AI model and each imaging procedure is performed on the X-ray mammography system sequentially. [Embodiment 7] 2. The method of embodiment 1, wherein the training data set is fed to the feature detection AI model and each imaging procedure is performed on the X-ray mammography system automatically. [Embodiment 8] 2. The method of claim 1, wherein the method is a federated method in which the training data set is fed into the feature detection AI model and each imaging procedure is performed on the X-ray mammography system. [Embodiment 9] The method of claim 1 , wherein the training data set is maintained on the X-ray mammography system. [Embodiment 10] 2. The method of embodiment 1, wherein the training data set is not transmitted externally from the X-ray mammography system. [Embodiment 11] 2. The method of embodiment 1, wherein acquiring the sensor data includes acquiring a plurality of images of a gantry. [Embodiment 12] 2. The method of embodiment 1, wherein acquiring the sensor data includes acquiring an image of at least the patient, the patient's breast, or a combination thereof. [Embodiment 13] 2. The method of embodiment 1, wherein the step of acquiring sensor data includes acquiring a plurality of images of a position and configuration of the gantry and at least one of a position and shape of at least one of the patient, the patient's breast, and combinations thereof. [Embodiment 14] The image processing system is configured to sense one or more triggering events of the X-ray mammography system, and the step of acquiring the plurality of sensor data includes: a. sensing a triggering event; and b. operating the sensor detection system such that the sensor data is acquired. 2. The method of embodiment 1, comprising: [Embodiment 15] The step of positioning the patient's breast between the detector and the compression paddle includes: a. positioning the patient's breast between the compression paddle and the detector; b. moving the compression paddle to a desired position such that the patient's breast is compressed between the compression paddle and the detector; and c. locking the compression paddle in the desired position. Including, 15. The method of embodiment 14, wherein the sensed triggering event is a locking of the compression plate. [Embodiment 16] 2. The method of embodiment 1, wherein the step of providing the sensor data to a training module includes the step of providing pre-processed sensor data, post-processed sensor data, and a combination thereof to the training module. [Embodiment 17] 1. An X-ray mammography system operable in an imaging mode and in an intervention / biopsy mode, the system comprising: a. a gantry disposed on a support surface, the gantry including an x-ray source, an x-ray detector alignable with the x-ray source, and a compression plate movable relative to the detector, the compression plate securing a patient's breast between the detector and the compression plate; b. an image processing system operatively connected to the gantry and controlling operation of the x-ray source and the x-ray detector to generate x-ray image data in an imaging mode of the mammography system, the image processing system including a processor for processing the x-ray image data from the detector, a database operatively connected to the processor for storing instructions for operating a feature detection AI model and a training module, a display operatively connected to the image processing system for presenting information to a user, and a user interface operatively connected to the image processing system for allowing user input to the image processing system; c. a sensor detection system disposed on the gantry and operably connected to the image processing system, the sensor detection system including at least one sensor operable to generate sensor data of the gantry and the patient, the feature detection AI model operable to detect features of one or both of the gantry and the patient in the sensor data to assess the gantry and the patient. Including, The training module is configured to receive at least one of user input parameters from the user interface for an imaging procedure to be performed by the X-ray mammography system, one or more internal system outputs from a control algorithm stored in the database, and combinations thereof, and sensor data from the sensor detection system to form a training data set for use in training the feature detection AI model, wherein the training data set is not transmitted externally from the X-ray mammography system. [Embodiment 18] 18. An X-ray mammography system as described in embodiment 17, wherein the training module is configured to operate simultaneously with an imaging procedure performed using the X-ray mammography system. [Embodiment 19] 18. An X-ray mammography system as described in embodiment 17, wherein the training module is configured to operate in a continuous learning mode. [Embodiment 20] 18. An X-ray mammography system as described in embodiment 17, wherein the training module is configured to operate in a federated learning mode. [Explanation of symbols]

[0070] 10,210 X-ray mammography systems 15 Gantry 16 X-ray source 18 X-ray detector 40 Compression Plate 50 User Interface 56 Display 236 patients 254 Camera 258 processors 260 Database 262 Feature Detection Artificial Intelligence (AI) Model 263 Training Module 264 Image Data 306 Sensor Data 312 Input parameters 314 Internal System Output 320 training dataset 44,256 Image Processing Systems 100,252 Sensor Detection System 100,256 Image Processing System 101, 102, 154, Sensor or camera

Claims

1. 1. A method for training a sensor-based feature detection artificial intelligence (AI) model (262) for an X-ray mammography system (10, 210), the method comprising: a. providing an x-ray mammography system (10, 210), said x-ray mammography system (10, 210) comprising: i. a gantry (15) disposed on a support surface, the gantry (15) including an x-ray source (16), an x-ray detector (18) alignable with said x-ray source (16), and a compression paddle (40) movable relative to said detector (18), the compression paddle (40) for securing a patient's breast between said detector and said compression paddle; ii. an image processing system (44, 256) operably connected to the gantry (15) and controlling operation of the X-ray source (16) and the X-ray detector (18) to generate X-ray image data in an imaging mode of the mammography system (10), the image processing system (44, 256) including a processor (258) for processing the X-ray image data from the detector (18), a database (260) operably connected to the processor (258) and storing instructions for operating a feature detection AI model (262) and a training module (263), a display (56) operably connected to the image processing system (44, 256) for presenting information to a user, and a user interface (50) operably connected to the image processing system (100, 256) for allowing user input to the image processing system (44, 256); iii. a sensor detection system (100, 252) operatively connected to the image processing system (44, 256), the sensor detection system (100, 252) including at least one sensor (101, 102, 154, 254) operable to generate sensor data of the gantry (15) and the patient (236), the feature detection AI model (262) operable to detect features of one or both of the gantry (15) and the patient (236) in the sensor data to assess at least one of a user, the gantry, and the patient; Providing an X-ray mammography system (10, 210), comprising: b. inputting parameters of an imaging procedure to be performed by said X-ray mammography system (10, 210); c. positioning the patient's breast (236) between the detector (18) and the compression paddle (40); d. acquiring sensor data (306) with said sensor detection system (100, 252); e. providing at least one of the input parameters (312), one or more internal system outputs (314) from a control algorithm stored in said database (260), and combinations thereof, along with the sensor data (306) to a training module (264) to form a training data set (320); e. feeding said training data set (320) to said feature detection AI model (262). The method includes:

2. 2. The method of claim 1, comprising providing the training data set (320) to the feature detection AI model (262) while performing the imaging procedure.

3. 2. The method of claim 1, wherein the at least one sensor is a camera operable to acquire camera images of the position of the gantry and the patient as the sensor data.

4. a. receiving one or more internal system outputs (314) from a control algorithm stored in the database (260) in response to the input parameters (312) for the imaging procedure; and b. providing said system output (314) along with said input parameters (312) and said sensor data (306) to said training module (263) to form said training data set (320); The method of claim 1 , comprising:

5. 5. The method of claim 4, wherein providing the input parameters (312), the internal system outputs (314), and the sensor data (306) to a training module (263) comprises using the input parameters (312) and the system outputs (314) as a ground truth (318) for the training data set (320).

6. 2. The method of claim 1, wherein the training data set (320) is fed to the feature detection AI model (262) and each imaging procedure is performed on the X-ray mammography system (10, 210) in a sequential manner.

7. 2. The method of claim 1, wherein the training data set (320) is fed to the feature detection AI model (262) and each imaging procedure is performed on the X-ray mammography system (10, 210) automatically.

8. 2. The method of claim 1, wherein the method is a federated method, the training data set (320) being fed to the feature detection AI model (262) and each imaging procedure being performed on the X-ray mammography system (10, 210).

9. The method of claim 1 , wherein the training data set (320) is not transmitted externally from the X-ray mammography system (10, 210).

10. 2. The method of claim 1, wherein acquiring the sensor data (306) comprises acquiring images of at least the patient, the patient's breast, or the combination of the patient, the patient's breast, and combinations thereof.

11. The image processing system (44, 256) is configured to sense one or more triggering events of the X-ray mammography system (10, 210), and the step of acquiring (306) the sensor data includes: a. sensing a triggering event (302); and b. operating said sensor detection system (100, 252) such that said sensor data (306) is acquired. The method of claim 1 , comprising:

12. The step of positioning the patient's breast (236) between the detector (18) and the compression paddle (40) comprises: a. positioning the patient's breast (236) between the compression paddle (40) and the detector (18); b. moving the compression paddle (40) to a desired position such that the patient's breast (236) is compressed between the compression paddle (40) and the detector (18); and c. locking the compression paddle (40) in the desired position. Including, The method of claim 11 , wherein the sensed triggering event is a locking of the compression paddle (40).

13. 1. An X-ray mammography system (10, 210) operable in an imaging mode and in an intervention / biopsy mode, the system (10, 210) comprising: a. a gantry (15) disposed on a support surface, the gantry (15) including an x-ray source (16), an x-ray detector (18) alignable with said x-ray source (16), and a compression paddle (40) movable relative to said detector (18), the compression paddle (40) securing a patient's breast (236) between said detector and said compression paddle; b. an image processing system (44, 256) operably connected to the gantry (15) and controlling operation of the X-ray source (16) and the X-ray detector (18) to generate X-ray image data in an imaging mode of the mammography system (10, 210), the image processing system (44, 256) including a processor (258) for processing the X-ray image data from the detector (18), a database (260) operably connected to the processor (258) and storing instructions for operating a feature detection AI model (262) and a training module (263), a display (56) operably connected to the image processing system (44, 256) for presenting information to a user, and a user interface (50) operably connected to the image processing system (44, 256) for allowing user input (312) to the image processing system (44, 256); c. a sensor detection system (100, 252) disposed on the gantry (15) and operatively connected to the image processing system (44, 256), the sensor detection system (100, 252) including at least one sensor (101, 102, 154, 254) operable to generate sensor data (306) of the gantry (15) and the patient (236), the feature detection AI model (262) operable to detect features of one or both of the gantry (15) and the patient (236) in the sensor data (306) to assess the gantry (15) and the patient (236); Including, The training module (263) is configured to receive at least one of user input parameters (312) from the user interface (50) for a shooting procedure to be performed by the X-ray mammography system (10, 210), one or more internal system outputs (314) from a control algorithm stored in the database (260), and combinations thereof, and sensor data (306) from the sensor detection system (100, 252), to form a training data set (320) used to train the feature detection AI model (262), wherein the training data set (320) is not transmitted externally from the X-ray mammography system (10, 210).

14. 14. The X-ray mammography system (10, 210) of claim 13, wherein the training module (263) is configured to operate in a continuous learning mode.

15. The X-ray mammography system (10, 210) of claim 13, wherein the training module (263) is configured to operate in an associative learning mode.

16. The input parameters (312) include laterality, field of view, and / or patient position; and / or The method of claim 1 , wherein the one or more internal system outputs (314) include an output related to a detected current breast position.

Images