A method, apparatus, device, and medium for fusing DR images and ultrasound images

Abstract

This invention relates to the field of medical image processing, and discloses a method, apparatus, computer device, and storage medium for fusing DR images and ultrasound images. The method includes: acquiring a DR image containing the position of the DR probe array; calibrating the position of the DR probe array using an optical positioning device; acquiring an ultrasound image containing probe array position data; calibrating the probe array position data using an optical positioning device; determining the projection mapping relationship between the DR image and the ultrasound image based on the DR probe array position and the probe array position data; and fusing the DR image and the ultrasound image according to the projection mapping relationship to obtain DR-ultrasound dual-modal fused data. This invention effectively improves the accuracy and efficiency of DR image and ultrasound image fusion, providing reliable dual-modal visualization support for medical image diagnosis.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and in particular to a method, apparatus, device, and medium for fusing DR images and ultrasound images. Background Technology

[0002] In clinical medical imaging diagnosis, digital X-ray (DR) and ultrasound imaging are two commonly used and complementary examination methods, each with unique advantages in displaying bone structures and dynamically assessing soft tissues, respectively. DR images have high spatial resolution, clearly presenting the anatomical location and contour of lesions, and are widely used for preliminary screening and localization; while ultrasound imaging has the characteristics of being real-time, radiation-free, and allowing dynamic observation of blood flow, making it suitable for the functional assessment of soft tissue lesions.

[0003] However, current imaging data from these two modalities are typically acquired and stored independently, forming "data silos" and lacking effective spatial correlation mechanisms. This forces doctors to rely on subjective experience to match lesions, easily introducing diagnostic biases. Furthermore, ultrasound examinations are highly dependent on the operator's scanning technique, making it difficult to accurately record and reproduce the spatial position of the scanning plane, resulting in challenges in follow-up comparisons. Existing medical image fusion technologies mostly focus on three-dimensional registration or projection mapping of CT, MRI, and ultrasound; research on real-time spatial localization and fusion of DR and ultrasound—two-dimensional, heterogeneous, and time-varying images—remains lacking.

[0004] Therefore, there is an urgent need for a method that can achieve accurate spatial registration and real-time fusion of DR images and ultrasound images. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for fusing DR images and ultrasound images to improve the accuracy and repeatability of collaborative diagnosis using DR and ultrasound images.

[0006] A method for fusing DR images and ultrasound images includes:

[0007] Acquire DR images containing the location of the DR detector array; the location of the DR detector array is calibrated using an optical positioning device;

[0008] Acquire an ultrasound image containing probe array position data; the probe array position data is calibrated using the optical positioning device.

[0009] The projection mapping relationship between the DR image and the ultrasound image is determined based on the position data of the DR detection array and the probe detection array.

[0010] The DR image and the ultrasound image are fused according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data.

[0011] Optionally, acquiring the DR image containing the location of the DR detector array includes:

[0012] The DR image of a designated area of ​​the patient is acquired through the DR detection component;

[0013] The optical positioning device identifies the first position information data of multiple DR marker points set on the DR detection component;

[0014] The location of the DR detection array is determined based on the first location information data.

[0015] Optionally, if the DR detection component includes a flat panel detector, then the plurality of DR marker points include the four corners of the flat panel detector;

[0016] The location of the DR detection array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image.

[0017] Optionally, acquiring the ultrasound image containing probe array position data includes:

[0018] The ultrasound image of the designated area of ​​the patient is acquired using an ultrasound probe;

[0019] The optical positioning device identifies the second position information data of multiple probe markers set on the ultrasonic probe.

[0020] The probe detection array position data is determined based on the second position information data.

[0021] Optionally, the number of the plurality of probe identification points is four;

[0022] The four probe markers are coplanar and not equidistant from each other;

[0023] If the ultrasound probe is a linear probe, the probe detection array position data includes the positions of the two endpoints of the linear probe.

[0024] Optionally, the location of the DR detector array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image;

[0025] Determining the projection mapping relationship between the DR image and the ultrasound image based on the DR detector array position and the probe detector array position data includes:

[0026] An ultrasonic detection projection model is constructed based on the probe array position data.

[0027] Determine whether all projection points of the ultrasonic detection projection model on the plane of the DR image are within the boundary range;

[0028] If all the projection points are within the boundary range, it is determined that there is a projection mapping relationship between the DR image and the ultrasound image, and the probe projection coordinates of the projection points and the projection distance of the ultrasound detection projection model are recorded.

[0029] Optionally, after fusing the DR image and the ultrasound image according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data, the method further includes:

[0030] When displaying the ultrasound image in the DR ultrasound dual-modal fusion data, the DR image associated with the ultrasound image is loaded, and the probe projection coordinates associated with the ultrasound image are extracted from the DR ultrasound dual-modal fusion data. The ultrasound probe identifier is then displayed on the DR image according to the probe projection coordinates.

[0031] When displaying the ultrasound movie in the DR ultrasound dual-modal fusion data, the DR image associated with the ultrasound movie is loaded, and the scanning trajectory of the ultrasound probe is synchronously played back on the DR image.

[0032] When displaying the DR image in the DR ultrasound dual-modal fusion data, the ultrasound image / ultrasound movie associated with the ultrasound movie is loaded.

[0033] A device for fusing DR images and ultrasound images, comprising:

[0034] A DR image acquisition module is used to acquire DR images containing the positions of the DR detector array; the positions of the DR detector array are calibrated by an optical positioning device.

[0035] An ultrasound image acquisition module is used to acquire ultrasound images containing probe array position data; the probe array position data is calibrated by the optical positioning device.

[0036] The projection mapping relationship determination module is used to determine the projection mapping relationship between the DR image and the ultrasound image based on the position data of the DR detector array and the position data of the probe detector array;

[0037] A data fusion generation module is used to fuse the DR image and the ultrasound image according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data.

[0038] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the above-described method for fusing DR images and ultrasound images.

[0039] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for fusing DR images and ultrasound images.

[0040] The aforementioned method, apparatus, equipment, and medium for fusing DR and ultrasound images utilize optical positioning equipment to spatially calibrate the DR detection array and ultrasound probe, achieving precise registration of the two modalities of medical images in a unified coordinate system. By directly establishing a projection mapping relationship through positional data, it avoids the computational errors and time-consuming problems associated with traditional image registration algorithms. This invention effectively improves the accuracy and efficiency of DR and ultrasound image fusion, providing reliable dual-modal visualization support for medical image diagnosis. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of an application environment for a method of fusing DR images and ultrasound images according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of a flat panel detector in one embodiment of the present invention;

[0044] Figure 3 This is a flowchart of a method for fusing DR images and ultrasound images according to an embodiment of the present invention;

[0045] Figure 4 This is a DR image projected with ultrasound probe position information in one embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of a device for fusing DR images and ultrasound images according to an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] This invention provides a method for fusing DR images and ultrasound images, which can be applied to applications such as... Figure 1 The application environment shown. For example... Figure 1 As shown, an optical positioning device is positioned above and to the left of the DR device's flat panel detector. This optical positioning device consists of at least one camera. It has a wide field of view and can track optical markers in the space to the right. The DR device's flat panel detector is a square panel with four optical markers a, b, c, and d at its four vertices. The spatial coordinates of these markers can be captured in real-time by the optical positioning device to determine the position of the DR detection array on the flat panel detector. Figure 2 As shown, Figure 2 This is a simplified schematic diagram of a flat panel detector in one embodiment. The DR detector array is located inside the flat panel detector, with its four vertices labeled i, j, k, and l. The area where the DR detector array is located is the effective imaging area of ​​the flat panel detector. The relative position of the flat panel detector and the optical positioning device remains fixed. The optical positioning device identifies the four vertices a, b, c, and d of the flat panel detector and registers their three-dimensional spatial information. The patient can have DR images taken through the flat panel detector. When the patient completes the DR image taking, the position of the DR detector array can be obtained based on the physical distance from the vertices of the flat panel detector to the DR detector array, which is the planar coordinate information of the current DR image in the three-dimensional space calibrated by the optical positioning device.

[0050] Next, an ultrasound probe with multiple markers is used to perform an ultrasound examination on the patient, acquiring ultrasound images. During the ultrasound examination, the patient's position remains unchanged from that during the DR examination. An optical positioning device identifies the coordinates of multiple markers on the ultrasound probe in real time. The physical distance from each marker to the probe array is determined during marker installation and is a known quantity; therefore, the probe array position data is calculated based on spatial geometric relationships. Furthermore, the projection mapping relationship between the DR image and the ultrasound image can be evaluated based on the DR array position and the probe array position data. If a projection mapping relationship exists between the DR image and the ultrasound image, DR ultrasound dual-modal fusion data can be generated. Figure 1 In this process, the same optical positioning system is used to track the DR flat panel detector (marker ad) and the ultrasound probe (marker eh), thereby achieving precise correlation between DR images and ultrasound images in a unified world coordinate system.

[0051] In one embodiment, such as Figure 3 As shown, a method for fusing DR images and ultrasound images is provided, including the following steps S10~S40.

[0052] S10. Acquire a DR image containing the position of the DR detector array; the position of the DR detector array is calibrated by an optical positioning device.

[0053] In essence, a DR (Digital Radiography) device is used to scan a specific area of ​​a patient (such as the chest cavity) to obtain a DR image (digital X-ray). Simultaneously, optical markers placed on the DR detector are continuously tracked by an optical positioning device to precisely determine the exact position of the DR detector array in global space. This results in a DR image that includes the position of the DR detector array.

[0054] Optionally, step S10, namely acquiring the DR image containing the location of the DR detector array, includes:

[0055] S101. Acquire the DR image of the designated area of ​​the patient through the DR detection component;

[0056] S102. The optical positioning device identifies the first position information data of multiple DR marker points set on the DR detection component;

[0057] S103. Determine the position of the DR detection array based on the first position information data.

[0058] Understandably, the area to be examined (such as the chest) is placed above the DR probe (flat panel detector), as shown in the attached image. Figure 1 As shown in the central position. Subsequently, the X-ray tube of the DR device (located above the patient area, not shown in the figure) emits X-rays. After the X-rays penetrate the patient area, they are received by the DR detection unit below, thereby generating a two-dimensional digital X-ray image showing the internal bone or tissue structure, i.e., a DR image.

[0059] The initial positional information of multiple DR markers set on the DR detector component can be identified using optical positioning equipment. In some examples, the multiple DR markers may refer to the four corners of the DR detector component. The position of the DR detector array can be calculated based on the physical distance from the corners of the DR detector component to the DR detector array (i.e., the effective imaging area of ​​the flat panel detector). The position of the DR detector array includes the boundary range of the DR image. In some examples, the positional information of each DR marker can be registered before DR image capture.

[0060] This embodiment achieves high-precision calibration of the DR detection array position by setting multiple marker points on the DR detection component and using optical positioning equipment to obtain its spatial position in real time. This avoids positioning errors caused by relying on internal parameters of the equipment or preset geometric models, and improves the accuracy of the spatial coordinates of DR images.

[0061] Optionally, if the DR detection component includes a flat panel detector, then the plurality of DR marker points include the four corners of the flat panel detector;

[0062] The location of the DR detection array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image.

[0063] Understandably, such as Figure 1 As shown, the DR detection component can be a flat panel detector, and the four corners a, b, c, and d of the flat panel detector can be set as DR marker points. The plane equation of the plane containing the DR image can be expressed as:

[0064]

[0065] Where A, B, and C are the normal vector components of the plane containing the DR image, and D is a constant, the normal vector... Any point on the plane where the DR image is located All satisfy the above plane equations. The plane containing the DR image can also have a normal space obtained in the three-dimensional space provided by the optical positioning equipment. .

[0066] like Figure 2 As shown, Figure 2 This is a schematic diagram of a flat panel detector in one embodiment. The DR detector array is located inside the flat panel detector, and its four vertices are i, j, k, and l, respectively. The boundary of the DR image can be a rectangular area enclosed by i, j, k, and l.

[0067] In this embodiment, DR markers are placed at the four corners of the flat panel detector, facilitating rapid and stable identification of the detector's spatial attitude by the optical positioning device. The four corners allow for precise fitting of the plane equation of the plane containing the DR image, determining the image's boundary range and achieving complete spatial modeling of the DR imaging area. This embodiment offers high calibration efficiency, significantly improving the positioning accuracy and geometric consistency of DR images in three-dimensional space.

[0068] S20. Acquire an ultrasound image containing probe array position data; the probe array position data is calibrated by the optical positioning device.

[0069] In essence, an ultrasound probe is used to scan the same area of ​​a patient, obtaining ultrasound images. Multiple ultrasound images can be combined to form an ultrasound movie. Optical markers placed on the ultrasound probe are continuously tracked by an optical positioning device, thereby accurately determining the precise position of the probe array in global space, i.e., the probe array position data. Each frame of the ultrasound image is associated with a probe array spatial coordinate and probe orientation.

[0070] Optionally, step S20, namely acquiring the ultrasound image containing probe array position data, includes:

[0071] S201. Acquire ultrasound images of the designated area of ​​the patient using an ultrasound probe;

[0072] S202. The second position information data of multiple probe marker points set on the ultrasonic probe are identified by the optical positioning device;

[0073] S203. Determine the probe detection array position data based on the second position information data.

[0074] Understandably, the patient's examination posture can be kept unchanged, and the ultrasound probe can be placed on the designated area to be examined (such as the chest cavity). The piezoelectric transducer inside the ultrasound probe emits ultrasound waves and receives echo signals from the patient's internal tissues. These signals are processed by the ultrasound host to generate an ultrasound image in real time.

[0075] While acquiring ultrasound images, multiple probe markers set on the ultrasound probe are captured by the camera of the optical positioning device. The system identifies these probe markers in real time and accurately measures the coordinates (X, Y, Z) of each probe marker in three-dimensional space, which is the second position information data. The physical distance from each probe marker to the probe detection array is determined when the probe markers are set, and the position data of the probe detection array can then be calculated based on this physical distance and the second position information data.

[0076] This embodiment achieves high-precision dynamic calibration of the probe array position by setting multiple marker points on the ultrasonic probe and using optical positioning equipment to acquire their spatial positions in real time. Based on the marker point data, the spatial attitude and position of the ultrasonic imaging plane can be accurately reconstructed, overcoming the registration deviation caused by the traditional reliance on the probe's internal coordinate system. This embodiment effectively improves the positioning accuracy of ultrasonic images in the global coordinate system, providing a reliable spatial reference for multimodal image fusion.

[0077] Optionally, the number of the plurality of probe identification points is four;

[0078] The four probe markers are coplanar and not equidistant from each other;

[0079] If the ultrasound probe is a linear probe, the probe detection array position data includes the positions of the two endpoints of the linear probe.

[0080] Understandably, such as Figure 1As shown, four probe markers can be set on a linear ultrasound probe. These four markers are arranged on a plane (i.e., coplanar), and the spacing between them is unequal (i.e., pairwise unequal), forming a unique spatial pattern. When the ultrasound probe scans the designated area of ​​the patient, the optical positioning device continuously tracks these four markers and obtains their precise three-dimensional coordinates. The coordinates of these four points can be used to verify whether they are coplanar, ensuring the validity of the data. Then, based on the unique spatial configuration formed by these four points, the position and orientation of the entire probe (including its probe array) in three-dimensional space are calculated. Since the ultrasound probe is a linear probe, its probe array is approximately a straight line. Based on the calculated probe spatial orientation, combined with a pre-calibrated probe geometric model, the coordinates of the two endpoints of the probe array in three-dimensional space, i.e., the endpoint positions, can be accurately derived.

[0081] This embodiment employs four coplanar probe markers, each pairwise unequally spaced, to uniquely determine the planar orientation of the ultrasound probe in space, avoiding positioning ambiguities caused by symmetrical or equidistant layouts. For linear probes, the spatial positions of the two endpoints of the probe array are accurately derived through the markers, precisely characterizing the geometric range of the ultrasound imaging area. This embodiment balances calibration stability and computational simplicity, significantly improving the accuracy and robustness of spatial positioning for linear ultrasound probes.

[0082] S30. Determine the projection mapping relationship between the DR image and the ultrasound image based on the position data of the DR detection array and the probe detection array.

[0083] Understandably, since DR images and ultrasound images are acquired in the same world coordinate system defined by the same optical positioning system, the projection point of the probe array onto the DR image can be calculated in this world coordinate system. If the projection point is within the boundary range of the DR image, it can be determined that there is a correlation between the DR image and the ultrasound image; if the projection point is outside the boundary range of the DR image, it can be determined that there is no correlation between the DR image and the ultrasound image.

[0084] Optionally, the location of the DR detector array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image;

[0085] Step S30, namely, determining the projection mapping relationship between the DR image and the ultrasound image based on the DR detector array position and the probe detector array position data, includes:

[0086] S301. Construct an ultrasonic detection projection model based on the probe array position data;

[0087] S302. Determine whether all projection points of the ultrasonic detection projection model on the plane of the DR image are within the boundary range.

[0088] S303. If all the projection points are within the boundary range, it is determined that there is a projection mapping relationship between the DR image and the ultrasound image, and the probe projection coordinates of the projection points and the projection distance of the ultrasound detection projection model are recorded.

[0089] Understandably, the imaging area scanned by an ultrasound probe can be abstracted into a geometric model based on the probe's array position data. For common linear probes, this model can be simplified to a rectangular plane; for convex or phased array probes, it may be a sector-shaped surface. This model includes the origin point of the ultrasound beam in space (probe contact point), the principal direction (central axis of the sound beam), and the scanning range.

[0090] The constructed ultrasound probe projection model (such as a rectangle or sector) is "projected" onto the plane containing the DR image. This process is similar to shining a beam of light along the normal direction of the DR detector onto the ultrasound model, creating a shadow on the DR plane. Next, the system calculates all vertices or contour points of this "shadow" (i.e., the projection area) and checks whether each point falls within the boundary range of the DR image determined in step S103. If all projection points are within the boundary, it means that the content of the ultrasound image is completely contained within the anatomical area covered by the DR image, and effective fusion can be performed. If any point falls outside the boundary, it means that part of the ultrasound scan area is "invisible" on the DR image, and the matching may be invalid, requiring readjustment or discard.

[0091] An ultrasound probe tip model can be created using markers provided by an optical positioning device. Taking a linear probe as an example, a straight line model can be used to represent the probe tip model. The coordinates of all points on the probe tip model should satisfy the condition that the foot of the perpendicular H of each point lies within the plane of the DR image. (Plane normal vector) any point On a plane (select a three-dimensional coordinate point on the DR image) ,but,

[0092]

[0093] The foot of the perpendicular, H, is:

[0094]

[0095] When the probe tip model is located in the normal space of the plane containing the DR image, it can obtain a vertical projection onto the DR image. Once the system determines that all projection points are within the boundary range, it formally confirms the projection mapping relationship between the DR image and the ultrasound image. Subsequently, the system accurately records two key pieces of information: probe projection coordinates and projection distance. Probe projection coordinates refer to the three-dimensional coordinates of each pixel on the ultrasound image, calculated based on its relative position within the ultrasound model, and projected onto the DR image plane. .

[0096] Projection distance This refers to the center point of the model at the top of the probe. Vertical distance to the DR image plane. Projection distance. It can be represented as:

[0097]

[0098] As the ultrasound probe moves, its trajectory on the DR image can be displayed in real time. It's important to note the projection distance. It can refer to the THK body thickness parameter, which represents the human body thickness between the center of the current probe tip and the corresponding projection point in the DR image.

[0099] This embodiment achieves geometric verification of the spatial relationship between dual-modal images by constructing an ultrasound probe projection model and projecting it onto the DR image plane. By determining whether the projection point falls within the boundary of the DR image, image pairs with effective overlapping areas are effectively screened, avoiding invalid fusion. The probe projection coordinates and projection distance are recorded to provide accurate geometric parameters for subsequent pixel-level mapping and image fusion. This embodiment significantly improves the accuracy, reliability, and computational efficiency of DR and ultrasound fusion.

[0100] S40. The DR image and the ultrasound image are fused according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data.

[0101] Understandably, after determining the projection mapping relationship, DR images and ultrasound images can be fused to obtain DR ultrasound dual-modal fusion data. In one example, such as... Figure 4 As shown, the position of the ultrasound probe can be projected onto the DR image, which is DR ultrasound dual-modal fusion data. Doctors can directly see the examination trajectory of the ultrasound probe on a clear DR image, achieving complementary advantages and providing more comprehensive information for diagnosis or surgical navigation.

[0102] This embodiment utilizes optical positioning equipment to spatially calibrate the DR detection array and ultrasound probe, achieving precise registration of the two modalities of medical images in a unified coordinate system. By directly establishing a projection mapping relationship through positional data, it avoids the computational errors and time-consuming issues associated with traditional image registration algorithms. This embodiment effectively improves the accuracy and efficiency of fusion between DR and ultrasound images, providing reliable dual-modal visualization support for medical image diagnosis.

[0103] Optionally, after step S40, that is, after fusing the DR image and the ultrasound image according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data, the method further includes:

[0104] S50. When displaying the ultrasound image in the DR ultrasound dual-modal fusion data, load the DR image associated with the ultrasound image, extract the probe projection coordinates associated with the ultrasound image from the DR ultrasound dual-modal fusion data, and display the ultrasound probe identifier on the DR image according to the probe projection coordinates.

[0105] S60. When displaying the ultrasound movie in the DR ultrasound dual-modal fusion data, load the DR image associated with the ultrasound movie and synchronously play back the scanning trajectory of the ultrasound probe on the DR image.

[0106] S70. When displaying the DR image in the DR ultrasound dual-modal fusion data, load the ultrasound image / ultrasound movie associated with the ultrasound movie.

[0107] Understandably, when displaying ultrasound images in DR ultrasound dual-modal fusion data, DR images associated with ultrasound images can be loaded, and probe projection coordinates associated with ultrasound images can be extracted from DR ultrasound dual-modal fusion data. Ultrasound probe identifiers can then be displayed on the DR images based on the probe projection coordinates.

[0108] For example, when a doctor observes a static ultrasound image from a fused dual-modal DR ultrasound dataset on the diagnostic interface, showing a suspicious lesion in the patient's liver, the corresponding DR image captured at that location can be automatically loaded into another view or picture-in-picture window on the screen, based on the spatial correlation information recorded in the fused data. This DR image serves as a background image displaying the skeletal and overall anatomical structures. The probe projection coordinates on the DR image plane when the ultrasound image was acquired can be extracted from the fused data. These coordinates are then overlaid on the loaded DR image with a prominent, graphical ultrasound probe icon. The doctor can clearly see at a glance the angle from which the current ultrasound section was acquired and its corresponding specific location on the DR image.

[0109] When displaying ultrasound movies in DR ultrasound dual-modal fusion data, the associated DR images can be loaded, and the scanning trajectory of the ultrasound probe can be simultaneously replayed on the DR images. For example, when a doctor plays back a previously scanned and saved ultrasound movie (i.e., a dynamic ultrasound image sequence, usually a few seconds of real-time scanning video), the complete DR image corresponding to this ultrasound movie can be loaded as a static background. When the doctor clicks to play the ultrasound movie, the movement path of the ultrasound probe during the acquisition of this movie is dynamically drawn on the DR image, i.e., the scanning trajectory. This trajectory may be a gradually extending bright line or a series of continuous points, the position of which is calculated entirely from the probe projection coordinates corresponding to each frame of the movie. This function allows doctors to "review" the entire ultrasound scanning process. By observing the movement trajectory of the probe on the DR background, it is possible to clearly understand how the ultrasound section traverses the target area, ensuring that all regions of interest have been completely scanned, effectively avoiding missed diagnoses due to incomplete scanning.

[0110] When displaying DR images from dual-modal fusion ultrasound data, ultrasound images / movies associated with ultrasound films can be loaded. For example, when a doctor clicks or selects a specific area on a DR image (such as a rib or lung shadow), all ultrasound images or films acquired near the selected location on the DR image can be automatically found based on the spatial mapping relationship recorded in the fusion data. This associated ultrasound data (which may be a set of static images or a short film) is then automatically loaded and displayed in another window. This achieves a seamless switch from macroscopic anatomy (DR) to microscopic tissue characteristics (ultrasound). When doctors see abnormalities on DR images, they can immediately view more detailed soft tissue information about the area, such as the presence of fluid accumulation or the internal echo characteristics of a mass, providing crucial information for accurate diagnosis.

[0111] This embodiment enhances the visualization and correlation of bimodal data by loading and marking corresponding DR image regions when displaying ultrasound images, enabling physicians to intuitively understand the spatial location of lesions. Simultaneously displaying the probe scanning trajectory while playing ultrasound films not only provides a dynamic operational perspective but also helps to accurately track key frames and positions during the examination process. Loading associated ultrasound data when viewing DR images achieves seamless switching from anatomical structure to functional information, greatly improving diagnostic efficiency and accuracy. This embodiment strengthens the interactivity and complementarity between multimodal image data, providing a powerful support tool for clinical diagnosis and treatment.

[0112] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0113] In one embodiment, an apparatus for fusing DR images and ultrasound images is provided, which corresponds one-to-one with the method for fusing DR images and ultrasound images described in the above embodiments. For example... Figure 5 As shown, the device for fusing DR images and ultrasound images includes:

[0114] The DR image acquisition module 10 is used to acquire DR images containing the positions of the DR detector array; the positions of the DR detector array are calibrated by an optical positioning device.

[0115] The ultrasound image acquisition module 20 is used to acquire ultrasound images containing probe array position data; the probe array position data is calibrated by the optical positioning device.

[0116] The projection mapping relationship determination module 30 is used to determine the projection mapping relationship between the DR image and the ultrasound image based on the DR detector array position and the probe detector array position data.

[0117] The data fusion generation module 40 is used to fuse the DR image and the ultrasound image according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data.

[0118] Optionally, the DR image acquisition module 10 includes:

[0119] A DR imaging unit is used to acquire DR images of a specified area of ​​the patient via a DR detection component;

[0120] The first position unit is used to identify first position information data of multiple DR marker points set on the DR detection component through the optical positioning device;

[0121] A DR array position determination unit is used to determine the position of the DR detection array based on the first position information data.

[0122] Optionally, if the DR detection component includes a flat panel detector, then the plurality of DR marker points include the four corners of the flat panel detector;

[0123] The location of the DR detection array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image.

[0124] Optionally, the ultrasound image acquisition module 20 includes:

[0125] An ultrasound image acquisition unit is used to acquire ultrasound images of the specified area of ​​the patient via an ultrasound probe;

[0126] The second position unit is used to identify second position information data of multiple probe marker points set on the ultrasonic probe through the optical positioning device;

[0127] The probe array position determination unit is used to determine the probe detection array position data based on the second position information data.

[0128] Optionally, the number of the plurality of probe identification points is four;

[0129] The four probe markers are coplanar and not equidistant from each other;

[0130] If the ultrasound probe is a linear probe, the probe detection array position data includes the positions of the two endpoints of the linear probe.

[0131] Optionally, the location of the DR detector array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image; the module 30 for determining the projection mapping relationship includes:

[0132] A projection model unit is constructed to construct an ultrasonic detection projection model based on the probe detection array position data.

[0133] The projection point determination unit is used to determine whether all projection points of the ultrasonic detection projection model on the plane of the DR image are within the boundary range.

[0134] The projection mapping relationship determination unit is used to determine that there is a projection mapping relationship between the DR image and the ultrasound image if all the projection points are within the boundary range, and to record the probe projection coordinates of the projection points and the projection distance of the ultrasound detection projection model.

[0135] Optionally, the apparatus for fusing DR images and ultrasound images further includes:

[0136] The first fusion data display module is used to load the DR image associated with the ultrasound image when displaying the ultrasound image in the DR ultrasound dual-modal fusion data, extract the probe projection coordinates associated with the ultrasound image from the DR ultrasound dual-modal fusion data, and display the ultrasound probe identifier on the DR image according to the probe projection coordinates.

[0137] The second fusion data display module is used to load the DR image associated with the ultrasound movie when displaying the ultrasound movie in the DR ultrasound dual-modal fusion data, and to synchronously play back the scanning trajectory of the ultrasound probe on the DR image.

[0138] The third fusion data display module is used to load ultrasound images / ultrasound films associated with the ultrasound films when displaying DR images in the DR ultrasound dual-modal fusion data.

[0139] Specific limitations regarding the device for fusing DR and ultrasound images can be found in the limitations of the method for fusing DR and ultrasound images described above, and will not be repeated here. Each module in the aforementioned device for fusing DR and ultrasound images can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0140] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data related to the method of fusing DR images and ultrasound images. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for fusing DR images and ultrasound images.

[0141] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for fusing DR images and ultrasound images described in the above embodiment; to avoid repetition, it will not be described again here. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in this embodiment of the apparatus for fusing DR images and ultrasound images; to avoid repetition, it will not be described again here.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the method for fusing DR images and ultrasound images described in the above embodiment. To avoid repetition, this will not be described again here. Alternatively, when executed by a processor, the computer program implements the functions of each module / unit in this embodiment of the apparatus for fusing DR images and ultrasound images. To avoid repetition, this will not be described again here.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0144] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0145] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for fusing DR images and ultrasound images, characterized in that, include: Acquire DR images containing the location of the DR detector array; the location of the DR detector array is calibrated using an optical positioning device; Acquire ultrasound images containing probe array position data; The probe detection array position data is calibrated by the optical positioning device; The projection mapping relationship between the DR image and the ultrasound image is determined based on the position data of the DR detection array and the probe detection array. The DR image and the ultrasound image are fused according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data; The acquisition of DR images containing the location of the DR detector array includes: The DR image is acquired by the DR detection component at a designated location on the patient. The optical positioning device identifies the first position information data of multiple DR marker points set on the DR detection component; The position of the DR detection array is determined based on the first location information data; The location of the DR detection array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image. Determining the projection mapping relationship between the DR image and the ultrasound image based on the position data of the DR detector array and the probe detector array includes: An ultrasonic detection projection model is constructed based on the probe array position data. Determine whether all projection points of the ultrasonic detection projection model on the plane of the DR image are within the boundary range; If all the projection points are within the boundary range, it is determined that there is a projection mapping relationship between the DR image and the ultrasound image, and the probe projection coordinates of the projection points and the projection distance of the ultrasound detection projection model are recorded.

2. The method for fusing DR images and ultrasound images according to claim 1, characterized in that, If the DR detection component includes a flat panel detector, then the plurality of DR marker points include the four corners of the flat panel detector; The location of the DR detection array includes the plane equation of the plane where the DR image is located and the boundary range of the DR image.

3. The method for fusing DR images and ultrasound images according to claim 1, characterized in that, The acquisition of the ultrasound image containing probe array position data includes: The ultrasound image of the designated area of ​​the patient is acquired using an ultrasound probe; The optical positioning device identifies second position information data of multiple probe marker points set on the ultrasonic probe. The probe detection array position data is determined based on the second position information data.

4. The method for fusing DR images and ultrasound images according to claim 3, characterized in that, The number of the multiple probe identification points is four; The four probe markers are coplanar and not equidistant from each other; If the ultrasound probe is a linear probe, the probe detection array position data includes the positions of the two endpoints of the linear probe.

5. The method for fusing DR images and ultrasound images according to claim 1, characterized in that, After fusing the DR image and the ultrasound image according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data, the method further includes: When displaying the ultrasound image in the DR ultrasound dual-modal fusion data, the DR image associated with the ultrasound image is loaded, and the probe projection coordinates associated with the ultrasound image are extracted from the DR ultrasound dual-modal fusion data. The ultrasound probe identifier is then displayed on the DR image according to the probe projection coordinates. When displaying the ultrasound movie in the DR ultrasound dual-modal fusion data, the DR image associated with the ultrasound movie is loaded, and the scanning trajectory of the ultrasound probe is synchronously played back on the DR image. When displaying the DR image in the DR ultrasound dual-modal fusion data, the ultrasound image / ultrasound movie associated with the ultrasound movie is loaded.

6. A device for fusing DR images and ultrasound images, characterized in that, include: A DR image acquisition module is used to acquire DR images containing the positions of the DR detector array; the positions of the DR detector array are calibrated by an optical positioning device. An ultrasound image acquisition module is used to acquire ultrasound images containing probe array position data; the probe array position data is calibrated by the optical positioning device. The projection mapping relationship determination module is used to determine the projection mapping relationship between the DR image and the ultrasound image based on the position data of the DR detector array and the position data of the probe detector array; A data fusion generation module is used to fuse the DR image and the ultrasound image according to the projection mapping relationship to obtain DR ultrasound dual-modal fusion data; The DR image acquisition module includes: A DR imaging unit is used to acquire DR images of a specified area of ​​the patient via a DR detection component; The first position unit is used to identify first position information data of multiple DR marker points set on the DR detection component through the optical positioning device; A DR array position determination unit is used to determine the position of the DR detection array based on the first position information data; The location of the DR detection array includes the plane equation of the plane containing the DR image and the boundary range of the DR image; the module for determining the projection mapping relationship includes: A projection model unit is constructed to construct an ultrasonic detection projection model based on the probe detection array position data. The projection point determination unit is used to determine whether all projection points of the ultrasonic detection projection model on the plane of the DR image are within the boundary range. The projection mapping relationship determination unit is used to determine that there is a projection mapping relationship between the DR image and the ultrasound image if all the projection points are within the boundary range, and to record the probe projection coordinates of the projection points and the projection distance of the ultrasound detection projection model.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for fusing DR images and ultrasound images according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for fusing DR images and ultrasound images according to any one of claims 1 to 5.

Images