Image acquisition method and device, electronic equipment and storage medium

Abstract

The invention provides an image acquisition method and device, electronic equipment and a storage medium, and the method is applied to the field of medical image processing, and comprises the steps: obtaining a first region image, a first marking image, a second region image and a second marking image; and inputting the first area image, the first marking image, the second area image and the second marking image into a pre-trained registration model, obtaining a registration image obtained by registering the area image of the bracket contained in the second area image based on the output of the registration model, and obtaining the registration image of the bracket while utilizing the overall information in the area image. The information of the mark image is added, the registration image obtained based on the registration model not only can be aligned with the first area image, but also can be aligned with the first marker in the first mark image, and the registration image which is accurately displayed and comprises the support is obtained.

Description

Technical Field

[0001] The present application relates to the field of medical image processing, and more specifically, to an image acquisition method, device, electronic device and storage medium in the field of medical image processing. Background Art

[0002] Stent implantation is commonly used to treat cardiovascular diseases. It can dilate blood vessels and keep blood flowing smoothly in the blood vessels. After the stent is implanted at the lesion site, it is necessary to observe the deployment morphology of the stent to achieve fine control of the implantation. However, the appearance of the stent is complex, and after it is implanted at the lesion site, the morphology of the stent cannot be accurately displayed in the perspective image due to the low contrast between the blood vessels and surrounding tissues and imaging defects. Summary of the invention

[0003] The present application provides an image acquisition method, device, electronic device and storage medium. The registration image obtained by the method through the registration model can be aligned with the first area image and the first marker in the first marker image, thereby realizing accurate display of the registration image containing the bracket.

[0004] In a first aspect, a method for acquiring an image is provided, the method comprising:

[0005] Acquire a first perspective image, and identify a first marker in the first perspective image;

[0006] Based on the first marker, a first perspective image is intercepted to obtain a first area image, and a first marker image corresponding to the first marker in the first area image is acquired;

[0007] Acquire a second region image of the second marker in the second perspective image and a second marker image corresponding to the second marker in the second region image;

[0008] The first area image, the first marked image, the second area image and the second marked image are input into the registration model to obtain the registered image of the second area image and the second marked image through the deformation field output by the registration model. The loss function of the registration model includes a first loss function and a second loss function obtained based on the registration image, the first area image, and the first marked image.

[0009] In a second aspect, an image acquisition device is provided, the device comprising:

[0010] A first acquisition unit, used to acquire a first perspective image and identify a first marker in the first perspective image;

[0011] A second acquisition unit is used to intercept the first perspective image based on the first marker to obtain a first area image, and obtain a first marker image corresponding to the first marker in the first area image;

[0012] a third acquiring unit, configured to acquire a second region image of the second perspective image and a second marker image corresponding to a second marker in the second region image;

[0013] An input unit is used to input the first area image, the first marked image, the second area image and the second marked image into the registration model to obtain the registered image of the second area image and the second marked image through the deformation field output by the registration model, and the loss function of the registration model includes a first loss function and a second loss function obtained based on the registration image, the first area image, and the first marked image.

[0014] In a third aspect, an electronic device is provided, comprising: a memory for storing executable program code; and a processor for calling and running the executable program code from the memory, so that the electronic device executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0015] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0016] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0017] In an embodiment of the present application, after obtaining the first region image, the first marked image, the second region image, and the second marked image, the first region image, the first marked image, the second region image, and the second marked image are input into a pre-trained registration model, and a registered image is obtained by registering the region image of the stent contained in the second region image through the output of the registration model. While utilizing the overall information in the region image, the information of the marked image is added, so that the registered image obtained by the registration model can be aligned with both the first region image and the first marker in the first marked image, thereby achieving accurate display of the registered image containing the stent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1A is a scene schematic diagram of a first area image of an image acquisition method provided in an embodiment of the present application;

[0019] Figure 1B is a scene schematic diagram of a second region image of an image acquisition method provided in an embodiment of the present application;

[0020] Figure 1C This is a scene diagram of a registration image of an image acquisition method provided in an embodiment of the present application.

[0021] Figure 2 It is a flowchart of an image acquisition method provided in an embodiment of the present application;

[0022] Figure 3 It is a flowchart of an image acquisition method provided in an embodiment of the present application;

[0023] Figure 4A It is a scene schematic diagram of a background removal image of an image acquisition method provided in an embodiment of the present application;

[0024] Figure 4B A schematic diagram of a DSA image scene of an image acquisition method provided in an embodiment of the present application

[0025] Figure 5A is a schematic diagram of a scene of an enhanced image of an image acquisition method provided in an embodiment of the present application;

[0026] Figure 5B is a schematic diagram of a scene of displaying an image in an image acquisition method provided in an embodiment of the present application;

[0027] Figure 6 It is a flowchart of an image acquisition method provided in an embodiment of the present application;

[0028] Figure 7 It is a structural schematic diagram of an image acquisition method and device provided in an embodiment of the present application;

[0029] Figure 8 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0031] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0032] Figure 1A It is a schematic diagram of a first region image of an image acquisition method provided in an embodiment of the present application. With the development of science and technology, medical imaging based on technologies such as computer-assisted imaging and magnetic resonance imaging has become an indispensable means of examination in the clinical diagnosis and treatment process. After a coronary stent is implanted in a patient, it is usually necessary to observe the deployment morphology of the stent. However, due to factors such as the low contrast between blood vessels and surrounding tissues and defects in the imaging process, the stent is almost invisible in the perspective image, and the deployment morphology of the stent cannot be accurately obtained.

[0033] Based on the above problems, an embodiment of the present specification provides an image acquisition method, which obtains a first perspective image, identifies a first marker in the first perspective image, and then intercepts a regional image in the first perspective image based on the first marker to obtain the first regional image, obtains a first marker image corresponding to the first marker in the first regional image, obtains a second perspective image, and obtains a second regional image of the second marker in the second perspective image, intercepts a second marker image corresponding to the second marker in the second regional image, and inputs the first regional image, the first marker image, the second regional image, and the second marker image into a pre-trained registration model, the loss function of the registration model includes a first loss function and a second loss function obtained based on the registration image, the first regional image, and the first marker image, and then obtains a registered image after registering the regional image containing the stent in the second regional image based on the output result of the registration model. In the process of obtaining the registration image through the registration model, the first region image, the second region image, the first marker image and the second marker image are used for registration at the same time. While utilizing the overall information of the marker in the region image, the information of the marker image is added. Thus, the registration image obtained by the registration model can be aligned with both the region image and the marker in the marker image, thus realizing accurate display of the registration image containing the stent. The registration result is shown in Figure 2. Figure 1C As shown, it should be noted that Figure 1A is the first region image, Figure 1B is the second region image, it can be seen Figure 1B The shape trend of the stent in Figure 1A More gentle, that is, after the alignment of this scheme Figure 1C , its shape is closer to Figure 1A .

[0034] It should be noted that the execution subject of the hardware operating environment involved in the embodiments of this specification may be an image acquisition device, which has functions such as data processing, data communication and program operation, and the image acquisition device may be a controller inside an electronic device. Of course, it may also be a module in the image acquisition device for implementing the image acquisition method, and this implementation condition does not limit this.

[0035] based on Figure 1A , 1B , 1C shows the scene, the following will be combined Figure 2-Figure 8 , the image acquisition method provided in the embodiment of the present application is introduced in detail.

[0036] See also Figure 2 , which is a flow chart of an image acquisition method provided in an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps S101-S104.

[0037] S101, acquiring a first perspective image, and identifying a first marker in the first perspective image;

[0038] During the stent implantation operation, the balloon is introduced into the blood vessel through a guide wire and placed at the site of blood vessel stenosis. After inflation, the stent is introduced into the site and adheres closely to the blood vessel wall. There are two characteristic marker balls at both ends of the balloon. The position of the stent in the fluoroscopic image (the image obtained by an X-ray detector) is determined by searching for the characteristic marker balls in the fluoroscopic image.

[0039] In one embodiment, the first perspective image may be a perspective image obtained after the stent is implanted during the treatment process; the first marker is a marker for locating the position of the stent in the first perspective image, which is specifically two characteristic marker balls and leads of the balloon in the first perspective image.

[0040] S102, intercepting a first perspective image based on a first marker to obtain a first region image, and acquiring a first marker image corresponding to the first marker in the first region image;

[0041] In one embodiment, after the first marker is determined, the first perspective image is segmented based on the first marker, and a first region image of the region where the first marker is located is captured, the first region image including the first marker and the background region of the first marker. A first marker image corresponding to the first marker is acquired in the first region image including the first marker.

[0042] S103, acquiring a second region image of the second perspective image and a second marker image corresponding to a second marker in the second region image;

[0043] Specifically, first obtain a second perspective image and identify a second marker in the second perspective image, wherein the second marker is a marker for positioning the stent in the second perspective image, and specifically is two characteristic marker balls and a lead of the balloon in the second perspective image. Then, based on the second marker, a second area image in the second perspective image is intercepted to obtain a second marker image corresponding to the second marker in the second area image.

[0044] In the embodiment of the present application, the first perspective image may be a perspective image acquired at the current moment, and the second perspective image may be a perspective image acquired before the first perspective image.

[0045] S104, input the first area image, the first marked image, the second area image and the second marked image into the registration model to obtain the registered image of the second area image and the second marked image through the deformation field output by the registration model, and the loss function of the registration model includes a first loss function and a second loss function obtained based on the registered image, the first area image, and the first marked image.

[0046] In one embodiment, after the first area image, the first marked image, the second area image and the second marked image are input into the trained registration model, the deformation field is outputted by the registration model, and the deformation field is applied to the second area image and the second marked image respectively to obtain a first predicted image of the first area image and a second predicted image of the first marked image, and then the loss between the first predicted image and the first area image is obtained to obtain a first loss function, and the loss between the second predicted image and the first marked image is obtained to obtain a second loss function, wherein the registered image includes the first predicted image and the second predicted image.

[0047] In an embodiment of the present application, after obtaining the first area image, the first marked image, the second area image, and the second marked image, the first area image, the first marked image, the second area image, and the second marked image are input into a pre-trained registration model, and a registered image is obtained by registering the area image of the stent contained in the second area image based on the output of the registration model. While utilizing the overall information in the area image, the information of the marked image is added, so that the registered image obtained by the registration model can be aligned with both the first area image and the first marker in the first marked image, thereby achieving accurate display of the registered image containing the stent.

[0048] See also Figure 3 , which is a flow chart of an image acquisition method provided in an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps S201-S203.

[0049] S201, determining an image area corresponding to a first marker;

[0050] In one embodiment, after identifying the first marker included in the first perspective image, the image area of ​​the first marker in the first perspective image is determined. Optionally, in the embodiment of the present application, after identifying the first marker, the area formed by the endpoints of the first marker is determined as the image area of ​​the first marker in the first perspective image. Specifically, in the embodiment of the present application, a recognition model can be used to identify the first marker, and the recognition model can be a target detection model (You Only Look Once, YOLO).

[0051] Furthermore, in an embodiment of the present application, before identifying the first marker in the first perspective image, in order to improve the accuracy of YOLO model recognition, the first perspective image can also be normalized to obtain a normalized image, and then the normalized image can be background removed to obtain a background removed image, and then the background removed image can be input into the recognition model to obtain the first marker in the first perspective image through the output result of the recognition model.

[0052] It is understandable that, referring to Figure 4B , Figure 4B A schematic diagram of a scene in which a background is removed from an image in the image acquisition method of the present application. Figure 4B The background removal image can be seen in the figure compared to the original DSA image ( Figure 4A ), the blood vessels are more visualized, so that the first marker can be accurately identified in the image, which improves the guarantee for the subsequent accurate display of the regional image of the stent.

[0053] S202, constructing diagonal lines of the image area, and determining the intersection of the diagonal lines as the center point of the image area;

[0054] S203: Determine that an image region corresponding to a preset pixel value with the region center point as the center is a first region image.

[0055] In one embodiment, after determining the image area of ​​the first marker in the first perspective image, diagonals of the image area are constructed, and the intersection of each diagonal line is determined as the regional center point of the image area. After obtaining the regional center point, the image area constructed within a preset pixel value with the regional center point as the center is determined as the first regional image.

[0056] Exemplarily, in an embodiment of the present application, the first markers are two characteristic marker balls and a lead of the balloon, and the corresponding image area is a rectangle, thereby determining that the intersection of the two diagonals of the image area with a rectangular shape is the center point of the image area. Then, with the center point of the area as the center and a preset pixel value of 256*256, the first perspective image is segmented to obtain a first area image with a pixel size of 256*256.

[0057] In an embodiment of the present application, the preset pixel value can also be determined based on the size of the first perspective image and / or the position of the first marker in the first perspective image, ensuring that the first area image obtained according to the preset pixel value contains the first marker. Since the installation position of the first bracket is consistent with the position of the first marker, it is ensured that a complete area image of the bracket can be obtained in the first area image.

[0058] Further, after obtaining the first region image, the first marker image corresponding to the first marker in the first region image is obtained. Specifically, in the embodiment of the present application, a lightweight segmentation network (such as LiteNeXt) can be used to obtain the first marker image corresponding to the first marker in the first region image. It can be understood that what is obtained in the first region image is an image containing the first marker and the background part of the first marker. When the first region image is segmented using the lightweight segmentation network, the first marker and its background part can be segmented from the first region image to obtain a first marker image containing only the first marker. In the embodiment of the present application, the first marker in the first perspective image is first identified by a recognition model, and then the first perspective image is divided to obtain the first region image of the area where the first marker is located, and then the first marker in the first region image is segmented by a lightweight segmentation network to obtain the first marker image. The lightweight network segments the first region image obtained by segmentation, which can reduce the segmentation area of ​​the lightweight network, improve the segmentation speed of the lightweight network, and ensure the accuracy of the segmentation.

[0059] After obtaining the first area image and the first marker image corresponding to the first marker in the first area image, obtain the second area image of the second marker in the second perspective image and the second marker image corresponding to the second marker in the second area image, and input the first area image, the second area image, the first marker image, and the second marker image into the trained registration model, and obtain the registration image through the output of the registration model. It is understandable that during the operation, the patient's position may change, or the acquired image may have position deviations due to factors such as equipment movement. These deviations can be eliminated by registration with the registration model, thereby accurately evaluating the morphology of the stent. Furthermore, in an embodiment of the present application, after obtaining the registration image through the registration model, it also includes:

[0060] S204, performing image enhancement on the registered image to obtain a plurality of enhanced images;

[0061] S205, respectively determining a plurality of pixel values ​​of a first pixel point in the plurality of enhanced images, and constructing a second pixel point corresponding to the first pixel point based on the plurality of pixel values;

[0062] S206, updating the first pixel point until a second pixel point corresponding to the first pixel point in the enhanced image is obtained;

[0063] S207: Obtain a fused image of the enhanced image based on the second pixel point, and perform denoising processing on the fused image to obtain a display image.

[0064] In one embodiment, see Figure 5A , Figure 5A The schematic diagram of the scene of the enhanced image of the image acquisition method of the present application is shown in FIG. Specifically, the USM (Unsharpen Mask) sharpening algorithm can be used to perform image enhancement processing on the registered image to obtain an enhanced image.

[0065] In an embodiment of the present application, multiple registered images can be obtained based on different first perspective images and second perspective images, and then image enhancement is performed on the multiple registered images to obtain multiple enhanced images. After the enhanced images are obtained, a first pixel is determined among the pixels contained in a certain enhanced image, and the pixel value of the first pixel is obtained. Then, the pixel corresponding to the first pixel is determined to be the same as the first pixel in the remaining enhanced images to obtain multiple first pixels, and further obtain multiple pixel values ​​of the first pixels of all enhanced images. It can be understood that the number of pixels of the multiple registered images obtained in the above manner is the same, and thus the number of pixels of the multiple enhanced images obtained after image enhancement is also the same.

[0066] Exemplarily, the multiple enhanced images include a first enhanced image and a second enhanced image. The first pixel point can be the pixel point in the first row and first column in the first enhanced image. Similarly, the pixel point corresponding to the first pixel point is the pixel point in the first row and first column in the second enhanced image.

[0067] After obtaining the pixel values ​​of the plurality of first pixel points, a second pixel point corresponding to the first pixel point is constructed based on the plurality of pixel values.

[0068] Optionally, in an embodiment of the present application, an average value of pixel values ​​of a plurality of first pixel points may be determined as the pixel value of a second pixel point, and then the second pixel point may be constructed based on the pixel value.

[0069] It is understandable that weights of pixel values ​​of different first pixel points may also be assigned through similarity detection, thereby constructing the second pixel point.

[0070] After constructing the second pixel corresponding to the current first pixel, the first pixel is updated in the registered image until a second pixel corresponding to the first pixel in the enhanced image is obtained, and a fused image of the enhanced image is obtained based on the second pixel.

[0071] Furthermore, in the embodiment of the present application, the step of performing denoising on the fused image to obtain a display image includes step S2071.

[0072] S2071: Input the fused image to an image denoising model so that the image denoising model denoises the enhanced image to obtain the displayed image.

[0073] It is understandable that there is noise in the perspective image, such as Gaussian noise, Poisson noise, etc., which will affect the clarity and contrast of the perspective. In the subsequent process of enhancing the fused image based on the perspective image, the details and noise are amplified at the same time, resulting in obvious noise in the fused image obtained after image enhancement and image fusion processing. Therefore, it is necessary to denoise the fused image.

[0074] Further, in the embodiment of the present application, after obtaining the fused image, the fused image is input into the image denoising model to perform denoising on the fused image to obtain a display image, see Figure 5B , Figure 5B A schematic diagram of a scene showing an image of the image acquisition method of the present application.

[0075] Specifically, in the embodiment of the present application, an image denoising model can be obtained through training. In the process of training the initial image denoising model using noisy training enhanced images and noise-free training display images, the initial image denoising model is trained using training enhanced images and training display images with a large number of fused frames to obtain the image denoising model.

[0076] In an embodiment of the present application, the center point of the image area is obtained by constructing a diagonal line in the image area corresponding to the first marker, and then the first area image containing the first marker is obtained according to the preset pixel value with the center point as the center. Since the installation position of the bracket is consistent with the position of the first marker, the first area image is obtained with the center point of the area image corresponding to the first marker, which can ensure that a complete area image containing the bracket is obtained in the first area image, and can ensure that the display image containing the bracket can be accurately displayed subsequently; and a fused image is obtained by fusing multiple enhanced images. By fusing the enhanced images, the error caused by a single image is avoided, and it can be ensured that the display image containing the bracket can be accurately displayed subsequently; and the fused image is input into the image enhancement model to enhance the image to obtain an enhanced image, and then the enhanced image is input into the image denoising model for denoising to obtain the display image, thereby avoiding the influence of noise, thereby being able to clearly display the bracket in the display image, and improving the visualization effect of the area image of the bracket in the display image.

[0077] See also Figure 6, which is a flow chart of an image acquisition method according to an embodiment of the present application. As shown in FIG4 , the method according to an embodiment of the present application may include the following steps S301 to S303.

[0078] S301, acquiring a sample image, where the sample image includes a third region image and a third marked image corresponding to a third marker in the third region image, a fourth region image and a fourth marked image corresponding to a fourth marker in the fourth region image;

[0079] S302, inputting the third region image, the fourth region image, the third marked image and the fourth marked image into an initial registration model to obtain a deformation field between the third region image, the fourth region image, the third marked image and the fourth marked image;

[0080] S303, constructing a loss function based on the deformation field, the third region image, the fourth region image, the third labeled image and the fourth labeled image, and optimizing the initial registration model based on the loss function until a convergence condition is met to obtain a registration model.

[0081] In one embodiment, a third area image, a third marked image, a fourth area image and a fourth marked image are obtained and input into an initial registration model. The initial registration model outputs a deformation field, and a third predicted image obtained after the fourth area image is registered with the deformation field and a fourth predicted image obtained after the fourth marked image is registered with the deformation field are calculated. Then, a third loss function between the third predicted image and the third area image and a fourth loss function between the fourth predicted image and the third marked image are calculated. The third loss function and the fourth loss function are added to obtain a total loss function. The initial registration model is back-propagated, and the parameters of the initial registration model are updated until the convergence conditions are met. The final registration model is obtained by iterative training, wherein the third predicted image and the fourth predicted image are registration images obtained based on the deformation field.

[0082] In the embodiment of the present application, the total loss function can be expressed as in represents the deformation field, I 0 represents the third labeled image, I 1 represents the third region image, The fourth labeled image is deformed by the deformation field The fourth predicted image generated after registration, Represents the deformation field of the fourth region image The third predicted image generated after registration, θ represents the network parameters, α and β are the hyperparameters that balance the two loss functions.

[0083] In an embodiment of the present application, when training the registration model, the third region image, the third marker image, the fourth region image and the fourth marker image are simultaneously included and input into the initial registration model, and the loss function of the initial registration model is determined based on the region image including the marker image and the marker image. While utilizing the overall information of the marker in the region image, the information of the marker image is added, so that the registration image output by the registration model can be aligned with both the region image and the marker of the marker image, thereby realizing a registration model that can be trained to accurately display the registration image including the stent.

[0084] based on Figure 1A , 1B , 1C scenario diagram, the following will be combined Figure 7 , the image acquisition device provided in the embodiment of the present application is introduced in detail. It should be noted that, Figure 7 The image acquisition device is used to execute the present application Figure 2-Figure 6 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2-Figure 6 Specifically, the image acquisition device includes: a first acquisition unit 11, a second acquisition unit 12, a third acquisition unit 13, and an input unit 14.

[0085] A first acquisition unit 11 is used to acquire a first perspective image and identify a first marker in the first perspective image;

[0086] The second acquisition unit 12 is used to intercept the first perspective image based on the first marker to obtain the first area image, and obtain the first marker image corresponding to the first marker in the first area image;

[0087] A third acquisition unit 13 is used to acquire a second region image of the second perspective image and a second marker image corresponding to a second marker in the second region image;

[0088] The input unit 14 is used to input the first area image, the first marked image, the second area image and the second marked image into the registration model to obtain the registered image of the second area image and the second marked image through the deformation field output by the registration model. The loss function of the registration model includes a first loss function and a second loss function obtained based on the registration image, the first area image, and the first marked image.

[0089] Optionally, the second acquiring unit 12 includes: a first determining subunit 121 , a constructing subunit 122 , and a second determining subunit 123 .

[0090] A first determining subunit 121, configured to determine an image region corresponding to a first marker;

[0091] A construction subunit 122, used to construct diagonal lines of the image area, and determine the intersection of the diagonal lines as the area center point of the image area;

[0092] The second determining subunit 123 is used to determine that the image area corresponding to the preset pixel value with the area center point as the center is the first area image.

[0093] Optionally, the input unit 14 further includes an enhancement subunit 141 , a third determination subunit 142 , an updating subunit 143 , and a processing subunit 143 .

[0094] An enhancement subunit 141, configured to perform image enhancement on the registered image to obtain a plurality of enhanced images;

[0095] A third determining subunit 142 is used to respectively determine a plurality of pixel values ​​of a first pixel point in the plurality of enhanced images, and construct a second pixel point corresponding to the first pixel point based on the plurality of pixel values;

[0096] An updating subunit 143, configured to update the first pixel point until a second pixel point corresponding to the first pixel point in the enhanced image is obtained;

[0097] The processing subunit 143 is used to obtain a fused image of the enhanced image based on the second pixel points, and perform denoising on the fused image to obtain a display image.

[0098] Optionally, the processing subunit 143 is specifically configured to:

[0099] The fused image is input into an image denoising model so that the image denoising model denoises the enhanced image to obtain the displayed image.

[0100] Optionally, the processing subunit 143 is specifically configured to:

[0101] An average value of the plurality of pixel values ​​is determined, and the second pixel point is constructed based on the average value.

[0102] Optionally, the input unit 14 further includes: an acquisition subunit 144 , an input subunit 145 , and a construction subunit 146 .

[0103] an acquisition subunit 144, configured to acquire a sample image, the sample image comprising a third region image and a third labeled image corresponding to a third marker in the third region image, a fourth region image and a fourth labeled image corresponding to a fourth marker in the fourth region image;

[0104] An input subunit 145, configured to input the third region image, the fourth region image, the third marked image and the fourth marked image into the initial registration model to obtain a deformation field between the third region image, the fourth region image, the third marked image and the fourth marked image;

[0105] The construction subunit 146 is used to construct a loss function based on the deformation field, the third region image, the fourth region image, the third labeled image and the fourth labeled image, and optimize the initial registration model based on the loss function until convergence to obtain a registration model.

[0106] Optionally, the first acquisition unit includes: a first processing subunit 111 , a second processing subunit 112 , and an input subunit 113 .

[0107] A first processing subunit 111 is used to perform normalization processing on the first perspective image to obtain a normalized image;

[0108] The second processing subunit 112 is used to perform background removal processing on the normalized image to obtain a background removed image;

[0109] The input subunit 113 is used to input the background-removed image into the recognition model, so as to obtain the first marker in the first perspective image through the output result of the recognition model.

[0110] In an embodiment of the present application, after obtaining the first area image, the first marked image, the second area image, and the second marked image, the first area image, the first marked image, the second area image, and the second marked image are input into a pre-trained registration model, and a registered image is obtained by registering the area image of the stent contained in the second area image through the output of the registration model. While utilizing the overall information in the area image, the information of the marked image is added, so that the registered image obtained by the registration model can be aligned with both the first area image and the first marked object image in the first marked image, thereby achieving accurate display of the registered image containing the stent.

[0111] See also Figure 8 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device 500 includes a processor 501 and a memory 502. The processor 501 is electrically connected to the memory 502.

[0112] The processor 501 is the control center of the electronic device 500 and may include one or more processing cores. The processor 501 uses various interfaces and lines to connect the various parts of the entire electronic device 500, and executes various functions and processes data of the electronic device 500 by running or calling the computer program stored in the memory 502, and calling the data stored in the memory 502, so as to control the electronic device 500 as a whole. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 501 can integrate one or a combination of CPU, denoising processor (Graphics Processing Unit, GPU) and modem. Among them, the CPU mainly processes the operating system, user pages and applications, etc.; the GPU is responsible for rendering and drawing the display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 501, and it can be implemented separately through a communication chip.

[0113] The memory 502 can be used to store software programs and modules, and the processor 501 executes various functional applications and data processing by running the computer programs and modules stored in the memory 502. The memory 502 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, a computer program required for at least one function, etc.; the data storage area can store data created according to the use of the electronic device 500, etc.

[0114] In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0115] In this embodiment, the processor 501 in the electronic device 500 loads instructions corresponding to the processes of one or more computer programs into the memory 502 according to the following steps, and the processor 501 runs the computer program stored in the memory 502 to implement various functions, as follows:

[0116] Acquire a first perspective image, and identify a first marker in the first perspective image;

[0117] Based on the first marker, a first perspective image is intercepted to obtain a first area image, and a first marker image corresponding to the first marker in the first area image is acquired;

[0118] Acquire a second region image of the second marker in the second perspective image and a second marker image corresponding to the second marker in the second region image;

[0119] The first area image, the first marked image, the second area image and the second marked image are input into the registration model to obtain the registered image of the second area image and the second marked image through the deformation field output by the registration model. The loss function of the registration model includes a first loss function and a second loss function obtained based on the registration image, the first area image, and the first marked image.

[0120] Optionally, when the processor 501 intercepts the first perspective image based on the first marker to obtain the first regional image, the processor 501 specifically executes:

[0121] determining an image region corresponding to the first marker;

[0122] Construct the diagonal lines of the image area, and determine the intersection of the diagonal lines as the center point of the image area;

[0123] An image region corresponding to a preset pixel value with the region center point as the center is determined as a first region image.

[0124] Optionally, after inputting the first region image, the first marked image, the second region image, and the second marked image into the registration model, the processor 501 further executes:

[0125] Performing image enhancement on the registered image to obtain a plurality of enhanced images;

[0126] Determine a plurality of pixel values ​​of a first pixel point in the plurality of enhanced images respectively, and construct a second pixel point corresponding to the first pixel point based on the plurality of pixel values;

[0127] Updating the first pixel point until a second pixel point corresponding to the first pixel point in the enhanced image is obtained;

[0128] A fused image of the enhanced image is obtained based on the second pixel point, and denoising is performed on the fused image to obtain a display image.

[0129] Optionally, when the processor 501 performs denoising processing on the fused image to obtain a display image, the processor 501 specifically performs:

[0130] The fused image is input into an image denoising model so that the image denoising model denoises the enhanced image to obtain the displayed image.

[0131] Optionally, when constructing a second pixel point corresponding to the first pixel point based on the multiple pixel values, the processor 501 specifically performs:

[0132] Determine an average value of the plurality of pixel values, and construct the second pixel point based on the average value. Optionally, before inputting the first region image, the first marked image, the second region image, and the second marked image into the registration model, the processor 501 may further perform:

[0133] Acquire a sample image, where the sample image includes a third region image and a third marked image corresponding to a third marker in the third region image, a fourth region image and a fourth marked image corresponding to a fourth marker in the fourth region image;

[0134] Inputting the third region image, the fourth region image, the third marked image and the fourth marked image into the initial registration model to obtain a deformation field between the third region image, the fourth region image, the third marked image and the fourth marked image;

[0135] A loss function is constructed based on the deformation field, the third region image, the fourth region image, the third labeled image and the fourth labeled image, and the initial registration model is optimized based on the loss function until convergence to obtain a registration model.

[0136] Optionally, when the processor 501 acquires the first perspective image and identifies the first marker in the first perspective image, the processor 501 specifically executes:

[0137] Normalizing the first perspective image to obtain a normalized image;

[0138] Perform background removal processing on the normalized image to obtain a background removed image;

[0139] The background-removed image is input into the recognition model to obtain the first marker in the first perspective image through the output result of the recognition model.

[0140] In an embodiment of the present application, after obtaining the first region image, the first marked image, the second region image, and the second marked image, the first region image, the first marked image, the second region image, and the second marked image are input into a pre-trained registration model, and a registered image is obtained by registering the region image of the stent contained in the second region image through the output of the registration model. While utilizing the overall information in the region image, the information of the marked image is added, so that the registered image obtained by the registration model can be aligned with both the first region image and the first marker in the first marked image, thereby achieving accurate display of the registered image containing the stent.

[0141] It should be understood that the device provided in the embodiment of the present application is used to execute the above-mentioned image acquisition method, and thus can achieve the same effect as the above-mentioned implementation method.

[0142] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to an electronic device, the processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device in executing related program codes, etc.

[0143] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0144] In addition, the device provided in the embodiment of the present application can specifically be a chip, component or module, and the chip may include a connected processor and memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute an image acquisition method provided in the above embodiment.

[0145] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement an image acquisition method provided in the above-mentioned embodiment.

[0146] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement an image acquisition method provided by the above-mentioned embodiment.

[0147] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0148] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0149] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0150] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image acquisition method, characterized in that: The method comprises: Acquire a first perspective image, and identify a first marker in the first perspective image; intercepting the first perspective image based on the first marker to obtain a first region image, and acquiring a first marker image corresponding to the first marker in the first region image; Acquire a second region image of the second marker in the second perspective image and a second marker image corresponding to the second marker in the second region image; The first area image, the first marked image, the second area image and the second marked image are input into a registration model to obtain a registered image of the second area image and the second marked image through the deformation field output by the registration model, and the loss function of the registration model includes a first loss function and a second loss function obtained based on the registration image, the first area image, and the first marked image.

2. The method according to claim 1, characterized in that The step of intercepting the first perspective image based on the first marker to obtain a first regional image includes: determining an image region corresponding to the first marker; Constructing diagonal lines of the image area, and determining the intersection of the diagonal lines as the area center point of the image area; An image area corresponding to a preset pixel value with the center point of the area as the center is determined as the first area image.

3. The method according to any one of claims 1 or 2, characterized in that: After inputting the first region image, the first marked image, the second region image and the second marked image into the registration model, the method further includes: Performing image enhancement on the registered image to obtain a plurality of enhanced images; Determine a plurality of pixel values ​​of a first pixel point in the plurality of enhanced images respectively, and construct a second pixel point corresponding to the first pixel point based on the plurality of pixel values; Updating the first pixel point until a second pixel point corresponding to the first pixel point in the enhanced image is obtained; A fused image of the enhanced image is obtained based on the second pixel point, and denoising is performed on the fused image to obtain a display image.

4. The method according to claim 3, characterized in that The denoising process is performed on the fused image to obtain a display image, including: The fused image is input into an image denoising model so that the image denoising model denoises the enhanced image to obtain the displayed image.

5. The method according to claim 3, characterized in that: The constructing a second pixel point corresponding to the first pixel point based on the plurality of pixel values ​​comprises: An average value of the plurality of pixel values ​​is determined, and the second pixel point is constructed based on the average value.

6. The method according to claim 1, characterized in that Before inputting the first region image, the first marked image, the second region image and the second marked image into the registration model, the method further includes: Acquire a sample image, wherein the sample image includes a third region image and a third marked image corresponding to a third marker in the third region image, a fourth region image and a fourth marked image corresponding to a fourth marker in the fourth region image; Inputting the third region image, the fourth region image, the third marker image and the fourth marker image into an initial registration model to obtain a deformation field between the third region image, the fourth region image, the third marker image and the fourth marker image; A loss function is constructed based on the deformation field, the third region image, the fourth region image, the third marked image and the fourth marked image, and the initial registration model is optimized based on the loss function until convergence to obtain the registration model.

7. The method according to claim 1, characterized in that The acquiring a first perspective image and identifying a first marker in the first perspective image includes: performing normalization processing on the first perspective image to obtain a normalized image; Performing background removal processing on the normalized image to obtain a background removed image; The background-removed image is input into a recognition model to obtain a first marker in the first perspective image through an output result of the recognition model.

8. An image acquisition device, characterized in that: The device comprises: A first acquisition unit, configured to acquire a first perspective image and identify a first marker in the first perspective image; a second acquisition unit, configured to intercept the first perspective image based on the first marker to obtain a first region image, and acquire a first marker image corresponding to the first marker in the first region image; a third acquisition unit, configured to acquire a second region image of the second perspective image and a second marker image corresponding to a second marker in the second region image; An input unit is used to input the first area image, the first marked image, the second area image and the second marked image into a registration model to obtain a registered image of the second area image and the second marked image through the deformation field output by the registration model, and the loss function of the registration model includes a first loss function and a second loss function obtained based on the registration image, the first area image, and the first marked image.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program code, and when the computer program code is executed, the method according to any one of claims 1 to 7 is implemented.

Images