Image splicing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image splicing method and device, electronic equipment and a storage medium, and the method comprises the steps: photographing a first X-ray image and a second X-ray image which are different and have an overlapping region, calculating the relative translation amount of the second X-ray image and the first X-ray image, and splicing the first X-ray image and the second X-ray image according to the relative translation amount. According to the image splicing method and device, the image splicing can be accurately and automatically completed, the visual field of the C-shaped arm can be increased under the condition that the C-shaped arm is not moved, the operation success rate and the operation efficiency of a doctor in an operation can be improved, the human input caused by manual image splicing in the prior art is saved, the problems of low splicing accuracy and low efficiency caused by manual splicing are solved, and the operation efficiency of the doctor is improved. And the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image stitching method and device, an electronic device and a storage medium. Background Art

[0002] Currently, X-ray splicing is widely used in DR equipment (DR, direct digital X-ray photography system).

[0003] In the medical field, X-ray machines, such as X-ray machines, are often used to image the area of ​​interest of a patient, and doctors then diagnose and treat the patient based on the images obtained. Fig.11 As shown, an X-ray machine is illustrated, whose main components include an X-ray tube 1, an X-ray beam limiter 2, a patient fixer 3, and a detector 4, wherein the main function of the X-ray tube 1 is to emit X-rays, the main function of the X-ray beam limiter 2 is to limit the irradiation range of the light field of the X-rays emitted by the X-ray tube 1, the function of the detector 4 is to receive X-rays and form an image which is then transmitted to a workstation for further processing, and the role of the patient fixer 3 is to have two points, the first point is to isolate the patient from the detector 4 for safety reasons, and the second point is to fix the patient so as to minimize movement during the entire shooting process.

[0004] It can be seen from the X-ray machine that the tube 1 emits X-rays that pass through the region of interest and then reach the X-ray detector 4 to obtain an image of the region of interest. The size of the obtained image is generally equal to the size of the X-ray detector. If the field of view of the region of interest is within the size of the X-ray detector, the entire region of interest can be fully presented in one image. For example, the field of view of regions of interest such as the heart and lungs is within the size of the X-ray detector, so regions of interest such as the heart and lungs can be fully presented in one picture. However, there are some regions of interest whose field of view exceeds the size of the X-ray detector, such as the spine and thighs, and the entire region of interest cannot be presented in one picture. This requires taking several pictures and then stitching these pictures together to form a complete image that can present the entire region of interest.

[0005] At present, there are several methods to image the region of interest whose field of view exceeds the size of the X-ray detector. Among them, there are two main categories. One is the angle acquisition method, which takes multiple pictures of the region of interest by rotating the tube, that is, changing the angle of the tube. In other words, when the tube is at a certain angle, take a picture of the region of interest, then rotate the tube to another angle and take another picture of the region of interest, and so on, until the region of interest is completely included in all pictures, and finally all the pictures taken are stitched together to form an image of the region of interest.

[0006] Another method is to use the method of parallel movement of the tube and the X-ray detector, that is, when the tube and the X-ray detector are in the first position, take a picture, then move the tube and the X-ray detector to the second position at the same time, take another picture, and so on, move the tube and the X-ray detector in parallel until the end of the region of interest, and finally stitch the taken pictures together to form an image of the region of interest. This image stitching method uses manual parallel movement of the tube and the X-ray detector, that is, after each picture is taken, the operator manually moves the tube and the X-ray detector to the next position in parallel according to his experience. Due to manual operation, the work efficiency is low, and due to the different experiences of different operators, the final image of the region of interest is often inaccurate.

[0007] Mobile C-arm X-ray machines used in surgery are usually pushed or pulled manually to stitch X-ray images. This results in an unknown amount of movement offset, which makes image stitching extremely difficult. In the past, image stitching of C-arms was generally done by manually moving images using software, which was time-consuming and labor-intensive. Summary of the invention

[0008] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide an image stitching method and device, an electronic device and a storage medium to solve the problem in the related art that the X-ray machine is usually pushed or pulled manually, resulting in an unknown movement offset and time-consuming and labor-intensive stitching of X-ray images.

[0009] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0010] According to a first aspect of the present invention, there is provided an image stitching method, comprising:

[0011] Step S1, obtaining a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken around the part of a human body to be detected;

[0012] Step S2, calculating the relative translation amount between the second X-ray image and the first X-ray image;

[0013] Step S3, stitching the first X-ray image and the second X-ray image according to the relative translation amount and the stitching direction selected by the user;

[0014] Step S4: if the stitching result meets the user's requirements, take the next X-ray image, and use the second X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1;

[0015] Step S5, repeating the above steps S1 to S4 until all X-ray images of the body part to be detected are stitched together;

[0016] Step S6: Render the stitched long image in blocks and perform image fusion.

[0017] According to a second aspect of the present invention, there is provided an image stitching device, comprising:

[0018] An acquisition module, used for acquiring a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken around the corner of the part to be detected;

[0019] A calculation module, used for calculating the relative translation amount between the second X-ray image and the first X-ray image;

[0020] a stitching module, configured to stitch the first X-ray image and the second X-ray image according to the relative translation amount and a stitching direction selected by a user;

[0021] It is also used to take the next X-ray image if the stitching result meets the user's needs, and use the second X-ray image obtained by the stitching module as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to the acquisition module until all X-ray images of the body part to be detected are stitched;

[0022] The fusion module is used to render the stitched long image in blocks and perform image fusion.

[0023] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0024] A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0025] Memory, used to store computer programs;

[0026] The processor is used to implement the above method when executing the program stored in the memory.

[0027] According to a fourth aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the above method.

[0028] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:

[0029] By taking different first X-ray images and second X-ray images with overlapping areas, calculating the relative translation between the second X-ray image and the first X-ray image, and splicing the first X-ray image and the second X-ray image according to the relative translation, the image stitching can be accurately and automatically completed, and the field of view of the C-arm can be increased without moving the C-arm, which helps to improve the doctor's intraoperative operation success rate and surgical efficiency, saves the manpower investment caused by manual image stitching in the prior art, avoids the problems of low stitching accuracy and low efficiency caused by manual stitching, and improves the user experience.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flow chart of an image stitching method according to an exemplary embodiment;

[0032] Figure 2 is a schematic diagram showing an overlapping area of ​​a first X-ray image and a second X-ray image according to an exemplary embodiment;

[0033] Figure 3 is a schematic diagram showing a result of stitching a first X-ray image and a second X-ray image according to an exemplary embodiment;

[0034] Figure 4A to Figure 4B is a long image after splicing is completed according to an exemplary embodiment;

[0035] Figure 5 is a flow chart of an image stitching method according to an exemplary embodiment;

[0036] Figure 6 is a schematic diagram of a similarity matrix calculation method according to an exemplary embodiment;

[0037] Figure 7 is a flow chart of an image fusion method according to an exemplary embodiment;

[0038] Figure 8 is a schematic diagram showing correction of a second cutting image according to an exemplary embodiment;

[0039] Fig. 9 is a schematic diagram showing block-by-block rendering of a long image after splicing according to an exemplary embodiment;

[0040] Figures 10A to 10C is a schematic diagram of an algorithm for fusing the area to be fused in the spliced ​​long image according to an exemplary embodiment;

[0041] Fig.11It is a structural schematic diagram of an X-ray machine shown according to the background technology;

[0042] Fig.12 is a schematic diagram of corner surround shooting according to an exemplary embodiment;

[0043] Figures 13A to 13B is a schematic diagram showing projecting a second X-ray image to a viewing angle of 0° according to an exemplary embodiment;

[0044] Fig.14 is a schematic block diagram of an image stitching device according to an exemplary embodiment;

[0045] Fig.15 It is a schematic block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] As described in the previous background technology, there is a problem in the related art that the X-ray machine is usually pushed or pulled manually, resulting in an unknown movement offset and time-consuming and labor-intensive stitching of X-ray images.

[0048] In order to effectively solve the problems in the related art, the present invention provides an image stitching method and device, an electronic device and a storage medium, which are described in detail below.

[0049] Embodiment 1

[0050] Figure 1 is a flow chart of an image stitching method according to an exemplary embodiment. Figure 1 As shown, the method comprises:

[0051] Step S1, obtaining a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken around the part of a human body to be detected;

[0052] Step S2, calculating the relative translation amount between the second X-ray image and the first X-ray image;

[0053] Step S3, stitching the first X-ray image and the second X-ray image according to the relative translation amount and the stitching direction selected by the user;

[0054] Step S4: if the stitching result meets the user's requirements, take the next X-ray image, and use the second X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1;

[0055] Step S5, repeating the above steps S1 to S4 until all X-ray images of the body part to be detected are stitched together;

[0056] Step S6: Render the stitched long image in blocks and perform image fusion.

[0057] It should be noted that, in specific practice, the technical solution provided in this embodiment is loaded into the controller of a C-arm X-ray device or a DR device for operation.

[0058] Taking the operation of a C-arm X-ray device as an example, in step S1, the first X-ray image and the second X-ray image to be stitched are obtained. In specific practice, they can be obtained by the following methods, including:

[0059] Step S11: The user uses a mobile C-arm X-ray machine to select a multiple X-ray scanning stitching protocol (the X-ray scanning stitching protocol represents a scanning method with a stitching process, and the protocol supports the image stitching method described in this embodiment).

[0060] Step S12, after entering the protocol, the user selects the splicing direction (for example, splicing from top to bottom, or from left to right, or from bottom to top, or from right to left, etc.; for another example, the spine can be from top to bottom, or from bottom to top; the pelvis can be from left to right, or from right to left).

[0061] Step S13, operate the C-arm X-ray machine, position it correctly, generally, try to make the human body close to the detector direction, and take the first X-ray image. At this time, the system will display the stitching queue. It should be noted that the first X-ray image and the second X-ray image mentioned in this embodiment refer to the previous X-ray image and the next X-ray image of the two consecutively taken X-ray images, and do not specifically refer to the first X-ray image and the second X-ray image.

[0062] Step S14: Rotate the C-arm at a certain angle (for example, rotate 17.5 degrees or 35 degrees to the left) to capture a second X-ray image. At this time, the captured second X-ray image needs to be overlapped with the first X-ray image at a preset overlap ratio (the preset overlap ratio can be set to 20%, 30%, 40%, 50%, etc., preferably 50%, according to user needs). Figure 2 As shown in the figure, the box area is the overlapping area of ​​the two X-ray images. Figure 2 The splicing result is as follows Figure 3 shown.

[0063] It can be understood that the technical solution provided in this embodiment, by shooting different first X-ray images and second X-ray images with overlapping areas, calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the stitching of images can be completed accurately and automatically, and the field of view of the C-arm can be increased without moving the C-arm, which helps to improve the doctor's intraoperative operation success rate and surgical efficiency, saves the manpower investment caused by manual image stitching in the prior art, avoids the problems of low stitching accuracy and low efficiency caused by manual stitching, and improves the user experience.

[0064] In specific practice, the step S4 further includes:

[0065] If the stitching result does not meet the user's requirements, the second X-ray image is retaken, and the process returns to step S1 to stitch the retaken second X-ray image with the first X-ray image.

[0066] The “does not meet user needs” refers to that the image resolution of the overlapping area of ​​the first X-ray image and the second X-ray image after stitching, or the size of the overlapping area does not meet the stitching requirements.

[0067] In this case, the user deletes the second X-ray image on the interface and imports the re-taken second X-ray image into the stitching queue; the user can also complete the queue by manual stitching. If the stitching result meets the user's needs, repeat the above shooting until the shooting length meets the clinical requirements. Click End Collection, the system completes all stitching and outputs a long image that meets the requirements. Assume that Figure 4A As shown, X-ray images are taken at five angles: -35° / -17.5° / 0 / 17.5° / 35° from left to right. The stitching result is as follows Figure 4B shown.

[0068] It is understandable that the technical solution provided in this embodiment first splices the first X-ray image and the second X-ray image, and only when the splicing result meets the user's needs will the next image be spliced, so that the quality and shooting angle of each sub-image in the final spliced ​​long image can meet the user's needs, avoiding rework and low efficiency caused by subsequent changes. At the same time, it can also reduce the number of shooting angle changes of the X-ray equipment and reduce the wear and tear of the X-ray equipment.

[0069] Preferably, see Figure 5 , the step S2 comprises:

[0070] Step S21: Cut the first X-ray image and the second X-ray image at four sides according to a preset cropping range to obtain a first cut image I 1 and the second cut image I 2 ;

[0071] Assume that for the first X-ray image, the cropping amplitudes for top, bottom, left, and right are c 1 、c 2 、c 3 、c 4 ; The initial size of the first X-ray image is w 0 ×h 0 , then the image size after cropping is w 1 ×h1, then:

[0072] w 1 =w 0 -c 3 -c 4

[0073] h 1 =h 0 -c 1 -c 2

[0074] c 1 、c 2 、c 3 、c 4 It can be set according to user needs, for example, 1 to c 4 Take 50 for both.

[0075] Step S22: According to the splicing direction selected by the user, the first cut image I 1 Select the region of interest (ROI);

[0076] Step S23, traverse the second cutting image I 2 All possible regions, calculating the similarity between all possible regions and the ROI to obtain a similarity matrix;

[0077] Step S24: determine the possible region corresponding to the maximum similarity in the similarity matrix, and determine the relative translation amount between the possible region and the ROI as the second cut image I 2 With the first cut image I 1 The relative translation amount is calculated and the maximum similarity is outputted (usually greater than 0.6 is better).

[0078] In specific practice, the step S22, the first cutting image I 1 Select the region of interest (ROI), including:

[0079] Get the height and width of the ROI determined by the user;

[0080] According to the splicing direction selected by the user, determine the first cutting image I 1The edge to be spliced ​​with the second cut image is taken as a reference, the first preset height from the edge is the bottom edge of the ROI, and the first preset height from the edge plus the height of the ROI is the top edge of the ROI;

[0081] The area enclosed by the bottom edge of the ROI and the top edge of the ROI is extended left and right by the width of the ROI in the middle to obtain the region of interest ROI.

[0082] It should be noted that the first preset height is set according to user needs, or according to experimental data, or according to historical experience values. For example:

[0083] Set the height of ROI to I 1 1 / 4 of the height; in principle, assuming that the stitching direction selected by the user is from top to bottom, the lower the ROI is, the better. Excluding the dead area that may not be removed, select the ROI distance I 1 The bottom 1 / 16. Therefore, the top of the ROI is at I 1 11 / 16 of

[0084] Set the width of ROI. Considering the lateral movement problem caused by manual movement of the C-arm, the width of ROI is set to I. 1 3 / 4 of the ROI and centered: That is, the starting positions of the left and right sides of the ROI are: I 1 1 / 8 and 7 / 8;

[0085] It should be noted that the biggest difference between the splicing of mobile C-arm X-ray equipment and DR equipment is that the movement of DR equipment is controlled by the system mechanical device, but the C-arm is manually operated;

[0086] Although DR equipment has SOP (Standard Operating Procedure) to guide user operation, it is still affected by human factors, and inevitably has large deviations in the up, down, left, and right directions. In order to mitigate the above impact, this embodiment adopts the above scheme to select ROI to reduce the impact of human factors on image stitching.

[0087] In specific practice, Figure 6 As shown, step S23, traverse all possible regions of the second cut image, calculate the similarity between all possible regions and the ROI, and obtain a similarity matrix, which is specifically:

[0088] From the second cut image I 2 From the upper left corner of the image to the lower right corner of the second cut image, select an area R of the same size as the ROI 2 , calculate each region R 2The similarity with the ROI is obtained to obtain a similarity matrix M.

[0089] Assume that the initial size of the first X-ray image is w 0 ×h 0 , the image size after cropping is w 1 ×h 1 , assuming that the ROI area is R1 and the size is w R ×h R ;

[0090] Calculate R using the following formula: 2 and R 1 Similarities include:

[0091] S=S 1 +S 2

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] Among them, R 1 (x,y) is the image R 1 The pixel value at (x,y), R 2 (x,y) is the image R 2 The pixel value at (x,y), R′ 1 (x, y) is the relative pixel value of image R1 at (x, y), R′ 2 (x,y) is the relative pixel value of image R2 at (x,y), is the average pixel value of image R1, is the average pixel value of image R2. The relative pixel value is relative to the average pixel value in each region.

[0099] It can be understood that the size of the similarity matrix M is (w 1 -w R )×(h 1 -h R ), each element in the similarity matrix M corresponds to the similarity at a translation position.

[0100] See also Figure 7The step S5, rendering the stitched long image in blocks and performing image fusion, includes:

[0101] Step S51: read the relative translation amount (dx) between the second cutting image and the first cutting image. 1 ,dy 1 ), and the fusion area ratio set by the user (e.g. 10%);

[0102] Step S52: Calculate the second cut image I according to the relative translation amount and the fusion area ratio. 2 With the first cut image I 1 Correction size of overlap area (such as Figure 8 As shown), and the second cutting image I is cut according to the modified size 2 , get the third cutting image I 3 ;

[0103] Step S53: Based on the third cutting image I 3 With the first cut image I 1 The relative translation (dx 2 ,dy 2 ), dx1=dx 2 , calculate the size of the long image after splicing, wherein the overlapping area of ​​the third cut image and the first cut image is the area to be fused;

[0104] Step S54: if the spliced ​​long image (size is denoted as (w l ,h l )) is smaller than the long image size required by clinical practice, the third cutting image obtained in step S53 is used as the first cutting image, the cutting image spliced ​​behind the third cutting image is used as the second cutting image, and the process returns to step S51;

[0105] Step S55, repeating the above steps S51 to S54 until the size of the rejoined long image meets the clinical requirement;

[0106] Step S56: Render the reassembled long image in blocks, where the rendered blocks include the area that needs to be merged (such as Fig. 9 As shown in the figure); adopt different fusion strategies (including but not limited to: copy, fill and merge) for each block, and return the fused long image splicing result.

[0107] Among them, for the area to be fused, the pixel value of each pixel is replaced by the weighted average of the overlapping pixels of the two previous and next cut images to be fused.

[0108] like Figures 10A to 10C As shown, it can be considered that when images are fused, the value of a pixel is the weighted average of the overlapping areas of the upper and lower images, as shown in the following formula:

[0109] p(x,y)=w 1 *p 1 (x,y)+w 2 *p 2 (x,y)

[0110] Among them, p i (x, y) is the pixel value of point (x, y) on the i-th image (i = 1 or 2), p 1 (x, y) is the pixel value at point (x, y) on the first cut image, p 2 (x, y) is the third cut image with p 1 (x,y) The pixel value of the overlapping pixel. 1 and w 2 is the corresponding weight, obtained by the following formula:

[0111] w i =d i / d S

[0112]

[0113]

[0114] Among them, d i_j Represents the distance of a pixel position from the jth edge on the i-th image, d i is the minimum distance from the edge of the i-th image at the pixel position, d s is the minimum distance d of all overlapping images at that position i The sum of the two images, usually there are two images that overlap at this position, that is, d S =d 1 +d 2 .

[0115] In summary, the technical solution provided in this embodiment calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the image stitching can be completed accurately and automatically, saving the manpower investment caused by manual image stitching in the prior art, avoiding the problems of low stitching accuracy and low efficiency caused by manual stitching, and improving the user experience.

[0116] When using a C-arm X-ray machine during surgery, it is limited by its own field of view and cannot fully observe the entire space required. By taking multiple X-ray images with overlapping areas and automatically stitching the X-ray images, the C-arm's field of view is increased, which helps improve the doctor's success rate and efficiency during surgery.

[0117] Embodiment 2

[0118] According to another exemplary embodiment, an image stitching method is shown, the method comprising:

[0119] Step S1, obtaining a first X-ray image and a second X-ray image to be stitched, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, taken around a corner facing the part to be detected by the human body (a schematic diagram of the corner view shooting is shown in FIG. Fig.12 As shown, Fig.12 The scanning angles in are: -35° / 0° / 35°);

[0120] Step S20, projecting the second X-ray image to a 0° viewing angle to obtain a third X-ray image;

[0121] Step S21, cutting the first X-ray image and the third X-ray image around according to a preset cropping range to obtain a first cut image and a second cut image;

[0122] Step S22: selecting a region of interest (ROI) for the first cut image according to the stitching direction selected by the user;

[0123] Step S23, traversing all possible regions of the second cut image, calculating the similarity between all possible regions and the ROI, and obtaining a similarity matrix;

[0124] Step S24, determining a possible region corresponding to the maximum similarity in the similarity matrix, determining a relative translation amount between the possible region and the ROI as a relative translation amount between the second cut image and the first cut image, and outputting the maximum similarity at the same time;

[0125] Step S3, stitching the projection image of the first X-ray image and the second X-ray image at a 0° viewing angle to form a third X-ray image according to the relative translation amount and the stitching direction selected by the user;

[0126] Step S4: if the stitching result meets the user's requirements, take the next X-ray image, and use the third X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1;

[0127] Step S5, repeating the above steps S1 to S4 until all X-ray images of the body part to be detected are stitched together;

[0128] Step S6: Render the stitched long image in blocks and perform image fusion.

[0129] It should be noted that, in specific practice, the technical solution provided in this embodiment is loaded into the controller of a C-arm X-ray device or a DR device for operation.

[0130] In specific practice, the step S1 includes:

[0131] The line connecting the X-ray tube and the center point of the part of the human body to be detected is determined as the center line of the 0° viewing angle;

[0132] The left and right sides of the center line of the 0° viewing angle are incremented by a preset angle, and each time the preset angle is incremented, the position is determined as the position to be photographed;

[0133] The X-ray tube is rotated from the 0° viewing angle position to the leftmost position to be photographed, then returned to the 0° viewing angle position, and then rotated to the rightmost position to be photographed;

[0134] The X-ray images captured at two adjacent corners are respectively determined as the first X-ray image and the second X-ray image.

[0135] Referring to FIG. 13 , in specific practice, the step S20 projects the second X-ray image to a 0° viewing angle to obtain a third X-ray image, including:

[0136] The second X-ray image (in FIG. Figure 2 ) viewing angle, the area of ​​the X-ray detector surface projected to the 0° viewing angle is determined as the third X-ray image (in FIG. 13 Figure 2 ');

[0137] The pixel value of any point on the third X-ray image (Pi-2' in FIG. 13) is equal to the pixel value of the intersection point (Pi-2 in FIG. 13) of the line connecting the point and the center of the X-ray tube on the second X-ray image. The pixel value of the intersection point (Pi-2 in FIG. 13) is calculated by interpolating the pixel values ​​of the two adjacent points on the left and right of the second X-ray image:

[0138] P i-2 =P j +(P j+1 -P j )(ll j ) / (l j+1 -l j )

[0139] Among them, Pj and Pj+1 are the pixel values ​​of the two adjacent points on the left and right of the point Pi-2 in image 2, l represents the image position of point Pi-2 on image 2, and l j represents the image position of point Pj on image 2, l j+1 The image position of the representative point Pj+1 on image 2.

[0140] See also Fig.13A and Fig. 13B, assuming that the distance between X-ray detectors is d, the distance from the X-ray tube to the X-ray detector is SID, the length of the X-ray detector is L, so the length of image 1 is also L, assuming that the length of image 2' is L2, the angle of the rotation angle is θ, and each time the X-ray tube takes an image, the viewing angle of the probe is 2α, and the angle with the surface of the X-ray detector is β. Draw a perpendicular line from the boundary point of the viewing angle 2α to image 2', and the perpendicular line divides the supplementary angle of β into γ and τ. The supplementary angle of β is γ+τ. According to the sine theorem, L2 can be obtained, and then the angle in Figure 13 can be obtained. Figure 1 and Figure 2 ''s relative translation.

[0141] According to the law of sine, L2 can be obtained as follows:

[0142]

[0143]

[0144]

[0145]

[0146]

[0147]

[0148]

[0149] La=L*cosθ

[0150] Lc=L*sinθ

[0151] Lb=Lc*tanτ

[0152] L2=La+Lb=L*cosθ+Lc*tanT

[0153]

[0154] Among them, the variables on the right side of the L2 calculation formula are all known, and the specific value of L2 can be obtained by calculation.

[0155] It can be understood that the technical solution provided in this embodiment, compared with the first embodiment, is that before the first X-ray image and the second X-ray image taken at different viewing angles are spliced, the second X-ray image is first projected to a viewing angle of 0°, and the third X-ray image is obtained and then stitched. This can improve the accuracy of stitching, make the viewing angle of the stitched image more natural, and make the image fusion more realistic, which can further enhance the visual comfort of the user.

[0156] Embodiment 3

[0157] Fig.14 is a schematic block diagram of an image stitching device 100 according to an exemplary embodiment. Fig.14 As shown, the device 100 comprises:

[0158] An acquisition module 101 is used to acquire a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken around the part of a human body to be detected;

[0159] A calculation module 102, configured to calculate a relative translation amount between the second X-ray image and the first X-ray image;

[0160] A stitching module 103, configured to stitch the first X-ray image and the second X-ray image according to the relative translation amount and a stitching direction selected by a user;

[0161] It is also used to take the next X-ray image if the stitching result meets the user's needs, and use the second X-ray image obtained by the stitching module as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to the acquisition module until all X-ray images of the body part to be detected are stitched;

[0162] The fusion module 104 is used to render the stitched long image in blocks and perform image fusion.

[0163] It should be noted that, in specific practice, the technical solution provided in this embodiment is loaded into the controller of a C-arm X-ray device or a DR device for operation.

[0164] The implementation methods and beneficial effects of the above modules are described in detail in the first embodiment, and will not be described in detail in this embodiment.

[0165] It can be understood that the technical solution provided in this embodiment, by shooting different first X-ray images and second X-ray images with overlapping areas, calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the stitching of images can be completed accurately and automatically, and the field of view of the C-arm can be increased without moving the C-arm, which helps to improve the doctor's intraoperative operation success rate and surgical efficiency, saves the manpower investment caused by manual image stitching in the prior art, avoids the problems of low stitching accuracy and low efficiency caused by manual stitching, and improves the user experience.

[0166] Embodiment 4

[0167] See also Fig.15 According to an exemplary embodiment, an electronic device includes:

[0168] Processor 701, communication interface 702, memory 703 and communication bus 704, wherein the processor 701, communication interface 702, memory 703 communicate with each other via the communication bus 704;

[0169] Memory 703, used for storing computer programs;

[0170] The processor 701 is used to implement the above method when executing the program stored in the memory.

[0171] It can be understood that the technical solution provided in this embodiment, by shooting different first X-ray images and second X-ray images with overlapping areas, calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the stitching of images can be completed accurately and automatically, and the field of view of the C-arm can be increased without moving the C-arm, which helps to improve the doctor's intraoperative operation success rate and surgical efficiency, saves the manpower investment caused by manual image stitching in the prior art, avoids the problems of low stitching accuracy and low efficiency caused by manual stitching, and improves the user experience.

[0172] Embodiment 5

[0173] According to an exemplary embodiment, a non-transitory computer-readable storage medium storing computer instructions is shown, where the computer instructions are used to enable a computer to execute the above method.

[0174] It can be understood that the technical solution provided in this embodiment, by shooting different first X-ray images and second X-ray images with overlapping areas, calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the stitching of images can be completed accurately and automatically, and the field of view of the C-arm can be increased without moving the C-arm, which helps to improve the doctor's intraoperative operation success rate and surgical efficiency, saves the manpower investment caused by manual image stitching in the prior art, avoids the problems of low stitching accuracy and low efficiency caused by manual stitching, and improves the user experience.

[0175] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, an optical disk, etc.

[0176] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An image stitching method, characterized in that: include: Step S1, obtaining a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken around the part of a human body to be detected; Step S2, calculating the relative translation amount between the second X-ray image and the first X-ray image; Step S3, stitching the first X-ray image and the second X-ray image according to the relative translation amount and the stitching direction selected by the user; Step S4: if the stitching result meets the user's requirements, take the next X-ray image, and use the second X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1; Step S5, repeating the above steps S1 to S4 until all X-ray images of the body part to be detected are stitched together; Step S6: Render the stitched long image in blocks and perform image fusion.

2. The method according to claim 1, characterized in that: The step S4 further comprises: If the stitching result does not meet the user's requirements, the second X-ray image is retaken, and the process returns to step S1 to stitch the retaken second X-ray image with the first X-ray image.

3. The method according to claim 1, characterized in that The step S1 comprises: The line connecting the X-ray tube and the center point of the part of the human body to be detected is determined as the center line of the 0° viewing angle; The left and right sides of the center line of the 0° viewing angle are incremented by a preset angle, and each time the preset angle is incremented, the position is determined as the position to be photographed; The X-ray tube is rotated from the 0° viewing angle position to the leftmost position to be photographed, then returned to the 0° viewing angle position, and then rotated to the rightmost position to be photographed; The X-ray images captured at two adjacent corners are respectively determined as the first X-ray image and the second X-ray image.

4. The method according to claim 1, characterized in that: The step S2 comprises: Step S21, cutting the first X-ray image and the second X-ray image respectively around the edges according to a preset cropping range to obtain a first cut image and a second cut image; Step S22: selecting a region of interest (ROI) for the first cut image according to the stitching direction selected by the user; Step S23, traversing all possible regions of the second cut image, calculating the similarity between all possible regions and the ROI, and obtaining a similarity matrix; Step S24, determining a possible region corresponding to the maximum similarity in the similarity matrix, determining the relative translation between the possible region and the ROI as the relative translation between the second cut image and the first cut image, and outputting the maximum similarity.

5. The method according to claim 4, characterized in that Before step S21, the step further includes: step S20, projecting the second X-ray image to a 0° viewing angle to obtain a third X-ray image; The step S21 is specifically as follows: According to a preset cropping range, the first X-ray image and the third X-ray image are respectively cropped around four sides to obtain a first cropped image and a second cropped image; The step S3 is specifically as follows: splicing the projection images of the first X-ray image and the second X-ray image at a 0° viewing angle to a third X-ray image according to the relative translation amount and the stitching direction selected by the user; The step S4 is specifically as follows: If the stitching result meets the user's needs, take the next X-ray image, and use the third X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1.

6. The method according to claim 5, characterized in that The step S20, projecting the second X-ray image to a 0° viewing angle to obtain a third X-ray image, comprises: Determine the area of ​​the X-ray detector surface projected to the 0° viewing angle under the viewing angle of the second X-ray image as the third X-ray image; The pixel value of any point on the third X-ray image is equal to the pixel value of the intersection of the line connecting the point and the center of the X-ray tube on the second X-ray image, and the pixel value of the intersection is obtained by interpolating the pixel values ​​of the two adjacent points on the left and right on the second X-ray image.

7. The method according to claim 4, characterized in that The selecting of a region of interest (ROI) for the first cutting image includes: Get the height and width of the ROI determined by the user; According to the stitching direction selected by the user, determine the edge of the first cut image to be stitched with the second cut image, take the edge as the reference, the first preset height from the edge is the bottom edge position of the ROI, and the first preset height from the edge plus the height of the ROI is the top edge position of the ROI; The area enclosed by the bottom edge of the ROI and the top edge of the ROI is extended left and right by the width of the ROI in the middle to obtain the region of interest ROI.

8. The method according to claim 4, characterized in that The method traverses all possible regions of the second cut image, calculates the similarity between all possible regions and the ROI, and obtains a similarity matrix, which is specifically: From the upper left corner of the second cut image to the lower right corner of the second cut image, a region of the same size as the ROI is selected, and the similarity between each region and the ROI is calculated to obtain a similarity matrix.

9. The method according to any one of claims 4 to 8, characterized in that: The step S5, rendering the spliced ​​long image in blocks and performing image fusion, includes: Step S51, reading the relative translation amount between the second cut image and the first cut image, and the fusion area ratio set by the user; Step S52, calculating the correction size of the overlapping area between the second cut image and the first cut image according to the relative translation amount and the ratio of the fusion area, and cropping the second cut image according to the correction size to obtain a third cut image; Step S53, calculating the size of the long image after splicing according to the relative translation amount between the third cut image and the first cut image, wherein the overlapping area of ​​the third cut image and the first cut image is the area to be fused; Step S54: if the spliced ​​long image is smaller than the clinically required long image size, the third cut image obtained in step S53 is used as the first cut image, the cut image spliced ​​after the third cut image is used as the second cut image, and the process returns to step S51; Step S55, repeating the above steps S51 to S54 until the size of the rejoined long image meets the clinical requirement; Step S56: Render the re-stitched long image in blocks, where the rendered blocks include the area that needs to be fused; adopt a different fusion strategy for each block, and return the fused long image stitching result.

10. The method according to claim 9, characterized in that The method uses different fusion strategies for each block and returns the long image stitching result, including: For the area to be fused, the pixel value of each pixel is replaced by the weighted average of the overlapping pixels of the two previous and next cut images to be fused.

11. An image stitching device, characterized in that: include: An acquisition module, used for acquiring a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken around the corner of the part to be detected; A calculation module, used for calculating the relative translation amount between the second X-ray image and the first X-ray image; A stitching module, used for stitching the first X-ray image and the second X-ray image according to the relative translation amount and a stitching direction selected by a user; It is also used to take the next X-ray image if the stitching result meets the user's needs, and use the second X-ray image obtained by the stitching module as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to the acquisition module until all X-ray images of the body part to be detected are stitched; The fusion module is used to render the stitched long image in blocks and perform image fusion.

12. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 10 when executing a program stored in a memory.

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 10.

Images