Image splicing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image splicing method and device, electronic equipment and a storage medium, and the method comprises the steps: calculating the relative translation amount of a second X-ray image and a first X-ray image, splicing the first X-ray image and the second X-ray image according to the relative translation amount, enabling the splicing of the images to be accurately and automatically completed, and improving the splicing efficiency. The human input caused by manual image splicing in the prior art is saved, the problems of low splicing accuracy and low efficiency caused by manual splicing are avoided, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image stitching method and device, an electronic device and a storage medium. Background Art

[0002] Currently, X-ray splicing is widely used in DR equipment (DR, direct digital X-ray photography system).

[0003] In the medical field, X-ray machines, such as X-ray machines, are often used to image the area of ​​interest of a patient, and doctors then diagnose and treat the patient based on the images obtained. Figure 1A As shown, an X-ray machine is illustrated, whose main components include an X-ray tube 1, an X-ray beam limiter 2, a patient fixer 3, and a detector 4, wherein the main function of the X-ray tube 1 is to emit X-rays, the main function of the X-ray beam limiter 2 is to limit the irradiation range of the light field of the X-rays emitted by the X-ray tube 1, the function of the detector 4 is to receive X-rays and form an image which is then transmitted to a workstation for further processing, and the role of the patient fixer 3 is to have two points, the first point is to isolate the patient from the detector 4 for safety reasons, and the second point is to fix the patient so as to minimize movement during the entire shooting process.

[0004] It can be seen from the X-ray machine that the tube 1 emits X-rays that pass through the region of interest and then reach the X-ray detector 4 to obtain an image of the region of interest. The size of the obtained image is generally equal to the size of the X-ray detector. If the field of view of the region of interest is within the size of the X-ray detector, the entire region of interest can be fully presented in one image. For example, the field of view of regions of interest such as the heart and lungs is within the size of the X-ray detector, so regions of interest such as the heart and lungs can be fully presented in one picture. However, there are some regions of interest whose field of view exceeds the size of the X-ray detector, such as the spine and thighs, and the entire region of interest cannot be presented in one picture. This requires taking several pictures and then stitching these pictures together to form a complete image that can present the entire region of interest.

[0005] At present, there are several methods to image the region of interest whose field of view exceeds the size of the X-ray detector. Among them, there are two main categories. One is the angle acquisition method, which takes multiple pictures of the region of interest by rotating the tube, that is, changing the angle of the tube. In other words, when the tube is at a certain angle, take a picture of the region of interest, then rotate the tube to another angle and take another picture of the region of interest, and so on, until the region of interest is completely included in all pictures, and finally all the pictures taken are stitched together to form an image of the region of interest.

[0006] Another method is to use the method of parallel movement of the tube and the X-ray detector, that is, when the tube and the X-ray detector are in the first position, take a picture, then move the tube and the X-ray detector to the second position at the same time, take another picture, and so on, move the tube and the X-ray detector in parallel until the end of the region of interest, and finally stitch the taken pictures together to form an image of the region of interest. This image stitching method uses manual parallel movement of the tube and the X-ray detector, that is, after each picture is taken, the operator manually moves the tube and the X-ray detector to the next position in parallel according to his experience. Due to manual operation, the work efficiency is low, and due to the different experiences of different operators, the final image of the region of interest is often inaccurate.

[0007] Mobile C-arm X-ray machines used in surgery are usually pushed or pulled manually to stitch X-ray images. This results in an unknown amount of movement offset, which makes image stitching extremely difficult. In the past, image stitching of C-arms was generally done by manually moving images using software, which was time-consuming and labor-intensive. Summary of the invention

[0008] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide an image stitching method and device, an electronic device and a storage medium to solve the problem in the related art that the X-ray machine is usually pushed or pulled manually, resulting in an unknown movement offset and time-consuming and labor-intensive stitching of X-ray images.

[0009] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0010] According to a first aspect of the present invention, there is provided an image stitching method, comprising:

[0011] Step S1, obtaining a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body that are taken along a part of a human body to be detected and have partial image overlap;

[0012] Step S2, calculating the relative translation amount between the second X-ray image and the first X-ray image;

[0013] Step S3, stitching the first X-ray image and the second X-ray image according to the relative translation amount and the stitching direction selected by the user;

[0014] Step S4: if the stitching result meets the user's requirements, take the next X-ray image, and use the second X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1;

[0015] Step S5, repeating the above steps S1 to S4 until all X-ray images of the body part to be detected are stitched together;

[0016] Step S6: Render the stitched long image in blocks and perform image fusion.

[0017] Preferably, the step S4 further includes:

[0018] If the stitching result does not meet the user's requirements, the second X-ray image is retaken, and the process returns to step S1 to stitch the retaken second X-ray image with the first X-ray image.

[0019] Preferably, step S2 comprises:

[0020] According to a preset cropping range, the first X-ray image and the second X-ray image are respectively cropped around to obtain a first cropped image and a second cropped image;

[0021] According to the stitching direction selected by the user, the region of interest (ROI) is selected for the first cutting image;

[0022] Traversing all possible regions of the second cut image, calculating the similarities between all possible regions and the ROI, and obtaining a similarity matrix;

[0023] A possible region corresponding to the maximum similarity in the similarity matrix is ​​determined, a relative translation amount between the possible region and the ROI is determined as a relative translation amount between the second cut image and the first cut image, and the maximum similarity is outputted at the same time.

[0024] Preferably, the selecting of a region of interest (ROI) on the first cutting image includes:

[0025] Get the height and width of the ROI determined by the user;

[0026] According to the stitching direction selected by the user, determine the edge of the first cut image to be stitched with the second cut image, take the edge as the reference, the first preset height from the edge is the bottom edge position of the ROI, and the first preset height from the edge plus the height of the ROI is the top edge position of the ROI;

[0027] The area enclosed by the bottom edge of the ROI and the top edge of the ROI is extended left and right by the width of the ROI in the middle to obtain the region of interest ROI.

[0028] Preferably, the traversing of all possible regions of the second cut image, calculating the similarity between all possible regions and the ROI, and obtaining a similarity matrix is ​​specifically:

[0029] From the upper left corner of the second cut image to the lower right corner of the second cut image, a region of the same size as the ROI is selected, and the similarity between each region and the ROI is calculated to obtain a similarity matrix.

[0030] Preferably, the step S5, rendering the stitched long image in blocks and performing image fusion, includes:

[0031] Step S51, reading the relative translation amount between the second cut image and the first cut image, and the fusion area ratio set by the user;

[0032] Step S52, calculating the correction size of the overlapping area between the second cut image and the first cut image according to the relative translation amount and the ratio of the fusion area, and cropping the second cut image according to the correction size to obtain a third cut image;

[0033] Step S53, calculating the size of the long image after splicing according to the relative translation amount between the third cut image and the first cut image, wherein the overlapping area of ​​the third cut image and the first cut image is the area to be fused;

[0034] Step S54: if the spliced ​​long image is smaller than the clinically required long image size, the third cut image obtained in step S53 is used as the first cut image, the cut image spliced ​​after the third cut image is used as the second cut image, and the process returns to step S51;

[0035] Step S55, repeating the above steps S51 to S54 until the size of the rejoined long image meets the clinical requirement;

[0036] Step S56: Render the re-stitched long image in blocks, where the rendered blocks include the area that needs to be fused; adopt a different fusion strategy for each block, and return the fused long image stitching result.

[0037] Preferably, the method of adopting a different fusion strategy for each block and returning a long image stitching result includes:

[0038] For the area to be fused, the pixel value of each pixel is replaced by the weighted average of the overlapping pixels of the two previous and next cut images to be fused.

[0039] According to a second aspect of the present invention, there is provided an image stitching device, comprising:

[0040] An acquisition module, used for acquiring a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken along the part of a human body to be detected;

[0041] A calculation module, used for calculating the relative translation amount between the second X-ray image and the first X-ray image;

[0042] A stitching module, used for stitching the first X-ray image and the second X-ray image according to the relative translation amount and a stitching direction selected by a user;

[0043] It is also used to take the next X-ray image if the stitching result meets the user's needs, and use the second X-ray image obtained by the stitching module as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to the acquisition module until all X-ray images of the body part to be detected are stitched;

[0044] The fusion module is used to render the stitched long image in blocks and perform image fusion.

[0045] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0046] A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0047] Memory, used to store computer programs;

[0048] The processor is used to implement the above method when executing the program stored in the memory.

[0049] According to a fourth aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the above method.

[0050] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:

[0051] By calculating the relative translation between the second X-ray image and the first X-ray image, and splicing the first X-ray image and the second X-ray image according to the relative translation, the image stitching can be completed accurately and automatically, saving the manpower investment caused by manual image stitching in the prior art, avoiding the problems of low stitching accuracy and low efficiency caused by manual stitching, and improving the user experience.

[0052] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1A It is a structural schematic diagram of an X-ray machine shown according to the background technology;

[0054] Figure 1 is a flow chart of an image stitching method according to an exemplary embodiment;

[0055] Figure 2A and Figure 2B is a schematic diagram showing an overlapping area of ​​a first X-ray image and a second X-ray image according to an exemplary embodiment;

[0056] Figure 3 is a schematic diagram showing a result of stitching a first X-ray image and a second X-ray image according to an exemplary embodiment;

[0057] Figure 4 is a long image after splicing is completed according to an exemplary embodiment;

[0058] Figure 5 is a flow chart of an image stitching method according to an exemplary embodiment;

[0059] Figure 6 is a schematic diagram of a similarity matrix calculation method according to an exemplary embodiment;

[0060] Figure 7 is a flow chart of an image fusion method according to an exemplary embodiment;

[0061] Figure 8 is a schematic diagram showing correction of a second cutting image according to an exemplary embodiment;

[0062] Fig. 9 is a schematic diagram showing block-by-block rendering of a long image after splicing according to an exemplary embodiment;

[0063] Figures 10A to 10C is a schematic diagram of an algorithm for fusing the area to be fused in the spliced ​​long image according to an exemplary embodiment;

[0064] Fig.11 is a schematic block diagram of an image stitching device according to an exemplary embodiment;

[0065] Fig.12 It is a schematic block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0067] As described in the previous background technology, there is a problem in the related art that the X-ray machine is usually pushed or pulled manually, resulting in an unknown movement offset and time-consuming and labor-intensive stitching of X-ray images.

[0068] In order to effectively solve the problems in the related art, the present invention provides an image stitching method and device, an electronic device and a storage medium, which are described in detail below.

[0069] Embodiment 1

[0070] Figure 1 is a flow chart of an image stitching method according to an exemplary embodiment. Figure 1 As shown, the method comprises:

[0071] Step S1, obtaining a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body that are taken along a part of a human body to be detected and have partial image overlap;

[0072] Step S2, calculating the relative translation amount between the second X-ray image and the first X-ray image;

[0073] Step S3, stitching the first X-ray image and the second X-ray image according to the relative translation amount and the stitching direction selected by the user;

[0074] Step S4: if the stitching result meets the user's requirements, take the next X-ray image, and use the second X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1;

[0075] Step S5, repeating the above steps S1 to S4 until all X-ray images of the body part to be detected are stitched together;

[0076] Step S6: Render the stitched long image in blocks and perform image fusion.

[0077] It should be noted that, in specific practice, the technical solution provided in this embodiment is loaded into the controller of a C-arm X-ray device or a DR device for operation.

[0078] Taking the operation of a C-arm X-ray device as an example, in step S1, the first X-ray image and the second X-ray image to be stitched are obtained. In specific practice, the acquisition can be performed by the following methods, including:

[0079] Step S11: The user uses a mobile C-arm X-ray machine to select a multiple X-ray scanning stitching protocol (the X-ray scanning stitching protocol represents a scanning method with a stitching process, and the protocol supports the image stitching method described in this embodiment).

[0080] Step S12, after entering the protocol, the user selects the splicing direction (for example, splicing from top to bottom, or from left to right, or from bottom to top, or from right to left, etc.; for another example, the spine can be from top to bottom, or from bottom to top; the pelvis can be from left to right, or from right to left).

[0081] Step S13, operate the C-arm X-ray machine, position it correctly, generally, try to make the human body close to the detector direction, and take the first X-ray image. At this time, the system will display the stitching queue. It should be noted that the first X-ray image and the second X-ray image mentioned in this embodiment refer to the previous X-ray image and the next X-ray image of the two consecutively taken X-ray images, and do not specifically refer to the first X-ray image and the second X-ray image.

[0082] Step S14: translate the C-arm to capture a second X-ray image. At this point, the captured second X-ray image needs to be overlapped with the first X-ray image at a preset overlap ratio (the preset overlap ratio can be set to 20%, 30%, 40%, 50%, etc., preferably 50%, according to user needs). Figure 2A and Figure 2B As shown, the box area is the overlapping area of ​​the two X-ray images.

[0083] It can be understood that the technical solution provided in this embodiment calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the image stitching can be completed accurately and automatically, saving the manpower investment caused by manual image stitching in the prior art, avoiding the problems of low stitching accuracy and low efficiency caused by manual stitching, and improving the user experience.

[0084] In specific practice, the step S4 further includes:

[0085] If the stitching result does not meet the user's requirements, the second X-ray image is retaken, and the process returns to step S1 to stitch the retaken second X-ray image with the first X-ray image.

[0086] The “does not meet user needs” refers to that the image resolution of the overlapping area of ​​the first X-ray image and the second X-ray image after stitching, or the size of the overlapping area does not meet the stitching requirements.

[0087] In this case, the user deletes the second X-ray image on the interface and imports the re-taken second X-ray image into the stitching queue; the user can also complete the queue by manually stitching. If the stitching result meets the user's needs, repeat the above shooting until the shooting length meets the clinical requirements. Click End Collection, the system completes all stitching and outputs a long image that meets the requirements, such as Figure 4 shown.

[0088] It is understandable that the technical solution provided in this embodiment first splices the first X-ray image and the second X-ray image, and only when the splicing result meets the user's needs will the next image be spliced, so that the quality and shooting angle of each sub-image in the final spliced ​​long image can meet the user's needs, avoiding rework and low efficiency caused by subsequent changes. At the same time, it can also reduce the number of shooting angle changes of the X-ray equipment and reduce the wear and tear of the X-ray equipment.

[0089] Preferably, see Figure 5 , the step S2 comprises:

[0090] Step S21, cutting the first X-ray image and the second X-ray image at four sides according to a preset cutting range to obtain a first cut image I1 and a second cut image I2;

[0091] Assume that for the first X-ray image, the cropping amplitudes of top, bottom, left and right are c1, c2, c3, and c4 respectively; the initial size of the first X-ray image is w0×h0, and the image size after cropping is w1×h1, then:

[0092] w1=w0-c3-c4

[0093] h1=h0-c1-c2

[0094] c1, c2, c3, and c4 can be set according to user needs. For example, c1 to c4 are all set to 50.

[0095] Step S22, selecting a region of interest ROI for the first cut image I1 according to the stitching direction selected by the user;

[0096] Step S23, traversing all possible regions of the second cut image I2, calculating the similarity between all possible regions and the ROI, and obtaining a similarity matrix;

[0097] Step S24, determine the possible area corresponding to the maximum similarity in the similarity matrix, determine the relative translation between the possible area and the ROI as the relative translation between the second cut image I2 and the first cut image I1, and output the maximum similarity (usually greater than 0.6 is better).

[0098] In specific practice, the step S22, selecting a region of interest ROI from the first cut image I1, includes:

[0099] Get the height and width of the ROI determined by the user;

[0100] According to the stitching direction selected by the user, determine the edge of the first cut image I1 to be stitched with the second cut image, take the edge as the reference, the first preset height from the edge is the bottom edge of the ROI, and the first preset height from the edge plus the height of the ROI is the top edge of the ROI;

[0101] The area enclosed by the bottom edge of the ROI and the top edge of the ROI is extended left and right by the width of the ROI in the middle to obtain the region of interest ROI.

[0102] It should be noted that the first preset height is set according to user needs, or according to experimental data, or according to historical experience values. For example:

[0103] Set the height of ROI to 1 / 4 of the height of I1. In principle, assuming that the stitching direction selected by the user is from top to bottom, the lower the ROI is, the better. Excluding the dead zone that may not be removed, select ROI 1 / 16 from the bottom of I1. Therefore, the top of ROI is at 11 / 16 of I1.

[0104] Set the width of ROI. Considering the lateral shift caused by manual movement of the C-arm, the width of ROI is set to 3 / 4 of I1 and centered. That is, the starting positions of the left and right sides of ROI are 1 / 8 and 7 / 8 of I1 respectively.

[0105] It should be noted that the biggest difference between the splicing of mobile C-arm X-ray equipment and DR equipment is that the movement of DR equipment is controlled by the system's mechanical device, but the C-arm is manually operated;

[0106] Although DR equipment has SOP (Standard Operating Procedure) to guide user operation, it is still affected by human factors, and inevitably has large deviations in the up, down, left, and right directions. In order to mitigate the above impact, this embodiment adopts the above scheme to select ROI to reduce the impact of human factors on image stitching.

[0107] In specific practice, Figure 6 As shown, step S23, traverse all possible regions of the second cut image, calculate the similarity between all possible regions and the ROI, and obtain a similarity matrix, which is specifically:

[0108] From the upper left corner of the second cut image I2 to the lower right corner of the second cut image, a region R2 of the same size as the ROI is selected, and the similarity between each region R2 and the ROI is calculated to obtain a similarity matrix M.

[0109] Assume that the initial size of the first X-ray image is w0×h0, and the image size after cropping is w1×h1. Assume that the ROI area is R1 and the size is w R ×h R ;

[0110] The similarity between R2 and R1 is calculated using the following formula:

[0111] S=S1+S2

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118] Among them, R1(x,y) is the pixel value of image R1 at (x,y), R2(x,y) is the pixel value of image R2 at (x,y), R′1(x,y) is the relative pixel value of image R1 at (x,y), and R′2(x,y) is the relative pixel value of image R2 at (x,y). is the average pixel value of image R1, is the average pixel value of image R2. The relative pixel value is relative to the average pixel value in each region.

[0119] It can be understood that the size of the similarity matrix M is (w1-w R )×(h1-h R ), each element in the similarity matrix M corresponds to the similarity at a translation position.

[0120] See also Figure 7 The step S5, rendering the stitched long image in blocks and performing image fusion, includes:

[0121] Step S51, reading the relative translation amount (dx1, dy1) between the second cut image and the first cut image, and the fusion area ratio set by the user (for example, 10%);

[0122] Step S52: Calculate the corrected size of the overlapped area between the second cut image I2 and the first cut image I1 according to the relative translation amount and the ratio of the fused area (eg Figure 8 As shown), and the second cutting image I2 is cut according to the modified size to obtain a third cutting image I3;

[0123] Step S53, according to the relative translation amount (dx2, dy2) between the third cut image I3 and the first cut image I1, dx1=dx2, the size of the long image after splicing is calculated, wherein the overlapping area of ​​the third cut image and the first cut image is the area to be fused;

[0124] Step S54: if the spliced ​​long image (size is denoted as (w l ,h l )) is smaller than the long image size required by clinical practice, the third cutting image obtained in step S53 is used as the first cutting image, the cutting image spliced ​​behind the third cutting image is used as the second cutting image, and the process returns to step S51;

[0125] Step S55, repeating the above steps S51 to S54 until the size of the rejoined long image meets the clinical requirement;

[0126] Step S56: Render the reassembled long image in blocks, where the rendered blocks include the area that needs to be merged (such as Fig. 9 As shown in the figure); adopt different fusion strategies (including but not limited to: copy, fill and merge) for each block, and return the fused long image splicing result.

[0127] Among them, for the area to be fused, the pixel value of each pixel is replaced by the weighted average of the overlapping pixels of the two previous and next cut images to be fused.

[0128] like Figures 10A to 10C As shown, it can be considered that when images are fused, the value of a pixel is the weighted average of the overlapping areas of the upper and lower images, as shown in the following formula:

[0129] p(x,y)=w1*p1(x,y)+w2*p2(x,y)

[0130] Among them, p i (x, y) is the pixel value of point (x, y) on the i-th image (i = 1 or 2), p1(x, y) is the pixel value at point (x, y) on the first cut image, and p2(x, y) is the pixel value of the pixel overlapping with p1(x, y) on the third cut image. w1 and w2 are the corresponding weights, obtained by the following formula:

[0131] w i =d i / d s

[0132] d s =∑d i

[0133]

[0134] Among them, d i_j Represents the distance of a pixel position from the jth edge on the i-th image, d i is the minimum distance from the edge of the i-th image at the pixel position, d s is the minimum distance d of all overlapping images at that position i The sum of the two images, usually there are two images that overlap at this position, that is, d s =d1+d2.

[0135] In summary, the technical solution provided in this embodiment calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the image stitching can be completed accurately and automatically, saving the manpower investment caused by manual image stitching in the prior art, avoiding the problems of low stitching accuracy and low efficiency caused by manual stitching, and improving the user experience.

[0136] When using a C-arm X-ray machine during surgery, it is limited by its own field of view and cannot fully observe the entire space required. By taking multiple X-ray images with overlapping areas and automatically stitching the X-ray images, the C-arm's field of view is increased, which helps improve the doctor's success rate and efficiency during surgery.

[0137] Embodiment 2

[0138] Fig.11 is a schematic block diagram of an image stitching device 100 according to an exemplary embodiment. Fig.11 As shown, the device 100 comprises:

[0139] An acquisition module 101 is used to acquire a first X-ray image and a second X-ray image to be stitched, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken along the part of a human body to be detected;

[0140] A calculation module 102, configured to calculate a relative translation amount between the second X-ray image and the first X-ray image;

[0141] A stitching module 103, configured to stitch the first X-ray image and the second X-ray image according to the relative translation amount and a stitching direction selected by a user;

[0142] It is also used to take the next X-ray image if the stitching result meets the user's needs, and use the second X-ray image obtained by the stitching module as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to the acquisition module until all X-ray images of the body part to be detected are stitched;

[0143] The fusion module 104 is used to render the stitched long image in blocks and perform image fusion.

[0144] It should be noted that, in specific practice, the technical solution provided in this embodiment is loaded into the controller of a C-arm X-ray device or a DR device for operation.

[0145] The implementation methods and beneficial effects of the above modules are described in detail in the first embodiment, and will not be described in detail in this embodiment.

[0146] It can be understood that the technical solution provided in this embodiment calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the image stitching can be completed accurately and automatically, saving the manpower investment caused by manual image stitching in the prior art, avoiding the problems of low stitching accuracy and low efficiency caused by manual stitching, and improving the user experience.

[0147] Embodiment 3

[0148] See also Fig.12 According to an exemplary embodiment, an electronic device includes:

[0149] Processor 701, communication interface 702, memory 703 and communication bus 704, wherein the processor 701, communication interface 702, memory 703 communicate with each other via the communication bus 704;

[0150] Memory 703, used for storing computer programs;

[0151] The processor 701 is used to implement the above method when executing the program stored in the memory.

[0152] It can be understood that the technical solution provided in this embodiment calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the image stitching can be completed accurately and automatically, saving the manpower investment caused by manual image stitching in the prior art, avoiding the problems of low stitching accuracy and low efficiency caused by manual stitching, and improving the user experience.

[0153] Embodiment 4

[0154] According to an exemplary embodiment, a non-transitory computer-readable storage medium storing computer instructions is shown, where the computer instructions are used to enable a computer to execute the above method.

[0155] It can be understood that the technical solution provided in this embodiment calculates the relative translation between the second X-ray image and the first X-ray image, and splices the first X-ray image and the second X-ray image according to the relative translation, so that the image stitching can be completed accurately and automatically, saving the manpower investment caused by manual image stitching in the prior art, avoiding the problems of low stitching accuracy and low efficiency caused by manual stitching, and improving the user experience.

[0156] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, an optical disk, etc.

[0157] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An image stitching method, characterized in that: include: Step S1, obtaining a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body that are taken along a part of a human body to be detected and have partial image overlap; Step S2, calculating the relative translation amount between the second X-ray image and the first X-ray image; Step S3, stitching the first X-ray image and the second X-ray image according to the relative translation amount and the stitching direction selected by the user; Step S4: if the stitching result meets the user's requirements, take the next X-ray image, and use the second X-ray image obtained in step S3 as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to step S1; Step S5, repeating the above steps S1 to S4 until all X-ray images of the body part to be detected are stitched together; Step S6: Render the stitched long image in blocks and perform image fusion.

2. The method according to claim 1, characterized in that: The step S4 further comprises: If the stitching result does not meet the user's requirements, the second X-ray image is retaken, and the process returns to step S1 to stitch the retaken second X-ray image with the first X-ray image.

3. The method according to claim 1, characterized in that The step S2 comprises: According to a preset cropping range, the first X-ray image and the second X-ray image are respectively cropped around to obtain a first cropped image and a second cropped image; According to the stitching direction selected by the user, the region of interest (ROI) is selected for the first cutting image; Traversing all possible regions of the second cut image, calculating the similarities between all possible regions and the ROI, and obtaining a similarity matrix; A possible region corresponding to the maximum similarity in the similarity matrix is ​​determined, a relative translation amount between the possible region and the ROI is determined as a relative translation amount between the second cut image and the first cut image, and the maximum similarity is outputted at the same time.

4. The method according to claim 3, characterized in that The selecting of a region of interest (ROI) for the first cutting image includes: Get the height and width of the ROI determined by the user; According to the stitching direction selected by the user, determine the edge of the first cut image to be stitched with the second cut image, take the edge as the reference, the first preset height from the edge is the bottom edge position of the ROI, and the first preset height from the edge plus the height of the ROI is the top edge position of the ROI; The area enclosed by the bottom edge of the ROI and the top edge of the ROI is extended left and right by the width of the ROI in the middle to obtain the region of interest ROI.

5. The method according to claim 3, characterized in that: The method traverses all possible regions of the second cut image, calculates the similarity between all possible regions and the ROI, and obtains a similarity matrix, which is specifically: From the upper left corner of the second cut image to the lower right corner of the second cut image, a region of the same size as the ROI is selected, and the similarity between each region and the ROI is calculated to obtain a similarity matrix.

6. The method according to any one of claims 3 to 5, characterized in that: The step S5, rendering the spliced ​​long image in blocks and performing image fusion, includes: Step S51, reading the relative translation amount between the second cut image and the first cut image, and the fusion area ratio set by the user; Step S52, calculating the correction size of the overlapping area between the second cut image and the first cut image according to the relative translation amount and the ratio of the fusion area, and cropping the second cut image according to the correction size to obtain a third cut image; Step S53, calculating the size of the long image after splicing according to the relative translation amount between the third cut image and the first cut image, wherein the overlapping area of ​​the third cut image and the first cut image is the area to be fused; Step S54: if the spliced ​​long image is smaller than the clinically required long image size, the third cut image obtained in step S53 is used as the first cut image, the cut image spliced ​​after the third cut image is used as the second cut image, and the process returns to step S51; Step S55, repeating the above steps S51 to S54 until the size of the rejoined long image meets the clinical requirement; Step S56: Render the re-stitched long image in blocks, where the rendered blocks include the area that needs to be fused; adopt a different fusion strategy for each block, and return the fused long image stitching result.

7. The method according to claim 6, characterized in that The method uses different fusion strategies for each block and returns the long image stitching result, including: For the area to be fused, the pixel value of each pixel is replaced by the weighted average of the overlapping pixels of the two previous and next cut images to be fused.

8. An image stitching device, characterized in that: include: An acquisition module, used for acquiring a first X-ray image and a second X-ray image to be spliced, wherein the first X-ray image and the second X-ray image represent X-ray images of adjacent parts of a human body with partial image overlap, which are taken along the part of a human body to be detected; A calculation module, used for calculating the relative translation amount between the second X-ray image and the first X-ray image; a stitching module, configured to stitch the first X-ray image and the second X-ray image according to the relative translation amount and a stitching direction selected by a user; It is also used to take the next X-ray image if the stitching result meets the user's needs, and use the second X-ray image obtained by the stitching module as the first X-ray image, and use the newly taken X-ray image as the second X-ray image, and return to the acquisition module until all X-ray images of the body part to be detected are stitched; The fusion module is used to render the stitched long image in blocks and perform image fusion.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 7.

Images