Image stitching method, device and equipment

Abstract

The present application discloses an image stitching method, device and equipment, which relates to the field of medical image processing technology. For images with no overlapping area or very small overlapping area, the stitching requirements can also be met, thereby improving the adaptability of image stitching scenarios. The method includes: obtaining the tag information associated with each segment of the image to be stitched, the tag information including image position information and image attribute information; by traversing and querying the tag information associated with each segment of the image, selecting the image attribute information that meets the stitching conditions as the stitching attribute information of each segment of the image; according to the stitching attribute information of each segment of the image, processing each segment of the image into each segment of the image containing the same image attribute information; stacking and stitching the images with the same image attribute information segment by segment according to the image position information to obtain the stitched image.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to an image stitching method, device and equipment. Background Art

[0002] In recent years, whole-body magnetic resonance imaging technology has been increasingly widely used in clinical applications. For example, whole-spine magnetic resonance imaging has a high diagnostic value for diseases such as spinal deformity, scoliosis, and multiple myeloma. Diffusion-weighted imaging with whole-body background suppression (DWIBS) has been increasingly used in the detection of tumor metastasis and high-end physical examination tumor screening. Head and neck, lower limb and whole-body magnetic resonance vascular imaging technology (MRA) has been increasingly widely used in the examination of vascular lesions due to its advantages of being non-invasive, not requiring contrast agents, being able to perform three-dimensional imaging, and being able to observe from multiple angles. However, the magnet length and gradient range of current magnetic resonance equipment are physically limited, and the farther away from the center an image is, the worse the uniformity of the magnetic field. Currently, whole-body magnetic resonance imaging mostly uses multiple segments for scanning, and then obtains the entire image through image stitching technology.

[0003] In the related art, for magnetic resonance images scanned in the sagittal or coronal plane, taking magnetic resonance whole spine imaging as an example, in order to obtain the image stitching result, the overlapping area of ​​adjacent segments of the image to be stitched can be determined, and then the overlapping area or the extended area of ​​the overlapping area can be registered to obtain the registered image, and the registered image can be stitched; for magnetic resonance images scanned in the transverse plane, taking magnetic resonance angiography MRA as an example, in order to obtain the image stitching result, the MIP post-processing reconstruction can be performed on each segment of the MRA original image obtained by the scan, and then each segment of the MIP post-processing reconstructed image can be rotated 360 degrees in the same rotation direction and the same step length, and the image at each step length is screenshotted during the rotation process to obtain the MIP screenshot sequence image of each segment of the original image, so as to obtain an image parallel to the long axis direction of the human body, and the obtained MIP screenshot sequence image of each segment is used as input, and the same stitching process as the image scanned in the sagittal or coronal plane is performed to finally obtain the image stitching result.

[0004] The above-mentioned image stitching process is mostly based on first registering the overlapping areas in the image and then stitching them. However, the stitching based on registration is time-consuming and requires a certain overlapping area between the image segments to be stitched. If there is no overlapping area or the overlapping area is very small, it is difficult for the image stitching based on registration to successfully perform the stitching operation, which makes the adaptability of the image stitching scene poor. Summary of the invention

[0005] In view of this, the present application provides an image stitching method, device and equipment, the main purpose of which is to solve the problem in the prior art that for images with no overlapping areas or very small overlapping areas, it is difficult to successfully perform stitching operations based on registration, resulting in poor adaptability of image stitching scenarios.

[0006] According to a first aspect of the present application, there is provided an image stitching method, the method comprising:

[0007] Acquire tag information associated with each segment of the image to be stitched, wherein the tag information includes image position information and image attribute information;

[0008] By traversing and querying the tag information associated with each segment of the image, image attribute information that meets the splicing condition is selected as the splicing attribute information of each segment of the image;

[0009] According to the splicing attribute information of each segment of the image, the each segment of the image is processed into each segment of the image containing the same image attribute information;

[0010] The images having the same image attribute information are stacked and spliced ​​segment by segment according to the image position information to obtain a spliced ​​image.

[0011] Furthermore, the obtaining of the tag information associated with each segment of the image to be spliced ​​specifically includes:

[0012] By querying the image data imported from the preset standard protocol file, the label data of the standard format image contained in each segment of the image to be spliced ​​is read from the image data;

[0013] The label information associated with the image is obtained for the label data of the standard format image contained in each segment of the image, so as to obtain the label information associated with each segment of the image to be spliced.

[0014] Furthermore, after obtaining the tag information associated with each segment of the image to be spliced, the method further includes:

[0015] The image segments are sorted within and between segments according to the direction sequence set by the image position information to obtain the image segments with the same sorting rule.

[0016] Furthermore, the image attribute information records the attribute values ​​marked on the image on different attribute features, and the traversal query of the tag information associated with each segment of the image selects the image attribute information that meets the splicing condition as the splicing attribute information of each segment of the image, specifically including:

[0017] Traversing and querying the tag information associated with each segment of the image, respectively extracting the attribute values ​​marked on different attribute features of each segment of the image;

[0018] By comparing the attribute values ​​marked on the different attribute features of the image segments, the image attribute information that meets the splicing condition is selected as the splicing attribute information of the image segments.

[0019] Furthermore, the selecting of image attribute information that meets the splicing condition as the splicing attribute information of each image segment by comparing the attribute values ​​marked on different attribute features of each image segment specifically includes:

[0020] By comparing the attribute values ​​marked on different attribute features of each image segment, the minimum attribute value marked on the pixel attribute feature and the layer spacing attribute feature of each image segment is selected respectively, and the maximum attribute value of the cross-sectional field of view feature attribute calculated using the attribute value marked on the preset attribute feature of the image is selected as the splicing attribute information of each image segment.

[0021] Furthermore, the processing of the image segments into images each containing the same image attribute information according to the splicing attribute information of the image segments specifically includes:

[0022] According to the splicing attribute information of each segment of the image, respectively extracting the splicing attribute value used in the pixel attribute feature, the layer spacing attribute feature and the cross-sectional field of view attribute feature of each segment of the image;

[0023] Using the splicing attribute values ​​used in the pixel attribute features and the cross-sectional field of view attribute features of each image segment, the attribute values ​​of the internal multi-layer images in each image segment in the pixel attribute features and the cross-sectional field of view attribute features are resampled and zero-filled, so that each image segment contains the same image attribute information within the segment;

[0024] By using the splicing attribute value used in the layer spacing attribute feature of each image segment, the attribute value of each image segment in the layer spacing attribute feature is resampled and filled with zeros, so that each image segment contains the same image attribute information between segments.

[0025] Furthermore, stacking and splicing the images having the same image attribute information segment by segment according to the image position information to obtain the spliced ​​image specifically includes:

[0026] Selecting a reference segment image from the segments of images having the same image attribute information according to a direction sequence set by the image position information;

[0027] The reference segment image is used as a reference image, and the reference segment image is spliced ​​with adjacent segment images according to the image position information. During the splicing process, the reference segment image is continuously updated until all segment images are spliced ​​to obtain a spliced ​​image.

[0028] Furthermore, the continuously updating the reference segment image during the stitching process specifically includes:

[0029] During the stitching process, determining whether there is an overlapping area between the reference segment image and the adjacent segment image;

[0030] If yes, weighted fusion processing is performed on the overlapping area formed by the reference segment image and the adjacent segment image to update the reference segment image; otherwise, stacking and splicing are performed on the reference segment image and the adjacent segment image to update the reference segment image.

[0031] According to a second aspect of the present application, an image stitching device is provided, the device comprising:

[0032] An acquisition unit, used to acquire tag information associated with each segment of the image to be spliced, wherein the tag information includes image position information and image attribute information;

[0033] A selection unit, configured to select image attribute information that meets the splicing condition as the splicing attribute information of each image segment by traversing and querying the tag information associated with each image segment;

[0034] A processing unit, configured to process the image segments into images each containing the same image attribute information according to the splicing attribute information of the image segments;

[0035] The splicing unit is used to stack and splice the images with the same image attribute information segment by segment according to the image position information to obtain a spliced ​​image.

[0036] Furthermore, the acquisition unit includes:

[0037] A reading module, used for reading the label data of the standard format images contained in each segment of the image to be spliced ​​from the image data by querying the image data imported from the preset standard protocol file;

[0038] The acquisition module is used to acquire the label information associated with the image according to the label data of the standard format image contained in each segment of the image, so as to obtain the label information associated with each segment of the image to be spliced.

[0039] Furthermore, the device also includes:

[0040] The sorting unit is used to sort the image segments within and between segments according to the direction sequence set by the image position information after obtaining the mark information associated with the image segments to be spliced, so as to obtain the image segments with the same sorting rule.

[0041] Furthermore, the image attribute information records attribute values ​​marked on different attribute features of the image, and the selection unit includes:

[0042] A query module, used for traversing and querying the tag information associated with each segment of the image, and respectively extracting the attribute values ​​marked on different attribute features of each segment of the image;

[0043] The comparison module is used to select the image attribute information that meets the splicing condition as the splicing attribute information of each image segment by comparing the attribute values ​​marked on different attribute features of each image segment.

[0044] Furthermore, the comparison module is specifically used to compare the attribute values ​​marked on different attribute features of each image segment, select the minimum attribute value marked on the pixel attribute feature and the layer spacing attribute feature for each image segment, and select the maximum attribute value of the cross-sectional field of view feature attribute calculated using the attribute value marked on the preset attribute feature of the image as the splicing attribute information of each image segment.

[0045] Furthermore, the processing unit includes:

[0046] An extraction module, used for extracting the splicing attribute values ​​used in pixel attribute features, layer spacing attribute features and cross-sectional field of view attribute features of each image segment according to the splicing attribute information of each image segment;

[0047] The first processing module is used to use the splicing attribute values ​​used in the pixel attribute features and the cross-sectional field of view attribute features of each segment of the image to resample and fill in zeros on the attribute values ​​of the internal multi-layer images in each segment of the image in the pixel attribute features and the cross-sectional field of view attribute features, so that each segment of the image contains the same image attribute information within the segment;

[0048] The second processing module is used to use the splicing attribute value used by each image segment on the layer spacing attribute feature to resample and fill in zeros on the attribute value of each image segment on the layer spacing attribute feature, so that each image segment contains the same image attribute information between segments.

[0049] Furthermore, the splicing unit comprises:

[0050] A selection module, used for selecting a reference segment image from the segments of images having the same image attribute information according to a direction sequence set by the image position information;

[0051] The splicing module is used to use the reference segment image as a reference image, splice the reference segment image with adjacent segment images according to the image position information, and continuously update the reference segment image during the splicing process until the splicing of all segment images is completed to obtain a spliced ​​image.

[0052] Furthermore, the stitching module is specifically used to determine whether there is an overlapping area between the reference segment image and the adjacent segment image during the stitching process;

[0053] The splicing module is further configured to perform weighted fusion processing on the overlapping area formed by the reference segment image and the adjacent segment image to update the reference segment image if yes; otherwise, perform stacking and splicing on the reference segment image and the adjacent segment image to update the reference segment image.

[0054] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned image stitching method is implemented.

[0055] According to a fourth aspect of the present application, an image stitching device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned image stitching method when executing the program.

[0056] By means of the above technical scheme, the present application provides an image stitching method, device and equipment. Compared with the current prior art method based on first aligning the overlapping areas in the image and then stitching, the present application obtains the marking information associated with each segment of the image to be stitched, and the marking information includes image position information and image attribute information. Further, by traversing and querying the marking information associated with each segment of the image, the image attribute information that meets the stitching conditions is selected as the stitching attribute information of each segment of the image, and each segment of the image is processed into an image containing the same image attribute information according to the stitching attribute information. The images with the same image attribute information are stacked and stitched segment by segment according to the image position information. This process relies on the image position information in the marking information for image stitching, which not only has high stitching efficiency, but also avoids the problem that there is no overlapping area or the overlapping area is very small in the image and the stitching operation cannot be performed, meets the needs of real-time image stitching, and improves the adaptability of image stitching scenarios.

[0057] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0059] Figure 1 A schematic diagram of a process of an image stitching method provided in an embodiment of the present application is shown;

[0060] Figure 2A schematic diagram of a process flow of another image stitching method provided in an embodiment of the present application is shown;

[0061] Figure 3 A flowchart of an image stitching process provided by an embodiment of the present application is shown;

[0062] Figure 4 A schematic diagram of the structure of an image stitching device provided in an embodiment of the present application is shown;

[0063] Figure 5 A structural schematic diagram of another image stitching device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0064] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0065] In the related art, most image stitching processes are implemented by first registering the overlapping areas and then stitching them. However, this method has several disadvantages. First, the image stitching process based on registration is usually time-consuming. If the MRI operator needs to check the stitching results in time during the scanning process to decide whether the patient needs to be re-scanned, the stitching efficiency cannot meet the technician's requirements. Second, the image stitching method based on registration requires a certain overlap area between the images to be stitched. If there is no overlap area or the overlap area is very small, it is difficult to successfully perform the stitching operation. Third, the existing image stitching method is mainly for images obtained by coronal or sagittal scanning, and is usually used for stitching of full spine images. For images scanned in the transverse direction, direct stitching based on the scanned images cannot be achieved. It is also necessary to perform MPR or MIP post-processing reconstruction on the transverse scanned images, first obtain images parallel to the long axis of the human body, and then stitch the obtained images parallel to the long axis of the human body layer by layer. On the one hand, the entire image stitching operation is cumbersome, which affects the image stitching efficiency. On the other hand, the stitching results can no longer be reconstructed by MIP, which cannot meet the clinical needs of observing MPR or MIP reconstructed images of the stitching results from any angle.

[0066] In order to solve this problem, this embodiment provides an image stitching method, such as Figure 1 As shown, this method relies on the marking information in the image to perform the image stitching process, which improves the efficiency of image stitching and avoids the problem that there is no overlapping area or the overlapping area is very small and image stitching cannot be performed. This method can be directly applied to the server of the medical platform, including the following steps:

[0067] 101. Obtain tag information associated with each image segment to be spliced.

[0068] Among them, the image segments to be spliced ​​can be magnetic resonance images obtained by scanning different parts of the human body using medical equipment. The scanning method of the magnetic resonance image is not limited here, and it can be a sagittal or coronal scanning method, or a transverse scanning method. Usually, in order to unify the storage format of images in the medical field, the magnetic resonance images scanned by medical equipment will be stored in a preset standard protocol file. The preset standard protocol is DICOM (Digital Imaging and Communications in Medicine). The DICOM file will treat each layer of the scanned image as an independent file. The DICOM file contains a file header and a data set. The file header contains relevant information for identifying the data set. The data set contains information related to the medical image, such as medical instrument information, image acquisition information, and patient information. It is specifically composed of DICOM data elements arranged in a certain order. The DICOM data elements record the tag information associated with the image. Here, the tag information associated with the image segments to be spliced ​​can be obtained by reading the DICOM file.

[0069] The above-mentioned marking information includes image position information and image attribute information. Images obtained by different scanning methods will store marking information. Here, the image position information may include image position information Image Position Patient and image encoding direction information Image Orientataion Patient. The image attribute information may include image pixel size information Pixel Spacing, layer spacing information Spacing Between Slices, image width information Columns, image height information Rows, etc.

[0070] The executor of the embodiment of the present invention may be an image stitching device, which may be specifically configured on the server side of a medical platform. By acquiring the marking information associated with each image segment to be stitched, the marking information has clear image location information and image attribute information, which can provide a possibility for the subsequent stitching process of each image segment. In the case where there is no overlapping area or the overlapping area is small between the image segments, the stitching operation can be directly performed without a cumbersome stitching process, thereby improving the image stitching efficiency.

[0071] 102. By traversing and querying the tag information associated with each image segment, image attribute information that meets the splicing condition is selected as the splicing attribute information of each image segment.

[0072] Since each segment of the stitched image represents different parts of the human body, different marking information is usually formed in each segment of the image and between each image during the process of using medical equipment to scan each segment of the image layer by layer. Specifically, each layer of the image has different image position information, and each segment of the image has different pixel value size, layer spacing information, cross-sectional field of view size, etc. In order to ensure the image stitching effect, it is necessary to unify the image attribute information of each segment of the image so that the unified image attribute information can be used as the stitching attribute information for subsequent image stitching.

[0073] It can be understood that in the process of traversing and querying the tag information associated with each segment of the image, the image attribute information can be extracted for the tag information associated with each segment of the image. Since the image attribute information may have multiple attributes, the image attribute information with the largest attribute value or the smallest attribute value can be selected here as the splicing attribute information of each segment of the image. For example, each segment of the image has three attributes ABC, and the maximum value of attribute A and the minimum value of attribute B and attribute C can be selected as the splicing attribute information of each segment of the image. It is also possible to perform weighted calculation based on the multiple attribute values ​​contained in the attribute information of each segment of the image to generate image attribute information that meets the splicing conditions as the splicing attribute information of each segment of the image. For example, each segment of the image has three attributes ABC, and the three attribute values ​​of attribute ABC can be weighted and averaged as the splicing attribute information of each segment of the image.

[0074] 103. According to the splicing attribute information of each segment of the image, process each segment of the image into segments of images each containing the same image attribute information.

[0075] Considering that adjacent images need to be consistent during the stitching process, the stitching attribute information is used as the stitching basis here to process each segment of the image into an image containing the same image attribute information. Specifically, intra-segment processing can be performed on each segment of the image, and each layer of the image in the segment can be resampled and zero-filled using the stitching attribute information so that each layer of the image has the same image attribute information; inter-segment image processing can be performed by resampling and zero-filling each image between segments using the stitching attribute information so that the images between segments have the same image attribute information.

[0076] 104. Stack and splice the images having the same image attribute information segment by segment according to the image position information to obtain a spliced ​​image.

[0077] It is understandable that the orientation direction used by the marking information associated with each image segment may be irregular, or the rules between different segments are different. For example, segment A uses the orientation direction from head to toe, and segment B uses the orientation direction from toe to head. In the image stitching process, each image segment needs to use the same orientation direction. Here, before stitching the image segments, each image segment can be sorted using a unified orientation direction according to the image position information so that each image segment uses the same orientation direction.

[0078] The image stitching method provided in the embodiment of the present application is compared with the method based on first aligning the overlapping areas in the image and then stitching in the current prior art. The present application obtains the marking information associated with each segment of the image to be stitched, and the marking information includes image position information and image attribute information. The marking information associated with each segment of the image is further traversed and queried, and the image attribute information that meets the stitching conditions is selected as the stitching attribute information of each segment of the image, and each segment of the image is processed into an image containing the same image attribute information according to the stitching attribute information. The images with the same image attribute information are stacked and stitched segment by segment according to the image position information. This process relies on the image position information in the marking information for image stitching, which not only has high stitching efficiency, but also avoids the problem that there is no overlapping area or the overlapping area is very small in the image and the stitching operation cannot be performed, meets the demand for real-time image stitching, and improves the adaptability of image stitching scenarios.

[0079] It should be added that the embodiments of the present invention do not limit the medical equipment to which the above-mentioned image stitching method is applicable. In actual application scenarios, since magnetic resonance imaging currently uses multiple segments for scanning, the above-mentioned image stitching process can be applied to medical equipment for magnetic resonance whole-body imaging, but the above-mentioned image stitching process is also applicable to each segment image obtained by segmented scanning using other medical equipment, for example, each segment image obtained by segmented scanning of CT medical equipment and each segment image obtained by segmented scanning of PET medical equipment can all be applied to the above-mentioned image stitching process.

[0080] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, this embodiment provides another image stitching method, such as Figure 2 As shown, the method includes:

[0081] 201. By querying the image data imported from the preset standard protocol file, label data of the standard format image included in each segment of the image to be spliced ​​is read from the image data.

[0082] It can be understood that the preset standard protocol here is equivalent to the standard output format of medical images. For medical image files of different formats generated by various medical imaging devices, a complete storage mechanism is set up in the preset standard protocol file, which can store the image data in the medical image files in a unified format.

[0083] Since the preset standard protocol file organizes and stores the information in the image according to the DICOM standard, the label data of the standard format image contained in each segment of the image to be spliced ​​in the preset standard protocol file can be read by programming, and the label data is specified in accordance with the DICOM standard.

[0084] 202. Obtaining label information associated with the image for label data of the standard format image contained in each image segment, and obtaining label information associated with each image segment to be spliced.

[0085] Since image position information and image attribute information are needed in the image stitching process, the label data related to image position information and image attribute information are mainly obtained here. As the mark information associated with the image, the label data related to the image position information mainly includes the image position and the image encoding direction. The image position indicates the image coordinates of the first pixel in the upper left corner of the image, that is, the first pixel coordinates transmitted by the DICOM file. The image direction indicates the direction cosine value of the first row and the first column of the image relative to the patient. The direction of the coordinate axis is determined according to the patient's direction. The label data related to the image attribute information mainly includes image pixels, layer spacing, image width and image height, etc. The layer spacing refers to the distance between two scanned layer images in the image.

[0086] 203. Perform intra-segment sorting and inter-segment sorting on the image segments according to the direction sequence set by the image position information to obtain image segments with the same sorting rule.

[0087] The direction order set by the image position here is equivalent to the human body orientation order. The direction from the human head to the legs can be selected, or the direction from the human legs to the head can be selected. For intra-segment sorting, the scanning layer images can be sorted according to the position of each scanning layer image in each segment image and the image encoding direction according to the set direction order, so that each scanning layer in each image uses a unified human body orientation order. For inter-segment sorting, each segment image is also sorted according to the position of each segment image and the image encoding direction according to the set direction order, so that each segment image uses a unified human body orientation order, thereby obtaining each segment image with the same sorting rule.

[0088] 204 , traverse and query the tag information associated with each image segment, and respectively extract the attribute values ​​marked on different attribute features of each image segment.

[0089] 205. By comparing the attribute values ​​marked on different attribute features of the image segments, image attribute information that meets the splicing condition is selected as the splicing attribute information of each image segment.

[0090] Among them, the image attribute information records the attribute values ​​marked on different attribute features of the image. Since the scanning parameters used in the layer-by-layer scanning process of each segment of the image may be different, the attribute values ​​marked on different attribute features between each segment of the image may be different. The attribute features here mainly involve the image in pixel attribute features, layer spacing attribute features and cross-sectional field of view attribute features.

[0091] As a preferred implementation, the attribute values ​​marked on different attribute features of each image segment can be compared, and the minimum attribute value marked on the pixel attribute feature and the layer spacing attribute feature of each image segment can be selected respectively, and the maximum attribute value of the cross-sectional field of view feature attribute calculated using the attribute value marked on the preset attribute feature of the image can be selected as the splicing attribute information of each image segment.

[0092] It can be understood that the above-mentioned pixel attribute features and layer spacing attribute features can be directly read from the tag information associated with the image, while the cross-sectional field of view feature attributes cannot be directly read from the tag information associated with the image, but need to be indirect attribute features obtained by calculating from the attribute values ​​of the image on the preset attribute tags.

[0093] 206. Process the image segments into images each containing the same image attribute information according to the splicing attribute information of the image segments.

[0094] Since the attribute features required for the internal and external parts of each image segment in the image stitching process are different, here we can specifically extract the stitching attribute values ​​used in the pixel attribute features, layer spacing attribute features and cross-sectional field of view attribute features of each image segment according to the stitching attribute information of each image segment, and use the stitching attribute values ​​used in the pixel attribute features and cross-sectional field of view attribute features of each image segment to resample and fill in zeros on the attribute values ​​of the internal multi-layer images in each image segment in the pixel attribute features and cross-sectional field of view attribute features, so that each image segment contains the same image attribute information within the segment, and use the stitching attribute values ​​used in the layer spacing attribute features of each image segment to resample and fill in zeros on the attribute values ​​in the layer spacing attribute features of each image segment, so that each image segment contains the same image attribute information between segments.

[0095] The resampling and zero-filling process here can be to use an interpolation algorithm to perform difference on the image layer by layer, and to perform a zero-filling operation on the expanded field of view. After the interpolation and zero-filling process, each layer of the image has the same attribute value in the cross-sectional field of view attribute characteristics and the image pixel attribute characteristics.

[0096] 207. Select a reference segment image from the segments of images having the same image attribute information according to a direction sequence set by the image position information.

[0097] 208. Using the reference segment image as a reference image, splicing the reference segment image with adjacent segment images according to the image position information, and continuously updating the reference segment image during the splicing process until all segment images are spliced ​​to obtain a spliced ​​image.

[0098] Specifically, in the process of stacking and stitching each segment image after the sorting processing, a reference segment image can be set, and the adjacent segment images are stitched with the reference segment image according to the image position information, and the reference segment image is updated. Here, the reference segment image is equivalent to the benchmark image for stitching, that is, which segment image is used as the benchmark for stitching, and the stitching process is to stitch two by two segment by segment. For example, the image to be stitched includes four segments A, B, C, and D, corresponding to the position of the human body from foot to head or from head to foot. When stitching, image A is first used as the reference segment image to stitch images A and B, and the result of stitching is image AB. Then, image AB is used as the updated reference segment image to stitch images AB and C to obtain image ABC. Then, image ABC is used as the updated reference segment image to stitch images ABC and D to obtain image ABCD, thereby obtaining the stitched image.

[0099] Furthermore, considering the overlapping areas between the image segments, specifically in the process of updating the reference segment image, it can be determined in the stitching process whether the reference segment image has overlapping areas with the adjacent segment images. If so, weighted fusion processing is performed on the overlapping areas formed by the reference segment image and the adjacent segment images, and the reference segment image is updated. Otherwise, the reference segment image and the adjacent segment images are stacked and stitched, and the reference segment image is updated.

[0100] In practical application scenarios, the image stitching process can be Figure 3 As shown, firstly, the image information is obtained, that is, the marking information associated with each image segment to be stitched, and the images are sorted. Here, the images can be sorted according to the direction order set by the image position information, and then the stitching attribute information is determined. The stitching attribute information is the same image attribute information used in the stitching process, and then the image layer is resampled. Specifically, each image segment is processed into an image with the same image attribute information using the stitching attribute information, and then the reference segment is resampled between layers, and each image segment is stitched according to the position layer resampling. Specifically, the same layer spacing in the stitching attribute information is used to stack and stitch each image segment layer by layer to obtain the final image stitching result.

[0101] It can be understood that, specifically, after determining the stitching attribute information, one method is to directly perform intra-segment and inter-segment processing on each segment image, so that the same attribute information is used within and between the segments of each segment image, and then directly stack and stitch the segments image. Another method is to first perform intra-segment processing on each segment image, so that the same attribute information is used within the segment of each segment image, select a reference segment image, and resample the segments image to the layer spacing information in the stitching attribute information during the stitching process, and then stack and stitch the reference segment image with the adjacent segment image according to the position of each segment image. Stitching here refers to stacking and sorting the image layer according to the position information, and in the stitching process, according to the position information of each segment image, it is determined whether there is an overlapping area between the segment images. If there is no overlapping area, the images are directly stacked and sorted according to the position information. If there is an overlapping area, the overlapping area is weighted fused to obtain updated image data, and the non-overlapping area data remains unchanged. The result is used as the updated reference segment image, and the stitching operation is repeated on the updated reference segment image and the adjacent segment image in turn until the stitching of all segment images is completed to obtain the stitched image.

[0102] Further, as Figure 1-Figure 2 The specific implementation of the method, the embodiment of the present application provides an image stitching device, such as Figure 4 As shown, the device includes: an acquisition unit 31, a selection unit 32, a processing unit 33, and a splicing unit 34.

[0103] An acquisition unit, used to acquire tag information associated with each segment of the image to be spliced, wherein the tag information includes image position information and image attribute information;

[0104] A selection unit, configured to select image attribute information that meets the splicing condition as the splicing attribute information of each image segment by traversing and querying the tag information associated with each image segment;

[0105] A processing unit, configured to process the image segments into images each containing the same image attribute information according to the splicing attribute information of the image segments;

[0106] The splicing unit is used to stack and splice the images with the same image attribute information segment by segment according to the image position information to obtain a spliced ​​image.

[0107] The image stitching device provided by the embodiment of the present invention is compared with the method of first aligning the overlapping areas in the image and then stitching in the existing technology. The present application obtains the marking information associated with each segment of the image to be stitched, and the marking information includes image position information and image attribute information. Further, by traversing and querying the marking information associated with each segment of the image, the image attribute information that meets the stitching conditions is selected as the stitching attribute information of each segment of the image, and each segment of the image is processed into an image containing the same image attribute information according to the stitching attribute information. The images with the same image attribute information are stacked and stitched segment by segment according to the image position information. This process relies on the image position information in the marking information for image stitching, which not only has high stitching efficiency, but also avoids the problem that there is no overlapping area in the image or the overlapping area is very small and the stitching operation cannot be performed, meets the demand for real-time image stitching, and improves the adaptability of image stitching scenarios.

[0108] In specific application scenarios, such as Figure 5 As shown, the acquisition unit 31 includes:

[0109] The reading module 311 may be used to read the label data of the standard format image contained in each segment of the image to be spliced ​​from the image data by querying the image data imported from the preset standard protocol file;

[0110] The acquisition module 312 may be used to acquire the tag information associated with the image according to the tag data of the standard format image contained in each segment of the image, so as to obtain the tag information associated with each segment of the image to be spliced.

[0111] In specific application scenarios, such as Figure 5 As shown, the device also includes:

[0112] The sorting unit 35 may be used to, after acquiring the tag information associated with each segment of the image to be spliced, sort the image segments within and between segments according to the direction sequence set by the image position information to obtain the image segments with the same sorting rule.

[0113] In specific application scenarios, such as Figure 5 As shown, the image attribute information records the attribute values ​​marked on the image on different attribute features, and the selection unit 32 includes:

[0114] The query module 321 can be used to traverse and query the tag information associated with each image segment, and respectively extract the attribute values ​​marked on different attribute features of each image segment;

[0115] The comparison module 322 may be used to select image attribute information that meets the splicing condition as the splicing attribute information of each image segment by comparing the attribute values ​​marked on different attribute features of each image segment.

[0116] In a specific application scenario, the comparison module 322 can be specifically used to compare the attribute values ​​marked on different attribute features of each image segment, select the minimum attribute value marked on the pixel attribute feature and the layer spacing attribute feature for each image segment, and select the maximum attribute value of the cross-sectional field of view feature attribute calculated using the attribute value marked on the preset attribute feature of the image as the splicing attribute information of each image segment.

[0117] In specific application scenarios, such as Figure 5 As shown, the processing unit 33 includes:

[0118] The extraction module 331 can be used to extract the splicing attribute values ​​used in the pixel attribute features, the layer spacing attribute features and the cross-sectional field of view attribute features of each image segment according to the splicing attribute information of each image segment;

[0119] The first processing module 332 may be used to use the splicing attribute values ​​used in the pixel attribute features and the cross-sectional field of view attribute features of each image segment to resample and fill in zeros on the attribute values ​​of the internal multi-layer images in each image segment in the pixel attribute features and the cross-sectional field of view attribute features, so that each image segment contains the same image attribute information in the segment;

[0120] The second processing module 333 can be used to use the splicing attribute value used in the layer spacing attribute feature of each segment image to resample and fill in zeros on the attribute value of the layer spacing attribute feature of each segment image, so that each segment image contains the same image attribute information between segments.

[0121] In specific application scenarios, such as Figure 5 As shown, the splicing unit 34 includes:

[0122] A selection module 341 may be used to select a reference segment image from the segments of images having the same image attribute information according to a direction sequence set by the image position information;

[0123] The stitching module 342 can be used to use the reference segment image as a reference image, stitch the reference segment image with adjacent segment images according to the image position information, and continuously update the reference segment image during the stitching process until all segment images are stitched together to obtain a stitched image.

[0124] In a specific application scenario, the stitching module 342 may be used to determine whether there is an overlapping area between the reference segment image and the adjacent segment image during the stitching process;

[0125] The stitching module 342 may be specifically configured to perform weighted fusion processing on the overlapping area formed by the reference segment image and the adjacent segment image to update the reference segment image if yes; otherwise, stack and stitch the reference segment image and the adjacent segment image to update the reference segment image.

[0126] It should be noted that for other corresponding descriptions of the functional units involved in the image stitching device applicable to the server side provided in this embodiment, reference can be made to Figure 1 and Figure 2 The corresponding description in will not be repeated here.

[0127] Based on the above Figure 1-Figure 2 The method shown in the embodiment of the present application accordingly provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned Figure 1-Figure 2 The image stitching method shown;

[0128] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0129] Based on the above Figure 1-Figure 2 The method shown, and Figure 4-Figure 5 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a server-side entity device, which can be a computer, a server, or other network devices, etc. The entity device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1-Figure 2 The image stitching method shown.

[0130] Optionally, the above-mentioned physical devices may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0131] Those skilled in the art will appreciate that the physical device structure for image stitching provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0132] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device for image stitching, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between components within the storage medium, and communication with other hardware and software in the information processing physical device.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or by hardware. By applying the technical solution of the present application, compared with the current existing methods, the present application can rely on the image position information in the marking information for image stitching, which not only has high stitching efficiency, but also avoids the problem that there is no overlapping area in the image or the overlapping area is very small and the stitching operation cannot be performed, meets the needs of real-time image stitching, and improves the adaptability of image stitching scenes.

[0134] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0135] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. An image stitching method, characterized in that: include: Acquire tag information associated with each segment of the image to be stitched, wherein the tag information includes image position information and image attribute information; By traversing and querying the tag information associated with each segment of the image, image attribute information that meets the splicing condition is selected as the splicing attribute information of each segment of the image, specifically, image attribute information with the largest attribute value or the smallest attribute value is selected from the image attribute information as the splicing attribute information of each segment of the image; According to the splicing attribute information of each segment of the image, the segment of the image is processed into each segment of the image containing the same image attribute information, specifically, intra-segment processing is performed on each segment of the image, and each layer of the image in the segment is resampled and zero-filled using the splicing attribute information, so that each layer of the image has the same image attribute information; inter-segment images are processed, and each inter-segment image is resampled and zero-filled using the splicing attribute information, so that the inter-segment images have the same image attribute information; The images having the same image attribute information are stacked and spliced ​​segment by segment according to the image position information to obtain a spliced ​​image.

2. The method according to claim 1, characterized in that The step of obtaining the tag information associated with each segment of the image to be spliced ​​specifically includes: By querying the image data imported from the preset standard protocol file, the label data of the standard format image contained in each segment of the image to be spliced ​​is read from the image data; The label information associated with the image is obtained for the label data of the standard format image contained in each segment of the image, so as to obtain the label information associated with each segment of the image to be spliced.

3. The method according to claim 1, characterized in that After obtaining the tag information associated with each segment of the image to be spliced, the method further includes: The image segments are sorted within and between segments according to the direction sequence set by the image position information to obtain image segments with the same sorting rule.

4. The method according to claim 1, characterized in that: The image attribute information records the attribute values ​​marked on the image on different attribute features. The traversal query of the tag information associated with each segment of the image selects the image attribute information that meets the splicing condition as the splicing attribute information of each segment of the image, specifically including: Traversing and querying the tag information associated with each segment of the image, respectively extracting the attribute values ​​marked on different attribute features of each segment of the image; By comparing the attribute values ​​marked on the different attribute features of the image segments, the image attribute information that meets the splicing condition is selected as the splicing attribute information of the image segments.

5. The method according to claim 4, characterized in that The selecting of image attribute information that meets the splicing condition as the splicing attribute information of each image segment by comparing the attribute values ​​marked on different attribute features of each image segment specifically includes: By comparing the attribute values ​​marked on different attribute features of each image segment, the minimum attribute value marked on the pixel attribute feature and the layer spacing attribute feature of each image segment is selected respectively, and the maximum attribute value of the cross-sectional field of view feature attribute calculated using the attribute value marked on the preset attribute feature of the image is selected as the splicing attribute information of each image segment.

6. The method according to any one of claims 1 to 5, characterized in that The processing of the image segments into images each containing the same image attribute information according to the splicing attribute information of the image segments specifically includes: According to the splicing attribute information of each segment of the image, respectively extracting the splicing attribute value used in the pixel attribute feature, the layer spacing attribute feature and the cross-sectional field of view attribute feature of each segment of the image; Using the splicing attribute values ​​used in the pixel attribute features and the cross-sectional field of view attribute features of each image segment, the attribute values ​​of the internal multi-layer images in each image segment in the pixel attribute features and the cross-sectional field of view attribute features are resampled and zero-filled, so that each image segment contains the same image attribute information within the segment; By using the splicing attribute value used in the layer spacing attribute feature of each image segment, the attribute value of each image segment in the layer spacing attribute feature is resampled and filled with zeros, so that each image segment contains the same image attribute information between segments.

7. The method according to any one of claims 1 to 5, characterized in that The step of stacking and splicing the images having the same image attribute information segment by segment according to the image position information to obtain the spliced ​​image specifically includes: Selecting a reference segment image from the segments of images having the same image attribute information according to a direction sequence set by the image position information; The reference segment image is used as a reference image, and the reference segment image is spliced ​​with adjacent segment images according to the image position information. During the splicing process, the reference segment image is continuously updated until all segment images are spliced ​​to obtain a spliced ​​image.

8. The method according to claim 7, characterized in that: The continuously updating the reference segment image during the stitching process specifically includes: During the stitching process, determining whether there is an overlapping area between the reference segment image and the adjacent segment image; If yes, weighted fusion processing is performed on the overlapping area formed by the reference segment image and the adjacent segment image to update the reference segment image; otherwise, stacking and splicing are performed on the reference segment image and the adjacent segment image to update the reference segment image.

9. An image stitching device, characterized in that: include: An acquisition unit, used to acquire tag information associated with each segment of the image to be spliced, wherein the tag information includes image position information and image attribute information; A selection unit is used to select image attribute information that meets the splicing condition as the splicing attribute information of each image segment by traversing and querying the tag information associated with each image segment, and specifically select image attribute information with the largest attribute value or the smallest attribute value from the image attribute information as the splicing attribute information of each image segment; a processing unit, configured to process the image segments into images containing the same image attribute information according to the splicing attribute information of the image segments, and specifically perform intra-segment processing on each image segment, by resampling and zero-filling operations on each layer of images within the segment using the splicing attribute information, so that each layer of images has the same image attribute information; and perform processing on inter-segment images, by resampling and zero-filling operations on each image between segments using the splicing attribute information, so that the images between segments have the same image attribute information; The splicing unit is used to stack and splice the images with the same image attribute information segment by segment according to the image position information to obtain a spliced ​​image.

10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image stitching method according to any one of claims 1 to 8 is implemented.

11. An image stitching device, comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the program, the image stitching method according to any one of claims 1 to 8 is implemented.

Images