Image deformation method, device, equipment and storage medium based on interpolation algorithm

Abstract

This application relates to an image deformation method, apparatus, device, and storage medium based on an interpolation algorithm. The method comprises: obtaining an image to be processed and selecting a corresponding number of deformation parameters based on processing requirements; processing the image to be processed based on the deformation parameters and an interpolation algorithm to obtain a deformed image; expanding the deformation parameters to obtain expanded deformation parameters; obtaining weights based on the expanded deformation parameters, and obtaining a target image based on the deformed image and the weights. This application ensures conservation of image integral values ​​based on existing interpolation algorithms, resolving the issue of achieving smooth image deformation while failing to ensure conservation of integral values ​​before and after deformation.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image deformation method, apparatus, device and storage medium based on an interpolation algorithm. Background Art

[0002] Currently, many technologies in the field of image processing involve image deformation and image interpolation algorithms. The quality of the interpolation algorithm directly affects the degree of image distortion. Due to the limitations of existing interpolation algorithms, which use the values ​​of surrounding pixels to perform different linear or nonlinear combinations to achieve interpolation, the interpolated pixel value is not equal to the sum of the contribution values ​​of the pixels involved in the interpolation. Therefore, when performing image deformation based on the interpolation algorithm, the overall integral value of the deformed image is often not conserved, which is directly reflected in the non-conservation of the image brightness before and after deformation. For some specific images, such as during motion correction of coronary CT images, brightness non-conservation can affect the changes in the CT value of the corrected image. These changes can easily affect the doctor's diagnosis in clinical practice.

[0003] Since the related technology cannot ensure the conservation of the integral value of the image during the deformation process, it is necessary to adjust the brightness of the deformed image. This is very cumbersome in processing and cannot be directly applied to existing interpolation algorithms.

[0004] Currently, no effective solution has been proposed to the problem that in related technologies, while achieving smooth image deformation, conservation of integral values ​​before and after deformation cannot be guaranteed. Summary of the Invention

[0005] In this embodiment, an image deformation method, apparatus, device and storage medium based on an interpolation algorithm are provided to solve the problem in related technologies that, while achieving smooth deformation of an image, conservation of integral values ​​before and after deformation cannot be guaranteed.

[0006] First, in this embodiment, a method for image deformation based on an interpolation algorithm is provided, which is characterized by comprising:

[0007] Obtain the image to be processed and select the corresponding number of deformation parameters according to the processing requirements;

[0008] Based on the deformation parameters and the interpolation algorithm, the image to be processed is processed to obtain a deformed image;

[0009] Expanding the deformation parameters to obtain expanded deformation parameters;

[0010] A weight is obtained according to the expanded deformation parameters, and a target image is obtained based on the deformed image and the weight value.

[0011] In some embodiments, processing the image to be processed to obtain the deformed image based on the deformation parameters and the interpolation algorithm includes:

[0012] Normalizing all coordinates of the image to be processed to obtain normalized coordinates;

[0013] Performing deformation processing on the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed;

[0014] Based on the deformation coordinates, sampling and interpolation processing is performed on the image to be processed by an interpolation algorithm to obtain the deformed image.

[0015] In some embodiments, deforming the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed after deformation includes:

[0016] The deforming the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed after deformation, includes:

[0017] determining a method for acquiring the deformation coordinates according to whether a preset condition is met;

[0018] The preset condition is that the number of the deformation parameters and the size of the image to be processed must meet a preset relationship.

[0019] In some embodiments, deforming the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed after deformation includes:

[0020] When the preset condition is met, deformation processing is performed on each of the normalized coordinates according to the deformation parameter to directly obtain the deformed coordinates.

[0021] In some embodiments, deforming the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed after deformation includes:

[0022] When the preset condition is not met, partitioning the image to be processed to obtain a plurality of image blocks; deforming the normalized coordinates located at the vertices of the image blocks according to the deformation parameters, and obtaining the corresponding deformation coordinates through a positioning function;

[0023] Based on the obtained deformation coordinates, all deformation coordinates are obtained through an interpolation algorithm.

[0024] In some embodiments, based on the deformation coordinates, sampling and interpolating the image to be processed by an interpolation algorithm to obtain the deformed image includes:

[0025] According to the acquired deformation coordinates of each pixel point, sampling and interpolation processing is performed by selecting an interpolation algorithm to obtain the deformed image.

[0026] In some embodiments, the step of expanding the deformation parameters to obtain expanded deformation parameters, obtaining weights according to the expanded deformation parameters, and obtaining a target image based on the deformed image and the weights includes:

[0027] Expanding the number of the deformation parameters to a number of deformation parameters corresponding to the coordinates in the image to be processed based on an interpolation algorithm;

[0028] Calculating the expanded deformation parameters to obtain corresponding weights; the weights represent the ratio of the integral values ​​of the image to be processed before and after the deformation processing;

[0029] Based on the weight value and the deformed image, a target image with a conserved integral value after deformation is calculated.

[0030] In a second aspect, an image deformation device based on an interpolation algorithm is provided in this embodiment, comprising: an acquisition module, a deformation module, and an integral value conservation module;

[0031] The acquisition module is used to acquire the image to be processed and select a corresponding number of deformation parameters according to processing requirements;

[0032] The deformation module is used to process the image to be processed to obtain a deformed image based on the deformation parameters and the interpolation algorithm;

[0033] The integral value conservation module is used to expand the deformation parameters to obtain expanded deformation parameters; obtain weights according to the expanded deformation parameters, and obtain a target image based on the deformed image and the weight values.

[0034] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image deformation method based on the interpolation algorithm described in the first aspect is implemented.

[0035] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the image deformation method based on the interpolation algorithm described in the first aspect is implemented.

[0036] Compared with related technologies, an image deformation method, device, equipment and storage medium based on an interpolation algorithm provided in this embodiment obtains an image to be processed and selects a corresponding number of deformation parameters according to processing requirements; based on the deformation parameters and the interpolation algorithm, the image to be processed is processed to obtain a deformed image; the deformation parameters are expanded to obtain expanded deformation parameters; weights are obtained according to the expanded deformation parameters, and a target image is obtained based on the deformed image and the weight values. This solves the problem of being unable to ensure the conservation of the integral value before and after deformation while achieving smooth deformation of the image, and achieves the effect of being able to simultaneously ensure the conservation of the image integral value before and after deformation when performing image deformation processing based on the interpolation algorithm.

[0037] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 FIG1 is a hardware structure diagram of an image deformation method based on an interpolation algorithm in one embodiment;

[0040] Figure 2 is a flow chart of an image deformation method based on an interpolation algorithm in one embodiment;

[0041] Figure 3 Schematic diagram of a partition of an image to be processed with a height and width equal to 5 in one embodiment;

[0042] Figure 4 is a flow chart of an image deformation method based on an interpolation algorithm in a preferred embodiment;

[0043] Figure 5 FIG. 4 is a structural block diagram of an image deformation device based on an interpolation algorithm in one embodiment.

[0044] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 510, acquisition module; 520, deformation module; 530, integral value conservation module. DETAILED DESCRIPTION

[0045] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0047] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a block diagram of the hardware structure of the terminal of the image deformation method based on the interpolation algorithm of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0048] Memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the interpolation algorithm-based image deformation method in this embodiment. Processor 102 executes the computer programs stored in memory 104 to execute various functional applications and data processing, thereby implementing the above-mentioned method. Memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, and such remote memory may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. The network may include a wireless network provided by the terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0050] Currently, many image processing technologies involve image deformation and interpolation algorithms, such as image registration, motion simulation, motion correction, and video or photographic image processing. The quality of the interpolation algorithm directly affects the degree of image distortion. Due to the limitations of existing interpolation algorithms, which use linear or nonlinear combinations of the values ​​of surrounding pixels to achieve interpolation, the interpolated pixel value is not equal to the sum of the contributions of the pixels involved in the interpolation. Therefore, when using interpolation algorithms to deform an image, the overall integral value of the deformed image is often not conserved, which is directly reflected in the difference in brightness between the image before and after deformation.

[0051] In order to solve the above problems, an image deformation method based on an interpolation algorithm is provided in the following embodiment, which can achieve conservation of the integral value before and after image deformation on the basis of the existing interpolation algorithm.

[0052] In this embodiment, an image deformation method based on an interpolation algorithm is provided. Figure 2 is a flow chart of the method of this embodiment, such as Figure 2 As shown, the method includes the following steps:

[0053] Step S210: Obtain an image to be processed and select a corresponding number of deformation parameters according to processing requirements.

[0054] Specifically, the image to be processed and the size of the image to be processed are obtained, and deformation parameters are obtained by outputting the training results of network learning. Further, according to the processing requirements, the network training output parameters are set to obtain a corresponding number of deformation parameters. The processing requirements include overall deformation processing or partitioned deformation processing of the entire image to be processed. In the overall deformation processing, the image to be processed can be deformed as a whole by one deformation parameter or multiple identical deformation parameters. In the partitioned deformation processing, the image to be processed can be partitioned, and each image block can be deformed separately by multiple non-complete deformation parameters. Furthermore, a specific area of ​​the image to be processed can be deformed. By setting the corresponding deformation parameters, the image portion other than the specific area is kept unchanged, and deformation occurs only in the specific area.

[0055] Generally, under ideal conditions, the more deformation parameters there are, the better the deformation effect will be. However, in actual applications, the selection of the number of deformation parameters also needs to consider factors such as computational complexity. Moreover, in partition deformation processing, in order to achieve the partition effect of the image to be processed, the number of deformation parameters also needs to be set appropriately according to the size of the image to be processed.

[0056] Step S220 : Based on the deformation parameters and the interpolation algorithm, the image to be processed is processed to obtain a deformed image.

[0057] Specifically, based on the deformation parameters obtained in the above steps, the image to be processed is deformed, and the coordinates of the pixels in the image to be processed after being deformed by all the deformation parameters are obtained. Based on the deformed coordinates, all coordinates in the deformed image are obtained, and sampling and interpolation processing is then performed on the image to obtain the pixel values ​​corresponding to all coordinates in the deformed image, thereby obtaining the deformed image. The sampling and interpolation processing can be implemented using various existing interpolation algorithms, such as bilinear interpolation, nearest neighbor interpolation, thin plate spline interpolation, and bicubic interpolation.

[0058] Step S230 , expanding the deformation parameters to obtain expanded deformation parameters.

[0059] Step S240 , obtaining weights according to the expanded deformation parameters, and obtaining a target image based on the deformed image and the weights.

[0060] Specifically, the existing number of deformation parameters is expanded through the interpolation algorithm to obtain the number of deformation parameters corresponding to the coordinates in the image to be processed. The determinant value of each deformation parameter after interpolation and expansion is calculated to obtain the corresponding determinant matrix, and the determinant matrix is ​​used as the weight of the deformed image.

[0061] The weight represents the area ratio of the deformed mesh that changes when the processed image is deformed into the deformed image. This can also be understood as the ratio of the integral value of the processed image before and after deformation, or the ratio of the light intensity of the processed image before and after deformation. Furthermore, the number of coordinates in the processed image is determined by the size of the processed image; each pixel in the processed image can be considered a coordinate.

[0062] Furthermore, matrix operations are performed on the deformed image and the determinant matrix, which is equivalent to performing deformation processing on the basis of the image to be processed. At the same time, the determinant matrix is ​​added as a weight, and the pixel value of each pixel coordinate of the deformed image is multiplied by the corresponding weight value to obtain the target image with conserved integral value after deformation.

[0063] Since the related technology cannot ensure the conservation of the integral value of the image during the deformation process, it is necessary to adjust the brightness of the deformed image. This is very cumbersome in processing and cannot be directly applied to existing interpolation algorithms.

[0064] This embodiment effectively supplements the existing technology. Through the above steps, a corresponding number of deformation parameters are selected to perform deformation processing on the image to be processed to obtain a deformed image. The existing number of deformation parameters are interpolated and expanded to obtain a number of deformation parameters corresponding to the coordinates of the image to be processed. The determinant matrix corresponding to these deformation parameters can represent the ratio of the integral values ​​before and after deformation. Combined with the determinant matrix, the deformed image is calculated and processed to obtain a target image with conserved integral values. The target image can be flexibly applied to existing interpolation algorithms and inserted into existing neural networks for back propagation and training. It can not only achieve smooth deformation of the image in shape, but also achieve conservation of the integral value of the image, thereby solving the problem of being unable to ensure conservation of the integral value before and after deformation while achieving smooth deformation of the image.

[0065] Since the processing requirement may be an overall deformation processing or a partition deformation processing, in the following embodiments, how to perform the partition deformation of the image to be processed is further described in detail.

[0066] First, when the processing requirement is to perform partition deformation, a corresponding number of deformation parameters are selected according to the complexity of the processing requirement and the size of the image to be processed, and the deformation parameters are set to be partially different or all different.

[0067] Specifically, since the image to be processed needs to be deformed in different areas, the deformation parameters are first set to be partially or completely different. That is, when deforming the areas, the deformation parameters are not completely set to ensure that different parts of the image to be processed can be deformed differently when partitioning.

[0068] Furthermore, in order to ensure that the image to be processed can be evenly partitioned, it is necessary to select a corresponding number of deformation parameters according to the size of the image to be processed (the height and width of the image to be processed), for example, Figure 3 is a schematic diagram of the partition of an image to be processed with a height and width equal to 5, such as Figure 3 As shown in the figure, an image to be processed with a height and width of 5 is divided into 4 image blocks. The partitioned image blocks (the areas shown by the solid lines in the figure) have a total of 9 pixel coordinates, and 9 deformation parameters can be set accordingly. In addition, considering the computational complexity in actual application, the size of the partitions can be adjusted at any time to a partition size that ensures accurate deformation while being suitable for the current hardware level. For example, an image to be processed with a height and width of 5 can be divided into 16 image blocks. The partitioned image blocks have a total of 25 pixel coordinates, and 25 deformation parameters can be set accordingly.

[0069] It should be noted that the above process of setting the number of deformation parameters can directly obtain the corresponding number of deformation parameters by controlling the output of the network training results.

[0070] In this embodiment, when performing partition deformation, a corresponding number of deformation parameters are output through network training, so that different deformations can be performed on different parts of the image to be processed while ensuring accurate deformation and meeting the partition size suitable for the existing hardware level.

[0071] In some embodiments, the above-mentioned processing of the image to be processed based on the deformation parameters and the interpolation algorithm to obtain the deformed image includes the following steps:

[0072] Step S310 , normalizing all coordinates of the image to be processed to obtain normalized coordinates.

[0073] Specifically, we create two column vectors between [-1, 1] based on the height and width of the image to be processed. This means that all coordinates on the image to be processed are normalized to the range [-1, 1] to obtain normalized coordinates. All coordinates can be considered as all pixels on the image to be processed. Assuming the height and width of the image to be processed are H and W, respectively, there are H times W coordinates in the corresponding figure.

[0074] In step S320 , the normalized coordinates are deformed based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed.

[0075] Specifically, affine transformations include scaling, translation, rotation, reflection, and shear mapping. After an affine transformation, straight lines remain straight lines, and parallel lines remain parallel lines. By using the affine transformation matrix as the deformation parameter and applying an affine transformation to the normalized coordinates based on the affine transformation matrix, the deformed coordinates of the image to be processed are obtained.

[0076] Each time an affine transformation is performed, an affine map of the same size as the image to be processed and all the deformed coordinates on the affine map after the affine transformation will be obtained. Through this step, after all the affine transformation matrices are affine transformed, the same number of affine maps as the number of affine transformation matrices and all the deformed coordinates on the affine map after the affine transformation will be obtained.

[0077] Furthermore, the method for obtaining the deformation coordinates is determined based on whether the preset conditions are met, wherein the preset conditions are that the number of deformation parameters and the size of the image to be processed must conform to a preset relationship. Specifically, the number of deformation parameters is expressed as n multiplied by m, wherein n and m may be the same or different. When n and m are the same, if n is equal to the height and width of the image to be processed at the same time, it is considered that the preset conditions are met; if n is not equal to the height and width of the image to be processed at the same time, it is considered that the preset conditions are not met.

[0078] For the case where the preset conditions are met, where n is equal to the height and width of the image to be processed, it is equivalent to partitioning each pixel point, deforming each normalized coordinate according to the deformation parameter, and directly obtaining the deformed coordinate.

[0079] Specifically, it also includes the steps of: creating an intermediate image corresponding to the image to be processed, and obtaining the deformation coordinates corresponding to each pixel point in the intermediate image.

[0080] Specifically, when n is equal to the height and width of the image to be processed, it is equivalent to partitioning at each pixel point. Then, by traversing each point in the intermediate image, the corresponding affine map is directly found, and the corresponding accurate coordinates in the affine map are obtained as the deformation coordinates. Figure 3 Taking the image to be processed with a height and width both equal to 5 as an example, when the number of deformation parameters is 25 (n=5), each normalized coordinate of the image to be processed is deformed directly based on all deformation parameters, and 25 affine maps and 25 deformed coordinates on each map are obtained accordingly. The deformation coordinates of the corresponding position in the corresponding affine map are directly obtained according to the coordinate point.

[0081] For cases where the preset conditions are not met, where n is not equal to both the height and width of the image to be processed, the following steps are performed to obtain the deformation coordinates:

[0082] The image to be processed is partitioned to obtain several image blocks; the normalized coordinates of the vertices of the image blocks are deformed according to the deformation parameters, and the corresponding deformation coordinates are obtained through the positioning function;

[0083] Based on the obtained deformation coordinates, all deformation coordinates are obtained through interpolation algorithm.

[0084] Specifically, partitioning is based on the number of deformation parameters (n*n), dividing the image to be processed into (n-1)*(n-1) uniform image blocks, and then deforming the normalized coordinates of the vertices of each image block according to the deformation parameters. The partitioning process is equivalent to sparsely processing all the normalized coordinates in the image to be processed and deforming the sparse normalized coordinates. At the same time, it can also reduce the computational complexity to a certain extent. Figure 3 Taking the image to be processed with a height and width of 5 as an example, when the number of deformation parameters is 9 (n=3), the image to be processed is partitioned into 4 image blocks (the 4 area blocks shown by the solid lines in the figure). After partitioning, there are 9 coordinate points in total. Then, based on all the deformation parameters, the 9 coordinate vertices of these 4 image blocks are deformed in turn to obtain the coordinates of the deformation processing.

[0085] Furthermore, it also includes the steps of: creating an intermediate image corresponding to the image to be processed, obtaining the deformation coordinates corresponding to the vertices of the image block in the intermediate image through a positioning function, as the deformation coordinates; based on the obtained deformation coordinates, obtaining all deformation coordinates through an interpolation algorithm.

[0086] Specifically, when n is not equal to both the height and width of the image to be processed, when traversing the coordinate points of the intermediate image, determine whether the coordinate point corresponds exactly to a vertex of the partitioned image block. If so, use the positioning function to find the corresponding affine map and obtain the corresponding accurate coordinates. Assuming that the coordinate point is (x, y), the number of deformation parameters is n*n, the number of image blocks is m*m, and corresponding to the kth affine map, calculate whether x%m and y%m are equal to 0. If both are equal to 0, it means that the coordinate point corresponds exactly to a vertex of the image block, otherwise it is not. The corresponding positioning function is as follows:

[0087] k=int(n*(y / m)+(x / m)),

[0088] Int represents rounding, and all coordinates at the vertices of the image block are accurate coordinates that can be directly obtained.

[0089] Furthermore, if the coordinate point does not correspond to the vertex of the partitioned image block, it is calculated based on the accurate deformation coordinates already obtained around it through an interpolation algorithm, wherein the interpolation algorithm includes but is not limited to a bilinear interpolation algorithm. Figure 3 Take the image to be processed in as an example, where the vertices of the four image blocks are accurate coordinates, and the inaccurate coordinates (such as the coordinate points at the intersection of the dotted lines in the figure) can be obtained based on linear interpolation of the surrounding accurate coordinates.

[0090] By judging whether the preset relationship is satisfied in this step, a specific deformation coordinate acquisition method is determined according to the specific partitioning situation to obtain all coordinates of the image to be processed after affine transformation.

[0091] Step S330 : Based on the deformation coordinates, sampling and interpolation processing is performed on the image to be processed by an interpolation algorithm to obtain a deformed image.

[0092] Specifically, based on the acquired deformed coordinates of each pixel, an interpolation algorithm is selected for sampling and interpolation to obtain a deformed image. The original image is sampled and interpolated using existing interpolation methods, such as bilinear interpolation, nearest neighbor interpolation, thin plate spline interpolation, and bicubic interpolation, to obtain the pixel values ​​corresponding to all coordinates in the deformed image, ultimately yielding the deformed image. This process can be thought of as obtaining the pixel values ​​corresponding to each deformed image and assigning them to each coordinate point in the intermediate image, which was all zero in the above step. This step ultimately yields a complete deformed image with the deformed coordinates and corresponding pixel values ​​from the intermediate image.

[0093] Preferably, the grid_sample function in the pytorch framework is used, that is, the coordinates obtained above are first scaled to between [0, H] and [0, W], where H and W represent the height and width of the image to be processed, respectively. If the coordinates correspond exactly to the correct ones, the pixel value of the coordinate point is directly obtained from the pixel value of the corresponding point on the affine map. If the coordinates fall on inaccurate ones, the pixel values ​​of the four surrounding points need to be interpolated.

[0094] In this embodiment, when performing partition deformation, a corresponding number of non-complete deformation parameters are first set, and then the normalized coordinates are affine transformed for different partition situations according to whether the preset relationship is satisfied to obtain the deformation coordinates. An intermediate image is further initialized, and the deformation coordinates are obtained from the deformation coordinates according to the coordinate points of the intermediate image. Finally, the pixel value corresponding to each deformation coordinate is calculated based on the interpolation algorithm to obtain a complete deformed image. In this way, the effect of performing different deformations on different parts of the processed image can be achieved, and problems such as image distortion and edge blurring after deformation can be effectively avoided.

[0095] The following embodiments specifically describe how to perform the overall deformation of the image to be processed.

[0096] When the processing requirement is to perform overall deformation, first, a corresponding number of deformation parameters are selected according to the complexity of the processing requirement, and the deformation parameters are set to be exactly the same.

[0097] Furthermore, when the deformation parameters are completely the same, only one deformation parameter may be set to reduce the complexity of calculation.

[0098] Secondly, the deformed image is obtained by the following steps:

[0099] Normalize all coordinates of the image to be processed to obtain normalized coordinates;

[0100] Through affine transformation, the normalized coordinates are deformed based on the deformation parameters to obtain the deformed coordinates of the pixel points of the image to be processed;

[0101] Based on the deformation coordinates, the image to be processed is sampled and interpolated by an interpolation algorithm to obtain a deformed image.

[0102] The specific embodiment of obtaining the deformed image can be implemented with reference to the above embodiment in which the number of deformation parameters and the size of the image to be processed satisfy a preset relationship. This embodiment can achieve overall deformation of the image to be processed.

[0103] In some of the embodiments, deformation of a specific area in the image to be processed may also be achieved, including:

[0104] By setting corresponding deformation parameters for the image portion except the specific area, the image portion is kept unchanged after being processed by the deformation parameters, that is, kept as it is;

[0105] Then, deformation parameters are set corresponding to specific areas so that deformation occurs only in the specific areas.

[0106] Furthermore, overall deformation and partition deformation can be performed in a specific area by setting the same or different deformation parameters in the specific area. The specific deformation processing refers to the overall deformation and partition deformation processing method of the entire image to be processed provided in the above embodiment.

[0107] According to the above embodiments, other deformation embodiments are conceivable by setting deformation parameters to achieve deformation in different areas and to different degrees.

[0108] After obtaining a deformed image through the partition processing, overall processing, and specific area processing of all the above embodiments, the conservation of integral values ​​before and after deformation is further achieved through the following embodiments. The deformation parameters are expanded to obtain expanded deformation parameters; weights are obtained based on the expanded deformation parameters, and a target image is obtained based on the deformed image and the weights. Specifically, the following steps are included:

[0109] Step S100 : Expanding the number of deformation parameters to a number of deformation parameters corresponding to the coordinates in the image to be processed based on an interpolation algorithm.

[0110] Specifically, assuming that the number of deformation parameters is n*n, the height and width of the image to be processed are H and W respectively. Corresponding to the given number of deformation parameters, the bilinear interpolation algorithm is used to interpolate and expand the number of deformation parameters corresponding to the coordinates in the image to be processed, that is, the number after interpolation and expansion is H*W.

[0111] Step S200 , calculating the corresponding weights of the expanded deformation parameters; the weights represent the ratios of the integral values ​​of the image to be processed before and after the deformation processing.

[0112] Specifically, a determinant matrix is ​​calculated for each deformation parameter as a weight. The resulting determinant matrix represents the proportion of the deformed portion of the image after the deformation process. It can also be understood as the ratio of the integral value of the image before and after the deformation. Preferably, the determinant matrix is ​​calculated using the torch.linalg.det() function provided by the PyTorch framework.

[0113] Step S300 : Based on the weight value and the deformed image, a target image with a conservative integral value after deformation is calculated.

[0114] Specifically, matrix multiplication is performed on the deformed image obtained in the above embodiment and the determinant matrix, which is equivalent to adding a weight to the deformed image, multiplying the pixel value of each pixel point coordinate of the deformed image by the weight value, and finally calculating the target image with conserved integral value.

[0115] In this embodiment, the deformation parameters are interpolated and expanded, and a corresponding determinant matrix representing the ratio of the integral values ​​before and after deformation is obtained. The deformed image obtained after deformation is calculated and processed based on the determinant matrix. This can solve the problem of non-conservation of the integral values ​​before and after deformation in the prior art, and can be applied to the deformation processing of the existing interpolation algorithm, so that while achieving smooth deformation of the image, conservation of the integral value of the image can also be achieved.

[0116] The present embodiment is described and illustrated below through preferred embodiments.

[0117] Figure 4 FIG. 1 is a flow chart of an image deformation method based on an interpolation algorithm according to a preferred embodiment of the present invention. Figure 4 As shown, the method includes the following steps:

[0118] Step S410 , obtaining an image to be processed, wherein the height and width of the image to be processed are set to H and W respectively, and the number of deformation parameters is set to n*n, and a corresponding number of deformation parameters is selected according to processing requirements.

[0119] The deformation parameters may be completely the same or not completely the same.

[0120] Step S420 , normalizing all coordinates of the image to be processed to obtain normalized coordinates.

[0121] Step S430: Determine whether n!=H and n!=W hold.

[0122] If so, execute step S431; if not, execute step S432.

[0123] Step S431: Divide the image to be processed into (n-1)*(n-1) uniform image blocks; perform affine transformation on the normalized coordinates of the vertices of the image blocks according to the deformation parameters to obtain an affine map and the deformed coordinates in the map.

[0124] Step S432: Perform an affine transformation on each normalized coordinate according to the deformation parameters to obtain an affine map and the deformed coordinates in the map.

[0125] Step S440: Initialize an intermediate image of the same size as the image to be processed, and traverse each coordinate point in the image.

[0126] If the above step S430 is satisfied, then proceed to step S450; if not, proceed to step S460.

[0127] Step S450: determine whether the coordinate point corresponds to a vertex of the image block.

[0128] If yes, execute step S451; if no, execute step S452.

[0129] In step S451 , the corresponding affine image in the intermediate image is obtained through the positioning function, and the accurate coordinates of the corresponding position in the image are directly obtained as the deformation coordinates.

[0130] Step S452 : Based on the deformation coordinates obtained around the point, all deformation coordinates are calculated by an interpolation algorithm.

[0131] Step S460: Obtain the corresponding affine map in the intermediate image, and directly obtain the accurate coordinates of the corresponding position in the map as the deformation coordinates.

[0132] Step S470 : Based on the deformation coordinates, the image to be processed is sampled and interpolated by an interpolation algorithm to obtain a pixel value corresponding to each deformation coordinate to obtain a deformed image.

[0133] Step S480 : For a given number of n*n deformation parameters, interpolation and expansion are performed to obtain H*W deformation parameters, and a determinant matrix is ​​calculated for each deformation parameter.

[0134] In step S490 , the deformed image and the determinant matrix are correspondingly multiplied to obtain a target image with a conserved integral value after deformation.

[0135] Through this preferred embodiment, the image to be processed can be deformed according to the processing requirements. After obtaining the deformed image, it is calculated by the determinant matrix of the deformation parameters after interpolation and expansion, so that the integral value of the calculated deformed image remains unchanged, thereby achieving the effect of ensuring the conservation of the integral value before and after deformation while performing different local deformations or overall deformations on the image.

[0136] It should be noted that the steps shown in the above process or the flowcharts in the accompanying drawings can be executed in a computer system, such as a set of computer-executable instructions. Furthermore, although the flowcharts illustrate a logical order, in some cases, the steps shown or described may be executed in a different order. For example, the process of interpolating and expanding the deformation parameters in step S480 is not limited to being performed after the deformable image is obtained; it can also be performed before or during the process of obtaining the deformable image.

[0137] This embodiment also provides an image deformation device based on an interpolation algorithm. This device is used to implement the above-mentioned embodiments and preferred implementations. Details already described will not be repeated here. The terms "module," "unit," "subunit," etc. used below may refer to a combination of software and / or hardware that implements the predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0138] Figure 5 This is a structural block diagram of the image deformation device based on the interpolation algorithm of this embodiment. Figure 5 As shown, the device includes: an acquisition module 510, a deformation module 520 and an integral value conservation module 530;

[0139] An acquisition module 510 is used to acquire an image to be processed and select a corresponding number of deformation parameters according to processing requirements;

[0140] A deformation module 520 is used to process the image to be processed to obtain a deformed image based on deformation parameters and an interpolation algorithm;

[0141] The integral value conservation module 530 is used to expand the deformation parameters to obtain expanded deformation parameters; obtain weights according to the expanded deformation parameters, and obtain a target image based on the deformed image and the weights.

[0142] Through the device provided by this embodiment, a corresponding number of deformation parameters are selected to perform deformation processing on the image to be processed to obtain a deformed image, and the existing number of deformation parameters are interpolated and expanded to obtain a number of deformation parameters corresponding to the coordinates of the image to be processed. The weights corresponding to these deformation parameters can represent the ratio of the integral values ​​before and after deformation. Combined with the weight values, the deformed image is calculated and processed to obtain a target image with conserved integral values. The target image can be flexibly applied to existing interpolation algorithms and inserted into existing neural networks for back propagation and training. It can not only achieve smooth deformation of the image in shape, but also achieve conservation of the integral value of the image, thereby solving the problem of being unable to ensure conservation of the integral value before and after deformation while achieving smooth deformation of the image.

[0143] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0144] This embodiment further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0145] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0146] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0147] In addition, in conjunction with the interpolation algorithm-based image deformation method provided in the above embodiments, this embodiment may also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the interpolation algorithm-based image deformation methods in the above embodiments.

[0148] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0149] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0150] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0151] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An image deformation method based on an interpolation algorithm, characterized in that: include: Obtain the image to be processed and select the corresponding number of deformation parameters according to the processing requirements; Based on the deformation parameters and the interpolation algorithm, the image to be processed is processed to obtain a deformed image; Expanding the deformation parameters to obtain expanded deformation parameters; Obtaining weights according to the expanded deformation parameters, and obtaining a target image based on the deformed image and the weights; wherein the method includes the following steps: Expanding the existing number of deformation parameters using an interpolation algorithm to obtain a number of deformation parameters corresponding to the coordinates in the image to be processed, calculating the determinant value of each deformation parameter after interpolation and expansion to obtain a corresponding determinant matrix, and using the determinant matrix as the weight of the deformable image; the weight represents the ratio of the integral value of the image to be processed before and after the deformation processing; The pixel value of each pixel point coordinate of the deformed image is multiplied by the weight value of the deformed image to obtain a target image with a conserved integral value after deformation.

2. The image deformation method based on interpolation algorithm according to claim 1, characterized in that: The step of processing the image to be processed based on the deformation parameters and the interpolation algorithm to obtain a deformed image includes: Normalizing all coordinates of the image to be processed to obtain normalized coordinates; Performing deformation processing on the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed; Based on the deformation coordinates, sampling and interpolation processing is performed on the image to be processed by an interpolation algorithm to obtain the deformed image.

3. The image deformation method based on interpolation algorithm according to claim 2, characterized in that: The deforming the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed after deformation, includes: determining a method for acquiring the deformation coordinates according to whether a preset condition is met; The preset condition is that the number of the deformation parameters and the size of the image to be processed must meet a preset relationship.

4. The image deformation method based on interpolation algorithm according to claim 3, characterized in that: The deforming the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed after deformation, includes: When the preset condition is met, deformation processing is performed on each of the normalized coordinates according to the deformation parameter to directly obtain the deformed coordinates.

5. The image deformation method based on interpolation algorithm according to claim 3, characterized in that: The deforming the normalized coordinates based on the deformation parameters through affine transformation to obtain the deformed coordinates of the pixel points of the image to be processed after deformation, includes: When the preset condition is not met, partitioning the image to be processed to obtain a plurality of image blocks; deforming the normalized coordinates located at the vertices of the image blocks according to the deformation parameters, and obtaining the corresponding deformation coordinates through a positioning function; Based on the obtained deformation coordinates, all deformation coordinates are obtained through an interpolation algorithm.

6. The image deformation method based on interpolation algorithm according to claim 2, characterized in that: Based on the deformation coordinates, sampling and interpolation processing is performed on the image to be processed by an interpolation algorithm to obtain the deformed image, including: According to the acquired deformation coordinates of each pixel point, sampling and interpolation processing is performed by selecting an interpolation algorithm to obtain the deformed image.

7. An image deformation device based on an interpolation algorithm, characterized in that: include: Acquisition module, deformation module and integral value conservation module; The acquisition module is used to acquire the image to be processed and select a corresponding number of deformation parameters according to processing requirements; The deformation module processes the image to be processed to obtain a deformed image based on the deformation parameters and the interpolation algorithm; The integral value conservation module is used to expand the deformation parameter to obtain the expanded deformation parameter; Obtaining weights according to the expanded deformation parameters, and obtaining a target image based on the deformed image and the weights; wherein the method includes the following steps: Expanding the existing number of deformation parameters using an interpolation algorithm to obtain a number of deformation parameters corresponding to the coordinates in the image to be processed, calculating the determinant value of each deformation parameter after interpolation and expansion to obtain a corresponding determinant matrix, and using the determinant matrix as the weight of the deformable image; the weight represents the ratio of the integral value of the image to be processed before and after the deformation processing; The pixel value of each pixel point coordinate of the deformed image is multiplied by the weight value of the deformed image to obtain a target image with a conserved integral value after deformation.

8. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the image deformation method based on an interpolation algorithm according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image deformation method based on the interpolation algorithm according to any one of claims 1 to 6 are implemented.

Images