Image Processing Method and Apparatus, Electronic Device, and Storage Medium

Abstract

The present invention discloses an image processing method, apparatus, electronic device and storage medium. Among them, the image processing method includes: obtaining a first image and a second image obtained by photographing the same object at different times; performing registration processing on the second image by using a non-linear transformation model to obtain a third image; wherein, the non-linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy; calculating a deformation field based on the third image and the first image, and synthesizing the first image with the deformation field to obtain a synthesized picture for characterizing the deformation effect of the target part. The present invention uses a non-linear transformation model to register the second image, calculates a deformation field based on the registered third image and the first image, and synthesizes the first image with the deformation field, so that the user can intuitively observe the deformation effect of the target part from the synthesized picture, improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to an image processing method and apparatus, an electronic device, and a storage medium. Background Art

[0002] With the development of modern science and technology, people can use mobile phones to record the appearances of themselves, their relatives, or others at any time. If they want to intuitively view the changing trend of the face shape, they can modify and compare the photos through image editing software after taking the photos or form the pictures into a video. However, this method has disadvantages such as the need to install professional image editing software additionally, the modification operation is relatively complex, and other problems will occur. In addition, although most image editing software on the market can compare the appearance changes, it is necessary to fuse multiple images, and the deviation of the picture shooting angle and the operation difficulty of software comparison both result in a poor final film effect, and it is difficult to distinguish the changing trend of the face shape with the naked eye. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to overcome the defects in the prior art such as the need to install software additionally, complex operations, or the need to fuse multiple images, and provide an image processing method and apparatus, an electronic device, and a storage medium.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] The first aspect of the present invention provides an image processing method, which is characterized by including the following steps:

[0006] Obtain a first image and a second image; wherein, the first image and the second image are respectively obtained by photographing the same object at different times;

[0007] Perform registration processing on the second image by using a non-linear transformation model to obtain a third image; wherein, the non-linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy;

[0008] Calculate a deformation field according to the third image and the first image, and synthesize the first image with the deformation field to obtain a synthesized picture for characterizing the deformation effect of the target part.

[0009] Optionally, before the step of performing registration processing on the second image by using the non-linear transformation model, it further includes:

[0010] Extract at least three first feature points of the target part of the object in the first image, and extract at least three second feature points of the target part in the second image; wherein, the second feature points correspond to the first feature points one by one;

[0011] Determine an affine transformation matrix according to the coordinates of the first feature points and the second feature points;

[0012] Perform coordinate transformation processing on the second image according to the affine transformation matrix.

[0013] Optionally, the non-linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy, and specifically includes:

[0014] Extract several feature points of the target parts of the object in the first image and the second image respectively;

[0015] Input the coordinates of all the extracted feature points into a preset non-linear transformation model to solve the parameters of the non-linear transformation model;

[0016] Perform registration processing on the second image by using the non-linear transformation model according to the solved parameters to obtain an intermediate image;

[0017] Compare the intermediate image with the first image to obtain the registration accuracy;

[0018] Optimize the extracted feature points, and input the coordinates of the optimized feature points into the preset non-linear transformation model to solve the parameters until the registration accuracy meets the requirements.

[0019] Optionally, the image processing method further includes the following steps:

[0020] Calculate the offset direction and offset size of the grid nodes in the deformation field according to the parameters of the non-linear transformation model;

[0021] Add indication marks in the deformation field according to the offset direction and offset size; wherein, the indication marks are used to characterize the deformation trend of the target part.

[0022] The second aspect of the present invention provides an image processing apparatus, including:

[0023] An image acquisition module, configured to acquire a first image and a second image; wherein, the first image and the second image are respectively obtained by photographing the same object at different times;

[0024] An image registration module, configured to perform registration processing on the second image by using a non - linear transformation model to obtain a third image; wherein, the non - linear transformation model is determined according to feature points respectively extracted from the target parts of the object in the first image and the second image and the registration accuracy;

[0025] An image synthesis module, configured to calculate a deformation field based on the third image and the first image, and synthesize the first image with the deformation field to obtain a synthesized picture for characterizing the deformation effect of the target part.

[0026] Optionally, the image processing device further includes an image processing module; the image processing module includes:

[0027] A first extraction unit, configured to extract at least three first feature points of the target part of the object in the first image, and extract at least three second feature points of the target part in the second image; wherein, the second feature points correspond to the first feature points one by one;

[0028] A matrix determination unit, configured to determine an affine transformation matrix according to the coordinates of the first feature points and the second feature points;

[0029] A coordinate transformation unit, configured to perform coordinate transformation processing on the second image according to the affine transformation matrix.

[0030] Optionally, the image processing device further includes a model determination module, and the model determination module includes a second extraction unit, a parameter solving unit, a registration processing unit, and an image comparison unit;

[0031] The second extraction unit is configured to extract a plurality of feature points of the target part of the object in the first image and the second image respectively;

[0032] The parameter solving unit is configured to input the coordinates of all the extracted feature points into a preset non - linear transformation model to solve the parameters of the non - linear transformation model;

[0033] The registration processing unit is configured to perform registration processing on the second image by using the non - linear transformation model according to the solved parameters to obtain an intermediate image;

[0034] The image comparison unit is configured to compare the intermediate image with the first image to obtain the registration accuracy, and optimize the extracted feature points in the case that the registration accuracy does not meet the requirements, and call the parameter solving unit.

[0035] Optionally, the image synthesis module is specifically configured to calculate the offset direction and offset size of the grid nodes in the deformation field according to the parameters of the non-linear transformation model, and add an indication mark in the deformation field according to the offset direction and offset size; wherein, the indication mark is used to characterize the deformation trend of the target part.

[0036] The third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the image processing method described in the first aspect is implemented.

[0037] The fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image processing method described in the first aspect is implemented.

[0038] On the basis of conforming to the common knowledge in the art, the above optional conditions can be combined arbitrarily to obtain various preferred embodiments of the present invention.

[0039] The positive and progressive effect of the present invention is that by using the first image as a reference image, registering the second image using a non-linear transformation model, calculating the deformation field according to the registered third image and the reference image, and synthesizing the reference image and the deformation field, the user can intuitively observe the deformation effect of the target part in the object from the synthesized picture, improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic flowchart of an image processing method provided in Embodiment 1 of the present invention.

[0041] Figure 2 It is a schematic diagram of a method for processing a second image provided in Embodiment 1 of the present invention.

[0042] Figure 3 It is a schematic diagram of a second image provided in Embodiment 1 of the present invention.

[0043] Figure 4 It is a schematic diagram of a method for determining a non-linear transformation model provided in Embodiment 1 of the present invention.

[0044] Figure 5 It is a schematic diagram of a deformation field provided in Embodiment 1 of the present invention.

[0045] Figure 6 It is a structural block diagram of an image processing device provided in Embodiment 1 of the present invention.

[0046] Figure 7 It is a schematic structural diagram of an electronic device provided in Embodiment 2 of the present invention. Detailed Implementation Modes

[0047] The present invention will be further described below by way of embodiments, but the present invention is not limited to the scope of the described embodiments.

[0048] Embodiment 1

[0049] Figure 1 FIG. is a schematic flowchart of an image processing method provided in this embodiment. The image processing method can be executed by an image processing device, which can be implemented in a software and / or hardware manner, and the image processing device can be part or all of an electronic device.

[0050] Among them, the electronic device in this embodiment can be a personal computer (PC), such as a desktop computer, an all-in-one computer, a laptop computer, a tablet computer, etc., and can also be a terminal device such as a mobile phone, a wearable device, a personal digital assistant (PDA), etc. The image processing method provided in this embodiment will be introduced below with the electronic device as the execution subject.

[0051] As Figure 1 shown, the image processing method provided in this embodiment may include the following steps S1 to S3:

[0052] Step S1, obtain a first image and a second image. Wherein, the first image and the second image are respectively obtained by photographing the same object at different times.

[0053] Specifically, the object can be a person, an animal, etc. In the specific process of photographing the object, in addition to the different photographing times, the photographing location, the photographing angle, etc. may also be different, so as to obtain two different images, namely the first image and the second image.

[0054] In an example of specific implementation, both the first image and the second image are two-dimensional images, which can be directly obtained by the camera of the electronic device or downloaded from a website. For example, different users can photograph the same object at different times respectively, and then upload the different photographed images to the website for other users to download and use.

[0055] In another example of specific implementation, both the first image and the second image are three-dimensional images, which can be obtained by a professional instrument such as a CT (Computed Tomography) device.

[0056] In order to improve the accuracy of determining the deformation effect of the target part, the coordinate systems of the first image and the second image can be unified so that the directions of the object in the two images are the same.

[0057] In an optional implementation, as Figure 2 shown, after step S1, it further includes:

[0058] Step S11: Extract at least three first feature points of the target part of the object in the first image, and extract at least three second feature points of the target part in the second image. Among them, the second feature points and the first feature points correspond one by one, and both correspond to the same position of the target part.

[0059] In a specific implementation, the above object can be a person, and the above target part can be a human face. Figure 3 A schematic diagram for showing a second image. In Figure 3 the example shown, multiple second feature points in the human face area of the second image can be extracted, such as Figure 3 shown by the multiple white dots in

[0060] In a specific example, two first feature points A1 and A2 of the human face in the first image are extracted, and two second feature points B1 and B2 of the human face in the second image are extracted. Among them, the first feature point A1 and the second feature point B1 both correspond to the position of the left eye corner of the human face, the first feature point A2 and the second feature point B2 both correspond to the position of the right eye corner of the human face, and the first feature point A3 and the second feature point B3 both correspond to the position of the tip of the nose of the human face.

[0061] In the specific implementation of step S11, the SIFT (Scale Invariant Feature Transform) operator is used to extract feature points. Among them, SIFT features are invariant to rotation, scale scaling, brightness change, etc., and are a very stable local feature.

[0062] Step S12: Determine the affine transformation matrix according to the coordinates of the first feature points and the second feature points.

[0063] Among them, affine transformation refers to a linear transformation and a translation transformation in a vector space in geometry to transform into another vector space.

[0064]

[0065] Among them, the above formula is an affine transformation in a two-dimensional space. To cover translation during calculation, homogeneous coordinates are introduced, and an additional dimension is augmented in the two-dimensional coordinates. In the affine transformation matrix, the parameters (c, f) are used to represent the translation amount, and the parameters (a, b, d, e) are used to represent linear transformations such as rotation and scaling of the second image. x is the abscissa of the first feature point, y is the ordinate of the first feature point, x′ is the abscissa of the corresponding second feature point, and y′ is the ordinate of the corresponding second feature point. A linear equation system is established based on the coordinates of at least three first feature points and the coordinates of the corresponding second feature points, and then the affine transformation matrix can be solved based on methods such as the least squares method or SVD (Singular Value Decomposition) decomposition.

[0066] Step S13: Perform coordinate transformation processing on the second image according to the affine transformation matrix. Specifically, multiply the coordinates of all pixel points in the second image by the affine transformation matrix respectively to obtain the second image after coordinate transformation. Among them, the second image after coordinate transformation is in the same coordinate system as the first image, so that the direction of the object or the target part in the object is the same in the first image and the second image.

[0067] It should be noted that in the example where the object is a person and the target part is a human face, deep learning face alignment algorithms such as MTCNN (Multi-task convolutional neural network) and PFLD (a face key point detection model) can also be used to align the face.

[0068] Step S2: Perform registration processing on the second image using a non-linear transformation model to obtain a third image. Among them, the non-linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy.

[0069] Since the deformation trend of the object target part is non-linear, a non-linear transformation model is used to perform registration processing on the second image. The specific parameters of the non-linear transformation model need to be determined according to the image to be registered, that is, the second image, and the reference image, that is, the first image.

[0070] In an optional implementation manner, as Figure 4 shown, the non-linear transformation model is determined through the following steps S21 to S26:

[0071] Step S21: Extract a number of feature points of the target part of the object in the first image and the second image respectively. Among them, the feature points extracted from the first image correspond one by one to the feature points extracted from the second image, and the two correspond to the same position of the target part. In specific implementation, the SIFT operator can be used to extract the feature points in the two images.

[0072] Step S22: Input the coordinates of all the extracted feature points into a preset non - linear transformation model to solve the parameters of the non - linear transformation model.

[0073] In specific implementation, the preset non - linear transformation model can be a thin - plate spline transformation model or a Smooth Thin Plate Spline (STPS) transformation model. The following is the Smooth Thin Plate Spline transformation model:

[0074]

[0075] where \(\{x i \},\{y i \}, i = 1,2,\cdots,n\) are corresponding point pairs, \(f\) is a mapping function, and \(\lambda\) is a control smoothing coefficient. In specific implementation, the above - mentioned Smooth Thin Plate Spline transformation model can be converted into the following formula:

[0076]

[0077] where \(A\) is an affine transformation matrix, \(\omega k \) is a non - affine deformation coefficient, \(\varphi(r)=r 2 \log(r)\) is the radial basis function of STPS, indicating that the deformation of a point on a certain surface is affected by the deformation of all control points. \(c j \) is an arbitrary feature point in the second image, \(u j \) is the corresponding feature point in the second image. In the process of solving the STPS parameters, two feature points of the first image and the second image are respectively required. Specifically, the least - squares results of the parameters \(A\) and \(\omega k \) of STPS can be calculated through QR decomposition, and then the parameters \(A\) and \(\omega k \) of STPS are obtained.

[0078] Step S23: Perform registration processing on the second image using the solved parameters with the non - linear transformation model to obtain an intermediate image.

[0079] In specific implementation, all the pixel points in the second image are input into the non - linear transformation model after solving the parameters, so as to obtain the intermediate image after registration processing. In the above example, the coordinates of all the pixel points in the second image can be multiplied by the solved parameter \(A\) respectively, and then added with \(\omegak The offset is used to obtain the coordinates of all pixel points of the intermediate image.

[0080] Step S24: Compare the intermediate image with the first image to obtain the registration accuracy. In this embodiment, the first image is used as the reference image, and the intermediate image is used as the floating image.

[0081] In a specific implementation, the registration accuracy can be measured according to the correlation coefficient CC:

[0082]

[0083] where x i , y i are the intensities of the i-th pixel point in the floating image and the reference image respectively; x m , y m are the average intensities of the floating image and the reference image respectively.

[0084] It should be noted that other methods can also be used to measure the registration accuracy.

[0085] Step S25: Determine whether the registration accuracy meets the requirements. If so, execute step S26; if not, optimize the feature points extracted in step S21, return to step S22, input the coordinates of the optimized feature points into a preset non-linear transformation model to solve for new parameters, and then sequentially execute steps S23 - S25 until the registration accuracy meets the requirements. Among them, the random gradient descent method can be used to optimize the feature points extracted in step S21.

[0086] In a specific implementation, a preset accuracy can be set. If the registration accuracy reaches the preset accuracy, it is considered that the registration accuracy meets the requirements; otherwise, it is considered that the registration accuracy does not meet the requirements.

[0087] Step S26: Obtain the determined non-linear transformation model.

[0088] Step S3: Calculate the deformation field based on the third image and the first image, and synthesize the first image with the deformation field to obtain a synthesized picture for characterizing the deformation effect of the target part. In a specific implementation, the user can intuitively know the deformation effect of the target part in the object through the synthesized picture, which is convenient for observation. In the example where the target part is a human face, the above deformation effects can be getting fatter, thinner, sharper, rounder, etc.

[0089] Figure 5 A schematic diagram for showing a deformation field is provided. In a specific example, by comparing the registered third image and the first image, the following can be obtained: Figure 5The deformation field shown is combined with the first image to obtain a composite image.

[0090] In an alternative embodiment, after calculating the deformation field based on the third image and the first image, and before combining the first image with the deformation field, step S3 specifically includes steps S31 to S32:

[0091] Step S31: Calculate the offset direction and offset magnitude of the grid nodes in the deformation field according to the parameters of the non - linear transformation model.

[0092] Step S32: Add indication marks in the deformation field according to the offset direction and offset magnitude. Wherein, the indication marks are used to characterize the deformation trend of the target part.

[0093] In a specific implementation, the indication marks can be arrows. Users can intuitively observe the deformation trend of the target part according to the direction and distance of the arrows. In a specific implementation, the indication marks can also be marking symbols of different colors, etc.

[0094] In this embodiment, by using the first image as a reference image, registering the second image using a non - linear transformation model, calculating the deformation field based on the registered third image and the reference image, and combining the reference image with the deformation field, users can intuitively observe the deformation effect of the target part in the object from the composite image, improving the user experience.

[0095] As Figure 6 shown, this embodiment also provides an image processing device 40, including an image acquisition module 41, an image registration module 42, and an image synthesis module 43.

[0096] The image acquisition module 41 is used to acquire the first image and the second image. Wherein, the first image and the second image are respectively obtained by photographing the same object at different times.

[0097] The image registration module 42 is used to perform registration processing on the second image using a non - linear transformation model to obtain a third image. Wherein, the non - linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy.

[0098] The image synthesis module 43 is used to calculate the deformation field according to the third image and the first image, and combine the first image with the deformation field to obtain a composite image for characterizing the deformation effect of the target part.

[0099] In an alternative embodiment, as Figure 6As shown in the figure, the above-mentioned image processing device 40 further includes an image processing module 44. The image processing module 44 includes a first extraction unit, a matrix determination unit, and a coordinate transformation unit. The first extraction unit is configured to extract at least three first feature points of the target part of the object in the first image, and extract at least three second feature points of the target part in the second image; wherein, the second feature points correspond to the first feature points one by one. The matrix determination unit is configured to determine an affine transformation matrix according to the coordinates of the first feature points and the second feature points. The coordinate transformation unit is configured to perform coordinate transformation processing on the second image according to the affine transformation matrix.

[0100] In an alternative embodiment, as Figure 6 shown in the figure, the above-mentioned image processing device 40 further includes a model determination module 45. The model determination module 45 includes a second extraction unit, a parameter solving unit, a registration processing unit, and an image comparison unit. The second extraction unit is configured to extract a plurality of feature points of the target part of the object in the first image and the second image respectively. The parameter solving unit is configured to input the coordinates of all the extracted feature points into a preset non-linear transformation model, and solve the parameters of the non-linear transformation model. The registration processing unit is configured to perform registration processing on the second image by using the non-linear transformation model according to the solved parameters to obtain an intermediate image. The image comparison unit is configured to compare the intermediate image with the first image to obtain a registration accuracy, and optimize the extracted feature points in the case where the registration accuracy does not meet the requirements, and call the parameter solving unit.

[0101] In an alternative embodiment, the above-mentioned image synthesis module is specifically configured to calculate the offset direction and offset size of the grid nodes in the deformation field according to the parameters of the non-linear transformation model, and add an indication mark in the deformation field according to the offset direction and offset size, wherein the indication mark is used to characterize the deformation trend of the target part.

[0102] It should be noted that in this embodiment, the image processing device may specifically be a separate chip, a chip module or an electronic device, or may also be a chip or a chip module integrated in an electronic device.

[0103] Regarding each module / unit included in the image processing device described in this embodiment, it may be a software module / unit, a hardware module / unit, or may also be partly a software module / unit and partly a hardware module / unit.

[0104] Embodiment 2

[0105] Figure 7A schematic structural diagram of an electronic device provided in this embodiment. The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. Among them, the memory stores a computer program that can be run by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image processing method of Embodiment 1. The electronic device provided in this embodiment can be a personal computer, such as a desktop computer, an all-in-one computer, a laptop computer, a tablet computer, etc., and can also be a terminal device such as a mobile phone, a wearable device, a palm computer, etc. Figure 7 The electronic device 3 shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0106] The components of the electronic device 3 may include but are not limited to: the above-mentioned at least one processor 4, the above-mentioned at least one memory 5, and a bus 6 connecting different system components (including the memory 5 and the processor 4).

[0107] The bus 6 includes a data bus, an address bus, and a control bus.

[0108] The memory 5 may include a volatile memory, such as a random access memory (RAM) 51 and / or a cache memory 52, and may further include a read-only memory (ROM) 53.

[0109] The memory 5 may further include a program / utility 55 having a set (at least one) of program modules 54. Such program modules 54 include but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0110] The processor 4 executes various functional applications and data processing by running the computer program stored in the memory 5, such as the above-mentioned image processing method.

[0111] The electronic device 3 can also communicate with one or more external devices 7 (such as a keyboard, a pointing device, etc.). Such communication can be carried out through an input / output (I / O) interface 8. And, the electronic device 3 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 9. As Figure 7 shown, the network adapter 9 communicates with other modules of the electronic device 3 through the bus 6. It should be understood that although Figure 7 not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 3, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0112] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / modules. Conversely, the features and functions of one unit / modules described above can be further divided and embodied by multiple units / modules.

[0113] Embodiment 3

[0114] This embodiment provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the image processing method of Embodiment 1.

[0115] Among them, the more specific forms that the readable storage medium can adopt may include but are not limited to: portable disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0116] In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on an electronic device, the program code is used to cause the electronic device to execute the image processing method of Embodiment 1.

[0117] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed entirely on the electronic device, partially on the electronic device, executed as an independent software package, partially on the electronic device and partially on a remote device, or entirely on a remote device.

[0118] Although the specific implementation manners of the present invention are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present invention is defined by the appended claims. Without departing from the principles and essence of the present invention, those skilled in the art can make various changes or modifications to these implementation manners, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, it includes the following steps: Obtain a first image and a second image; wherein, the first image and the second image are respectively obtained by photographing the same object at different times; Use a non - linear transformation model to perform registration processing on the second image to obtain a third image; wherein, the non - linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy; Calculate a deformation field based on the third image and the first image, and synthesize the first image with the deformation field to obtain a synthesized picture for characterizing the deformation effect of the target part; The non - linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy, specifically including: Extract a number of feature points from the target parts of the object in the first image and the second image respectively; Input the coordinates of all the extracted feature points into a preset non - linear transformation model to solve the parameters of the non - linear transformation model; Use the non - linear transformation model to perform registration processing on the second image according to the solved parameters to obtain an intermediate image; Compare the intermediate image with the first image to obtain the registration accuracy; Optimize the extracted feature points, and input the coordinates of the optimized feature points into the preset non - linear transformation model to solve the parameters until the registration accuracy meets the requirements.

2. The image processing method according to claim 1, characterized in that, before the step of using the non - linear transformation model to perform registration processing on the second image, it further includes: Extract at least three first feature points of the target part of the object in the first image, and extract at least three second feature points of the target part in the second image; wherein, the second feature points correspond to the first feature points one by one; Determine an affine transformation matrix according to the coordinates of the first feature points and the second feature points; Perform coordinate transformation processing on the second image according to the affine transformation matrix.

3. The image processing method according to any one of claims 1 or 2, characterized in that, the image processing method further includes the following steps: Calculate the offset direction and offset size of the grid nodes in the deformation field according to the parameters of the non - linear transformation model; Add an indication mark in the deformation field according to the offset direction and offset size; wherein, the indication mark is used to characterize the deformation trend of the target part.

4. An image processing device, characterized in that, it includes: An image acquisition module for obtaining a first image and a second image; wherein, the first image and the second image are respectively obtained by photographing the same object at different times; An image registration module for using a non - linear transformation model to perform registration processing on the second image to obtain a third image; wherein, the non - linear transformation model is determined according to the feature points extracted from the target parts of the object in the first image and the second image respectively and the registration accuracy; An image synthesis module, configured to calculate a deformation field based on the third image and the first image, and synthesize the first image with the deformation field to obtain a synthesized picture for characterizing the deformation effect of the target part; The image processing device further includes a model determination module, and the model determination module includes a second extraction unit, a parameter solving unit, a registration processing unit, and an image comparison unit; The second extraction unit is configured to respectively extract a plurality of feature points of the target part of the object in the first image and the second image; The parameter solving unit is configured to input the coordinates of all the extracted feature points into a preset non-linear transformation model to solve the parameters of the non-linear transformation model; The registration processing unit is configured to perform registration processing on the second image by using the non-linear transformation model according to the solved parameters to obtain an intermediate image; The image comparison unit is configured to compare the intermediate image with the first image to obtain a registration accuracy, and optimize the extracted feature points when the registration accuracy does not meet the requirements, and call the parameter solving unit.

5. The image processing device according to claim 4, wherein, the image processing device further includes an image processing module; the image processing module includes: A first extraction unit, configured to extract at least three first feature points of the target part of the object in the first image, and extract at least three second feature points of the target part in the second image; wherein, the second feature points correspond to the first feature points one by one; A matrix determination unit, configured to determine an affine transformation matrix according to the coordinates of the first feature points and the second feature points; A coordinate transformation unit, configured to perform coordinate transformation processing on the second image according to the affine transformation matrix.

6. The image processing device according to any one of claims 4 or 5, wherein, The image synthesis module is specifically configured to calculate the offset direction and offset size of the grid nodes in the deformation field according to the parameters of the non-linear transformation model, and add an indication mark in the deformation field according to the offset direction and offset size; wherein, the indication mark is used to characterize the deformation trend of the target part.

7. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the computer program, the image processing method according to any one of claims 1-3 is implemented.

8. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the image processing method according to any one of claims 1-3 is implemented.

Images