Image processing method and device, storage medium, communication equipment and chip

Abstract

The invention provides an image processing method which comprises the steps that a second image is determined based on a first image, the first image is determined through first direction mapping of a third image, and image areas of the first image and the second image are the same; based on the first image, the second image and the third image, a fourth image is determined, the fourth image is determined through second direction mapping of the second image, and the second direction mapping is opposite to the first direction mapping in direction. According to the scheme, on the basis of the third image and the first image obtained by the third image through mesh deformation in the first direction, the fourth image obtained by performing mesh deformation on the first image in the second direction is determined through a mode of determining the mesh point coordinates and performing coordinate transformation, so that the inverse deformation solving method of mesh deformation is provided; the method can be flexibly applied to an image processing scene needing bidirectional deformation mapping, and the application range of a grid deformation algorithm is widened.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, apparatus, storage medium, communication equipment and chip. Background Art

[0002] With the popularization of smart terminal devices and the development of image processing, the processing algorithms used in image processing have become more diverse. Among them, mesh grid deformation is a commonly used image processing algorithm in the field of image processing. It sets a certain distribution of grid points on the image to be processed as the deformation parameters of the image warping, takes each grid point as a deformation control point, gives the deformation displacement, and then determines the position of each control point after deformation to achieve the deformation of the entire image. However, the above-mentioned mesh deformation process is one-way, that is, it can only be deformed from the image to be processed to the processed image, and it is impossible to reversely deform from the processed image to the image to be processed. Summary of the invention

[0003] The present disclosure provides an image processing method, apparatus, storage medium, communication equipment and chip to solve the problems in related technologies and realize the inverse deformation solution of mesh deformation, so that mesh deformation can be flexibly applied to image processing scenarios requiring bidirectional deformation mapping, thereby improving the scope of application of mesh deformation algorithms and the accuracy of image processing.

[0004] A first aspect embodiment of the present disclosure proposes an image processing method, comprising: determining a second image based on a first image, wherein the first image is determined by performing a first direction mapping on the third image, and the image areas of the first image and the second image are the same; determining a fourth image based on the first image, the second image and the third image, wherein the fourth image is determined by performing a second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping.

[0005] In some embodiments, determining the second image based on the first image includes: determining an image area of ​​the second image based on the first image; and determining a first grid point of the second image.

[0006] In some embodiments, the coordinate systems of the first image, the second image, the third image, and the fourth image are the same.

[0007] In some embodiments, based on the first image, the second image and the third image, determining the fourth image includes: determining the first coordinates of the first grid point in the second grid of the first image; based on the first coordinates and the second grid, determining the first interpolation coefficient; based on the first interpolation coefficient, the first image and the third image, determining the fourth image.

[0008] In some embodiments, determining the second grid of the first coordinate of the first grid point in the first image includes: determining the second grid point of the first image, wherein the distance between the second grid point and the first coordinate satisfies a preset condition; determining the second grid point as the first vertex of the second grid; and determining the second grid based on the first vertex.

[0009] In some embodiments, determining the first interpolation coefficient based on the first coordinate and the second mesh includes: determining the first interpolation coefficient based on the first coordinate and coordinates of a second vertex of the second mesh, the second vertex including the first vertex.

[0010] In some embodiments, determining the fourth image based on the first interpolation coefficient, the first image, and the third image includes: determining a second coordinate of the first grid point in the fourth image; and determining the fourth image based on the second coordinate.

[0011] In some embodiments, determining the second coordinates of the first grid in the fourth image includes: determining the third coordinates of the first vertex in the third image; determining a first interval between adjacent grid points in the third image; and determining the second coordinate based on the third coordinate, the first interval, and the first interpolation coefficient.

[0012] A second aspect of the present disclosure provides an image processing device, which includes: a first determination unit, used to determine a second image based on a first image, wherein the first image is determined by performing a first direction mapping on the third image, and the image areas of the first image and the second image are the same; a second determination unit, used to determine a fourth image based on the first image, the second image and the third image, wherein the fourth image is determined by performing a second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping.

[0013] The third method embodiment of the present disclosure proposes a computer-readable storage medium, wherein when the computer program is executed by a processor, the method described in the first aspect embodiment of the present disclosure is performed.

[0014] The fourth aspect embodiment of the present disclosure proposes a communication device, including: a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the method described in the first aspect embodiment of the present disclosure when executing the computer program.

[0015] The fifth aspect embodiment of the present disclosure proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of a communication device and send signals to the processor, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, the communication device executes the method described in the first aspect embodiment of the present disclosure.

[0016] In summary, according to the image processing method proposed in the present disclosure, the method includes: determining the second image based on the first image, wherein the first image is determined by performing first direction mapping on the third image, and the image areas of the first image and the second image are the same; determining the fourth image based on the first image, the second image and the third image, wherein the fourth image is determined by performing second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping. The scheme of the present disclosure is based on the third image and the first image obtained by mesh deformation of the third image in the first direction, and by determining the coordinates of the grid points and performing coordinate transformation, the fourth image obtained by mesh deformation of the first image in the second direction is determined, thereby proposing an inverse deformation solution method for mesh deformation. The method can be flexibly applied to image processing scenarios that require bidirectional deformation mapping, thereby improving the scope of application of mesh deformation algorithms.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0019] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0020] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0021] Figure 3 An example diagram of an image processing method provided by an embodiment of the present disclosure;

[0022] Figure 4 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0023] Figure 5 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0024] Figure 6 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0025] Figure 7 A flowchart of an image processing method provided by the embodiment of the present disclosure

[0026] Figure 8 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0027] Fig. 9A schematic diagram of the structure of a communication device provided in an embodiment of the present disclosure;

[0028] Fig.10 A schematic diagram of the structure of a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0030] Therefore, in order to solve the problems existing in the related art, the present disclosure proposes an image processing method, which determines a fourth image obtained by mesh deformation of the first image in a second direction based on a third image and a first image obtained by meshing the third image in a first direction, by determining the coordinates of the grid points and performing coordinate transformation. The method can be flexibly applied to image processing scenarios that require bidirectional deformation mapping, thereby improving the scope of application of the mesh deformation algorithm.

[0031] Before introducing the detailed solution of the present disclosure, the scenario to which the solution of the present disclosure is applied is described first. Figure 1 FIG. 1 is an application scenario diagram of an image processing method in an embodiment. Figure 1 As shown, the application scenario includes an electronic device 104, in which a camera module and several applications can be installed. The application can initiate an image acquisition instruction to acquire an image, and the camera module acquires the image 102. Among them, the camera module can include a front camera module and / or a rear camera module. Finally, the target image is sent to the target application. Among them, the electronic device 104 can be a smart phone, a tablet computer, a personal digital assistant, a wearable device, etc.

[0032] In some optional embodiments, the electronic device may also be a vehicle-mounted device or an Internet of Vehicles device, such as a smart car, etc. In the present disclosure, only a smart phone is taken as an example, but it does not limit the scope of the present disclosure.

[0033] A camera can be installed on an electronic device, and images can be acquired through the installed camera. Cameras can be divided into laser cameras, visible light cameras and other types according to the different images acquired. Laser cameras can acquire images formed by irradiating lasers onto objects, and visible light cameras can acquire images formed by irradiating visible light onto objects. Several cameras can be installed on an electronic device, and the installation locations are not limited. For example, a camera can be installed on the front panel of an electronic device, and two cameras can be installed on the back panel. The camera can also be installed inside the electronic device in an embedded manner, and then the camera can be opened by rotating or sliding. Specifically, a front camera and a rear camera can be installed on an electronic device. The front camera and the rear camera can acquire images from different perspectives. Generally, the front camera can acquire images from the front perspective of the electronic device, and the rear camera can acquire images from the back perspective of the electronic device.

[0034] It should be understood that in the present disclosure, the front camera or the rear camera is only used as an example to distinguish the shooting angles of different cameras, rather than to limit the functions of multiple cameras. The multiple cameras in the present disclosure can be rear cameras or front cameras at the same time, which is not limited in the present disclosure.

[0035] An electronic device can have several applications installed. An application is software written for a certain purpose in an electronic device. An electronic device can use an application to provide services to users. When an application needs to capture an image, it will initiate an image capture instruction. The electronic device will call the camera module to capture the image according to the image capture instruction. An image capture instruction is an instruction used to trigger an image capture operation.

[0036] The electronic device is also provided with a processor, and the image processing module in the processor can perform image processing on the image captured by the camera module, for example, execute the image processing method provided in the present disclosure.

[0037] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure. The method can be applied to Figure 1 The application scenario shown is, for example, executed by a terminal with an integrated image processing function or an image processor in the terminal, and the present disclosure is not limited thereto. Figure 2 As shown, the image processing method includes steps 201-204.

[0038] Step 201: determine a second image based on a first image.

[0039] In some embodiments, the first image is determined by performing first direction mapping on the third image, wherein the first direction may be a positive direction or a negative direction, which is not limited in the present disclosure.

[0040] For example, Figure 3 A schematic diagram of a grid forward mapping and reverse mapping. Figure 3 As shown, when the third image is the target image, its grid point is G ref_dst When the first direction mapping is reverse mapping, the first image is the source image, and its grid point is G warp_src ; When the third image is the target image, its grid point is G ref_src When the first mapping direction is forward mapping, the first image is the source image, and its grid point is G warp_dst .

[0041] In some embodiments, the first image, the second image, the third image, and the fourth image are images comprising grid points.

[0042] In some embodiments, determining the second image based on the first image includes determining the image area of ​​the second image based on the first image, and determining the first grid points of the second image, wherein the number and spacing of the first grid points of the second image may be the same as or different from the number and spacing of the grid points of the third image, and the present disclosure is not limited to this.

[0043] In some embodiments, the first image corresponds to the second grid points, and the second grid points constitute the second grid; the second image corresponds to the first grid points, and the first grid points constitute the first grid.

[0044] Step 202: Determine a fourth image based on the first image, the second image, and the third image.

[0045] In some embodiments, based on the first image, the second image and the third image, determining the fourth image includes: determining the first coordinates of the first grid point in the first image in the second grid in the first image; determining the first interpolation coefficient based on the first coordinates and the second grid; determining the fourth image based on the first interpolation coefficient, the first image and the third image.

[0046] In some embodiments, the fourth image is determined by performing a second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping.

[0047] For example, Figure 3 As shown, when the first direction mapping is reverse mapping, the second direction mapping is forward mapping, the second image is the source image, and its grid point is G ref_src , the fourth image is the target image, and its grid points are G warp_dst ; When the first direction mapping is forward mapping, the second direction mapping is reverse mapping, the second image is the target image, and its grid point is G ref_dst , the fourth image is the source image, and its grid points are G warp_src .

[0048] In some embodiments, the coordinate systems of the first image, the second image, the third image and the fourth image are the same to facilitate determination of the coordinate positions of the grid points in different images, wherein the same coordinate systems mean that the positions of the origins of the coordinate systems are the same and the scale sizes of the coordinate axes are the same.

[0049] It should be understood that the number of grid points in each row and each column in the first image and the third image is the same, and the number of grid points in each row and each column in the second image and the fourth image is the same.

[0050] In summary, according to the image processing method proposed in the present disclosure, the method includes: determining the second image based on the first image, wherein the first image is determined by performing first direction mapping on the third image, and the image areas of the first image and the second image are the same; determining the fourth image based on the first image, the second image and the third image, wherein the fourth image is determined by performing second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping. The scheme of the present disclosure determines the interpolation coefficient based on the third image and the first image obtained by meshing the third image in the first direction, and then determines the fourth image obtained by meshing the first image in the second direction, thereby proposing an inverse deformation solution method for mesh deformation. The method can be flexibly applied to image processing scenarios that require bidirectional deformation mapping, thereby improving the scope of application of mesh deformation algorithms.

[0051] Figure 4 The following further shows a flowchart of an image processing method proposed in the present disclosure. Figure 2 The embodiment shown further explains step 201. Figure 4 The following steps may be included.

[0052] Step 401: determine an image area of ​​a second image based on a first image.

[0053] In some embodiments, the second image area may be determined based on the first image, that is, the image area of ​​the second image may be the entire image area of ​​the first image or a partial image area of ​​the first image, which is not limited in the present disclosure.

[0054] Step 402: determine the first grid points of the second image.

[0055] In some embodiments, the first grid points can be determined based on the processing accuracy requirement of the second image, that is, the higher the processing accuracy required for the second image, the more first grid points are determined and the smaller the spacing between the first grid points; the first grid points can also be determined based on the capabilities of the device executing this solution, that is, the greater the device capabilities, the higher the available computing power, the more first grid points are determined and the smaller the spacing between the first grid points, but not limited to this, the present disclosure does not limit the method for determining the first grid points of the second image.

[0056] In summary, according to the image processing method proposed in the present disclosure, the method includes: determining the image area of ​​the second image based on the first image; determining the first grid point of the second image. The method of the present disclosure determines the second image by determining the image area of ​​the second image and the first grid point of the second image, laying a foundation for determining the fourth image. At the same time, the first grid point can be flexibly determined based on conditions such as the accuracy required for image processing and the capability of the execution device, thereby increasing the scope of application of the present solution.

[0057] Figure 5 The following further shows a flowchart of an image processing method proposed in the present disclosure. Figure 5 The embodiment shown further explains step 202. Figure 5 The following steps may be included.

[0058] Step 501: determine a second grid in a first image with a first coordinate of a first grid point.

[0059] In some embodiments, a second grid in the first image may be determined for first coordinates of first grid points to determine first interpolation coefficients based on the first coordinates and the second grid.

[0060] In some embodiments, the second grid of the first image is an irregular trapezoid. The second grid point of the first image can be determined to further determine the second grid of the first coordinate in the first image, wherein the first coordinate can be within or outside the second grid, and the present disclosure does not limit this.

[0061] In some embodiments, the distance between the second grid point and the first coordinate satisfies a preset condition. In other words, the second grid point may be the grid point closest to the first coordinate point in the first image, but is not limited thereto. The present disclosure does not impose any restrictions on the preset condition.

[0062] Step 502: Determine a first interpolation coefficient based on the first coordinate and the second grid.

[0063] In some embodiments, a first interpolation coefficient may be determined based on the first coordinates and the second grid, so as to determine a fourth image based on the first interpolation coefficient.

[0064] In some embodiments, a first interpolation coefficient may be determined based on the first coordinates and coordinates of a second vertex of the second mesh, the second vertex including the first vertex.

[0065] In some embodiments, the first vertex is a second grid point that meets a preset condition in step 501, and the second vertices are four vertices of the second grid.

[0066] In some embodiments, the first interpolation coefficient may be determined by:

[0067] set up Where A, B, C, and D are the coordinates of the second vertex of the second mesh, u and v are the first interpolation coefficients, p is the first coordinate, and M and N are intermediate variables. The above formula group can be simplified to obtain the formula:

[0068]

[0069] Substituting the first coordinate and the coordinate of the second vertex of the second grid into equation 1 and solving it, the values ​​of the first interpolation coefficients u and v can be determined.

[0070] Step 503: determine a fourth image based on the first interpolation coefficient, the first image and the third image.

[0071] In some embodiments, the second coordinates of the first grid point in the fourth image may be determined based on the first interpolation coefficient, the first image, and the third image. In other words, the second coordinates are the coordinates corresponding to the first grid point after grid deformation.

[0072] In some embodiments, based on the second coordinates, the grid points of the first grid points after grid deformation, that is, the grid points of the fourth image, can be determined, thereby achieving determination of the fourth image.

[0073] In summary, the image processing method proposed in the present disclosure includes: determining the second grid of the first coordinate of the first grid point in the first image; determining the first interpolation coefficient based on the first coordinate and the second grid; and determining the fourth image based on the first interpolation coefficient, the first image and the third image. The method of the present disclosure determines the second grid of the first coordinate of the first grid point in the first image, and then determines the interpolation coefficient based on the first coordinate and the determined second grid, so as to determine the fourth image according to the interpolation coefficient, thereby realizing the inverse deformation solution of the grid deformation, so that the grid deformation can be flexibly applied to image processing scenarios that require bidirectional deformation mapping, thereby improving the scope of application of the grid deformation algorithm.

[0074] Figure 6 The following further shows a flowchart of an image processing method proposed in the present disclosure. Figure 5 The embodiment shown further explains step 501. Figure 6 The following steps may be included.

[0075] Step 601, determining second grid points of the first image.

[0076] In some embodiments, the second grid point may be determined based on the first coordinate of the first grid point in the second image, wherein the distance between the second grid point and the first coordinate satisfies a preset condition.

[0077] In some embodiments, the preset condition may be the closest distance to the first coordinate, that is, the second grid point may be the grid point in the first image that is closest to the first coordinate point, but the present disclosure does not limit the preset condition.

[0078] In some embodiments, the second grid point can be determined by traversing all grid points in the first image and calculating the distance between each grid point and the first coordinate. The second grid point can also be determined based on prior results, such as the maximum deformation distance of the grid deformation, by calculating the distance between each grid point and the first coordinate within the maximum deformation distance. The present disclosure does not limit the method for determining the second grid point of the first image.

[0079] Step 602: determine the second grid point as the first vertex of the second grid.

[0080] In some embodiments, the second mesh point may be determined as a first vertex of the second mesh to determine the second mesh based on the first vertex.

[0081] In some embodiments, the first vertex may be any vertex among the upper left vertex, the upper right vertex, the lower left vertex, and the lower right vertex of the second mesh, which is not limited in the present disclosure.

[0082] Step 603: Determine a second mesh based on the first vertex.

[0083] In some embodiments, the second grid can be determined based on the first vertex. For example, if the first vertex is the upper left vertex of the second grid, then the grid point adjacent to the right side of the first vertex can be determined to be the upper right vertex of the second grid, the grid point adjacent to the lower side of the first vertex can be determined to be the lower left vertex of the second grid, the grid point adjacent to the right side of the lower left vertex can be determined to be the lower right vertex, and the lower right vertex is adjacent to the upper right vertex.

[0084] It should be understood that the upper left vertex and the upper right vertex of the second grid are in the same row, the lower left vertex and the lower right vertex are in the same row, the upper left vertex and the lower left vertex are in the same column, and the upper right vertex and the lower right vertex are in the same column.

[0085] In some embodiments, the first coordinate may be within the second grid or outside the second grid, which is not limited in the present disclosure.

[0086] In summary, the image processing method proposed in the present disclosure includes: determining a second grid point of the first image, wherein the distance between the second grid point and the first coordinate satisfies a preset condition; determining the second grid point as the first vertex of the second grid; and determining the second grid based on the first vertex. The scheme of the present disclosure determines the grid point of the first image that satisfies the preset condition as the second grid point, and then determines the grid of the first coordinate in the second image based on the second grid point, thereby realizing a method for quickly determining the second grid corresponding to the first coordinate, thereby improving the image processing efficiency.

[0087] Figure 7 The following further shows a flowchart of an image processing method proposed in the present disclosure. Figure 5 The embodiment shown further explains the determination of the second coordinates of the first grid in the fourth image. Figure 7 The following steps may be included.

[0088] Step 701, determining a third coordinate of a first vertex in a third image.

[0089] In some embodiments, by determining a third coordinate of the first vertex in the third image, a foundation is laid for determining the second coordinate.

[0090] In some embodiments, the coordinates of the first vertex in the third image can be determined by determining the row number and column number of the first vertex in the first image, and determining the row number and column number of the first vertex in the third image after mesh deformation, wherein the row number and column number of the first vertex in the first image are the same as the row number and column number of the first vertex in the third image.

[0091] Step 702: determine a first interval between adjacent grid points in a third image.

[0092] In some embodiments, a basis for determining the second coordinate is provided by determining a first interval between adjacent grid points in the third image.

[0093] In some embodiments, the first interval includes an interval between left-right adjacent grid points and an interval between top-bottom adjacent grid points in the third image.

[0094] Step 703, determining the second coordinate based on the third coordinate, the first interval and the first interpolation coefficient.

[0095] In some embodiments, the second coordinates are determined based on the third coordinates, the first interval, and the first interpolation coefficient to determine the fourth image.

[0096] In some embodiments, the second coordinate is the coordinate corresponding to the first grid point after grid deformation.

[0097] In some embodiments, the second coordinate may be determined by:

[0098] Let K = [u, v] T ,G E =[E x ,E y ] T , S = [s x ,s y ] T , where u and v are the first interpolation coefficients, (E x ,E y) is the third coordinate, (s x ,s y ) is the first interval, K, G E , S is the intermediate variable, T represents the transposition, then the above formula group can be simplified to obtain the formula:

[0099] q=[q x ,q y ] T =G E +K·S (Formula 2)

[0100] Among them, (q x ,q y ) represents the second coordinate, and · represents the vector inner product. It should be understood that when the first coordinate is within the second grid, K·S represents an interpolation operation; when the first coordinate is outside the second grid, K·S represents an extrapolation operation, and an interpolation operation.

[0101] In summary, the image processing method proposed in the present disclosure includes: determining the third coordinate of the first vertex in the third image; determining the first interval between adjacent grid points in the third image; and determining the second coordinate based on the third coordinate, the first interval, and the first interpolation coefficient. The scheme of the present disclosure determines the coordinate position of the first grid point when the inverse deformation of the grid deformation is performed by determining the third coordinate and the first interval, and determining the second coordinate based on the third coordinate, the first interval, and the first interpolation coefficient, thereby laying a foundation for determining the fourth image.

[0102] Therefore, this solution has the following beneficial effects:

[0103] 1. The scheme disclosed in the present invention determines the fourth image obtained by meshing the first image in the second direction by determining the coordinates of the grid points and performing coordinate transformation, thereby proposing a method for solving the inverse deformation of the grid deformation, solving the problem that there is no analytical solution to the inverse of the grid deformation process.

[0104] 2. The scheme disclosed in the present invention can be flexibly applied to image processing scenarios that require bidirectional deformation mapping. At the same time, the commonly used matrices of affine and perspective transformations can be expressed as mesh grids and then applied to the present method for image processing, thereby improving the scope of application of the mesh deformation algorithm.

[0105] 3. The scheme disclosed in the present invention determines the grid points of the first image that meet the preset conditions as the second grid points, and then determines the grid of the first coordinate in the second image based on the second grid points, thereby realizing a method for quickly determining the second grid corresponding to the first coordinate, thereby improving image processing efficiency.

[0106] Figure 8The structure diagram of an image processing device 800 provided in an embodiment of the present disclosure includes:

[0107] A first determining unit 810 is configured to determine a second image based on a first image, wherein the first image is determined by performing first direction mapping on the third image, and the first image and the second image have the same image area;

[0108] The second determining unit 820 is used to determine a fourth image based on the first image, the second image and the third image, wherein the fourth image is determined by performing second direction mapping on the second image, and the second mapping direction is opposite to the first mapping direction.

[0109] In some embodiments, the first determination unit 810 is further configured to determine an image area of ​​the second image based on the first image; and determine a first grid point of the second image.

[0110] In some embodiments, the coordinate systems of the first image, the second image, the third image, and the fourth image are the same.

[0111] In some embodiments, the second determination unit 820 is further used to determine the second grid of the first coordinates of the first grid point in the first image; determine the first interpolation coefficient based on the first coordinates and the second grid; and determine the fourth image based on the first interpolation coefficient, the first image and the third image.

[0112] In some embodiments, the second determination unit 820 is further used to: determine a second grid point of the first image, where the distance between the second grid point and the first coordinate satisfies a preset condition; determine the second grid point as a first vertex of the second grid; and determine the second grid based on the first vertex.

[0113] In some embodiments, the second determination unit 820 is further configured to determine a first interpolation coefficient based on the first coordinate and a coordinate of a second vertex of the second mesh, the second vertex including the first vertex.

[0114] The second determining unit 820 is further configured to: determine a second coordinate of the first grid point in the fourth image; and determine the fourth image based on the second coordinate.

[0115] The second determination unit 820 is further used to: determine a third coordinate of the first vertex in the third image; determine a first interval between adjacent grid points in the third image; and determine a second coordinate based on the third coordinate, the first interval and the first interpolation coefficient.

[0116] According to the image processing device proposed in the present disclosure, a second image is determined based on a first image, wherein the first image is determined by performing a first direction mapping on the third image, and the image areas of the first image and the second image are the same; a fourth image is determined based on the first image, the second image and the third image, wherein the fourth image is determined by performing a second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping. The scheme of the present disclosure is based on a first image obtained by deforming the third image and the third image through a mesh grid in a first direction, and by determining the coordinates of the grid points and performing a coordinate transformation, a fourth image obtained by performing a mesh deformation on the first image in a second direction is determined, thereby proposing an inverse deformation solution method for mesh deformation. The method can be flexibly applied to image processing scenarios requiring bidirectional deformation mapping, thereby improving the scope of application of mesh deformation algorithms.

[0117] Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation of the method is also applicable to the device provided in the embodiment and will not be described in detail in this embodiment.

[0118] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the communication device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0119] Fig. 9 1 is a schematic diagram of the structure of a communication device 900 provided in an embodiment of the present application. The communication device 900 may be a network device, or a terminal device, or a chip, a chip system, or a processor that supports the network device to implement the above method, or a chip, a chip system, or a processor that supports the terminal device to implement the above method. The device may be used to implement the method described in the above method embodiment, and the details may refer to the description in the above method embodiment.

[0120] The communication device 900 may include one or more processors 901. The processor 901 may be a general-purpose processor or a dedicated processor, etc. For example, it may be a baseband processor or a central processing unit. The baseband processor may be used to process the communication protocol and the communication data, and the central processing unit may be used to control the communication device (such as a base station, a baseband chip, a terminal device, a terminal device chip, a DU or a CU, etc.), execute a computer program, and process the data of the computer program.

[0121] Optionally, the communication device 900 may further include one or more memories 902, on which a computer program 904 may be stored, and the processor 901 executes the computer program 904 so that the communication device 900 performs the method described in the above method embodiment. Optionally, data may also be stored in the memory 902. The communication device 900 and the memory 902 may be provided separately or integrated together.

[0122] Optionally, the communication device 900 may further include a transceiver 905 and an antenna 906. The transceiver 905 may be referred to as a transceiver unit, a transceiver, or a transceiver circuit, etc., and is used to implement a transceiver function. The transceiver 905 may include a receiver and a transmitter, the receiver may be referred to as a receiver or a receiving circuit, etc., and is used to implement a receiving function; the transmitter may be referred to as a transmitter or a transmitting circuit, etc., and is used to implement a transmitting function.

[0123] Optionally, the communication device 900 may further include one or more interface circuits 907. The interface circuit 907 is used to receive code instructions and transmit them to the processor 901. The processor 901 executes the code instructions to enable the communication device 900 to execute the method described in the above method embodiment.

[0124] In one implementation, the processor 901 may include a transceiver for implementing receiving and sending functions. For example, the transceiver may be a transceiver circuit, an interface, or an interface circuit. The transceiver circuit, interface, or interface circuit for implementing the receiving and sending functions may be separate or integrated. The above-mentioned transceiver circuit, interface, or interface circuit may be used for reading and writing code / data, or the above-mentioned transceiver circuit, interface, or interface circuit may be used for transmitting or delivering signals.

[0125] In one implementation, the processor 901 may store a computer program 903, which runs on the processor 901 and enables the communication device 900 to perform the method described in the above method embodiment. The computer program 903 may be fixed in the processor 901, in which case the processor 901 may be implemented by hardware.

[0126] In one implementation, the communication device 900 may include a circuit that can implement the functions of sending, receiving or communicating in the aforementioned method embodiments. The processor and transceiver described in the present application can be implemented in an integrated circuit (IC), an analog IC, a radio frequency integrated circuit RFIC, a mixed signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductor (CMOS), N-type metal oxide semiconductor (nMetal-oxide-semiconductor, NMOS), P-type metal oxide semiconductor (positive channelmetal oxide semiconductor, PMOS), bipolar junction transistor (bipolar junction transistor, BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

[0127] The communication device described in the above embodiments may be a network device or a terminal device, but the scope of the communication device described in this application is not limited thereto, and the structure of the communication device may not be limited thereto. Fig. 9 The communication device may be a stand-alone device or may be part of a larger device. For example, the communication device may be:

[0128] (1) Independent integrated circuit IC, or chip, or chip system or subsystem;

[0129] (2) having a set of one or more ICs, and optionally, the IC set may also include a storage component for storing data and computer programs;

[0130] (3) ASIC, such as modem;

[0131] (4) Modules that can be embedded in other devices;

[0132] (5) Receivers, terminal devices, intelligent terminal devices, cellular phones, wireless devices, handheld devices, mobile units, vehicle-mounted devices, network devices, cloud devices, artificial intelligence devices, etc.;

[0133] (6)Others

[0134] For the case where the communication device may be a chip or a chip system, see Fig.10 Schematic diagram of the chip structure shown.

[0135] The embodiment of the present disclosure also provides a chip, such as Fig.10 The chip shown includes a processor 1001 and an interface 1002. The number of the processor 1001 may be one or more, and the number of the interface 1002 may be multiple.

[0136] Optionally, the chip further includes a memory 1003, and the memory 1003 is used to store necessary computer programs and data.

[0137] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the above embodiments of the present disclosure.

[0138] Those skilled in the art may also appreciate that the various illustrative logic blocks and steps listed in the embodiments of the present application may be implemented by electronic hardware, computer software, or a combination of the two. Whether such functions are implemented by hardware or software depends on the design requirements of the specific application and the entire system. Those skilled in the art may use various methods to implement the functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0139] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0140] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0141] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0143] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (P GA), a field programmable gate array (FPGA), etc.

[0144] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0145] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0146] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises: Determine a second image based on a first image, wherein the first image is determined by performing first direction mapping on a third image, and the first image and the second image have the same image area; A fourth image is determined based on the first image, the second image, and the third image, wherein the fourth image is determined by performing a second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping.

2. The method according to claim 1, characterized in that The determining the second image based on the first image comprises: Based on the first image, determining an image area of ​​the second image; A first grid of points of the second image is determined.

3. The method according to claim 1, characterized in that The coordinate systems of the first image, the second image, the third image, and the fourth image are the same.

4. The method according to claim 1, characterized in that The determining the fourth image based on the first image, the second image, and the third image comprises: Determine a second grid in the first image with the first coordinates of the first grid point; Determining a first interpolation coefficient based on the first coordinates and the second grid; The fourth image is determined based on the first interpolation coefficients, the first image, and the third image.

5. The method according to claim 4, characterized in that Determining a second grid of first coordinates of the first grid point in the first image comprises: Determine a second grid point of the first image, wherein a distance between the second grid point and the first coordinate satisfies a preset condition; Determining the second grid point as a first vertex of the second grid; Based on the first vertices, the second mesh is determined.

6. The method according to claim 4, characterized in that The determining a first interpolation coefficient based on the first coordinate and the second grid comprises: The first interpolation coefficient is determined based on the first coordinates and coordinates of second vertices of the second mesh, the second vertices including the first vertex.

7. The method according to claim 4, characterized in that The determining the fourth image based on the first interpolation coefficient, the first image, and the third image comprises: determining a second coordinate of the first grid point in the fourth image; Based on the second coordinates, the fourth image is determined.

8. The method according to claim 7, characterized in that The determining the second coordinates of the first grid in the fourth image comprises: determining a third coordinate of the first vertex in the third image; determining a first interval between adjacent grid points in the third image; The second coordinate is determined based on the third coordinate, the first interval, and the first interpolation coefficient.

9. An image processing device, characterized in that: The device comprises: A first determining unit is used to determine a second image based on a first image, wherein the first image is determined by performing first direction mapping on a third image, and the first image and the second image have the same image area; The second determining unit is used to determine a fourth image based on the first image, the second image and the third image, wherein the fourth image is determined by performing a second direction mapping on the second image, and the second direction mapping is opposite to the first direction mapping.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. A communication device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

12. A chip, characterized in that: The invention comprises one or more interfaces and one or more processors; the interface is used to receive a signal from a memory of a communication device and send the signal to the processor, wherein the signal includes a computer instruction stored in the memory; when the processor executes the computer instruction, the communication device executes the method described in any one of claims 1 to 8.

Images