An image data processing method, computer device and medium

Abstract

Embodiments of the present application disclose an image data processing method, a computer device, and a medium. The method includes: obtaining a horizontal gradient map and a vertical gradient map of a target image, and generating a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map; based on the covariance feature map, mapping each pixel point in the target image to a spatial coordinate system to obtain the corresponding three-dimensional spatial coordinates of each pixel point; according to the three-dimensional spatial coordinates of each pixel point, determining the target cone to which each pixel point belongs, and obtaining the cone gradient feature of the target cone to which each pixel point belongs; the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which they belong; according to the cone gradient feature to which each pixel point belongs, determining the image gradient feature of the target image. By using the present application, the quality of the obtained image gradient feature can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image data processing method, computer equipment and medium. Background Art

[0002] With the rapid development of computer technology and multimedia technology, digital images have been increasingly used in daily life, work, and study. Among them, image gradient features are a basic image feature that can be widely used in various image processing scenarios, such as image super-resolution, denoising, and high dynamic range (HDR) processing.

[0003] Image gradient feature is an image feature calculated based on gradient. Generally speaking, the image can be convolved using a gradient feature operator to obtain the gradient value of each pixel in the image. The image gradient feature of the image can be obtained by mathematically transforming the obtained gradient value. However, the image gradient feature obtained in this way has a poor discreteness. When the image gradient feature with poor discreteness is applied to the super-resolution processing scenario of the image (or other scenarios), the effect of super-resolution processing of the image will be poor. Therefore, how to obtain better quality image gradient features (such as image gradient features with better discreteness) has become a hot research issue in current computer vision technology. Summary of the invention

[0004] The embodiments of the present application provide an image data processing method, a computer device, and a medium, which can improve the quality of the obtained image gradient features, so that when the image gradient features are subsequently applied to various image processing applications, the image processing effect is better.

[0005] The first aspect of the embodiment of the present application discloses an image data processing method, the method comprising:

[0006] Acquire a horizontal gradient map and a vertical gradient map of a target image, and generate a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map;

[0007] Based on the cosquare feature map, each pixel point in the target image is mapped to a spatial coordinate system to obtain the three-dimensional spatial coordinates corresponding to each pixel point;

[0008] According to the three-dimensional space coordinates of each pixel point, the target cone to which each pixel point belongs is determined, and the cone gradient feature of the target cone to which each pixel point belongs is obtained; the coordinate position indicated by the three-dimensional space coordinates of each pixel point is respectively located on the cone arc surface of the target cone to which it belongs;

[0009] Determine the image gradient feature of the target image according to the conical gradient feature to which each pixel point belongs.

[0010] Optionally, the specific implementation manner of determining the horizontal gradient map and the vertical gradient map of the target image according to the pixel values of each pixel point in the target image is as follows:

[0011] Perform a convolution operation on the target image based on a horizontal gradient convolution kernel to obtain the horizontal gradient map;

[0012] Perform a convolution operation on the target image based on a vertical gradient convolution kernel to obtain the vertical gradient map.

[0013] Optionally, the specific implementation manner of generating the covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map is as follows:

[0014] Generate an initial covariance feature map based on the horizontal gradient map and the vertical gradient map;

[0015] Perform a convolution operation on the initial covariance feature map based on an image blur convolution kernel to obtain the covariance feature map after image blur.

[0016] Optionally, the horizontal gradient map includes the horizontal gradient values corresponding to each pixel point respectively; the vertical gradient map includes the vertical gradient values corresponding to each pixel point respectively; the specific implementation manner of generating the cross-covariance feature map of the target image according to the horizontal gradient map and the vertical gradient map is as follows:

[0017] Determine the horizontal feature extraction region corresponding to each pixel point in the horizontal gradient map and the vertical feature extraction region corresponding to each pixel point in the vertical gradient map;

[0018] Determine the cross-covariance feature value corresponding to each pixel point according to the horizontal gradient value in the horizontal feature extraction region corresponding to each pixel point and the vertical gradient value in the vertical feature extraction region corresponding to each pixel point;

[0019] Generate the cross-covariance feature map according to the cross-covariance feature values corresponding to each pixel point.

[0020] Optionally, the specific implementation manner of obtaining the conical gradient feature of the target cone to which each pixel point belongs is as follows:

[0021] Obtain the cone radius value of the target cone to which each pixel point belongs;

[0022] Obtain a second included angle between the hypotenuse and the base of the target cone to which each pixel point belongs;

[0023] Respectively determine the cone radius value and the second included angle corresponding to each pixel point as the cone gradient feature to which each pixel point belongs.

[0024] Optionally, the method further includes:

[0025] Train an initial super-resolution model based on the image gradient feature of the target image;

[0026] Determine the trained initial super-resolution model as the super-resolution model;

[0027] Input the image to be adjusted into the super-resolution model to obtain a resolution-adjusted image of the image to be adjusted; the image resolution of the resolution-adjusted image is higher than the image resolution of the image to be adjusted.

[0028] A second aspect of the embodiments of the present application discloses an image data processing device, the device includes:

[0029] A generation unit, configured to obtain a horizontal gradient map and a vertical gradient map of a target image, and generate a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map;

[0030] A first determination unit, configured to map each pixel point in the target image to a spatial coordinate system based on the covariance feature map, and obtain the respective three-dimensional spatial coordinates corresponding to each pixel point;

[0031] An acquisition unit, configured to determine the target cone to which each pixel point belongs according to the three-dimensional spatial coordinates of each pixel point, and obtain the cone gradient feature of the target cone to which each pixel point belongs; the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which they belong;

[0032] A second determination unit, configured to determine the image gradient feature of the target image according to the cone gradient feature to which each pixel point belongs.

[0033] A third aspect of the embodiments of the present application discloses a computer device, including a processor, a memory, and a network interface, the processor, the memory, and the network interface are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method in the first aspect above.

[0034] In the fourth aspect of the embodiments of the present application, a computer-readable storage medium is disclosed. The computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method of the first aspect above.

[0035] In the fifth aspect of the embodiments of the present application, a computer program product or a computer program is disclosed. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to execute the method of the first aspect above.

[0036] In the embodiments of the present application, a computer device may obtain a horizontal gradient map and a vertical gradient map of a target image, generate a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map, and then, based on the covariance feature map, map each pixel point in the target image to a spatial coordinate system to obtain the corresponding three-dimensional spatial coordinates of each pixel point. Then, according to the three-dimensional spatial coordinates of each pixel point, the target cone to which each pixel point belongs is determined. Next, the cone gradient feature of the target cone to which each pixel point belongs is obtained. Further, the image gradient feature of the target image may be determined according to the cone gradient feature of each pixel point. By implementing the above method, the discreteness of the image gradient feature of the image is better and can be more evenly distributed in the three-dimensional space, so that the quality of the obtained image gradient feature can be improved, and when the image gradient feature is subsequently applied to various image processing applications, the image processing effect is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1a is a schematic structural diagram of an image data processing system provided by an embodiment of the present application;

[0039] Figure 1b is a schematic flowchart of an image data processing method provided by an embodiment of the present application;

[0040] Figure 2a is a schematic structural diagram of a target image, a horizontal gradient map, and a vertical gradient map provided by an embodiment of the present application;

[0041] Figure 2bIt is a schematic structural diagram of another target image, horizontal gradient map, and vertical gradient map provided by an embodiment of the present application;

[0042] Figure 2c It is a schematic structural diagram of a horizontal covariance feature map, a vertical covariance feature map, and a cross-covariance feature map provided by an embodiment of the present application;

[0043] Figure 3a It is a schematic structural diagram of a cone provided by an embodiment of the present application;

[0044] Figure 3b It is a schematic structural diagram of another cone provided by an embodiment of the present application;

[0045] Figure 4 It is a schematic flowchart of another image data processing method provided by an embodiment of the present application;

[0046] Figure 5a It is a schematic structural diagram of a horizontal covariance feature map provided by an embodiment of the present application;

[0047] Figure 5b It is a schematic structural diagram of yet another cone provided by an embodiment of the present application;

[0048] Figure 5c It is a schematic structural diagram of yet another cone provided by an embodiment of the present application;

[0049] Figure 6a It is a schematic flowchart of yet another image data processing method provided by an embodiment of the present application;

[0050] Figure 6b It is a schematic spatial diagram of an image gradient feature provided by an embodiment of the present application;

[0051] Figure 7 It is a schematic structural diagram of an image data processing device provided by an embodiment of the present application;

[0052] Figure 8 It is a schematic structural diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0053] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0054] Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making.

[0055] Artificial intelligence technology is an interdisciplinary subject with a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0056] Computer Vision Technology (CV) Computer vision is a science that studies how to enable machines to "see". More specifically, it refers to using cameras and computers to replace human eyes for tasks such as target recognition and measurement in machine vision, and further performing graphic processing to make the images processed by the computer more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, Optical Character Recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0057] The solution provided in the embodiments of this application relates to technologies such as image processing in the computer vision technology of artificial intelligence, and is specifically described through the following embodiments:

[0058] Please refer to Figure 1a , Figure 1a which is a schematic diagram of the architecture of an image data processing system provided in the embodiments of this application. This application relates to a terminal 101 and a server 102.

[0059] Taking the terminal 101 as an example, the terminal 101 can obtain the image to be processed and send the image to the server 102. The server 102 obtains the horizontal gradient map and the vertical gradient map of the image, determines the covariance feature map of the image based on the horizontal gradient map and the vertical gradient map, and based on the covariance feature map, determines the three-dimensional space coordinates corresponding to each pixel point, and according to the three-dimensional space coordinates, determines the cone to which each pixel point belongs. The server 102 will obtain the cone gradient feature of each pixel point respectively, and according to the cone gradient feature, determine the image gradient feature of the image.

[0060] Subsequently, the server 102 can apply the image gradient feature to the image features required in various image processing scenarios. For example, it can be applied to scenarios such as super-resolution, denoising, and HDR processing of images.

[0061] Figure 1a The illustrated terminal 101 can be a smart device with video processing functions such as a mobile phone, a tablet computer, a laptop computer, a handheld computer, a mobile internet device (MID), a wearable device, etc. The terminal 101 and the server 102 can be directly or indirectly connected through wired or wireless communication methods, and the present application does not limit this here.

[0062] In one implementation, an image data processing method provided in an embodiment of the present application can be executed by a computer device, where the computer device can be a terminal or a server in the above image data processing system; the general principle is as follows: the computer device can obtain the horizontal gradient map and the vertical gradient map of the target image, and generate the covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map. Then, based on the covariance feature map, each pixel point in the target image is mapped to the space coordinate system to obtain the three-dimensional space coordinates corresponding to each pixel point respectively, and according to the three-dimensional space coordinates of each pixel point, determine the target cone to which each pixel point belongs. Then, obtain the cone gradient feature of the target cone to which each pixel point belongs respectively. Further, the image gradient feature of the target image can be determined according to the cone gradient feature of each pixel point respectively. By implementing the above method, the quality of the obtained image gradient feature can be improved, so that when the image gradient feature is subsequently applied to various image processing applications, the image processing effect is better.

[0063] The implementation details of the technical solutions of the embodiments of the present application are elaborated in detail below:

[0064] Please refer to Figure 1b , Figure 1bIt is a schematic flowchart of an image data processing method provided by an embodiment of the present application. This method is applied to a computer device and can be executed by the computer device. Among them, the computer device can be a server or a terminal. As Figure 1b shown, the image data processing method may include:

[0065] S101: Obtain the horizontal gradient map and vertical gradient map of the target image, and generate the covariance feature map of the target image based on the horizontal gradient map and vertical gradient map.

[0066] In one implementation, the target image can be a low-resolution image obtained by magnifying the original image in a super-resolution application scenario.

[0067] In one implementation, the computer device can perform a convolution operation on the target image using an image gradient operator to obtain the horizontal gradient map and vertical gradient map of the target image. Among them, the image gradient operator includes a horizontal gradient convolution kernel and a vertical gradient convolution kernel. The size of the convolution kernel can be 3×3 or 5×5, which is not limited in the present application. Then, a convolution operation can be performed on the target image based on the horizontal gradient convolution kernel to obtain the horizontal gradient map of the target image, and a convolution operation can be performed on the target image based on the vertical gradient convolution kernel to obtain the vertical gradient map of the target image. For example, the horizontal gradient convolution kernel in the image gradient operator can be a 3×3 convolution kernel, such as [0,0,0;-0.5,0,0.5;0,0,0], and the vertical gradient convolution kernel can be a 3×3 convolution kernel, such as [0,-0.5,0;0,0,0;0,0.5,0]. It can also be other image gradient operators, such as the Roberts operator, Sobel operator, Prewitt operator, Laplace operator, which is not limited in the present application.

[0068] For example, taking the horizontal gradient convolution kernel in the image gradient operator as [0,0,0;-0.5,0,0.5;0,0,0] and the vertical gradient convolution kernel as [0,-0.5,0;0,0,0;0,0.5,0] as an example for illustration. For the horizontal gradient map of the target image, for each pixel point on the target image, subtract the pixel value of the pixel point to the left of this pixel point from the pixel value of the pixel point to the right of this pixel point, and then divide the difference by 2. The obtained value is the horizontal gradient value corresponding to this pixel point on the horizontal gradient map. For the vertical gradient map of the target image, for each pixel point on the target image, subtract the pixel value of the pixel point above this pixel point from the pixel value of the pixel point below this pixel point, and then divide the difference by 2. The obtained value is the vertical gradient value corresponding to this pixel point on the vertical gradient map. For example, Figure 2aThe image marked by 21 in the figure is a schematic diagram of the structure of a certain pixel point in the target image and its 8-neighborhood. Among them, the numbers in each box of the image marked by 21 represent the pixel values corresponding to the pixel point. Then, the horizontal gradient value of the pixel point with a pixel value of S5 is (S6 - S4) / 2, and the vertical gradient value is (S8 - S2) / 2. Among them, the pixel value is any value from 0 to 255.

[0069] Then, according to the above method, the horizontal gradient value and vertical gradient value of each pixel point in the target image can be determined. Then, according to the horizontal gradient value of each pixel point, a horizontal gradient map can be formed, and according to the vertical gradient value of each pixel point, a vertical gradient map can be formed. For example, Figure 2a The image marked by 22 in the figure is the horizontal gradient map corresponding to the image marked by 21, and the image marked by 23 is the vertical gradient map corresponding to the image marked by 21. Among them, the numbers in each box of the image marked by 22 represent the horizontal gradient values corresponding to the pixel points in the image marked by 21, and the numbers in each box of the image marked by 23 represent the vertical gradient values corresponding to the pixel points in the image marked by 21. The obtained (S6 - S4) / 2 is P5 in the image marked by 22, and (S8 - S2) / 2 is Q5 in the image marked by 23.

[0070] In one implementation, after the computer device determines the horizontal gradient map and vertical gradient map of the target image, it can generate the covariance feature map of the target image based on the horizontal gradient map and vertical gradient map. Among them, the covariance feature map includes a horizontal covariance feature map, a vertical covariance feature map, and a cross-covariance feature map. Optionally, the computer device can generate the horizontal covariance feature map of the target image according to the horizontal gradient map; generate the vertical covariance feature map of the target image according to the vertical gradient map; generate the cross-covariance feature map of the target image according to the horizontal gradient map and vertical gradient map.

[0071] Optionally, the specific implementation of the computer device generating the horizontal covariance feature map of the target image according to the horizontal gradient map can be described as follows:

[0072] The computer device can determine the horizontal feature extraction region corresponding to each pixel point in the horizontal gradient map, and the horizontal feature extraction region can be preset. Optionally, the horizontal feature extraction region can be an image block with a preset size centered on each pixel point. The shape of the image block can be square, rectangular, circular, etc., which is not limited in this application. Then, for a square-shaped image block, the preset size can be 7×7, 5×5, etc., and for a rectangular-shaped image block, the preset size can be 5×7, 4×8, etc. For example, Figure 2bThe image marked by 24 is the target image, where each square in the image represents a pixel. Taking the pixel S in the image marked by 24 as an example, the position of the pixel S in the horizontal gradient map can be determined. For example, the horizontal gradient map can be the image marked by 25. The position of the pixel S in the horizontal gradient map is P. Assuming that the image block of the horizontal feature extraction region is square and the preset size is 5×5, then the horizontal feature extraction region can be the square marked by 201 in the image marked by 25.

[0073] It should be noted that for the pixels on the edge of the target image, such as Figure 2b the pixel S0 in the target image marked by 24 in the figure, when determining the horizontal feature extraction region (assuming it is a 5×5 image block) of the pixel S0, it may happen that a 5×5 image block cannot be obtained on the target image. In this case, the horizontal feature extraction region can be filled to a 5×5 image block by filling. Among them, the filling method can be zero-padding, that is, padding with zeros in the insufficient area to fill to a 5×5 image block; or the replication method, that is, replicating the pixel values of the outermost pixels to fill to a 5×5 image block; or the reflection method, that is, flipping with the outermost pixel as the axis of symmetry to obtain a 5×5 image block; or other methods, which are not limited in this application. The same understanding applies to the subsequent vertical feature extraction region and cross feature extraction region.

[0074] After determining the horizontal feature extraction region corresponding to each pixel, the horizontal covariance feature value corresponding to each pixel can be determined according to the horizontal gradient value in the horizontal feature extraction region corresponding to each pixel. Specifically, for a certain pixel, the computer device can obtain the horizontal gradient value in the horizontal feature extraction region corresponding to the pixel, and then sum up each horizontal gradient value. The sum result can be the horizontal covariance feature value corresponding to the pixel. Optionally, after the above sum result, the sum result can also be averaged, and the average result can be the horizontal covariance feature value corresponding to the pixel. Optionally, before the summation process, each horizontal gradient value can also be weighted, and then the weighted processing results of each horizontal gradient value are summed up, and the sum result is the horizontal covariance feature value corresponding to the pixel; or each horizontal gradient value can be squared, and then the squared results of each horizontal gradient value are summed up, and the sum result is the horizontal covariance feature value corresponding to the pixel. It can also include any other combination of weighted processing, summation processing, or averaging processing to determine the horizontal covariance feature value corresponding to the pixel, which is not limited in this application. For example, for the above pixel S, the horizontal feature values corresponding to each small square in the square marked by 201 can be obtained first. There are 25 horizontal feature values, which can be denoted as [p1, p2,..., p25 , and then square each of the 25 horizontal eigenvalue values, and sum up the squared results of each target horizontal gradient eigenvalue value to obtain p1 2 +p2 2 +,..., +p 25 2 , then the summation result is the horizontal covariance eigenvalue corresponding to the pixel point S. After determining the horizontal covariance eigenvalue corresponding to each pixel point, a horizontal covariance feature map can be generated using the horizontal covariance eigenvalue corresponding to each pixel point. For example, Figure 2c the image marked by 27 in [] is the horizontal covariance feature map corresponding to the target image, where each square in the image represents the horizontal covariance eigenvalue corresponding to a pixel point.

[0075] Optionally, the specific implementation manner for the computer device to generate the vertical covariance feature map based on the vertical gradient map can be described as follows:

[0076] The specific implementation manner for generating the vertical covariance feature map is similar to the specific implementation manner for generating the horizontal covariance feature map described above. The computer device can determine the vertical feature extraction region corresponding to each pixel point in the vertical gradient map, and the determination method and region size of this vertical feature extraction region are the same as those of the above horizontal feature extraction region. For example, Figure 2b the image marked by 26 in [] is the vertical gradient map. Taking the pixel point S as an example, it can be determined that the position of the pixel point S in the vertical gradient map is Q, then the vertical feature extraction region can be the square marked by 202 in the image marked by 26. After determining the vertical feature extraction region corresponding to each pixel point, the vertical covariance eigenvalue corresponding to each pixel point can be determined according to the vertical gradient values in the vertical feature extraction region corresponding to each pixel point. For example, for a certain pixel point, the computer device can obtain the vertical gradient values in the vertical feature extraction region corresponding to this pixel point, then square each vertical gradient value, and then sum up the squared results of each vertical gradient value, then the summation result is the vertical covariance eigenvalue corresponding to this pixel point. There can also be other methods for determining the vertical covariance eigenvalue corresponding to each pixel point, which can refer to the method for determining the horizontal covariance eigenvalue corresponding to each pixel point described above, and will not be elaborated here. For example, Figure 2c the image marked by 28 in [] is the vertical covariance feature map corresponding to the target image, where each square in the image represents the vertical covariance eigenvalue corresponding to a pixel point.

[0077] Optionally, the specific implementation manner for the computer device to generate the cross covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map can be described as follows:

[0078] The computer device can determine the horizontal feature extraction regions corresponding to each pixel point in the horizontal gradient map and the vertical feature extraction regions corresponding to each pixel point in the vertical gradient map. For example, taking the pixel point S as an example, the horizontal feature extraction region of the pixel point S can be the square marked by 201 in the image marked by 25, and the vertical feature extraction region can be the square marked by 202 in the image marked by 26. After determining the horizontal and vertical feature extraction regions of each pixel point, the cross-covariance feature value corresponding to each pixel point can be determined according to the horizontal gradient value in the horizontal feature extraction region corresponding to each pixel point and the vertical gradient value in the vertical feature extraction region corresponding to each pixel point. Specifically, for a certain pixel point, the computer device can obtain the horizontal gradient value in the horizontal feature extraction region corresponding to the pixel point and the vertical gradient value in the vertical feature extraction region, then multiply the horizontal gradient value and the vertical gradient value at each corresponding position, and then sum up each multiplication result. The sum result is the cross-covariance feature value corresponding to the pixel point. Optionally, the above sum result can also be averaged, and the average result can be the cross-covariance feature value corresponding to the pixel point. Optionally, before the summation process, it can also be to weight each horizontal gradient value, weight each vertical gradient value, then multiply the weighted results of the horizontal gradient value and the vertical gradient value at each corresponding position, and then sum up each multiplication result. The sum result is the cross-covariance feature value corresponding to the pixel point. It can also include any other combination of weighting, summing, or averaging processes to determine the cross-covariance feature value corresponding to the pixel point, which is not limited in this application. For example, for the above pixel point S, the 25 horizontal feature values and 25 vertical feature values obtained can be respectively denoted as [p1, p2,..., p 25 , [q1, q2,..., q 25 , and then multiply the 25 horizontal feature values and 25 vertical feature values correspondingly and sum up the multiplication results to obtain p1×q1 + p2×q2 +,..., + p 25 ×q 25 . The sum result is the cross-covariance feature value corresponding to the pixel point S. After determining the cross-covariance feature value corresponding to each pixel point, the cross-covariance feature map can be generated using the cross-covariance feature value corresponding to each pixel point. For example, Figure 2c the image marked by 29 in the figure is the cross-covariance feature map corresponding to the target image, where each square in the image represents the cross-covariance feature value corresponding to a pixel point.

[0079] S102: Based on the covariance feature map, map each pixel point in the target image to a spatial coordinate system to obtain the corresponding three-dimensional spatial coordinates for each pixel point.

[0080] Among them, the spatial coordinate system may include a horizontal coordinate axis, a vertical coordinate axis, and a spatial coordinate axis, which can be denoted as the x-axis, y-axis, and z-axis respectively.

[0081] In one implementation, the computer device can determine the horizontal coordinate value corresponding to each pixel point according to the horizontal covariance feature map, determine the vertical coordinate value corresponding to each pixel point according to the vertical covariance feature map, and determine the spatial coordinate value corresponding to each pixel point according to the cross-covariance feature map. Then, according to the horizontal coordinate value corresponding to each pixel point, the vertical coordinate value corresponding to each pixel point, and the spatial coordinate value corresponding to each pixel point, the three-dimensional spatial coordinates corresponding to each pixel point can be determined. Optionally, each horizontal covariance feature value in the horizontal covariance feature map is the horizontal coordinate value corresponding to each pixel point. Similarly, each vertical covariance feature value in the vertical covariance feature map is the vertical coordinate value corresponding to each pixel point, and each cross-covariance feature value in the cross-covariance feature map is the spatial coordinate value corresponding to each pixel point. For example, for Figure 2b the pixel point S in the target image marked by 24, the horizontal coordinate value, vertical coordinate value, and spatial coordinate value of this pixel point S are respectively Figure 2c a in the horizontal covariance feature map marked by 27, b in the vertical covariance feature map marked by 28, and c in the cross-covariance feature map marked by 29. Then, the three-dimensional spatial coordinates corresponding to this pixel point S are (a, b, c).

[0082] S103: According to the three-dimensional spatial coordinates of each pixel point, determine the target cone to which each pixel point belongs, and obtain the cone gradient feature of the target cone to which each pixel point belongs.

[0083] In one implementation, after obtaining the three-dimensional spatial coordinates corresponding to each pixel point, through mathematical proof, the three-dimensional spatial coordinates corresponding to each pixel point in the target image are distributed within an infinitely large cone. For the convenience of subsequent calculations, the horizontal axis, vertical axis, and spatial axis of the above spatial coordinate system can be denoted as the x-axis, z-axis, and y-axis respectively. Correspondingly, the three-dimensional spatial coordinates of each pixel point will also change. For example, when the horizontal axis, vertical axis, and spatial axis of the spatial coordinate system of pixel point S are denoted as the x-axis, y-axis, and z-axis respectively, its three-dimensional spatial coordinates are (a, b, c), while when the horizontal axis, vertical axis, and spatial axis of the spatial coordinate system are denoted as the x-axis, z-axis, and y-axis respectively, its three-dimensional spatial coordinates are (A1, B1, C1), where A1 = a, B1 = c, and C1 = b. Then, an infinitely large cone in which the three-dimensional spatial coordinates corresponding to each pixel point in the target image are distributed can be as Figure 3a shown. Among them, the vertex coordinates of the cone are the origin (0, 0, 0), the ratio of the bottom radius of the cone to the height of the cone is 1, and the direction vector from the vertex to the center of the bottom of the cone coordinates is the same as the direction of the vector (1, 1, 0). Further, the cone can be simplified to Figure 3b the cone marked by 31 shown, where the three-dimensional spatial coordinates of each pixel point in the target image are all located within Figure 3b the cone marked by 31 shown. Optionally, for the three-dimensional spatial coordinates of each pixel point, there is a target cone to which it belongs. Among them, the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which it belongs. For example, the target cone of the three-dimensional spatial coordinates (A1, B1, C1) of pixel point S can be as Figure 3b the cone marked by 32 shown. Then, after determining the target cone, the computer device can obtain the cone radius value of the target cone to which each pixel point belongs respectively, and the second angle between the conical hypotenuse and the conical base in the target cone to which each pixel point belongs, and determine the cone radius value and the second angle corresponding to each pixel point as the cone gradient feature of each pixel point respectively.

[0084] It should be noted that in addition to determining the cone radius value and the second angle corresponding to each pixel point as the cone gradient feature of each pixel point respectively, the cone radius value corresponding to each pixel point can also be determined as the cone gradient feature of each pixel point, or the second angle corresponding to each pixel point can be determined as the cone gradient feature of each pixel point, or the angle between the conical hypotenuse and the conical height in the target cone to which each pixel point belongs can be determined as the cone gradient feature of each pixel point. That is, when determining the cone gradient feature of each pixel point, various characteristics of the pixel point in the corresponding target cone can be determined as the cone gradient feature of the pixel point.

[0085] S104: Determine the image gradient feature of the target image according to the conical gradient features to which each pixel point belongs respectively.

[0086] In one implementation, the computer device may determine the radius gradient feature of the target image according to the conical radius values respectively corresponding to each pixel point, and determine the included angle gradient feature of the target image according to the second included angles respectively corresponding to each pixel point. Then, the radius gradient feature and the included angle gradient feature are determined as the image gradient feature of the target image. For example, there are N pixel points in the target image, and the N pixel points correspond to N conical radius values and N second included angles. Then, the N conical radius values are the radius gradient feature, and the N second included angles are the included angle gradient feature.

[0087] In the embodiment of the present application, the computer device may obtain the horizontal gradient map and the vertical gradient map of the target image, generate the covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map. Then, based on the covariance feature map, map each pixel point in the target image to the spatial coordinate system to obtain the three-dimensional spatial coordinates respectively corresponding to each pixel point, and determine the target cone to which each pixel point belongs according to the three-dimensional spatial coordinates of each pixel point. Next, obtain the conical gradient feature of the target cone to which each pixel point belongs respectively. Further, the image gradient feature of the target image may be determined according to the conical gradient features to which each pixel point belongs respectively. By implementing the above method, the discreteness of the image gradient feature of the image can be better, and it can be more evenly distributed in the three-dimensional space, so as to improve the quality of the obtained image gradient feature, and make the image processing effect better when the image gradient feature is subsequently applied to various image processing applications.

[0088] Please refer to Figure 4 , Figure 4 which is a schematic flowchart of another image data processing method provided by the embodiment of the present application. This method is applied to a computer device and can be executed by the computer device. Among them, the computer device may be a server or a terminal. As Figure 4 shown, the image data processing method may include:

[0089] S401: Obtain the horizontal gradient map and the vertical gradient map of the target image, and generate the covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map.

[0090] In one implementation, the computer device can obtain the horizontal gradient map and the vertical gradient map of the target image, and generate an initial covariance feature map based on the obtained horizontal gradient map and vertical gradient map. The generation method of the initial covariance feature map can refer to the method of generating the covariance feature map in step S101 above, which will not be elaborated here. After determining the initial covariance feature map, the computer device can also perform a blurring operation on the initial covariance feature map to obtain the covariance feature map of the blurred image. Through blurring processing, the noise of the target image can be reduced. It should be noted that the initial covariance feature map here includes the initial horizontal covariance feature map (the same as the horizontal covariance feature map above), the initial vertical covariance feature map (the same as the vertical covariance feature map above), and the initial cross-covariance feature map (the same as the cross-covariance feature map above). Optionally, the blurring operation can be a mean blurring operation, a median blurring operation, a Gaussian blurring operation, or other blurring operations, which are not limited in this application. Specifically, the computer device can perform a convolution operation on the initial covariance feature map based on the image blurring convolution kernel to obtain the covariance feature map of the blurred image. That is, based on the image blurring convolution kernel, convolution operations are respectively performed on the initial horizontal covariance feature map, the initial vertical covariance feature map, and the initial cross-covariance feature map included in the initial covariance feature map to obtain the horizontal covariance feature map after blurring, the vertical covariance feature map after blurring, and the cross-covariance feature map after blurring. For example, taking the mean blurring operation as an example, the image blurring convolution kernel corresponding to the mean blurring operation can be a 3×3 mean blurring matrix, and each element value in the mean blurring matrix is 1 / 9, that is, [1 / 9, 1 / 9, 1 / 9; 1 / 9, 1 / 9, 1 / 9; 1 / 9, 1 / 9, 1 / 9]. The essence of performing a convolution operation on the initial horizontal covariance feature map using this image blurring convolution kernel is to sum an initial horizontal covariance feature value in the initial horizontal covariance feature map and the 8 surrounding initial horizontal covariance feature values, and then calculate the average value of the sum of these 9 initial horizontal covariance feature values, and update the central initial horizontal covariance feature value to the average value. As Figure 5a In the figure, what is marked by 51 is the initial horizontal covariance feature map. Taking the initial horizontal covariance feature value "7" marked by 501 as an example, for this initial horizontal covariance feature value, the mean blurring operation is to sum the data within the square marked by 501 and then take the average value to get (5 + 1 + 2 + 1 + 7 + 3 + 6 + 7 + 4) / 9 = 4. Then, what is marked by 52 is the horizontal covariance feature map obtained after blurring, and the initial horizontal covariance feature value "7" is updated to "4" marked by 503.

[0091] S402: Based on the covariance feature map, map each pixel point in the target image to the spatial coordinate system to obtain the corresponding three-dimensional spatial coordinates of each pixel point.

[0092] S403: Determine the initial cone to which each pixel belongs according to the three-dimensional spatial coordinates respectively corresponding to each pixel.

[0093] Among them, the initial cone can refer to the target cone described in step S103, which will not be elaborated here. For example, taking pixel S as an example, the initial cone to which this pixel S belongs can be the cone marked by 32 as shown in Figure 3b the figure.

[0094] S404: Determine the target coordinate axis from the horizontal coordinate axis, vertical coordinate axis, and spatial coordinate axis included in the spatial coordinate system.

[0095] In one implementation, the computer device can determine the target coordinate axis from the horizontal coordinate axis, vertical coordinate axis, and spatial coordinate axis included in the spatial coordinate system, and the target coordinate axis can be any one of the horizontal coordinate axis, vertical coordinate axis, and spatial coordinate axis. Optionally, the target coordinate axis can also be any vector in the space corresponding to the spatial coordinate system. For example, for the three-dimensional space shown in Figure 3b the figure, the target coordinate axis can be the x-axis, or the y-axis, or the z-axis, or the vector (1, 1, 0) can be used as the target coordinate axis.

[0096] S405: Rotate the initial cone to which each pixel belongs according to the target coordinate axis to obtain the target cone to which each pixel belongs.

[0097] In one implementation, the computer device can rotate the initial cone to which each pixel belongs according to the target coordinate axis to obtain the target cone to which each pixel belongs. Optionally, the cone axis of the target cone to which each pixel belongs can be made parallel to the target coordinate axis. Then, compared with the initial cone, it is more convenient and faster to determine the cone gradient feature to which each pixel belongs when using the target cone obtained after rotation. For example, if the target coordinate axis is the z-axis and the set rotation angle is 45 degrees, an infinite cone in which the three-dimensional spatial coordinates corresponding to each pixel in the target image are distributed can be rotated from the cone shown in Figure 3a the figure to the cone shown in Figure 5b the figure. Then, simplifying the cone shown in Figure 5b the figure, the cone marked by 51 as shown in Figure 5c the figure can be obtained. The cone marked by 52 is the target cone obtained after the rotation of pixel S. Among them, the cone axis of the target cone to which each pixel belongs coincides with the z-axis.

[0098] S406: Obtain the cone gradient feature of the target cone to which each pixel belongs.

[0099] Among them, the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which it belongs. For example, pixel point S is on the conical arc surface of the cone marked 52 as shown in 5c.

[0100] In one implementation, the computer device can respectively determine the pixel vector to which each pixel point belongs on the conical arc surface of the target cone to which each pixel point belongs, and obtain the vector length of the pixel vector to which each pixel point belongs. Optionally, the computer device can determine the spatial coordinate axis as the target coordinate axis. When the target coordinate axis is the spatial coordinate axis, the computer device can obtain the projection direction of the pixel vector to which each pixel point belongs in the target plane, and the first included angle between the pixel vector to which each pixel point belongs and the spatial coordinate axis. Among them, the target plane is determined by the horizontal coordinate axis and the vertical coordinate axis, that is, the target plane is a plane composed of the horizontal coordinate axis and the vertical coordinate axis. Then, after obtaining the vector length, projection direction and first included angle corresponding to each pixel point, the vector length, projection direction and first included angle corresponding to each pixel point can be respectively determined as the conical gradient feature to which each pixel point belongs.

[0101] It should be noted that the computer device can determine the spatial coordinate axis as the target coordinate axis, or can also determine the horizontal coordinate axis or the vertical coordinate axis, or other vectors in space as the target coordinate axis, which is not limited in this application.

[0102] For example, taking pixel point S as an example, among them, the target cone to which the pixel point S belongs can be Figure 5c the cone marked 52 as shown. And the target cone is obtained by rotating the cone marked 32 in Figure 3b with the target coordinate axis as the z-axis and the rotation angle of 45 degrees. It should be noted that the three-dimensional spatial coordinates of each pixel point will change after rotation. For example, if the three-dimensional spatial coordinates of pixel point S before rotation are (A1, B1, C1), then the three-dimensional spatial coordinates of pixel point S after rotation are (A2, B2, C2), where A2 = sin(180 / 4)(A1 - B1), B2 = sin(180 / 4)(A1 + B1), C2 = C1. Then, the pixel vector of pixel point SS after rotation is (A2, B2, C2), and the vector length of this pixel vector is The projection direction of the pixel vector in the target plane composed of the x-axis and the y-axis is α = atan2(A2, C2), and the first included angle between the pixel vector and the y-axis is

[0103] It should be noted that, in addition to determining the vector length, projection direction, and first included angle corresponding to each pixel as the conical gradient feature to which each pixel belongs, other values can also be used as the conical gradient feature to which each pixel belongs. For example, the conical radius value of the target cone to which each pixel belongs as described above, and the second included angle between the conical hypotenuse and the conical base in the target cone to which each pixel belongs can both be used as the conical gradient feature to which each pixel belongs, or it can be other values, which are not limited in this application. Moreover, the types of conical gradient features to which each pixel belongs can include one or more, which are not limited in this application. For example, the vector length, projection direction, and first included angle corresponding to each pixel can be respectively determined as the conical gradient feature to which each pixel belongs, or the vector length and projection direction corresponding to each pixel can be determined as the conical gradient feature to which each pixel belongs.

[0104] S407: Determine the image gradient feature of the target image according to the conical gradient feature to which each pixel belongs.

[0105] In one implementation, the computer device can determine the length gradient feature of the target image according to the vector length corresponding to each pixel respectively, determine the direction gradient feature of the target image according to the projection direction corresponding to each pixel respectively, and determine the included angle gradient feature of the target image according to the first included angle corresponding to each pixel respectively. Then, the length gradient feature, direction gradient feature, and included angle gradient feature are determined as the image gradient feature of the target image. For example, there are N pixels in the target image, and the N pixels correspond to N vector lengths, N projection directions, and N first included angles. Then the N vector lengths are the length gradient feature, the N projection directions are the direction gradient feature, and the N first included angles are the included angle gradient feature.

[0106] Among them, for the specific implementation manners of steps S401 - S403, reference can be made to the specific descriptions of steps S101 - S103 in the above embodiments, which will not be elaborated here.

[0107] In an embodiment of the present application, a computer device may obtain a horizontal gradient map and a vertical gradient map of a target image, generate a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map, and then, based on the covariance feature map, map each pixel point in the target image to a spatial coordinate system to obtain the corresponding three-dimensional spatial coordinates of each pixel point, and determine the initial cone to which each pixel point belongs according to the corresponding three-dimensional spatial coordinates of each pixel point. Then, a target coordinate axis is determined from the horizontal coordinate axis, the vertical coordinate axis, and the spatial coordinate axis included in the spatial coordinate system. To rotate the initial cone to which each pixel point belongs according to the target coordinate axis to obtain the target cone to which each pixel point belongs, and then obtain the cone gradient feature of the target cone to which each pixel point belongs. Further, according to the cone gradient feature to which each pixel point belongs, the image gradient feature of the target image is determined. By implementing the above method, the initial cone to which each pixel point in the image belongs can be arbitrarily rotated to obtain a target cone, and the target cone can be used to more conveniently and quickly obtain the image gradient feature of the image. Similarly, the dispersion of the image gradient feature of the image can be better, and it can be more evenly distributed in the three-dimensional space, so as to improve the quality of the obtained image gradient feature, so that when the image gradient feature is subsequently applied to various image processing applications, the image processing effect is better.

[0108] Please refer to Figure 6a , Figure 6a which is a schematic flowchart of another image data processing method provided by an embodiment of the present application. As can be seen from the Figure 6a flowchart, the image data processing method can be summarized into five processes, including: calculating horizontal and vertical gradients, calculating covariance features, blurring processing, feature three-dimensional rotation transformation, and feature cone transformation.

[0109] In one implementation, for the process of calculating horizontal and vertical gradients, the computer device may calculate the horizontal gradient value and the vertical gradient value corresponding to each pixel point in the target image, and thus obtain the horizontal gradient map and the vertical gradient map of the low-resolution image. Among them, the horizontal gradient map and the vertical gradient map can be obtained by performing a convolution operation on the target image with a gradient operator, and the gradient operator includes a horizontal direction gradient operator and a vertical direction gradient operator. Then, the horizontal gradient map can be obtained by performing a convolution operation on the target image with the horizontal direction gradient operator, and the vertical gradient map can be obtained by performing a convolution operation on the target image with the vertical direction gradient operator.

[0110] In one implementation, for the process of calculating co-variance features, the computer device can use the obtained horizontal gradient map and vertical gradient map to determine the co-variance feature map, which includes a horizontal co-variance feature map, a vertical co-variance feature map, and a cross co-variance feature map. Among them, each horizontal co-variance feature value in the horizontal co-variance feature map can be obtained according to the horizontal feature extraction region of each pixel point in the horizontal gradient map, each vertical co-variance feature value in the vertical co-variance feature map can be obtained according to the vertical feature extraction region of each pixel point in the vertical gradient map, and the cross co-variance feature map is obtained according to the horizontal feature extraction region and the vertical feature extraction region.

[0111] In one implementation, for the blurring process, after the computer device determines the horizontal co-variance feature map, the vertical co-variance feature map, and the cross co-variance feature map of the target image, it can perform blurring processing on the horizontal co-variance feature map, the vertical co-variance feature map, and the cross co-variance feature map respectively to obtain the blurred horizontal co-variance feature map, the vertical co-variance feature map, and the cross co-variance feature map. Optionally, the blurring processing can be mean blurring, median blurring, Gaussian blurring, etc.

[0112] In one implementation, for the feature three-dimensional rotation transformation process, based on the horizontal co-variance feature map, the vertical co-variance feature map, and the cross co-variance feature map, each pixel point in the target image can be mapped to a spatial coordinate system to obtain the corresponding three-dimensional coordinates of each pixel point, that is, the three-dimensional coordinates. For example, the horizontal coordinate value included in the three-dimensional coordinates of each pixel point is the horizontal co-variance feature value corresponding to the pixel point in the horizontal co-variance feature map. Similarly, the vertical coordinate value included in the three-dimensional coordinates of each pixel point is the vertical co-variance feature value corresponding to the pixel point in the vertical co-variance feature map, and the spatial coordinate value included in the three-dimensional coordinates of each pixel point is the cross co-variance feature value corresponding to the pixel point in the cross co-variance feature map. Through mathematical proof, the three-dimensional coordinates of each pixel point are distributed within an infinitely large cone, and according to the three-dimensional coordinates of each pixel point, the initial cone to which each pixel point belongs can be determined. For the convenience of subsequent determination of the image gradient features of the target image, the infinitely large cone can be rotated to obtain the rotated infinitely large cone. Then, for the initial cone corresponding to each pixel point, after rotation, the target cone corresponding to each pixel point can also be obtained, and after rotation, the three-dimensional coordinates of each pixel point will also change. For example, for the pixel point S described above, the three-dimensional coordinates before rotation are (A1, B1, C1), and the three-dimensional coordinates after rotation become (A2, B2, C2). The specific transformation process of its three-dimensional coordinates will not be elaborated here. Among them, Figure 6aFeature A therein represents the horizontal coordinate value in the three-dimensional space coordinates after rotation of each pixel point, Feature B represents the vertical coordinate value in the three-dimensional space coordinates after rotation of each pixel point, and Feature C represents the space coordinate value in the three-dimensional space coordinates after rotation of each pixel point.

[0113] In one implementation, for the feature conic transformation process, after determining the three-dimensional space coordinates of each pixel point after rotation, the gradient image feature of the target image can be determined according to the three-dimensional space coordinates. For example, for any pixel point, taking pixel point S as an example, the three-dimensional space coordinates of pixel point S are (A2, B2, C2), then the pixel vector to which the pixel point S belongs is the three-dimensional space coordinates (A2, B2, C2), and the vector length of the pixel vector, the projection direction of the pixel vector in the target plane, and the angle between the pixel vector and the target coordinate axis can be determined as the image gradient feature of the pixel point S. Then, the corresponding vector length, projection direction, and angle of each pixel point can be determined as the image gradient feature of the target image. Among them, Figure 6a the image gradient feature 1 therein can represent the vector length corresponding to each pixel point in the target image, the image gradient feature 2 can represent the projection direction corresponding to each pixel point in the target image, and the image gradient feature 3 can represent the angle corresponding to each pixel point in the target image.

[0114] In one implementation, the dispersion of the image gradient feature determined by the embodiments of the present application is relatively good. As Figure 6b shown, the image gradient feature can be approximately evenly distributed in a cube where x ∈ [0, 1], y ∈ [0, 1], and z ∈ [0, 1]. Then, taking super-resolution as an example, when training a model using a super-resolution image reconstruction dataset, such as common datasets like BSD100, it can ensure that each subspace has sufficient training data, and the method is simple and stable without introducing subsequent additional operations, so that the super-resolution filter can be effectively fitted, the overall super-resolution speed is faster, and the super-resolution effect is guaranteed. For example, in a super-resolution application scenario, an initial super-resolution model can be trained based on the image gradient feature of the above-mentioned target image, and the trained initial super-resolution model can be determined as the super-resolution model. Then, after determining the super-resolution model, a high-resolution image can be obtained using the super-resolution model. Specifically, the image to be adjusted can be input into the super-resolution model, and a resolution-adjusted image of the image to be adjusted can be obtained. Among them, the image resolution of the resolution-adjusted image is higher than that of the image to be adjusted. In addition, in practical applications, the image gradient feature determined by the embodiments of the present application can achieve better super-resolution image quality in super-resolution applications. If it is applied to a terminal and under the same bitstream, it can achieve better image quality compared with existing super-resolution algorithms. Or when used for server transcoding, it can reduce the loss of image quality while compressing the bitstream, or achieve a higher compression ratio.

[0115] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of an image data processing device provided by an embodiment of the present application. In this embodiment, the described image data processing device includes:

[0116] A generating unit 701, configured to obtain a horizontal gradient map and a vertical gradient map of a target image, and generate a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map;

[0117] A first determining unit 702, configured to map each pixel point in the target image to a spatial coordinate system based on the covariance feature map, so as to obtain the corresponding three-dimensional spatial coordinates of each pixel point;

[0118] An obtaining unit 703, configured to determine a target cone to which each pixel point belongs according to the three-dimensional spatial coordinates of each pixel point, and obtain a cone gradient feature of the target cone to which each pixel point belongs; the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which they belong;

[0119] A second determining unit 704, configured to determine an image gradient feature of the target image according to the cone gradient features to which each pixel point belongs.

[0120] In one implementation manner, the generating unit 701 is specifically configured to:

[0121] Generate a horizontal covariance feature map of the target image according to the horizontal gradient map;

[0122] Generate a vertical covariance feature map of the target image according to the vertical gradient map;

[0123] Generate a cross covariance feature map of the target image according to the horizontal gradient map and the vertical gradient map;

[0124] Determine the horizontal covariance feature map, the vertical covariance feature map, and the cross covariance feature map as the covariance feature map.

[0125] In one implementation manner, the horizontal gradient map includes the horizontal gradient values corresponding to each pixel point; the generating unit 701 is specifically configured to:

[0126] The step of generating a horizontal covariance feature map of the target image according to the horizontal gradient map includes:

[0127] Determine a horizontal feature extraction region corresponding to each pixel point in the horizontal gradient map;

[0128] Determine the horizontal covariance feature value corresponding to each pixel point according to the horizontal gradient value in the horizontal feature extraction region corresponding to each pixel point;

[0129] Generate the horizontal covariance feature map according to the horizontal covariance feature value corresponding to each pixel point.

[0130] In one implementation, the vertical gradient map includes the vertical gradient value corresponding to each pixel point; the generating unit 701 is specifically configured to:

[0131] The generating the vertical covariance feature map of the target image according to the vertical gradient map includes:

[0132] Determine the vertical feature extraction region corresponding to each pixel point in the vertical gradient map;

[0133] Determine the vertical covariance feature value corresponding to each pixel point according to the vertical gradient value in the vertical feature extraction region corresponding to each pixel point;

[0134] Generate the vertical covariance feature map according to the vertical covariance feature value corresponding to each pixel point.

[0135] In one implementation, the vertical gradient map includes the vertical gradient value corresponding to each pixel point; the first determining unit 702 is specifically configured to:

[0136] Determine the horizontal coordinate value corresponding to each pixel point according to the horizontal covariance feature map;

[0137] Determine the vertical coordinate value corresponding to each pixel point according to the vertical covariance feature map;

[0138] Determine the spatial coordinate value corresponding to each pixel point according to the cross-covariance feature map;

[0139] Determine the three-dimensional spatial coordinate corresponding to each pixel point according to the horizontal coordinate value corresponding to each pixel point, the vertical coordinate value corresponding to each pixel point, and the spatial coordinate value corresponding to each pixel point.

[0140] In one implementation, the spatial coordinate system includes a horizontal coordinate axis, a vertical coordinate axis, and a spatial coordinate axis; the obtaining unit 703 is specifically configured to:

[0141] Determine the initial cone to which each pixel point belongs according to the three-dimensional spatial coordinate corresponding to each pixel point;

[0142] Determine the target coordinate axis from the horizontal coordinate axis, the vertical coordinate axis, and the spatial coordinate axis;

[0143] Rotate the initial cone to which each pixel belongs according to the target coordinate axis to obtain the target cone to which each pixel belongs; the cone axis of the target cone to which each pixel belongs is parallel to the target coordinate axis.

[0144] In one implementation, the obtaining unit 703 is specifically configured to:

[0145] On the conical arc surface of the target cone to which each pixel belongs, respectively determine the pixel vector to which each pixel belongs;

[0146] Obtain the vector length of the pixel vector to which each pixel belongs;

[0147] When the target coordinate axis is the space coordinate axis, obtain the projection direction of the pixel vector to which each pixel belongs in the target plane, and the first included angle between the pixel vector to which each pixel belongs and the space coordinate axis; the target plane is determined by the horizontal coordinate axis and the vertical coordinate axis;

[0148] Respectively determine the vector length, projection direction, and first included angle corresponding to each pixel as the conical gradient feature to which each pixel belongs.

[0149] In one implementation, the second determining unit 704 is specifically configured to:

[0150] Determine the length gradient feature of the target image according to the vector length corresponding to each pixel respectively;

[0151] Determine the direction gradient feature of the target image according to the projection direction corresponding to each pixel respectively;

[0152] Determine the included angle gradient feature of the target image according to the first included angle corresponding to each pixel respectively;

[0153] Determine the length gradient feature, the direction gradient feature, and the included angle gradient feature as the image gradient feature of the target image.

[0154] It can be understood that the division of units in the embodiments of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation. Each functional unit in the embodiments of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0155] Please refer to Figure 8 , Figure 8It is a schematic structural diagram of a computer device provided by an embodiment of the present application. In this embodiment, the computer device described can be a server or a terminal. The computer device includes: a processor 801, a memory 802, and a network interface 803. Data can be exchanged between the above-mentioned processor 801, memory 802, and network interface 803.

[0156] The above-mentioned processor 801 can be a Central Processing Unit (CPU), and this processor can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or this processor can also be any conventional processor, etc.

[0157] The above-mentioned memory 802 can include a read-only memory and a random access memory, and provide program instructions and data to the processor 801. A part of the memory 802 can also include a non-volatile random access memory. Among them, when the processor 801 calls the program instructions, it is used to execute:

[0158] Obtain the horizontal gradient map and vertical gradient map of the target image, and generate the covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map;

[0159] Based on the covariance feature map, map each pixel point in the target image to a spatial coordinate system to obtain the corresponding three-dimensional spatial coordinates of each pixel point;

[0160] According to the three-dimensional spatial coordinates of each pixel point, determine the target cone to which each pixel point belongs, and obtain the cone gradient feature of the target cone to which each pixel point belongs; the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which they belong;

[0161] According to the cone gradient feature to which each pixel point belongs, determine the image gradient feature of the target image.

[0162] In one implementation, the processor 801 is specifically used for:

[0163] Generate the horizontal covariance feature map of the target image according to the horizontal gradient map;

[0164] Generate a vertical covariance feature map of the target image according to the vertical gradient map;

[0165] Generate a cross-covariance feature map of the target image according to the horizontal gradient map and the vertical gradient map;

[0166] Determine the horizontal covariance feature map, the vertical covariance feature map, and the cross-covariance feature map as the covariance feature map.

[0167] In one implementation, the horizontal gradient map includes horizontal gradient values respectively corresponding to each pixel point; the processor 801 is specifically configured to:

[0168] The generating of the horizontal covariance feature map of the target image according to the horizontal gradient map includes:

[0169] In the horizontal gradient map, determine horizontal feature extraction regions respectively corresponding to each pixel point;

[0170] According to the horizontal gradient values in the horizontal feature extraction regions respectively corresponding to each pixel point, determine horizontal covariance feature values respectively corresponding to each pixel point;

[0171] Generate the horizontal covariance feature map according to the horizontal covariance feature values respectively corresponding to each pixel point.

[0172] In one implementation, the vertical gradient map includes vertical gradient values respectively corresponding to each pixel point; the processor 801 is specifically configured to:

[0173] The generating of the vertical covariance feature map of the target image according to the vertical gradient map includes:

[0174] In the vertical gradient map, determine vertical feature extraction regions respectively corresponding to each pixel point;

[0175] According to the vertical gradient values in the vertical feature extraction regions respectively corresponding to each pixel point, determine vertical covariance feature values respectively corresponding to each pixel point;

[0176] Generate the vertical covariance feature map according to the vertical covariance feature values respectively corresponding to each pixel point.

[0177] In one implementation, the vertical gradient map includes vertical gradient values respectively corresponding to each pixel point; the processor 801 is specifically configured to:

[0178] Determine the horizontal coordinate value corresponding to each pixel point according to the horizontal covariance feature map;

[0179] Determine the vertical coordinate value corresponding to each pixel point according to the vertical covariance feature map;

[0180] Determine the spatial coordinate value corresponding to each pixel point according to the cross - covariance feature map;

[0181] Determine the three - dimensional spatial coordinates corresponding to each pixel point respectively according to the horizontal coordinate value, the vertical coordinate value and the spatial coordinate value corresponding to each pixel point.

[0182] In one implementation, the spatial coordinate system includes a horizontal coordinate axis, a vertical coordinate axis and a spatial coordinate axis; the processor 801 is specifically configured to:

[0183] Determine the initial cone to which each pixel point belongs respectively according to the three - dimensional spatial coordinates corresponding to each pixel point;

[0184] Determine a target coordinate axis from the horizontal coordinate axis, the vertical coordinate axis and the spatial coordinate axis;

[0185] Rotate the initial cone to which each pixel point belongs according to the target coordinate axis to obtain the target cone to which each pixel point belongs respectively; the cone axis of the target cone to which each pixel point belongs is parallel to the target coordinate axis.

[0186] In one implementation, the processor 801 is specifically configured to:

[0187] Determine the pixel vector to which each pixel point belongs respectively on the conical arc surface of the target cone to which each pixel point belongs;

[0188] Obtain the vector length of the pixel vector to which each pixel point belongs;

[0189] When the target coordinate axis is the spatial coordinate axis, obtain the projection direction of the pixel vector to which each pixel point belongs in the target plane, and the first included angle between the pixel vector to which each pixel point belongs and the spatial coordinate axis; the target plane is determined by the horizontal coordinate axis and the vertical coordinate axis;

[0190] Determine the vector length, projection direction and first included angle corresponding to each pixel point respectively as the conical gradient feature to which each pixel point belongs.

[0191] In one implementation, the processor 801 is specifically configured to:

[0192] Determine the length gradient feature of the target image according to the vector length corresponding to each pixel point respectively;

[0193] Determine the directional gradient feature of the target image according to the projection direction corresponding to each pixel point;

[0194] Determine the included angle gradient feature of the target image according to the first included angle corresponding to each pixel point;

[0195] Determine the length gradient feature, the directional gradient feature and the included angle gradient feature as the image gradient feature of the target image.

[0196] The embodiment of the present application also provides a computer storage medium, in which program instructions are stored, and when the program is executed, it may include some or all of the steps of the image data processing method in the corresponding embodiment as Figure 1b or Figure 4 corresponding to some or all of the steps of the image data processing method in the embodiment.

[0197] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0198] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and this program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disc, etc.

[0199] The embodiment of the present application also provides a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps performed in the above embodiments of the various methods.

[0200] The above has introduced in detail a method, apparatus, computer device and medium for image data processing provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation on the present application.

Claims

1. An image data processing method, characterized in that, Including: Obtaining a horizontal gradient map and a vertical gradient map of a target image, and generating a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map; the covariance feature map is determined based on a horizontal covariance feature map, a vertical covariance feature map, and a cross-covariance feature map, the horizontal covariance feature map is generated according to the horizontal gradient map, the vertical covariance feature map is generated according to the vertical gradient map, and the cross-covariance feature map is generated according to the horizontal gradient map and the vertical gradient map; Based on the covariance feature map, mapping each pixel point in the target image to a spatial coordinate system to obtain the corresponding three-dimensional spatial coordinates of each pixel point, including: determining the horizontal coordinate value corresponding to each pixel point according to the horizontal covariance feature map; determining the vertical coordinate value corresponding to each pixel point according to the vertical covariance feature map; determining the spatial coordinate value corresponding to each pixel point according to the cross-covariance feature map; determining the corresponding three-dimensional spatial coordinates of each pixel point according to the horizontal coordinate value, the vertical coordinate value, and the spatial coordinate value corresponding to each pixel point; According to the three-dimensional spatial coordinates of each pixel point, determining the target cone to which each pixel point belongs, and obtaining the cone gradient feature of the target cone to which each pixel point belongs; the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which they belong; According to the cone gradient feature to which each pixel point belongs, determining the image gradient feature of the target image.

2. The method according to claim 1, wherein The generating the covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map includes: Generating the horizontal covariance feature map of the target image according to the horizontal gradient map; Generating the vertical covariance feature map of the target image according to the vertical gradient map; Generating the cross-covariance feature map of the target image according to the horizontal gradient map and the vertical gradient map; Determining the horizontal covariance feature map, the vertical covariance feature map, and the cross-covariance feature map as the covariance feature map.

3. The method according to claim 2, wherein The horizontal gradient map includes the horizontal gradient value corresponding to each pixel point; The generating the horizontal covariance feature map of the target image according to the horizontal gradient map includes: In the horizontal gradient map, determining the horizontal feature extraction region corresponding to each pixel point; Determining the horizontal covariance feature value corresponding to each pixel point according to the horizontal gradient value in the horizontal feature extraction region corresponding to each pixel point; Generating the horizontal covariance feature map according to the horizontal covariance feature value corresponding to each pixel point.

4. The method according to claim 2, wherein The vertical gradient map includes the vertical gradient value corresponding to each pixel point; The generating the vertical covariance feature map of the target image according to the vertical gradient map includes: In the vertical gradient map, determining the vertical feature extraction region corresponding to each pixel point; Determine the vertical covariance feature value corresponding to each pixel point according to the vertical gradient value in the vertical feature extraction region corresponding to each pixel point. Generate the vertical covariance feature map according to the vertical covariance feature value corresponding to each pixel point.

5. The method according to claim 1, wherein The spatial coordinate system includes a horizontal coordinate axis, a vertical coordinate axis, and a spatial coordinate axis. The determining the target cone corresponding to each pixel point according to the three-dimensional spatial coordinate of each pixel point includes: Determine the initial cone to which each pixel point belongs according to the three-dimensional spatial coordinate of each pixel point. Determine the target coordinate axis from the horizontal coordinate axis, the vertical coordinate axis, and the spatial coordinate axis. Rotate the initial cone to which each pixel point belongs according to the target coordinate axis to obtain the target cone to which each pixel point belongs; the cone axis of the target cone to which each pixel point belongs is parallel to the target coordinate axis.

6. The method according to claim 5, characterized in that, The obtaining the cone gradient feature of the target cone to which each pixel point belongs includes: On the conical arc surface of the target cone to which each pixel point belongs, determine the pixel vector to which each pixel point belongs respectively. Obtain the vector length of the pixel vector to which each pixel point belongs. When the target coordinate axis is the spatial coordinate axis, obtain the projection direction of the pixel vector to which each pixel point belongs in the target plane, and the first included angle between the pixel vector to which each pixel point belongs and the spatial coordinate axis; the target plane is determined by the horizontal coordinate axis and the vertical coordinate axis. Determine the vector length, projection direction, and first included angle corresponding to each pixel point as the cone gradient feature of the target cone to which each pixel point belongs respectively.

7. The method according to claim 6, wherein The determining the image gradient feature of the target image according to the cone gradient feature of the target cone to which each pixel point belongs includes: Determine the length gradient feature of the target image according to the vector length corresponding to each pixel point. Determine the direction gradient feature of the target image according to the projection direction corresponding to each pixel point. Determine the included angle gradient feature of the target image according to the first included angle corresponding to each pixel point. Determine the length gradient feature, the direction gradient feature, and the included angle gradient feature as the image gradient feature of the target image.

8. An image data processing device, characterized in that, Includes: A generating unit, configured to obtain a horizontal gradient map and a vertical gradient map of a target image, and generate a covariance feature map of the target image based on the horizontal gradient map and the vertical gradient map; the covariance feature map is determined based on a horizontal covariance feature map, a vertical covariance feature map, and a cross-covariance feature map, the horizontal covariance feature map is generated according to the horizontal gradient map, the vertical covariance feature map is generated according to the vertical gradient map, and the cross-covariance feature map is generated according to the horizontal gradient map and the vertical gradient map. A first determination unit, configured to map each pixel point in the target image to a spatial coordinate system based on the covariance feature map of the covariance party, so as to obtain the corresponding three-dimensional spatial coordinates of each pixel point, including: determining the horizontal coordinate value corresponding to each pixel point according to the horizontal covariance feature map; determining the vertical coordinate value corresponding to each pixel point according to the vertical covariance feature map; determining the spatial coordinate value corresponding to each pixel point according to the cross-covariance feature map; determining the corresponding three-dimensional spatial coordinates of each pixel point according to the horizontal coordinate value corresponding to each pixel point, the vertical coordinate value corresponding to each pixel point, and the spatial coordinate value corresponding to each pixel point; An acquisition unit, configured to determine the target cone to which each pixel point belongs according to the three-dimensional spatial coordinates of each pixel point, and acquire the cone gradient feature of the target cone to which each pixel point belongs; the coordinate positions indicated by the three-dimensional spatial coordinates of each pixel point are respectively on the conical arc surface of the target cone to which they belong; A second determination unit, configured to determine the image gradient feature of the target image according to the cone gradient feature to which each pixel point belongs.

9. A computer device, characterized in that, It includes a processor, a memory, and a network interface, and the processor, the memory, and the network interface are interconnected. Among them, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1-7.

Images