Data processing method and device based on three-dimensional image, equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device based on a three-dimensional image, equipment and a storage medium. The method comprises the following steps: acquiring depth information and normal information of a target pixel point in a three-dimensional image; the depth information represents the shooting distance of the pixel points; the normal information represents the direction of a normal vector of the pixel point; determining initial edge information of the target pixel point according to the depth information and the normal information of the target pixel point and a preset first neighborhood range; according to initial edge information of all pixel points in a preset first neighborhood range, gradient information of a target pixel point is determined; the gradient information represents the change direction of the pixel value of the target pixel point; determining target edge information of the target pixel point according to the depth information, the normal information and the gradient information of the target pixel point; the target edge information represents whether the target pixel point is located at the edge of the three-dimensional image. The method is used for improving the accuracy of data processing of the three-dimensional image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a data processing method, apparatus, device and storage medium based on three-dimensional images. Background Technology

[0002] Image edge detection is a key step in image processing.

[0003] In related technologies, image edge detection techniques in 3D scenes determine edge information by analyzing single-dimensional data of the image, such as depth information or normal information, but this method has a high false detection rate. Therefore, there is an urgent need for an algorithm that can efficiently and accurately extract image edges in 3D scenes. Summary of the Invention

[0004] This application provides a data processing method, apparatus, device, and storage medium based on three-dimensional images to achieve the technical effect of improving the accuracy of three-dimensional image data processing.

[0005] In a first aspect, embodiments of this application provide a data processing method based on three-dimensional images, including:

[0006] The depth information and normal information of target pixels in a 3D image are obtained; the depth information represents the shooting distance of the pixel; the normal information represents the normal vector of the pixel.

[0007] Based on the depth information, normal information, and preset first neighborhood range of the target pixel, the initial edge information of the target pixel is determined; the initial edge information represents the preliminary determination result of whether the target pixel is located at the edge of the three-dimensional image; the edge of the three-dimensional image represents the boundary between different regions in the three-dimensional image; the preset first neighborhood range represents the region including the target pixel within a first preset range;

[0008] The gradient information of the target pixel is determined based on the initial edge information of all pixels in the preset first neighborhood range; the gradient information represents the direction of change of the pixel value of the target pixel.

[0009] The target edge information of the target pixel is determined based on the initial edge information, depth information, normal information, and gradient information of the target pixel; the target edge information indicates whether the target pixel is located at the edge of the three-dimensional image.

[0010] In one possible implementation, the initial edge information of the target pixel is determined based on its depth information, normal information, and a preset first neighborhood range, including:

[0011] Based on the depth information of the target pixel and a preset first neighborhood range, a first initial information is determined based on a preset first convolution operation; and based on the normal information of the target pixel and a preset first neighborhood range, a second initial information is determined based on a preset first convolution operation; the first initial information indicates whether the target pixel is located at the edge of the 3D image based on the depth information of the target pixel; the second initial information indicates whether the target pixel is located at the edge of the 3D image based on the normal information of the target pixel.

[0012] Based on the first initial information and the second initial information, the initial edge information of the target pixel is determined.

[0013] In one possible implementation, it also includes:

[0014] For each target pixel, if the initial edge information of the target pixel satisfies the first preset condition, then the initial edge information of each neighboring pixel of the target pixel is obtained.

[0015] If the initial edge information of the neighboring pixels does not meet the first preset condition, the initial edge information of the neighboring pixels is updated according to the depth information, normal information, and preset second neighborhood range of the neighboring pixels; the neighboring pixels are pixels adjacent to the target pixel; the preset second neighborhood range represents the region including the second preset range of the neighboring pixels; the second preset range is smaller than the first preset range.

[0016] In one possible implementation, the initial edge information of the neighboring pixels is updated based on the depth information, normal information, and a preset second neighborhood range, including:

[0017] Based on the depth information of the neighboring pixels and a preset second neighborhood range, a third initial information is determined based on a preset second convolution operation; and based on the normal information of the neighboring pixels and a preset second neighborhood range, a fourth initial information is determined based on a preset second convolution operation. The third initial information indicates whether the neighboring pixels are edges of the 3D image, determined based on the depth information of the neighboring pixels; the fourth initial information indicates whether the neighboring pixels are edges of the 3D image, determined based on the normal information of the neighboring pixels.

[0018] The initial edge information of the neighboring pixels is updated based on the third initial information and the fourth initial information.

[0019] In one possible implementation, determining the target edge information of the target pixel based on its initial edge information, depth information, normal information, and gradient information includes:

[0020] Based on the depth information of the target pixel and a preset first neighborhood range, a first initial information is determined based on a preset first convolution operation; the first initial information indicates whether the target pixel is located at the edge of the three-dimensional image based on the depth information of the target pixel.

[0021] The target edge information of the target pixel is determined based on the initial edge information, depth information, normal information, gradient information, and first initial information of the target pixel.

[0022] In one possible implementation, determining the target edge information of the target pixel based on its initial edge information, depth information, normal information, gradient information, and first initial information includes:

[0023] If the initial edge information satisfies the second preset condition, and the depth information, gradient information, and the first initial information satisfy the third preset condition, then the target edge information of the target pixel is determined to be the preset first identification information; the preset first identification information indicates that the target pixel is not located at the edge of the three-dimensional image.

[0024] In one possible implementation, determining the target edge information of the target pixel based on the initial edge information, depth information, normal information, gradient information, and first initial information of the target pixel further includes:

[0025] If the initial edge information satisfies the second preset condition, and the depth information, gradient information, and the first initial information do not satisfy the third preset condition, then the target edge information of the target pixel is determined according to the normal information.

[0026] In one possible implementation, the normal information includes a first sub-information, a second sub-information, and a third sub-information; the first sub-information represents the value of the normal information on a first preset coordinate axis; the second sub-information represents the value of the normal information on a second preset coordinate axis; and the third sub-information represents the value of the normal information on a third preset coordinate axis.

[0027] Based on the normal information, the target edge information of the target pixel is determined, including:

[0028] The target edge information of the target pixel is determined based on the first sub-information, the second sub-information, and the third sub-information of the normal information.

[0029] Secondly, embodiments of this application provide a data processing apparatus based on three-dimensional images, comprising:

[0030] The depth information and normal information of target pixels in a 3D image are obtained; the depth information represents the shooting distance of the pixel; the normal information represents the direction of the normal vector of the pixel.

[0031] Based on the depth information, normal information, and a preset first neighborhood range of the target pixel, the initial edge information of the target pixel is determined; the initial edge information represents the preliminary determination result of whether the target pixel is located at the edge of the three-dimensional image; the edge of the three-dimensional image represents the boundary between different regions in the three-dimensional image; the first neighborhood range represents the region including the first preset range of the target pixel;

[0032] The gradient information of the target pixel is determined based on the initial edge information of all pixels in the preset first neighborhood range; the gradient information represents the direction of change of the pixel value of the target pixel.

[0033] Based on the depth information, normal information, and gradient information of the target pixel, the target edge information of the target pixel is determined; the target edge information indicates whether the target pixel is located at the edge of the three-dimensional image.

[0034] Thirdly, embodiments of this application provide a data processing device based on a three-dimensional image, including: a memory and a processor;

[0035] The memory stores computer-executed instructions;

[0036] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0038] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0039] The data processing method, apparatus, device, storage medium, and program product based on three-dimensional images provided in this application acquire the depth information and normal information of target pixels in a three-dimensional image. By combining the pixel's shooting distance represented by the depth information with the pixel's normal vector represented by the normal information, the initial edge information of the target pixel is determined within a preset first neighborhood. This initial edge information represents a preliminary determination of whether the target pixel is located at the edge of the three-dimensional image. Because multi-dimensional data of the three-dimensional image is analyzed, compared to analyzing single-dimensional data in related technologies, multi-dimensional data analysis of three-dimensional images has higher accuracy, and the edges found in the three-dimensional image are more accurate. Furthermore, by comprehensively analyzing the initial edge information, depth information, normal information, and gradient information of the target pixel, the target edge information of the target pixel is obtained, removing one pixel from a two-pixel edge. This allows the edge of the three-dimensional image to be located on a more accurate single-pixel edge, further improving the accuracy of three-dimensional image data processing. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] Figure 1 This application provides a schematic diagram of the scenario.

[0042] Figure 2 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 1 ;

[0043] Figure 3 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 2 ;

[0044] Figure 4 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 3 ;

[0045] Figure 5 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 4 ;

[0046] Figure 6 A schematic diagram illustrating the initial information of the target pixel obtained by the convolution operation provided in this application;

[0047] Figure 7 A schematic diagram of the structure of a data processing device based on three-dimensional images provided in an embodiment of this application;

[0048] Figure 8This is a schematic diagram of the structure of a data processing device based on three-dimensional images provided in an embodiment of this application;

[0049] Figure 9 A schematic diagram of the structure of the data processing device based on three-dimensional images provided in this application;

[0050] Figure 10 A schematic diagram of the structure of the data processing device based on three-dimensional images provided in this application.

[0051] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0053] The application scenarios of this application include, but are not limited to, scenarios where mobile devices process data in real time. For example, under low computing power conditions, mobile phones, tablets and other terminal devices can perform fast edge detection on 2D images of 3D scene rendering for real-time feature extraction, image segmentation or motion detection.

[0054] This also includes, but is not limited to, scenarios involving 3D modeling and reconstruction, such as industrial inspection, medical imaging, or architectural scanning, where edge information is used to assist in the generation and optimization of 3D models.

[0055] This also includes, but is not limited to, game and film rendering scenarios, such as quickly identifying object outlines in dynamic scenes to improve rendering efficiency and reduce computing resource consumption.

[0056] This also includes, but is not limited to, robot visual navigation, such as using edge information to assist in environmental perception and achieve object recognition and obstacle avoidance.

[0057] In the above scenarios, the relevant technologies rely on grayscale information or single-dimensional data, such as using only depth information or only normal information. This results in problems such as high false detection rate, high false negative rate, and inability to distinguish between the mask and the masked edge, which limits the accuracy and practicality of the detection results.

[0058] For example, Figure 1 The scenario diagram provided for this application is as follows: Figure 1As shown, under some extreme shooting angles, 101 represents the camera, 102 represents the obstruction, 103 represents the obstructed object, and 104 represents a magnified pixel image of the edge between the obstruction 102 and the obstructed object 103 in the image captured by the camera. Each small square represents a pixel. When the plane being photographed is almost parallel to the camera's line of sight, detecting only the depth value of the pixels can lead to false positives due to drastic changes in the depth values ​​of the pixels in the captured image. When only the normal information of the pixels in the captured image is detected, if the normal vectors on both sides of the image edge have the same direction (e.g., the normal vectors of the planes above the obstruction 102 and the obstructed object 103 have the same direction), it is impossible to distinguish the pixels on both sides of the edge, resulting in missed detections. In the pixel magnification image 104, the small squares in the shaded area are the detected double-pixel edges. In some application scenarios, it is necessary to project the detected edges in reverse into three-dimensional space. At this time, the masking edge and the masked edge correspond to straight lines on two different planes in three-dimensional space. If this is not considered, processing straight lines on two different planes at the same time will cause a huge data processing error.

[0059] The data processing method, apparatus, device, and storage medium based on three-dimensional images provided in this application are intended to solve the aforementioned technical problems.

[0060] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0061] Figure 2 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 1 ,like Figure 2 As shown, the method includes:

[0062] S201. Obtain the depth information and normal information of the target pixel in the 3D image; the depth information represents the shooting distance of the pixel; the normal information represents the normal vector of the pixel.

[0063] This application can be applied to mobile electronic devices such as mobile phones and tablets, as well as other electronic devices or apparatuses capable of image processing, for detecting edges in three-dimensional images.

[0064] In a 3D graphics engine, a virtual camera renders a 3D scene into a 2D image based on its parameters. During this process, the image pipeline automatically generates depth and normal information for the image. During rendering, the engine creates a grayscale image that caches the distance from the object's surface to the camera for each pixel. The final 3D image includes this grayscale image; therefore, the depth information of target pixels in the 3D image can be obtained from the grayscale image. This depth information represents the shooting distance of the pixel, that is, the distance from the object's surface to the camera.

[0065] In a 3D model, each vertex has a predefined normal vector. During rendering, these normal vectors are interpolated and passed to each pixel, generating a normal map. The normal map is a three-channel (X, Y, and Z) map, where the X, Y, and Z channels store the components of the normal vector along the X, Y, and Z axes, respectively. The final 3D image includes this normal map; therefore, the normal information of target pixels in the 3D image can be obtained from the normal map. This normal information represents the normal vector of the pixel, i.e., the normal vector of the pixel in the normal map.

[0066] It should be noted that since the normal vectors of pixels may come from different interpolation processes, the initial lengths of the different normal vectors will vary. Therefore, after obtaining the normal vector information of the target pixel, a preset normalization process can be performed on the normal vector information to ensure that all normal vector information is compared under the same reference, thus guaranteeing the accuracy of subsequent data processing. The preset normalization process is used to map the data of each channel in the normal vector information to the range [-1, 1].

[0067] There are multiple target pixels, each of which is any pixel in the 3D image. For any given target pixel, it corresponds to one depth information and one normal information.

[0068] S202. Based on the depth information, normal information, and preset first neighborhood range of the target pixel, determine the initial edge information of the target pixel; the initial edge information represents the preliminary determination result of whether the target pixel is located at the edge of the three-dimensional image; the edge of the three-dimensional image represents the boundary between different regions in the three-dimensional image; the preset first neighborhood range represents the region including the first preset range of the target pixel.

[0069] For example, for each target pixel, the depth information of the target pixel is compared with the depth information of other pixels within a preset first neighborhood to obtain a first comparison value. For instance, the depth information of the target pixel is compared with the depth information of any other pixel within the preset first neighborhood (e.g., the difference is calculated) to obtain a first comparison result. The first comparison results corresponding to each of the other pixels are then combined to obtain the first comparison value. The first comparison value reflects the first change in the depth information of the target pixel compared to the depth information of all other pixels within the preset first neighborhood. The greater the first change, the greater the likelihood that the target pixel is located at the edge of the 3D image. The preset first neighborhood represents a region including the target pixel within a first preset range, such as a region centered on the target pixel. The area covered.

[0070] Simultaneously, the normal information of the target pixel is compared with the normal information of other pixels within a preset first neighborhood to obtain a second comparison value. For example, the normal information of the target pixel is compared with the normal information of any other pixel within the preset first neighborhood (e.g., the difference is calculated) to obtain a second comparison result. The second comparison results corresponding to each of the other pixels are then combined to obtain the second comparison value. The second comparison value reflects the second change in the normal information of the target pixel compared to the normal information of all other pixels within the preset first neighborhood. The greater the second change, the greater the probability that the target pixel is located at the edge of the 3D image.

[0071] Therefore, the initial edge information of the target pixel can be obtained by combining the first comparison value and the second comparison value corresponding to the target pixel. The initial edge information represents a preliminary determination of whether the target pixel is located at the edge of the 3D image, and the edge of the 3D image represents the boundary between different regions in the 3D image. For example, if the first comparison value is greater than a first preset threshold, or the second comparison value is greater than a second preset threshold, then the initial edge information of the target pixel is preliminarily confirmed to indicate that the target pixel is an edge of the 3D image; if the first comparison value is less than or equal to the first preset threshold, and the second comparison value is less than or equal to the second preset threshold, then the initial edge information of the target pixel is preliminarily confirmed to indicate that the target pixel is not an edge of the 3D image.

[0072] S203. Determine the gradient information of the target pixel based on the initial edge information of all pixels in the preset first neighborhood range; the gradient information represents the direction of change of the pixel value of the target pixel.

[0073] For example, the gradient information of a target pixel can be determined based on the initial edge information of all pixels within a preset first neighborhood range and a preset gradient extraction algorithm. The preset gradient extraction algorithm is used to extract the gradient information of the target pixel, and the gradient information represents the direction of change of the target pixel's pixel value, i.e., along which the pixel value of the target pixel changes the most.

[0074] For example, the preset gradient extraction algorithm uses the Sobel operator as shown in formulas (1)-(2) to extract the gradient information of the target pixel:

[0075] (1);

[0076] (2);

[0077] In the formula, This represents the lateral gradient operator, used to extract the gradient information of the target pixel in the lateral direction; This represents the vertical gradient operator, used to extract the gradient information of the target pixel in the vertical direction.

[0078] The horizontal gradient operator is convolved with the initial edge information of all pixels in a preset first neighborhood to obtain the horizontal gradient information; the vertical gradient operator is convolved with the initial edge information of all pixels in a preset first neighborhood to obtain the vertical gradient information. The horizontal gradient information represents the direction of change of the pixel value of the target pixel in the horizontal direction, and the vertical gradient information represents the direction of change of the pixel value of the target pixel in the vertical direction.

[0079] The gradient information of the target pixel is obtained by taking the arctangent of the ratio of the horizontal gradient information to the vertical gradient information, as shown in formula (3):

[0080] (3);

[0081] In the formula, This represents the gradient information of the target pixel. Represents lateral gradient information; It represents the gradient information in the vertical direction; arctan represents the arctangent function.

[0082] S204. Based on the initial edge information, depth information, normal information, and gradient information of the target pixel, determine the target edge information of the target pixel; the target edge information indicates whether the target pixel is located at the edge of the three-dimensional image.

[0083] For example, in a 3D image, the initial edge information of all target pixels collectively corresponds to the initially determined boundaries between different regions in the 3D image. For instance... Figure 1 In the pixel magnification of the edge portion in Figure 104, since the edges of both the occluding and occluded objects are edges of the 3D image, the edges of the 3D image correspond to two-pixel boundaries (small squares in the shaded area). To reduce errors in subsequent data processing, it is necessary to convert the two-pixel edges into single-pixel edges.

[0084] Therefore, based on the initial edge information of the target, the edges of the preliminarily determined 3D image can be located. For each target pixel in the preliminarily determined edges of the 3D image, the direction of change of the pixel value of the target pixel can be found based on the gradient information of that target pixel (e.g., Figure 1 The direction of the line connecting the two parallel shaded squares in the middle indicates the direction of change from edge pixels to non-edge pixels, and the pixel value changes the most along this direction.

[0085] Since depth information represents the shooting distance of a pixel, in the occluded and occluded edge parts, such as the edge of a two-pixel pixel, the pixel with the closer shooting distance is the occluded part, and the pixel with the farther shooting distance is the occluded part. Therefore, in the direction of change determined by the gradient information, the depth information of the target pixel and the neighboring pixels are compared. By removing the pixels with larger depth information and retaining the edge of the occluded part, the edge of a single pixel is obtained.

[0086] In some extreme cases, the depth information of the two pixels at a dual-pixel edge may not differ significantly, making it impossible to obtain a single-pixel edge simply by removing the pixel with the greater depth information. Therefore, we can further combine this with normal information to determine which pixel to remove from the dual-pixel edge. For example, since the camera screen coordinate system's axes are perpendicular to the screen plane during shooting, we can assume that the larger the value of the normal information on the camera screen coordinate axis, the closer the pixel is to the photographer, and the more information it provides. Therefore, we can determine which pixel to remove based on the values ​​of the normal information of the two pixels at the dual-pixel edge on the pixel screen coordinate axis, such as removing the pixel with the smaller value of the normal information on the camera screen coordinate axis. This yields the single-pixel edge.

[0087] The data processing method based on three-dimensional images provided in this application acquires the depth and normal information of target pixels in a three-dimensional image. By combining the pixel's shooting distance represented by the depth information with the pixel's normal vector represented by the normal information, the initial edge information of the target pixel is determined within a preset first neighborhood. This initial edge information represents a preliminary determination of whether the target pixel is located at the edge of the three-dimensional image. Because the method analyzes multi-dimensional data of the three-dimensional image, it has higher accuracy compared to analyzing single-dimensional data in related technologies, and the edges found in the three-dimensional image are more accurate. Furthermore, by comprehensively analyzing the initial edge information, depth information, normal information, and gradient information of the target pixel, the target edge information of the target pixel is obtained, removing one pixel from a two-pixel edge. This allows the edge of the three-dimensional image to be located on a more accurate single-pixel edge, further improving the accuracy of three-dimensional image data processing.

[0088] Figure 3 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 2 ,like Figure 3 As shown, the method includes:

[0089] S301. Obtain the depth and normal information of the target pixel in the 3D image.

[0090] S302. Based on the depth information of the target pixel and the preset first neighborhood range, determine the first initial information based on the preset first convolution operation; and based on the normal information of the target pixel and the preset first neighborhood range, determine the second initial information based on the preset first convolution operation.

[0091] For example, the depth information of the target pixel and the depth information of other pixels within a preset first neighborhood range are defined as follows: the preset first neighborhood range is centered on the target pixel. The neighborhood is subjected to a preset first convolution operation to obtain the first initial information.

[0092] Specifically, it can be calculated with the target pixel as the center. The Laplacian value of the depth information of each pixel in the neighborhood, where the Laplacian operator can be defined by the second derivative, as shown in formula (4):

[0093] (4);

[0094] In the formula, Indicates the target pixel as the center The Laplacian value of the depth information of each pixel in the neighborhood; This represents the second derivative in the x-axis direction; This represents the second derivative in the y-axis direction; This represents the depth information of the target pixel. This indicates the depth information of the pixel to the right of the target pixel; This indicates the depth information of the pixels to the left of the target pixel. This indicates the depth information of the pixels below the target pixel. This represents the depth information of the pixels above the target pixel. The Laplacian value of the calculated depth information is used as the first initial information.

[0095] The convolution kernel is shown in formula (5):

[0096] (5);

[0097] Simultaneously, the normal information of the target pixel and the normal information of other pixels within a preset first neighborhood range are also considered. The preset first neighborhood range is centered on the target pixel. The neighborhood is used to perform the first convolution operation by applying the convolution kernel of formula (5) to obtain the second initial information.

[0098] Specifically, based on the preset rules for calculating the normal vector angle, the normal vector angle between the target pixel and its right neighboring pixels is first calculated using the normal information of the target pixel and the normal information of its right neighboring pixels. This angle is the normal vector angle of the target pixel. Since the values ​​of the top-left, bottom-left, top-right, and bottom-right positions of the convolution kernel are all 0, only the normal vector angle between the neighboring pixels above the target pixel and its right neighboring pixels is calculated. The normal vector angle between the neighboring pixels below the target pixel and its right neighboring pixels is calculated. The normal vector angle between the neighboring pixels to the left of the target pixel and its right neighboring pixels is calculated. The normal vector angle between the neighboring pixels to the right of the target pixel and its right neighboring pixels is calculated. This yields the angle between the normal vectors of the target pixel, its neighboring pixels above, below, to the left, and to the right.

[0099] Then, using the convolution kernel as shown in Equation (5), the calculation is performed with the target pixel as the center. The Laplacian value of the angle between the normal vectors of each pixel in the neighborhood should be noted. When applying formula (4) to calculate the Laplacian value of the angle between the normal vectors, the depth information is replaced by the angle between the normal vectors. The Laplacian value of the angle between the normal vectors is used as the Laplacian value of the normal information. The calculated Laplacian value of the normal information is used as the second initial information.

[0100] S303. Determine the initial edge information of the target pixel based on the first initial information and the second initial information.

[0101] For example, if the first initial information is greater than a first preset threshold, or the second initial threshold is greater than a second initial threshold, then the initial edge information of the target pixel is determined to be the preset second identifier information. Otherwise, the initial edge information of the target pixel is determined to be the first identifier information. For example, if the preset second identifier information is 255, it indicates that the target pixel is located at the edge of the 3D image. If the preset first identifier information is 0, it indicates that the target pixel is not located at the edge of the 3D image.

[0102] The first preset threshold is the threshold corresponding to depth information, and the second preset threshold is the threshold corresponding to normal information, which can be set based on historical experience.

[0103] S304. Determine the gradient information of the target pixel based on the initial edge information of all pixels in the preset first neighborhood range.

[0104] S305. Determine the target edge information of the target pixel based on the initial edge information, depth information, normal information, and gradient information of the target pixel.

[0105] In this embodiment, based on a preset first convolution operation, first initial information is obtained according to the depth information of the target pixel and the depth information of other pixels within a preset first neighborhood. Whether the target pixel is located at the edge of the 3D image can be preliminarily determined based on whether the first initial information is greater than a first preset threshold. Simultaneously, second initial information is obtained according to the normal information of the target pixel and the normal information of other pixels within a preset first neighborhood. Whether the target pixel is located at the edge of the 3D image can also be preliminarily determined based on whether the second initial information is greater than the first preset threshold. Therefore, judging the initial edge information of the target pixel based on the first initial information and combined with the second initial information can effectively fuse multi-dimensional information such as depth information and normal information for comprehensive judgment, thereby improving the accuracy of data processing.

[0106] Figure 4 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 3 ,like Figure 4 As shown, the method includes:

[0107] S401. Obtain the depth and normal information of the target pixel in the 3D image.

[0108] S402. Determine the initial edge information of the target pixel based on its depth information, normal information, and a preset first neighborhood range.

[0109] S403. For each target pixel, if the initial edge information of the target pixel satisfies the first preset condition, then obtain the initial edge information of each neighboring pixel of the target pixel.

[0110] For example, for each target pixel, if the initial edge information of the target pixel satisfies the first preset condition, that is, the initial edge information of the target pixel is the preset second identification information 255, then the initial edge information of each neighboring pixel of the target pixel is obtained.

[0111] S404. If the initial edge information of the neighboring pixels does not meet the first preset condition, then the initial edge information of the neighboring pixels is updated according to the depth information, normal information, and preset second neighborhood range of the neighboring pixels. A neighboring pixel is a pixel adjacent to the target pixel; the preset second neighborhood range represents the region including the second preset range of the neighboring pixels; the second preset range is smaller than the first preset range.

[0112] If the initial edge information of a neighboring pixel does not meet the first preset condition, that is, the initial edge information of a neighboring pixel is not the preset second identifier information 255, that is, the neighboring pixel is not located at the edge of the three-dimensional image, then a neighbor detection is performed for that neighboring pixel.

[0113] For example, the initial edge information of neighboring pixels is updated based on their depth information, normal information, and a preset second neighborhood range. The preset second neighborhood range is smaller than a first preset range; for example, the preset first neighborhood range is... The neighborhood, the preset range of the second neighborhood is Neighborhood.

[0114] Specifically, for each neighboring pixel, taking that neighboring pixel as the top-left pixel, we obtain... Four neighboring pixels. Based on a preset gradient extraction algorithm, according to this... The depth information of each pixel in the four neighboring pixels is used to calculate the first gradient value. The first gradient value reflects the change in pixel value of the neighboring pixels, that is, whether the pixel value changes drastically from the neighboring pixel to any other pixel.

[0115] And based on a preset gradient extraction algorithm, according to this The normal information of each pixel in the four neighboring pixels is used to calculate the second gradient value. The first gradient value can also reflect the change in the pixel value of the neighboring pixels, that is, whether the pixel value changes drastically from the neighboring pixel to any other pixel.

[0116] Then, combining the first and second gradient values, the initial edge information of the neighboring pixels is updated. For example, if the first gradient value is greater than a third preset threshold, or the second gradient value is greater than a fourth preset threshold, the initial edge information of the neighboring pixels is updated to the preset second identifier information, that is, the neighboring pixels are located at the edge of the 3D image. The third preset threshold is the threshold corresponding to the depth information, and the fourth preset threshold is the threshold corresponding to the normal information, which can be set based on historical experience.

[0117] In some embodiments of this example, updating the initial edge information of neighboring pixels based on the depth information, normal information, and a preset second neighborhood range includes: determining third initial information based on the depth information and the preset second neighborhood range, using a preset second convolution operation; and determining fourth initial information based on the normal information and the preset second neighborhood range, using a preset second convolution operation. The third initial information indicates whether the neighboring pixels are edges of the 3D image based on the depth information; the fourth initial information indicates whether the neighboring pixels are edges of the 3D image based on the normal information; and updating the initial edge information of the neighboring pixels based on the third and fourth initial information.

[0118] For example, the preset second neighborhood range is The neighborhood is defined as follows: taking the pixel in the neighborhood as the top-left pixel within the second neighborhood range, and determining the fourth initial information based on the depth information of each pixel within the preset second neighborhood range and the preset second convolution operation. Specifically, the convolution kernel of the following formula (6) can be used, based on the Roberts operator, to calculate the third gradient value in the horizontal direction according to the depth information of each pixel within the preset second neighborhood range, and the convolution kernel of the following formula (7) can be used, based on the Roberts operator, to calculate the fourth gradient value in the vertical direction according to the depth information of each pixel within the preset second neighborhood range.

[0119] (6);

[0120] (7);

[0121] Among them, the convolution kernel of formula (6) is used to detect horizontal edges; the convolution kernel of formula (7) is used to detect vertical edges; the Roberts operator is used to calculate the first gradient value or the second gradient value; the third gradient value can reflect the change of the pixel value of the neighboring pixel in the horizontal direction; the fourth gradient value can reflect the change of the pixel value of the neighboring pixel in the vertical direction.

[0122] The third initial information is the sum of the absolute values ​​of the second and third gradient values.

[0123] For normal information, first calculate the angle between the normal vector of a neighboring pixel and the normal vector of the pixel to its right, and at the same time calculate the angle between the normal vector of each other pixel and the normal vector of the pixel to its right within the preset second neighborhood, which is the normal vector angle of that pixel.

[0124] Then, the convolution kernel of formula (6) above can be used to calculate the fourth gradient value in the horizontal direction based on the Roberts operator and the angle between the normal vectors of each pixel in the preset second neighborhood. The convolution kernel of formula (7) above can be used to calculate the fifth gradient value in the vertical direction based on the Roberts operator and the angle between the normal vectors of each pixel in the preset second neighborhood.

[0125] The fourth initial information is the sum of the absolute values ​​of the fourth gradient value and the fifth gradient value.

[0126] Then, combining the third and fourth initial information, the initial edge information of the neighboring pixels is updated. For example, if the third initial information is greater than the fourth preset threshold, or the fourth initial information is greater than the fifth preset threshold, the initial edge information of the neighboring pixels is updated to the preset second identifier information, that is, the neighboring pixels are located at the edge of the 3D image. Among them, the fifth preset threshold is the threshold corresponding to the depth information, and the sixth preset threshold is the threshold corresponding to the normal information, which can be set according to historical experience.

[0127] The beneficial effect of this setup is that, based on the depth information of neighboring pixels and the depth information of other pixels within a preset second neighborhood, third initial information reflecting the changes in pixel values ​​of neighboring pixels as reflected by the depth information can be calculated within a preset second neighborhood, which is smaller than the preset first neighborhood. Furthermore, based on the normal information of neighboring pixels and the normal information of other pixels within the preset second neighborhood, fourth initial information reflecting the changes in pixel values ​​of neighboring pixels as reflected by the normal information can be calculated within a preset second neighborhood, which is smaller than the preset first neighborhood. These third and fourth initial information provide more detailed information reflecting the changes in pixel values ​​of neighboring pixels. Updating the initial edge information of neighboring pixels based on this information yields more accurate initial edge information.

[0128] S405. Determine the gradient information of the target pixel based on the initial edge information of all pixels in the preset first neighborhood range.

[0129] S406. Determine the target edge information of the target pixel based on the initial edge information, depth information, normal information, and gradient information of the target pixel.

[0130] In this embodiment, for each target pixel located at the edge of a 3D image, if, based on the initial edge information, any neighboring pixel of the target pixel is not located at the edge of the 3D image, supplementary detection is performed based on the depth information, normal information, and a preset second neighborhood range of the neighboring pixels to update the initial edge information of the neighboring pixels. Since the preset second neighborhood range is smaller than the preset first neighborhood range, the result of the supplementary detection is more accurate than the detection result corresponding to the preset first neighborhood range, and more detailed edge information can be detected, thereby preventing edge pixels from being missed. Simultaneously, only the neighboring pixels of the target pixel located at the edge of the 3D image are detected, rather than all pixels in the 3D image, improving detection efficiency.

[0131] Figure 5 A flowchart illustrating the data processing method based on three-dimensional images provided in the embodiments of this application. Figure 4 ,like Figure 5 As shown, the method includes:

[0132] S501. Obtain the depth and normal information of the target pixel in the 3D image.

[0133] S502. Determine the initial edge information of the target pixel based on its depth information, normal information, and a preset first neighborhood range.

[0134] S503. Determine the gradient information of the target pixel based on the initial edge information of all pixels in the preset first neighborhood range.

[0135] It should be noted that, under certain special circumstances, when the initial edge information of pixels is symmetrically distributed within a preset first neighborhood (e.g., the preset first neighborhood is...), Within the preset first neighborhood, the initial edge information of all pixels is [0,255,0,0,255,0,0,255,0], and the horizontal and vertical gradient information calculated using symmetric operators (such as formulas (1) and (2)) are both 0. Therefore, the gradient information of the target pixel cannot be obtained. Then, the following method is used... The asymmetric operator of the neighborhood (such as any one of [0,-1,1,0], [-1,0,0,1], [0,0,-1,1], [0,-1,0,1]) and the first neighborhood range ( The initial edge information of all pixels in the neighborhood is convolved to calculate the gradient value of the target pixel.

[0136] S504. Based on the depth information of the target pixel and a preset first neighborhood range, and using a preset first convolution operation, determine the first initial information. The first initial information indicates whether the target pixel is located at the edge of the three-dimensional image based on the depth information of the target pixel.

[0137] The execution process for determining the first initial information in S504 is the same as the execution process for determining the first initial information in S302, and will not be repeated here.

[0138] S505. Determine the target edge information of the target pixel based on the initial edge information, depth information, normal information, gradient information, and first initial information of the target pixel.

[0139] For example, if the initial edge information satisfies the second preset condition, and the depth information, gradient information, and first initial information satisfy the third preset condition, then the target edge information of the target pixel is determined to be the preset first identification information; the preset first identification information indicates that the target pixel is not located at the edge of the three-dimensional image.

[0140] Figure 6 This is a schematic diagram illustrating how convolution operations obtain the initial information of the target pixel, as shown below. Figure 6 As shown, 601 is the masking part, 602 is the masked part, the red box on the left is the first initial information for calculating the target pixel points of the masking edge, and the red box on the right is the first initial information for calculating the target pixel points of the masked edge.

[0141] The initial information of the target pixel is used as the gradient magnitude and the gradient information is used as the gradient direction. By comparing the sign of the magnitude of the target pixel with that of the neighboring pixels in the gradient direction and the Laplacian value of the depth information, the masked edge pixels are eliminated.

[0142] If the initial edge information of the target pixel satisfies the second preset condition, that is, the initial edge information of the target pixel is the preset second identification information, meaning the target pixel is located at the edge of the 3D image, the first consideration is whether the target pixel is a masked pixel. Figure 6As shown, the edge of the orange area is the masking edge, and the edge of the blue area is the masked edge. In the camera screen coordinate system, the depth information of the orange area is less than that of the blue area. According to the definition of formula (5), the Laplacian value of the depth information of the masked edge (i.e., the first initial information) is less than 0, and the Laplacian value of the depth information of the masking edge is greater than 0. Therefore, if the initial edge information of the target pixel is the preset second identification information 255, that is, the target pixel is located at the edge of the three-dimensional image, and the first initial information of the target pixel is less than 0, and the first initial information of the neighboring pixels along the direction of the gradient information from the target pixel is greater than 0, and the difference between the depth information of the target pixel and the depth information of the neighboring pixels is greater than the preset difference threshold, the preset difference threshold is set according to historical experience, that is, the second preset condition is met, then the target pixel is considered to be a masked pixel. Therefore, the target edge information of the target pixel is the preset second identification information 0. That is, the target pixel is removed.

[0143] If the initial edge information satisfies the second preset condition, and the depth information, gradient information, and first initial information do not satisfy the third preset condition, then the target edge information of the target pixel is determined based on the normal information.

[0144] If the initial edge information of the target pixel satisfies the second preset condition, and the depth information, gradient information, and first initial information do not satisfy the third preset condition (i.e., the target pixel is not a masked pixel located at the edge of the 3D image), then the normal information is used to determine whether to remove the target pixel.

[0145] Specifically, the normal information includes a first sub-information, a second sub-information, and a third sub-information; the first sub-information represents the value of the normal information on the first preset coordinate axis; the second sub-information represents the value of the normal information on the second preset coordinate axis; and the third sub-information represents the value of the normal information on the third preset coordinate axis. The above-mentioned determination of the target edge information of the target pixel based on the normal information includes: determining the target edge information of the target pixel based on the first sub-information, the second sub-information, and the third sub-information of the normal information.

[0146] For example, the first sub-information is the value of the normal information on the Z-axis; the second sub-information is the value of the normal information on the Y-axis; and the third sub-information is the value of the normal information on the X-axis. Since the axes of the screen coordinate system are perpendicular to the screen plane, it can be assumed that the larger the first sub-information, the more directly the target pixel is facing the observer, the more information it contains, and the more important it is.

[0147] When the initial edge information, depth information, gradient information, and first initial information do not meet the second preset condition, the first sub-information of the target pixel is compared with the first sub-information of the neighboring pixels in the gradient direction.

[0148] If the first sub-information of the target pixel is not equal to the first sub-information of the neighboring pixels in the gradient direction, when the first sub-information of the target pixel is greater than the first sub-information of the neighboring pixels in the gradient direction, the target edge information of the target pixel is the preset second identifier information, and the target edge information of the neighboring pixels in the gradient direction is the preset first identifier information. The preset second identifier information indicates that the target pixel is located at the edge of the 3D image, that is, the neighboring pixels are removed; when the first sub-information of the target pixel is less than the first sub-information of the neighboring pixels in the gradient direction, the target edge information of the target pixel is the preset first identifier information, and the target edge information of the neighboring pixels in the gradient direction is the preset second identifier information, that is, the target pixel is removed.

[0149] If the first sub-information of the target pixel is equal to the first sub-information of the neighboring pixels along the gradient direction, then compare the second sub-information of the target pixel with the second sub-information of the neighboring pixels along the gradient direction.

[0150] If the second sub-information of the target pixel is not equal to the second sub-information of the neighboring pixels in the gradient direction, when the second sub-information of the target pixel is greater than the second sub-information of the neighboring pixels in the gradient direction, the target edge information of the target pixel is the preset second identifier information, and the target edge information of the neighboring pixels in the gradient direction is the preset first identifier information, that is, the neighboring pixels are removed; when the second sub-information of the target pixel is less than the second sub-information of the neighboring pixels in the gradient direction, the target edge information of the target pixel is the preset first identifier information, and the target edge information of the neighboring pixels in the gradient direction is the preset second identifier information, that is, the target pixel is removed.

[0151] If the second sub-information of the target pixel is equal to the second sub-information of the neighboring pixels in the gradient direction, compare the third sub-information of the target pixel with the third sub-information of the neighboring pixels in the gradient direction.

[0152] If the third sub-information of the target pixel is not equal to the third sub-information of the neighboring pixels in the gradient direction, when the third sub-information of the target pixel is greater than the third sub-information of the neighboring pixels in the gradient direction, the target edge information of the target pixel is the preset second identifier information, and the target edge information of the neighboring pixels in the gradient direction is the preset first identifier information, that is, the neighboring pixels are removed; when the third sub-information of the target pixel is less than the third sub-information of the neighboring pixels in the gradient direction, the target edge information of the target pixel is the preset first identifier information, and the target edge information of the neighboring pixels in the gradient direction is the preset second identifier information, that is, the target pixel is removed.

[0153] If the third sub-information of the target pixel is equal to the third sub-information of the neighboring pixels in the gradient direction, the target edge information of the target pixel is the preset second identifier information, and the target edge information of the neighboring pixels in the gradient direction is the preset first identifier information. That is, the neighboring pixels are removed by default.

[0154] If the initial edge information of the target pixel does not meet the second preset condition, then there is no need to remove the target pixel or neighboring pixels in the gradient direction.

[0155] Figure 7 A schematic diagram of the two-pixel edge of the three-dimensional image provided in this application, as shown below. Figure 6 As shown, the edges of the car in the 3D image were extracted. These edges are the edges of pixels without masked edges being removed. Small red circles enclose part of the edges, and the content within the larger red circles is a magnified display of the edge portion within the smaller red circles. From Figure 7 As can be seen, the extracted edges are relatively thick lines, that is, two-pixel edges.

[0156] Figure 8 A schematic diagram of a single-pixel edge of the three-dimensional image provided in this application, as shown below. Figure 7 As shown, the edges of the car in the 3D image were extracted. These edges are the edges from which pixels that were masked have been removed. Small red circles enclose part of the edges, and the content within the larger red circles is a magnified display of the edge portion within the smaller red circles. From Figure 8 As can be seen, the extracted edges are relatively thin lines, that is, single-pixel edges.

[0157] Figure 9 This is a schematic diagram of the structure of the data processing device based on three-dimensional images provided in this application, as shown below. Figure 9 As shown, the data processing device 90 based on three-dimensional images provided in this embodiment includes:

[0158] The acquisition module 901 is used to acquire the depth information and normal information of target pixels in a 3D image; the depth information represents the shooting distance of the pixel; the normal information represents the direction of the normal vector of the pixel.

[0159] The first determining module 902 is used to determine the initial edge information of the target pixel based on the depth information, normal information, and a preset first neighborhood range of the target pixel; the initial edge information represents the preliminary determination result of whether the target pixel is located at the edge of the three-dimensional image; the edge of the three-dimensional image represents the boundary between different regions in the three-dimensional image; the first neighborhood range represents the region including the first preset range of the target pixel;

[0160] The second determining module 903 is used to determine the gradient information of the target pixel based on the initial edge information of all pixels in the preset first neighborhood range; the gradient information represents the direction of change of the pixel value of the target pixel.

[0161] The third determining module 904 is used to determine the target edge information of the target pixel based on the depth information, normal information, and gradient information of the target pixel; the target edge information indicates whether the target pixel is located at the edge of the three-dimensional image.

[0162] In one possible implementation, the first determining module 902 is further configured to:

[0163] Based on the depth information of the target pixel and a preset first neighborhood range, a first initial information is determined based on a preset first convolution operation. Based on the normal information of the target pixel and a preset first neighborhood range, a second initial information is determined based on a preset first convolution operation. The first initial information indicates whether the target pixel is located at the edge of the 3D image based on its depth information. The second initial information indicates whether the target pixel is located at the edge of the 3D image based on its normal information.

[0164] Based on the first initial information and the second initial information, the initial edge information of the target pixel is determined.

[0165] In one possible implementation, the data processing device 70 based on the three-dimensional image further includes a supplementary detection module for:

[0166] For each target pixel, if the initial edge information of the target pixel satisfies the first preset condition, then the initial edge information of each neighboring pixel of the target pixel is obtained.

[0167] If the initial edge information of the neighboring pixels does not meet the first preset condition, the initial edge information of the neighboring pixels is updated according to the depth information, normal information, and preset second neighborhood range of the neighboring pixels. The neighboring pixels are the pixels adjacent to the target pixel. The preset second neighborhood range represents the region including the second preset range of the neighboring pixels. The second preset range is smaller than the first preset range.

[0168] In one possible implementation, the supplementary detection module is also used for:

[0169] Based on the depth information of neighboring pixels and a preset second neighborhood range, a third initial information is determined based on a preset second convolution operation. Similarly, based on the normal information of neighboring pixels and a preset second neighborhood range, a fourth initial information is determined based on a preset second convolution operation. The third initial information represents whether a neighboring pixel is an edge of the 3D image, determined based on its depth information. The fourth initial information represents whether a neighboring pixel is an edge of the 3D image, determined based on its normal information.

[0170] Update the initial edge information of neighboring pixels based on the third and fourth initial information.

[0171] In one possible implementation, the third determining module 904 is further configured to:

[0172] Based on the depth information of the target pixel and a preset first neighborhood range, and based on a preset first convolution operation, first initial information is determined; the first initial information represents whether the target pixel is located at the edge of the three-dimensional image based on the depth information of the target pixel.

[0173] The target edge information of the target pixel is determined based on the initial edge information, depth information, normal information, gradient information, and first initial information of the target pixel.

[0174] In one possible implementation, the third determining module 904 is further configured to:

[0175] If the initial edge information satisfies the second preset condition, and the depth information, gradient information, and first initial information satisfy the third preset condition, then the target edge information of the target pixel is determined to be the preset first identifier information; the preset first identifier information indicates that the target pixel is not located at the edge of the three-dimensional image.

[0176] In one possible implementation, the third determining module 904 is further configured to:

[0177] If the initial edge information satisfies the second preset condition, and the depth information, gradient information, and first initial information do not satisfy the third preset condition, then the target edge information of the target pixel is determined based on the normal information.

[0178] In one possible implementation, the normal information includes first sub-information, second sub-information, and third sub-information; the first sub-information represents the value of the normal information on a first preset coordinate axis; the second sub-information represents the value of the normal information on a second preset coordinate axis; the third sub-information represents the value of the normal information on a third preset coordinate axis; the third determining module 904 is further configured to:

[0179] The target edge information of the target pixel is determined based on the first, second, and third sub-information of the normal information.

[0180] The data processing device based on three-dimensional images provided in this embodiment can execute the methods provided in the above-described method embodiments. Its implementation principle and technical effects are similar, and will not be described in detail here.

[0181] Figure 10 This is a schematic diagram of the structure of the data processing device based on three-dimensional images provided in this application. Figure 10 As shown, the data processing device 100 based on three-dimensional images provided in this embodiment includes at least one processor 1001 and a memory 1002. Optionally, the data processing device 100 based on three-dimensional images further includes a communication component 1003. The processor 1001, memory 1002, and communication component 1003 are connected via a bus.

[0182] In a specific implementation, at least one processor 1001 executes computer execution instructions stored in memory 1002, causing at least one processor 1001 to perform the above-described method.

[0183] The specific implementation process of processor 1001 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0184] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0185] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0186] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0187] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0188] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0189] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0190] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0191] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0192] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0194] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0195] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0196] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A data processing method based on three-dimensional images, characterized in that, include: Obtain the depth and normal information of target pixels in a 3D image; The depth information represents the shooting distance of the pixel; The normal information represents the normal vector of a pixel; The initial edge information of the target pixel is determined based on the depth information, normal information, and a preset first neighborhood range of the target pixel. The initial edge information represents the preliminary determination result of whether the target pixel is located at the edge of the 3D image; the edge of the 3D image represents the boundary between different regions in the 3D image; The preset first neighborhood range represents the region including the first preset range of the target pixel; The gradient information of the target pixel is determined based on the initial edge information of all pixels in the preset first neighborhood range; The gradient information represents the direction of change in the pixel value of the target pixel; The target edge information of the target pixel is determined based on the initial edge information, depth information, normal information, and gradient information of the target pixel; the target edge information indicates whether the target pixel is located at the edge of the three-dimensional image.

2. The method according to claim 1, characterized in that, Based on the depth information, normal information, and a preset first neighborhood range of the target pixel, the initial edge information of the target pixel is determined, including: Based on the depth information of the target pixel and a preset first neighborhood range, a first initial information is determined based on a preset first convolution operation; and based on the normal information of the target pixel and a preset first neighborhood range, a second initial information is determined based on a preset first convolution operation; the first initial information indicates whether the target pixel is located at the edge of the 3D image based on the depth information of the target pixel; the second initial information indicates whether the target pixel is located at the edge of the 3D image based on the normal information of the target pixel. Based on the first initial information and the second initial information, the initial edge information of the target pixel is determined.

3. The method according to claim 1, characterized in that, Also includes: For each target pixel, if the initial edge information of the target pixel satisfies the first preset condition, then the initial edge information of each neighboring pixel of the target pixel is obtained. If the initial edge information of the neighboring pixels does not meet the first preset condition, the initial edge information of the neighboring pixels is updated according to the depth information, normal information, and preset second neighborhood range of the neighboring pixels; the neighboring pixels are pixels adjacent to the target pixel; the preset second neighborhood range represents the region including the second preset range of the neighboring pixels; the second preset range is smaller than the first preset range.

4. The method according to claim 3, characterized in that, Based on the depth information, normal information, and preset second neighborhood range of the neighboring pixels, the initial edge information of the neighboring pixels is updated, including: Based on the depth information of the neighboring pixels and a preset second neighborhood range, a third initial information is determined based on a preset second convolution operation; and based on the normal information of the neighboring pixels and a preset second neighborhood range, a fourth initial information is determined based on a preset second convolution operation. The third initial information indicates whether the neighboring pixels are edges of the 3D image, determined based on the depth information of the neighboring pixels; the fourth initial information indicates whether the neighboring pixels are edges of the 3D image, determined based on the normal information of the neighboring pixels. The initial edge information of the neighboring pixels is updated based on the third initial information and the fourth initial information.

5. The method according to claim 1, characterized in that, Based on the initial edge information, depth information, normal information, and gradient information of the target pixel, the target edge information of the target pixel is determined, including: Based on the depth information of the target pixel and a preset first neighborhood range, a first initial information is determined based on a preset first convolution operation; the first initial information indicates whether the target pixel is located at the edge of the three-dimensional image based on the depth information of the target pixel. The target edge information of the target pixel is determined based on the initial edge information, depth information, normal information, gradient information, and first initial information of the target pixel.

6. The method according to claim 5, characterized in that, Based on the initial edge information, depth information, normal information, gradient information, and first initial information of the target pixel, the target edge information of the target pixel is determined, including: If the initial edge information satisfies the second preset condition, and the depth information, gradient information, and the first initial information satisfy the third preset condition, then the target edge information of the target pixel is determined to be the preset first identification information; the preset first identification information indicates that the target pixel is not located at the edge of the three-dimensional image.

7. The method according to claim 5, characterized in that, Determining the target edge information of the target pixel based on its initial edge information, depth information, normal information, gradient information, and first initial information further includes: If the initial edge information satisfies the second preset condition, and the depth information, gradient information, and the first initial information do not satisfy the third preset condition, then the target edge information of the target pixel is determined according to the normal information.

8. The method according to claim 7, characterized in that, The normal information includes a first sub-information, a second sub-information, and a third sub-information; the first sub-information represents the value of the normal information on a first preset coordinate axis; the second sub-information represents the value of the normal information on a second preset coordinate axis; The third sub-information represents the value of the normal information on the third preset coordinate axis; Based on the normal information, the target edge information of the target pixel is determined, including: The target edge information of the target pixel is determined based on the first sub-information, the second sub-information, and the third sub-information of the normal information.

9. A data processing device based on three-dimensional images, characterized in that, include: The acquisition module is used to acquire the depth and normal information of target pixels in a 3D image; The depth information represents the shooting distance of the pixel; The normal information represents the direction of the normal vector of a pixel; The first determining module is used to determine the initial edge information of the target pixel based on the depth information, normal information, and a preset first neighborhood range of the target pixel. The initial edge information represents the preliminary determination result of whether the target pixel is located at the edge of the 3D image; the edge of the 3D image represents the boundary between different regions in the 3D image; The first neighborhood range represents the region that includes a first preset range of the target pixel; The second determining module is used to determine the gradient information of the target pixel based on the initial edge information of all pixels in the preset first neighborhood range; The gradient information represents the direction of change in the pixel value of the target pixel; The third determining module is used to determine the target edge information of the target pixel based on the depth information, normal information, and gradient information of the target pixel. The target edge information indicates whether the target pixel is located at the edge of the three-dimensional image.

10. A data processing device based on three-dimensional images, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

Images