Three-dimensional processing method of two-dimensional image, computer equipment and system

Abstract

The invention is suitable for the technical field of medical equipment, and provides a three-dimensional method of a two-dimensional image, computer equipment and a system. The three-dimensional processing method of the two-dimensional image comprises the following steps: determining a depth map of the two-dimensional image through a depth estimation model; generating a left and right eye disparity map according to the depth map; generating a left eye coordinate mapping graph and a right eye coordinate mapping graph according to the left eye disparity map and the right eye disparity map; obtaining a left eye view according to the two-dimensional image and the left eye coordinate mapping graph, and obtaining a right eye view according to the two-dimensional image and the right eye coordinate mapping graph; and obtaining a three-dimensional image corresponding to the two-dimensional image according to the left-eye view and the right-eye view, thereby realizing three-dimensional imaging, effectively reducing the size of equipment, reducing the cost of the equipment, and reducing the complexity of optical calibration.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical equipment, and particularly relates to a three-dimensional method of a two-dimensional image, a computer device and a system. BACKGROUND

[0002] With the continuous development of medical technology, minimally invasive surgery has become a common surgical method in clinical medicine due to its small trauma, light pain and fast postoperative recovery. When performing minimally invasive surgery, an endoscope system including an endoscope and a surgical instrument is usually placed in the patient's body through a small incision, so that the doctor can know the internal tissue condition of the human body in time through the image collected by the endoscope. However, the traditional endoscope system mainly provides two-dimensional images, and the doctor cannot accurately know the hierarchical relationship of the internal tissue of the human body and the accurate distance between the tip of the surgical instrument and the target tissue from the two-dimensional images. The lack of depth perception not only increases the difficulty of fine operations (such as suturing or stripping), but also increases the risk of surgery.

[0003] In order to solve the above problems, the prior art provides a three-dimensional endoscope system, which uses a binocular endoscope to collect a three-dimensional image of the internal tissue of the human body, so that the doctor can know the hierarchical relationship of the internal tissue of the human body and the accurate distance between the tip of the surgical instrument and the target tissue through the three-dimensional image. However, since the three-dimensional endoscope system needs to be configured with a binocular endoscope, there are problems of large equipment volume, high equipment cost and high complexity of optical calibration, which limit its popularization in clinical medicine. SUMMARY

[0004] Therefore, the embodiments of the present application provide a three-dimensional method of a two-dimensional image, a computer device and a system to solve the technical problems of large equipment volume, high equipment cost and high complexity of optical calibration of the existing three-dimensional endoscope system.

[0005] In a first aspect, the embodiments of the present application provide a three-dimensional method of a two-dimensional image, comprising: determining a depth map of the two-dimensional image through a depth estimation model; generating left and right eye disparity maps according to the depth map; generating left and right eye coordinate mapping maps according to the left and right eye disparity maps; obtaining a left eye view according to the two-dimensional image and the left eye coordinate mapping map, and obtaining a right eye view according to the two-dimensional image and the right eye coordinate mapping map; obtaining a three-dimensional image corresponding to the two-dimensional image according to the left eye view and the right eye view.

[0006] In an optional implementation manner of the first aspect, generating left and right eye disparity maps according to the depth map comprises: determining an original inverse depth map of the two-dimensional image according to the depth map; Normalizing the original inverse depth map to obtain a normalized inverse depth map; Performing median filtering on the normalized inverse depth map to obtain a smoothed inverse depth map; A left-eye and a right-eye disparity map are generated according to the smoothed inverse depth map.

[0007] In an optional implementation of the first aspect, the size of the left-eye and right-eye disparity maps and the size of the smoothed inverse depth map are both the same as the size of the two-dimensional image; Correspondingly, generating left-eye and right-eye disparity maps according to the smoothed inverse depth map includes: According to the pixel value of each pixel in the smoothed inverse depth map, the values ​​of the elements at the same position in the left and right eye disparity maps are calculated respectively by the following formula: disp ( i , j )= scale × phdepth ( i , j ) -1 ; in, disp ( i , j ) is the first i Rank j The value of the column element, scale is the scaling factor, phdepth ( i , j ) -1 is the first i Rank j The pixel value of the column pixel, 1≤ i ≤ h , 1≤ j ≤ w , i and j are all integers, h is the height of the two-dimensional image, w is the width of the two-dimensional image.

[0008] In an optional implementation of the first aspect, generating a left-eye coordinate mapping map and a right-eye coordinate mapping map according to the left-eye and right-eye disparity maps includes: Creating a first floating-point array and a second floating-point array of the same size as the two-dimensional image, and initializing all elements in the first floating-point array and the second floating-point array; According to the left and right eye disparity map i Rankj The value of the column element in the first floating-point array is calculated by the following formula i Rank j Predicted values ​​for column elements: float 1( i , j )=[ i , j + disp ( i , j ) / 2]; in, float 1( i , j ) is the first floating-point array i Rank j The predicted value of the column element, disp ( i , j ) is the first i Rank j The value of the column element, 1≤ i ≤ h , 1≤ j ≤ w , i and j are all integers, h is the height of the two-dimensional image, w is the width of the two-dimensional image; The integer part of the vertical coordinate of each predicted value in the first floating-point array is in [1, w ], updating the initial value with the predicted value of the first target element to obtain a first updated array, and generating a left-eye coordinate mapping map according to the first updated array; According to the left and right eye disparity map i Rank j The value of the column element in the second floating-point array is calculated by the following formula i Rank j Predicted values ​​for column elements: float 2( i , j )=[ i , j - disp ( i , j ) / 2]; in, float 2( i , j ) is the first floating point array in the second i Rank ja predicted value of a column element; a second target element whose integer part of a longitudinal coordinate of the predicted value in the second floating-point array is within [1, w ] and whose decimal part of the longitudinal coordinate of the predicted value in the second floating-point array is within [0,

[0009] In an optional implementation of the first aspect, generating the left-eye coordinate mapping map according to the first updated array comprises: updating the initial value of each third target element in the first updated array whose value is the initial value by using a value of a first reference element closest to the left of the third target element, to obtain a first reference array, wherein the value of the first reference element is not the initial value; performing an integer operation on values of all elements in the first reference array to obtain the left-eye coordinate mapping map.

[0010] In an optional implementation of the first aspect, obtaining the left-eye view according to the two-dimensional image and the left-eye coordinate mapping map comprises: creating a first blank image with the same size as the two-dimensional image; determining a first mapping pixel of a pixel in the two-dimensional image in the first row and the first column in the first blank image according to a value of an element in the left-eye coordinate mapping map in the first row and the first column; i determining a first mapping pixel of a pixel in the two-dimensional image in the first row and the first column in the first blank image according to a value of an element in the left-eye coordinate mapping map in the first row and the first column; j determining a first mapping pixel of a pixel in the two-dimensional image in the first row and the first column in the first blank image according to a value of an element in the left-eye coordinate mapping map in the first row and the first column; i determining a first mapping pixel of a pixel in the two-dimensional image in the first row and the first column in the first blank image according to a value of an element in the left-eye coordinate mapping map in the first row and the first column; j determining a first mapping pixel of a pixel in the two-dimensional image in the first row and the first column in the first blank image according to a value of an element in the left-eye coordinate mapping map in the first row and the first column; determining a first weight of the pixel in the two-dimensional image in the first row and the first column and a second weight of a pixel in the two-dimensional image in the first row and the first column+1 according to a decimal part of a value of an element in the first reference array in the first row and the first column; i determining a first weight of the pixel in the two-dimensional image in the first row and the first column and a second weight of a pixel in the two-dimensional image in the first row and the first column+1 according to a decimal part of a value of an element in the first reference array in the first row and the first column; j determining a first weight of the pixel in the two-dimensional image in the first row and the first column and a second weight of a pixel in the two-dimensional image in the first row and the first column+1 according to a decimal part of a value of an element in the first reference array in the first row and the first column; i determining a first weight of the pixel in the two-dimensional image in the first row and the first column and a second weight of a pixel in the two-dimensional image in the first row and the first column+1 according to a decimal part of a value of an element in the first reference array in the first row and the first column; j performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; i performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; j performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; i performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; j performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; i performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; j performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; i performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; j performing weighted summation on pixel values of the pixel in the two-dimensional image in the first row and the first column and the pixel in the two-dimensional image in the first row and the first column+1 according to the first weight and the second weight to obtain a pixel value of the first mapping pixel of the pixel in the two-dimensional image in the first row and the first column in the first blank image; filling pixel values of corresponding pixels in the first blank image with pixel values of the first mapping pixels respectively to obtain the left-eye view.

[0011] In an optional implementation of the first aspect, generating the right-eye coordinate mapping map according to the second updated array comprises: For a fourth target element in the second updated array, if a value of the fourth target element is the initial value, updating the initial value of the fourth target element by a value of a second reference element closest to the right of the fourth target element, to obtain a second reference array; the value of the second reference element is not the initial value; Performing an integer operation on values of all elements in the second reference array to obtain the right-eye coordinate mapping map.

[0012] In an optional implementation of the first aspect, obtaining the right-eye view according to the two-dimensional image and the right-eye coordinate mapping map comprises: Creating a second blank image with the same size as the two-dimensional image; According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; i According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; j According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image; According to a value of an element in the right-eye coordinate mapping map, determining a second mapping pixel of a pixel in the two-dimensional image in the second blank image;

[0013] In a second aspect, the embodiments of the present application provide a computer device, comprising a memory and a computer program stored in the memory and capable of running on a processor, and the processor implements the method according to any optional implementation of the first aspect when executing the computer program.

[0014] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the three-dimensional method of a two-dimensional image according to any one of the optional implementation manners of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the three-dimensional method of a two-dimensional image according to any one of the optional implementation manners of the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a three-dimensional system of a two-dimensional image, which includes a monocular camera, a computer device and a three-dimensional display; the computer device is connected with the monocular camera and the three-dimensional display. The monocular camera is configured to collect a two-dimensional image and send the two-dimensional image to the computer device. The computer device is configured to execute the method according to any one of the optional implementation manners of the first aspect. The three-dimensional display is configured to display the three-dimensional image generated by the computer device.

[0017] The three-dimensional method of a two-dimensional image, the computer device, the computer readable storage medium, the computer program product and the three-dimensional system of a two-dimensional image provided by the embodiments of the present application have the following beneficial effects: The three-dimensional method of a two-dimensional image provided by the embodiments of the present application can directly process the two-dimensional image collected by the monocular camera into the corresponding three-dimensional image, so that the acquisition of the three-dimensional image no longer depends on the traditional binocular camera. That is, the method provided by the embodiments of the present application can realize three-dimensional imaging while effectively reducing the device volume, reducing the device cost and reducing the complexity of optical calibration. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 FIG. 1 is a structural schematic diagram of a three-dimensional system of a two-dimensional image provided by an embodiment of the present application; Figure 2 FIG. 2 is a schematic flowchart of a three-dimensional method of a two-dimensional image provided by an embodiment of the present application; Figure 3A specific implementation flowchart of S202 in a three-dimensional method of a two-dimensional image provided by an embodiment of the present application is shown in the following table. Figure 4 A specific implementation flowchart of S203 in a three-dimensional method of a two-dimensional image provided by an embodiment of the present application is shown in the following table. Figure 5 A specific implementation flowchart of S204 in a three-dimensional method of a two-dimensional image provided by an embodiment of the present application is shown in the following table. Figure 6 Another specific implementation flowchart of S204 in a three-dimensional method of a two-dimensional image provided by an embodiment of the present application is shown in the following table. Figure 7 A structure schematic diagram of a computer device provided by an embodiment of the present application is shown in the following table. DETAILED DESCRIPTION

[0020] The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and thus only serve as examples, but cannot be used to limit the protection scope of the present application.

[0021] In the description of the embodiments of the present application, the technical terms "comprise", "contain", "have" and any variants thereof all mean "include but are not limited to", unless otherwise specifically emphasized. In the description of the embodiments of the present application, unless otherwise specified, the technical term "multiple" refers to two or more than two, and the technical terms "at least one" and "one or more" refer to one, two or more than two. The technical terms "first", "second" and the like are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. The technical term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects have an "or" relationship.

[0022] The embodiments of the present application first provide a three-dimensional system of a two-dimensional image. Figure 1 A structure schematic diagram of a three-dimensional system of a two-dimensional image provided by an embodiment of the present application is shown in the following table. As shown in the table, Figure 1 The system can include a camera 11, a computer device 12 and a three-dimensional (3D) display device 13.

[0023] A communication connection is established between the computer device 12 and the camera 11 and the 3D display device 13. The communication connection can be a wired communication connection (e.g., a connection based on a serial communication interface) or a wireless communication connection (e.g., a connection based on a wireless local area network). This embodiment of the application does not limit the communication connection method between the computer device 12 and the camera 11 and the 3D display device 13.

[0024] For example, the camera 11 can be a monocular camera, such as a monocular endoscope used in clinical medical scenarios. The camera 11 can be used to capture two-dimensional images and send the two-dimensional images to the computer device 12. A two-dimensional image refers to an image that does not contain depth information. The two-dimensional image can be a color image or a grayscale image. The embodiments of this application do not limit the specific type of the two-dimensional image.

[0025] For example, when the camera 11 is a monocular endoscope, the camera 11 can be used to capture a two-dimensional image of the inspected object and send the two-dimensional image of the inspected object to the computer device 12. The inspected object can be, for example, an organ (such as an intestine) or tissue in an animal body.

[0026] The computer device 12 can be used to generate a three-dimensional image corresponding to the two-dimensional image and send the three-dimensional image to the 3D display device 13. It should be noted that the specific process of the computer device 12 generating the three-dimensional image corresponding to the two-dimensional image can be referred to the relevant description in the subsequent method embodiments and will not be described in detail here.

[0027] The 3D display device 13 can be used to display a three-dimensional image corresponding to a two-dimensional image. For example, the 3D display device can be a head-mounted 3D display device or other types of 3D display devices. The embodiment of the present application does not limit the specific type of the 3D display device 13.

[0028] The embodiment of the present application also provides a method for converting a two-dimensional image into three dimensions. The execution subject of the method for converting a two-dimensional image into three dimensions can be Figure 1 In the computer device 12. For example, Figure 2 This is a schematic flow chart of a method for converting a two-dimensional image into three dimensions provided in an embodiment of the present application. Figure 2 As shown, the method may include S201 to S205, which are described in detail as follows: S201 : Determine a depth map of a two-dimensional image using a depth estimation model.

[0029] Optionally, when the two-dimensional image is a color image, the first i Rank j The pixel values ​​of the column pixels can be obtained by the three-dimensional array [ R 1( i ,j ), G 1( i , j ), B 1( i , j )] represents. R 1( i , j ) refers to the pixel value of the pixel in the red channel in the i th row and the j th column of the two-dimensional image, G 1( i , j ) refers to the pixel value of the pixel in the green channel in the i th row and the j th column of the two-dimensional image, B 1( i , j ) refers to the pixel value of the pixel in the blue channel in the i th row and the j th column of the two-dimensional image. R 1( i , j )、 G 1( i , j ) or B 1( i , j ) can be an integer between 0 and 255 (inclusive).

[0030] wherein, 1≤ i ≤ h , 1≤ j ≤ w , i and j are integers, h is the height of the two-dimensional image, w is the width of the two-dimensional image, i.e., the size of the two-dimensional image is w × h , i.e., the total number of rows of the two-dimensional image is h and the total number of columns is w .

[0031] Optionally, in the case of a two-dimensional image being a grayscale image, the pixel value of the pixel in the i th row and the j th column of the two-dimensional image can be represented by gray ( i , j ). Exemplarily, gray ( i , j ) can be an integer between 0 and 255 (inclusive).

[0032] The depth map of a two-dimensional image can be used to describe the depth information of the two-dimensional image. It should be noted that the size of the depth map of a two-dimensional image is the same as the size of the two-dimensional image, and both can be w × h The pixel value of each pixel in the depth map can be used to represent the depth value of the pixel at the same position in the two-dimensional image.

[0033] For example, the depth map i Rank j The pixel value of the column pixel can be depth ( i , j ). For example, suppose the pixel value of the pixel in the first row and first column of the depth map is depth (1,1) means that the depth value of the pixel in the first row and first column of the two-dimensional image is depth (1,1), that is, the distance between the spatial point corresponding to the pixel in the first row and first column of the two-dimensional image and the camera is depth (1,1).

[0034] Exemplarily, the depth estimation model may be obtained by training a depth-anything model (DAM) using a deep learning algorithm based on a preset sample set.

[0035] The preset sample set may include multiple pieces of sample data, each of which may consist of a sample image and a depth map of the sample image. When a computer device trains a DAM using the preset sample set, the sample image in each piece of sample data may be used as the DAM's input, and the depth map of the sample image in each piece of sample data may be used as the DAM's output, thereby enabling the DAM to learn image depth map estimation capabilities through training. In practical applications, after DAM training is completed, the computer device may determine the trained DAM as a depth estimation model.

[0036] It is understood that, in the case where the sample images are surgical images, training the DAM using a preset sample set can yield a surgical-depth-anything model (SDAM). This SDAM can be used as a depth estimation model for estimating depth maps of surgical images in clinical medical scenarios. In clinical medical scenarios, using the trained SDAM to estimate depth maps of surgical images can improve the accuracy of depth map estimation for these images.

[0037] S202: Generate left-eye and right-eye disparity maps according to the depth map.

[0038] It is understandable that there is usually parallax (i.e., visual deviation) between the left and right eyes. For example, for the same spatial point, its position in the left eye view and the position in the right eye view usually have a horizontal deviation. Based on this, the embodiment of the present application uses a left-eye disparity map to represent this parallax between the left and right eyes. The size of the left-eye disparity map is the same as that of the two-dimensional image, and both can be w × h .

[0039] The value of each element in the left-eye disparity map can be used to represent the vertical coordinate difference between the first position in the left-eye view and the second position in the right-eye view of the spatial point corresponding to the pixel at the same position in the two-dimensional image. i Rank j The value of an element can be disp ( i , j ). For example, suppose the value of the element in the 1st row and 5th column of the left and right eye disparity maps is disp (1,5) means that the difference between the vertical coordinates of the spatial point corresponding to the pixel in the first row and the fifth column of the two-dimensional image in the first position in the left eye view and the second position in the right eye view is disp (1,5) pixels.

[0040] Optionally, S202 may include: Figure 3 The S2021~S2024 shown are detailed as follows: S2021: Determine an original inverse depth map of the two-dimensional image based on the depth map.

[0041] The size of the original inverse depth map is the same as that of the two-dimensional image, and can be w × h .

[0042] Specifically, the computer device may determine the inverse of the pixel value of each pixel in the depth map as the pixel value of the pixel at the same position in the original inverse depth map. i Rank j The pixel value of the column pixel can be depth ( i , j ) -1 express.

[0043] It can be understood that since the inverse depth map usually uses 0 to represent the depth information of an infinite spatial point, it is more convenient for subsequent calculations than using ∞ to represent the depth information of an infinite spatial point in the depth map, can reduce computational complexity, and improve the efficiency of three-dimensionalization of two-dimensional images.

[0044] S2022: Normalize the original inverse depth map to obtain a normalized inverse depth map.

[0045] The embodiment of the present application normalizes the pixel values ​​of each pixel in the original inverse depth map to the interval [0, 1], so that the uncertain pixel values ​​in the original inverse depth map that are in an arbitrary interval can be mapped to a unified and predictable interval, thereby improving the consistency of the data scale and facilitating subsequent calculations.

[0046] Optionally, the computer device may normalize each pixel in the original inverse depth map using the following formula (1) to obtain a pixel value of each pixel in the normalized inverse depth map: ;Formula (1) in, p norm ( i , j ) is the first in the normalized inverse depth map i Rank j The pixel value of the column pixel, depth ( i , j ) -1 is the first i Rank j Pixel value of the column pixel, max val is the maximum pixel value in the original inverse depth map, min val is the minimum pixel value in the original inverse depth map.

[0047] S2023: Perform median filtering on the normalized inverse depth map to obtain a smoothed inverse depth map.

[0048] Optionally, the computer device can select a sliding window of a preset size, align the center pixel of the sliding window with each pixel in the normalized inverse depth map in turn, and for each pixel in the normalized inverse depth map, the computer device can replace the pixel value of the pixel with the median of all pixel values ​​in the sliding window corresponding to the pixel, thereby performing median filtering on the normalized inverse depth map to obtain a smooth inverse depth map after median filtering. Exemplarily, the above-mentioned preset size can be 3×3 pixels or 5×5 pixels, etc. The embodiment of the present application does not limit the size of the sliding window.

[0049] The embodiment of the present application performs median filtering on the normalized inverse depth map, which not only filters out the noise of the normalized inverse depth map, but also better preserves the edge details of the normalized inverse depth map, thereby improving the accuracy of subsequent left and right eye disparity map calculations, and further improving the accuracy of the final generated three-dimensional image.

[0050] S2024, generating left-eye and right-eye disparity maps according to the smoothed inverse depth map.

[0051] It can be understood that the disparity of the left eye and the right eye is proportional to the inverse depth value of the spatial point. Specifically, the closer the spatial point is to the camera, the greater the inverse depth value of the spatial point, and the greater the disparity of the left eye and the right eye; the farther the spatial point is from the camera, the smaller the inverse depth value of the spatial point, and the smaller the disparity of the left eye and the right eye.

[0052] Based on this, S2024 can specifically include the following steps: According to the pixel value of each pixel in the smoothed inverse depth map, the value of the element at the same position in the left-eye and right-eye disparity maps is calculated respectively by the following formula (2): disp i , j )= scale × phdepth i , j ) -1 ; formula (2) wherein, disp i , j ) is the value of the element in the left-eye and right-eye disparity maps, i j scale phdepth i , j ) -1 is the pixel value of the pixel in the smoothed inverse depth map, i j

[0053] In an optional implementation, the value of the scaling factor scale may be set according to experience.

[0054] In another optional implementation, the value of the scaling factor scale may be set according to the parameters (such as focal length and baseline) of the camera used to collect the two-dimensional image.

[0055] S203, generating a left-eye coordinate mapping map and a right-eye coordinate mapping map according to the left-eye and right-eye disparity maps.

[0056] wherein, the size of the left-eye coordinate mapping map and the size of the right-eye coordinate mapping map are both the same as the size of the two-dimensional image, and both can be w × h .

[0057] The value of the element in the left-eye coordinate mapping map can be used to represent the position of the spatial point in the two-dimensional image. i j The value of the element in the right-eye coordinate mapping map can be used to represent the position of the spatial point in the two-dimensional image.​​​​​​​​​​i Row j Column Row

[0058] Column i Row j Column i Row j Column Row

[0059] Column Figure 4 S2031-S2035, which are described as follows: S2031, creating a first floating-point array and a second floating-point array with the same size as the two-dimensional image, and initializing all elements in the first floating-point array and the second floating-point array.

[0060] wherein the size of the first floating-point array and the size of the second floating-point array are both the same as the size of the two-dimensional image, and both can be w × h .

[0061] In some embodiments, the computer device can initialize all elements in the first floating-point array and the second floating-point array with the two-dimensional coordinate value [0, 0], so that the initial value of all elements in the first floating-point array and the second floating-point array is [0, 0]. In other embodiments, the computer device can also initialize all elements in the first floating-point array and the second floating-point array with other two-dimensional coordinate values.

[0062] S2032, calculating the predicted value of the element in the first floating-point array at Row i Column j according to the value of the element in the left-eye disparity map at Row i Column j by the following formula (3): float 1( i , j )=[ i , j + disp ( i , j ) / 2];formula (3) wherein,float 1( i , j ) is a predicted value of an element in the first floating-point array, i row and the j column element of the left-eye disparity map, disp i , j is a value of an element in the right-eye disparity map, i row and the j column element of the left-eye disparity map.

[0063] Exemplarily, assuming that the value of the element in the first row and the first column of the right-eye disparity map (1, 1) is 5.5, it indicates that the first position of the spatial point corresponding to the pixel in the first row and the first column of the two-dimensional image in the left-eye view and the second position of the spatial point in the right-eye view are different by 5.5 pixels in the horizontal direction. Then, according to the above formula (3), the value of the element in the first row and the first column of the first floating-point array (1, 1) can be calculated as [1, 3.75]. disp float

[0064] S2033, for each first target element in the first floating-point array, whose integer part of the vertical coordinate is within [1, w ], the initial value is updated by using the predicted value of the first target element, to obtain a first updated array, and the left-eye coordinate mapping map is generated according to the first updated array.

[0065] It can be understood that the integer part of the vertical coordinate of the predicted value of some elements in the first floating-point array can not be within [1, w ]. Based on this, optionally, for the elements in the first floating-point array, whose integer part of the vertical coordinate is not within [1, w ], the computer device can discard the predicted value of the elements.

[0066] Optionally, for each first target element in the first floating-point array, whose integer part of the vertical coordinate is within [1, w ], the computer device can update the initial value of the first target element to the predicted value. Exemplarily, assuming that the width of the two-dimensional image is 10, the value of the element in the first row and the first column of the first floating-point array (1, 1) is [1, 3.75], and since the integer part of the vertical coordinate of the element is 3, which is within [1, 10], the computer device can update the initial value (for example, [0, 0]) of the first target element to [1, 3.75]. w float

[0067] ​​​​​It is understood that after the computer device updates the initial values ​​of all first target elements in the first floating-point array to the predicted values, the first updated data can be obtained. That is, the value of each first target element in the first updated data is the predicted value, and the value of each first target element in the first floating-point array is the initial value.

[0068] Optionally, S2033 may specifically include steps 1.1 to 1.2, which are described in detail as follows: Step 1.1: For each third target element in the first update array whose value is the initial value, use the value of the first reference element closest to the left of the third target element to update the initial value of the third target element to obtain the first reference array; the value of the first reference element is not the initial value.

[0069] It is understandable that since the integer part of the ordinate of the predicted value of the elements in the right columns of the first floating-point array is likely to exceed w Therefore, the computer device typically discards the predicted values ​​of the elements in the right columns of the first floating-point array, that is, does not update the initial values ​​of the elements in the right columns of the first floating-point array. This causes the values ​​of the elements in the right columns of the first update array to remain at the initial values. Based on this, for each third target element in the first update array that has an initial value, the computer device can use the value of the first reference element in the first floating-point array that is closest to the left of the third target element and whose value is not the initial value to update the initial value of the third target element, thereby obtaining the first reference array.

[0070] For example, assuming that the value of the element in the first row and column 5 of the first update array is [0, 0], and the value of the element in the first row and column 4 is [1, 6.75], the computer device may update the value of the element in the first row and column 5 of the first update array to [1, 6.75]. For another example, assuming that the value of the element in the first row and column 5 of the first update array is [0, 0], and the value of the element in the first row and column 4 is also [0, 0], and the value of the element in the first row and column 3 is [1, 5.75], the computer device may update the value of both the element in the first row and column 5 of the first update array and the element in the first row and column 4 of the first update array to [1, 5.75].

[0071] Step 1.2: perform a rounding operation on the values ​​of all elements in the first reference array to obtain a left eye coordinate mapping diagram.

[0072] Assuming that the value of the element in the 1st row and 4th column in the first reference array is [1, 6.75], after the computer device rounds it, the value of the element in the 1st row and 4th column in the left eye coordinate mapping image is [1, 6].

[0073] S2034, according to the left and right eye disparity map i Rank jThe value of the column element in the second floating-point array is calculated by the following formula (4): i Rank j Predicted values ​​for column elements: float 2( i , j )=[ i , j - disp ( i , j ) / 2]; formula (4) in, float 2( i , j ) is the first floating point in the second floating point array i Rank j The predicted values ​​of the column elements.

[0074] For example, assume that the element in the 1st row and 5th column of the left and right eye disparity maps is disp The value of (1,5) is 4.5, which means that the spatial point corresponding to the pixel in the first row and the fifth column of the two-dimensional image has a horizontal difference of 4.5 pixels between the first position in the left eye view and the second position in the right eye view. Then, according to the above formula (4), the element in the first row and the fifth column of the second floating-point array can be calculated. float The value of 2(1,5) is [1,2.75].

[0075] S2035, for each predicted value in the second floating-point array, the integer part of the vertical coordinate is in [1, w ], use the predicted value of the second target element to update the initial value to obtain a second updated array, and generate a right eye coordinate mapping map according to the second updated array.

[0076] It is understandable that the integer part of the ordinate of the predicted value of some elements in the second floating-point array may not be in the range [1, w Based on this, optionally, the integer part of the vertical coordinate of the predicted value in the second floating-point array is not in [1, w ], the computing device may discard the predicted values ​​of these elements.

[0077] Optionally, the integer part of the ordinate of each predicted value in the second floating-point array is in [1, w ], the computer device may update the initial value of the second target element to the predicted value. For example, assuming that the width of the two-dimensional image is w is 10, the element in the first row and fifth column of the second floating-point array floatThe value of the second target element in the second floating-point array 2(1, 5) is [1, 2.25], since the integer part of the ordinate of the element is 2 and 2 is within [1, 10], the computer device can update the initial value (for example, [0, 0]) of the second target element to [1, 2.25].

[0078] It can be understood that after the computer device updates the initial value of all the second target elements in the second floating-point array to the predicted value, the second updated data can be obtained. That is, the value of each second target element in the second updated data is the predicted value, and the value of each second target element in the second floating-point array is the initial value.

[0079] Optionally, S2035 can specifically include steps 2.1-2.2, which are described as follows. Step 2.1, for each fourth target element with an initial value in the second updated array, the initial value of the fourth target element is updated by using the value of the second reference element closest to the right side of the fourth target element, to obtain a second reference array; the value of the second reference element is not an initial value.

[0080] It can be understood that since the integer part of the ordinate of the predicted value of the left several columns of elements in the second floating-point array is likely to be less than 1, the computer device will usually discard the predicted value of the left several columns of elements in the second floating-point array, that is, the initial value of the left several columns of elements in the second floating-point array is not updated, so that the value of the left several columns of elements in the second updated array is still the initial value. Based on this, for each fourth target element with an initial value in the second updated array, the computer device can update the initial value of the fourth target element by using the value of the second reference element closest to the right side of the fourth target element and not an initial value in the second updated array, thereby obtaining a second reference array.

[0081] For example, assuming that the value of the element in the first row and the third column in the second updated array is [0, 0], the value of the element in the first row and the fourth column is [1, 1.25], the computer device can update the value of the element in the first row and the third column in the second updated array to [1, 1.25]. For another example, assuming that the value of the element in the first row and the third column in the second updated array is [0, 0], the value of the element in the first row and the fourth column is also [0, 0], and the value of the element in the first row and the fifth column is [1, 3.25], the computer device can update the value of the element in the first row and the third column and the value of the element in the first row and the fourth column in the second updated array to [1, 3.25].

[0082] Step 2.2, performing an integer operation on the value of all elements in the second reference array to obtain the right eye coordinate mapping diagram.

[0083] Assuming that the value of the element in the 1st row and 5th column in the second reference array is [1, 3.25], after the computer device rounds it, the value of the element in the 1st row and 5th column in the right eye coordinate mapping image is [1, 3].

[0084] S204 , obtaining a left-eye view according to the two-dimensional image and the left-eye coordinate mapping map, and obtaining a right-eye view according to the two-dimensional image and the right-eye coordinate mapping map.

[0085] Optionally, S204 may include: Figure 5 S2041 to S2045 shown are described in detail as follows: S2041: Create a first blank image with the same size as the two-dimensional image.

[0086] S2042, according to the left eye coordinate mapping i Rank j The value of the column element determines the i Rank j The column pixel is the first mapped pixel in the first blank image.

[0087] For example, assuming that the value of the element in the 1st row and 1st column of the left eye coordinate mapping map is [1,3], the computer device can determine the element in the 1st row and 3rd column of the first blank image as the first mapping pixel of the pixel in the 1st row and 1st column of the two-dimensional image in the first blank image.

[0088] S2043, according to the first reference array i Rank j The fractional part of the value of the column element determines the i Rank j The first weight of the column pixels and the i Rank j +2nd weight for the pixels in column 1.

[0089] The sum of the first weight and the second weight is 1.

[0090] Optionally, the computer device may store the first reference array i Rank j The decimal portion of the value of the column element is determined as the second weight, and the difference between 1 and the decimal portion is determined as the first weight. For example, assuming that the value of the element in the first row and first column of the first reference array is [1, 3.75], the computer device may determine the decimal portion of 3.75, 0.75, as the second weight, and determine 1-0.75=0.25 as the first weight.

[0091] S2044, according to the first weight and the second weight, i Rank j The pixel values ​​of the pixels in the column are the same as those in thei Rank j The pixel values ​​of the +1 column are weighted summed to obtain the first pixel in the two-dimensional image. i Rank j The pixel value of the column pixel is the first mapped pixel in the first blank image.

[0092] Optionally, the computer device can calculate the first i Rank j The first product of the pixel value of the column pixel and the first weight can be calculated, and the first i Rank j The second product of the pixel value of the +1 column pixel and the second weight, and the sum of the first product and the second product can be determined as the first product in the two-dimensional image. i Rank j The pixel value of the column pixel is the first mapped pixel in the first blank image.

[0093] S2045 , using the pixel values ​​of the first mapped pixels to fill the pixel values ​​of the corresponding pixels in the first blank image, to obtain a left-eye view.

[0094] Optionally, S204 may also include: Figure 6 S2046 to S2040 shown are described in detail as follows: S2046: Create a second blank image with the same size as the two-dimensional image.

[0095] S2047, according to the right eye coordinate mapping i Rank j The value of the column element determines the i Rank j The column pixel is the second mapped pixel in the second blank image.

[0096] For example, assuming that the value of the element in the 1st row and 5th column of the right eye coordinate mapping map is [1,2], the computer device can determine the element in the 1st row and 2nd column in the second blank image as the second mapping pixel of the pixel in the 1st row and 5th column in the two-dimensional image in the second blank image.

[0097] S2048, according to the first i Rank j The fractional part of the value of the column element determines the i Rank j The third weight of the column pixel and the i Rank j +4th weight for the pixels in column 1.

[0098] The sum of the third weight and the fourth weight is 1.

[0099] Optionally, the computer device can determine a decimal part of the value of the element in the first row and the first column of the second reference array as the fourth weight, and determine a difference between 1 and the decimal part as the third weight. For example, assuming that the value of the element in the first row and the fifth column of the second reference array is [1, 2.25], the computer device can determine a decimal part 0.25 of 2.25 as the fourth weight, and determine 1-0.25=0.75 as the third weight. i j Optionally, the computer device can determine a decimal part of the value of the element in the first row and the first column of the second reference array as the fourth weight, and determine a difference between 1 and the decimal part as the third weight. For example, assuming that the value of the element in the first row and the fifth column of the second reference array is [1, 2.25], the computer device can determine a decimal part 0.25 of 2.25 as the fourth weight, and determine 1-0.25=0.75 as the third weight.

[0100] S2049, according to the third weight and the fourth weight, performing weighted summation on the pixel value of the pixel in the first row and the first column of the two-dimensional image and the pixel value of the pixel in the first row and the second column, to obtain a pixel value of a second mapping pixel of the pixel in the first row and the second column in the second blank image. i j Optionally, the computer device can calculate a third product of the pixel value of the pixel in the first row and the first column of the two-dimensional image and the third weight, and can calculate a fourth product of the pixel value of the pixel in the first row and the second column of the two-dimensional image and the fourth weight, and can determine a sum of the third product and the fourth product as the pixel value of the second mapping pixel of the pixel in the first row and the second column in the second blank image. i j Optionally, the computer device can calculate a third product of the pixel value of the pixel in the first row and the first column of the two-dimensional image and the third weight, and can calculate a fourth product of the pixel value of the pixel in the first row and the second column of the two-dimensional image and the fourth weight, and can determine a sum of the third product and the fourth product as the pixel value of the second mapping pixel of the pixel in the first row and the second column in the second blank image. i j Optionally, the computer device can calculate a third product of the pixel value of the pixel in the first row and the first column of the two-dimensional image and the third weight, and can calculate a fourth product of the pixel value of the pixel in the first row and the second column of the two-dimensional image and the fourth weight, and can determine a sum of the third product and the fourth product as the pixel value of the second mapping pixel of the pixel in the first row and the second column in the second blank image.

[0101] Optionally, the computer device can calculate a third product of the pixel value of the pixel in the first row and the first column of the two-dimensional image and the third weight, and can calculate a fourth product of the pixel value of the pixel in the first row and the second column of the two-dimensional image and the fourth weight, and can determine a sum of the third product and the fourth product as the pixel value of the second mapping pixel of the pixel in the first row and the second column in the second blank image. i j Optionally, the computer device can calculate a third product of the pixel value of the pixel in the first row and the first column of the two-dimensional image and the third weight, and can calculate a fourth product of the pixel value of the pixel in the first row and the second column of the two-dimensional image and the fourth weight, and can determine a sum of the third product and the fourth product as the pixel value of the second mapping pixel of the pixel in the first row and the second column in the second blank image. i j Optionally, the computer device can calculate a third product of the pixel value of the pixel in the first row and the first column of the two-dimensional image and the third weight, and can calculate a fourth product of the pixel value of the pixel in the first row and the second column of the two-dimensional image and the fourth weight, and can determine a sum of the third product and the fourth product as the pixel value of the second mapping pixel of the pixel in the first row and the second column in the second blank image. i j Optionally, the computer device can calculate a third product of the pixel value of the pixel in the first row and the first column of the two-dimensional image and the third weight, and can calculate a fourth product of the pixel value of the pixel in the first row and the second column of the two-dimensional image and the fourth weight, and can determine a sum of the third product and the fourth product as the pixel value of the second mapping pixel of the pixel in the first row and the second column in the second blank image.

[0102] S2040, filling the pixel values of the corresponding pixels in the second blank image with the pixel values of the respective second mapping pixels, to obtain the right eye view.

[0103] S205, obtaining a three-dimensional image corresponding to the two-dimensional image according to the left eye view and the right eye view.

[0104] Optionally, the computer device can create a target blank image with a size of 2 w × h , copy all the pixels in the left eye view to the left half (i.e., from the first column to the second column) of the target blank image, and copy all the pixels in the right eye view to the right half (i.e., from the third column to the fourth column) of the target blank image, to obtain the three-dimensional image corresponding to the two-dimensional image. w w w

[0105] ​​​​​​​​​​Optionally, after obtaining the three-dimensional image corresponding to the two-dimensional image, the computer device may send the three-dimensional image corresponding to the two-dimensional image to the 3D display device 13 so that the 3D display device 13 displays the three-dimensional image.

[0106] As can be seen above, the method for converting 2D images into 3D images provided in the embodiments of this application can directly process 2D images captured by a monocular camera into corresponding 3D images, eliminating the need for traditional binocular cameras to obtain 3D images. In other words, the method provided in the embodiments of this application can achieve 3D imaging while effectively reducing the size of the device, lowering its cost, and reducing the complexity of optical calibration.

[0107] It can be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0108] An embodiment of the present application also provides a computer device. Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 7 As shown, the computer device 7 may include: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as a program corresponding to the method for converting a two-dimensional image into three dimensions. When the processor 70 executes the computer program 72, the steps in the embodiment of the method for converting a two-dimensional image into three dimensions are implemented, such as Figure 2 S201~S205 shown.

[0109] For example, the computer program 72 may be divided into one or more modules / units, one or more of which are stored in the memory 71 and executed by the processor 70 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 72 in the computer device 7.

[0110] Those skilled in the art will understand that Figure 7 This is merely an example of the computer device 7 and does not constitute a limitation on the computer device 7 , which may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0111] The processor 70 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0112] The memory 71 can be an internal storage unit of the computer device 7, for example, a hard disk or a memory of the computer device 7. The memory 71 can also be an external storage device of the computer device 7, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card or a flash card, etc. equipped on the computer device 7. Further, the memory 71 can include both the internal storage unit and the external storage device of the computer device 7. The memory 71 is used to store computer programs and other programs and data required by the computer device. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0113] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement each step in the method for three-dimensionalizing a two-dimensional image.

[0114] The embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the steps in each method embodiment.

[0115] In the above embodiments, the description of each embodiment has its own focus. The parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0116] It should be noted that, unless otherwise specified, all technical terms used in the embodiments of the present application have the same meanings as those commonly understood by the person skilled in the art to which the present application belongs. The technical terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0117] The phrase "in one embodiment" in the description of the embodiments of the present application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are merely example and that a specific example described herein can be combined with another example, even though the description might not expressly state the combination.

[0118] Those skilled in the art can understand that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0119] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for converting a two-dimensional image into three dimensions, characterized in that: include: Determining a depth map of the two-dimensional image using a depth estimation model; generating left-eye and right-eye disparity maps according to the depth map; generating a left-eye coordinate mapping map and a right-eye coordinate mapping map according to the left-eye and right-eye disparity maps; Obtaining a left-eye view according to the two-dimensional image and the left-eye coordinate mapping map, and obtaining a right-eye view according to the two-dimensional image and the right-eye coordinate mapping map; A three-dimensional image corresponding to the two-dimensional image is obtained according to the left-eye view and the right-eye view.

2. The method according to claim 1, characterized in that Generating left-eye and right-eye disparity maps according to the depth map includes: determining an original inverse depth map of the two-dimensional image according to the depth map; Normalizing the original inverse depth map to obtain a normalized inverse depth map; Performing median filtering on the normalized inverse depth map to obtain a smoothed inverse depth map; A left-eye and a right-eye disparity map are generated according to the smoothed inverse depth map.

3. The method according to claim 2, characterized in that The size of the left-eye and right-eye disparity maps and the size of the smoothed inverse depth map are both the same as the size of the two-dimensional image; Correspondingly, generating left-eye and right-eye disparity maps according to the smoothed inverse depth map includes: According to the pixel value of each pixel in the smoothed inverse depth map, the values ​​of the elements at the same position in the left and right eye disparity maps are calculated respectively by the following formula: disp ( i , j )= scale × phdepth ( i , j ) -1 ； in, disp ( i , j ) is the first i Rank j The value of the column element, scale is the scaling factor, phdepth ( i , j ) -1 is the first i Rank j The pixel value of the column pixel, 1≤ i ≤ h , 1≤ j ≤ w , i and j are all integers, h is the height of the two-dimensional image, w is the width of the two-dimensional image.

4. The method according to any one of claims 1 to 3, characterized in that Generating a left-eye coordinate mapping map and a right-eye coordinate mapping map according to the left-eye and right-eye disparity maps, including: Creating a first floating-point array and a second floating-point array of the same size as the two-dimensional image, and initializing all elements in the first floating-point array and the second floating-point array; According to the left and right eye disparity map i Rank j The value of the column element in the first floating-point array is calculated by the following formula i Rank j Predicted values ​​for column elements: float 1( i , j )=[ i , j + disp ( i , j ) / 2]； in, float 1( i , j ) is the first floating-point array i Rank j The predicted value of the column element, disp ( i , j ) is the first i Rank j The value of the column element, 1≤ i ≤ h , 1≤ j ≤ w , i and j are all integers, h is the height of the two-dimensional image, w is the width of the two-dimensional image; The integer part of the vertical coordinate of each predicted value in the first floating-point array is in [1, w ], updating the initial value with the predicted value of the first target element to obtain a first updated array, and generating a left-eye coordinate mapping map according to the first updated array; According to the left and right eye disparity map i Rank j The value of the column element in the second floating-point array is calculated by the following formula i Rank j Predicted values ​​for column elements: float 2( i , j )=[ i , j - disp ( i , j ) / 2]； in, float 2( i , j ) is the first floating point array in the second i Rank j The predicted value of the column element; The integer part of the vertical coordinate of each predicted value in the second floating-point array is in [1, w ], use the predicted value of the second target element to update the initial value to obtain a second updated array, and generate a right eye coordinate mapping map according to the second updated array.

5. The method according to claim 4, characterized in that Generating a left-eye coordinate mapping map according to the first update array includes: For each third target element in the first update array whose value is the initial value, update the initial value of the third target element using the value of the first reference element closest to the left of the third target element to obtain a first reference array; the value of the first reference element is not the initial value; A rounding operation is performed on the values ​​of all elements in the first reference array to obtain a left-eye coordinate mapping map.

6. The method according to claim 5, characterized in that Obtaining a left-eye view according to the two-dimensional image and the left-eye coordinate mapping diagram, comprising: creating a first blank image of the same size as the two-dimensional image; According to the left eye coordinate mapping i Rank j The value of the column element determines the first i Rank j a first mapped pixel of the column pixel in the first blank image; According to the first reference array i Rank j The decimal part of the value of the column element determines the i Rank j The first weight of the column pixels and the i Rank j +2nd weight of 1 column of pixels; According to the first weight and the second weight, i Rank j The pixel values ​​of the pixels in the column are the same as those in the i Rank j The pixel values ​​of the +1 column are weighted summed to obtain the first pixel in the two-dimensional image. i Rank j a pixel value of the first mapped pixel of the column pixel in the first blank image; The pixel values ​​of the corresponding pixels in the first blank image are filled with the pixel values ​​of the first mapped pixels to obtain a left-eye view.

7. The method according to claim 4, characterized in that Generating a right eye coordinate mapping map according to the second update array includes: For each fourth target element in the second update array whose value is the initial value, update the initial value of the fourth target element using the value of the second reference element closest to the right of the fourth target element to obtain a second reference array; the value of the second reference element is not the initial value; A rounding operation is performed on the values ​​of all elements in the second reference array to obtain a right eye coordinate mapping map.

8. The method according to claim 7, characterized in that Obtaining a right-eye view according to the two-dimensional image and the right-eye coordinate mapping diagram, comprising: creating a second blank image of the same size as the two-dimensional image; According to the first i Rank j The value of the column element determines the first i Rank j a second mapped pixel of the column pixel in the second blank image; According to the second reference array i Rank j The decimal part of the value of the column element determines the i Rank j The third weight of the column pixel and the i Rank j +4th weight of 1 column of pixels; According to the third weight and the fourth weight, the first i Rank j The pixel values ​​of the pixels in the column are the same as those in the i Rank j The pixel values ​​of the +1 column are weighted summed to obtain the first pixel in the two-dimensional image. i Rank j a pixel value of the second mapped pixel of the column pixel in the second blank image; The pixel values ​​of the corresponding pixels in the second blank image are filled with the pixel values ​​of the second mapped pixels to obtain a right-eye view.

9. A computer device, characterized in that: The method comprises a memory and a computer program stored in the memory and executable on a processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A system for converting a two-dimensional image into three dimensions, characterized in that: It includes a monocular camera, a computer device and a three-dimensional display; the computer device is connected to the monocular camera and the three-dimensional display; The monocular camera is used to capture a two-dimensional image and send the two-dimensional image to the computer device; The computer device is used to perform the method according to any one of claims 1 to 8; The three-dimensional display is used to display the three-dimensional image generated by the computer device.

Images