Image processing method and device, electronic equipment and readable storage medium
By acquiring and fusing the parallax information of multi-field images, the accuracy of depth values is improved, the problem of low accuracy of depth values in the prior art is solved, and the accuracy of stereoscopic display and machine vision applications is enhanced.
Patent Information
- Application Number
- CN202411352959.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-26
AI Technical Summary
When obtaining depth information, the accuracy of depth values is low, making it difficult to meet the needs of application scenarios such as stereoscopic display, machine vision and satellite remote sensing.
By acquiring the first depth image, the disparity value of N pixel points is determined, and the disparity information in the second field of view image is used for fusion calculation, the disparity value is corrected to improve the accuracy of the depth value, and a more accurate depth map is generated .
The accuracy of depth values is improved, and the generated depth map is more in line with the depth information of actual three-dimensional objects, enhancing the accuracy of applications such as stereoscopic display and machine vision.
Smart Images

Figure CN119071458A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of image processing technology, and specifically relates to an image processing method, device, electronic device and readable storage medium. Background Art
[0002] With the development of three-dimensional (3D) technology, application scenarios such as stereoscopic display, machine vision and satellite remote sensing all need to obtain depth information of the scene.
[0003] In related technologies, the depth information of a scene can be obtained by binocular ranging. Specifically, a binocular camera synchronously captures two images with slight horizontal parallax to obtain a binocular stereo image, and the parallax characteristics of human binocular vision are utilized to calculate the three-dimensional geometric information of the object being measured from multiple images obtained by the binocular camera using a stereo matching algorithm, that is, to obtain the depth value of the target object.
[0004] However, since the stereo matching algorithm is only applicable to generating a depth image of the target object by calculating the depth value after the image is input, it may cause corresponding errors in the depth value obtained after the electronic device converts the disparity value, thereby making the calculated depth value too inaccurate. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide an image processing method, device, electronic device and readable storage medium, which can improve the accuracy of depth values in depth images.
[0006] In a first aspect, an embodiment of the present application provides an image processing method, which includes: obtaining a first depth image, the first depth image including M depth points, one depth point corresponding to a first depth value, and M being a positive integer; determining N first disparity values corresponding to N first pixels based on the M first depth values, the N first pixels being all pixels in a first field of view image, and N being a positive integer greater than M; determining second disparity information of each first pixel based on the first disparity value and first disparity information corresponding to each first pixel, the first disparity information being disparity information between the first pixel and its corresponding second pixel, the second pixel being a pixel in a second field of view image, the second field of view image and the first field of view image being images of different fields of view taken in the same scene; determining the second depth value of each first pixel based on each second disparity information, and generating a second depth map based on the N second depth values.
[0007] In a second aspect, an embodiment of the present application provides an image processing device, which includes: an acquisition module, a processing module and a generation module; the acquisition module is used to acquire a first depth image, the first depth image includes M depth points, one depth point corresponds to a first depth value, and M is a positive integer; the processing module is used to determine N first disparity values corresponding to N first pixels according to the M first depth values acquired by the above acquisition module, the N first pixels are all pixels in the first field of view image, and N is a positive integer greater than M; the above processing module is also used to determine the second disparity information of each first pixel according to the first disparity value and first disparity information corresponding to each first pixel, the first disparity information is the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is a pixel in the second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; the above processing module is also used to determine a second depth value according to each second disparity information; the above generation module is used to generate a second depth map according to the N second depth values.
[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.
[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.
[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.
[0012] In an embodiment of the present application, a first depth image is obtained, the first depth image includes M depth points, one depth point corresponds to a first depth value, and M is a positive integer; based on the M first depth values, N first disparity values corresponding to N first pixels are determined, the N first pixels are all pixels in the first field of view image, and N is a positive integer greater than M; based on the first disparity value and first disparity information corresponding to each first pixel, second disparity information of each first pixel is determined respectively, the first disparity information is the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is a pixel in the second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; based on each second disparity information, the second depth value of each first pixel is determined respectively, and a second depth map is generated based on the N second depth values. In this scheme, based on the depth value in a high-precision depth image, N high-precision disparity values corresponding to N first pixel points in the first field of view are determined, and then these high-precision disparity values are fused and calculated with the first disparity information, and corrected to obtain second high-precision disparity information, and a second depth value is determined based on the second disparity information, thereby improving the accuracy of the depth value, and then generating a depth map that is more consistent with the actual three-dimensional object, that is, improving the accuracy of the generated depth map. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 It is one of the schematic diagrams of an image processing method provided in an embodiment of the present application;
[0014] Figure 2 This is a second schematic diagram of an image processing method provided in an embodiment of the present application;
[0015] Figure 3 This is a third schematic diagram of an image processing method provided in an embodiment of the present application;
[0016] Figure 4 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;
[0017] Figure 5 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;
[0018] Figure 6 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0019] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0020] The following is an explanation of the technical terms involved in the embodiments of the present application:
[0021] 1) Direct-Time of Flight (d-ToF):
[0022] In the embodiment of the present application, d-ToF refers to the direct time-of-flight method. The d-ToF technology directly measures the time difference between the emission and reception of the light pulse, and uses single photon avalanche diodes (SPAD) to achieve high-sensitivity light detection, and uses the time-correlated single-photon technology method (Time-Correlated Single-PhotonCounting, TCSPC) to achieve picosecond time accuracy. d-ToF sends discrete laser pulses, which can achieve ultra-low duty cycle. Compared with the indirect time-of-flight method (indirect-Time of Flight, i-ToF), it is more power-saving and has faster imaging speed, but the technical barriers are high and the hardware requirements are high. Because d-ToF has advantages over i-ToF in terms of rapid information acquisition, anti-interference, and imaging clarity, with the continuous maturity and improvement of algorithm technology and hardware equipment, d-ToF is expected to become the mainstream solution for deep imaging technology.
[0023] 2) d-ToF sensor: Its principle is to use a short-pulse laser of nanosecond or even picosecond level directly at the transmitting end, and then quickly receive the reflected laser, and obtain the ranging information by measuring the flight time.
[0024] 3) Depth map: defines the distance component value of the line connecting the camera optical center and the object point on the optical axis. The depth map can be used to convert pixel coordinates (u, v) into spatial three-dimensional point coordinates (x, y, z).
[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0026] The terms "at least one (item)", "at least one of" and the like in the specification and claims of this application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" means two or more, and its meaning is similar to that of "at least one (item)".
[0027] It should be noted that the image processing method provided in the embodiments of the present application can be executed by electronic devices such as mobile phones, tablet computers, laptop computers, PDAs, and vehicle-mounted electronic devices. In some embodiments of the present application, the image processing method provided in the embodiments of the present application is described by taking the electronic device as the execution subject to execute the image processing method as an example.
[0028] The image processing method, device, electronic device and readable storage medium provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0029] Depth estimation algorithms are of great use in the fields of three-dimensional reconstruction and augmented / mixed reality. Depth maps can assist MR spatial perception and virtual-reality interaction. Correct depth results can also help achieve virtual and real occlusion effects, thereby reflecting convincing three-dimensional information of the real environment and enabling people to obtain natural spatial perception. In recent years, with the development of three-dimensional (3D) technology, application scenarios such as stereoscopic display, machine vision, and satellite remote sensing all require the acquisition of scene depth information. At present, the principles for obtaining depth information can be divided into three types: binocular ranging, structured light, and time of flight (TOF). Among them, TOF is divided into direct measurement of flight time (dTOF) and indirect measurement of flight time (iTOF) based on different ranging principles.
[0030] Due to the advantages of high measurement accuracy, compact structure, fast response speed and low power consumption, dTOF sensors are considered to be an ideal choice for 3D sensing of next-generation devices. In related technologies, the process of using dTOF sensors to obtain depth information is as follows: the transmitter emits modulated pulse light with concentrated light energy, and at the same time, the receiver is turned on to receive photons during the pulse cycle. After multiple pulse cycles of exposure, the receiver receives a sufficient number of photons, and then draws a distribution histogram of the number of photons received by each pixel at different times in a pulse cycle. Based on the distribution histogram, the depth information of the target to be measured can be restored.
[0031] However, on the one hand, for vision pro glasses, the area covered by dTOF points is much smaller than the field of view of the color camera, so the spatial reconstruction range is limited. On the other hand, due to manufacturing limitations, the number of laser points emitted by the dTOF sensor is small, making the spatial resolution of the acquired dTOF data low. The sparse 3D points acquired by dTOF can usually only obtain a relatively sparse depth map, which is difficult to directly apply in some MR scenarios (such as 3D reconstruction occlusion collision, etc.).
[0032] For the above problems, those skilled in the art usually use the following existing technologies to solve them:
[0033] Prior art 1:
[0034] By setting two different regional position information to obtain the depth value of each pixel, the angular resolution can be dynamically changed at low cost and high efficiency, solving the problem of insufficient angular resolution in related technologies, and obtaining depth points with different angular resolutions. Although this method improves the angular resolution of the point cloud to a certain extent, it is still unable to obtain pixel-by-pixel point cloud and depth information. At the same time, it is difficult to obtain high-frequency depth information due to the frame rate of dTOF, so it has certain limitations in actual AR / MR applications.
[0035] Prior art 2:
[0036] By collecting sparse dot matrix depth images of faces at different angles, multiple dot matrix depth images are fused and processed to obtain a dense dot matrix depth map. Recognizing faces on this basis can improve the recognition precision and accuracy to a certain extent. Although this method can obtain a dense depth map, it is difficult to require users to fuse and generate a higher-precision depth map after shooting at multiple angles in AR / MR applications, so this method has certain limitations.
[0037] Existing technology three:
[0038] By using high-resolution RGB images to enhance low-resolution dTof depth, the depth densification effect can be achieved. However, for AR / MR applications, the field of view of the fisheye RGB camera is usually much larger than the field of view of the dtof point cloud. In this case, most areas of the RGB image are not covered by the dtof point cloud, resulting in a decrease in depth accuracy. At the same time, due to the limitation of the dtof frame rate, it is difficult to meet the requirements of outputting high frame rate depth.
[0039] However, in the image processing method provided in the embodiment of the present application, a first depth image is obtained, the first depth image includes M first depth values, M is a positive integer; according to the M first depth values, N first disparity values corresponding to N first pixels are determined, the first pixel is a pixel in the first field of view image corresponding to the first depth image, and N is a positive integer greater than M; according to the first disparity value and the first disparity information corresponding to each first pixel, a second disparity information is determined respectively, the first disparity information is: the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is a pixel in the second field of view image; according to each second disparity information, a second depth value is determined respectively, and a second depth map is generated according to the N second depth values. In this scheme, based on the depth value in a high-precision depth image, N high-precision disparity values corresponding to N first pixel points in the first field of view are determined, and then these high-precision disparity values are fused and calculated with the first disparity information, and corrected to obtain second high-precision disparity information, and a second depth value is determined based on the second disparity information, thereby improving the accuracy of the depth value, and then generating a depth map that is more consistent with the actual three-dimensional object, that is, improving the accuracy of the generated depth map.
[0040] The image processing method provided in this embodiment may be executed by an image processing device, which may be an electronic device, or a control module or a processing module in the electronic device. The technical solution provided in the embodiment of this application is described below using an electronic device as an example.
[0041] The present application provides an image processing method. Figure 1 FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present application, which can be applied to electronic devices. Figure 1 As shown, the image processing method provided in the embodiment of the present application may include the following steps 201 to 204.
[0042] Step 201: An electronic device acquires a first depth image.
[0043] In some embodiments of the present application, the first depth image includes M depth points, where M is a positive integer.
[0044] In some embodiments of the present application, one depth point corresponds to one first depth value.
[0045] In some embodiments of the present application, the first depth image is acquired by a dTOF sensor.
[0046] Exemplarily, the first depth image is generated based on a depth value corresponding to the laser flight time acquired by the dTOF sensor.
[0047] Exemplarily, the depth points may be sparse point cloud points in the first depth image.
[0048] Exemplarily, each point cloud point represents a depth value of a location.
[0049] In other words, each first depth value is a depth value of a point cloud point in the first depth image.
[0050] Step 202: The electronic device determines N first disparity values corresponding to N first pixel points according to the M first depth values.
[0051] In some embodiments of the present application, the N first pixel points are all pixel points in the first field of view image, and N is a positive integer greater than M.
[0052] It can be understood that the first depth image and the first field of view image corresponding to the first depth image are images of the same scene.
[0053] In some embodiments of the present application, the first field of view image is a three-dimensional stereo image converted from one of the RGB images in the binocular RGB images acquired by a binocular camera.
[0054] In some embodiments of the present application, the above binocular RGB image includes a left-eye RGB image and a right-eye RGB image.
[0055] Exemplarily, the first field of view image may be a left-eye stereoscopic image obtained by stereoscopically correcting a left-eye RGB image, or may be a left-eye stereoscopic image obtained by stereoscopically correcting a right-eye RGB image.
[0056] It should be noted that the field of view image corresponding to the left-eye RGB image is usually processed as the main image.
[0057] Optionally, in some embodiments of the present application, before determining the M depth values in the first field of view image, the electronic device also needs to perform a stereoscopic correction operation on the image to be processed to obtain the first field of view image. For example, the image to be processed can be rotated by a rotation matrix so that the image to be processed is transformed into a space parallel to the optical axis.
[0058] In some embodiments of the present application, the electronic device projects the M point cloud points corresponding to the M depth values, i.e., the above-mentioned depth points, into the first field of view image, and determines the M first pixel points according to the positions of the point cloud points, and then calculates the first disparity values of the M first pixel points according to the depth values of the point cloud points corresponding to the M first pixel points, to obtain the M first disparity values. Since the M point cloud points cannot completely cover all the first pixel points in the first field of view image, the electronic device determines the disparity values of the first pixel points other than the M first pixel points as the preset disparity values, thereby obtaining the first disparity values of the remaining first pixel points, and finally determines the first disparity values corresponding to the M first pixel points and the first disparity values corresponding to the remaining first pixel points as the N first disparity values of the N first pixel points.
[0059] Exemplarily, the preset parallax value may be 0 or any value.
[0060] Exemplarily, the electronic device may calculate the disparity value of the pixel point according to the depth value of the pixel point, the focal length of the camera corresponding to the first field of view image, and the baseline of the binocular camera.
[0061] Optionally, in some embodiments of the present application, the above step 202 specifically includes steps 202a to 202d:
[0062] Step 202a: The electronic device projects M depth points onto the first field of view image to determine M first pixel points.
[0063] In some embodiments of the present application, the M first pixel points correspond one-to-one to the M depth points.
[0064] In some embodiments of the present application, the M first pixel points are M pixel points among the N first pixel points in the first field of view image.
[0065] Example 1, assuming that the first depth image includes point cloud point A, point cloud point B and point cloud point C, the point cloud point is the above-mentioned depth point, the first field of view image includes first pixel point A, first pixel point B, first pixel point C, first pixel point D and first pixel point E, the point cloud point in the first depth image is projected onto the first pixel point in the first field of view image, that is, point cloud point A corresponds to the first pixel point A, point cloud point B corresponds to the first pixel point B, and point cloud point C corresponds to the first pixel point C.
[0066] Step 202b: The electronic device determines, according to the first depth value of each depth point, a first disparity value of a first pixel point corresponding to each depth point, to obtain M first disparity values.
[0067] In some embodiments of the present application, the electronic device calculates a first disparity value of a first pixel point corresponding to a depth point based on a depth value of the depth point, a camera focal length in a first field of view image, and a binocular camera baseline.
[0068] Exemplarily, the electronic device calculates the first disparity value of the first pixel using formula (1).
[0069] For example, the above formula (1) is:
[0070]
[0071] Among them, f represents the focal length of the camera, B represents the baseline of the binocular camera, z represents the depth value, and d is the first disparity value.
[0072] Step 202c: The electronic device determines the preset disparity values as NM first disparity values corresponding to the NM first pixel points.
[0073] Step 202d: determine the NM first disparity values and the M first disparity values as N first disparity values.
[0074] In some embodiments of the present application, the NM first pixel points are pixel points other than the M first pixel points among the N first pixel points.
[0075] Example 2, combined with Example 1, since point cloud point A corresponds to the first pixel point A, point cloud point B corresponds to the first pixel point B, and point cloud point C corresponds to the first pixel point C, then the first pixel point A, the first pixel point B and the first pixel point C are the above-mentioned M first pixel points, and the first pixel point D and the first pixel point E are the above-mentioned NM first pixel points.
[0076] In this way, considering that the FOV of DTof is limited, and the binocular RGB often has a larger FOV field of view, for the RGB area beyond the coverage of DTof, this embodiment combines the first depth image with the first field of view image with a wider field of view angle as described above, calculates the first disparity value, thereby effectively increasing the field of view (FOV) of the depth map, thereby achieving the goal of increasing the frame rate of the depth map.
[0077] Step 203: The electronic device determines the second disparity information of each first pixel point according to the first disparity value and the first disparity information corresponding to each first pixel point.
[0078] In some embodiments of the present application, the first disparity information is: disparity information between a first pixel point and its corresponding second pixel point.
[0079] In some embodiments of the present application, the second pixel point is a pixel point in the second field of view image.
[0080] In some embodiments of the present application, the second field of view image and the first field of view image are images of different fields of view taken in the same scene.
[0081] In some embodiments of the present application, the first disparity information is disparity information obtained by performing binocular stereo matching between the first field of view image and the second field of view image.
[0082] Exemplarily, binocular stereo matching refers to the operation of stereo matching of field images of the same target scene acquired from different field angles. The cost cube is a set of matching data obtained in binocular stereo matching, including each disparity value of each pixel and the cost value corresponding to each disparity value of each pixel.
[0083] In some embodiments of the present application, the first disparity information includes a second disparity value between the first pixel and the second pixel, and a first generation value between the first pixel and the second pixel.
[0084] Exemplarily, the second disparity value indicates an offset of the second pixel point relative to the first pixel point.
[0085] Exemplarily, the first generation value indicates the similarity between the first pixel and the second pixel.
[0086] Among them, the cost value is a numerical value that can measure the correlation between the pixel points at two positions. Regardless of whether the positions of the two pixel points are corresponding points in binocular stereo matching, the matching cost value can be calculated by the matching cost function. The smaller the cost value, the greater the correlation, which means that the probability that the two pixel points are corresponding points is also greater.
[0087] In some embodiments of the present application, the electronic device calculates a cost weight according to the first disparity value corresponding to each first pixel and the second disparity value in the first disparity information, and updates each first generation value according to the cost weight to obtain the second disparity information.
[0088] In some embodiments of the present application, the second disparity information includes a second disparity value between the first pixel and the second pixel, and an updated first generation value between the first pixel and the second pixel.
[0089] Step 204: The electronic device determines a second depth value of each first pixel point according to each second disparity information, and generates a second depth map according to the N second depth values.
[0090] Specifically, in some embodiments of the present application, the above step 204 specifically includes steps 204a to 204c:
[0091] Step 204a: The electronic device determines a target cost value with the minimum cost value from the updated L first cost values.
[0092] In some embodiments of the present application, the L first cost values are cost values between a first pixel point and the corresponding L second pixel points.
[0093] In some embodiments of the present application, a target cost value with the minimum cost value is determined from the updated L first cost values, and the target cost value is used as the final cost value corresponding to the first pixel.
[0094] Step 204b: The electronic device determines, from the L second disparity values, a target disparity value corresponding to the same second pixel point as the target cost value.
[0095] In some embodiments of the present application, since the first pixel corresponds to different second pixels at different disparity values, the electronic device uses the second pixel corresponding to the target disparity value as the target pixel. In other words, the first pixel corresponds to the target disparity value.
[0096] Step 204c: The electronic device calculates a second depth value according to the target disparity value.
[0097] In some embodiments of the present application, the electronic device selects the minimum cost value based on the second cost value in the second field of view information, determines the second disparity value corresponding to the cost value as the optimal disparity value of the first pixel, and calculates the depth value of the first pixel using formula (2) based on the optimal disparity value. After obtaining the depth values of all the first pixels, the electronic device generates a final depth map.
[0098] Exemplarily, the above formula (2) is a modified formula of formula (1), and the specific formula is as follows:
[0099]
[0100] In the image processing method provided in the embodiment of the present application, a first depth image is obtained, the first depth image includes M depth points, one depth point corresponds to a first depth value, and M is a positive integer; according to the M first depth values, N first disparity values corresponding to N first pixels are determined, the N first pixels are all pixels in the first field of view image, and N is a positive integer greater than M; according to the first disparity value and first disparity information corresponding to each first pixel, the second disparity information of each first pixel is determined respectively, the first disparity information is the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is the pixel in the second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; according to each second disparity information, the second depth value of each first pixel is determined respectively, and a second depth map is generated according to the N second depth values. In this scheme, based on the depth value in a high-precision depth image, N high-precision disparity values corresponding to N first pixel points in the first field of view are determined, and then these high-precision disparity values are fused and calculated with the first disparity information, and corrected to obtain second high-precision disparity information, and a second depth value is determined based on the second disparity information, thereby improving the accuracy of the depth value, and then generating a depth map that is more consistent with the actual three-dimensional object, that is, improving the accuracy of the generated depth map.
[0101] Further, in some embodiments of the present application, combined with Figure 1 ,like Figure 2 As shown, before the above step 203 "the electronic device determines the second disparity information of each first pixel point according to the first disparity value and the first disparity information corresponding to each first pixel point", the image processing method provided in the embodiment of the present application also includes steps 301 to 303:
[0102] Step 301: The electronic device determines L second pixel points corresponding to the first pixel point according to the first pixel point and L second disparity values.
[0103] In some embodiments of the present application, each of the first pixel points corresponds to L second pixel points, where L is a positive integer.
[0104] In some embodiments of the present application, the above-mentioned first pixel point determines a second pixel point according to a second disparity value, and so on, each first pixel point corresponds to L second pixel points.
[0105] Example 3. In combination with Example 1, the first field of view image includes a first pixel point A, a first pixel point B, a first pixel point C, a first pixel point D and a first pixel point E. Taking the first pixel point A as an example, when the second disparity value is 0, the pixel point at the same coordinate point in the second field of view image is determined as the second pixel point A1. When the second disparity value is 1, the pixel point at the same coordinate point in the second field of view image offset to the right by 1 pixel point is determined as the second pixel point A2. When the second disparity value is 2, the pixel point at the same coordinate point in the second field of view image offset to the right by 2 pixel points is determined as the second pixel point A3. By analogy, the pixel points in the second field of view image under multiple second disparity values are calculated and determined as multiple second pixel points corresponding to the first pixel point A.
[0106] Step 302: The electronic device determines a first generation value of each first pixel point according to the grayscale value of each first pixel point and the grayscale values of L second pixels corresponding to the first pixel point.
[0107] In some embodiments of the present application, each of the first generation values is used to indicate the grayscale similarity between the first pixel and a second pixel.
[0108] Step 303: The electronic device determines the L second disparity values and the L first generation values as a first disparity information corresponding to the first pixel.
[0109] In some embodiments of the present application, the electronic device performs stereo matching on the first field of view image and the second field of view image to obtain a cost cube, that is, the first disparity information.
[0110] In some embodiments of the present application, the cost cube includes a cost value corresponding to each disparity value of each first pixel point obtained by stereo matching.
[0111] Exemplarily, the electronic device may calculate the cost cube of the first field of view image by using a cost calculation method.
[0112] Exemplarily, the above cost calculation method includes any of the following: SAD (Sum of Absolute Differences) image matching algorithm, Census similarity algorithm. It should be noted that in this application, in addition to the above method, other calculation methods can also be used, and this application does not specifically limit this.
[0113] Exemplarily, the process of calculating the cost cube is to calculate the cost value between the pixel point (x, y) in the left image and the dmax pixel points in the right image (i.e., (x, y) to (x+d, y)), so as to obtain a cost cube with a length of W, a width of H, and a height of dmax, wherein the left image is the first field of view image, the right image is the second field of view image, and the cost value can be recorded as C(x, y, d), where d is the disparity value between the two pixel points.
[0114] It can be understood that the above cost calculation method process is the process from step 301 to step 303.
[0115] In this way, for pixels without dtof points, relatively accurate depth map information can be estimated through the binocular stereo matching algorithm, so as to generate a more accurate depth map in combination with the depth value obtained by the dTOF sensor.
[0116] Optionally, in some embodiments of the present application, combined with Figure 2 ,like Figure 3 As shown, in combination with the above steps 301 to 303, the above step 203 "the electronic device determines a second disparity information according to the first disparity value and the first disparity information corresponding to each first pixel point" specifically includes steps 203a to 203c:
[0117] Step 203a: The electronic device determines a cost weight based on a first disparity value corresponding to the first pixel and a second disparity value corresponding to each second pixel corresponding to the first pixel.
[0118] In some embodiments of the present application, each of the above cost weights is used to represent the error probability of a first-generation value corresponding to a second pixel point.
[0119] Exemplarily, the error probability of the first cost value is used to indicate that the smaller the similarity between the second pixel point corresponding to the cost value and the first pixel point is, the lower the matching rate between the two is.
[0120] Exemplarily, the error probability of the first-generation value is proportional to the cost weight value, that is, the greater the error probability of the first-generation value, the higher the cost weight value.
[0121] In some embodiments of the present application, the electronic device uses formula (3) to calculate the cost weight of the cost value between each second disparity value and the corresponding first disparity value one by one.
[0122] For example, the above formula (3) is:
[0123]
[0124] Taking the first pixel point A as an example, x is a second disparity value of the first pixel point A, and μ is a first disparity value of the first pixel point A.
[0125] Step 203b: The electronic device updates a first-generation value corresponding to each second pixel point based on a cost weight corresponding to each second pixel point.
[0126] In some embodiments of the present application, the electronic device uses formula (4) to perform weighted update on each first-generation value.
[0127] For example, the above formula (4) is:
[0128]
[0129] Among them, C is the cost value corresponding to the pixel point p, Represents the updated cost value.
[0130] Step 203c: The electronic device determines the L second disparity values and the updated L first generation values as one piece of second disparity information.
[0131] In this way, since the cost weight represents the probability of an error, its value is set to be less than 1. Therefore, when the cost value is updated using the cost weight, the cost value can be further reduced, making it easier to select two corresponding pixels with high similarity, thereby obtaining a more accurate depth value.
[0132] Each of the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or any two or more of them can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.
[0133] The window fuzzy processing method provided in the embodiment of the present application can be executed by an electronic device or a window fuzzy processing device. In the embodiment of the present application, the window fuzzy processing device executing the window fuzzy processing method is taken as an example to illustrate the window fuzzy processing device provided in the embodiment of the present application.
[0134] Figure 4 A possible structural diagram of an image processing device involved in an embodiment of the present application is shown. Figure 4 As shown, the image processing device 700 may include: an acquisition module 701 , a processing module 702 and a generation module 703 .
[0135] Among them, the above-mentioned acquisition module 701 is used to acquire a first depth image, which includes M depth points, one depth point corresponds to a first depth value, and M is a positive integer; the above-mentioned processing module 702 is used to determine N first disparity values corresponding to N first pixels according to the M first depth values acquired by the above-mentioned acquisition module 701, the N first pixels are all pixels in the first field of view image, and N is a positive integer greater than M; the above-mentioned processing module 702 is also used to determine the second disparity information of each first pixel according to the first disparity value and first disparity information corresponding to each first pixel, the first disparity information is: the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is the pixel in the second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; the above-mentioned processing module 702 is also used to determine the second depth value of each first pixel according to each second disparity information; the above-mentioned generation module 703 is used to generate a second depth map according to the N second depth values.
[0136] Optionally, in some embodiments of the present application, the processing module 702 is specifically used to:
[0137] Projecting the M depth points onto the first field of view image, determining M first pixel points, where the M first pixel points correspond one-to-one to the M depth points;
[0138] Determine, according to the first depth value of each depth point, a first disparity value of a first pixel point corresponding to each depth point, to obtain M first disparity values;
[0139] The preset disparity values are respectively determined as NM first disparity values corresponding to NM first pixel points, where the NM first pixel points are pixel points other than the M first pixel points among the N first pixel points;
[0140] The NM first disparity values and the M first disparity values are determined as N first disparity values.
[0141] Optionally, in some embodiments of the present application, each of the first pixel points corresponds to L second pixel points, where L is a positive integer;
[0142] The processing module 702 is further configured to, before determining the second disparity information of each first pixel point according to the first disparity value and the first disparity information corresponding to each first pixel point:
[0143] Determine, according to the first pixel and the L second disparity values, L second pixel points corresponding to the first pixel point;
[0144] Determine a first generation value of each first pixel according to the gray value of each first pixel and the gray values of L second pixels corresponding to the first pixel, each first generation value being used to indicate a gray similarity between the first pixel and a second pixel;
[0145] The L second disparity values and the L first generation values are determined as a first disparity information corresponding to the first pixel.
[0146] Optionally, in some embodiments of the present application, the processing module 702 is specifically used to:
[0147] Determine a cost weight based on a first disparity value corresponding to the first pixel and a second disparity value corresponding to each second pixel corresponding to the first pixel, wherein each cost weight is used to represent an error probability of a first generation value corresponding to a second pixel;
[0148] Based on a cost weight corresponding to each second pixel point, a first generation value corresponding to each second pixel point is updated;
[0149] The L second disparity values and the updated L first generation values are determined as one piece of second disparity information.
[0150] Optionally, in some embodiments of the present application, the processing module 702 is specifically used to:
[0151] Determine the target cost value with the minimum cost value from the updated L first cost values;
[0152] Determine, from the L second disparity values, a target disparity value corresponding to the same second pixel point as the target cost value;
[0153] A second depth value is calculated according to the target disparity value.
[0154] In the image processing device provided in the embodiment of the present application, a first depth image is obtained, the first depth image includes M depth points, one depth point corresponds to a first depth value, and M is a positive integer; according to the M first depth values, N first disparity values corresponding to N first pixels are determined, the N first pixels are all pixels in the first field of view image, and N is a positive integer greater than M; according to the first disparity value and the first disparity information corresponding to each first pixel, the second disparity information of each first pixel is determined respectively, the first disparity information is the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is the pixel in the second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; according to each second disparity information, the second depth value of each first pixel is determined respectively, and a second depth map is generated according to the N second depth values. In this scheme, based on the depth value in a high-precision depth image, N high-precision disparity values corresponding to N first pixel points in the first field of view are determined, and then these high-precision disparity values are fused and calculated with the first disparity information, and corrected to obtain second high-precision disparity information, and a second depth value is determined based on the second disparity information, thereby improving the accuracy of the depth value, and then generating a depth map that is more consistent with the actual three-dimensional object, that is, improving the accuracy of the generated depth map.
[0155] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0156] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0157] The image processing device provided in the embodiment of the present application can implement each process implemented in the embodiment of the image processing method. To avoid repetition, they will not be described again here.
[0158] Alternatively, if Figure 5 As shown, an embodiment of the present application further provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores programs or instructions that can be executed on the processor 801, and when the program or instructions are executed by the processor 801, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.
[0159] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0160] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0161] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.
[0162] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0163] Among them, the above-mentioned processor 110 is used to obtain a first depth image, which includes M depth points, one depth point corresponds to a first depth value, and M is a positive integer; the above-mentioned processor 110 is used to determine N first disparity values corresponding to N first pixels according to the M first depth values, and the N first pixels are all pixels in the first field of view image, and N is a positive integer greater than M; the above-mentioned processor 110 is also used to determine the second disparity information of each first pixel according to the first disparity value and first disparity information corresponding to each first pixel, the first disparity information is: the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is the pixel in the second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; the above-mentioned processor 110 is also used to determine the second depth value of each first pixel according to each second disparity information; the above-mentioned processor 110 is used to generate a second depth map according to the N second depth values.
[0164] Optionally, in some embodiments of the present application, the processor 110 is specifically configured to:
[0165] Projecting the M depth points onto the first field of view image, determining M first pixel points, where the M first pixel points correspond one-to-one to the M depth points;
[0166] Determine, according to the first depth value of each depth point, a first disparity value of a first pixel point corresponding to each depth point, to obtain M first disparity values;
[0167] The preset disparity values are respectively determined as NM first disparity values corresponding to NM first pixel points, where the NM first pixel points are pixel points other than the M first pixel points among the N first pixel points;
[0168] The NM first disparity values and the M first disparity values are determined as N first disparity values.
[0169] Optionally, in some embodiments of the present application, each of the first pixel points corresponds to L second pixel points, where L is a positive integer;
[0170] The processor 110 is further configured to, before determining the second disparity information of each first pixel point according to the first disparity value and the first disparity information corresponding to each first pixel point:
[0171] Determine, according to the first pixel and the L second disparity values, L second pixel points corresponding to the first pixel point;
[0172] Determine a first generation value of each first pixel according to the gray value of each first pixel and the gray values of L second pixels corresponding to the first pixel, each first generation value being used to indicate a gray similarity between the first pixel and a second pixel;
[0173] The L second disparity values and the L first generation values are determined as a first disparity information corresponding to the first pixel.
[0174] Optionally, in some embodiments of the present application, the processing module 702 is specifically used to:
[0175] Determine a cost weight based on a first disparity value corresponding to the first pixel and a second disparity value corresponding to each second pixel corresponding to the first pixel, wherein each cost weight is used to represent an error probability of a first generation value corresponding to a second pixel;
[0176] Based on a cost weight corresponding to each second pixel point, a first generation value corresponding to each second pixel point is updated;
[0177] The L second disparity values and the updated L first generation values are determined as one piece of second disparity information.
[0178] Optionally, in some embodiments of the present application, the processor 110 is specifically configured to:
[0179] Determine the target cost value with the minimum cost value from the updated L first cost values;
[0180] Determine, from the L second disparity values, a target disparity value corresponding to the same second pixel point as the target cost value;
[0181] A second depth value is calculated according to the target disparity value.
[0182] In the electronic device provided in the embodiment of the present application, by acquiring a first depth image, the first depth image includes M depth points, one depth point corresponds to a first depth value, and M is a positive integer; according to the M first depth values, N first disparity values corresponding to N first pixels are determined, the N first pixels are all pixels in the first field of view image, and N is a positive integer greater than M; according to the first disparity value and the first disparity information corresponding to each first pixel, the second disparity information of each first pixel is determined respectively, the first disparity information is the disparity information between the first pixel and the second pixel corresponding to it, the second pixel is the pixel in the second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; according to each second disparity information, the second depth value of each first pixel is determined respectively, and a second depth map is generated according to the N second depth values. In this scheme, based on the depth value in a high-precision depth image, N high-precision disparity values corresponding to N first pixel points in the first field of view are determined, and then these high-precision disparity values are fused and calculated with the first disparity information, and corrected to obtain second high-precision disparity information, and a second depth value is determined based on the second disparity information, thereby improving the accuracy of the depth value, and then generating a depth map that is more consistent with the actual three-dimensional object, that is, improving the accuracy of the generated depth map.
[0183] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0184] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0185] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.
[0186] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0187] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0188] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0189] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0190] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0191] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0192] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0193] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image processing method, characterized in that: The method comprises: Acquire a first depth image, wherein the first depth image includes M depth points, one depth point corresponds to one first depth value, and M is a positive integer; Determine, according to the M first depth values, N first disparity values corresponding to N first pixel points, where the N first pixel points are all pixel points in the first field of view image, and N is a positive integer greater than M; Determine second disparity information of each first pixel point according to the first disparity value and first disparity information corresponding to each first pixel point, the first disparity information is disparity information between the first pixel point and its corresponding second pixel point, the second pixel point is a pixel point in a second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; According to each piece of the second disparity information, a second depth value of each first pixel is determined respectively, and a second depth map is generated according to the N second depth values.
2. The method according to claim 1, characterized in that The determining, according to the M first depth values, N first disparity values corresponding to the N first pixel points includes: Projecting the M depth points onto the first field of view image to determine M first pixel points, where the M first pixel points correspond one-to-one to the M depth points; Determine, according to the first depth value of each of the depth points, a first disparity value of the first pixel point corresponding to each depth point, to obtain M first disparity values; Determine the preset disparity values as NM first disparity values corresponding to NM first pixel points, respectively, where the NM first pixel points are pixel points other than the M first pixel points among the N first pixel points; The NM first disparity values and the M first disparity values are determined as N first disparity values.
3. The method according to claim 1, characterized in that Each of the first pixel points corresponds to L of the second pixel points, where L is a positive integer; Before determining the second disparity information of each first pixel point according to the first disparity value and the first disparity information corresponding to each first pixel point, the method further includes: Determine, according to the first pixel and the L second disparity values, L second pixel points corresponding to the first pixel point; Determine a first generation value of each first pixel according to the gray value of each first pixel and the gray values of L second pixels corresponding to the first pixel, each of the first generation values being used to indicate a gray similarity between the first pixel and one of the second pixels; The L second disparity values and the L first cost values are determined as one piece of the first disparity information corresponding to the first pixel.
4. The method according to claim 3, characterized in that The determining the second disparity information of each first pixel point respectively according to the first disparity value and the first disparity information corresponding to each first pixel point includes: Determine a cost weight based on the first disparity value corresponding to the first pixel and a second disparity value corresponding to each of the second pixels corresponding to the first pixel, wherein each cost weight is used to represent an error probability of a first cost value corresponding to the second pixel; Based on the cost weight corresponding to each of the second pixels, updating the first generation value corresponding to each of the second pixels; The L second disparity values and the updated L first cost values are determined as one piece of second disparity information.
5. The method according to claim 4, characterized in that The determining the second depth value of each first pixel point according to each second disparity information respectively includes: Determine a target cost value with the minimum cost value from the updated L first cost values; Determine, from the L second disparity values, a target disparity value corresponding to the same second pixel point as the target cost value; The second depth value is calculated according to the target disparity value.
6. An image processing device, characterized in that: The image processing device comprises: an acquisition module, a processing module and a generation module; The acquisition module is used to acquire a first depth image, wherein the first depth image includes M depth points, one depth point corresponds to one first depth value, and M is a positive integer; The processing module is used to determine N first disparity values corresponding to N first pixel points according to the first depth values obtained by the M obtaining modules, where the N first pixel points are all pixel points in the first field of view image, and N is a positive integer greater than M; The processing module is further used to determine the second disparity information of each first pixel point according to the first disparity value and the first disparity information corresponding to each first pixel point, the first disparity information is the disparity information between the first pixel point and the second pixel point corresponding to it, the second pixel point is a pixel point in a second field of view image, and the second field of view image and the first field of view image are images of different fields of view taken in the same scene; The processing module is further used to determine the second depth value of each first pixel point according to each second disparity information; The generating module is configured to generate a second depth map according to the N second depth values.
7. The device according to claim 6, characterized in that The processing module is specifically used for: Projecting the M depth points onto the first field of view image to determine M first pixel points, where the M first pixel points correspond one-to-one to the M depth points; Determine, according to the first depth value of each of the depth points, a first disparity value of the first pixel point corresponding to each depth point, to obtain M first disparity values; Determine the preset disparity values as NM first disparity values corresponding to NM first pixel points, respectively, where the NM first pixel points are pixel points other than the M first pixel points among the N first pixel points; The NM first disparity values and the M first disparity values are determined as N first disparity values.
8. The device according to claim 6, characterized in that Each of the first pixel points corresponds to L of the second pixel points, where L is a positive integer; The processing module is further configured to, before determining the second disparity information of each first pixel point respectively according to the first disparity value and the first disparity information corresponding to each first pixel point: Determine, according to the first pixel and the L second disparity values, L second pixel points corresponding to the first pixel point; Determine a first generation value of each first pixel according to the gray value of each first pixel and the gray values of the L second pixels corresponding to the first pixel, each of the first generation values being used to indicate a gray similarity between the first pixel and one of the second pixels; The L second disparity values and the L first cost values are determined as one piece of the first disparity information corresponding to the first pixel.
9. The device according to claim 8, characterized in that The processing module is specifically used for: Determine a cost weight based on the first disparity value corresponding to the first pixel and a second disparity value corresponding to each of the second pixels corresponding to the first pixel, wherein each cost weight is used to represent an error probability of a first cost value corresponding to the second pixel; Based on the cost weight corresponding to each of the second pixels, updating the first generation value corresponding to each of the second pixels; The L second disparity values and the updated L first cost values are determined as one piece of second disparity information.
10. The device according to claim 9, characterized in that The processing module is specifically used for: Determine a target cost value with the minimum cost value from the updated L first cost values; Determine, from the L second disparity values, a target disparity value corresponding to the same second pixel point as the target cost value; The second depth value is calculated according to the target disparity value.
11. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the image processing method as claimed in any one of claims 1 to 5.
12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
Image processing device, imaging device, and image processing method
CN103493093A
Image processing method and device
CN108496201A
Depth recovery method, electronic equipment and computer readable storage medium
CN114299129A
Image processing method and device, electronic equipment and computer readable storage medium
CN116912533A
Method for generating intermediate view-point depth image based on disparity increment of stereo depth images
WO2013108944A1
