Method, electronic device and storage medium for acquiring scene depth information

Abstract

The embodiments of the present invention relate to the field of 3D sensing measurement technology, and disclose a method, electronic device, and storage medium for acquiring scene depth information. The present invention includes: performing several rounds of seed growth on a scene image captured by an image acquisition device; acquiring target disparity information of the growth pixels that have successfully grown in each round; obtaining depth information of the scene image based on the target disparity information of each round; acquiring target disparity information of the growth pixels that have successfully grown in each round includes: acquiring disparity information of the seed pixels in the current round and the seed neighborhood of the seed pixels; correcting the initial disparity information of each growth pixel in the seed neighborhood to obtain target disparity information, and the initial disparity information is set as the disparity information of the seed pixels. By adopting this embodiment, the variable structured light system can increase the speed of resolving the depth information of the scene and improve the accuracy of the depth information.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of 3D sensing measurement technology, and in particular to a method, electronic device, and storage medium for acquiring scene depth information. Background Art

[0002] Depth perception technologies mainly include structured light methods, passive binocular stereo vision, time-of-flight methods, and other technologies. Among them, spatially coded structured light technology is the most commonly used depth perception technology, and is widely used in the fields of consumer electronics, smart home, and medical industry. The projector in the spatially coded structured light system projects a modulated pseudo-random speckle pattern into the scene to be measured, obtains the corresponding scene map through the optical sensor, and matches the scene map with one or more reference maps stored in the camera to obtain the position relationship of the same-name points. Based on the position relationship of the same-name points and the principle of triangulation, the depth information of the scene to be measured can be obtained. Compared with passive binocular matching, spatially coded structured light technology can greatly enhance the scene features by actively projecting speckle patterns, which greatly improves the accuracy and speed of matching.

[0003] However, if the positional relationship between the projector and the optical sensor in the structured light system changes, the reference image is calibrated and the depth information of the scene image is obtained based on the corrected reference image. This results in a slow speed for obtaining the depth information of the scene image and also causes the obtained depth information to be inaccurate. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method, electronic device and storage medium for obtaining scene depth information, so that when the positional relationship between the projector and the optical sensor in the structured light system changes, the speed of solving the depth information of the scene can be improved and the accuracy of the depth information can be improved.

[0005] To solve the above technical problems, an embodiment of the present invention provides a method for acquiring scene depth information, which is applied to a structured light system including a projector and an image acquisition device, and the method includes: performing several rounds of seed growth on the scene image captured by the image acquisition device; acquiring target disparity information of the grown pixels that have successfully grown in each round; obtaining the depth information of the scene image based on the target disparity information of each round; acquiring the target disparity information of the grown pixels that have successfully grown in each round, specifically including: acquiring the disparity information of the seed pixels in the current round and the seed neighborhood of the seed pixels, the disparity information of the seed pixels being the distance between the seed pixels and the corresponding same-name points in a pre-stored reference image; correcting the initial disparity information of each grown pixel in the seed neighborhood to obtain target disparity information, and the initial disparity information being set as the disparity information of the seed pixels.

[0006] An embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned method for acquiring scene depth information.

[0007] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned method for acquiring scene depth information when executed by a processor.

[0008] The method for obtaining scene depth information in the embodiment of the present application is applied to a structured light system, which includes a projector and an image acquisition device. If the positional relationship between the projector and the image acquisition device changes, the triangular relationship used to calculate the depth information of the scene image will change, and the pre-stored reference image will be inaccurate and unable to be calculated. In this example, in each round of regional growing of the scene image, the initial disparity information of the growing pixels of various sub-neighborhoods is corrected, and the target disparity information of each growing pixel in the seed neighborhood is obtained. Since there is no need to calculate the depth information of the scene image based on the corrected reference image, there is no need to correct the reference image, which improves the speed of solving the depth information; improves the real-time performance of the depth information acquisition, and increases the applicable scenarios.

[0009] In addition, before correcting the initial disparity information of each growing pixel in the seed neighborhood and obtaining the target disparity information, the method further includes: determining the correspondence between the disparity information of the seed pixel and the target disparity according to the relative rotation relationship between the reference image and the scene image, wherein the correspondence is expressed as: r =d s -|AS|+|TB|, where d r Indicates the target disparity information, d s represents the disparity information of the seed pixel, |AS| represents the distance between the seed pixel and the growth pixel, and |TB| represents the distance between the position of the homonymous point corresponding to the seed pixel and the position of the homonymous point corresponding to the growth pixel. During the region growing process, the seed pixel and the growth pixel in its seed neighborhood usually belong to the same depth region, so the initial disparity information of each growth pixel is the disparity information of the seed pixel. When the positional relationship between the projector and the image acquisition device changes, the correspondence between the disparity information of the seed pixel and the target disparity information can be determined based on the relative rotation relationship. Based on this correspondence, the accurate target disparity information can be quickly determined to achieve correction of the initial disparity information of the growth pixel.

[0010] In addition, correcting the initial disparity information of each growth pixel in the seed neighborhood to obtain the target disparity information includes: obtaining the distance between the seed pixel and the growth pixel as a first distance; obtaining the distance between the position of the same name point corresponding to the seed pixel and the position of the same name point corresponding to the growth pixel as a second distance; and obtaining the target disparity information based on the corresponding relationship, the disparity information of the seed pixel, the first distance, and the second distance. By obtaining the first distance and the second distance and using the corresponding relationship, the target disparity information can be quickly obtained.

[0011] In addition, obtaining disparity information for seed pixels in the current round includes: selecting candidate seed pixels from the scene image; obtaining a matching value between the candidate seed pixel and a corresponding point of the same name in the reference image as a candidate matching value; and if the candidate matching value exceeds a preset selection threshold, selecting the candidate seed pixel as the seed pixel and obtaining disparity information for the seed pixel. Before obtaining the seed neighborhood, seed pixels are obtained from the scene image, and disparity information for the seed pixels is obtained, and the seed pixels are quickly screened out using a selection threshold.

[0012] In addition, selecting candidate seed pixels from the scene image includes: selecting pixels in the scene image that are spaced apart by an n*n grid as candidate seed pixels in the scene image, where n is an integer greater than 0. This method can take into account the coverage of candidate seed pixels and can quickly obtain candidate seed pixels, thereby improving the speed of selecting candidate seed pixels.

[0013] In addition, obtaining a matching value between the candidate seed pixel and a corresponding homonymous point in the reference image as the candidate matching value includes: selecting a homonymous point of the candidate seed pixel from the reference image using a preset matching strategy, the preset matching strategy including any one of the following: binarization, sum of absolute differences, or zero-mean cross correlation; and calculating a matching value between the candidate seed pixel and the selected homonymous point as the candidate matching value. Multiple methods are used to obtain the candidate matching value, providing flexibility in the acquisition method.

[0014] In addition, several rounds of seed growth are performed on the scene image captured by the image capture device, specifically including: after each round of seed growth is completed, reducing the selection threshold to obtain the seed pixels of the next round.

[0015] In addition, before determining the correspondence between the disparity information of the seed pixel and the target disparity based on the relative rotation relationship between the reference image and the scene image, the method further includes: performing the following processing on each pixel in the seed neighborhood: obtaining a matching value between the pixel and its corresponding same-name point; if the matching value exceeds a preset growth threshold, determining that the pixel has been successfully grown and marking it as a grown pixel. Marking successfully grown pixels avoids the problem of duplicate growth. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0017] Figure 1 is a flowchart of a method for acquiring scene depth information provided according to a first embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of a specific implementation of obtaining target disparity information of successfully grown pixels in each round according to the first embodiment of the present invention;

[0019] Figure 3 is a flowchart of a method for acquiring scene depth information provided according to a second embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of a seed neighborhood provided according to a second embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of a triangular relationship provided according to a second embodiment of the present invention;

[0022] Figure 6 is a schematic diagram of a seed pixel and a growth pixel provided according to a second embodiment of the present invention;

[0023] Figure 7 This is a schematic diagram of a specific implementation of obtaining target disparity information of successfully grown pixels in each round according to the third embodiment of the present invention;

[0024] Figure 8 is a schematic diagram of candidate seed pixels provided according to a third embodiment of the present invention;

[0025] Figure 9 is a structural block diagram of an electronic device provided according to a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, each embodiment of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will appreciate that in each embodiment of the present invention, many technical details are provided to help readers better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0027] The following embodiments are divided for the convenience of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0028] The first embodiment of the present invention relates to a method for obtaining scene depth information. It is applied to a structured light system including a projector and an image acquisition device, and its process is as follows: Figure 1 and 2 As shown:

[0029] Step 101: Perform several rounds of seed growth on the scene image captured by the image capture device. Each round of growth process executes steps 102 and 103 until all rounds of seed growth are completed.

[0030] Step 102: Obtain target disparity information of the successfully grown pixel in each round. The target disparity information is the distance between the grown pixel and the corresponding point of the same name in the pre-stored reference image.

[0031] Step 103: Obtain depth information of the scene image based on the target disparity information of each round.

[0032] To obtain the target disparity information of the successfully grown pixels in each round, the specific process is as follows: Figure 2 As shown:

[0033] Step 1021: Obtain disparity information of the seed pixel in the current round and the seed neighborhood of the seed pixel.

[0034] Step 1022: Correct the initial disparity information of each growing pixel in the seed neighborhood to obtain target disparity information. The initial disparity information is set as the disparity information of the seed pixel. The disparity information of the seed pixel is the distance between the seed pixel and the corresponding point of the same name in the reference image.

[0035] The method for obtaining scene depth information in the embodiment of the present application is applied to a structured light system, which includes a projector and an image acquisition device. When the positional relationship between the projector and the image acquisition device changes, the triangular relationship used to calculate the depth information of the scene image changes, and the pre-stored reference image is inaccurate and cannot be calculated. In this example, in each round of regional growing of the scene image, the initial disparity information of the growing pixels of various sub-neighborhoods is corrected, and the target disparity information of each growing pixel in the seed neighborhood is obtained. Since there is no need to calculate the depth information of the scene image based on the corrected reference image, there is no need to correct the reference image, which improves the speed of solving the depth information; improves the real-time performance of the depth information acquisition, and increases the applicable scenarios.

[0036] The second embodiment of the present invention relates to a method for acquiring scene depth information. The second embodiment is a detailed introduction to the first embodiment.

[0037] This method for acquiring scene depth information is applied to a structured light system, which includes a projector and an image acquisition device. The projector may be a laser emitter, which can be used to project a pseudo-random speckle pattern onto a target object. The image acquisition device may be a camera. The camera captures an image of the target object to obtain a scene image, i.e., the scene image includes the pseudo-random speckle pattern projected by the projector. A reference image may be pre-stored, which is an image formed by projecting the projector onto a standard object at different distances. By matching the scene image with the reference image, the disparity between each pixel in the scene image and the corresponding point of interest in the reference image is obtained. The depth information in the scene image is determined based on the disparity of each pixel, the distance to a reference plane represented by the reference image, and a triangular relationship. The triangular relationship includes the relationship between multiple triangles formed by the position of the image acquisition device, the position of the projector, the position of the target object, and the position of the reference plane.

[0038] Regarding step 101 above, when a change in the positional relationship between the projector and the image acquisition device is detected, a region growing method can be used to perform several rounds of seed growth on the scene image. In this example, the region growing method is used to perform several rounds of seed growth on the scene image. The region growing algorithm is not further described here. The number of rounds can be at least two. The number of rounds of seed growth can be set according to actual needs. For example, in this example, the number of rounds is six.

[0039] Regarding step 103 above, after obtaining the target disparity information for each growth pixel in each round of growth, the initial disparity information of each growth pixel in the seed neighborhood is corrected to obtain the target disparity information. A new triangular relationship is reconstructed using the target disparity information of each growth pixel and the disparity information of the seed pixel to determine the depth information of the scene image. The method for constructing the triangular relationship is generally similar to existing methods and will not be further described here. A growth pixel is a pixel that has successfully grown.

[0040] Regarding the above step 102, the process is as follows: Figure 3 As shown, including:

[0041] Step 2021: Obtain the disparity information of the seed pixel in the current round and the seed neighborhood of the seed pixel.

[0042] Specifically, the scene image includes an image of a target object, and the depth information of the scene image is the distance from the target object to the image acquisition device in the structured light system. Several rounds of seed growth are performed on the scene image. During each round of seed growth, a seed pixel is obtained, and based on the seed pixel, a seed neighborhood of the seed pixel is obtained. The seed neighborhood is an area surrounded by a preset number of pixels around the seed pixel, for example, Figure 4 As shown, the seed pixel is a black area, and there are 8 pixels around the seed pixel forming the seed neighborhood of the seed pixel. The 8 pixels are as follows Figure 4 After obtaining the seed pixel, the disparity information of the seed pixel can be obtained.

[0043] Step 2022: Perform the following processing on each pixel in the seed neighborhood: obtain the matching value between the pixel and its corresponding point of the same name.

[0044] Specifically, the growth threshold can be set according to the actual application, and each pixel in the seed neighborhood is processed as follows: a matching value between the pixel and the corresponding point of the same name in the reference image is obtained; if the matching value exceeds the growth threshold, it indicates that the pixel and the seed pixel are in the same area, and step 2023 is executed.

[0045] It should be noted that the method of obtaining the matching value between a pixel and a point of the same name corresponding to the pixel in the reference image is similar to the existing method and will not be described in detail here.

[0046] Step 2023: If the matching value exceeds the preset growth threshold, it is determined that the pixel growth is successful and marked as a grown pixel.

[0047] Specifically, the identification information of the growth pixel can be set according to actual applications, and the growth pixel can be identified through the identification information.

[0048] Step 2024: Obtain the distance between the seed pixel and the growth pixel as the first distance.

[0049] Specifically, the distance between the seed pixel and the growth pixel is obtained in the scene image, and the distance is used as the first distance, which is recorded as |AS|, where |·| represents the distance.

[0050] Step 2025: Obtain the distance between the position of the seed pixel corresponding to the same-name point and the position of the growth pixel corresponding to the same-name point as the second distance.

[0051] Obtain the homonymous point position of the seed pixel in the reference image, denoted as T, and obtain the homonymous point position of the growth pixel, denoted as B; obtain the distance between the homonymous point position of the seed pixel and the homonymous point position of the growth pixel in the reference image, and use this distance as the second distance, denoted as |TB|, where |·| represents the distance.

[0052] Step 2026: Obtain target disparity information according to the corresponding relationship, the disparity information of the seed pixel, the first distance, and the second distance.

[0053] Specifically, for a monocular structured light system, the triangular relationship is as follows: Figure 5 In the figure, point P is a point on the target object, P' is the intersection of the projected light and the reference plane, the point P on the L plane is denoted as a1, the point P' on the L plane is denoted as a2, and the distance between a1 and a2 is the parallax of point P in the scene image. When the positional relationship between the projector and the image acquisition device in the monocular structured light system structure changes, the original triangular relationship no longer exists, that is, the reference image cannot be used to calculate the parallax and depth information, and the reference image needs to be corrected to re-establish the triangular relationship of the current structure. If the reference image is corrected and the triangular relationship is reconstructed, the parallax information of each pixel in the scene image is obtained based on the reconstructed three-piece relationship, and the solution speed is slow.

[0054] The change in the position of the projector relative to the image acquisition device is a rigid transformation, that is, a combination of translation and rotation transformations. However, for the speckle pattern projected by the projector received by the image acquisition device, the entire speckle image does not simply undergo translation and rotation, but rather follows the process of homography transformation, perspective transformation, and homography transformation.

[0055] In one example, the correspondence between the disparity information of the seed pixel and the target disparity can be determined based on the relative rotation relationship between the reference image and the scene image. This correspondence can be determined before or after executing step 2024.

[0056] Specifically, the first distance between the seed pixel S and its neighboring point A, and the second distance between its corresponding points T and B are approximately equal to each other. Figure 6 As shown, the seed pixel is S, the same name point of the seed pixel is T, the growth pixel is A, and the same name point of the growth pixel A is B. According to the relative rotation relationship, formula (1) can be obtained:

[0057] d r -d s ≈|TB|-|AS| Formula (1);

[0058] d r Indicates the target disparity information, d s represents the disparity information of the seed pixel, |AS| represents the first distance, and |TB| represents the second distance. By transforming formula (1), we can get the corresponding relationship, which is expressed as:

[0059] d r =d s -|AS|+|TB| Formula (2);

[0060] Among them, d r Indicates the target disparity information, d s represents the disparity information of the seed pixel, |AS| represents the first distance, and |TB represents the second distance.

[0061] According to formula (2) and the first distance, the disparity information of the seed pixel and the second distance, the target disparity information can be calculated.

[0062] It should be noted that after obtaining the target disparity information, the depth information of the scene image can be calculated.

[0063] In this example, during the region growing process, the seed pixel and the growth pixel in its seed neighborhood belong to the same depth region, the positional relationship between the projector and the image acquisition device changes, and the correspondence between the disparity information of the seed pixel and the target disparity information is determined based on the relative rotation relationship. Based on this correspondence, the accurate target disparity information can be quickly determined.

[0064] The third embodiment of the present invention relates to a method for obtaining scene depth information. The third embodiment is a further improvement to step 102 of the above embodiment. The main improvement is that pixels spaced in an n*n grid are used as candidate seed pixels in the scene image. The process is as follows Figure 7 As shown:

[0065] Step 3021: Select candidate seed pixels from the scene image.

[0066] Specifically, the scene image is composed of pixels, and the scene image is a grid diagram as shown in FIG. There are many ways to obtain candidate seed pixels. In one example, pixels spaced apart by an n*n grid can be used as candidate seed pixels in the scene image.

[0067] like Figure 8 As shown, n is 3, and the candidate seed pixels are selected at 3*3 grid intervals, such as Figure 8 The black pixels in the image are obtained, wherein the coordinate position of each pixel can be the position of the upper left corner of the pixel. In this example, 12 candidate seed pixels are obtained.

[0068] Step 3022: Obtain the matching value between the candidate seed pixel and the corresponding point of the same name in the reference image as the candidate matching value.

[0069] In one example, the specific process of obtaining the matching value between the candidate seed pixel and the corresponding homonymous point in the reference image as the candidate matching value can be: selecting the homonymous point of the candidate seed pixel from the reference image using a preset matching strategy, the preset matching strategy including any one of the following: through binarization, sum of absolute differences (SAD) or zero mean cross correlation (ZNCC); calculating the matching value between the candidate seed pixel and the selected homonymous point.

[0070] Specifically, due to changes in the positional relationship between the projector and the image acquisition device, the location of the seed pixel's homonymous point may be in a different row, requiring a matching search across multiple rows. However, during subsequent seed growth, a single-line search of several pixels along the direction of the seed point is sufficient. Cross-row searches using matching calculation methods such as binarization, SAD, or ZNCC can yield homonymous points for candidate seed pixels.

[0071] Step 3033: If the candidate matching value exceeds the preset selection threshold, the candidate seed pixel is used as the seed pixel, and the disparity information of the seed pixel is obtained.

[0072] Specifically, the selection threshold can be set according to actual application. If the matching value exceeds the selection threshold, the candidate seed pixel is used as the seed pixel, and the disparity information of the seed pixel is obtained. The disparity information of the seed pixel can be obtained based on the reference image and the scene image.

[0073] In one example, after each round of seed growth is completed, the selection threshold is reduced to obtain seed pixels for the next round.

[0074] Specifically, if there are no more growing pixels in the current round and growth stops, it is determined that the seed growth of the current round is completed, and the selection threshold can be reduced by a preset number. Based on the new selection threshold, the candidate seeds are screened again, and the candidate seed pixels that exceed the updated selection threshold are selected as the seed pixels of the next round.

[0075] Step 3034: Get the seed neighborhood of the seed pixel in the current round.

[0076] Step 3035: Perform the following processing on each pixel in the seed neighborhood: obtain the matching value between the pixel and its corresponding point of the same name; if the matching value exceeds the preset growth threshold, execute step 3036.

[0077] Step 3036: Determine that the pixel growth is successful and mark it as a grown pixel.

[0078] Step 3037: Obtain the distance between the seed pixel and the growth pixel as the first distance.

[0079] Step 3038: Obtain the distance between the position of the seed pixel corresponding to the same-name point and the position of the growth pixel corresponding to the same-name point as the second distance.

[0080] Step 3039: Obtain target disparity information according to the corresponding relationship, the disparity information of the seed pixel, the first distance, and the second distance.

[0081] The above steps 3034 to 3039 are substantially the same as steps 2021 to 2026 in the second embodiment and will not be described again here.

[0082] For ease of understanding, the process of determining scene depth information is described below:

[0083] Step S1 selects candidate seed pixels from the scene image at an n*n grid interval. Step S2 selects the same-name points of the candidate seed pixels using a matching calculation method such as binarization, SAD, or ZNCC. Step S3 performs the following processing on each candidate seed pixel: determines whether the candidate matching value of the candidate seed pixel exceeds a selection threshold. If so, the candidate seed pixel is used as a seed pixel and the disparity information of the seed pixel is calculated. Step S4 obtains the seed neighborhood of the seed pixel in the current round. Step S5 performs the following processing on each pixel in the seed neighborhood: obtains the matching value between the pixel and its corresponding same-name point. If the matching value exceeds a preset growth threshold, step S6 is executed. Step S6 determines that the pixel has been grown successfully and marks it as a grown pixel. Step S7 obtains the distance between the seed pixel and the grown pixel as a first distance. Step S8 obtains the distance between the same-name point position corresponding to the seed pixel and the same-name point position corresponding to the grown pixel as a second distance. Step S9 obtains the corresponding relationship between the target disparity information and the disparity information of the seed pixel based on the relative rotation relationship. Step S10: Obtain target disparity information based on the correspondence, disparity information of the seed pixel, the first distance, and the second distance. Step S11: Reduce the selection threshold to obtain the next round of seed pixels. Repeat steps S4 to S11 until region growing is complete. Step S12: Obtain depth information for the scene image based on the target disparity information obtained in each round.

[0084] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0085] It is worth noting that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovations of the present invention, this embodiment does not include units that are not closely related to solving the technical problems proposed by the present invention. However, this does not mean that other units do not exist in this embodiment.

[0086] A fourth embodiment of the present invention relates to an electronic device, a structural block diagram of which is shown in Figure 9, comprising: at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501; wherein the memory 502 stores instructions that can be executed by the at least one processor 501, and the instructions are executed by the at least one processor 501 to enable the at least one processor 501 to execute the above-mentioned method for obtaining scene depth information.

[0087] The memory 502 and processor 501 are connected using a bus. The bus can include any number of interconnected buses and bridges, linking various circuits of one or more processors 501 and memory 502. The bus can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0088] The processor 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory can be used to store data used by the processor when performing operations.

[0089] A fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for acquiring scene depth information is implemented.

[0090] Those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing related hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0091] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A method for obtaining scene depth information, characterized in that: Applied to a structured light system including a projector and an image acquisition device, the method includes: performing several rounds of seed growth on the scene images captured by the image capture device; Obtain the target disparity information of the successfully grown pixels in each round of growth; Obtaining depth information of the scene image according to the target disparity information of each round; The step of obtaining target disparity information of the successfully grown pixels in each round of growth specifically includes: Obtaining disparity information of a seed pixel in a current round and a seed neighborhood of the seed pixel, wherein the disparity information of the seed pixel is the distance between the seed pixel and a corresponding point of the same name in a pre-stored reference image; Correcting the initial disparity information of each growing pixel in the seed neighborhood to obtain the target disparity information, wherein the initial disparity information is set as the disparity information of the seed pixel; wherein, Before correcting the initial disparity information of each growing pixel in the seed neighborhood to obtain the target disparity information, a correspondence between the disparity information of the seed pixel and the target disparity is determined based on the relative rotation relationship between the reference image and the scene image. The correspondence is expressed as: ,in, represents the target disparity information, represents the disparity information of the seed pixel, represents the distance between the seed pixel and the growth pixel, represents the distance between the position of the seed pixel corresponding to the same name point and the position of the growth pixel corresponding to the same name point; Correcting the initial disparity information of each growing pixel in the seed neighborhood to obtain the target disparity information includes: Acquire the distance between the seed pixel and the growth pixel as a first distance; Obtaining the distance between the position of the same-name point corresponding to the seed pixel and the position of the same-name point corresponding to the growth pixel as a second distance; The target disparity information is acquired according to the corresponding relationship, the disparity information of the seed pixel, the first distance, and the second distance.

2. The method for obtaining scene depth information according to claim 1, wherein: The obtaining of disparity information of seed pixels in the current round includes: Selecting candidate seed pixels from the scene image; Obtaining a matching value between the candidate seed pixel and the corresponding point of the same name in the reference image as a candidate matching value; If the candidate matching value exceeds a preset selection threshold, the candidate seed pixel is used as the seed pixel, and disparity information of the seed pixel is obtained.

3. The method for obtaining scene depth information according to claim 2, wherein: The selecting candidate seed pixels from the scene image includes: Pixels spaced apart in an n*n grid in the scene image are used as the candidate seed pixels, where n is an integer greater than 0.

4. The method for obtaining scene depth information according to claim 2, wherein: The obtaining a matching value between the candidate seed pixel and the corresponding point of the same name in the reference image as a candidate matching value includes: Selecting the same-name points of the candidate seed pixels from the reference image using a preset matching strategy, wherein the preset matching strategy includes any one of the following: binarization, absolute difference, or zero-mean cross correlation; A matching value between the candidate seed pixel and the selected point of the same name is calculated as the candidate matching value.

5. The method for obtaining scene depth information according to claim 2, wherein: The performing several rounds of seed growth on the scene images acquired by the image acquisition device specifically includes: After each round of seed growth is completed, the selection threshold is reduced to obtain the seed pixels of the next round.

6. The method for obtaining scene depth information according to claim 1, wherein: Before correcting the initial disparity information of each growing pixel in the seed neighborhood to obtain the target disparity information, the method further includes: The following processing is performed on each pixel in the seed neighborhood: a matching value between the pixel and its corresponding point of the same name is obtained; If the matching value exceeds a preset growth threshold, it is determined that the pixel has grown successfully and is marked as a grown pixel.

7. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for acquiring scene depth information as described in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for acquiring scene depth information according to any one of claims 1 to 6 is implemented.

Images