Ranging method based on structural light source, depth camera, electronic equipment and medium

Abstract

The invention discloses a distance measuring method based on a structural light source, a depth camera, electronic equipment and a medium. The distance measurement method comprises the steps that a distance measurement image obtained in the distance measurement process based on a structural light source is acquired, the distance measurement image comprises a light spot bright area group and a light spot dark area group, the light spot bright area group comprises a plurality of bright area groups, and each bright area group is matched with at least one dark area group in the light spot dark area group; determining a multipath interference variable of each bright area group according to the data of each bright area group and the data of the dark area group corresponding to the bright area group; and correcting the data of each bright area group based on the multipath interference variable of each bright area group so as to screen out bright area group data participating in distance measurement calculation from the distance measurement image. Through the scheme of the invention, the calculation accuracy of the multipath interference variable can be improved, and accurate multipath correction is realized, so that the distance measurement accuracy is effectively improved.

Description

Technical Field

[0001] The present application generally relates to the field of distance measurement technology. More specifically, the present application relates to a distance measurement method based on a structured light source, and a depth camera, an electronic device and a computer-readable storage medium for executing the aforementioned distance measurement method. Background Art

[0002] Time of flight (TOF) is a method of calculating the distance of an object by measuring the time it takes for a light pulse to be emitted and reflected. As one of the 3D ranging solutions, it is being used in more and more scenarios. However, depth ranging achieved by TOF is usually affected by multipath interference in actual scenarios, resulting in reduced measurement accuracy.

[0003] At present, multipath correction schemes such as deep learning and multipath modulation solution are commonly used to solve the problem of multipath interference. However, these schemes have problems such as high computational complexity and difficulty in practical application, which makes the final correction effect not obvious.

[0004] In view of this, there is an urgent need to provide a ranging solution based on structured light source in order to achieve accurate multipath correction and improve ranging accuracy. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a distance measurement method based on a structured light source, an electronic device and a storage medium in multiple aspects.

[0006] In a first aspect, the present application provides a distance measurement method based on a structured light source, comprising: acquiring a distance measurement image obtained in a distance measurement process based on a structured light source, wherein the distance measurement image comprises a light spot bright area group and a light spot dark area group, and the light spot bright area group comprises a plurality of bright area groups, and each bright area group is matched with at least one dark area group in the light spot dark area group; determining a multipath interference amount of each bright area group according to data of each bright area group and data of its corresponding dark area group; and correcting the data of each bright area group based on the multipath interference amount of each bright area group, so as to filter out the bright area group data participating in the distance measurement calculation from the distance measurement image.

[0007] In some embodiments, determining the multipath interference amount of each bright area group based on the data of each bright area group and the data of its corresponding dark area group includes: obtaining a brightness ratio and a calibration ratio between the at least one dark area group and the corresponding bright area group; obtaining a baseline multipath correction amount for the at least one dark area group; and determining the multipath interference amount of the bright area group corresponding to the at least one dark area group based on the acquired brightness ratio, the calibration ratio and the baseline multipath correction amount.

[0008] In some embodiments, correcting the data of each bright area group based on the multipath interference amount of each bright area group includes: calculating the difference between the baseline multipath correction amount of the at least one dark area group and the multipath interference amount of the corresponding bright area group to complete the correction of the data of the bright area group corresponding to the at least one dark area group.

[0009] In some embodiments, obtaining the brightness ratio between the at least one dark area group and the corresponding bright area group includes: obtaining the average brightness value of the at least one dark area group; obtaining the average brightness value of the bright area group corresponding to the at least one dark area group; and calculating the ratio between the average brightness value of the at least one dark area group and the average brightness value of the corresponding bright area group to obtain the brightness ratio.

[0010] In some embodiments, obtaining a calibration ratio between the at least one dark area group and the corresponding bright area group includes: obtaining a calibration ratio between the at least one dark area group and the corresponding bright area group; obtaining a calibration image formed by the structured light source in an environment without multipath interference, wherein the calibration image includes the multiple bright area groups, and each bright area group is matched with the at least one dark area group; obtaining a calibration brightness value mean of the at least one dark area group; obtaining a calibration brightness value mean of the bright area group corresponding to the at least one dark area group; and calculating a ratio between the calibration brightness value mean of the at least one dark area group and the calibration brightness value mean of the corresponding bright area group to obtain the calibration ratio.

[0011] In some embodiments, determining the multipath interference amount of the bright area group corresponding to the at least one dark area group includes: determining the multipath correction coefficient of the at least one dark area group based on the acquired brightness ratio and the calibration ratio; and determining the multipath interference amount of the bright area group corresponding to the at least one dark area group based on the multipath correction coefficient of the at least one dark area group and the benchmark multipath correction amount.

[0012] In some embodiments, wherein the at least one dark area group includes a first dark area group and / or a second dark area group, determining the multipath interference amount of the bright area group corresponding to the at least one dark area group includes: in response to a brightness value of the current bright area group being greater than a threshold, determining a multipath correction coefficient of the first dark area group corresponding to the current bright area group according to a brightness ratio and a calibration ratio between the first dark area group corresponding to the current bright area group and the current bright area group, and determining the multipath interference amount of the current bright area group based on the multipath correction coefficient of the first dark area group corresponding to the current bright area group and a reference multipath correction amount; in response to a brightness value of the current bright area group being less than or equal to a threshold, determining the multipath correction coefficients of the first dark area group and the second dark area group corresponding to the current bright area group according to a brightness ratio and a calibration ratio between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, and determining the multipath interference amount of the current bright area group based on the multipath correction coefficients of the first dark area group and the second dark area group corresponding to the current bright area group and a reference multipath correction amount.

[0013] In some embodiments, the multipath correction coefficient of at least one dark area group is determined according to the following formula: Rd1=(R1-RC1) / Rref; Rd2=(R2-RC2) / Rref; wherein Rd1 and Rd2 are respectively the multipath correction coefficients of the first dark area group and the second dark area group, R1 and R2 are respectively the brightness ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, RC1 and RC2 are respectively the calibration ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, and Rref is a preset constant value.

[0014] In some embodiments, the data collected by the image acquisition device of the ranging image is four-phase data, and in response to the brightness value of the current bright area group being greater than a threshold, the multipath interference amount of the current bright area group is determined according to the following formula: ΔI= Rd1×I1, ΔQ= Rd1×Q1; or in response to the brightness value of the current bright area group being less than or equal to a threshold, the multipath interference amount of the current bright area group is determined according to the following formula: ΔI= Rd1×I1+ Rd2×I2, ΔQ=Rd1×Q1+ Rd2×Q2; wherein Rd1 and Rd2 are multipath correction coefficients of the first dark area group and the second dark area group respectively, ΔI and ΔQ are multipath interference amounts of the current bright area group, I1, Q1, I2, Q2 are benchmark multipath correction values ​​of the first dark area group and the second dark area group respectively, and I1=D11-D13, Q1=D12-D14, I2=D21-D23, Q2=D22-D24, D11~D14, D21~D24 are four-phase data collected by the image acquisition device respectively.

[0015] In some embodiments, the distance measurement method further includes: grouping the light spot bright area groups to obtain the multiple bright area groups; and acquiring a brightness value of each bright area group to configure at least one dark area group for each bright area group based on the acquired brightness value.

[0016] In some embodiments, configuring at least one dark area group for each bright area group based on the acquired brightness value includes: in response to the brightness value of each bright area group being greater than a threshold, configuring one dark area group for each bright area group; or in response to the brightness value of each bright area group being less than or equal to a threshold, configuring multiple dark area groups for each bright area group; wherein the dark area groups in the light spot dark area group are divided according to their positional relationship with the bright area groups.

[0017] In some embodiments, the ranging method further includes: grouping the light spot bright area groups to obtain the multiple bright area groups; and configuring multiple dark area groups for each bright area group, wherein the dark area groups in the light spot dark area groups are divided according to their positional relationship with the bright area groups.

[0018] In a second aspect, the present application provides a depth camera, comprising: a projection end, configured to project a structured light source for ranging; a receiving end, configured to receive the structured light source for ranging; a memory, which stores computer instructions for ranging based on the structured light source; and a processor, which runs the computer instructions so that the depth camera executes the ranging method based on the structured light source described in the foregoing and multiple embodiments below.

[0019] In a third aspect, the present application provides an electronic device, comprising: a depth camera as described in the third aspect.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium comprising program instructions for ranging based on a structured light source. When the program instructions are executed by a processor, the ranging method based on a structured light source described in the foregoing and following embodiments is implemented.

[0021] Through the distance measurement method based on the structured light source, the depth camera, the electronic device and the medium provided above, the embodiment of the present application groups the light spot bright area group and the light spot dark area group in the distance measurement image collected by the structured light source, and determines the multipath interference amount of each bright area group according to the data of each bright area group and the data of its corresponding dark area group, so as to correct the data of the corresponding bright area group based on the multipath interference amount to screen out the bright area group data participating in the distance measurement calculation. Therefore, in the distance measurement process, the scheme of the present application can reduce the influence of multipath interference to a certain extent compared with other light sources by using the structured light source for distance measurement. At the same time, considering that the brightness ratio of the bright area and the dark area in the image will change before and after being interfered with by multipath and the degree of influence of the bright and dark areas at different positions is different, the light spot bright area group in the distance measurement image can be grouped, and the corresponding dark area group can be matched for each bright area group obtained by the grouping, so as to make full use of the data of the grouped bright area group and the data of the corresponding dark area group to calculate the multipath interference amount, improve the calculation accuracy of the multipath interference amount, and realize accurate multipath correction, thereby effectively improving the distance measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 A schematic diagram showing a flow chart of a distance measurement method based on a structured light source according to an embodiment of the present application is shown; Figure 2 A schematic diagram showing a flow chart of a distance measurement method based on a structured light source according to another embodiment of the present application is shown; Figure 3 A schematic diagram showing the distribution of bright and dark areas in a ranging image according to an embodiment of the present application is shown; Figure 4a~Figure 4d A schematic diagram showing the distribution of bright area groups and corresponding dark area groups in ranging images collected by different structured light sources according to an embodiment of the present application; and Figure 5 A schematic structural block diagram of a depth camera according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0024] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0025] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0026] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0027] Exemplary application scenarios In the distance measurement scenario based on TOF technology, the TOF camera is required to emit light to the target scene, and the sensor receives the light reflected from the target scene and calculates the distance through the time-of-flight method. This principle is based on the fact that the light in the target scene will only be reflected once. However, in actual measurement, the light will be reflected and refracted multiple times in the target scene. The signal received by each pixel may contain reflected light from multiple paths, resulting in multipath interference. TOF technology is affected by multipath interference, which reduces the measurement accuracy. In this regard, existing technologies mostly use multipath correction solutions such as deep learning and multipath modulation solution, but these solutions are difficult to calculate and difficult to implement in practical applications.

[0028] In order to solve the problems in the above scenarios, the inventors have found through research that the brightness ratio of the bright and dark areas in the image will change before and after being affected by multipath interference, and the bright and dark areas at different positions are affected to different degrees. The relevant data of the bright and dark areas in the image can be used to accurately calculate the amount of multipath interference in the bright area. Based on this, a ranging scheme based on a structured light source is proposed, which can group the bright area groups of light spots in the ranging image, and match the corresponding dark area groups for each bright area group obtained by grouping, so as to make full use of the data of the grouped bright area group and the corresponding dark area group to calculate the amount of multipath interference, improve the calculation accuracy of the multipath interference, and achieve accurate multipath correction, thereby effectively improving the ranging accuracy.

[0029] The following combination Figure 1~Figure 5 The scheme of this application is described in detail.

[0030] Figure 1 A schematic flow chart of a distance measurement method 100 based on a structured light source according to an embodiment of the present application is shown.

[0031] like Figure 1 As shown, at step S101, a ranging image obtained in the ranging process based on the structured light source can be obtained. The ranging image includes a light spot bright area group and a light spot dark area group, and the light spot bright area group can include multiple bright area groups, and each bright area group is matched with at least one dark area group in the light spot dark area group.

[0032] In step S102, the multipath interference amount of each bright area group may be determined according to the aforementioned data of each bright area group and the corresponding data of the dark area group.

[0033] In step S103, the data of each bright area group may be corrected based on the multipath interference amount of each bright area group, so as to filter out the bright area group data involved in the ranging calculation from the ranging image.

[0034] In the solution of the present application, the aforementioned structured light source may specifically include a line light source and a point array light source (i.e., a SPOT light source), and the line light source further includes a single-line light source and a multi-line light source. Compared with other light sources (such as a planar array light source, etc.), these structured light sources are relatively less affected by multipath interference during the ranging process, but may still cause insufficient ranging accuracy, especially in scenarios where high ranging accuracy is required. In addition, the structured light source in the present application is not limited to the various light sources listed above, but may also include other structured light sources that can meet ranging requirements. The specific type of the structured light source is not limited here, and can be set and adjusted according to the ranging requirements.

[0035] In this embodiment, the distance measurement can be performed by a structured light source such as a single-line light source, a multi-line light source or a SPOT light source. For example, a device that supports projecting a structured light source and receiving a structured light source (such as a TOF camera device) can be used to project and receive a structured light source into a distance measurement scene to form the aforementioned distance measurement image.

[0036] Further, the obtained ranging image can be subdivided into a light spot bright area group and a light spot dark area group. In some embodiments, the light spot bright area and the light spot dark area can be distinguished according to the brightness of the pixels in the image, and then the light spot bright area and the light spot dark area are further grouped to obtain the light spot bright area group and the light spot dark area group. In other embodiments, it may not be necessary to distinguish the light spot bright area and the light spot dark area according to the pixel brightness. For example, when designing a device that supports projecting a structured light source and receiving a structured light source, it can be specified which positions on the image are light spot bright areas and which positions are light spot dark areas, and then it can be determined whether it belongs to a light spot bright area or a light spot dark area according to the position information of the pixel in the image.

[0037] In this embodiment, the aforementioned light spot bright area group may include multiple bright area groups, and the number of bright area groups is not limited here, and can be adjusted specifically according to the distance measurement accuracy requirement. Generally, the higher the distance measurement accuracy requirement, the more bright area groups are divided. For example, for a ranging image obtained based on a line structured light source, when the light spot bright area and the light spot dark area in the ranging image are distributed by row, for the light spot bright area of ​​any row, one pixel in the light spot bright area of ​​the row can be used as a bright area group, or two, three, or even more than three pixels can be used as a bright area group. That is, the solution of the present application does not limit the number of bright area groups in the light spot bright area group and the number of pixels in each bright area group.

[0038] After the bright area groups are divided, the dark areas of the light spots can be divided for each bright area group, so as to configure at least one dark area group for each bright area group. In this embodiment, the number of dark area groups that the bright area group needs to match can be determined according to the brightness value of the bright area group and / or the ranging accuracy, etc. For example, when the brightness value of the bright area group is high, it can be considered that it is less affected by multipath interference to a certain extent, and only one dark area group can be configured at this time. When the brightness value of the bright area group is low, it can be considered that it is greatly affected by multipath interference to a certain extent, and multiple dark area groups can be configured at this time. Of course, if the ranging requirement is high, multiple dark area groups can also be configured for each bright area group.

[0039] In addition, when configuring a dark area group for a bright area group, the positional relationship between the pixels in the dark area of ​​the light spot and the pixels in the bright area group is mainly considered, and the pixels in the dark area of ​​the light spot that are adjacent, close to, or within a certain range of distance are preferably used as the pixels of the dark area group corresponding to the bright area group. The solution of the present application does not limit the number of dark area groups corresponding to the bright area group and the number of pixels in each dark area group, which need to be adjusted and set according to the characteristics of the light source, the distribution of bright and dark areas in the image, the ranging accuracy, and other factors.

[0040] After obtaining each bright area group and its corresponding dark area group, the data of each bright area group and its corresponding dark area group can be obtained. In practical applications, when there is multipath interference, the data of the bright area group covers the data of the multipath interference light path and the data of the real light path required for ranging. Since the brightness ratio of the bright area and the dark area in the image will change before and after the multipath interference, especially the brightness ratio of the bright area and the dark area will change greatly before and after the multipath interference. Therefore, it is possible to consider combining the data of the dark area group to simulate or simulate the degree of multipath interference of the bright area, so as to effectively correct the data of the bright area group.

[0041] In this embodiment, it is also considered that the positions of bright and dark areas in the image are different, and the multipath interference effects on them are also different. Therefore, the bright area of ​​the light spot can be subdivided into multiple bright area groups, and a corresponding dark area group is configured for each bright area group, so as to combine the data of the bright area group and the data of the corresponding dark area group, accurately calculate the multipath interference amount of each bright area group, and accurately correct the data of the brightness group based on the obtained multipath interference amount.

[0042] By using structured light sources for ranging, the impact of multipath interference can be reduced to a certain extent compared to other light sources. At the same time, considering that the brightness ratio of the bright and dark areas in the image will change before and after multipath interference, and the bright and dark areas at different positions are affected to different degrees, the light spot bright area groups in the ranging image are grouped, and each bright area group obtained by grouping is matched with a corresponding dark area group, so as to fully utilize the data of the grouped bright area group and the corresponding dark area group to calculate the multipath interference amount, improve the calculation accuracy of the multipath interference amount, and realize accurate multipath correction, thereby effectively improving the ranging accuracy.

[0043] Figure 2 FIG. 2 is a flow chart of a distance measurement method 200 based on a structured light source according to another embodiment of the present application. It should be noted that: Figure 2 The ranging method 200 in the embodiment can be understood as Figure 1 Therefore, the above combined with the distance measurement method 100 Figure 1 The relevant description also applies to the following.

[0044] like Figure 2As shown, at step S201, a ranging image including a light spot bright area group and a light spot dark area group can be obtained. The ranging image is obtained in the ranging process based on the structured light source. The light spot bright area group can include multiple bright area groups, and each bright area group is matched with at least one dark area group in the light spot dark area group. The structured light source, the light spot bright area group, the light spot dark area group, and the bright area group and the dark area group can all be referred to Figure 1 Related description in .

[0045] In this embodiment, the distance measurement can be performed specifically by a structured light source such as a single-line light source, a multi-line light source or a SPOT light source. For example, a device that supports projecting a structured light source and receiving a structured light source can be used to project and receive a single-line light source, a multi-line light source, a SPOT light source or other structured light sources into the distance measurement scene to form the aforementioned distance measurement image. When the distance measurement image is obtained, the light spot bright area group can be grouped to obtain a plurality of bright area groups. Then, the brightness value of each bright area group is obtained, so as to configure at least one dark area group for each bright area group based on the obtained brightness value. For example, in response to the brightness value of each bright area group being greater than a threshold value (the threshold value can be set according to actual needs), a dark area group is configured for each bright area group. For another example, in response to the brightness value of each bright area group being less than or equal to a threshold value, multiple dark area groups are configured for each bright area group. For another example, the light spot bright area group can be grouped to obtain a plurality of bright area groups, and multiple dark area groups are configured for each bright area group. Wherein the dark area groups in the light spot dark area group are divided according to the positional relationship between the light area groups.

[0046] Generally, the higher the ranging accuracy requirement, the more bright area groups are divided. This embodiment does not limit the number of bright area groups and the number of pixels in each bright area group, and can be adjusted according to the ranging accuracy requirement. As mentioned above, for the ranging image obtained based on the line structure light source, the bright area and dark area of ​​the light spot in the ranging image can be distributed by row. For the bright area of ​​the light spot in any row, one pixel in the bright area of ​​the light spot in the row can be used as a bright area group, or two, three, or even more than three pixels can be used as a bright area group. In addition, the number of dark area groups corresponding to the bright area group and the number of pixels in each dark area group are not limited, and they need to be adjusted and set according to the characteristics of the light source, the distribution of bright and dark areas in the image, the ranging accuracy and other factors. For example, when the brightness value of the bright area group is high, it can be considered to be less affected by multipath interference to a certain extent, and only one dark area group can be configured at this time. When the brightness value of the bright area group is low, it can be considered to be more affected by multipath interference to a certain extent, and multiple dark area groups can be configured at this time. Of course, if the distance measurement requirement is high, multiple dark area groups can also be configured for each bright area group. Thus, different distance measurement requirements can be met through different grouping methods for bright and dark areas.

[0047] The following can be combined Figure 3~Figure 4d Let's further illustrate the division of bright area groups and dark area groups. It should be noted that Figure 3~Figure 4d The distribution of the bright area and the dark area of ​​the light spot and the division and distribution of the bright area group and the corresponding dark area group are only exemplary descriptions and can be adjusted according to actual application requirements (such as light source characteristics, ranging requirements, etc.).

[0048] As an example, Figure 3 A specific distribution diagram of the bright and dark areas of the light spot in the ranging image obtained based on the line structure light source is shown. Figure 3 In the range-finding image, the bright area and dark area of ​​the light spot can be distributed by row (only as an example). The white row area at position 301 in the image represents the bright area of ​​the light spot, while the gray row area and the black row area at position 302 represent the dark area of ​​the light spot. For the bright area of ​​the light spot in any row (the white row area at position 301), one pixel in the bright area of ​​the light spot can be used as a bright area group, or two, three, or even more than three pixels can be used as a bright area group. The dark area group corresponding to each bright area group can use the pixels in the gray row area and / or the pixels in the black row area adjacent to the pixels in the bright area group.

[0049] For example, Figure 4a In the figure, the spot bright area group includes multiple bright area groups, wherein one bright area group L1 can be assigned two dark area groups (e.g., the first dark area group and the second dark area group). The pixels in the first dark area group are the pixels of the corresponding area of ​​the previous row and the corresponding area of ​​the next row (at position A1 in the figure) of the bright area group L1. The pixels in the second dark area group are the pixels of the corresponding area of ​​the next row upward and the corresponding area of ​​the next row upward (at position A2 in the figure) of the bright area group L1. Figure 4a In the example, the bright area group L1 may include one pixel, and the first dark area group and the second dark area group may include two pixels respectively.

[0050] For example, Figure 4b In the figure, the spot bright area group includes multiple bright area groups, wherein one bright area group L1 can be assigned two dark area groups (e.g., the first dark area group and the second dark area group). The pixels in the first dark area group are the pixels of the neighboring area corresponding to the previous row and the neighboring area corresponding to the next row (at position A1 in the figure) of the bright area group L1. The pixels in the second dark area group are the pixels of the neighboring area corresponding to the next row upward and the neighboring area corresponding to the next row upward (at position A2 in the figure) of the bright area group L1. Figure 4b In the example, the bright area group L1 may include one pixel, and the first dark area group and the second dark area group may include 6 pixels respectively.

[0051] For another example, Figure 4cIn the figure, the spot bright area group includes multiple bright area groups, wherein one bright area group L1 can be assigned two dark area groups (e.g., the first dark area group and the second dark area group). The pixels in the first dark area group are the pixels in the corresponding area of ​​the previous row and the corresponding area of ​​the next row (at position A1 in the figure) of the bright area group L1. The pixels in the second dark area group are the pixels in the corresponding neighborhood area of ​​the previous row and the corresponding neighborhood area of ​​the next row (at position A2 in the figure) of the bright area group L1. Figure 4c In the example, the bright area group L1 may include one pixel, the first dark area group may include two pixels, and the second dark area group may include four pixels.

[0052] For example, Figure 4d In the , the ranging image is obtained based on the SPOT light source. The spot bright area group includes multiple bright area groups, where one bright area group L1 can be assigned two dark area groups (for example, the first dark area group and the second dark area group). The first dark area group is the pixels in the dark area neighborhood area (at position A1 in the figure) of the upper and lower rows (at least one row) of the corresponding column of each bright area group, and the second dark area group is the pixels in the dark area of ​​the spot (at position A2 in the figure) at a certain distance threshold from the pixels in the bright area group. Figure 4d In the example, the bright area group L1 may include one pixel, and the first dark area group and the second dark area group may include four pixels respectively.

[0053] After dividing the bright area group and the corresponding dark area group, return Figure 2 In step S202, a brightness ratio and a calibration ratio between the at least one dark area group and the corresponding bright area group may be obtained.

[0054] Among them, the average brightness value of at least one dark area group (for example, the average brightness value of all pixels in the dark area group) can be obtained, and the average brightness value of the bright area group corresponding to the at least one dark area group (for example, the average brightness value of all pixels in the bright area group) can be obtained, and then the ratio between the average brightness value of the at least one dark area group and the average brightness value of the corresponding bright area group is calculated to obtain the brightness ratio between the aforementioned at least one dark area group and the corresponding bright area group.

[0055] In practical applications, the calibration ratio between at least one dark area group and the corresponding bright area group can be obtained in advance, and of course, the calibration ratio between at least one dark area group and the corresponding bright area group can also be calculated in real time. Specifically, a calibration image formed by a structured light source in an environment without multipath interference (such as a flat white standard plate, etc.) is obtained. Among them, the calibration image includes multiple bright area groups, and each bright area group is matched with at least one dark area group. It should be noted that the grouping of bright area groups and dark area groups in the calibration image is consistent with the grouping of bright area groups and dark area groups in the ranging image during the actual ranging process. Next, the mean value of the calibration brightness value of at least one dark area group is obtained, and the mean value of the calibration brightness value of the bright area group corresponding to at least one dark area group is obtained. Then, the ratio between the mean value of the calibration brightness value of at least one dark area group and the mean value of the calibration brightness value of the corresponding bright area group is calculated to obtain the calibration ratio.

[0056] At step S203, a reference multipath correction value of at least one dark area group may be obtained. A signal is usually represented by two orthogonal components, namely, I (In-phase) and Q (Quadrature) components. The multipath effect will affect these two components, causing the amplitude and phase of the signal to change. Therefore, the reference multipath correction value may be divided into an I component and a Q component.

[0057] In step S204, the multipath interference amount of the bright area group corresponding to at least one dark area group may be determined based on the acquired brightness ratio, the calibration ratio and the reference multipath correction amount.

[0058] Specifically, the multipath correction coefficient of at least one dark area group can be determined according to the acquired brightness ratio and the calibration ratio. Then, based on the multipath correction coefficient of at least one dark area group and the reference multipath correction amount, the multipath interference amount of the bright area group corresponding to the at least one dark area group is determined.

[0059] In some embodiments, if the brightness value of the current bright area group is greater than a threshold, a dark area group (e.g., a first dark area group) may be configured for the current bright area group. Specifically, in response to the brightness value of the current bright area group being greater than the threshold, the multipath correction coefficient of the first dark area group corresponding to the current bright area group is determined based on the brightness ratio and calibration ratio between the first dark area group corresponding to the current bright area group and the current bright area group, and the multipath interference amount of the current bright area group is determined based on the multipath correction coefficient of the first dark area group corresponding to the current bright area group and the reference multipath correction amount. The multipath correction coefficient of the first dark area group may be determined according to the formula Rd1=(R1-RC1) / Rref. Rd1 is the multipath correction coefficient of the first dark area group, R1 is the brightness ratio between the first dark area group corresponding to the current bright area group and the current bright area group, RC1 is the calibration ratio between the first dark area group corresponding to the current bright area group and the current bright area group, and Rref is a preset constant value.

[0060] Furthermore, the data collected by the image acquisition device of the ranging image is four-phase data (for example, D11, D12, D13 and D14), and the multipath interference amount of the current bright area group is determined according to the following formula: ΔI = Rd1×I1, ΔQ = Rd1×Q1. Among them, Rd1 is the multipath correction coefficient of the first dark area group, ΔI and ΔQ are the multipath interference amounts of the current bright area group, I1 and Q1 are the reference multipath correction values ​​of the first dark area group, and I1=D11-D13, Q1=D12-D14.

[0061] In this implementation scenario, the brightness value of the current bright area group is greater than the threshold, indicating that the previous bright area group may be less affected by multipath interference. At this time, only one dark area group is used for multipath interference correction, which ensures the correction accuracy while reducing the overall calculation amount.

[0062] In other embodiments, if the brightness value of the current bright area group is less than or equal to a threshold, multiple dark area groups (e.g., a first dark area group and a second dark area group) may be configured for the current bright area group. Specifically, in response to the brightness value of the current bright area group being less than or equal to the threshold, the multipath correction coefficients of the first dark area group and the second dark area group corresponding to the current bright area group are determined respectively according to the brightness ratios and calibration ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, and the multipath interference amount of the current bright area group is determined based on the multipath correction coefficients of the first dark area group and the second dark area group corresponding to the current bright area group and the reference multipath correction amount. It should be noted that the multiple dark area groups here are not limited to the first and second dark area groups, and more dark area groups may be set as needed.

[0063] The multipath correction coefficients of the first dark area group and the second dark area group are determined according to the formula Rd1=(R1-RC1) / Rref, Rd2=(R2-RC2) / Rref. Rd1 and Rd2 are respectively the multipath correction coefficients of the first dark area group and the second dark area group, R1 and R2 are respectively the brightness ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, RC1 and RC2 are respectively the calibration ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, and Rref is a preset constant value.

[0064] Furthermore, the data collected by the image acquisition device of the ranging image is four-phase data (for example, D11, D12, D13 and D14, D21, D22, D23 and D24), and the multipath interference amount of the current bright area group is determined according to the following formula: ΔI=Rd1×I1+ Rd2×I2, ΔQ= Rd1×Q1+ Rd2×Q2. Among them, Rd1 and Rd2 are multipath correction coefficients of the first dark area group and the second dark area group respectively, ΔI and ΔQ are multipath interference amounts of the current bright area group, I1, Q1, I2, Q2 are reference multipath correction values ​​of the first dark area group and the second dark area group respectively, and I1=D11-D13, Q1=D12-D14, I2=D21-D23, Q2=D22-D24, and D11~D14 and D21~D24 are four-phase data collected by the image acquisition device respectively.

[0065] In this implementation scenario, the brightness value of the current bright area group is less than or equal to the threshold, indicating that the previous bright area group may be greatly affected by multipath interference. At this time, multiple dark area groups are needed to perform multipath interference correction to ensure the correction accuracy of the bright area group.

[0066] In some other embodiments, the brightness value of the current bright area group may be ignored, and multiple dark area groups may be fixedly configured for each bright area group. The multipath correction coefficients of all dark area groups corresponding to the current bright area group may be determined based on the brightness ratios and calibration ratios between all dark area groups and the current bright area group. The multipath interference amount of the current bright area group may be determined based on the multipath correction coefficients of all dark area groups corresponding to the current bright area group and the benchmark multipath correction amount.

[0067] At step S205, the difference between the reference multipath correction amount of at least one dark area group and the multipath interference amount of the corresponding bright area group can be calculated to complete the correction of the data of the bright area group corresponding to the at least one dark area group. At this point, the correction of the data of the bright area group is completed. Then, the data of the corrected bright area group can be used to perform subsequent phase, depth and other calculations to meet the ranging requirements.

[0068] Therefore, considering the fact that the brightness ratio of the bright area and the dark area in the image will change before and after being interfered by multipath, and that the bright and dark areas in the image are in different positions and are affected by different multipath interference, the bright area of ​​the light spot in the ranging image is subdivided into multiple bright area groups, and each bright area group obtained by grouping is matched with a corresponding dark area group, so as to simulate or simulate the degree of multipath interference of the bright area group in combination with the data of the dark area group, so as to realize the accurate calculation of the multipath interference amount of each bright area group, and realize the accurate correction of the data of the brightness group based on the obtained multipath interference amount. Secondly, by calibrating the brightness ratio of the dark area group to the bright area group in an environment without multipath interference, and applying the data for calibration in the actual measurement scene, the subsequent correction, measurement, etc. are more accurate. In addition, the data of the bright area group and the data of the dark area group are combined for correction, so that the correction result is more accurate.

[0069] In addition, the solution in this embodiment can be applied to ranging scenarios of various structured light sources, and compared with traditional solutions, the entire solution has a small amount of calculation and high calculation accuracy, and is easier to implement in practical applications.

[0070] Figure 5 A schematic block diagram of a depth camera 500 according to an embodiment of the present application is schematically shown. Figure 5 As shown, the depth camera 500 may include a processor 501, a memory 502, a projection end 503, and a receiving end 504. The projection end 503 is configured to project a structured light source for ranging, the receiving end 504 is configured to receive a structured light source for ranging, and the memory 502 stores a computer instruction for ranging based on the structured light source. When the computer instruction is executed by the processor 501, the depth camera 500 executes: obtaining a ranging image obtained in the ranging process based on the structured light source, wherein the ranging image includes a light spot bright area group and a light spot dark area group, and the light spot bright area group includes a plurality of bright area groups, and each bright area group is matched with at least one dark area group in the light spot dark area group, and the multipath interference amount of each bright area group is determined according to the data of each bright area group and the data of the corresponding dark area group; and the data of each bright area group is corrected based on the multipath interference amount of each bright area group, so as to filter out the bright area group data participating in the ranging calculation from the ranging image.

[0071] Through the above implementation, the depth camera can reduce the impact of multipath interference to a certain extent compared with other light sources by using structured light sources for ranging in the process of ranging based on structured light sources. At the same time, considering that the brightness ratio of the bright and dark areas in the image will change before and after being affected by multipath interference, and the bright and dark areas at different positions are affected to different degrees, the light spot bright area group in the ranging image can be grouped, and each bright area group obtained by grouping can be matched with a corresponding dark area group, so as to make full use of the data of the grouped bright area group and the corresponding dark area group to calculate the multipath interference amount, improve the calculation accuracy of the multipath interference amount, and achieve accurate multipath correction, thereby effectively improving the ranging accuracy.

[0072] It should be noted that the specific contents of the method steps of the depth camera operation are combined with the above Figure 1~Figure 4d The specific implementation methods of the described related methods are the same or similar, so they will not be repeated here.

[0073] In addition, the present application also provides an electronic device, which may include Figure 5 It should be noted that the electronic device in this embodiment may include any device that needs to measure distance based on a structured light source (such as a sweeping robot, etc.).

[0074] In addition, the present application also provides a computer-readable storage medium in which program instructions are stored, and the program instructions are configured to be executed at runtime. Figure 1 ~ Figure 2 The ranging method based on structured light source is shown.

[0075] Specifically, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0076] Although multiple embodiments of the present application have been shown and described herein, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art can think of many changes, modifications and alternatives without departing from the thought and spirit of the present application. It should be understood that in the process of practicing the present application, various alternatives to the embodiments of the present application described herein can be adopted. The attached claims are intended to limit the scope of protection of the present application, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A distance measurement method based on structured light source, characterized in that: include: Acquire a ranging image obtained in a ranging process based on a structured light source, wherein the ranging image includes a light spot bright area group and a light spot dark area group, and the light spot bright area group includes a plurality of bright area groups, and each bright area group is matched with at least one dark area group in the light spot dark area group; Determining the multipath interference amount of each bright area group according to the data of each bright area group and the data of its corresponding dark area group; and The data of each bright area group is corrected based on the multipath interference amount of each bright area group, so as to filter out the bright area group data participating in the ranging calculation from the ranging image.

2. The distance measurement method based on structured light source according to claim 1, characterized in that: Determining the multipath interference amount of each bright area group according to the data of each bright area group and the data of its corresponding dark area group comprises: Acquire a brightness ratio and a calibration ratio between the at least one dark area group and the corresponding bright area group; Acquiring a reference multipath correction amount for the at least one dark area group; and The multipath interference amount of the bright area group corresponding to the at least one dark area group is determined based on the acquired brightness ratio, the calibration ratio and the reference multipath correction amount.

3. The distance measurement method based on structured light source according to claim 2, characterized in that: Correcting the data of each bright area group based on the multipath interference amount of each bright area group comprises: The difference between the reference multipath correction amount of the at least one dark area group and the multipath interference amount of the corresponding bright area group is calculated to complete the correction of the data of the bright area group corresponding to the at least one dark area group.

4. The distance measurement method based on structured light source according to claim 2, characterized in that: Acquiring a brightness ratio between the at least one dark area group and the corresponding bright area group includes: Obtaining a mean brightness value of the at least one dark area group; Obtaining a mean value of brightness values ​​of a bright area group corresponding to the at least one dark area group; and A ratio between a mean luminance value of the at least one dark area group and a mean luminance value of the corresponding bright area group is calculated to obtain the luminance ratio.

5. The distance measurement method based on structured light source according to claim 2, characterized in that: Obtaining a calibration ratio between the at least one dark area group and the corresponding bright area group includes: Obtaining a calibration ratio between the at least one dark area group and the corresponding bright area group; Acquire a calibration image formed by the structured light source in an environment without multipath interference, wherein the calibration image includes the multiple bright area groups, and each bright area group is matched with the at least one dark area group; Obtaining a mean value of calibrated brightness values ​​of the at least one dark area group; Obtaining a calibrated brightness value average of the bright area group corresponding to the at least one dark area group; and The ratio between the average calibrated brightness value of the at least one dark area group and the average calibrated brightness value of the corresponding bright area group is calculated to obtain the calibration ratio.

6. The distance measurement method based on structured light source according to claim 2, characterized in that: Determining the multipath interference amount of the bright area group corresponding to the at least one dark area group includes: Determining a multipath correction coefficient of the at least one dark area group according to the acquired brightness ratio and the calibration ratio; and Based on the multipath correction coefficient of the at least one dark area group and the reference multipath correction amount, a multipath interference amount of the bright area group corresponding to the at least one dark area group is determined.

7. The distance measurement method based on structured light source according to claim 6, characterized in that: The at least one dark area group includes a first dark area group and / or a second dark area group, and determining the multipath interference amount of the bright area group corresponding to the at least one dark area group includes: In response to a brightness value of a current bright area group being greater than a threshold, determining a multipath correction coefficient of a first dark area group corresponding to the current bright area group according to a brightness ratio and a calibration ratio between the first dark area group corresponding to the current bright area group and the current bright area group, and determining a multipath interference amount of the current bright area group based on the multipath correction coefficient of the first dark area group corresponding to the current bright area group and a reference multipath correction amount; In response to the brightness value of the current bright area group being less than or equal to a threshold value, the multipath correction coefficients of the first dark area group and the second dark area group corresponding to the current bright area group are determined respectively according to the brightness ratios and calibration ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, and the multipath interference amount of the current bright area group is determined based on the multipath correction coefficients of the first dark area group and the second dark area group corresponding to the current bright area group and a reference multipath correction amount.

8. The distance measurement method based on structured light source according to claim 7, characterized in that: The multipath correction coefficient of the at least one dark area group is determined according to the following formula: Rd1=(R1-RC1) / Rref; Rd2=(R2-RC2) / Rref; Among them, Rd1 and Rd2 are respectively the multipath correction coefficients of the first dark area group and the second dark area group, R1 and R2 are respectively the brightness ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, RC1 and RC2 are respectively the calibration ratios between the first dark area group and the second dark area group corresponding to the current bright area group and the current bright area group, and Rref is a preset constant value.

9. The distance measurement method based on structured light source according to claim 8, characterized in that: The data collected by the image acquisition device of the ranging image is four-phase data. In response to the brightness value of the current bright area group being greater than a threshold, the multipath interference amount of the current bright area group is determined according to the following formula: ΔI= Rd1×I1, ΔQ= Rd1×Q1; or In response to the brightness value of the current bright area group being less than or equal to a threshold, the multipath interference amount of the current bright area group is determined according to the following formula: ΔI= Rd1×I1+ Rd2×I2, ΔQ= Rd1×Q1+ Rd2×Q2; Among them, Rd1 and Rd2 are multipath correction coefficients of the first dark area group and the second dark area group respectively, ΔI and ΔQ are multipath interference amounts of the current bright area group, I1, Q1, I2, Q2 are benchmark multipath correction values ​​of the first dark area group and the second dark area group respectively, and I1=D11-D13, Q1=D12-D14, I2=D21-D23, Q2=D22-D24, D11~D14, D21~D24 are four-phase data collected by the image acquisition device respectively.

10. The distance measurement method based on structured light source according to claim 1, characterized in that: The distance measurement method further comprises: Grouping the light spot bright area groups to obtain the plurality of bright area groups; and A brightness value of each bright area group is acquired, so as to configure at least one dark area group for each bright area group based on the acquired brightness value.

11. The distance measurement method based on structured light source according to claim 10, characterized in that: Wherein configuring at least one dark area group for each bright area group based on the acquired brightness value comprises: In response to the brightness value of each bright area group being greater than a threshold, configuring a dark area group for each bright area group; or In response to the brightness value of each bright area group being less than or equal to a threshold, a plurality of dark area groups are configured for each bright area group; wherein the dark area groups in the light spot dark area group are divided according to their positional relationship with the bright area groups.

12. The distance measurement method based on structured light source according to claim 1, characterized in that: The distance measurement method further comprises: Grouping the light spot bright area groups to obtain the plurality of bright area groups; and A plurality of dark area groups are configured for each bright area group, wherein the dark area groups in the light spot dark area group are divided according to their positional relationship with the bright area group.

13. A depth camera, characterized in that: include: A projection end, configured to project a structured light source for distance measurement; A receiving end configured to receive a structured light source for ranging; A memory storing computer instructions for distance measurement based on structured light sources; A processor that runs the computer instructions so that the depth camera performs the distance measurement method based on structured light source according to any one of claims 1 to 12.

14. An electronic device, characterized in that: include: The depth camera of claim 13.

15. A computer-readable storage medium, characterized in that: The invention comprises program instructions for distance measurement based on structured light source, and when the program instructions are executed by a processor, the distance measurement method based on structured light source according to any one of claims 1 to 12 is implemented.

Images