Image processing method and device of depth module, storage medium and depth camera

Abstract

The application discloses a kind of image processing method, device, readable storage medium and depth camera of depth module, the method includes: obtaining the depth image and gray image that the depth module is shot, the pixel position of the depth image and the gray image one-to-one correspondence;Calculate the gray scale value of each pixel point position in the gray image;According to the gray scale value, obtain the image processing area of the depth image;According to the image processing area, obtain the depth value of the corresponding pixel point of the depth image.The application can effectively process the depth image measured, make the depth image formed more smooth, facilitate later use.

Description

Technical Field

[0001] This invention relates to the field of depth image processing technology, and more particularly to image processing methods, apparatus, readable storage media, and depth cameras for depth modules. Background Technology

[0002] A depth module, also known as a TOF (Time of Flight) module, is a measurement device that emits light signals and receives reflected light signals, calculating the distance to an object by calculating the time difference.

[0003] When performing depth distance measurement in the depth module, the calculation unit is based on pixels. The distance calculation for each pixel is completed independently, which is equivalent to multiple measurement units measuring independently. The individual differences of different measurement units will cause differences in the depth distances obtained when measuring the same distance. This results in large jitter in the depth distance between adjacent pixels, making the resulting depth image less smooth and unfavorable for later use of the depth image.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] Therefore, in view of the problem that the depth distance measured by adjacent pixels is large and the resulting depth image is not smooth enough during current depth module measurement, it is necessary to provide an image processing method, device, readable storage medium and depth camera for a depth module, which can effectively process the measured depth image to make the resulting depth image smoother and more convenient for later use.

[0006] To achieve the above objectives, the present invention provides an image processing method for a depth module, the method comprising:

[0007] The depth image and grayscale image captured by the depth module are obtained, and the pixel positions of the depth image and the grayscale image correspond one-to-one;

[0008] Calculate the grayscale value of each pixel in the grayscale image;

[0009] Based on the grayscale values, the image processing area of ​​the depth image is obtained;

[0010] Based on the image processing area, the depth value of the corresponding pixel in the depth image is obtained.

[0011] Optionally, the step of calculating the grayscale value of each pixel in the grayscale image includes:

[0012] Select any pixel in the grayscale image as the first pixel;

[0013] Using the first pixel as the center point, a first selection range is generated based on a preset distance;

[0014] Extract grayscale data from the first selected range and sort the grayscale data by grayscale size;

[0015] The median value of the sorted grayscale data is used as the first grayscale value, and the first grayscale value is used as the grayscale brightness of the first pixel.

[0016] Optionally, the step of obtaining the image processing region of the depth image based on the grayscale value includes:

[0017] Based on the first grayscale value of the first pixel and combined with a preset reference distance, the processing distance of the depth image is calculated.

[0018] Based on the processing distance, an image processing area is generated.

[0019] Optionally, the step of obtaining the depth value of the corresponding pixel in the depth image based on the image processing area includes:

[0020] Obtain depth data within the image processing area, and sort the depth data by depth size;

[0021] The median value of the sorted depth data is the depth value corresponding to the first pixel.

[0022] Optionally, the step of calculating the grayscale value at each pixel location in the grayscale image includes the following prior steps:

[0023] A first expansion distance is set, and the image area of ​​the grayscale image is expanded according to the first expansion distance. The first expansion distance is greater than or equal to the preset distance, so that the first selection range is within the effective grayscale image area.

[0024] Optionally, the step of obtaining the depth value of the corresponding pixel in the depth image based on the image processing area includes the following before:

[0025] A second expansion distance is set, and the depth image area of ​​the depth image is expanded according to the second expansion distance. The second expansion distance is greater than or equal to the processing distance, so that the image processing area is within the effective depth image area.

[0026] Furthermore, to achieve the above objectives, the present invention also provides an image processing apparatus for a depth module, the apparatus comprising:

[0027] The acquisition module is used to acquire the depth image and grayscale image captured by the depth module, wherein the pixel positions of the depth image and the grayscale image correspond one-to-one;

[0028] The calculation module is used to calculate the grayscale value of each pixel in the grayscale image.

[0029] The generation module is used to obtain the image processing area of ​​the depth image based on the grayscale values;

[0030] The processing module is used to obtain the depth value of the corresponding pixel in the depth image based on the image processing area.

[0031] Optionally, the calculation module is further configured to select any pixel in the grayscale image as the first pixel; generate a first selection range with the first pixel as the center point and according to a preset distance; extract grayscale data within the first selection range and sort the grayscale data by grayscale size; obtain the median value of the sorted grayscale data as the first grayscale value, and use the first grayscale value as the grayscale brightness of the first pixel.

[0032] In addition, to achieve the above objectives, the present invention also provides a depth camera, which includes a depth module and a housing, wherein the depth module is disposed within the housing, and the depth module processes the depth image using the image processing method of the depth module described above.

[0033] In addition, to achieve the above objectives, the present invention also provides a readable storage medium storing an image processing program for a depth module, wherein when the image processing program for the depth module is executed by a processor, it implements the steps of the image processing method for the depth module as described above.

[0034] In the technical solution proposed in this invention, a depth module captures a depth image and a grayscale image, wherein the pixel positions of the depth image and the grayscale image are in one-to-one correspondence. The grayscale value corresponding to each pixel in the grayscale image is calculated, and the image processing area of ​​the depth image is calculated based on the magnitude of the grayscale value. Based on the image processing area, the depth value of the corresponding pixel in the depth image is obtained. The higher the grayscale value, the brighter the pixel, and vice versa. In other words, the image processing area of ​​the depth image is determined by the brightness value of the corresponding pixel. Thus, the size of the image processing area can change according to the brightness value of the pixel, effectively ensuring the smoothness of the positions of adjacent pixels and facilitating subsequent use. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the first embodiment of the image processing method for the depth module of the present invention;

[0037] Figure 2 This is a flowchart illustrating the second embodiment of the image processing method for the depth module of the present invention;

[0038] Figure 3 This is a flowchart illustrating the third embodiment of the image processing method for the depth module of the present invention;

[0039] Figure 4 This is a flowchart illustrating the fourth embodiment of the image processing method for the depth module of the present invention;

[0040] Figure 5 This is a flowchart illustrating the fifth embodiment of the image processing method for the depth module of the present invention;

[0041] Figure 6 This is a flowchart illustrating the sixth embodiment of the image processing method for the depth module of the present invention;

[0042] Figure 7 This is a schematic diagram of the depth module ranging device of the present invention;

[0043] Figure 8 This is a schematic diagram of a grayscale image in the image processing method of the depth module of the present invention;

[0044] Figure 9 This is a schematic diagram of the vertical expansion of a grayscale image in the image processing method of the depth module of the present invention;

[0045] Figure 10 This is a schematic diagram of the horizontal expansion of a grayscale image in the image processing method of the depth module of the present invention.

[0046] Explanation of icon numbers:

[0047] 100 Get Module 500 Extended Module 200 Calculation module 600 grayscale image 300 Generate module 610 Extended grayscale image 400 Processing module

[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0051] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0052] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0053] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0054] See Figure 1 As shown in the first embodiment of the present invention, the present invention provides an image processing method for a depth module, the method comprising:

[0055] Step S10: Obtain the depth image and grayscale image captured by the depth module. The pixel positions of the depth image and the grayscale image correspond one-to-one.

[0056] Specifically, the depth module is equipped with a photosensitive component. The depth module emits pulsed light, and the photosensitive component can receive the reflected pulsed light. The pulsed light is infrared laser. The photosensitive component receives the infrared laser and simultaneously generates a depth image and a grayscale image. The depth image and the grayscale image have the same area. It can be understood that the pixel positions of the depth image and the grayscale image are in one-to-one correspondence, that is, the depth image and the grayscale image have the same pixel positions.

[0057] Step S20: Calculate the grayscale value of each pixel in the grayscale image;

[0058] Specifically, the grayscale value corresponding to each pixel can be directly extracted from the grayscale image, or the average grayscale value can be calculated within a certain range with one pixel as the center, or all grayscale values ​​included in the range can be sorted by size and the median value can be selected as the grayscale value. Alternatively, the grayscale value of each pixel position can be set by setting a method. The grayscale value is an integer value between 0 and 255, where 0 represents black and 255 represents white.

[0059] Step S30: Obtain the image processing area of ​​the depth image based on the grayscale values;

[0060] Since grayscale values ​​are equivalent to brightness, the grayscale value is obtained directly based on the brightness of each pixel in the image. Pixels with higher brightness have higher grayscale values, and pixels with lower brightness have lower grayscale values. In other words, the size of the image processing area of ​​the depth image is calculated based on the brightness of the corresponding pixels.

[0061] Step S40: Based on the image processing area, obtain the depth value of the corresponding pixel in the depth image.

[0062] After determining the image processing area, the depth value of the corresponding pixel in the depth image is calculated. This can be done by taking the average depth distance within the image processing area as the depth value, or by sorting all depth distances within the image processing area by size and taking the median value as the depth value of the corresponding pixel.

[0063] In this implementation solution, the depth module captures a depth image and a grayscale image, where the pixel positions of the depth image and the grayscale image are in one-to-one correspondence. The grayscale value corresponding to each pixel in the grayscale image is calculated, and the image processing area of ​​the depth image is calculated based on the magnitude of the grayscale value. According to the image processing area, the depth value of the corresponding pixel in the depth image is obtained. The higher the grayscale value, the brighter the pixel, and vice versa. In other words, the image processing area of ​​the depth image is determined by the brightness value of the corresponding pixel. Thus, the size of the image processing area can change according to the brightness value of the pixel, effectively ensuring the smoothness of the positions of adjacent pixels and facilitating later use.

[0064] In addition, since the size of the image processing area can vary according to the brightness value of the pixels, the area of ​​the image processing area can be reduced around pixels with high brightness to improve the display details of the depth image, while the area of ​​the image processing area can be expanded around pixels with low brightness, making the depth values ​​of adjacent pixels in the depth image closer and the depth image smoother. This ensures both the display details of the depth image and the smoothness of the depth image data.

[0065] See Figure 2 As shown, based on the first embodiment of the present invention, a second embodiment of the present invention is proposed. Step S20, which calculates the grayscale value of each pixel in a grayscale image, includes:

[0066] Step S21: Select any pixel in the grayscale image as the first pixel;

[0067] In this context, the corresponding pixel in the grayscale image is at the same location as the pixel in the depth image, meaning that the first pixel also exists in the depth image.

[0068] Step S22: Using the first pixel as the center point, generate a first selection range based on a preset distance;

[0069] The preset distance can be set by the user or calculated based on the brightness of the first pixel. For example, if the preset distance is 'a', the generated first selection range is a × a square. The first selection range of this invention is not limited to a × a square, but can also be a circle with radius 'a', or other graphic areas that occupy a certain grayscale image area.

[0070] Step S23: Extract grayscale data from the first selected range and sort the grayscale data by grayscale size;

[0071] Specifically, the first selection range includes a portion of grayscale data in the grayscale image. Select a portion or all of the grayscale data, sort the grayscale data from low to high or from high to low, and select the median value as the grayscale value of the corresponding pixel.

[0072] Step S24: The median value of the sorted grayscale data is obtained as the first grayscale value, and the first grayscale value is used as the grayscale brightness of the first pixel.

[0073] For example, within the first selection range of an a×a square, the value of a×a / 2 is selected as the first grayscale value from the sorted grayscale data. Within the first selection range, it is inevitable that some pixels will display poorly or be measured inaccurately. However, poor or inaccurate cases are rare. For example, there may be abnormal black values, i.e., grayscale values ​​of 0. By sorting the grayscale data, a few abnormal values ​​can be eliminated, thus determining the median value as the first grayscale value, ensuring that the grayscale brightness of the first pixel is more accurate.

[0074] By reassigning values ​​to the second pixel, the third pixel, and all other pixels except the first pixel in the grayscale image through steps S21 to S24, the entire grayscale image is reassigned, ensuring that the grayscale brightness of the grayscale image is more accurate.

[0075] Furthermore, when reassigning values ​​to pixels in a grayscale image, the selection range is not limited to each pixel being within the first selection range; a second selection range, a third selection range, etc., can also be generated based on different preset distances.

[0076] See Figure 3 As shown, based on the second embodiment proposed in this invention, a third embodiment of this invention is proposed. Step S30, which obtains the image processing region of the depth image based on grayscale values, includes:

[0077] Step S31: Based on the first grayscale value of the first pixel and combined with the preset reference distance, calculate the processing distance of the depth image.

[0078] The reference distance can be changed by the user or calculated based on the first grayscale value of the first pixel.

[0079] Step S32: Generate the image processing area based on the processing distance.

[0080] For example, the following formula

[0081]

[0082] Where L is the reference distance, N ijP represents the processing distance. ij Let represent the first grayscale value, and i and j represent the pixel positions. For example, in a 640×480 image, i is between 1 and 640, and j is between 1 and 480. The image processing area can then be calculated using the formula above. From the formula, we can see that P... ij The higher the first gray level value, the higher the brightness. N ij The processing distance becomes smaller, P ij The smaller the first grayscale value, the lower the brightness. N ij The processing distance increases, which means that the area of ​​the image processing region can vary according to the grayscale brightness of the corresponding pixel, ensuring data smoothness while improving the display details of the depth image.

[0083] See Figure 4 As shown, based on the third embodiment proposed in this invention, a fourth embodiment of this invention is proposed. Step S40, which obtains the depth value of a pixel corresponding to a depth image based on the image processing area, includes:

[0084] Step S41: Obtain depth data within the image processing area and sort the depth data by depth size;

[0085] The image processing area includes depth data. Some or all of the depth data can be selected and sorted in ascending or descending order. Due to measurement or display imaging issues, inaccuracies may occur in the measured depth data, such as a depth of 0. By sorting the depth data, abnormal data can be sorted at one end or at both ends.

[0086] Step S42: The median value of the sorted depth data is the depth value corresponding to the first pixel.

[0087] By selecting the median value from the sorted depth data as the depth value corresponding to the first pixel, outlier data can be effectively eliminated, ensuring that the calculated depth value is more accurate.

[0088] Similarly, for each pixel in the depth image, calculations are performed according to steps S41 and S42. The second pixel, the third pixel, and so on, until all other pixels except the first pixel are reassigned, are calculated. This completes the reassignment of depth values ​​in the depth image, ensuring that the depth image measurement is more accurate.

[0089] See Figure 5 As shown, based on the second embodiment proposed in this invention, a fifth embodiment of this invention is proposed, which includes the following steps before step S20 of calculating the grayscale value of each pixel in the grayscale image:

[0090] Step S50: Set a first expansion distance, expand the image area of ​​the grayscale image according to the first expansion distance, the first expansion distance is greater than or equal to a preset distance, so that the first selection range is within the effective grayscale image area.

[0091] For example, see [link / reference] Figures 8 to 10 As shown, in a grayscale image 600 that is a W×H shape, when processing the boundary pixels of the grayscale image 600, the first selection range exceeds the boundary of the W×H shape, resulting in the first selection range including invalid grayscale images 600. Therefore, the first expansion distance is set to a, and the W×H shape is expanded outward to obtain an expanded grayscale image 610 (W+2a)×(H+2a). The first selection range is an a×a square. Thus, when selecting the first selection range of pixels at the boundary of the W×H shape, they are all within the effective grayscale image 600 area. Specifically, when expanding the grayscale image 600, the effective image area of ​​the grayscale image 600 is increased by symmetrically expanding outward at a distance a from the grayscale image 600 inside the grayscale image 600, ensuring that the first selection range is within the effective grayscale image 600 area.

[0092] See Figure 6 As shown, based on the fourth embodiment proposed in this invention, a fourth and sixth embodiment of this invention is proposed, which includes the following steps before step S40 of obtaining the depth value of the corresponding pixel in the depth image according to the image processing area:

[0093] Step S60: Set a second expansion distance, expand the depth image area of ​​the depth image according to the second expansion distance, and the second expansion distance is greater than or equal to the processing distance so that the image processing area is within the effective depth image area.

[0094] For example, in a depth image with dimensions W×H, when processing the boundary pixels of the depth image, if the selected image processing area exceeds the W×H boundary, the image processing area will include invalid depth image data. Therefore, a second expansion distance of b is set, and the W×H shape is expanded outwards to obtain an expanded grayscale image of (W+2b)×(H+2b). The image processing area is a b×b square. Thus, when selecting the image processing area for pixels at the boundary of the W×H shape, all pixels are within the valid depth image area. Specifically, when expanding the depth image, the corresponding b-distance from the depth image within the depth image is symmetrically expanded outwards, thereby increasing the effective image area of ​​the depth image and ensuring that the first selection range is within the valid depth image area.

[0095] The present invention also provides an image processing device for a depth module, the image processing device for the depth module comprising: an acquisition module 100, a calculation module 200, a generation module 300, and a processing module 400.

[0096] The acquisition module 100 is used to acquire the depth image and grayscale image captured by the depth module. The pixel positions of the depth image and the grayscale image are in one-to-one correspondence. Specifically, the depth module is equipped with a photosensitive component. The depth module emits pulsed light, and the photosensitive component can receive the reflected pulsed light. The pulsed light is an infrared laser. The photosensitive component includes an infrared sensor. In addition, the photosensitive component receives the infrared laser and simultaneously generates a depth image and a grayscale image. The depth image and the grayscale image have the same area. It can be understood that the pixel positions of the depth image and the grayscale image are in one-to-one correspondence, that is, the depth image and the grayscale image have the same pixel positions.

[0097] The calculation module 200 is used to calculate the grayscale value of each pixel in the grayscale image. Specifically, the grayscale value corresponding to each pixel can be directly extracted from the grayscale image, or the average value of its grayscale values ​​can be calculated within a certain range with one pixel as the center, or all grayscale values ​​included in the range can be sorted by size and the median value can be selected as the grayscale value. Alternatively, the grayscale value of each pixel can be set by a set method. The grayscale value is an integer value between 0 and 255, where 0 represents black and 255 represents white.

[0098] The generation module 300 is used to obtain the image processing area of ​​the depth image based on the grayscale value. Since the grayscale value is equivalent to the brightness, the grayscale value is obtained directly based on the brightness of each pixel in the image. The grayscale value is higher for pixels with higher brightness and lower for pixels with lower brightness. In other words, the size of the image processing area of ​​the depth image is calculated based on the brightness of the corresponding pixel.

[0099] The processing module 400 is used to obtain the depth value of the corresponding pixel in the depth image based on the image processing area. Specifically, after determining the image processing area, the depth value of the corresponding pixel in the depth image can be calculated by taking the average depth distance within the image processing area as the depth value, or by sorting all depth distances within the image processing area by size and taking the median value as the depth value of the corresponding pixel.

[0100] In this implementation solution, the depth module captures a depth image and a grayscale image, where the pixel positions of the depth image and the grayscale image are in one-to-one correspondence. The grayscale value corresponding to each pixel in the grayscale image is calculated, and the image processing area of ​​the depth image is calculated based on the magnitude of the grayscale value. According to the image processing area, the depth value of the corresponding pixel in the depth image is obtained. The higher the grayscale value, the brighter the pixel, and vice versa. In other words, the image processing area of ​​the depth image is determined by the brightness value of the corresponding pixel. Thus, the size of the image processing area can change according to the brightness value of the pixel, effectively ensuring the smoothness of the positions of adjacent pixels and facilitating later use.

[0101] In addition, since the size of the image processing area can vary according to the brightness value of the pixels, the area of ​​the image processing area can be reduced around pixels with high brightness to improve the display details of the depth image, while the area of ​​the image processing area can be expanded around pixels with low brightness, making the depth values ​​of adjacent pixels in the depth image closer and the depth image smoother. This ensures both the display details of the depth image and the smoothness of the depth image data.

[0102] Furthermore, the calculation module 200 is also used to select any pixel in the grayscale image as the first pixel; generate a first selection range with the first pixel as the center point and according to a preset distance; extract grayscale data within the first selection range and sort the grayscale data by grayscale size; obtain the median value of the sorted grayscale data as the first grayscale value, and use the first grayscale value as the grayscale brightness of the first pixel. Specifically, the corresponding pixel in the grayscale image and the same position in the depth image, that is, there is also a first pixel in the depth image. The preset distance can be set by the user or calculated based on the brightness of the first pixel. For example, if the preset distance is 'a', then the generated first selection range is an a×a square. The first selection range of the present invention is not limited to an a×a square, but can also be a circle with a radius of 'a', or other graphic areas occupying a certain area of ​​the grayscale image. The first selection range includes a portion of the grayscale data in the grayscale image. Selecting a portion or all of the grayscale data, sorting the grayscale data according to their numerical values ​​from small to large or from large to small, and selecting the median value as the grayscale value of the corresponding pixel. For example, within a first selection range of an a×a square, the value of a×a / 2 is selected as the first grayscale value from the sorted grayscale data. Within the first selection range, some pixels may inevitably display poorly or be inaccurately measured. However, these poor or inaccurate cases are rare. For example, there may be abnormal black values, i.e., grayscale values ​​of 0. By sorting the grayscale data, a few abnormal values ​​can be eliminated, thus determining the intermediate value as the first grayscale value, ensuring that the grayscale brightness of the first pixel is more accurate. The calculation module 200 is also used to reassign values ​​to all pixels except the first pixel, completing the reassignment of values ​​for the entire grayscale image, ensuring that the grayscale brightness of the grayscale image is more accurate.

[0103] Furthermore, when reassigning values ​​to pixels in a grayscale image, the selection range is not limited to each pixel being within the first selection range; a second selection range, a third selection range, etc., can also be generated based on different preset distances.

[0104] Furthermore, the generation module 300 is also used to calculate the processing distance of the depth image based on the first grayscale value of the first pixel and a preset reference distance; wherein, the reference distance can be changed by the user setting, or the reference distance can be calculated based on the first grayscale value of the first pixel.

[0105] The generation module 300 is also used to generate an image processing area based on the processing distance.

[0106] For example, the following formula

[0107]

[0108] Where L is the reference distance, N ij P represents the processing distance. ij Let represent the first grayscale value, and i and j represent the pixel positions. For example, in a 640×480 image, i is between 1 and 640, and j is between 1 and 480. The image processing area can then be calculated using the formula above. From the formula, we can see that P... ij The higher the first gray level value, the higher the brightness. N ij The processing distance becomes smaller, P ij The smaller the first grayscale value, the lower the brightness. N ij The processing distance increases, which means that the area of ​​the image processing region can vary according to the grayscale brightness of the corresponding pixel, ensuring data smoothness while improving the display details of the depth image.

[0109] Furthermore, the processing module 400 is also used to acquire depth data within the image processing area and sort the depth data by depth size. The image processing area includes depth data, and some or all of the depth data are selected. The selected depth data is sorted in ascending order or descending order. Due to measurement or display imaging reasons, inaccurate depth data may inevitably occur, such as a depth data of 0. By sorting the depth data, abnormal data can be sorted at one end or at both ends.

[0110] The processing module 400 is also used to obtain the median value of the sorted depth data as the depth value corresponding to the first pixel. Selecting the median value from the sorted depth data as the depth value corresponding to the first pixel can effectively eliminate abnormal data and ensure that the calculated depth value is more accurate.

[0111] Similarly, for each pixel in the depth image, the processing module 400 also performs calculations, reassigning values ​​to the second pixel, the third pixel, and so on, up to the remaining pixels excluding the first pixel. After the calculations for the remaining pixels in the depth image are completed, the depth values ​​in the depth image are reassigned, ensuring that the measurement of the depth image is more accurate.

[0112] Furthermore, the image processing device of the depth module includes an expansion module 500, which is used to set a first expansion distance and expand the image area of ​​the grayscale image according to the first expansion distance. The first expansion distance is greater than or equal to a preset distance so that the first selection range is within the effective grayscale image area.

[0113] For example, in a grayscale image with dimensions W×H, when processing the boundary pixels, if the first selection range exceeds the W×H boundary, it includes invalid grayscale images. Therefore, a first expansion distance of 'a' is set, and the W×H image is expanded outwards to obtain an expanded grayscale image of (W+2a)×(H+2a). The first selection range becomes an a×a square. Thus, when selecting pixels at the boundary of the W×H image, the first selection range is within the valid grayscale image area. Specifically, when expanding the grayscale image, the corresponding distance 'a' within the grayscale image is symmetrically expanded outwards, thereby increasing the effective image area and ensuring that the first selection range is within the valid grayscale image area.

[0114] Furthermore, the expansion module 500 is also used to set a second expansion distance, expand the depth image area of ​​the depth image according to the second expansion distance, and the second expansion distance is greater than or equal to the processing distance, so that the image processing area is within the effective depth image area.

[0115] For example, in a depth image with dimensions W×H, when processing the boundary pixels of the depth image, if the selected image processing area exceeds the W×H boundary, the image processing area will include invalid depth image data. Therefore, a second expansion distance of b is set, and the W×H shape is expanded outwards to obtain an expanded grayscale image of (W+2b)×(H+2b). The image processing area is a b×b square. Thus, when selecting the image processing area for pixels at the boundary of the W×H shape, all pixels are within the valid depth image area. Specifically, when expanding the depth image, the corresponding b-distance from the depth image within the depth image is symmetrically expanded outwards, thereby increasing the effective image area of ​​the depth image and ensuring that the first selection range is within the valid depth image area.

[0116] The present invention also provides a depth camera, the depth camera including a depth module and a housing, the depth module being disposed within the housing, and the depth module processing the depth image using the image processing method of the depth module as described above.

[0117] The specific implementation of the depth camera of the present invention can be referred to the various embodiments of the image processing method of the depth module described above, and will not be repeated here.

[0118] The present invention also provides a readable storage medium storing an image processing program for a depth module, wherein when the image processing program for the depth module is executed by a processor, it implements the steps of the image processing method for the depth module as described above.

[0119] Specific embodiments of the readable storage medium of the present invention can be referred to the various embodiments of the image processing method of the depth module described above, and will not be repeated here.

[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0121] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0123] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural transformations made using the contents of the specification and drawings of the present invention under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the scope of patent protection of the present invention.

Claims

1. An image processing method for a depth module, characterized in that, The method includes: The depth image and grayscale image captured by the depth module are obtained, and the pixel positions of the depth image and the grayscale image correspond one-to-one; Calculate the grayscale value of each pixel in the grayscale image; Based on the grayscale values, the image processing area of ​​the depth image is obtained; Based on the image processing area, the depth value of the corresponding pixel in the depth image is obtained; wherein, the step of calculating the grayscale value of each pixel in the grayscale image includes: Select any pixel in the grayscale image as the first pixel; Using the first pixel as the center point, a first selection range is generated based on a preset distance; Extract grayscale data from the first selected range and sort the grayscale data by grayscale size; The step of obtaining the median value of the sorted grayscale data as the first grayscale value, and using the first grayscale value as the grayscale brightness of the first pixel; the step of obtaining the image processing region of the depth image based on the grayscale value includes: Based on the first grayscale value of the first pixel and a preset reference distance, the processing distance of the depth image is calculated according to a preset formula. Based on the processing distance, an image processing area is generated; The formula for the processing distance is as follows: Where L is the reference distance. Represents the processing distance. Let represent the first grayscale value, and i and j represent the pixel positions. For example, in a 640×480 image, i is between 1 and 640, and j is between 1 and 480. The image processing area can then be calculated using the formula above. As can be seen from the formula... The higher the first gray level value, the higher the brightness. The processing distance becomes smaller. The smaller the first gray level value, the lower the brightness. The processing distance then increases.

2. The image processing method for the depth module as described in claim 1, characterized in that, The step of obtaining the depth value of the corresponding pixel in the depth image based on the image processing area includes: Obtain depth data within the image processing area, and sort the depth data by depth size; The median value of the sorted depth data is the depth value corresponding to the first pixel.

3. The image processing method for the depth module as described in claim 1, characterized in that, Before the step of calculating the grayscale value of each pixel in the grayscale image, the following steps are included: A first expansion distance is set, and the image area of ​​the grayscale image is expanded according to the first expansion distance. The first expansion distance is greater than or equal to the preset distance, so that the first selection range is within the effective grayscale image area.

4. The image processing method for the depth module as described in claim 2, characterized in that, Before the step of obtaining the depth value of the corresponding pixel in the depth image based on the image processing area, the following steps are included: A second expansion distance is set, and the depth image area of ​​the depth image is expanded according to the second expansion distance. The second expansion distance is greater than or equal to the processing distance, so that the image processing area is within the effective depth image area.

5. An image processing apparatus for a depth module, characterized in that, The device includes: The acquisition module is used to acquire the depth image and grayscale image captured by the depth module, wherein the pixel positions of the depth image and the grayscale image correspond one-to-one; The calculation module is used to calculate the grayscale value of each pixel in the grayscale image. The generation module is used to obtain the image processing area of ​​the depth image based on the grayscale values; The processing module is used to obtain the depth value of the corresponding pixel in the depth image based on the image processing area; wherein, the calculation module is further used to select any pixel in the grayscale image as the first pixel; generate a first selection range with the first pixel as the center point and according to a preset distance; extract grayscale data within the first selection range; sort the grayscale data by grayscale size; obtain the median value of the sorted grayscale data as the first grayscale value; and use the first grayscale value as the grayscale brightness of the first pixel. The process of obtaining the image processing region of the depth image based on the grayscale value includes: Based on the first grayscale value of the first pixel and a preset reference distance, the processing distance of the depth image is calculated according to a preset formula. Based on the processing distance, an image processing area is generated; The formula for the processing distance is as follows: Where L is the reference distance. Represents the processing distance. Let represent the first grayscale value, and i and j represent the pixel positions. For example, in a 640×480 image, i is between 1 and 640, and j is between 1 and 480. The image processing area can then be calculated using the formula above. As can be seen from the formula... The higher the first gray level value, the higher the brightness. The processing distance becomes smaller. The smaller the first gray level value, the lower the brightness. The processing distance then increases.

6. A depth camera, characterized in that, The depth camera includes a depth module and a housing, the depth module being disposed within the housing, and the depth module processing the depth image using the image processing method of the depth module as described in any one of claims 1 to 4.

7. A readable storage medium, characterized in that, The readable storage medium stores an image processing program for a depth module, which, when executed by a processor, implements the steps of the image processing method for a depth module as described in any one of claims 1 to 4.

Images