An adaptive semi-global stereo matching parameter setting method and device

Abstract

The present disclosure relates to an adaptive semi-global stereo matching parameter setting method, device, electronic equipment and storage medium. The method comprises: based on a left camera image of a binocular camera shooting a target, obtaining the coordinates of a preset left target area and calculating the image proportion; calculating the global variance and the local variance; calculating the local inverse luminosity according to the local variance, generating the minimum disparity number based on a preset correspondence relationship; calculating the joint inverse luminosity according to the image proportion, the global variance and the local variance, generating the sum of absolute differences cost calculation window based on a preset correspondence relationship; configuring the minimum disparity number and the sum of absolute differences cost calculation window to complete the adaptive setting of the semi-global stereo matching parameters. By calculating the minimum disparity number and the sum of absolute differences cost calculation window size of the image, the present disclosure can adapt to various lighting conditions and perform autonomous parameter setting under non-human control.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image processing, and in particular, to an adaptive semi-global stereo matching parameter setting method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] Binocular stereo vision obtains the information of three-dimensional world by processing two-dimensional images of binoculars, and is an important technology in computer vision, the core of which is stereo matching of images. Among various stereo matching algorithms, the semi-global stereo matching method is widely used because it can maintain high matching accuracy while consuming less computing time. However, the disparity image calculated by the semi-global stereo matching method is greatly affected by parameter setting, and different parameter settings are required under different light intensity conditions. In actual application, the algorithm is usually mounted on a detection equipment, and manual parameter adjustment cannot be performed in real time. If good matching effect is to be obtained, an adaptive parameter setting method is needed.

[0003] The core of distance measurement using binocular cameras is binocular stereo matching. The semi-global stereo matching (SGBM) algorithm is a stereo matching algorithm that balances speed and accuracy. However, the accuracy of the SGBM algorithm depends on the setting of two most important parameters: the sum of absolute differences (SAD) matching window size and the minimum disparity number. Different parameters are required under different light conditions. Binocular cameras are usually used for real-time shooting. During this process, once the parameters are determined, they cannot be adjusted, which may cause poor stereo matching effect under backlight or other light condition changes.

[0004] Therefore, one or more methods are needed to solve the above problems.

[0005] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The purpose of the present disclosure is to provide an adaptive semi-global stereo matching parameter setting method, device, electronic equipment and computer readable storage medium, thereby at least partially overcoming one or more problems caused by the limitations and defects of related technologies.

[0007] According to one aspect of the present disclosure, an adaptive semi-global stereo matching parameter setting method is provided, comprising:

[0008] The left camera image of the target is captured by a binocular camera, coordinates of a preset left target region in the left camera image are obtained, and a proportion of the left target region in the left camera image is calculated.

[0009] A global normalized gray histogram of the left camera image and a variance of the global normalized gray histogram are calculated, a local normalized gray histogram of the left target region and a variance of the local normalized gray histogram are calculated.

[0010] A local inverse luminance is calculated according to the variance of the local normalized gray histogram, a minimum disparity number is generated based on a preset local inverse luminance and a corresponding relationship of the minimum disparity number, a joint inverse luminance is calculated according to the proportion of the left target region in the left camera image, the variance of the global normalized gray histogram and the variance of the local normalized gray histogram, and an absolute difference sum cost calculation window is generated based on a preset joint inverse luminance and a corresponding relationship of the absolute difference sum cost calculation window.

[0011] The minimum disparity number and the absolute difference sum cost calculation window are configured to complete adaptive setting of a semi-global stereo matching parameter.

[0012] In an exemplary embodiment of the present disclosure, the method further comprises:

[0013] The left camera image F L of the target is captured by a binocular camera, the F L has a width of M pixels and a height of N pixels.

[0014] A preset left target region F P defined in the form of a bounding box in the left camera image is obtained, the F L has a top-left corner vertex as an origin (0, 0), a top-left corner coordinate of the F P is (x1, y1), and a right-bottom corner coordinate is (x2, y2).

[0015] A proportion of the left target region in the left camera image is calculated

[0016]

[0017] In an exemplary embodiment of the present disclosure, the method further comprises:

[0018] A global normalized gray histogram of the left camera image F L is calculated.

[0019]

[0020] wherein p L (r k) is the probability value of the gray level r k in the left camera image F L k n is the discrete gray level, MN is the number of pixels of the gray level r k in the left camera image F k L MN is the total number of pixels in the left camera image F L l-1 is the number of all possible gray levels in the left camera image F

[0021] The mean of the global normalized gray histogram is calculated according to the global normalized gray histogram:

[0022]

[0023] where μ is the mean of the global normalized gray histogram, l-1 is the number of all possible gray levels in the left camera image F L r is the discrete gray level, p(r L ) is the probability value of the gray level r k in the left camera image F k k L

[0024] The variance is calculated according to the mean of the global normalized gray histogram:

[0025]

[0026] where σ is the variance of the global normalized gray histogram, μ is the mean of the global normalized gray histogram, l-1 is the number of all possible gray levels in the left camera image F L 2 r is the discrete gray level, p(r L ) is the probability value of the gray level r L in the left camera image F k k k L

[0027] In an exemplary embodiment of the present disclosure, the method further comprises:

[0028] calculating the local normalized gray histogram of the left target region F P

[0029]

[0030] where p(r P ) is the probability value of the gray level r k in the left target region F k P ​​​​​​​​​​​r k is the number of pixels of the discrete gray level, n k is the number of pixels of the discrete gray level, r k is the number of pixels of the discrete gray level, MN is the left target region F P is the number of pixels of the discrete gray level, l-1 is the left target region F P is the number of pixels of the discrete gray level, n

[0031] According to the local normalized gray histogram, the mean value of the local normalized gray histogram is calculated:

[0032]

[0033] wherein μ P is the mean value of the local normalized gray histogram, l-1 is the left target region F P is the number of pixels of the discrete gray level, n k is the number of pixels of the discrete gray level, p(r k ) is the probability value of the gray level r k in the left target region F P .

[0034] According to the mean value of the local normalized gray histogram, the variance is calculated:

[0035]

[0036] wherein σ P 2 is the variance of the local normalized gray histogram, μ P is the mean value of the local normalized gray histogram, l-1 is the left target region F P is the number of pixels of the discrete gray level, n k is the number of pixels of the discrete gray level, p(r k ) is the probability value of the gray level r k in the left target region F P .

[0037] In an exemplary embodiment of the present disclosure, the method further comprises:

[0038] According to the variance σ P of the local normalized gray histogram, the local inverse luminance is calculated:

[0039]

[0040] The minimum disparity number n is generated based on the correspondence between the preset local inverse luminance and the minimum disparity number. disparities

[0041] In an exemplary embodiment of the present disclosure, the method further comprises:

[0042] ​According to the proportion s of the left target region in the left camera image, the variance σ of the global normalized gray histogram L , the variance σ of the local normalized gray histogram P Calculate the joint inverse luminance:

[0043]

[0044] Generate the absolute difference sum cost calculation window w based on the preset corresponding relationship between the joint inverse luminance and the absolute difference sum cost calculation window SAD .

[0045] In an aspect of the present disclosure, an adaptive semi-global stereo matching parameter setting device is provided, comprising:

[0046] An image preprocessing module is configured to acquire the coordinates of a preset left target region in a left camera image of a binocular camera based on the left camera image of the binocular camera capturing a target, and calculate the proportion of the left target region in the left camera image.

[0047] A normalized gray histogram variance calculation module is configured to calculate a global normalized gray histogram of the left camera image and the variance of the global normalized gray histogram, and calculate a local normalized gray histogram of the left target region and the variance of the local normalized gray histogram.

[0048] A parameter calculation module is configured to calculate a local inverse luminance based on the variance of the local normalized gray histogram, generate a minimum disparity number based on a preset corresponding relationship between the local inverse luminance and the minimum disparity number, calculate a joint inverse luminance based on the proportion of the left target region in the left camera image, the variance of the global normalized gray histogram, and the variance of the local normalized gray histogram, and generate an absolute difference sum cost calculation window based on a preset corresponding relationship between the joint inverse luminance and the absolute difference sum cost calculation window.

[0049] A parameter setting module is configured to configure the minimum disparity number and the absolute difference sum cost calculation window, so as to complete adaptive setting of semi-global stereo matching parameters.

[0050] In an aspect of the present disclosure, an electronic device is provided, comprising:

[0051] A processor; and

[0052] A memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method according to any one of the above.

[0053] In one aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the method according to any one of the above.

[0054] In one of the example embodiments of the present disclosure, an adaptive semi-global stereo matching parameter setting method is provided, which comprises: obtaining the coordinates of a preset left target region based on a left camera image of a target captured by a binocular camera, and calculating an image proportion; calculating a global variance and a local variance; calculating a local inverse light intensity according to the local variance, and generating a minimum disparity number based on a preset correspondence relationship; calculating a joint inverse light intensity according to the image proportion, the global variance and the local variance, and generating an absolute difference sum cost calculation window based on a preset correspondence relationship; and configuring the minimum disparity number and the absolute difference sum cost calculation window to complete adaptive setting of semi-global stereo matching parameters. The minimum disparity number and the absolute difference sum cost calculation window size of the image are calculated, so that in the case of non-human control, adaptive parameter setting can be performed under various lighting conditions.

[0055] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0056] The above and other features and advantages of the present disclosure will become more apparent by describing in detail example embodiments thereof with reference to the attached drawings.

[0057] Figure 1 A flowchart of an adaptive semi-global stereo matching parameter setting method according to an example embodiment of the present disclosure is shown;

[0058] Figure 2 A typical parallel binocular imaging model schematic diagram of an adaptive semi-global stereo matching parameter setting method according to an example embodiment of the present disclosure is shown;

[0059] Figure 3 An input image of adaptive parameter calculation of an adaptive semi-global stereo matching parameter setting method according to an example embodiment of the present disclosure is shown;

[0060] Figure 4 A schematic block diagram of an adaptive semi-global stereo matching parameter setting device according to an example embodiment of the present disclosure is shown;

[0061] Figure 5 A block diagram of an electronic device according to an example embodiment of the present disclosure is shown schematically; and

[0062] Figure 6 A schematic diagram of a computer readable storage medium according to an example embodiment of the present disclosure is shown schematically. DETAILED DESCRIPTION

[0063] Example embodiments now will be described more fully hereinafter with reference to the accompanying drawings. Example embodiments, however, can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example embodiments to those skilled in the art. Like reference numerals refer to like elements throughout the several views.

[0064] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the

[0065] The block diagrams in the drawings show only the functionality of the embodiments and do not imply that the described functionality must be implemented in a specific order. Further, the blocks can represent modules, segments, or portions of an application specific integrated circuit (ASIC) and data processing device. Therefore, the boundaries of these blocks are not necessarily the physical boundaries of the ASIC and data processing device. It is a matter of design choice and engineering predilection to partition a given ASIC and data processing device into different blocks.

[0066] In the present example embodiments, firstly, an adaptive semi-global stereo matching parameter setting method is provided; referring to FIG. 1, the adaptive semi-global stereo matching parameter setting method can include the following steps: Figure 1

[0067] In step S110, based on a left camera image of a target photographed by a binocular camera, coordinates of a preset left target region in the left camera image are obtained, and an occupancy ratio of the left target region in the left camera image is calculated.

[0068] In step S120, a global normalized gray histogram of the left camera image and a variance of the global normalized gray histogram are calculated; a local normalized gray histogram of the left target region and a variance of the local normalized gray histogram are calculated.

[0069] ​In step S130, a local inverse lightness is calculated according to the variance of the local normalized gray histogram, a minimum disparity number is generated based on a preset corresponding relationship between the local inverse lightness and the minimum disparity number, a joint inverse lightness is calculated according to the proportion of the left target region in the left camera image, the variance of the global normalized gray histogram, and the variance of the local normalized gray histogram, and an absolute difference sum cost calculation window is generated based on a preset corresponding relationship between the joint inverse lightness and the absolute difference sum cost calculation window.

[0070] In step S140, the minimum disparity number and the absolute difference sum cost calculation window are configured to complete adaptive setting of the semi-global stereo matching parameters.

[0071] An adaptive semi-global stereo matching parameter setting method in an example embodiment of the present disclosure, wherein the method comprises: obtaining coordinates of a preset left target region and calculating an image proportion based on a left camera image of a target photographed by a binocular camera; calculating a global variance and a local variance; calculating a local inverse lightness according to the local variance, generating a minimum disparity number based on a preset corresponding relationship; calculating a joint inverse lightness according to the image proportion, the global variance, and the local variance, and generating an absolute difference sum cost calculation window based on a preset corresponding relationship; and configuring the minimum disparity number and the absolute difference sum cost calculation window to complete adaptive setting of semi-global stereo matching parameters. The present disclosure calculates the minimum disparity number and the absolute difference sum cost calculation window size of the image, so that in the case of non-human control, adaptive parameter setting can be performed under each lighting condition.

[0072] In the following, the adaptive semi-global stereo matching parameter setting method in the example embodiment will be further described.

[0073] Embodiment one:

[0074] In step S110, coordinates of a preset left target region in a left camera image of a target photographed by a binocular camera can be obtained, and a proportion of the left target region in the left camera image can be calculated.

[0075] In the example embodiment, the method further comprises:

[0076] Based on a left camera image F of a target photographed by a binocular camera, L , the width of F L is M pixels, and the height is N pixels;

[0077] A preset left target region F P defined in the form of a bounding box in the left camera image is obtained, L with the top-left corner vertex of F PThe coordinates of the top left corner are (x1, y1), and the coordinates of the bottom right corner are (x2, y2).

[0078] Calculate the proportion of the left target region in the left camera image.

[0079]

[0080] In step S120, the global normalized grayscale histogram of the left camera image and the variance of the global normalized grayscale histogram can be calculated; the local normalized grayscale histogram of the left target region and the variance of the local normalized grayscale histogram can also be calculated.

[0081] In this example embodiment, the method further includes:

[0082] Calculate the left camera image F L Globally normalized grayscale histogram:

[0083]

[0084] Where, p L (r k ) represents the grayscale level r k In the left-hand camera image F L The probability value of its occurrence, r k For discrete gray levels, n k For r to appear k The number of pixels at each gray level, MN is the image F of the left camera. L Total number of pixels, l-1 is the left camera image F L All possible gray levels;

[0085] Calculate the mean of the globally normalized grayscale histogram based on the globally normalized grayscale histogram:

[0086]

[0087] Where, μ L The mean of the globally normalized grayscale histogram is l, and l-1 is the mean of the left camera image F. L r represents all possible gray levels in the dataset. k For discrete gray levels, p(r) k ) represents the grayscale level r k In the left-hand camera image F L The probability value of its appearance in;

[0088] Calculate the variance based on the mean of the globally normalized grayscale histogram:

[0089]

[0090] Where, σL 2 μ is the variance of the globally normalized grayscale histogram. L The mean of the globally normalized grayscale histogram is l, and l-1 is the mean of the left camera image F. L r represents all possible gray levels in the dataset. k For discrete gray levels, p(r) k ) represents the grayscale level r k In the left-hand camera image F L The probability value of its appearance.

[0091] In this example embodiment, the method further includes:

[0092] Calculate the left target region F P Locally normalized grayscale histogram:

[0093]

[0094] Where, p P (r k ) represents the grayscale level r k In the target area F on the left P The probability value of its occurrence, r k For discrete gray levels, n k For r to appear k The number of pixels at each gray level, MN is the target region F on the left. P Total number of pixels, l-1 is the target area F on the left. P All possible gray levels;

[0095] Calculate the mean of the locally normalized grayscale histogram based on the locally normalized grayscale histogram:

[0096]

[0097] Where, μ P The mean of the locally normalized grayscale histogram is l, and l-1 represents the target region F on the left. P r represents all possible gray levels in the dataset. k For discrete gray levels, p(r) k ) represents the grayscale level r k In the target area F on the left P The probability value of its appearance in;

[0098] Calculate the variance based on the mean of the locally normalized grayscale histogram:

[0099]

[0100] Where, σ P 2 μ is the variance of the locally normalized grayscale histogram. Pis the mean of the local normalized gray histogram, and l-1 is the left target region F P is the number of all possible gray levels, and r k is a discrete gray level, and p(r k ) is the probability value of the gray level r k appearing in the left target region F P .

[0101] In step S130, the local inverse brightness can be calculated according to the variance of the local normalized gray histogram, the minimum disparity number n is generated based on the preset corresponding relationship between the local inverse brightness and the minimum disparity number, and the corresponding relationship should ensure that the set minimum disparity number is greater than 0 and is an integral multiple of 16; the joint inverse brightness is calculated according to the proportion of the left target region in the left camera image, the variance of the global normalized gray histogram, and the variance of the local normalized gray histogram, and the sum of absolute differences cost calculation window w is generated based on the preset corresponding relationship between the joint inverse brightness and the sum of absolute differences cost calculation window, and the corresponding relationship should ensure that the set sum of absolute differences cost calculation window is greater than 5 and is an odd number.

[0102] In the embodiment of the present example, the method further comprises:

[0103] The local inverse brightness is calculated according to the variance σ P of the local normalized gray histogram:

[0104]

[0105] The minimum disparity number n is generated based on the preset corresponding relationship between the local inverse brightness and the minimum disparity number. disparities

[0106] In the embodiment of the present example, the method further comprises:

[0107] The joint inverse brightness is calculated according to the proportion s of the left target region in the left camera image, the variance σ L of the global normalized gray histogram, and the variance σ P of the local normalized gray histogram:

[0108]

[0109] The sum of absolute differences cost calculation window w is generated based on the preset corresponding relationship between the joint inverse brightness and the sum of absolute differences cost calculation window. SAD

[0110] In step S140, the minimum disparity number n and the sum of absolute differences cost calculation window w can be configured to complete the adaptive setting of the semi-global stereo matching parameters.

[0111] ​​In the embodiment of the present example, the method of the present disclosure adaptively adjusts the semi-global stereo matching parameters, which is simple in calculation and strong in real-time performance. By calculating the joint inverse luminance and the local inverse luminance in the image, the corresponding minimum disparity number and the SAD matching window size are calculated according to different inverse luminances, so that in the case of non-human control, the adaptive parameter setting can be performed under different illumination conditions.

[0112] Embodiment two:

[0113] In the embodiment of the present example, the binocular image is captured by the binocular camera conforming to the binocular parallel vision model as shown in the figure, and the left and right camera images F L and F R for binocular stereo matching calculation are obtained after correction. L The imaging planes corresponding to the two cameras are F R and F W , and the projection points of P(X W , Y W , Z L ) on F R and F L are P1 and P2 respectively.

[0114] In the embodiment of the present example, the whole image of the corrected left camera is F P , which is M pixels wide and N pixels high. As shown in the figure, the image includes a target area F L defined in the form of a bounding box. The top-left corner of F P is taken as the origin (0, 0), and the coordinates of the top-left corner of F L are (x1, y1), and the coordinates of the bottom-right corner are (x2, y2). Figure 3 In the embodiment of the present example, the global normalized gray histogram p L of F P and the local normalized gray histogram p P of F k are calculated. The calculation formula of the normalized gray histogram is:

[0115]

[0116]

[0117] In the formula, p(r k ) represents the probability value of the gray level r k appearing in the image, r k is the discrete gray level, n k is the number of pixels appearing in the gray level r L , MN is the total number of image pixels, and l-1 represents the number of all possible gray levels in the image.

[0118] ​In the embodiment of the present example, the mean μ of the global normalized gray histogram is calculated L and the variance and the mean μ of the local normalized gray histogram within the bounding box region P and the variance The mean of the normalized gray histogram is calculated as follows:

[0119]

[0120] where μ is the mean of the normalized gray histogram, l-1 represents the number of all possible gray levels in the image, r k is a discrete gray level, and p(r k ) represents the probability value of the gray level r k in the image.

[0121] The variance of the normalized gray histogram is calculated as follows:

[0122]

[0123] where σ 2 is the variance of the normalized gray histogram, μ is the mean of the normalized gray histogram, l-1 represents the number of all possible gray levels in the image, r k is a discrete gray level, and p(r k ) represents the probability value of the gray level r k in the image.

[0124] In the embodiment of the present example, the percentage of the target region in the global image is calculated, and the calculation formula is as follows:

[0125]

[0126] In the embodiment of the present example, the local inverse brightness DBL P and the joint inverse brightness DBL U are calculated. The calculation formula of the local inverse brightness is as follows:

[0127]

[0128] where DBL is the inverse brightness, and σ 2 is the variance of the normalized gray histogram.

[0129] The calculation formula of the joint inverse brightness is as follows:

[0130]

[0131] In the embodiment of the present example, the minimum disparity number n P is determined according to the image local inverse brightness DBL disparities .

[0132] The correspondence between the local inverse lightness and the minimum disparity number is shown in the following table.

[0133] Local inverse lightness Minimum disparity number DBL P ≤45]]> 16 <![CDATA[45<DBL P ≤61]]> 32 61<DBL P ≤67]]> 48 <![CDATA[67<DBL P ]]> 64

[0134] In the embodiment of the present example, the joint inverse lightness DBL is calculated according to the local inverse lightness DBL U The SAD cost calculation window size w is determined SAD .

[0135] The correspondence between the joint inverse lightness and the SAD cost calculation window size is shown in the following table.

[0136] Joint inverse lightness SAD cost computation window size DBL U ≤50]]> 11 or 13 50 < DBL U ≤ 55 13 55 < DBL U ≤ 61 13 or 15 or 17 61<DBL U ≤67]]> 17 or 21

[0137] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0138] In addition, in the embodiment of the present example, an adaptive semi-global stereo matching parameter setting device is also provided. Referring to FIG. 4, Figure 4 The adaptive semi-global stereo matching parameter setting device 400 can include an image preprocessing module 410, a normalized gray histogram variance calculation module 420, a parameter calculation module 430, and a parameter setting module 440. Wherein:

[0139] The image preprocessing module 410 is configured to acquire the coordinates of a preset left target region in a left camera image of a binocular camera based on the left camera image of the binocular camera shooting a target, and calculate the proportion of the left target region in the left camera image.

[0140] The normalized gray histogram variance calculation module 420 is configured to calculate a global normalized gray histogram of the left camera image and the variance of the global normalized gray histogram, and calculate a local normalized gray histogram of the left target region and the variance of the local normalized gray histogram.

[0141] The parameter calculation module 430 is configured to calculate a local inverse lightness according to the variance of the local normalized gray histogram, generate a minimum disparity number based on a preset correspondence between the local inverse lightness and the minimum disparity number, calculate a joint inverse lightness according to the proportion of the left target region in the left camera image, the variance of the global normalized gray histogram, and the variance of the local normalized gray histogram, and generate an absolute difference sum cost calculation window based on a preset correspondence between the joint inverse lightness and the absolute difference sum cost calculation window.

[0142] The parameter setting module 440 is configured to configure the minimum disparity number and the sum of absolute difference cost calculation window to complete adaptive setting of the semi-global stereo matching parameters.

[0143] The specific details of each of the above-described adaptive semi-global stereo matching parameter setting device modules have been described in detail in the corresponding adaptive semi-global stereo matching parameter setting method, and thus will not be described here again.

[0144] It should be noted that although several modules or units of the adaptive semi-global stereo matching parameter setting device 400 are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into embodied by multiple modules or units.

[0145] In addition, in the exemplary embodiments of the present disclosure, an electronic device capable of implementing the above-described method is also provided.

[0146] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0147] The electronic device 500 according to this embodiment of the present disclosure will be described below with reference to Figure 5 Figure 5 The display electronic device 500 is only an example and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0148] As shown in Figure 5 The electronic device 500 is shown in the form of a general computing device. The components of the electronic device 500 can include, but are not limited to, the above-mentioned at least one processing unit 510, the above-mentioned at least one storage unit 520, a bus 530 connecting different system components (including the storage unit 520 and the processing unit 510), and a display unit 540.

[0149] The storage unit stores program code which can be executed by the processing unit 510, so that the processing unit 510 performs the steps described in the above "Exemplary Method" section according to various exemplary embodiments of the present disclosure. For example, the processing unit 510 can execute the steps described in the above "Exemplary Method" section according to various exemplary embodiments of the present disclosure. Figure 1 ​Steps S110 to S140 shown in FIG. 10.

[0150] The storage unit 520 can include a readable medium in the form of volatile storage such as a random access memory (RAM) 5201 and / or cache memory 5202, and further can include a read-only memory (ROM) 5203.

[0151] The storage unit 520 can also include a program / utility 5204 having a set (at least one) of program modules 5205 that include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, which may

[0152] The bus 530 can represent one or more of several types of bus structures, including a storage bus or bus controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures, and the like.

[0153] The electronic device 500 can also communicate with one or more external devices 570 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with the electronic device 500; and / or one or more devices that enable the electronic device 500 to communicate with one or more other computing devices. Such communication can be facilitated by an Input / Output (I / O) interface 550. Still yet, the electronic device 500 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network such as the Internet, via a network adapter 560. As depicted, the network adapter 560 is in communication with the other components of the electronic device 500 through the bus 530. It should be appreciated that the electronic device 500 can be a part of another device or be a stand-alone device. It also should be appreciated that the electronic device 500 can be connected to another electronic device in a similar or different manner as the network adapter 560 is connected to the electronic device 500. In this regard, the electronic device 500 can be a part of a larger device, such as a part of a computer system or other device that includes a plurality of devices.

[0154] From the above description of the embodiments, those skilled in the art will readily perceive that the example embodiments described herein can be implemented by software and / or by hardware coupled with software. Accordingly, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a plurality of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0155] In exemplary embodiments of the present disclosure, a computer readable storage medium having stored thereon a program product capable of implementing the above-described methods of the specification is also provided. In some possible embodiments, various aspects of the present application can also be implemented in the form of a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the above "Exemplary Methods" section according to various exemplary embodiments of the present application.

[0156] Reference Figure 6 As shown, a program product 600 for implementing the above-described methods according to embodiments of the present application is described, which can take the form of a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in the present document, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus or device.

[0157] The program product can take any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0158] The computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave, in which readable program code is carried. Such propagated data signal can take various forms, including but not limited to electromagnetic signal, optical signal or any suitable combination thereof. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.

[0159] The program code contained on the readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0160] The program code may be implemented in any of various ways, including procedure-based, narrative-based, object-based, and / or architectural-based versions. In procedure-based implementations, the program code is implemented in a series of isolated procedures, which rely primarily on extrinsic (or global) variables to communicate with one another. In narrative-based implementations, the program code is implemented at various levels by an interpreter or virtual machine. In object-based versions, the program code is implemented as a series of objects, which interact to perform the functionality of the present application. In architectural-based versions, the program code is implemented based on a component model, which organizes functionality into components that are interconnected to achieve the functionality of the present application.

[0161] Furthermore, the above-described diagrams are only schematic and are non-limiting. As such, the present application is intended to encompass all changes and modifications of the herein described methods which do not constitute departures from the true spirit and scope of the application. For example, the order of the steps of the methods can be modified, and the use of different programming languages and / or different computer systems can be utilized. Also for example, each of the above-described processes could be performed synchronously or asynchronously.

[0162] Other embodiments of the present disclosure will be apparent to those of ordinary skill in the art from the disclosure hereof. The present application is intended to cover any and all adaptations or variations of various embodiments of the disclosure including its general description. It is intended that the term "including" as used herein will be interpreted to mean "including, but not limited to." Further, it is intended that the term "comprising" as used herein will be interpreted to mean "comprising, but not limited to." It is further intended that certain embodiments of the present disclosure, which consist of combinations of the features of the above described embodiments, fall within the scope of the present application. The true scope and spirit of the present application are indicated by the appended claims.

[0163] It should be understood that the present disclosure is not limited to the precise structures herein described and illustrated, and that various modifications and changes can be made without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims appended hereto.

Claims

1. An adaptive semi-global stereo matching parameter setting method, characterized in that, The method includes: Based on the left camera image of the target captured by the binocular camera, the coordinates of the preset left target region in the left camera image are obtained, and the proportion of the left target region in the left camera image is calculated. Calculate the global normalized grayscale histogram of the left-side camera image and the variance of the global normalized grayscale histogram; calculate the local normalized grayscale histogram of the left-side target region and the variance of the local normalized grayscale histogram. The local backlight intensity is calculated based on the variance of the local normalized grayscale histogram, and the minimum disparity number is generated based on the correspondence between the preset local backlight intensity and the minimum disparity number. Based on the proportion s of the left target region in the left camera image and the variance σ of the global normalized grayscale histogram... L 2 The variance σ of the locally normalized grayscale histogram P 2 Calculate the combined backlight intensity: The absolute difference sum cost calculation window is generated based on the pre-defined correspondence between the joint backlight intensity and the absolute difference sum cost calculation window; Configure the minimum disparity number and the sum of absolute differences cost calculation window to complete the adaptive setting of semi-global stereo matching parameters.

2. The method as described in claim 1, characterized in that, The method further includes: Image F of the target taken by the left camera based on the stereo camera L The F L The width is M L Number of pixels, height is N. L 1 pixel; Obtain the preset left target region F defined by a bounding box in the left camera image. P , with F L The top left vertex is the origin (0,0), F P The coordinates of the top left corner are (x1, y1), and the coordinates of the bottom right corner are (x2, y2). Calculate the proportion of the left target region in the left camera image.

3. The method as described in claim 2, characterized in that, The method further includes: Calculate the left camera image F L Globally normalized grayscale histogram: Where, p L (r k ) represents the grayscale level r k In the left-hand camera image F L The probability value of its occurrence, r k For discrete gray levels, n k For r to appear k Number of pixels at each gray level, M L N L Image F from the left camera L Total number of pixels, l is the left camera image F L All gray levels in the image; Calculate the mean of the globally normalized grayscale histogram based on the globally normalized grayscale histogram: Where, μ L Let l be the mean of the globally normalized grayscale histogram, and let f be the value of the left-side camera image. L All gray levels in r k For discrete gray levels, p L (r k ) represents the grayscale level r k In the left-hand camera image F L The probability value of its appearance in; Calculate the variance based on the mean of the globally normalized grayscale histogram: Where, σ L 2 μ is the variance of the globally normalized grayscale histogram. L Let l be the mean of the globally normalized grayscale histogram, and let f be the value of the left-side camera image. L All gray levels in r k For discrete gray levels, p L (r k ) represents the grayscale level r k In the left-hand camera image F L The probability value of its appearance.

4. The method as described in claim 3, characterized in that, The method further includes: Calculate the left target region F P Locally normalized grayscale histogram: Where, p P (r k ) represents the grayscale level r k In the target area F on the left P The probability value of its occurrence, r k For discrete gray levels, n k For r to appear k Number of pixels at each gray level, M p N p For the target area F on the left P Total number of pixels, l is the target area F on the left P All gray levels in the image; Calculate the mean of the locally normalized grayscale histogram based on the locally normalized grayscale histogram: Where, μ P Let l be the mean of the locally normalized grayscale histogram, and let l be the target region F on the left. P All gray levels in r k For discrete gray levels, p P (r k ) represents the grayscale level r k In the target area F on the left P The probability value of its appearance in; Calculate the variance based on the mean of the locally normalized grayscale histogram: Where, σ P 2 μ is the variance of the locally normalized grayscale histogram. P Let l be the mean of the locally normalized grayscale histogram, and let l be the target region F on the left. P All gray levels in r k For discrete gray levels, p P (r k ) represents the grayscale level r k In the target area F on the left P The probability value of its appearance.

5. The method as described in claim 4, characterized in that, The method further includes: Based on the variance σ of the locally normalized grayscale histogram P 2 Calculate local backlight intensity: The minimum disparity number n is generated based on the pre-defined correspondence between local backlight intensity and minimum disparity number. disparities The corresponding relationship should ensure that the minimum disparity number after setting is an integer multiple of 16 greater than 0.

6. The method as described in claim 5, characterized in that, The method further includes: The absolute difference sum cost calculation window w is generated based on the pre-defined correspondence between the joint backlight intensity and the absolute difference sum cost calculation window. SAD The corresponding relationship should ensure that the sum of absolute differences and cost calculation windows after setting are odd numbers greater than 5.

7. An adaptive semi-global stereo matching parameter setting device, based on the method described in any one of claims 1-6, characterized in that, The device includes: The image preprocessing module is used to obtain the coordinates of a preset left target region in the left camera image based on the left camera image captured by the binocular camera, and to calculate the proportion of the left target region in the left camera image. The normalized grayscale histogram variance calculation module is used to calculate the global normalized grayscale histogram of the left camera image and the variance of the global normalized grayscale histogram; and to calculate the local normalized grayscale histogram of the left target region and the variance of the local normalized grayscale histogram. The parameter calculation module is used to calculate the local backlight intensity based on the variance of the local normalized grayscale histogram, and generate the minimum disparity number based on the preset correspondence between the local backlight intensity and the minimum disparity number; calculate the joint backlight intensity based on the proportion of the left target region in the left camera image, the variance of the global normalized grayscale histogram, and the variance of the local normalized grayscale histogram, and generate the absolute difference sum cost calculation window based on the preset correspondence between the joint backlight intensity and the absolute difference sum cost calculation window; The parameter setting module is used to configure the minimum disparity number and the sum of absolute differences cost calculation window to complete the adaptive setting of the semi-global stereo matching parameters.

8. An electronic device, characterized in that, include Processor; and A memory storing computer-readable instructions that, when executed by the processor, implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the method according to any one of claims 1 to 6 when executed by a processor.

Images