Height fusion method and device combined with gradient information, equipment and medium

Abstract

The invention discloses a height fusion method and device combined with gradient information, equipment and a medium, and belongs to the technical field of image processing. The method comprises the following steps: carrying out image height measurement on a to-be-measured object through at least two measuring devices to obtain height images output by the measuring devices; in each height image, acquiring a height value of each pixel point in a neighborhood range of the target pixel point, and performing weighted calculation on the height value of each pixel point in the neighborhood range in combination with gradient information of a pre-acquired standard image to obtain weighted height distribution data of the target pixel point; and carrying out fusion calculation on the weighted height distribution data of the target pixel points in each height image by adopting adaptive gain to obtain a height fusion result of the target pixel points. According to the technical scheme, the weighted calculation is performed by combining the gradient information, and the fusion calculation is performed by adopting the adaptive gain, so that the gain can be dynamically adjusted in the fusion calculation process, and the fusion of the height values is more accurate.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a method, device, equipment and medium for highly fusion combining gradient information. Background Art

[0002] At present, PCB boards are one of the indispensable core components of various smart devices. By performing three-dimensional reconstruction of PCB boards, a data basis can be provided for subsequent PCB board inspection. However, this technology is very sensitive to light signals and often has imaging anomalies. Therefore, multiple measuring devices can be added to measure height values ​​for collaborative imaging to meet inspection needs.

[0003] For the same location, multiple measuring devices can measure multiple height values, and these height values ​​need to be fused. Currently, the Min-Max fusion strategy is generally used, but the Min-Max fusion strategy is separated from the gradient information and has high requirements for the consistency of the height values, resulting in a small scope of application and low accuracy. Therefore, how to use gradient information for height fusion is an urgent problem that people in this field need to solve. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, device, equipment and medium for height fusion combined with gradient information, which can be dynamically adjusted according to the height distribution characteristics of different areas to make the fusion of height values ​​more accurate.

[0005] In a first aspect, an embodiment of the present application provides a high-level fusion method combining gradient information, the method comprising:

[0006] Performing image height measurement of the object to be measured by at least two measuring devices to obtain a height image output by each measuring device;

[0007] In each height image, the height value of each pixel in the neighborhood of the target pixel is obtained, and the height value of each pixel in the neighborhood is weightedly calculated in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction;

[0008] The weighted height distribution data of the target pixel points in each height image are fused and calculated using adaptive gain to obtain the height fusion result of the target pixel points.

[0009] The beneficial effect of this solution is that by combining gradient information for weighted calculation and using adaptive gain for fusion calculation, the gain can be dynamically adjusted during the fusion calculation process, making the fusion of height values ​​more accurate.

[0010] Furthermore, the height value of each pixel point within the neighborhood is weightedly calculated in combination with the gradient information of the pre-acquired standard image to obtain weighted height distribution data of the target pixel point, including:

[0011] Acquiring gradient information of a standard image of the object to be measured;

[0012] Within the neighborhood, determining a weight value of each pixel point according to the gradient information of each pixel point;

[0013] The height value of each pixel point within the neighborhood is weightedly calculated according to the weight value to obtain weighted height distribution data of the target pixel point.

[0014] The beneficial effect of this solution is that by combining the gradient information of the pre-acquired standard image for weighted calculation, the weighted height distribution data of the target pixel point is obtained, which can effectively reduce the calculation deviation of the target pixel point that is exactly at the edge of the object and improve the accuracy of the height distribution curve of the obtained target pixel point.

[0015] Furthermore, within the neighborhood, the weight value of each pixel is determined according to the gradient information of each pixel, including:

[0016] Within the neighborhood, the gradient information of each pixel is substituted into a pre-constructed semantic penalty function to determine the weight value of each pixel.

[0017] The beneficial effect of this solution is that by substituting the gradient information of each pixel into a pre-constructed semantic penalty function to determine the weight value of each pixel, the consistency of the subsequently determined weighted height distribution data and the gradient information can be improved.

[0018] Furthermore, within the neighborhood, the gradient information of each pixel point is substituted into a pre-constructed semantic penalty function to determine the weight value of each pixel point, including:

[0019] The weight value of each pixel is determined according to the difference between the gradient information of each pixel in the neighborhood and the gradient information of the target pixel, and the penalty intensity control parameter.

[0020] The beneficial effect of this scheme is that by determining the weight value of each pixel point based on the difference between the gradient information of each pixel point in the neighborhood range and the gradient information of the target pixel point and the penalty intensity control parameter, the reference of the height value of each pixel point in the neighborhood range for calculating the height value of the target pixel point can be determined, thereby further determining the reference of the height distribution data of the height image in the neighborhood range of the target pixel point for calculating the height value of the target pixel point.

[0021] Furthermore, the weight value of each pixel is determined according to the difference between the gradient information of each pixel in the neighborhood and the gradient information of the target pixel, and the penalty intensity control parameter, including:

[0022] The calculation is done using the following formula:

[0023]

[0024] Among them, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, g(v ij ) is the gradient information of the pixel point in the i-th row and j-th column within the neighborhood, g(v c ) is the gradient information of the target pixel, and α is the penalty intensity control parameter.

[0025] The beneficial effect of this scheme is that is an exponential penalty function. By adopting the exponential penalty function, a smoother penalty effect can be produced.

[0026] Furthermore, the height values ​​of each pixel point within the neighborhood are weighted according to the weight value to obtain weighted height distribution data of the target pixel point, including:

[0027] According to the weight value of each pixel point in the neighborhood range and the height value of each pixel point in the neighborhood range, a weighted average calculation is performed to obtain the mean value of the weighted height distribution data of the target pixel point;

[0028] According to the weight value of each pixel point in the neighborhood range and the difference between the height value of each pixel point in the neighborhood range and the mean value, weighted calculation and square root operation are performed to obtain the variance of the weighted height distribution data of the target pixel point.

[0029] The beneficial effect of this solution is that by calculating the mean and variance of the weighted height distribution data, a data basis can be provided for the subsequent calculation of the size of the adaptive gain.

[0030] Furthermore, the following formula is used to calculate the mean and variance of the weighted height distribution data:

[0031]

[0032]

[0033] Among them, μ is the mean of the weighted height distribution data of the target pixel point, σ is the variance of the weighted height distribution data of the target pixel point, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, a ij is the height value of the pixel in the i-th row and j-th column within the neighborhood.

[0034] The beneficial effect of this solution is that by calculating the mean and variance of the weighted height distribution data, a data basis can be provided for the subsequent calculation of the size of the adaptive gain.

[0035] Furthermore, the weighted height distribution data of the target pixel points in each height image are fused and calculated using adaptive gain to obtain the height fusion result of the target pixel points, including:

[0036] Obtain weighted height distribution data of target pixel points in each height image;

[0037] Determining the size of the adaptive gain according to the weighted height distribution data;

[0038] The adaptive gain is used to perform fusion calculation on the weighted height distribution data of the target pixel point in each height image to obtain a height fusion result.

[0039] The beneficial effect of this solution is that by using adaptive gain to perform fusion calculation on the weighted height distribution data of the target pixel points in each height image, the height fusion result of the target pixel points is obtained. The height values ​​collected by multiple measuring devices can be fused to achieve complementary advantages between different measuring devices and obtain a height value close to the true value.

[0040] Further, determining the size of the adaptive gain according to the weighted height distribution data includes:

[0041] The size of the adaptive gain is determined according to the variance in the weighted height distribution data.

[0042] The beneficial effect of this scheme is that since the variance σ can represent the degree of discreteness of each pixel in the neighborhood, that is, it can represent the importance or influence of the weighted height distribution data on the height fusion result, therefore, determining the size of the adaptive gain according to the variance in the weighted height distribution data can improve the accuracy of the height fusion result.

[0043] Further, determining the size of the adaptive gain according to the variance in the weighted height distribution data includes:

[0044] The variance of the weighted height distribution data of the target pixel point in the two height images is squared and summed to obtain the square sum data;

[0045] According to the variance of the weighted height distribution data of the target pixel in the two height images, the squares are calculated respectively, the proportion of the calculated square of the variance in the square sum data is determined, and the size of the adaptive gain of the two height images is determined.

[0046] The beneficial effect of this solution is that by controlling the proportion of the two weighted height distribution data in the fusion process according to the variance in the two weighted height distribution data, the accuracy of the height fusion result can be improved.

[0047] Further, the adaptive gain is determined by the following formula;

[0048]

[0049]

[0050] Among them, σ p represents the variance of the weighted height distribution data of the target pixel in the pth height image, σ q represents the variance of the weighted height distribution data of the target pixel in the qth height image, k pq k represents the size of the adaptive gain of the target pixel of the qth height image when the pth height image is fused with the qth height image; qp It represents the size of the adaptive gain of the target pixel of the p-th height image when the p-th height image is fused with the q-th height image.

[0051] The beneficial effect of this scheme is that if σ p Compared to σ q is smaller, it means that the p-th height image is more accurate than the q-th height image, and σ p Compared to σ q Smaller leads to k qp Compared to k pq is larger, that is, when the p-th height image and the q-th height image are fused, the p-th height image accounts for a larger proportion, which increases the proportion of the height image with higher accuracy, thereby improving the accuracy of the height fusion result; if σ p Compared to σ q is larger, it means that the accuracy of the p-th height image is lower than that of the q-th height image, and σ p Compared to σ q Greater leads to k qp Compared to k pq Smaller, that is, when the p-th height image is fused with the q-th height image, the proportion of the p-th height image is smaller, reducing the proportion of height images with lower accuracy, thereby improving the accuracy of the height fusion result.

[0052] Furthermore, the adaptive gain is used to perform fusion calculation on the weighted height distribution data of the target pixel in each height image to obtain a height fusion result, including:

[0053] Taking the weighted height distribution data of the target pixel points in each height image as the basic operator, using the adaptive gain to perform fusion calculation on any two basic operators to obtain an intermediate operator;

[0054] Repeat the fusion calculation for all basic operators and intermediate operators until a final fusion result is obtained;

[0055] The mean value in the final fusion result is taken as the height fusion result of the target pixel.

[0056] The beneficial effect of this scheme is that by using adaptive gain to perform fusion calculation to obtain an intermediate operator, and fusing the basic operator with the intermediate operator again until the final fusion result is obtained, the contribution of specific weighted height distribution data to the fusion result can be more emphasized or weakened in the calculation fusion process, thereby improving the robustness of the calculation fusion process and the accuracy of the fusion result.

[0057] In a second aspect, an embodiment of the present application provides a highly fused device combining gradient information, the device comprising:

[0058] A height image acquisition module is used to perform image height measurement of the object to be measured by at least two measuring devices to obtain a height image output by each measuring device;

[0059] A weighted height distribution data calculation module is used to obtain the height value of each pixel in the neighborhood of the target pixel in each height image, and perform weighted calculation on the height value of each pixel in the neighborhood in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction;

[0060] The fusion module is used to perform fusion calculation on the weighted height distribution data of the target pixel points in each height image using adaptive gain to obtain the height fusion result of the target pixel points.

[0061] The beneficial effect of this solution is that by combining gradient information for weighted calculation and using adaptive gain for fusion calculation, the gain can be dynamically adjusted during the fusion calculation process, making the fusion of height values ​​more accurate.

[0062] Furthermore, the weighted height distribution data calculation module includes:

[0063] A gradient information acquisition unit, used to acquire the gradient information of the standard image of the object to be measured;

[0064] A pixel weight determination unit, used to determine the weight value of each pixel point within the neighborhood according to the gradient information of each pixel point;

[0065] The weighted height determination unit is used to perform weighted calculation on the height value of each pixel point within the neighborhood according to the weight value to obtain weighted height distribution data of the target pixel point.

[0066] The beneficial effect of this solution is that by combining the gradient information of the pre-acquired standard image for weighted calculation, the weighted height distribution data of the target pixel point is obtained, which can effectively reduce the calculation deviation of the target pixel point that is exactly at the edge of the object and improve the accuracy of the height distribution curve of the obtained target pixel point.

[0067] Furthermore, the pixel weight determination unit is specifically used to:

[0068] Within the neighborhood, the gradient information of the pixel points is substituted into a pre-constructed semantic penalty function to determine the weight value of each pixel point.

[0069] The beneficial effect of this solution is that by substituting the gradient information of each pixel into a pre-constructed semantic penalty function to determine the weight value of each pixel, the consistency of the subsequently determined weighted height distribution data and the gradient information can be improved.

[0070] Furthermore, the pixel weight determination unit is specifically used to:

[0071] The weight value of each pixel is determined according to the difference between the gradient information of each pixel in the neighborhood and the gradient information of the target pixel, and the penalty intensity control parameter.

[0072] The beneficial effect of this scheme is that by determining the weight value of each pixel point based on the difference between the gradient information of each pixel point in the neighborhood range and the gradient information of the target pixel point and the penalty intensity control parameter, the reference of the height value of each pixel point in the neighborhood range for calculating the height value of the target pixel point can be determined, thereby further determining the reference of the height distribution data of the height image in the neighborhood range of the target pixel point for calculating the height value of the target pixel point.

[0073] Furthermore, the pixel weight determination unit is specifically used to:

[0074] The calculation is done using the following formula:

[0075]

[0076] Among them, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, g(v ij ) is the gradient information of the pixel point in the i-th row and j-th column within the neighborhood, g(v c ) is the gradient information of the target pixel, and α is the penalty intensity control parameter.

[0077] The beneficial effect of this scheme is that is an exponential penalty function (the general formula of the exponential penalty function is: ), by adopting the exponential penalty function, a smoother penalty effect can be produced.

[0078] Furthermore, the weighted height determination unit is specifically used to:

[0079] According to the weight value of each pixel point in the neighborhood range and the height value of each pixel point in the neighborhood range, a weighted average calculation is performed to obtain the mean value of the weighted height distribution data of the target pixel point;

[0080] According to the weight value of each pixel point in the neighborhood range and the difference between the height value of each pixel point in the neighborhood range and the mean value, weighted calculation and square root operation are performed to obtain the variance of the weighted height distribution data of the target pixel point.

[0081] The beneficial effect of this solution is that by calculating the mean and variance of the weighted height distribution data, a data basis can be provided for the subsequent calculation of the size of the adaptive gain.

[0082] Furthermore, the weighted height determination unit is specifically used to:

[0083] The mean and variance of weighted height distribution data are calculated using the following formula:

[0084]

[0085]

[0086] Among them, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, a ij is the height value of the pixel in the i-th row and j-th column within the neighborhood, μ is the mean of the weighted height distribution data of the target pixel, and σ is the variance of the weighted height distribution data of the target pixel.

[0087] The beneficial effect of this solution is that by calculating the mean and variance of the weighted height distribution data, a data basis can be provided for the subsequent calculation of the size of the adaptive gain.

[0088] Furthermore, the fusion module includes:

[0089] A weighted height acquisition unit is used to acquire weighted height distribution data of target pixel points in each height image;

[0090] An adaptive gain determination unit, used to determine the size of the adaptive gain according to the weighted height distribution data;

[0091] The fusion result determination unit is used to use the adaptive gain to perform fusion calculation on the weighted height distribution data of the target pixel point in each height image to obtain a height fusion result.

[0092] The beneficial effect of this solution is that by using adaptive gain to perform fusion calculation on the weighted height distribution data of the target pixel points in each height image, the height fusion result of the target pixel points is obtained. The height values ​​collected by multiple measuring devices can be fused to achieve complementary advantages between different measuring devices and obtain a height value close to the actual height value.

[0093] Furthermore, the adaptive gain determination unit is specifically used to:

[0094] The size of the adaptive gain is determined according to the variance in the weighted height distribution data.

[0095] The beneficial effect of this scheme is that since the variance σ can represent the degree of discreteness of each pixel in the neighborhood, that is, it can represent the importance or influence of the weighted height distribution data on the height fusion result, therefore, determining the size of the adaptive gain according to the variance in the weighted height distribution data can improve the accuracy of the height fusion result.

[0096] Furthermore, the adaptive gain determination unit is specifically used to:

[0097] The variance of the weighted height distribution data of the target pixel point in the two height images is squared and summed to obtain the square sum data;

[0098] According to the variance of the weighted height distribution data of the target pixel in the two height images, the squares are calculated respectively, the proportion of the calculated square of the variance in the square sum data is determined, and the size of the adaptive gain of the two height images is determined.

[0099] The beneficial effect of this solution is that by controlling the proportion of the two weighted height distribution data in the fusion process according to the variance in the two weighted height distribution data, the accuracy of the height fusion result can be improved.

[0100] Furthermore, the adaptive gain determination unit is specifically used to:

[0101] The adaptive gain is determined using the following formula;

[0102]

[0103]

[0104] Among them, σ p represents the variance of the weighted height distribution data of the target pixel in the pth height image, σq represents the variance of the weighted height distribution data of the target pixel in the qth height image, k pq k represents the size of the adaptive gain of the target pixel of the qth height image when the pth height image is fused with the qth height image; qp It represents the size of the adaptive gain of the target pixel of the p-th height image when the p-th height image is fused with the q-th height image.

[0105] The beneficial effect of this scheme is that if σ p Compared to σ q is smaller, it means that the p-th height image is more accurate than the q-th height image, and σ p Compared to σ q Smaller leads to k qp Compared to k pq is larger, that is, when the p-th height image and the q-th height image are fused, the p-th height image accounts for a larger proportion, which increases the proportion of the height image with higher accuracy, thereby improving the accuracy of the height fusion result; if σ p Compared to σ q is larger, it means that the accuracy of the p-th height image is lower than that of the q-th height image, and σ p Compared to σ q Greater leads to k qp Compared to k pq Smaller, that is, when the p-th height image is fused with the q-th height image, the proportion of the p-th height image is smaller, reducing the proportion of height images with lower accuracy, thereby improving the accuracy of the height fusion result.

[0106] Furthermore, the fusion result determination unit is specifically used to:

[0107] Taking the weighted height distribution data of the target pixel points in each height image as the basic operator, using the adaptive gain to perform fusion calculation on any two basic operators to obtain an intermediate operator;

[0108] Repeat the fusion calculation for all basic operators and intermediate operators until a final fusion result is obtained;

[0109] The mean value in the final fusion result is taken as the height fusion result of the target pixel.

[0110] The beneficial effect of this scheme is that by using adaptive gain to perform fusion calculation to obtain an intermediate operator, and fusing the basic operator with the intermediate operator again until the final fusion result is obtained, the contribution of specific weighted height distribution data to the fusion result can be more emphasized or weakened in the calculation fusion process, thereby improving the robustness of the calculation fusion process and the accuracy of the fusion result.

[0111] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0112] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0113] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0114] In an embodiment of the present application, at least two measuring devices are used to measure the image height of the object to be measured, and a height image output by each measuring device is obtained; in each height image, the height value of each pixel point within the neighborhood range of the target pixel point is obtained, and the height value of each pixel point within the neighborhood range is weightedly calculated in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel point; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction;

[0115] The weighted height distribution data of the target pixel points in each height image are fused and calculated using adaptive gain to obtain the height fusion result of the target pixel points. The technical solution provided in the embodiment of the present application combines gradient information for weighted calculation and uses adaptive gain for fusion calculation, and can dynamically adjust the gain during the fusion calculation process, so that the fusion of the height value is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] Figure 1 It is a flowchart of a high-level fusion method combining gradient information provided in an embodiment of the present application;

[0117] Figure 2 This is an example diagram of the placement of the measuring device provided in the embodiment of the present application;

[0118] Figure 3 It is a flowchart of a high-level fusion method combining gradient information provided in an embodiment of the present application;

[0119] Figure 4 This is an example diagram of a highly fused result combining gradient information provided in an embodiment of the present application;

[0120] Figure 5is an example diagram of the semantic penalty function provided in an embodiment of the present application;

[0121] Figure 6 It is a flowchart of a high-level fusion method combining gradient information provided in an embodiment of the present application;

[0122] Figure 7 It is a schematic diagram of the structure of a highly fused device combining gradient information provided in an embodiment of the present application;

[0123] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0124] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all of the content is shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0125] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0126] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0127] In conjunction with the accompanying drawings, the highly fused method, device, equipment and medium combined with gradient information provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0128] In the prior art, the Min-Max fusion strategy is generally used to fuse multiple height values ​​of a pixel point collected. The specific rules of the Min-Max fusion strategy are as follows:

[0129] If there is a measuring device that outputs a height value h1, then h1 is determined to be the final height fusion result;

[0130] If there are two measuring devices that output altitude values ​​h1 and h2 respectively, if the absolute value of the difference between h1 and h2 exceeds the set threshold, h1 and h2 are determined to be invalid altitude values; if the absolute value of the difference between h1 and h2 does not exceed the set threshold, a relatively smaller altitude value is determined as the final altitude fusion result;

[0131] If there are three measuring devices that output altitude values ​​h1, h2 and h3 respectively, then determine the maximum and minimum values ​​among h1, h2 and h3. If the difference between the maximum and minimum values ​​exceeds the set threshold, then determine h1, h2 and h3 as invalid altitude values; if the difference between the maximum and minimum values ​​does not exceed the set threshold, then determine the middle value among h1, h2 and h3 as the final altitude fusion result;

[0132] If there are more than three measuring devices that output height values ​​(h1, h2, h3, ...), if the difference between the maximum and minimum values ​​of each height value exceeds the set threshold, each height value is determined to be an invalid height value; if the difference between the maximum and minimum values ​​does not exceed the set threshold, the average value of each height value is determined as the final height fusion result.

[0133] It can be seen that the Min-Max fusion strategy has high requirements for the consistency of the collected height values. It solves the height pixel by pixel without gradient information, which has low accuracy and small scope of application.

[0134] In response to the problems existing in the Min-Max fusion strategy, this application performs weighted calculation on the surrounding height information of each pixel based on the gradient information of the standard image, and adaptively optimizes the fusion gain during the height fusion process to improve the accuracy of the height fusion result, so that the fused height is close to the actual height of the pixel.

[0135] Figure 1 is a flow chart of a high-level fusion method combining gradient information provided in an embodiment of the present application. Figure 1 As shown, the specific steps include:

[0136] S101, performing image height measurement of an object to be measured by at least two measuring devices to obtain a height image output by each measuring device;

[0137] First, the application scenario of this solution can be a scenario of three-dimensional reconstruction of the object to be measured. Based on the above application scenario, it can be understood that the execution subject of this solution can be an intelligent terminal with image segmentation, image recognition and data processing capabilities, such as: mobile phones, tablet computers and desktop computers, etc., which are not limited here.

[0138] The measuring device may be a camera, a sensor, etc. used in structured light imaging technology to capture structured light projected onto an object.

[0139] The object to be tested may refer to an object that needs to be 3D reconstructed. As an example, the object to be tested in this solution may refer to a PCB (Printed Circuit Board), which is a basic circuit board for supporting and connecting electronic components.

[0140] The height image may refer to an image in which each pixel includes height value information.

[0141] In one embodiment, the image height measurement is performed to obtain the height image output by each measuring device, and structured light imaging technology, laser ranging technology, radar ranging technology, etc. Among them, structured light imaging technology is the mainstream technology used for three-dimensional reconstruction of the PCB board to be measured. Structured light imaging technology projects structured light sources and uses cameras, sensors, etc. to capture the deformation information of the object surface, thereby realizing three-dimensional reconstruction of the object.

[0142] Figure 2 This is an example diagram of the measurement equipment placement measurement provided by the embodiment of the present application. Figure 2 As shown, there are three measuring devices, namely measuring device 1, measuring device 2 and measuring device 3, and each measuring device can output a height image.

[0143] S102, in each height image, obtaining the height value of each pixel in the neighborhood of the target pixel, and performing weighted calculation on the height value of each pixel in the neighborhood in combination with the gradient information of the pre-acquired standard image to obtain weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction;

[0144] The standard image may refer to a color image containing the object to be tested. Specifically, a color image is an image containing information of three channels, namely red, green and blue, and presents rich colors and details by combining the brightness values ​​of the three color channels.

[0145] The camera is a device for capturing images and videos; the preset direction may refer to a preset placement direction of the camera relative to the object to be measured, which may include directly above and / or directly below the object to be measured. Specifically, if the object to be measured is a PCB board, the camera is generally placed directly above the PCB board and faces the PCB board.

[0146] Among them, the pixel point may refer to each pixel point in the height image, and the height value of the pixel point may refer to the height value of the pixel point corresponding to the specific position on the object to be measured in the height image. For example, the object to be measured is a PCB board, and the height values ​​of the positions where no electronic components are arranged may be the same, for example, all are 0. At the welding resistor, due to the different height values ​​of each point on the top surface of the resistor, there are different corresponding height values, which are written into the height image.

[0147] Furthermore, by calibrating according to the external and internal parameters of the measuring device and the camera, the correspondence between each pixel in the height image and each pixel in the standard image can be determined; after the height image is output, the height value of each pixel in the standard image can be determined according to the correspondence between each pixel in the height image and each pixel in the standard image.

[0148] Among them, if the preset direction of the camera is directly above the PCB board and facing the PCB board, the pixels in the height image correspond one-to-one with the pixels in the standard image; however, in one case, if the resolution of the standard image is different from the resolution of the height image, the one-to-one correspondence of the pixels cannot be achieved. The resolution of the standard image can be adjusted to make the number of pixels in the standard image consistent with the number of pixels in the height image, thereby making the pixels in the height image correspond one-to-one with the pixels in the standard image.

[0149] The target pixel point may refer to a pixel point in the standard image that has at least two valid height values. Figure 2 As shown, since the measuring device is generally located on the side and above the object to be measured, the measuring device cannot obtain the height value of the back side of the object to be measured (that is, the object to be measured cannot reflect structured light), so there will be a situation where although there are at least two measuring devices outputting height images, there is only one valid height value for a certain pixel point; if a pixel point has only one valid height value, the pixel point does not need to be highly fused and the final height value is directly output, so the pixel point may not be used as the target pixel point for subsequent height fusion in this scheme.

[0150] In a height image, for pixels whose height values ​​cannot be obtained by the current measuring device, the height value of the pixel can be assigned a value of negative infinity to indicate that the height value is an invalid height value.

[0151] The neighborhood range may refer to a square pixel area consisting of n×n (n is a positive integer) pixels selected with the target pixel as the center point. As an example, n in this solution may be 3, 5, 7, etc. For example, the neighborhood range may be a square pixel area consisting of 5×5 pixels with the target pixel as the center point.

[0152] In one embodiment, the method for obtaining the height value of each pixel in the neighborhood range of the target pixel point is to determine the neighborhood range and use it as a reading window to read the height value information corresponding to each pixel in the neighborhood range.

[0153] Among them, the gradient information may refer to the gradient value of each pixel in the standard image. If the gradient value of a pixel point in the standard image is larger, it means that the height difference between the pixel point and the adjacent pixel points in a certain direction is larger, and further indicates that the specific position of the object to be measured corresponding to the pixel point is on the edge of the object to be measured. Based on the gradient value of each pixel point, each pixel point in the standard image can be divided to determine the specific object corresponding to each pixel point on the object to be measured.

[0154] Specifically, taking a resistor component on a PCB board as an example, the height values ​​of various positions of the resistor component are generally the same, and the height value of the middle part of the edge of the resistor component is significantly different from the height value of the edge part of the resistor component, resulting in the gradient value of the pixel point corresponding to the edge position of the resistor component in the standard image being not 0; the height values ​​of various objects on the PCB board are generally different, and accordingly, the gradient values ​​of the pixel points corresponding to the edge positions of various objects in the standard image are also different. The gradient value corresponding to each object can be obtained by shooting a standard image of a standard PCB board and performing data annotation; the pixel area enclosed by pixel points having a gradient value consistent with the gradient value corresponding to the resistor component corresponds to the entire resistor component.

[0155] It can be understood that the gradient information also reflects the semantic information of each pixel in the standard image to a certain extent. The semantic information describes whether the pixel belongs to the PCB substrate, or to a resistor component, a capacitor component, a chip component, etc.

[0156] Among them, weighted calculation can refer to the process of determining the weight value of each pixel point in the neighborhood range, and determining the mean and variance of the height values ​​of each pixel point in the neighborhood range according to the weight value.

[0157] The weighted height distribution data of the target pixel point may refer to a distribution curve of the height values ​​of each pixel point in a height image within the neighborhood range in the standard image.

[0158] In one embodiment, the weighted height distribution data of the target pixel point can be obtained by acquiring the gradient information of the standard image of the object to be measured, determining the weight value of each pixel point within the neighborhood according to the gradient information of each pixel point, and performing weighted calculation on the height value of each pixel point within the neighborhood according to the weight value to obtain the weighted height distribution data of the target pixel point.

[0159] S103, performing fusion calculation on the weighted height distribution data of the target pixel points in each height image using an adaptive gain to obtain a height fusion result of the target pixel points.

[0160] Among them, using adaptive gain for fusion calculation can be a process of automatically adjusting the fusion gain of weighted height distribution data of each target pixel point in the process of fusion calculation of the height values ​​of target pixel points of different target images to obtain the final output height value of the target pixel point.

[0161] In one embodiment, the method of obtaining the height fusion result of the target pixel point can be to obtain the weighted height distribution data of the target pixel point in each height image, determine the size of the adaptive gain according to the weighted height distribution data, and use the adaptive gain to perform fusion calculation on the weighted height distribution data of the target pixel point in each height image to obtain the height fusion result. For example, if the height value of the same pixel point in height image 1 is h1, and the height value in height image 2 is h2, then a gain value e can be determined, and e×h1 and (1-e)×h2 are added to obtain the height fusion result of the target pixel point.

[0162] The technical solution provided in the embodiment of the present application is to measure the image height of the object to be measured by at least two measuring devices to obtain a height image output by each measuring device; in each height image, the height value of each pixel in the neighborhood of the target pixel is obtained, and the height value of each pixel in the neighborhood is weighted calculated in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction; the weighted height distribution data of the target pixel in each height image is fused and calculated using adaptive gain to obtain the height fusion result of the target pixel. The above-mentioned height fusion method combining gradient information, by combining gradient information for weighted calculation and using adaptive gain for fusion calculation, can dynamically adjust the gain during the fusion calculation process to make the fusion of height values ​​more accurate.

[0163] Figure 3 is a flow chart of a high-level fusion method combining gradient information provided in an embodiment of the present application. Figure 3 As shown, the specific steps include:

[0164] S301, performing image height measurement of an object to be measured by at least two measuring devices to obtain a height image output by each measuring device;

[0165] S302, in each height image, obtaining the height value of each pixel point in the neighborhood of the target pixel point;

[0166] S303, obtaining gradient information of the standard image of the object to be measured;

[0167] Figure 4 is an example diagram of a highly fused result combining gradient information provided by an embodiment of the present application. Figure 4 As shown, (1) is a standard image of the object to be measured, (2) is the gradient information of the standard image, and (3) is a height fusion result of the object to be measured calculated based on the gradient information.

[0168] In one embodiment, the method for obtaining the gradient information of the standard image can be to convert the standard image into a target space image; wherein the target space image includes three channels; for the current scale, the gradient information of the target space image in different directions on each channel is obtained; the gradient information of the current scale is determined according to the gradient information in different directions on each channel; all scales are traversed, and the gradient information of the standard image is determined according to the gradient information of each scale.

[0169] The target space image may refer to image data in the LAB space. Specifically, the LAB space is a color space used to describe color information in an image.

[0170] The three channels refer to the three channels of the LAB space: L (brightness), A (red and green), and B (yellow and blue). The L channel represents the brightness information of the image, and the value range is usually 0 to 100, where 0 represents black and 100 represents white; the A channel represents the change from green to red, and the value range is from -128 to 127; the b channel represents the change from blue to yellow, and the value range is also from -128 to 127.

[0171] In one embodiment, the step of converting a standard image into a target space image includes: linearizing the RGB values ​​of the standard image, that is, dividing the values ​​of the RGB channels by 255 and normalizing them to the range [0, 1]; converting the linearized RGB values ​​into XYZ values ​​using a conversion matrix multiplication; converting the XYZ values ​​into LAB values ​​using a nonlinear transformation function; and dividing the image into three images, namely, an L channel image, an A channel image, and a B channel image, according to the LAB value.

[0172] The scale can refer to the size or dimension of the convolution kernel used to calculate the gradient. The convolution kernel, also known as a filter or convolution matrix, is a commonly used tool in image processing and computer vision. It is a small matrix or window used to perform convolution operations on the input image. The convolution operation is a basic mathematical operation. In image processing, the convolution operation is often used for tasks such as image filtering, feature extraction, and image enhancement.

[0173] In one embodiment, the gradient information of the target space image in different directions on each channel may be obtained by constructing a structure tuple, and determining the gradient information of the target space image in different directions on each channel based on the structure tuple.

[0174] Among them, the structure element is a template or window used in mathematical morphology operations to define the shape and size of the neighborhood considered during the operation. The structure element can be a small binary matrix or kernel, in which the elements can be 0 or 1. In mathematical morphology, the structure element is used to perform morphological operations such as erosion, dilation, opening and closing operations, which are used in shape analysis, edge detection, and image enhancement in image processing.

[0175] The structure tuple can be an array with structure elements as elements, and each structure element corresponds to a direction. An array is a data structure used to store a collection of elements of the same type. An array provides a continuous, fixed-size storage space, in which each element can be accessed by an index.

[0176] In one embodiment, the structure tuple may be constructed by defining an array with the structure element as the data type and a size of at least 2, and assigning each structure element to the array.

[0177] In one embodiment, the method of determining the gradient information of the target space image in different directions on each channel based on the structure tuple can be adopted by calculating the gradient information of each structure element in the structure tuple on each channel. Wherein, the method of calculating the gradient information of a structure element on a channel can be adopted by extracting the channel image of the target space image, and performing morphological erosion and morphological dilation operations according to the structure element and the channel image, and calculating the difference between the morphological erosion operation result and the morphological dilation operation result as the gradient information of the structure element on the channel. Wherein, the morphological erosion operation refers to using the structure element to compare the channel image pixel by pixel, and retaining the pixels that are completely contained in the channel image area of ​​the structure element, otherwise setting them to 0; the morphological dilation operation refers to comparing the structure element with the channel image pixel by pixel, and setting the pixels in the image to the maximum value as long as the structure element and the image have overlapping parts.

[0178] In one embodiment, the method for determining the gradient information of the current scale can be to use the average of the gradient information of the current channel in different directions for any channel as the gradient information of the current channel, traverse each channel to obtain the gradient information of each channel of the current scale, and use the maximum value of the gradient information of the three channels of the current scale as the gradient information of the current scale.

[0179] In one embodiment, assuming that the structure element of the current scale includes structure elements in four directions, and the gradient information of the target space image in each direction on the L (brightness) channel has been calculated according to the above steps, which are gs1, gs2, gs3 and gs4 respectively, then the calculation formula of the gradient information l of the L (brightness) channel of the current scale is:

[0180] l = (gs1 + gs2 + gs3 + gs4) / 4;

[0181] In one embodiment, by calculating the mean of the gradient information of each channel in different directions as the gradient information, the gradient information l, gradient information a and gradient information b of the current scale can be obtained. The gradient information l is the gradient information of the L (brightness) channel of the current scale, the gradient information a is the gradient information of the A (red and green) channel of the current scale, and the gradient information b is the gradient information of the B (yellow and blue) channel of the current scale.

[0182] In one embodiment, the max() function can be used to obtain the maximum value of the three gradient information of the current scale gradient information l, gradient information a and gradient information b, and the maximum value is used as the gradient information esi of the current scale. The calculation formula of the gradient information esi of the current scale is:

[0183] esi = max(l, a, b);

[0184] Where esi represents the gradient information of the i-th scale.

[0185] In one embodiment, the edge information of the object to be detected may be determined by traversing all scales and determining the edge information of the object to be detected according to the mean of the gradient information of each scale.

[0186] In one embodiment, assuming that three scales are set, and the gradient information of the three scales calculated according to the above steps are es1, es2, and es3, respectively, then the calculation formula of the edge information edge of the object to be measured is:

[0187] edge=(es1+es2+es3) / 3;

[0188] S304, determining a weight value of each pixel point within the neighborhood according to the gradient information of each pixel point;

[0189] The weight value of a pixel point may refer to a measure used to adjust the importance or influence strength of the pixel point for determining weighted height distribution data of a target pixel point.

[0190] In one embodiment, the weight value of each pixel point may be determined by substituting the gradient information of each pixel point into a pre-built semantic penalty function within a neighborhood to determine the weight value of each pixel point.

[0191] In a feasible embodiment, optionally, within the neighborhood, determining the weight value of each pixel point according to the gradient information of each pixel point includes:

[0192] Within the neighborhood, the gradient information of each pixel is substituted into a pre-constructed semantic penalty function to determine the weight value of each pixel.

[0193] The semantic penalty function may refer to a function that calculates the importance or influence of a pixel point on determining the weighted height distribution data of a target pixel point based on the gradient information of the pixel point. As an example, the semantic penalty function pre-constructed in this solution is:

[0194]

[0195] Accordingly, in one embodiment, the weight value of each pixel can be determined by using the following formula: Among them, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, g(v ij ) is the gradient information of the pixel point in the i-th row and j-th column within the neighborhood, g(v c ) is the gradient information of the target pixel, and α is the penalty intensity control parameter.

[0196] This solution, by substituting the gradient information of each pixel into a pre-constructed semantic penalty function to determine the weight value of each pixel, can improve the consistency of the subsequently determined weighted height distribution data and the gradient information.

[0197] In a feasible embodiment, optionally, within the neighborhood, the gradient information of each pixel point is substituted into a pre-constructed semantic penalty function to determine the weight value of each pixel point, including:

[0198] The weight value of each pixel is determined according to the difference between the gradient information of each pixel in the neighborhood and the gradient information of the target pixel, and the penalty intensity control parameter.

[0199] The smaller the difference between the gradient information of the pixel point and the gradient information of the target pixel point, the greater the probability that the pixel point and the target pixel point belong to the same object, and the greater the contribution of the pixel point to the height distribution curve; the larger the difference between the gradient information of the pixel point and the gradient information of the target pixel point, the smaller the probability that the pixel point and the target pixel point belong to the same object, and the smaller the contribution of the pixel point to the height distribution curve. The object may refer to an electronic component arranged on the object to be measured, for example, for a PCB board, the object may be a resistor component.

[0200] Among them, the penalty intensity control parameter refers to a parameter used to adjust the strength or degree of the penalty term in the penalty function, and the penalty intensity control parameter affects the degree of penalty imposed on the violation of the constraint; the penalty intensity control parameter can be set based on the historical calculation results of the high fusion method combined with gradient information.

[0201] This scheme determines the weight value of each pixel point based on the difference between the gradient information of each pixel point in the neighborhood and the gradient information of the target pixel point, and the penalty intensity control parameter, so as to determine the reference of the height value of each pixel point in the neighborhood for calculating the height value of the target pixel point, thereby further determining the reference of the height distribution data of the height image in the neighborhood of the target pixel point for calculating the height value of the target pixel point.

[0202] In a feasible embodiment, optionally, within the neighborhood, the weight value of each pixel is determined by substituting the gradient information of each pixel into a pre-built semantic penalty function, including:

[0203] The calculation is done using the following formula:

[0204]

[0205] Among them, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, g(v ij ) is the gradient information of the pixel point in the i-th row and j-th column within the neighborhood, g(v c ) is the gradient information of the target pixel, and α is the penalty intensity control parameter.

[0206] Figure 5 is an example diagram of the semantic penalty function provided in the embodiment of the present application. c ) is 50, and the penalty intensity control parameter α is selected as 10, the semantic penalty function image is as follows Figure 5 shown.

[0207] This program, is an exponential penalty function. By adopting the exponential penalty function, a smoother penalty effect can be produced.

[0208] S305, performing weighted calculation on the height value of each pixel point within the neighborhood range according to the weight value to obtain weighted height distribution data of the target pixel point;

[0209] In one embodiment, the weighted height distribution data of the target pixel point is obtained by performing a weighted average calculation based on the weight value of each pixel point in the neighborhood range and the height value of each pixel point in the neighborhood range to obtain the mean of the weighted height distribution data of the target pixel point; and performing a weighted calculation and a square root operation based on the weight value of each pixel point in the neighborhood range and the difference between the height value of each pixel point in the neighborhood range and the mean to obtain the variance of the weighted height distribution data of the target pixel point.

[0210] In a feasible embodiment, optionally, weighted calculation is performed on the height value of each pixel point within the neighborhood according to the weight value to obtain weighted height distribution data of the target pixel point, including:

[0211] According to the weight value of each pixel point in the neighborhood range and the height value of each pixel point in the neighborhood range, a weighted average calculation is performed to obtain the mean value of the weighted height distribution data of the target pixel point;

[0212] According to the weight value of each pixel point in the neighborhood range and the difference between the height value of each pixel point in the neighborhood range and the mean value, weighted calculation and square root operation are performed to obtain the variance of the weighted height distribution data of the target pixel point.

[0213] Among them, the mean can represent the concentration trend of the height values ​​of each pixel in the neighborhood. If the variance at this time is small enough, the mean at this time can be regarded as the height fusion result of the target pixel.

[0214] In one embodiment, a weighted average calculation is performed to obtain the mean of the weighted height distribution data of the target pixel point. The calculation formula based on the mean can be used. Perform calculations.

[0215] Among them, the variance can represent the difference between the height value of each pixel in the neighborhood range and the mean. If the variance is small, it means that the difference between the height value of each pixel in the neighborhood range and the mean is small. Further, it means that the reference value of the height value of each pixel in the neighborhood range is greater, that is, the reference value of the mean is greater; if the variance is large, it means that the difference between the height value of each pixel in the neighborhood range and the mean is large. Further, it means that the reference value of the height value of each pixel in the neighborhood range is smaller, that is, the reference value of the mean is smaller.

[0216] In one embodiment, the weighted calculation and square root operation are performed to obtain the variance of the weighted height distribution data of the target pixel point, which can be calculated based on the general formula of the variance: Perform calculations.

[0217] This solution, by calculating the mean and variance of weighted height distribution data, can provide a data basis for the subsequent calculation of the size of the adaptive gain.

[0218] In a feasible embodiment, optionally, weighted calculation is performed on the height distribution data of each pixel point within the neighborhood according to the weight value to obtain weighted height distribution data of the target pixel point, including:

[0219] The mean and variance of weighted height distribution data are calculated using the following formula:

[0220]

[0221]

[0222] Among them, μ is the mean of the weighted height distribution data of the target pixel point, σ is the variance of the weighted height distribution data of the target pixel point, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, a ij is the height value of the pixel in the i-th row and j-th column within the neighborhood.

[0223] Furthermore, the weighted height distribution data can be a normal distribution curve, which can be represented by h, that is, h≈N(μ,σ 2 ).

[0224] This solution, by calculating the mean and variance of weighted height distribution data, can provide a data basis for the subsequent calculation of the size of the adaptive gain.

[0225] S307, performing fusion calculation on the weighted height distribution data of the target pixel points in each height image using an adaptive gain to obtain a height fusion result of the target pixel points.

[0226] This scheme, by combining the gradient information of the pre-acquired standard image for weighted calculation, obtains the weighted height distribution data of the target pixel point, which can effectively reduce the calculation deviation of the target pixel point that is exactly at the edge of the object and improve the accuracy of the height distribution curve of the obtained target pixel point.

[0227] Figure 6 is a flow chart of a high-level fusion method combining gradient information provided in an embodiment of the present application. Figure 6 As shown, the specific steps include:

[0228] S601, performing image height measurement of an object to be measured by at least two measuring devices to obtain a height image output by each measuring device;

[0229] S602, in each height image, obtaining the height value of each pixel in the neighborhood of the target pixel, and performing weighted calculation on the height value of each pixel in the neighborhood in combination with the gradient information of the pre-acquired standard image to obtain weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction;

[0230] S603, obtaining weighted height distribution data of the target pixel in each height image;

[0231] In one embodiment, a method for obtaining weighted height distribution data of a target pixel in each height image may be to determine the weight value of each pixel in a neighborhood according to the gradient information of each pixel for each height image, and perform weighted calculation on the height value of each pixel in the neighborhood according to the weight value to obtain the weighted height distribution data of the target pixel.

[0232] S604, determining the size of the adaptive gain according to the weighted height distribution data;

[0233] The adaptive gain refers to a gain coefficient that is automatically adjusted according to the characteristics of the observed data and the dynamic changes of the system. Specifically, it may refer to a gain coefficient that is automatically adjusted according to the variance in the weighted height distribution data.

[0234] In one embodiment, the size of the adaptive gain may be determined by determining the size of the adaptive gain according to the variance in the weighted height distribution data.

[0235] In a feasible embodiment, optionally, determining the size of the adaptive gain according to the weighted height distribution data includes:

[0236] The size of the adaptive gain is determined according to the variance in the weighted height distribution data.

[0237] The size of the adaptive gain can be determined by calculating the squares of the variances of the weighted height distribution data of the target pixel points in the two height images and summing them to obtain the sum of squares data. According to the variances of the weighted height distribution data of the target pixel points in the two height images, the squares are calculated respectively, and the proportion of the square of the calculated variance in the sum of squares data is determined to determine the size of the adaptive gain of the two height images.

[0238] In this scheme, since the variance σ can represent the degree of discreteness of each pixel in the neighborhood, that is, it can represent the importance or influence of the weighted height distribution data on the height fusion result, the size of the adaptive gain is determined according to the variance in the weighted height distribution data, which can improve the accuracy of the height fusion result.

[0239] In a feasible embodiment, optionally, determining the size of the adaptive gain according to the variance in the weighted height distribution data includes:

[0240] The variance of the weighted height distribution data of the target pixel point in the two height images is squared and summed to obtain the square sum data;

[0241] According to the variance of the weighted height distribution data of the target pixel in the two height images, the squares are calculated respectively, the proportion of the calculated square of the variance in the square sum data is determined, and the size of the adaptive gain of the two height images is determined.

[0242] The square sum data may refer to the sum of the square values ​​of the variances of the weighted height distribution data of the target pixel in the two height images. The square sum data may be obtained by using the following formula:

[0243]

[0244] Among them, σ p represents the variance of the weighted height distribution data of the target pixel in the pth height image, σ q represents the variance in the weighted height distribution data of the target pixel in the qth height image, and sum represents the sum of squares data of the target pixel between the pth height image and the qth height image.

[0245] The proportion of the square of the variance in the sum of squares data is the calculation result obtained by dividing the square of the variance by the sum of squares data. The proportion of the square of the variance in the sum of squares data is the size of the adaptive gain when the height image corresponding to another variance is fused between two height images. To determine the size of the adaptive gain when a height image is fused between two height images, the following formula can be used for calculation:

[0246]

[0247]

[0248] Among them, σ p represents the variance of the weighted height distribution data of the target pixel in the pth height image, k pq represents the size of the adaptive gain of the target pixel of the qth height image when the pth height image is fused with the qth height image, σq represents the variance of the weighted height distribution data of the target pixel in the qth height image, k qp It represents the size of the adaptive gain of the target pixel of the p-th height image when the p-th height image is fused with the q-th height image.

[0249] in, It can be derived according to the following formula:

[0250]

[0251]

[0252] in, It can be derived according to the following formula:

[0253]

[0254]

[0255] This solution can improve the accuracy of the height fusion result by controlling the proportion of the two weighted height distribution data in the fusion process according to the variance in the two weighted height distribution data.

[0256] In a feasible embodiment, optionally, the adaptive gain is determined by the following formula:

[0257]

[0258]

[0259] Among them, σ p represents the variance of the weighted height distribution data of the target pixel in the pth height image, σ q represents the variance of the weighted height distribution data of the target pixel in the qth height image, k pq k represents the size of the adaptive gain of the target pixel of the qth height image when the pth height image is fused with the qth height image; qp It represents the size of the adaptive gain of the target pixel of the p-th height image when the p-th height image is fused with the q-th height image.

[0260] In this scheme, if σ p Compared to σ q is smaller, it means that the p-th height image is more accurate than the q-th height image, and σ p Compared to σ q Smaller leads to k qp Compared to k pqis larger, that is, when the p-th height image and the q-th height image are fused, the p-th height image accounts for a larger proportion, which increases the proportion of the height image with higher accuracy, thereby improving the accuracy of the height fusion result; if σ p Compared to σ q is larger, it means that the accuracy of the p-th height image is lower than that of the q-th height image, and σ p Compared to σ q Greater leads to k qp Compared to k pq Smaller, that is, when the p-th height image is fused with the q-th height image, the proportion of the p-th height image is smaller, reducing the proportion of height images with lower accuracy, thereby improving the accuracy of the height fusion result.

[0261] S605, using the adaptive gain to perform fusion calculation on the weighted height distribution data of the target pixel point in each height image to obtain a height fusion result.

[0262] In one embodiment, a method for obtaining a height fusion result can be adopted, which is to use the weighted height distribution data of the target pixel point in each height image as the basic operator, use adaptive gain to fuse any two basic operators to obtain an intermediate operator, repeat the fusion calculation for all basic operators and intermediate operators until a final fusion result is obtained, and use the average of the final fusion result as the height fusion result of the target pixel point.

[0263] In a feasible embodiment, optionally, the adaptive gain is used to perform fusion calculation on the weighted height distribution data of the target pixel in each height image to obtain a height fusion result, including:

[0264] Taking the weighted height distribution data of the target pixel points in each height image as the basic operator, using the adaptive gain to perform fusion calculation on any two basic operators to obtain an intermediate operator;

[0265] Repeat the fusion calculation for all basic operators and intermediate operators until a final fusion result is obtained;

[0266] The mean value in the final fusion result is taken as the height fusion result of the target pixel.

[0267] The basic operator may refer to the mean and variance of the weighted height distribution data of the target pixel points in each height image.

[0268] The intermediate operator may refer to the mean and variance obtained by any pairwise fusion calculation of all basic operators.

[0269] In one embodiment, the intermediate operator can be obtained by performing fusion calculation on every two adjacent basic operators in the first round of fusion calculation, and performing adjacent fusion calculation on each fusion calculation result again in the second round of fusion calculation, and performing the next round of fusion calculation until only one fusion calculation result remains, and the fusion calculation result is the intermediate operator. If there are an odd number of basic operators or fusion calculation results in a round of fusion calculation, the last basic operator or fusion calculation result directly enters the next round of fusion calculation.

[0270] As an example, in the first round of fusion calculation, there are initially four basic operators A1, A2, A3 and A4. A1 and A2 are fused to obtain a fusion calculation result B1, and A3 and A4 are fused to obtain a fusion calculation result B2. In the second round of fusion calculation, there are currently two fusion calculation results B1 and B2. B1 and B2 are fused to obtain a fusion calculation result C1, and the fusion calculation result C1 is the intermediate operator.

[0271] As an example, in the first round of fusion calculation, there are initially six basic operators A1, A2, A3, A4, A5 and A6. A1 and A2 are fused to obtain a fusion calculation result B1, A3 and A4 are fused to obtain a fusion calculation result B2, and A5 and A6 are fused to obtain a fusion calculation result B3; in the second round of fusion calculation, there are currently three fusion calculation results B1, B2 and B3, and B1 and B2 are fused to obtain a fusion calculation result C1; in the third round of fusion calculation, there are currently two fusion calculation results C1 and B3, and C1 and B3 are fused to obtain a fusion calculation result D1, and the fusion calculation result D1 is the intermediate operator.

[0272] As an example, in the first round of fusion calculation, there are initially five basic operators A1, A2, A3, A4 and A5. A1 and A2 are fused to obtain a fusion calculation result B1, and A3 and A4 are fused to obtain a fusion calculation result B2; in the second round of fusion calculation, there are currently two fusion calculation results B1 and B2 and one basic operator A5. B1 and B2 are fused to obtain a fusion calculation result C1; in the third round of fusion calculation, there is currently one fusion calculation result C1 and one basic operator A5. C1 and A5 are fused to obtain a fusion calculation result D1, and the fusion calculation result D1 is the intermediate operator.

[0273] As an example, in the first round of fusion calculation, there are initially seven basic operators A1, A2, A3, A4, A5, A6 and A7. A1 and A2 are fused to obtain a fusion calculation result B1, A3 and A4 are fused to obtain a fusion calculation result B2, and A5 and A6 are fused to obtain a fusion calculation result B3; in the second round of fusion calculation, there are currently three fusion calculation results B1, B2 and B3 and one basic operator A7. B1 and B2 are fused to obtain a fusion calculation result C1, and B3 and A7 are fused to obtain a fusion calculation result C2; in the third round of fusion calculation, there are currently two fusion calculation results C1 and C2. C1 and C2 are fused to obtain a fusion calculation result D1, and the fusion calculation result D1 is the intermediate operator.

[0274] In one embodiment, the fusion calculation method can be calculated using the following formula:

[0275] h ij =k ji h i +k ij h j ;

[0276] Among them, h i represents the i-th basic operator or fusion calculation result, h j represents the jth basic operator or fusion calculation result, k ji Indicates the fusion time between the i-th basic operator or fusion calculation result and the j-th basic operator or fusion calculation result. i The size of the adaptive gain, k ij Indicates the fusion time between the i-th basic operator or fusion calculation result and the j-th basic operator or fusion calculation result. j The size of the adaptive gain, h ij Represents the fusion calculation result of the i-th basic operator or fusion calculation result and the target pixel point of the j-th basic operator or fusion calculation result.

[0277] The final fusion result may be a fusion calculation result with a mean close to the actual height and a variance close to 0.

[0278] In one embodiment, the final fusion result can be obtained by replacing the mean and variance of the basic operator with the largest variance with the mean and variance of the intermediate operator, and re-fusing the means and variances to obtain the next intermediate operator, until the variance or varianceized value of the intermediate operator is less than a preset threshold, and the intermediate operator is determined as the final fusion result.

[0279] As an example, assume that there are currently four measuring devices, and the input data (i.e., the weighted height distribution data of the target pixel in the height images output by the four measuring devices) is as follows:

[0280] Sequential Number Mean variance 1 4718.637 1166724.8 2 26.442688 12.140282 3 28.313896 3.3460467 4 25.023556 5.843145

[0281] The fusion calculation results (intermediate operator) of the input data in the above table are: the mean is 27.02272334736978, and the variance is 1.8103737410190859.

[0282] Among them, the variance of sequence number 1 is the largest, so the mean and variance of sequence number 1 are replaced with the fusion calculation results (intermediate operators) of the basic operators in the above table, and the new input data is obtained as shown in the following table:

[0283] Sequential Number Mean variance 1 27.02272334736978 1.8103737410190859 2 26.442688 12.140282 3 28.313896 3.3460467 4 25.023556 5.843145

[0284] Repeat the above steps and iterate to get the fusion result as follows:

[0285] Iterations Mean variance 1 27.02272334736978 1.8103737410190859 2 27.019069441488664 0.9051875781375058 3 27.041389224138946 0.4701200213061964 4 27.125965670477274 0.24491243063481988 5 27.080839140632577 0.12710802352354453

[0286] According to the above table, it can be clearly seen that as the number of iterations increases, the variance approaches 0, indicating that the mean approaches the actual height value.

[0287] The average of the final fusion results of each pixel is taken as the height fusion result of the pixel, and the height fusion result of all pixels is calculated to obtain the height fusion result of the entire object to be tested. Figure 4 As shown, (3) is the height fusion result of the object to be measured calculated based on the gradient information.

[0288] This scheme uses adaptive gain to perform fusion calculation to obtain an intermediate operator, and fuses the basic operator with the intermediate operator again until the final fusion result is obtained. In the calculation fusion process, it can pay more attention to or weaken the contribution of specific weighted height distribution data to the fusion result, thereby improving the robustness of the calculation fusion process and the accuracy of the fusion result.

[0289] This solution uses adaptive gain to fuse the weighted height distribution data of the target pixel points in each height image to obtain the height fusion result of the target pixel point. It can fuse the height values ​​collected by multiple measuring devices to achieve complementary advantages between different measuring devices and obtain a height value close to the actual height value.

[0290] Figure 7 Schematic diagram of the structure of the highly integrated device combining gradient information provided in the embodiment of the present application. Figure 7 As shown, specifically including the following:

[0291] A height image acquisition module 710 is used to perform image height measurement on the object to be measured by at least two measuring devices to obtain a height image output by each measuring device;

[0292] The weighted height distribution data calculation module 720 is used to obtain the height value of each pixel in the neighborhood of the target pixel in each height image, and perform weighted calculation on the height value of each pixel in the neighborhood in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction;

[0293] The fusion module 730 is used to perform fusion calculation on the weighted height distribution data of the target pixel points in each height image by using an adaptive gain to obtain a height fusion result of the target pixel points.

[0294] The beneficial effect of this solution is that by combining gradient information for weighted calculation and using adaptive gain for fusion calculation, the gain can be dynamically adjusted during the fusion calculation process, making the fusion of height values ​​more accurate.

[0295] Furthermore, the weighted height distribution data calculation module 720 includes:

[0296] A gradient information acquisition unit, used to acquire the gradient information of the standard image of the object to be measured;

[0297] A pixel weight determination unit, used to determine the weight value of each pixel point within the neighborhood according to the gradient information of each pixel point;

[0298] The weighted height determination unit is used to perform weighted calculation on the height value of each pixel point within the neighborhood according to the weight value to obtain weighted height distribution data of the target pixel point.

[0299] The beneficial effect of this solution is that by combining the gradient information of the pre-acquired standard image for weighted calculation, the weighted height distribution data of the target pixel point is obtained, which can effectively reduce the calculation deviation of the target pixel point that is exactly at the edge of the object and improve the accuracy of the height distribution curve of the obtained target pixel point.

[0300] Furthermore, the pixel weight determination unit is specifically used to:

[0301] Within the neighborhood, the gradient information of the pixel points is substituted into a pre-constructed semantic penalty function to determine the weight value of each pixel point.

[0302] The beneficial effect of this solution is that by substituting the gradient information of each pixel into a pre-constructed semantic penalty function to determine the weight value of each pixel, the consistency of the subsequently determined weighted height distribution data and the gradient information can be improved.

[0303] Furthermore, the pixel weight determination unit is specifically used to:

[0304] The weight value of each pixel is determined according to the difference between the gradient information of each pixel in the neighborhood and the gradient information of the target pixel, and the penalty intensity control parameter.

[0305] The beneficial effect of this scheme is that by determining the weight value of each pixel point based on the difference between the gradient information of each pixel point in the neighborhood range and the gradient information of the target pixel point and the penalty intensity control parameter, the reference of the height value of each pixel point in the neighborhood range for calculating the height value of the target pixel point can be determined, thereby further determining the reference of the height distribution data of the height image in the neighborhood range of the target pixel point for calculating the height value of the target pixel point.

[0306] Furthermore, the pixel weight determination unit is specifically used to:

[0307] The calculation is done using the following formula:

[0308]

[0309] Among them, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, g(v ij ) is the gradient information of the pixel point in the i-th row and j-th column within the neighborhood, g(v c ) is the gradient information of the target pixel, and α is the penalty intensity control parameter.

[0310] The beneficial effect of this scheme is that is an exponential penalty function (the general formula of the exponential penalty function is: ), by adopting the exponential penalty function, a smoother penalty effect can be produced.

[0311] Furthermore, the weighted height determination unit is specifically used to:

[0312] According to the weight value of each pixel point in the neighborhood range and the height value of each pixel point in the neighborhood range, a weighted average calculation is performed to obtain the mean value of the weighted height distribution data of the target pixel point;

[0313] According to the weight value of each pixel point in the neighborhood range and the difference between the height value of each pixel point in the neighborhood range and the mean value, weighted calculation and square root operation are performed to obtain the variance of the weighted height distribution data of the target pixel point.

[0314] The beneficial effect of this solution is that by calculating the mean and variance of the weighted height distribution data, a data basis can be provided for the subsequent calculation of the size of the adaptive gain.

[0315] Furthermore, the weighted height determination unit is specifically used to:

[0316] The mean and variance of weighted height distribution data are calculated using the following formula:

[0317]

[0318]

[0319] Among them, μ is the mean of the weighted height distribution data of the target pixel point, σ is the variance of the weighted height distribution data of the target pixel point, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, a ij is the height value of the pixel in the i-th row and j-th column within the neighborhood.

[0320] The beneficial effect of this solution is that by calculating the mean and variance of the weighted height distribution data, a data basis can be provided for the subsequent calculation of the size of the adaptive gain.

[0321] Furthermore, the fusion module 730 includes:

[0322] A weighted height acquisition unit is used to acquire weighted height distribution data of target pixel points in each height image;

[0323] An adaptive gain determination unit, used to determine the size of the adaptive gain according to the weighted height distribution data;

[0324] The fusion result determination unit is used to use the adaptive gain to perform fusion calculation on the weighted height distribution data of the target pixel point in each height image to obtain a height fusion result.

[0325] The beneficial effect of this solution is that by using adaptive gain to perform fusion calculation on the weighted height distribution data of the target pixel points in each height image, the height fusion result of the target pixel points is obtained. The height values ​​collected by multiple measuring devices can be fused to achieve complementary advantages between different measuring devices and obtain a height value close to the actual height value.

[0326] Furthermore, the adaptive gain determination unit is specifically used to:

[0327] The size of the adaptive gain is determined according to the variance in the weighted height distribution data.

[0328] The beneficial effect of this scheme is that since the variance σ can represent the degree of discreteness of each pixel in the neighborhood, that is, it can represent the importance or influence of the weighted height distribution data on the height fusion result, therefore, determining the size of the adaptive gain according to the variance in the weighted height distribution data can improve the accuracy of the height fusion result.

[0329] Furthermore, the adaptive gain determination unit is specifically used to:

[0330] The variance of the weighted height distribution data of the target pixel point in the two height images is squared and summed to obtain the square sum data;

[0331] According to the variance of the weighted height distribution data of the target pixel in the two height images, the squares are calculated respectively, the proportion of the calculated square of the variance in the square sum data is determined, and the size of the adaptive gain of the two height images is determined.

[0332] The beneficial effect of this solution is that by controlling the proportion of the two weighted height distribution data in the fusion process according to the variance in the two weighted height distribution data, the accuracy of the height fusion result can be improved.

[0333] Furthermore, the adaptive gain determination unit is specifically used to:

[0334] The adaptive gain is determined using the following formula;

[0335]

[0336]

[0337] Among them, σ p represents the variance of the weighted height distribution data of the target pixel in the pth height image, σ q represents the variance of the weighted height distribution data of the target pixel in the qth height image, k pq k represents the size of the adaptive gain of the target pixel of the qth height image when the pth height image is fused with the qth height image; qq It represents the size of the adaptive gain of the target pixel of the p-th height image when the p-th height image is fused with the q-th height image.

[0338] The beneficial effect of this scheme is that if σ q Compared to σ q is smaller, it means that the p-th height image is more accurate than the q-th height image, and σ q Compared to σ q Smaller leads to k qp Compared to k pqis larger, that is, when the p-th height image and the q-th height image are fused, the p-th height image accounts for a larger proportion, which increases the proportion of the height image with higher accuracy, thereby improving the accuracy of the height fusion result; if σ p Compared to σ q is larger, it means that the accuracy of the p-th height image is lower than that of the q-th height image, and σ p Compared to σ q Greater leads to k qp Compared to k pq Smaller, that is, when the p-th height image is fused with the q-th height image, the proportion of the p-th height image is smaller, reducing the proportion of height images with lower accuracy, thereby improving the accuracy of the height fusion result.

[0339] Furthermore, the fusion result determination unit is specifically used to:

[0340] Taking the weighted height distribution data of the target pixel points in each height image as the basic operator, using the adaptive gain to perform fusion calculation on any two basic operators to obtain an intermediate operator;

[0341] Repeat the fusion calculation for all basic operators and intermediate operators until a final fusion result is obtained;

[0342] The mean value in the final fusion result is taken as the height fusion result of the target pixel.

[0343] The beneficial effect of this scheme is that by using adaptive gain to perform fusion calculation to obtain an intermediate operator, and fusing the basic operator with the intermediate operator again until the final fusion result is obtained, the contribution of specific weighted height distribution data to the fusion result can be more emphasized or weakened in the calculation fusion process, thereby improving the robustness of the calculation fusion process and the accuracy of the fusion result.

[0344] The technical solution provided by the embodiment of the present application includes a height image acquisition module, which is used to measure the image height of the object to be measured by at least two measuring devices to obtain the height image output by each measuring device; a weighted height distribution data calculation module, which is used to obtain the height value of each pixel in the neighborhood range of the target pixel in each height image, and to perform weighted calculation on the height value of each pixel in the neighborhood range in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction; a fusion module, which is used to perform fusion calculation on the weighted height distribution data of the target pixel in each height image using an adaptive gain to obtain the height fusion result of the target pixel. The above-mentioned height fusion device combined with gradient information can dynamically adjust the gain during the fusion calculation process by combining gradient information for weighted calculation and using adaptive gain for fusion calculation, so that the fusion of the height value is more accurate.

[0345] The highly fused device combining gradient information in the embodiment of the present application can be a device, or a component, integrated circuit or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0346] The highly fused device combining gradient information in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0347] The highly fused device combined with gradient information provided in the embodiment of the present application can realize each process realized by the above-mentioned method embodiments, and will not be described again here to avoid repetition.

[0348] Figure 8 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 8As shown, an embodiment of the present application further provides an electronic device 800, including a processor 801, a memory 802, and a program or instruction stored in the memory 802 and executable on the processor 801. When the program or instruction is executed by the processor 801, each process of the embodiment of the highly fused device combining gradient information is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0349] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0350] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned embodiment of the highly fused device combining gradient information is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0351] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0352] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the highly fused device combining gradient information, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0353] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0354] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0355] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0356] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0357] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A high degree of fusion method combining gradient information, characterized in that: The method comprises: Performing image height measurement of the object to be measured by at least two measuring devices to obtain a height image output by each measuring device; In each height image, the height value of each pixel in the neighborhood of the target pixel is obtained, and the height value of each pixel in the neighborhood is weightedly calculated in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction; The weighted height distribution data of the target pixel points in each height image are fused and calculated using adaptive gain to obtain the height fusion result of the target pixel points.

2. The high-level fusion method combining gradient information according to claim 1, characterized in that: The height value of each pixel point within the neighborhood is weightedly calculated in combination with the gradient information of the pre-acquired standard image to obtain weighted height distribution data of the target pixel point, including: Acquiring gradient information of a standard image of the object to be measured; Within the neighborhood, determining a weight value of each pixel point according to gradient information of each pixel point; The height value of each pixel point within the neighborhood is weightedly calculated according to the weight value to obtain weighted height distribution data of the target pixel point.

3. The high-level fusion method combining gradient information according to claim 2, characterized in that: Within the neighborhood, the weight value of each pixel is determined according to the gradient information of each pixel, including: Within the neighborhood, the gradient information of each pixel is substituted into a pre-constructed semantic penalty function to determine the weight value of each pixel.

4. The high-level fusion method combining gradient information according to claim 3 is characterized in that: In the neighborhood, the gradient information of each pixel is substituted into a pre-built semantic penalty function to determine the weight value of each pixel, including: The weight value of each pixel is determined according to the difference between the gradient information of each pixel in the neighborhood and the gradient information of the target pixel, and the penalty intensity control parameter.

5. The high-level fusion method combining gradient information according to claim 4 is characterized in that: According to the difference between the gradient information of each pixel in the neighborhood and the gradient information of the target pixel, and the penalty intensity control parameter, the weight value of each pixel is determined, including: The calculation is done using the following formula: Among them, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, g(v ij ) is the gradient information of the pixel point in the i-th row and j-th column within the neighborhood, g(v c ) is the gradient information of the target pixel, and α is the penalty intensity control parameter.

6. The high-level fusion method combining gradient information according to claim 2, characterized in that: The height values ​​of each pixel point within the neighborhood are weightedly calculated according to the weight value to obtain weighted height distribution data of the target pixel point, including: According to the weight value of each pixel point in the neighborhood range and the height value of each pixel point in the neighborhood range, a weighted average calculation is performed to obtain the mean value of the weighted height distribution data of the target pixel point; According to the weight value of each pixel point in the neighborhood range and the difference between the height value of each pixel point in the neighborhood range and the mean value, weighted calculation and square root operation are performed to obtain the variance of the weighted height distribution data of the target pixel point.

7. The high-level fusion method combining gradient information according to claim 6 is characterized in that: The mean and variance of weighted height distribution data are calculated using the following formula: Among them, μ is the mean of the weighted height distribution data of the target pixel point, σ is the variance of the weighted height distribution data of the target pixel point, β ij is the weight value of the pixel in the i-th row and j-th column in the neighborhood, a ij is the height value of the pixel in the i-th row and j-th column within the neighborhood.

8. The high-level fusion method combining gradient information according to claim 1, characterized in that: The weighted height distribution data of the target pixel points in each height image are fused and calculated using adaptive gain to obtain the height fusion result of the target pixel points, including: Obtain weighted height distribution data of target pixel points in each height image; Determining the size of the adaptive gain according to the weighted height distribution data; The adaptive gain is used to perform fusion calculation on the weighted height distribution data of the target pixel point in each height image to obtain a height fusion result.

9. The high-level fusion method combining gradient information according to claim 8, characterized in that: Determining the size of the adaptive gain according to the weighted height distribution data includes: The size of the adaptive gain is determined according to the variance in the weighted height distribution data.

10. The high-level fusion method combining gradient information according to claim 9, characterized in that: Determining the size of the adaptive gain according to the variance in the weighted height distribution data includes: The variance of the weighted height distribution data of the target pixel point in the two height images is squared and summed to obtain the square sum data; According to the variance of the weighted height distribution data of the target pixel in the two height images, the squares are calculated respectively, the proportion of the calculated square of the variance in the square sum data is determined, and the size of the adaptive gain of the two height images is determined.

11. The high-level fusion method combining gradient information according to claim 10, characterized in that: The adaptive gain is determined using the following formula; Among them, σ p represents the variance of the weighted height distribution data of the target pixel in the pth height image, σ q represents the variance of the weighted height distribution data of the target pixel in the qth height image, k pq k represents the size of the adaptive gain of the target pixel of the qth height image when the pth height image is fused with the qth height image; qp It represents the size of the adaptive gain of the target pixel of the p-th height image when the p-th height image is fused with the q-th height image.

12. The high-level fusion method combining gradient information according to claim 8, characterized in that: The adaptive gain is used to perform fusion calculation on the weighted height distribution data of the target pixel point in each height image to obtain a height fusion result, including: Taking the weighted height distribution data of the target pixel points in each height image as the basic operator, using the adaptive gain to perform fusion calculation on any two basic operators to obtain an intermediate operator; Repeat the fusion calculation for all basic operators and intermediate operators until a final fusion result is obtained; The mean value in the final fusion result is taken as the height fusion result of the target pixel.

13. A highly fused device combining gradient information, characterized in that: The device comprises: A height image acquisition module is used to perform image height measurement of the object to be measured by at least two measuring devices to obtain a height image output by each measuring device; A weighted height distribution data calculation module is used to obtain the height value of each pixel in the neighborhood of the target pixel in each height image, and perform weighted calculation on the height value of each pixel in the neighborhood in combination with the gradient information of the pre-acquired standard image to obtain the weighted height distribution data of the target pixel; wherein the standard image is obtained by photographing the object to be measured by a camera in a preset direction; The fusion module is used to perform fusion calculation on the weighted height distribution data of the target pixel points in each height image using adaptive gain to obtain the height fusion result of the target pixel points.

14. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the high-degree fusion method combining gradient information as described in any one of claims 1 to 12.

15. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the high-degree fusion method combining gradient information as described in any one of claims 1 to 12 are implemented.

Images