A method and device for measuring three-dimensional information of a target

Abstract

The present application discloses a method for measuring three-dimensional information of a target, the method comprising: obtaining multiple time series of image data for characterizing the distance between the target to be measured and the image acquisition device, obtaining a height map for characterizing the height value of each pixel point in each time series based on the obtained time series image data, traversing each height map in a time series, using the height value of each pixel point in each height map, dynamically updating the height value of each pixel point, so that the height value of each pixel point in the missing area caused by the self-occlusion between the targets to be measured is completed, and the height value of each pixel point in the unoccluded area is maintained, and an updated height map is obtained, and based on the updated height map, the three-dimensional information of the target to be measured is measured. The present application improves the accuracy of three-dimensional information measurement. The present application can also perform differential modeling on the image data in the acquisition scene and the measurement scene, improving the adaptability of the application scene and the accuracy of measurement.

Description

Technical Field

[0001] The present invention relates to the field of visual three-dimensional information measurement, and in particular, to a method for measuring target three-dimensional information. Background Art

[0002] In many scenarios, it is necessary to measure the three-dimensional information of an object. The measurement of the three-dimensional information includes at least one of volume measurement, area measurement, and height measurement.

[0003] Recently, image-based three-dimensional information measurement has been widely used. In practical applications, due to the mutual occlusion between objects in the image, the accuracy of the measurement of the target's three-dimensional information is affected. Summary of the invention

[0004] The invention provides a method for measuring three-dimensional information of a target, so as to improve the accuracy of measuring the three-dimensional information of the target.

[0005] A first aspect of the present invention provides a method for measuring three-dimensional information of a target, the method comprising:

[0006] Acquire multiple time series of image data for representing the distance between the target to be measured and the image acquisition device,

[0007] Based on the acquired time-series image data, a height map for representing the height value of each pixel point in each time sequence is obtained.

[0008] Each height map is traversed sequentially, and the height value of each pixel in each height map is dynamically updated by using the height value of each pixel, so that the height value of each pixel in the missing area caused by the self-occlusion between the targets to be measured is supplemented, and the height value of each pixel in the unoccluded area is maintained, and the updated height map is obtained.

[0009] Based on the updated height map, three-dimensional information of the target to be measured is measured.

[0010] Preferably, the dynamically updating the height value of each pixel point by using the height value of each pixel point in each height map includes:

[0011] Check whether there is self-occlusion in the height value of each pixel in the current height map traversed. If there is self-occlusion, the height value of each pixel with self-occlusion is completed based on the current height map, and the height value of each pixel without self-occlusion is updated using the height value of each pixel in the current height map.

[0012] The detection is repeatedly performed to determine whether the height values ​​of each pixel point in the current height map traversed are blocked by the target to be measured until all height maps are traversed.

[0013] Preferably, the detecting whether the height values ​​of each pixel point in the traversed current height map have self-occlusion between the objects to be measured includes:

[0014] Convert the spatial points corresponding to each pixel point and its height value in the current height map into spatial points in the polar coordinate system to obtain the polar coordinate position information of the current spatial point.

[0015] For the height value of any pixel point, the spatial point corresponding to the pixel point and its historical height value is converted into a spatial point in the polar coordinate system to obtain the polar coordinate system position information of the historical spatial point of the pixel point. The historical height value is the height value of the pixel point in each historical height map whose time sequence is earlier than the time sequence of the current height map.

[0016] The polar coordinate system position information includes: azimuth, polar angle, and radius.

[0017] Based on the azimuth and polar angle in the polar coordinate system position information of the historical spatial point, the traversal interval of the pixel point is set.

[0018] According to the traversal interval of the pixel point, traverse the polar coordinate system position information of the current spatial point, and determine whether the difference between the radius of any historical spatial point of the pixel point and the radius of the traversed current spatial point is greater than the set distance threshold. If the difference is greater than the distance threshold, it is determined that the height value of the pixel point in the current height map is self-occluded.

[0019] Preferably, the dynamically updating the height value of each pixel point by using the height value of each pixel point in each height map includes:

[0020] For the current height value of any pixel in the current height map,

[0021] Determine whether the current height value of the pixel is valid.

[0022] If the current height value is valid and there is no self-occlusion, compare the current height value with the previous height value of the pixel.

[0023] If the current height value is greater than the previous height value, the current height value remains unchanged.

[0024] If the current height value is not greater than the previous height value, delete all historical height values ​​greater than or equal to the current height value and keep the current height value unchanged.

[0025] If the current height value is invalid, the historical height values ​​of the pixel are traversed in reverse order of time to detect whether the traversed current historical height value is self-occluded.

[0026] If the current historical height value is self-occluded, the traversal is terminated and the current historical height value is used as the current height value; if the current historical height value is not self-occluded, the current historical height value is deleted, the next historical height value of the pixel is traversed, and the step of detecting whether the traversed current historical height value is self-occluded is returned until a historical height value with self-occlusion is detected; if no historical height value with self-occlusion is detected after the historical height value traversal of the pixel is completed, the current height value is marked as invalid.

[0027] Traverse the height value of the pixel in the next height map adjacent to the current height map, return to execute the step of the current height value of any pixel in the current height map, until all height maps of the pixel are traversed, and obtain the updated height value of the pixel.

[0028] Preferably, before comparing the current altitude value with the previous altitude value, the method further comprises:

[0029] Determine whether the previous height value of the pixel in the previous height map adjacent to the current height map is valid.

[0030] If the previous altitude value is valid, the step of comparing the current altitude value with the previous altitude value is performed,

[0031] If the previous altitude value is invalid, keep the current altitude value unchanged;

[0032] Preferably, before the detecting whether the traversed current historical height value has self-occlusion, the method further comprises:

[0033] Determine whether the current historical height value is valid.

[0034] If the current historical height value is valid, the step of detecting whether the traversed current historical height value is self-occluded is performed.

[0035] If the current historical height value is invalid, then traverse the previous historical height value of the pixel point, and return to execute the step of detecting whether the traversed current historical height value has self-occlusion.

[0036] Preferably, the dynamically updating the height value of each pixel point by using the height value of each pixel point in each height map further includes:

[0037] Establishing a height table for each pixel point for recording the historical information of each height map of the pixel point, the height table includes: a height array for recording the historical height value of the pixel point, a status array for recording the status information representing whether the historical height value of the pixel point is valid, and a header array for recording the historical header height value, wherein the height values ​​in the height array are arranged from small to large, each height value corresponds to the status information, and the header height value is used to represent the currently updated height value of the pixel point in the height map;

[0038] Preferably, the determining whether the previous height value of the pixel point in the previous height map adjacent to the current height map is valid includes:

[0039] According to the status array in the pixel height table, determine whether the header height value in the pixel height table is valid.

[0040] If the header height value is valid, compare the current height value with the header height value.

[0041] If the current height value is greater than the header height value, the current height value is recorded in the header array as the header height value, the current height value is recorded in the height array, and the status information of the current height value is recorded in the status array.

[0042] If the current height value is not greater than the header height value, delete the height values ​​in the height array that are greater than or equal to the current height value, then add the current height value to the height array, record the current height value as a valid state in the status array, and record it in the header array as the header height value.

[0043] If the table header height value is invalid, the current height value is recorded in the height array, and the current height value is recorded as a valid state in the state array, and is recorded in the table header array as the table header height value;

[0044] Preferably, the traversing each historical height value of the pixel point includes: traversing each header height value of a header array in a height table of the pixel point;

[0045] The determining whether the previous height value of the current pixel point being traversed is valid includes:

[0046] According to the status array in the current altimeter traversed, determine whether the header altitude value in the current altimeter is valid.

[0047] If the header height value is valid, then if the header height value is not blocked, delete the header height value from the header array.

[0048] If the header height value is invalid, get the previous header height value in the header array;

[0049] Preferably, the three-dimensional information measurement of the target to be measured based on the updated height map includes:

[0050] The header height value in the current height table of each pixel point is used as the updated height value of each pixel point.

[0051] Based on the updated height value of each pixel point, at least one of the height, occupied area and volume of the target to be measured is determined.

[0052] Preferably, converting the spatial point corresponding to the pixel point and its historical height value into a spatial point in a polar coordinate system includes:

[0053] The space point corresponding to the height value of the table header and its pixel point is converted into a space point in the polar coordinate system to obtain the polar coordinate system position information corresponding to the height value of the table header;

[0054] Preferably, the step of setting the traversal interval of the pixel point based on the azimuth and polar angle in the polar coordinate system position information of the historical spatial point includes:

[0055] Taking the azimuth and polar angle in the polar coordinate system position information corresponding to the header height value as the center, respectively, the azimuth traversal area and polar angle traversal area of ​​the pixel point are set;

[0056] Preferably, the determining whether a difference obtained by subtracting a radius of the traversed current spatial point from a radius of any historical spatial point of the pixel point is greater than a set distance threshold comprises:

[0057] Subtract the radius of the current space point traversed from the radius in the polar coordinate system position information corresponding to the header height value to obtain the difference. When the difference is greater than the distance threshold, it is determined that the header height value is blocked by the current height map.

[0058] Preferably, the polar coordinate system position information is stored in a two-dimensional array, one dimension represents the azimuth, the other dimension represents the polar angle, and the radius is stored in an array of the azimuth and polar angle corresponding to the radius.

[0059] The step of setting the azimuth traversal area and polar angle traversal area of ​​the pixel point with the azimuth and polar angle of the spatial point in the historical polar coordinate system of the pixel point as the center respectively includes:

[0060] Based on the dimension where the azimuth angle is located and the dimension where the polar angle is located, the azimuth angle traversal array and the polar angle traversal array are set with the azimuth angle and polar angle of the spatial point in the historical polar coordinate system of the pixel point as the center;

[0061] Preferably, the traversing the polar coordinate system position information of the current space point includes:

[0062] According to the azimuth traversal array and the polar angle traversal array, determine the two-dimensional array where the polar coordinate system position information of the current space point to be traversed is located.

[0063] Iterate over the radius values ​​in this two-dimensional array.

[0064] Preferably, the step of acquiring image data at each time sequence for representing the distance between the target to be measured and the image acquisition device includes:

[0065] Acquire at least one frame of first image data and at least one frame of second image data respectively, wherein the first image data does not include the target image to be measured or includes the first target image to be measured, the second image data includes the second target image to be measured, and the first target image to be measured is different from the second target image to be measured;

[0066] Preferably, the step of obtaining a height map for representing the height value of each pixel point at each time sequence based on the acquired time sequence image data includes:

[0067] acquiring a first height map based on the first image data, acquiring a second height map based on the second image data,

[0068] For each pixel in the height map, a second height value of the pixel in each second height map is differentially operated with the first height value of the pixel in the first height map to obtain the height value of the pixel in each second height map.

[0069] A second aspect of the present application provides a device for measuring three-dimensional information of a target, the device comprising:

[0070] An image acquisition module is used to acquire multiple time series of image data for representing the distance between the target to be measured and the image acquisition device.

[0071] The height map acquisition module acquires the height map of each time sequence for representing the height value of each pixel point based on the acquired time sequence image data.

[0072] The height map modeling module traverses each height map in time sequence, and dynamically updates the height value of each pixel point using the height value of each pixel point in each height map, so that the height value of each pixel point in the missing area caused by the self-occlusion between the targets to be measured is completed, and the height value of each pixel point in the unoccluded area is maintained, thereby obtaining an updated height map.

[0073] The measurement module measures the three-dimensional information of the target to be measured based on the updated height map.

[0074] A third aspect of the present application provides a computer-readable storage medium storing a computer program, which is executed by a processor to implement the steps of the above-mentioned method for measuring target three-dimensional information.

[0075] A method for measuring three-dimensional information of a target provided in an embodiment of the present application utilizes corresponding height map information obtained based on multi-frame image data to complete height values ​​of self-occluded image areas formed by mutual occlusion between targets, thereby completing the height information of the target to be measured, avoiding the loss of height information caused by a single frame, and improving the accuracy of measuring the three-dimensional information of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic flow chart of a method for measuring three-dimensional information of a target according to an embodiment of the present application.

[0077] Figure 2 A schematic diagram of a process for dynamically updating the height value of each pixel point according to an embodiment of the present application.

[0078] Figure 3 A schematic diagram of a flow chart of the method for measuring three-dimensional information of a target in this embodiment.

[0079] Figure 4 A schematic diagram of scene acquisition and differential modeling in this embodiment.

[0080] Figure 5 A schematic diagram of the height table for a pixel in the height map

[0081] Figure 6 A schematic diagram of a process of updating a height map based on multiple frames of height maps in this embodiment.

[0082] Figure 7 A schematic diagram for modeling a height map as a target goes from unoccluded to occluded to unoccluded again as time goes on.

[0083] Figure 8 Another diagram for modeling a height map as an object progresses from unoccluded to occluded to unoccluded again as time goes on.

[0084] Fig. 9 A schematic diagram of detecting whether a height value is blocked by the current height map in this embodiment.

[0085] Fig.10 A schematic diagram of the polar coordinate system.

[0086] Fig.11 A schematic diagram of a two-dimensional array of polar coordinate position information.

[0087] Fig.12 A schematic diagram of the height map modeling process of this embodiment.

[0088] Fig.13 This is an example of an application scenario of the target three-dimensional information measurement method of this embodiment.

[0089] Fig.14 A schematic diagram of a target three-dimensional information measuring device according to an embodiment of the present application.

[0090] Fig.15 Another schematic diagram of the target three-dimensional information measuring device according to an embodiment of the present application. DETAILED DESCRIPTION

[0091] In order to make the objectives, technical means and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings.

[0092] An embodiment of the present application provides a method for measuring three-dimensional information of a target, which utilizes a multi-frame height map of the target to be measured to dynamically update the height value of each pixel point, so that the height value of each pixel point in the missing area caused by self-occlusion between the targets to be measured is completed, and the height value of each pixel point in the unoccluded area is maintained, thereby obtaining an updated height map to improve the accuracy of measuring the three-dimensional information of the target.

[0093] See also Figure 1 As shown, Figure 1 A flow chart of a method for measuring three-dimensional information of a target in an embodiment of the present application. The method comprises:

[0094] Step 101, obtaining a plurality of time series of image data for representing the distance between the target to be measured and the image acquisition device,

[0095] As an example, multiple time-series image data refer to image data at different moments, that is, multiple frames of image data. Each moment can be continuous or discontinuous, so that the image data can present the historical change process of the target to be measured, including but not limited to dynamic changes such as increase, decrease, and restacking of the target to be measured. For example, in a manual or mechanical stacking scenario, the target to be measured is multiple boxes, which are stacked one by one.

[0096] As an example, at least one frame of first image data and at least one frame of second image data are respectively acquired through the same image acquisition device, wherein the first image data does not include the image of the target to be measured or includes the first target image to be measured, the second image data includes the second target image to be measured, and the first target image to be measured is different from the second target image to be measured. For example, at least one frame of first image data in an acquisition scene and at least one frame of second image data in a measurement scene are respectively acquired, wherein the first image data does not include the image of the target to be measured, and the second image data includes the image of the target to be measured. For another example, at least one frame of first image data in a first scene and at least one frame of second image data in a second measurement scene are respectively acquired, wherein the first scene has a first target to be measured, and the second scene is a second target to be measured stacked on the basis of the first target to be measured, so that the first target image to be measured and the second target image to be measured are at least different in height.

[0097] Step 102, based on the acquired time-series image data, a height map for representing the height value of each pixel point in each time sequence is acquired.

[0098] As an example, algorithms such as visual matching are used to perform visual estimation on each frame of image data to obtain a corresponding height map.

[0099] As an example, a first height map is obtained based on the first image data, and a second height map is obtained based on the second image data. For each pixel in the height map, the second height value of the pixel in each second height map is differentially operated with the first height value of the pixel in the first height map to obtain the height value of the pixel in each second height map, so as to improve the accuracy of the height map.

[0100] Step 103, traverse each height map in time sequence, and dynamically update the height value of each pixel point using the height value of each pixel point in each height map, so that the height value of each pixel point in the missing area caused by the self-occlusion between the targets to be measured is completed, and the height value of each pixel point in the unoccluded area is maintained, thereby obtaining an updated height map.

[0101] As an example, the dynamic update of the height value of each pixel can be performed as follows:

[0102] Check whether there is self-occlusion in the height value of each pixel in the current height map traversed. If there is self-occlusion, the height value of each pixel with self-occlusion is completed based on the current height map, and the height value of each pixel without self-occlusion is updated using the height value of each pixel in the current height map.

[0103] For example, see Figure 2 As shown, Figure 2A schematic diagram of a process for dynamically updating the height value of each pixel. For the current height value of any pixel in the current height map,

[0104] Step 1031, determine whether the current height value of the pixel is valid.

[0105] If the current height value is valid and there is no self-occlusion, the current height value is compared with the previous height value of the pixel, where the previous height value is the height value of the pixel in the previous height map adjacent to the current height map.

[0106] If the current height value is greater than the previous height value, it is determined that the height of the target to be measured has increased, for example, a new target to be measured is stacked, so the current height value is kept unchanged, and the height value of the pixel point in the current height map is used for updating.

[0107] If the current height value is not greater than the previous height value, it is determined that the height of the target to be measured has decreased, for example, a target to be measured has been removed, so all historical height values ​​greater than or equal to the current height value are deleted to remove redundant historical height values, and the current height value is kept unchanged to be updated using the height value of the pixel point in the current height map;

[0108] If the current height value is valid and there is self-occlusion, delete all historical height values ​​that are greater than or equal to the current height value and keep the current height value unchanged;

[0109] If the current height value is invalid, check whether the previous height value is self-occluded.

[0110] If it exists, it means that the height value of the pixel is blocked by the current height map, and the previous height value is used as the current height value to complete the height value of the pixel.

[0111] If it does not exist, delete the previous height value to remove redundant historical height values, take the previous height value adjacent to the previous height value as the detection target, trace back the height value at an earlier time sequence, and return to execute the step of detecting whether the previous height value has self-occlusion until the detected height value has self-occlusion. In other words, detect whether each historical height value has self-occlusion in reverse time sequence. Once a height value with self-occlusion is detected, the height value with self-occlusion is used as the current height value to complete the height value of the pixel. If all historical height values ​​of the pixel have been detected for self-occlusion and no self-occlusion exists, mark the current height value as invalid.

[0112] Step 1032, traverse the height value of the pixel in the next height map adjacent to the current height map, return to execute the step of the current height value of any pixel in the current height map, until all height maps of the pixel are traversed, and obtain the updated height value of the pixel.

[0113] To improve the robustness and efficiency of the dynamic update process, before comparing the current height value with the previous height value, further comprising:

[0114] Determine whether the previous height value is valid.

[0115] In the case where the previous height value is invalid, the current height value is kept unchanged, and the height value of the pixel point in the current height map is used for updating.

[0116] Before detecting whether the traversed current historical height value is self-occluded, further comprising:

[0117] Determine whether the previous height value is valid.

[0118] As an example, self-occlusion can be performed as follows:

[0119] Convert the spatial points corresponding to each pixel point and its height value in the current height map into spatial points in the polar coordinate system to obtain the polar coordinate system position information of the current spatial point.

[0120] For the height value of any pixel point, the spatial point corresponding to the pixel point and its historical height value is converted into a spatial point in the polar coordinate system to obtain the polar coordinate system position information of the historical spatial point of the pixel point. The historical height value is the height value of the pixel point in each historical height map whose time sequence is earlier than the time sequence of the current height map.

[0121] The polar coordinate system position information includes: azimuth, polar angle, and radius.

[0122] Based on the azimuth and polar angle in the polar coordinate system position information of the historical spatial point, the traversal interval of the pixel point is set.

[0123] According to the traversal interval of the pixel point, traverse the polar coordinate system position information of the current spatial point, and determine whether the difference between the radius of any historical spatial point of the pixel point and the radius of the traversed current spatial point is greater than the set distance threshold. If the difference is greater than the distance threshold, it is determined that the height value of the pixel point in the current height map is self-occluded.

[0124] Step 104 , based on the updated height map, three-dimensional information measurement is performed on the target to be measured.

[0125] The embodiment of the present application dynamically updates the height value of each pixel through the height map information at each time sequence. During the dynamic update process, the height value of each pixel in the missing area caused by the self-occlusion between the targets to be measured is completed, and the height value of each pixel in the unoccluded area is maintained, so that the accuracy of the height map is improved pixel by pixel, thereby avoiding the defects of occlusion missing and inaccurate measurement caused by single-frame self-occlusion. The embodiment of the present application can not only perform static measurement, for example, a one-time measurement of the target to be measured without dynamic changes, but also perform dynamic measurement, for example, measuring the three-dimensional information of the target to be measured with dynamic changes at different time sequences.

[0126] To facilitate understanding of the embodiments of the present application, the following description is made by taking the image data as a depth map as an example. It should be understood that the embodiments of the present application are not limited to depth maps, and other image data may also be flexibly applicable.

[0127] See also Figure 3 As shown, Figure 3 A schematic flow chart of a method for measuring three-dimensional information of a target in this embodiment. The method comprises:

[0128] Step 301: Acquire multiple frames of depth maps and / or point cloud images of a target to be measured.

[0129] For example, a depth map can be obtained from a binocular camera, a TOF camera, etc., a point cloud image can be obtained by a camera such as a laser or a millimeter-wave radar, and the multi-frame images can be images that are continuous in time sequence, such as video images, or can be discontinuous images, such as extracted from video images.

[0130] Step 302: convert the acquired depth map and / or point cloud image into a point cloud in a world coordinate system frame by frame.

[0131] As an example, for any frame depth map and / or point cloud image, the pixel point position information in the image frame is converted into the spatial point position information in the world coordinate system by using the camera parameters to obtain the spatial point position information of the image frame.

[0132] Step 303: Based on the point cloud in the world coordinate system, a height map for representing pixel height information is obtained frame by frame.

[0133] As an example, for any image frame, the spatial point position information of the image frame is mapped to a height map. Each pixel in the height map is used to represent the smallest target unit, that is, the target point. The position information of each pixel is the position information of the pixel corresponding to the spatial point. The pixel value of the pixel represents the height value. In this way, the height map of the image frame can be obtained. Since the height map is obtained based on the point cloud in the world coordinate system, even image data from different cameras can be mapped to the corresponding height map.

[0134] In some specific scenarios, such as uneven ground scenes or uneven containers such as granaries, in order to improve the accuracy of measurement, this embodiment can respectively obtain depth maps or point cloud images of the acquisition scene and the measurement scene, wherein the acquisition scene is the scene before the target to be measured is placed in the scene, and the measurement scene is the scene after the target to be measured is acquired.

[0135] Perform differential modeling based on the height maps obtained from the acquisition scene and the measurement scene. Figure 4 As shown, Figure 4 A schematic diagram of scene acquisition and differential modeling. Scene acquisition is performed at time T0, and the acquisition scene information is saved. The target to be measured is placed at time Tn, and the information of the target to be measured and the acquisition scene is saved. The height value of each pixel point on the height map is differentially operated, that is, the height value h of the pixel point (u, v) in the height map at time Tn is Tn (u,v) minus the height h of the pixel (u,v) in the height map at time T0 T0 (u,v), then the height value h of the pixel point (u,v) is obtained D (u,v) only retains the information of the target to be measured, which is conducive to more accurate measurement of three-dimensional information.

[0136] In another application scenario, for example, in the same measurement scenario, a first target to be measured is stacked in the first scene, and a second target to be measured is stacked on the basis of the first target to be measured in the second scene. Similarly, differential modeling can also be performed based on the height maps obtained from the first scene and the second scene.

[0137] In order to facilitate the modeling of height maps for three-dimensional information measurement, for each frame of height map, each pixel in the height map also corresponds to a height table for storing the historical information of the pixel. Figure 5 As shown, Figure 5 A schematic diagram of the height table of a pixel in the height map. Figure 1 ,high Figure 2 , ... The time sequence of the height map n increases successively, and the height table is updated with the time sequence. For example, the height table includes:

[0138] A height array used to record the historical height values ​​of the pixel in each height map frame.

[0139] A status array used to record whether the historical height value of the pixel in each height map frame is valid, and

[0140] Header array used to record the historical header height value of the pixel point;

[0141] The header height value is used to represent the currently updated height value of the pixel in the height map. The height values ​​in the height array are arranged in ascending order, so that when the height value of the pixel is greater than or equal to 0, it is valid, otherwise it is invalid. In order to make the array capacity and storage space controllable, the height array and the state array are circular arrays.

[0142] Step 304: dynamically update the height value of each pixel based on the multi-frame height map to achieve height map modeling for three-dimensional information measurement.

[0143] See also Figure 6 As shown, Figure 6 This is a schematic diagram of a process for updating a height map based on a multi-frame height map in this embodiment. For any pixel point in the first frame height map,

[0144] Step 3041, determine whether the current height value of the pixel is valid.

[0145] When the current height value is greater than or equal to 0, that is, valid and there is no self-occlusion, step 3042 is executed. When the current height value is valid and there is self-occlusion, the historical height values ​​greater than or equal to the current height value in the height array are deleted, and the current height value is stored at the deletion position, and its state information is marked as valid in the state array, and the current height value is stored in the header array as a new header position;

[0146] When the current height value is less than 0, it is determined that the current height value is invalid, and step 3043 is executed.

[0147] Step 3042, judging whether the table header height value is valid according to the table header height value,

[0148] If valid, compare the current height value with the height value at the current header. If the current height value is greater than the height value at the current header, store the current height value as the new header height value in the header array of the height table of the pixel point. Otherwise, delete the historical height values ​​in the height array that are greater than or equal to the current height value, store the current height value at the deletion point, mark its status information as valid in the status array, and store the current height value as the new header position in the header array.

[0149] If invalid, the current altitude value is stored as the new header altitude value in the header array of the altitude table of the pixel point, so as to update the altitude table in time and reduce the accumulated error. The current altitude value is recorded in the altitude array and its status information is marked as valid.

[0150] Step 3043, traverse the header height values ​​in the pixel height table in reverse order of time sequence, and determine whether the traversed current history header height value is valid.

[0151] If it is valid, determine whether the current history header height value is blocked by the current height map, that is, determine whether there is self-occlusion.

[0152] If yes, the traversal is terminated, and the current history table header height value is used as the current height value, so as to retain the current history table header height value, and step 3044 is executed.

[0153] Otherwise, delete the current historical header height value, return to step 3043, and traverse the previous historical header height value in the header array until the header height values ​​in the header array are traversed. If the obscured header height value is still not detected when the header height values ​​in the header array are traversed, record the current height value in the height array, mark the current height value as invalid in the state array, and then execute step 3044.

[0154] If invalid, return to step 3043 to traverse the previous historical header height value in the header array.

[0155] Step 3044, traverse the height value of the pixel in the next height map, return to step 3041, until the traversal of the pixel in all height maps is completed, and update the height value of the pixel according to the header height value.

[0156] Steps 3041 to 3044 are repeatedly performed until the height values ​​of all pixels in the height map are updated, thereby obtaining an updated height map to achieve height map modeling.

[0157] To facilitate understanding of the height map modeling process, the following example is given.

[0158] See also Figure 7 As shown, Figure 7 This is a schematic diagram of the modeling height map in the process of the target being unobstructed, obstructed, and then unobstructed as time increases. In the figure, the dotted line represents the camera line of sight. At time T1, the target is not self-occluded. At time T2, the target is self-occluded. At time T3, the target is not self-occluded. Among them, the imaging height map is the height map before modeling, that is, the height map that is not dynamically updated, and the modeling height map is the height map after modeling, that is, the height map after dynamic update.

[0159] Assuming that the height value of the height table header of a certain pixel point of the target is invalid before time T1, the height value h1 at time T1 can be added to the height array and used as the header height value; because the height value h1 is added at time T1, the header height value is valid, then at time T2, the current height value h2 is larger than the height value h1 at the current header, then the current height value h2 is added to the corresponding position in the height array from small to large, and the current height value h2 is recorded in the header array as the header height value; at time T3, since the high-rise box at time T2 was taken away, the current height value h1 is smaller than the header height value h2, so the height values ​​h2 and h1 in the height table that are greater than or equal to the current height value h1 are deleted, and the height value h1 is added to the corresponding position in the height array from small to large in the height array, and the current height value h1 is used as the header height value.

[0160] If the current height value is invalid, determine whether the state of the header height value is valid. If valid, determine whether the header height value is self-occluded, that is, whether it is blocked by the current height map.

[0161] If it is blocked, the height of the historical goods is retained and the previous historical header height value is traversed. For example, at time T2, a cargo is placed to block the historical goods. The black area in the imaging height map is the missing part due to the blockage. The height table of the corresponding pixel points of this part remains unchanged. The height arrays in the height table of the corresponding pixel points of the blocked part are h1 and h2. The header height value h2 is the height of the blockage. In this way, the height information of the corresponding pixel points of the blocked and missing parts can be retained.

[0162] If it is not blocked, delete the header height value, for example, see Figure 8 As shown, corresponding to the situation where the unobstructed goods are taken away, at time T7, the height of box 5 and box 6 is not obstructed and they are taken away, the header array of the pixel points corresponding to box 5 will be updated from {0,h2,h3,h5} to {0,h2,h3}, and the header array of the pixel points corresponding to box 6 will be updated from {0,h6} to {0}.

[0163] In this embodiment, the header array may also be used to indicate the position information of the currently updated height value of the pixel point in the height array, and the height value corresponding to the position information is the header height value.

[0164] See also Fig. 9 As shown, Fig. 9 This is a schematic diagram of detecting whether the height value is blocked by the current height map in this embodiment. It includes:

[0165] Step 901, converting the space points corresponding to each pixel point and its height value in the current height map into space points in the polar coordinate system to obtain the polar coordinate system position information of the current space point,

[0166] There are two steps to convert the spatial point corresponding to the pixel point in the height map to the position point of the polar coordinate system. First, the spatial point corresponding to the pixel point of the height map is a three-dimensional spatial point in the world coordinate system, which needs to be converted to the spatial point of the camera's Cartesian coordinate system through the camera's external parameters; then the Cartesian coordinate system is converted to the point of the polar coordinate system.

[0167] Polar coordinates, also known as spherical coordinates, are a coordinate system that describes points in three-dimensional space. It uses radius r, polar angle θ, and azimuth φ to represent the position of a point. The spherical coordinate transformation formula can convert spherical coordinates to rectangular coordinates (Cartesian coordinates) or convert rectangular coordinates to spherical coordinates.

[0168] See also Fig.10 As shown, Fig.10 is a schematic diagram of the polar coordinate system. Assume that the three coordinates of point P in three-dimensional space are (r, θ, φ), where:

[0169] r is the distance from the origin to point P, 0≤r;

[0170] θ is the angle between the line from the origin to point P and the positive z-axis, 0≤θ≤π;

[0171] φ is the angle between the projection line of the line from the origin to point P on the xy-plane and the positive x-axis, 0≤φ<2π.

[0172] The conversion formula from rectangular coordinates to spherical coordinates is as follows:

[0173]

[0174] θ = arccos(z / r)

[0175] φ = artan(y, x)

[0176] Among them, arccos represents the inverse cosine function, and artan represents the inverse tangent function.

[0177] Step 902: for any height value to be detected, the space points corresponding to the historical height values ​​of the pixel point where the height value to be detected is located are converted into space points in the polar coordinate system to obtain the polar coordinate system position information of the historical space point of the pixel point.

[0178] There is no strict order between the above steps 901 and 902 and they can be executed in parallel.

[0179] As an example, in step 3043, the height value to be detected may be a header height value, and in step 3041, the height value to be detected may also be a current height value of any pixel point in the current height map.

[0180] Step 903, setting a traversal interval with a certain value as a range, taking the azimuth and polar angle in the historical spatial point polar coordinate system position information of the pixel point as the center respectively.

[0181] As an example, the polar coordinate system position information of the historical space point may be the azimuth and polar angle of the table header height value. For example, the azimuth traversal area and the polar angle traversal area are set with the azimuth and polar angle of the polar coordinate system position information corresponding to the table header height value as the center respectively.

[0182] Step 904: traverse the polar coordinate system position information of the current spatial point according to the set traversal interval, and determine whether the difference between the radius of any historical height value of the pixel point and the radius of the traversed current height value is greater than the set distance threshold.

[0183] If the difference is greater than the set distance threshold, it is determined that the height value to be detected is blocked.

[0184] Otherwise, it is determined that the height value to be detected is not blocked.

[0185] For example, if the difference between the radius in the polar coordinate system position information corresponding to the table header height value and the radius of the traversed current height value is greater than the distance threshold, it is determined that the table header height value is self-occluded.

[0186] Step 905, return to step 904, until the traversal of the polar coordinate system position information of the current space point is completed.

[0187] As an example, to improve the processing speed, the polar coordinate system position information is stored in a two-dimensional array, one dimension represents the azimuth, the range is [0, 360], the other dimension represents the polar angle, the range is [0, 180], and the array stores the radius value. Fig.11 As shown, Fig.11 A schematic diagram of a two-dimensional array of polar coordinate position information. The row direction represents the azimuth, the column direction represents the polar angle, and the radius value of the space point corresponding to the position of the azimuth and polar angle is stored. In this way, for the same pixel point, the height map at different times can correspond to different two-dimensional arrays. These two-dimensional data at different times represent the polar coordinate position information of the historical space point of the pixel point, which is conducive to setting the traversal area based on these two-dimensional data at different times, and is also conducive to improving the efficiency of self-occlusion detection.

[0188] For example, based on the dimension where the azimuth angle and the polar angle are located, the azimuth angle traversal array and the polar angle traversal array are set with the azimuth angle and polar angle of the spatial point in the historical polar coordinate system of the pixel point as the center; according to the azimuth angle traversal array and the polar angle traversal array, the two-dimensional array where the polar coordinate system position information of the current spatial point to be traversed is located is determined, and the radius value in the two-dimensional array is traversed to compare with the radius corresponding to the height value to be detected.

[0189] See also Fig.12 As shown, Fig.12 This is a schematic diagram of the height map modeling process. The figure shows the color map, depth map, height map, self-occlusion recognition effect map, and self-occlusion completion height map in sequence.

[0190] Step 305: perform three-dimensional information measurement based on the modeled height map.

[0191] As an example, three-dimensional information includes volume, area, height,

[0192] in,

[0193] The height can be obtained directly from the height map;

[0194] The area can be obtained by integrating the pixels on the height map, which can be expressed mathematically as:

[0195] S=s×N

[0196] N is the number of pixels corresponding to the effective height, and s is the area corresponding to the pixel;

[0197] The volume can be obtained by integrating the pixels on the height map, which can be expressed mathematically as:

[0198] V = s × ∑ h i ,

[0199] ∑h i is the accumulation of the i-th valid height value, and s is the area corresponding to the pixel;

[0200] The effective height refers to the height value obtained by correcting the height value obtained from the height map according to the scene. When the bearing surface where the target to be measured is located also has a height value, for example, when the target to be measured is located on a pallet, a vehicle-borne object such as an AGV, or a transport vehicle, for example Fig.13 In a and c, the height value obtained from the height map can be corrected according to the height value of the bearing surface. For example, the height value of the bearing surface can be subtracted to obtain the effective height to avoid the three-dimensional measurement result of the target to be measured being too large, which is beneficial to improve the accuracy of the three-dimensional measurement.

[0201] In this embodiment, the height map can be processed as conveniently as an image frame. Compared with the three-dimensional information measurement method based on point cloud three-dimensional space, it is conducive to improving the efficiency of height information processing and reducing interference to improve the accuracy of height information; by performing self-occlusion detection on the height value and using the historical height value to complete the height value of the pixel point with self-occlusion, the height value of the target to be measured is complete, the accuracy of the measurement is improved, and the unobstructed pixel point is updated with the current height, which is conducive to reducing the cumulative error. In the process of self-occlusion detection of the height value, the detection is performed through the position relationship in the polar coordinate system, which avoids false detection and improves the accuracy of self-occlusion, thereby helping to improve the accuracy of the self-occluded pixel point height value completion; optionally, by obtaining the height map in a differential manner, it is conducive to improving the adaptability to the scene and the measurement accuracy.

[0202] See also Fig.13 As shown, Fig.13 This is an example of an application scenario of the target three-dimensional information measurement method of the present embodiment; the target three-dimensional information measurement method of the present embodiment can be used for overall measurement under a single camera, as shown in Figure a, and can also be used for non-overall measurement, for example, bulk package stacking, as shown in Figure b, and can also be applied to whole pallet measurement with multiple cameras, as shown in Figure c, and can also be used for duty cycle detection, stacking detection, etc. of storage space, as shown in Figure d.

[0203] See also Fig.14 As shown, Fig.14 This is a schematic diagram of a target three-dimensional information measurement device according to an embodiment of the present application. The device includes:

[0204] An image acquisition module is used to acquire multiple time series of image data for representing the distance between the target to be measured and the image acquisition device.

[0205] The height map acquisition module is used to acquire a height map for representing the height value of each pixel point at each time sequence based on the acquired time sequence image data.

[0206] The height map modeling module is used to traverse each height map in time sequence, and dynamically update the height value of each pixel point by using the height value of each pixel point in each height map, so that the height value of each pixel point in the missing area caused by the self-occlusion between the targets to be measured is supplemented, and the height value of each pixel point in the unoccluded area is maintained, so as to obtain an updated height map.

[0207] The measurement module is used to measure the three-dimensional information of the target to be measured based on the updated height map.

[0208] See also Fig.15 As shown, Fig.15Another schematic diagram of a target three-dimensional information measuring device according to an embodiment of the present application. The device comprises: a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to implement the steps of the target three-dimensional information measuring method according to an embodiment of the present application. As an example, the device is a visual measurement camera.

[0209] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0210] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0211] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the target three-dimensional information measurement method of the embodiment of the present application are implemented.

[0212] As for the apparatus / network-side device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0213] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0214] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for measuring three-dimensional information of a target, characterized in that: The method includes: Acquire multiple time series of image data for representing the distance between the target to be measured and the image acquisition device, Based on the acquired time-series image data, a height map for representing the height value of each pixel point in each time sequence is obtained. A height table for recording the historical information of each height map of the pixel is established for each pixel point, and the height table includes: a height array for recording the historical height values ​​of the pixel point, a status array for recording the status information representing whether the historical height values ​​of the pixel point are valid, and a header array for recording the historical header height values, wherein each height value corresponds to the status information, and the header height value is used to represent the currently updated height value of the pixel point in the height map, Each height map is traversed sequentially, and the height value of each pixel in each height map is dynamically updated by using the height value of each pixel, so that the height value of each pixel in the missing area caused by the self-occlusion between the targets to be measured is supplemented, and the height value of each pixel in the unoccluded area is maintained, and the updated height map is obtained. Based on the updated height map, three-dimensional information measurement of the target to be measured is performed; in, The step of traversing each height map in time sequence and dynamically updating the height value of each pixel point in each height map by using the height value of each pixel point in each height map comprises: For the current height value of any pixel in the current height map traversed, If the current height value of the pixel is valid, determine whether the current height value is self-occluded. In the case of self-occlusion of the current height value, delete the height values ​​in the height array that are greater than or equal to the current height value, then add the current height value to the height array, record the current height value as a valid state in the state array, and record it in the header array as the header height value; In the case where the current height value does not have self-occlusion, the state array in the pixel height table is used to determine whether the header height value in the pixel height table is valid. If the header height value is valid, compare the current height value with the header height value. If the current height value is greater than the header height value, the current height value is recorded in the header array as the header height value, the current height value is recorded in the height array, and the status information of the current height value is recorded in the status array. If the current height value is not greater than the header height value, delete the height values ​​in the height array that are greater than or equal to the current height value, then add the current height value to the height array, record the current height value as a valid state in the status array, and record it in the header array as the header height value. If the table header height value is invalid, the current height value is recorded in the height array, and the current height value is recorded as a valid state in the state array, and is recorded in the table header array as the table header height value; The header height value in the current height table of each pixel point is used as the updated height value of each pixel point.

2. The measuring method according to claim 1, characterized in that: The method of completing the height value of each pixel in the missing area caused by the self-occlusion between the targets to be measured, and maintaining the height value of each pixel in the unoccluded area, includes: Check whether there is self-occlusion in the height value of each pixel in the current height map traversed. If there is self-occlusion, the height value of each pixel with self-occlusion is completed based on the current height map, and the height value of each pixel without self-occlusion is updated using the height value of each pixel in the current height map. The detection is repeatedly performed to determine whether the height values ​​of each pixel point in the current height map traversed are blocked by the target to be measured until all height maps are traversed.

3. The measuring method according to claim 2, characterized in that: The step of detecting whether the height values ​​of each pixel point in the traversed current height map are self-occluded by the target to be measured includes: Convert the spatial points corresponding to each pixel point and its height value in the current height map into spatial points in the polar coordinate system to obtain the polar coordinate position information of the current spatial point. For the height value of any pixel point, the spatial point corresponding to the pixel point and its historical height value is converted into a spatial point in the polar coordinate system to obtain the polar coordinate system position information of the historical spatial point of the pixel point. The historical height value is the height value of the pixel point in each historical height map whose time sequence is earlier than the time sequence of the current height map. The polar coordinate system position information includes: azimuth, polar angle, and radius. Based on the azimuth and polar angle in the polar coordinate system position information of the historical spatial point, the traversal interval of the pixel point is set. According to the traversal interval of the pixel point, traverse the polar coordinate system position information of the current spatial point, and determine whether the difference between the radius of any historical spatial point of the pixel point and the radius of the traversed current spatial point is greater than the set distance threshold. If the difference is greater than the distance threshold, it is determined that the height value of the pixel point in the current height map is self-occluded.

4. The measuring method according to claim 3, characterized in that: The method of dynamically updating the height value of each pixel point by using the height value of each pixel point in each height map further includes: If the current height value of the pixel is invalid, the historical height values ​​of the pixel are traversed in reverse order of time to determine whether the traversed current historical header height value is valid. If it is valid, check whether the traversed current historical height value is self-occluded. If the current historical height value is self-occluded, the traversal is terminated and the current historical height value is used as the current height value; if the current historical height value is not self-occluded, the current historical height value is deleted, the next historical height value of the pixel is traversed, and the step of detecting whether the traversed current historical height value is self-occluded is returned until a historical height value with self-occlusion is detected; if no historical height value with self-occlusion is detected after the historical height value traversal of the pixel is completed, the current height value is marked as invalid. Traverse the height value of the pixel in the next height map adjacent to the current height map, return to execute the step of the current height value of any pixel in the current height map, until all height maps of the pixel are traversed, and obtain the updated height value of the pixel.

5. The measuring method according to claim 4, characterized in that: Before the detection of whether the traversed current historical height value is self-occluded, the method further includes: Determine whether the current historical height value is valid. If the current historical height value is valid, the step of detecting whether the traversed current historical height value is self-occluded is performed. If the current historical height value is invalid, then traverse the previous historical height value of the pixel point, and return to execute the step of detecting whether the traversed current historical height value has self-occlusion.

6. The measuring method according to claim 5, characterized in that: The height values ​​in the height array are arranged from small to large, and traversing the historical height values ​​of the pixel point includes: traversing the header height values ​​in the header array in the height table of the pixel point; The determining whether the traversed current history table header height value is valid further includes: If the header height value is valid, then if the header height value is not blocked, delete the header height value from the header array. If the header height value is invalid, get the previous header height value in the header array; The three-dimensional information measurement of the target to be measured based on the updated height map includes: Based on the updated height value of each pixel point, at least one of the height, occupied area and volume of the target to be measured is determined.

7. The measuring method according to claim 6, characterized in that: The step of converting the spatial point corresponding to the pixel point and its historical height value into a spatial point in a polar coordinate system includes: The space point corresponding to the height value of the table header and its pixel point is converted into a space point in the polar coordinate system to obtain the polar coordinate system position information corresponding to the height value of the table header; The step of setting the traversal interval of the pixel point based on the azimuth and polar angle in the polar coordinate system position information of the historical spatial point includes: Taking the azimuth and polar angle in the polar coordinate system position information corresponding to the header height value as the center, respectively, the azimuth traversal area and polar angle traversal area of ​​the pixel point are set; The determining whether a difference between the radius of any historical spatial point of the pixel point and the radius of the traversed current spatial point is greater than a set distance threshold includes: Subtract the radius of the current space point traversed from the radius in the polar coordinate system position information corresponding to the header height value to obtain the difference. When the difference is greater than the distance threshold, it is determined that the header height value is blocked by the current height map.

8. The measuring method according to claim 7, characterized in that: The polar coordinate system position information is stored in a two-dimensional array, one dimension represents the azimuth, the other dimension represents the polar angle, and the radius is stored in the array of the azimuth and polar angle corresponding to the radius. The step of setting the azimuth traversal area and polar angle traversal area of ​​the pixel point with the azimuth and polar angle of the spatial point in the historical polar coordinate system of the pixel point as the center respectively includes: Based on the dimension where the azimuth angle is located and the dimension where the polar angle is located, the azimuth angle traversal array and the polar angle traversal array are set with the azimuth angle and polar angle of the spatial point in the historical polar coordinate system of the pixel point as the center; The traversing of the polar coordinate system position information of the current space point includes: According to the azimuth traversal array and the polar angle traversal array, determine the two-dimensional array where the polar coordinate system position information of the current space point to be traversed is located. Iterate over the radius values ​​in this two-dimensional array.

9. The measuring method according to claim 1, characterized in that: The step of acquiring image data at each time sequence for representing the distance between the target to be measured and the image acquisition device includes: Acquire at least one frame of first image data and at least one frame of second image data respectively, wherein the first image data does not include the target image to be measured or includes the first target image to be measured, the second image data includes the second target image to be measured, and the first target image to be measured is different from the second target image to be measured; The step of obtaining a height map for representing the height value of each pixel point at each time sequence based on the obtained time sequence image data includes: acquiring a first height map based on the first image data, acquiring a second height map based on the second image data, For each pixel in the height map, a second height value of the pixel in each second height map is differentially operated with the first height value of the pixel in the first height map to obtain the height value of the pixel in each second height map.

10. A device for measuring three-dimensional information of a target, characterized in that: The device includes: An image acquisition module is used to acquire multiple time series of image data for representing the distance between the target to be measured and the image acquisition device. The height map acquisition module acquires the height map of each time sequence for representing the height value of each pixel point based on the acquired time sequence image data. A height map modeling module establishes a height table for each pixel point for recording the historical information of each height map of the pixel point, wherein the height table includes: a height array for recording the historical height values ​​of the pixel point, a state array for recording the state information for characterizing whether the historical height values ​​of the pixel point are valid, and a header array for recording the historical header height value, wherein each height value corresponds to the state information, and the header height value is used to characterize the currently updated height value of the pixel point in the height map; each height map is traversed sequentially, and the height value of each pixel point in each height map is dynamically updated by using the height value of each pixel point, so that the height value of each pixel point in the missing area caused by the self-occlusion between the targets to be measured is completed, and the height value of each pixel point in the unoccluded area is maintained, so as to obtain an updated height map, The measurement module measures the three-dimensional information of the target to be measured based on the updated height map; in, The step of traversing each height map in time sequence and dynamically updating the height value of each pixel point in each height map by using the height value of each pixel point in each height map comprises: For the current height value of any pixel in the current height map traversed, If the current height value of the pixel is valid, determine whether the current height value is self-occluded. In the case of self-occlusion of the current height value, delete the height values ​​in the height array that are greater than or equal to the current height value, then add the current height value to the height array, record the current height value as a valid state in the state array, and record it in the header array as the header height value; In the case where the current height value does not have self-occlusion, the state array in the pixel height table is used to determine whether the header height value in the pixel height table is valid. If the header height value is valid, compare the current height value with the header height value. If the current height value is greater than the header height value, the current height value is recorded in the header array as the header height value, the current height value is recorded in the height array, and the status information of the current height value is recorded in the status array. If the current height value is not greater than the header height value, delete the height values ​​in the height array that are greater than or equal to the current height value, then add the current height value to the height array, record the current height value as a valid state in the status array, and record it in the header array as the header height value. If the table header height value is invalid, the current height value is recorded in the height array, and the current height value is recorded as a valid state in the state array, and is recorded in the table header array as the table header height value; The header height value in the current height table of each pixel point is used as the updated height value of each pixel point.

Images