Rapid labeling method for container lockhole RGBD data

Abstract

The invention relates to a rapid labeling method for container lockhole RGBD data, and aims to solve the technical problems that the labeling result of the existing data labeling method is easy to have errors, the labeling efficiency is low, and the labeling cost is high. According to the rapid labeling method for container lockhole RGBD data, the corresponding pixel point in the filtered depth image is calculated through any pixel point on the container lockhole in the color image, and then the pixel point in the depth image is obtained by adopting an edge point searching algorithm based on Gaussian attenuation. Determining the minimum edge pixel point and the maximum edge pixel point of the container lockhole in the filtered depth image in the axial direction and in the axial direction; and determining annotation data according to the minimum edge pixel point and the maximum edge pixel point, thereby completing rapid annotation of the container lockhole RGBD data.

Description

Technical Field

[0001] This invention relates to an RGBD data annotation method, specifically a rapid annotation method for RGBD data of container keyholes. Background Technology

[0002] The RGBD (color image and depth image) data annotation of container keyholes is the key data used to provide training and inference for the subsequent container keyhole recognition model. It is an indispensable part of the training and inference of the container keyhole recognition model, and high-quality RGBD data annotation of container keyholes directly affects the accuracy of model recognition.

[0003] In existing technologies, the annotation of RGBD data of container lock holes is usually done manually or using self-annotation software. When using manual annotation, different annotators may have different understandings of the same task, which can lead to errors in the annotation results and low annotation efficiency. Using existing self-annotation software for image annotation requires software purchase or usage fees, which increases the annotation cost of the project. Summary of the Invention

[0004] The purpose of this invention is to solve the technical problems of existing RGBD data annotation methods for container lock holes, which are prone to errors, have low annotation efficiency, and high annotation costs, and to provide a fast annotation method for RGBD data of container lock holes.

[0005] To achieve the above objectives, the technical solution provided by this invention is as follows: A rapid annotation method for RGBD data of container keyholes, characterized by the following steps: S1. Acquire RGBD data of container lock holes using a depth camera, wherein the RGBD data includes depth images and color images; S2. Perform bilateral filtering on the depth image to obtain the filtered depth image; S3. Based on any pixel on the lock hole of the container in the color image, calculate its corresponding pixel in the filtered depth image using the intrinsic and extrinsic parameters of the depth camera. S4. Based on the corresponding pixel points determined in step S3, an edge point search algorithm based on Gaussian attenuation is used to determine the locations of the container lock holes in the filtered depth image. axial direction and in The minimum and maximum edge pixels along the axis; S5. Based on the container lock holes, respectively... axial direction and in The minimum and maximum edge pixels along the axis are used to determine the annotation data. Based on the determined annotation data, the RGBD data of the container lock hole is quickly annotated.

[0006] Furthermore, the process of step S4 is as follows: S4.1. Using the corresponding pixel point determined in step S3 as the reference point, and centering on the reference point, determine multiple detection directions, including the four directions of up, down, left, and right. S4.2, Taking any one of the plurality of detection directions as the detection direction, perform the first... The number of tests is determined by the number of tests. Determine step size Based on step size Determine the pixel to be detected, and let ; S4.3. Calculate the gradient value of the pixel to be detected determined in step S4.2 using the Sobel operator, and determine whether the gradient value is greater than the preset edge gradient value. If yes, the pixel to be detected is the edge pixel in the detection direction, and proceed to step S4.5; otherwise, proceed to step S4.4. S4.4, Order And based on the number of tests Determine step size , step length Perform an update based on the updated step size. Using the current pixel to be detected as the center, continue to determine the next pixel to be detected along the current detection direction, and then return to step S4.3; S4.5. Using the reference point described in step S4.1 as the center, and employing the methods of steps S4.2 to S4.4, determine the edge pixels in the remaining detection directions. The edge pixels in the left detection direction are those in... The smallest edge pixel in the axial direction, the edge pixel in the right detection direction is the smallest edge pixel in the axial direction. The maximum edge pixel point along the axial direction, the edge pixel point in the upper detection direction is the one in... The smallest edge pixel in the axial direction, the edge pixel in the downward detection direction is the smallest edge pixel in the axial direction. The largest edge pixel in the axial direction.

[0007] Furthermore, in steps S4.2 and S4.4, based on the number of detections... Determine step The formula is as follows: ; In the formula, The maximum step size in the detection direction is determined empirically. This represents the attenuation rate.

[0008] Further, in step S2, the filtered depth image is defined as: ; In the formula, For any pixel point Centered Size range, where N is a positive integer. In order to be in Pixels within the range, These are weighting coefficients. This is the original depth map; where, The calculation formula is as follows: ; In the formula, Weighting coefficients for the spatial domain The calculation formula is as follows: ; In the formula, For spatial domain parameters; In the formula, Pixel range weighting coefficient The calculation formula is as follows: ; In the formula, This refers to the pixel value range parameter.

[0009] Further, in step S4.3, the process of calculating the gradient value of the pixel to be detected determined in step S4.2 using the Sobel operator is as follows: S4.3.1 Calculate the pixel to be detected in gradient in the axial direction : ; S4.3.2 Calculate the pixel to be detected in gradient in the axial direction : ; S4.3.3 Calculate the gradient value of the pixel to be detected. : ; Furthermore, the process of step S5 is as follows: According to the container lock holes axial direction and in The minimum and maximum edge pixels along the axis are used to construct a rectangular outer frame. The data within the rectangular outer frame is the labeled data of the container keyhole RGBD data. The labeled data is then labeled with the corresponding target tags to complete the rapid labeling of the container keyhole RGBD data.

[0010] The beneficial effects of this invention are: 1. This invention provides a fast annotation method for RGBD data of container keyholes. By taking advantage of the consistent depth of container keyhole images, an edge point search algorithm based on Gaussian decay is used to quickly find the rectangular outer bounding box of the container keyhole, thereby completing the fast annotation of the RGBD data of container keyholes. This solves the problems of existing annotation methods, such as easy errors in annotation results, low annotation efficiency, and high annotation cost. 2. The present invention provides a fast annotation method for RGBD data of container keyholes. By performing bilateral filtering on the depth image, outliers on the depth map are reduced, improving the accuracy of subsequent edge region detection and further enhancing the accuracy of data annotation. 3. The present invention provides a rapid annotation method for RGBD data of container lock holes. By selecting any point on the color map and combining it with the regional information on the depth map, the rectangular outer frame of the container lock hole is detected, converting the original multi-point annotation method into single-point annotation, thereby improving the efficiency of data annotation. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating an embodiment of a rapid annotation method for RGBD data of container lock holes according to the present invention. Figure 2 This is a schematic diagram of the rectangular outer frame in step S5 of an embodiment of a method for rapid annotation of RGBD data of container lock holes according to the present invention. Detailed Implementation

[0012] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0013] like Figure 1 As shown in the figure, this embodiment presents a rapid annotation method for RGBD data of container lock holes. Through user interaction, it effectively improves the efficiency and accuracy of data annotation while reducing annotation costs. The method includes the following steps: S1. Acquire RGBD data of container lock holes using a depth camera. The RGBD data includes depth images and color images. In this embodiment, approximately 300 RGBD data images of container lock holes are acquired.

[0014] S2. To address potential noise in the depth image, perform bilateral filtering on the depth image to remove noise while preserving edges, resulting in a filtered depth image. The filtered depth image is defined as: ; In the formula, For any pixel point Centered Size range, where N is a positive integer. In order to be in Pixels within the range, The weighting coefficients are calculated using the following formula: ; In the formula, the spatial domain weighting coefficients The calculation formula is as follows: ; In the formula, For spatial domain parameters; In the formula, the pixel value range weighting coefficient The calculation formula is as follows: ; In the formula, This refers to the pixel value range parameter.

[0015] S3. Based on any pixel on the lock hole of the container in the color image, find the corresponding pixel on the depth map, and calculate its corresponding pixel in the depth image using the intrinsic and extrinsic parameters of the depth camera.

[0016] S4. Based on the corresponding pixel points determined in step S3, an edge point search algorithm based on Gaussian attenuation is used to determine the location of the container lock holes. axial direction and in The minimum and maximum edge pixels along the axis; the specific process is as follows: S4.1. Using the corresponding pixel point determined in step S3 as the reference point, and centering on the reference point, determine multiple detection directions, including the four directions of up, down, left, and right. S4.2, Taking any one of the plurality of detection directions as the detection direction, perform the first... The second detection, the The number of tests, based on the number of tests. Determine step size Based on step size Determine the pixel to be detected, where, ; Based on the number of tests Determine step The formula is as follows: ; In the formula, The maximum step size for manually inputting the detection direction. As the decay rate increases, the search step size should be gradually reduced to ensure the accuracy of the search.

[0017] S4.3 Calculate the gradient value of the pixel to be detected determined in step S4.3 using the Sobel operator, and determine whether the gradient value is greater than the preset edge gradient value. , As a hyperparameter, in this embodiment, If the value is 3, then the pixel to be detected is an edge pixel, and step S4.5 is executed; otherwise, step S4.4 is executed. The process of calculating the gradient value of the pixel to be detected determined in step S4.3 using the Sobel operator is as follows: S4.3.1 Calculate the pixel to be detected in gradient in the axial direction : ; S4.3.2 Calculate the pixel to be detected in gradient in the axial direction : ; S4.3.3 Calculate the gradient value of the pixel to be detected. : ; S4.4, Order , step length Perform an update based on the updated step size. Using the current pixel to be detected as the center, continue to determine the next pixel to be detected along the current detection direction, and then return to step S4.3; S4.5. Using the reference point described in step S4.1 as the center, take any of the remaining detection directions as the detection direction and return to step S4.2 until the edge pixels in the four detection directions (up, down, left, and right) are obtained. The edge pixels in the left detection direction are the edge pixels in the left detection direction. The smallest edge pixel in the axial direction, the edge pixel in the right detection direction is the smallest edge pixel in the axial direction. The maximum edge pixel point along the axial direction, the edge pixel point in the upper detection direction is the one in... The smallest edge pixel in the axial direction, the edge pixel in the downward detection direction is the smallest edge pixel in the axial direction. The largest edge pixel in the axial direction.

[0018] S5. Based on the container lock holes, respectively... axial direction and in The minimum and maximum edge pixels along the axis are used to construct a rectangular outer frame. The data within this rectangular outer frame is the labeled data for the RGBD data of the container keyhole. For example... Figure 2 As shown, The coordinates of the center point are determined by the reference point. The step size taken in the left detection direction. The step size taken in the right detection direction. The step size taken in the upward detection direction. Let be the step size traversed in the lower detection direction, from which we can obtain: exist The minimum edge pixel point along the axis is defined as follows: ; exist The maximum edge pixel point along the axis is defined as follows: ; exist The maximum edge pixel point along the axis is defined as follows: ; exist The minimum edge pixel point along the axis is defined as follows: ; S6. Add the corresponding target labels to the labeled data of the container keyhole RGBD data found in step S5, thereby completing the rapid labeling of the container keyhole RGBD data.

[0019] This embodiment proposes a fast annotation method for RGBD data of lock holes on containers. It acquires a depth map and color map aligned to the current scene using a depth camera. Noise on the depth map surface is removed using filtering. A point is arbitrarily selected on the color map and directly mapped to its corresponding pixel on the depth map. The edge information of the region is obtained by calculating the gradient information of the depth map. Using this point as the center, the annotation grows horizontally and vertically until a certain condition is no longer met. The maximum and minimum coordinates of the growth point in the horizontal and vertical directions form the bounding box of the target. Finally, the category of the target is selected, completing the fast image annotation. This fast annotation method effectively improves the efficiency of existing annotation methods. By selecting a single point, target detection data annotation can be completed directly according to requirements. Compared to bounding box or rule-based methods, it can efficiently complete image annotation and reduce project costs.

Claims

1. A rapid annotation method for RGBD data of container lock holes, characterized in that, Includes the following steps: S1. Acquire RGBD data of container lock holes using a depth camera, wherein the RGBD data includes depth images and color images; S2. Perform bilateral filtering on the depth image to obtain the filtered depth image; S3. Based on any pixel on the lock hole of the container in the color image, calculate its corresponding pixel in the filtered depth image using the intrinsic and extrinsic parameters of the depth camera. S4. Based on the corresponding pixel points determined in step S3, an edge point search algorithm based on Gaussian attenuation is used to determine the locations of the container lock holes in the filtered depth image. axial direction and in The minimum and maximum edge pixels along the axis; S5. Based on the container lock holes, respectively... axial direction and in The minimum and maximum edge pixels along the axis are used to determine the annotation data. Based on the determined annotation data, the RGBD data of the container lock hole is quickly annotated.

2. The method for rapid annotation of RGBD data of container lock holes according to claim 1, characterized in that, The process of step S4 is as follows: S4.

1. Using the corresponding pixel point determined in step S3 as the reference point, and centering on the reference point, determine multiple detection directions, including the four directions of up, down, left, and right. S4.2, Taking any one of the plurality of detection directions as the detection direction, perform the first... The number of tests is determined by the number of tests. Determine step size Based on step size Determine the pixel to be detected, and let ; S4.

3. Calculate the gradient value of the pixel to be detected determined in step S4.2 using the Sobel operator, and determine whether the gradient value is greater than the preset edge gradient value. If yes, the pixel to be detected is the edge pixel in the detection direction, and proceed to step S4.5; otherwise, proceed to step S4.

4. S4.4, Order And based on the number of tests Determine step size , step length Perform an update based on the updated step size. Using the current pixel to be detected as the center, continue to determine the next pixel to be detected along the current detection direction, and then return to step S4.3; S4.

5. Using the reference point described in step S4.1 as the center, and employing the methods of steps S4.2 to S4.4, determine the edge pixels in the remaining detection directions. The edge pixels in the left detection direction are those in... The smallest edge pixel in the axial direction, the edge pixel in the right detection direction is the smallest edge pixel in the axial direction. The maximum edge pixel point along the axial direction, the edge pixel point in the upper detection direction is the one in... The smallest edge pixel in the axial direction, the edge pixel in the downward detection direction is the smallest edge pixel in the axial direction. The largest edge pixel in the axial direction.

3. The method for rapid annotation of RGBD data of container lock holes according to claim 2, characterized in that: In steps S4.2 and S4.4, based on the number of detections... Determine step size The formula is as follows: ； In the formula, The maximum step size in the detection direction is determined empirically. This represents the attenuation rate.

4. The rapid annotation method for RGBD data of container lock holes according to claim 3, characterized in that: In step S2, the filtered depth image Defined as: ； In the formula, For any pixel point Centered Size range, where N is a positive integer. In order to be in Pixels within the range, These are weighting coefficients. This is the original depth map; where, The calculation formula is as follows: ； In the formula, The spatial domain weighting coefficients are calculated using the following formula: ； In the formula, For spatial domain parameters; In the formula, The weighting coefficients for the pixel value range are calculated using the following formula: ； In the formula, This refers to the pixel value range parameter.

5. A method for rapid annotation of RGBD data of container lock holes according to claim 4, characterized in that: In step S4.3, the process of calculating the gradient value of the pixel to be detected determined in step S4.2 using the Sobel operator is as follows: S4.3.1 Calculate the pixel to be detected in gradient in the axial direction : ； S4.3.2 Calculate the pixel to be detected in gradient in the axial direction : ； S4.3.3 Calculate the gradient value of the pixel to be detected. : 。 6. A method for rapid annotation of RGBD data of container lock holes according to claim 5, characterized in that, The process of step S5 is as follows: According to the container lock holes axial direction and in The minimum and maximum edge pixels along the axis are used to construct a rectangular outer frame. The data within the rectangular outer frame is the labeled data of the container keyhole RGBD data. The labeled data is then labeled with the corresponding target tags to complete the rapid labeling of the container keyhole RGBD data.

Images