Method and device for determining damaged area of a ship steel plate

Abstract

The application provides a kind of ship steel plate damage area determination method and device, it is related to industrial surface damage detection and area measurement field.The method comprises: obtaining the image to be detected of the surface to be measured collected by the shooting device, and the image to be detected includes the color image of the surface to be measured and the depth image of the surface to be measured;The target region segmentation processing is carried out to the color image of the surface to be measured, and the target pixel level mask of the target damage region is obtained;Based on the target pixel level mask, the target depth data set corresponding to the target damage region is extracted from the depth image of the surface to be measured;According to the target depth data set and the calibration parameters of the shooting device, the initial projection physical area of the target damage region is determined;According to the spatial geometric characteristics, the initial projection physical area is corrected, and the target surface area of the target damage region is obtained.The scheme of the application realizes the automatic quantitative calculation of the damage area of the ship steel plate.

Description

Technical Field

[0001] This invention relates to the field of industrial surface damage detection and area measurement, and particularly to a method and apparatus for determining the area of ​​damaged regions on ship steel plates. Background Technology

[0002] Coating, as a common surface treatment technology, is widely used in industrial settings such as bridges, ships, building components, and aerospace vehicles. Its main function is to provide corrosion protection, oxidation resistance, and wear resistance to the substrate. In industrial maintenance and quality inspection, the condition of the coating directly affects the operational safety and service life of equipment. Therefore, rapid and accurate detection and assessment of coating defect areas is of significant engineering importance. In the field of inspection of large steel structures such as ships, the detection methods for coating damage and corrosion areas mainly rely on manual visual inspection and instrument-assisted inspection based on physical principles. Inspectors assess the degree and area of ​​damage based on surface characteristics such as coating peeling, blistering, breakage, and substrate corrosion, and formulate corresponding maintenance strategies accordingly. However, this traditional manual inspection method is not only inefficient, labor-intensive, and costly in terms of equipment, but its accuracy is also easily affected by the subjective factors of the operator, lighting conditions, and fatigue level. When facing complex ship cabin structures, the bottom of high-altitude bridges, or dangerous areas, traditional methods often face problems such as poor accessibility and insufficient coverage. Summary of the Invention

[0003] This invention provides a method and apparatus for determining the area of ​​damaged regions on ship steel plates. It solves the problems of existing industrial applications where coating damage and corrosion areas mainly rely on manual inspection, resulting in low inspection efficiency, susceptibility to subjective factors, and difficulty in automatically quantifying the actual physical area of ​​the target damaged region.

[0004] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0005] An embodiment of the present invention proposes a method for determining the area of ​​damaged regions on ship steel plates, comprising:

[0006] Acquire the image to be detected of the surface to be tested captured by the imaging device. The image to be detected includes a time-synchronized and spatially registered color image of the surface to be tested and a depth image of the surface to be tested.

[0007] The color image of the surface to be tested is subjected to target region segmentation processing to obtain a target pixel-level mask of the target damage region. The target pixel-level mask includes at least one target damage instance mask. The target damage region includes at least one of a coating damage region and a corrosion region.

[0008] Based on the target pixel-level mask, extract the target depth dataset corresponding to the target damage region from the surface depth image to be tested;

[0009] The initial projected physical area of ​​the target damage region is determined based on the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device.

[0010] Based on the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damage region.

[0011] Embodiments of the present invention also provide a device for determining the area of ​​damaged regions in ship steel plates, comprising:

[0012] The acquisition module is used to acquire the image of the surface to be tested captured by the imaging device. The image to be tested includes a time-synchronized and spatially registered color image of the surface to be tested and a depth image of the surface to be tested.

[0013] The processing module is used to perform target region segmentation processing on the color image of the surface under test to obtain a target pixel-level mask of the target damage region. The target pixel-level mask includes at least one target damage instance mask. The target damage region includes at least one of a coating damage region and a corrosion region. Based on the target pixel-level mask, the module extracts the target depth dataset corresponding to the target damage region from the depth image of the surface under test. According to the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device, the module determines the initial projected physical area of ​​the target damage region. According to the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, the module corrects the initial projected physical area to obtain the target surface area of ​​the target damage region.

[0014] The above-described solution of the present invention has at least the following beneficial effects:

[0015] This invention achieves automated quantitative calculation of the area of ​​damaged areas on ship steel plates by jointly processing time-synchronized and spatially registered color and depth images of the surface to be tested, extracting corresponding depth data using the target pixel-level mask of the target damage area, determining the initial projected physical area based on camera calibration parameters, and further correcting the area based on the spatial geometric features of the surface where the target damage area is located. This helps to improve the consistency and efficiency of area detection and reduce the workload of manual close-range inspection. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the method for determining the present invention;

[0017] Figure 2 This is a schematic diagram of the model architecture of the preset segmentation network model of the method of the present invention;

[0018] Figure 3 This is a schematic diagram of the architecture of each module in the preset segmentation network model of the method of the present invention;

[0019] Figure 4 This is a schematic diagram of the imaging principle of the imaging device used in the method of this invention;

[0020] Figure 5 This is a schematic diagram of the template measurement area correction method of the present invention;

[0021] Figure 6 This is a schematic diagram of the target damage area calculation process in a single-category instance segmentation embodiment of the method of the present invention;

[0022] Figure 7 This is a schematic diagram of the module block of the device for determining the area of ​​damaged steel plate of a ship according to the present invention. Detailed Implementation

[0023] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0024] like Figures 1 to 6 As shown, an embodiment of the present invention provides a method for determining the area of ​​a damaged region on a ship's steel plate, comprising:

[0025] Step 11: Acquire the image to be detected of the surface to be tested captured by the imaging device. The image to be detected includes a time-synchronized and spatially registered color image of the surface to be tested and a depth image of the surface to be tested.

[0026] Step 12: Perform target region segmentation processing on the color image of the surface to be tested to obtain a target pixel-level mask of the target damage region. The target pixel-level mask includes at least one target damage instance mask. The target damage region includes at least one of a coating damage region and a corrosion region.

[0027] Step 13: Based on the target pixel-level mask, extract the target depth dataset corresponding to the target damage area from the surface depth image to be tested;

[0028] Step 14: Determine the initial projected physical area of ​​the target damage region based on the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device.

[0029] Step 15: Based on the spatial geometric features of the surface where the target damage area is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damage area.

[0030] In this embodiment, the target damage area refers to at least one of the coating damage area and the corrosion area on the surface to be measured, which require area measurement. The coating damage area and the corrosion area can be identified separately or identified as a unified target damage category. The target pixel-level mask is used to mark the pixel positions covered by the target damage area. The target pixel-level mask may include one or more target damage instance masks. When multiple target damage instance masks are included, i.e., under a mutually exclusive category labeling method, the target pixel-level masks corresponding to each category do not overlap at the pixel level. When it is necessary to determine the area of ​​the coating damage area and the area of ​​the corrosion area separately, the area can be calculated based on the target pixel-level mask corresponding to each category. When it is necessary to determine the total target damage area, the target pixel sets corresponding to each category can be summarized, or the areas of each category can be accumulated to obtain the total area of ​​the target damage area.

[0031] In this embodiment, the target region segmentation process can be performed using a preset single-class instance segmentation model or a preset multi-class segmentation model. When using the single-class instance segmentation model, one or more categories, such as coating damage and corrosion, are uniformly mapped to the target damage category, and the target damage region corresponding to the target damage category is identified, which helps reduce interference from category features between different target damage types. When using the multi-class segmentation model, coating damage and corrosion regions are identified separately, and corresponding target pixel-level masks are output respectively. This embodiment uses the target pixel-level mask to obtain the spatially registered depth image of the surface under test. The target depth dataset is extracted, and based on the target depth dataset and the calibration parameters of the imaging device, the uncorrected single-pixel projected physical area corresponding to each target pixel is determined. This area is accumulated and corrected using a geometric scaling factor to obtain the initial projected physical area of ​​the target damage region. This processing method considers the depth differences between different target pixels within the target damage region, reducing the area calculation deviation caused by depth differences between different pixels in the target damage region. Based on the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface of the target damage region.

[0032] The method for determining the area of ​​damaged areas in ship steel plates described in this invention automatically completes the target area segmentation, depth data processing, area calculation, and result output after acquiring time-synchronized and spatially registered color and depth images of the surface to be tested. Under the condition that the imaging equipment and configuration remain unchanged and offline calibration has been completed, there is no need for manual selection or repeated parameter calibration during the on-site inspection stage. This reduces the influence of subjective judgment on the area measurement results in traditional manual inspection, and achieves objective consistency and repeatability in the detection of damaged areas in ship steel plates. Thus, it realizes the automated quantitative calculation of the area of ​​damaged areas in ship steel plates. It is applicable to coating maintenance and quality inspection in industrial scenarios such as ships, bridges, and building components, and helps to improve the consistency and efficiency of the detection of damaged areas in ship steel plates. At the same time, it helps to reduce the workload of manual close-range inspection and has the feasibility of deployment on a lightweight computing platform.

[0033] In an optional embodiment of the present invention, step 11, acquiring the image of the surface to be tested captured by the imaging device, may include:

[0034] Step 111: Acquire color and depth images of the surface to be measured using a depth camera. The depth image of the surface to be measured records the axial depth value of each pixel's corresponding spatial point in the coordinate system of the shooting device.

[0035] Step 112: Spatially register the color image and depth image of the surface to be tested to generate a time-synchronized and spatially registered color image and depth image of the surface to be tested, thus obtaining the image to be detected.

[0036] In this embodiment, step 111 can specifically be as follows: the depth camera can be an Orbbec Gemini336 depth camera as the acquisition device. This camera has a built-in hardware-level depth calculation chip, which can output a color image with a resolution of 640×400 and the corresponding depth image in real time; the depth image of the surface to be measured records the axial depth value of each pixel's corresponding spatial point in the coordinate system of the shooting device. The axial depth value is the Z coordinate of the corresponding spatial point in the coordinate system of the shooting device, and the unit is millimeters; during acquisition, the camera is aimed at the surface to be measured (such as the bulkhead of a ship) to ensure that it is within the effective field of view of the camera; the depth camera can be directly attached to a drone for remote photography without the need for personnel to observe at close range, thereby expanding the detection coverage of areas that are difficult for humans to reach, solving the problem of poor accessibility during personnel inspection, and improving detection efficiency.

[0037] In one embodiment of close-up inspection of ship cabins using unmanned aerial vehicles (UAVs), the imaging device is mounted on the UAV and acquires color and depth images of the surface under test under imaging conditions that meet preset close-up inspection image resolution requirements. The close-up inspection image resolution requirements can be confirmed through resolution verification targets and detail verification targets; for example, horizontal and vertical stripes at the 4mm, 2mm, 1mm, 0.5mm, and 0.25mm levels in the verification image are clearly distinguishable, and character details of 1.8mm in size are still discernible. The resolution requirements are used to evaluate the image sharpness of the imaging device under the current imaging conditions, without limiting the imaging device to a fixed shooting distance. The actual shooting distance can be determined based on the imaging configuration of the imaging device, the ambient lighting conditions, the condition of the surface under test, and the resolution verification results.

[0038] In this embodiment, step 112 can specifically be to complete spatial registration through the hardware registration module or supporting software development kit of the shooting device. For example, by jointly calibrating the optical parameters of the two sensors, a pixel position mapping relationship between the color image pixels and the registered depth image pixels is established. During the alignment process, the depth image is reprojected onto the color image coordinate system based on the color image, so that for the same spatial point in the field of view, the color image and the registered depth image establish a pixel position correspondence relationship. For example, the pixel coordinates (320, 200) of a target damage area edge point 1200 mm away from the camera in the field of view correspond to each other in the color image and the coordinates (320, 200) in the depth image within the preset registration error range.

[0039] In this embodiment, by using the hardware registration module of the imaging device or the supporting software development kit, the depth image is reprojected onto the color image coordinate system, so that the pixel position correspondence between the color image of the surface to be measured and the registered depth image is established, thereby reducing the impact of spatial registration deviation on subsequent depth extraction and area calculation.

[0040] In an optional embodiment of the present invention, step 12 involves performing target region segmentation processing on the color image of the surface to be tested to obtain a target pixel-level mask of the target damage region. The target pixel-level mask includes at least one target damage instance mask, which may include:

[0041] Step 120: Using a preset single-class instance segmentation model, one or more categories of coating damage category and corrosion category in the color image of the surface to be tested are uniformly mapped to the target damage category, the target damage region corresponding to the target damage category is identified, and the target pixel-level mask corresponding to the target damage region is output.

[0042] In this embodiment, by employing the single-class instance segmentation model, one or more categories among the coating damage category and the corrosion category are uniformly mapped to the target damage category, and the target damage area corresponding to the target damage category is identified. This can reduce the interference of category feature differences between different target damage types on model training, allowing the model to focus on the visual differences between the target damage area and the background area. It should be noted that the single-class instance segmentation model is an optional implementation of step 120, rather than a limitation on the target area segmentation processing method described in step 120.

[0043] In an optional embodiment of the present invention, the training process of the single-class instance segmentation model in step 120 may include:

[0044] Step 121: Obtain multiple target damage images with pixel-level target damage instance annotations, wherein the target damage instance annotations include target damage category labels for training and target damage instance masks for training.

[0045] Step 122: While keeping the pixel range of each target damage instance mask used for training unchanged, merge the target damage category labels into a unified category label to obtain a single-class labeled dataset.

[0046] Step 123: Expand the single-class labeled dataset using data augmentation strategies to obtain a surface damage dataset. The data augmentation strategies include at least one of random rotation, random scaling, and color space enhancement.

[0047] Step 124: Divide the surface damage dataset into a training set, a validation set, and a test set according to a preset ratio. Use the training set to train the preset segmentation network model, and select hyperparameters, determine early stopping, and select the optimal model weights based on the performance evaluation results on the validation set to obtain the single-class instance segmentation model. The test set is only used for the final independent performance evaluation.

[0048] In this embodiment, the target damage category mentioned in step 121 includes coating damage category and rust category; wherein, coating peeling belongs to coating damage category; hard rust and edge or weld corrosion belong to rust category; step 122 may include: for each target damage instance image, uniformly mapping the category label corresponding to each target damage instance to the "Damage" category, while retaining the original pixel-level instance mask and its pixel range of each target damage instance, to obtain a single-category instance segmentation annotation dataset; for example, an image that originally contained both hard rust instances and coating peeling instances, after category merging, each instance is uniformly labeled as the "Damage" category, but each target damage instance still retains its corresponding independent pixel-level instance mask, and no longer distinguishes the specific damage type.

[0049] In this embodiment, in order to suppress model overfitting and enhance its generalization ability under complex lighting and variable perspectives, a variety of data augmentation methods are used to expand the dataset to obtain a surface damage dataset. The random rotation of the data augmentation strategy in step 123 is to randomly rotate the images in the single-class labeled dataset from -45° to +45°. The random scaling is to randomly scale the images in the single-class labeled dataset by a ratio of 0.5 to 1.5. The color space enhancement is to randomly shift the hue H of the images in the single-class labeled dataset by ±10, randomly scale the saturation S by 0.5 to 1.5 times, and randomly scale the brightness V by 0.5 to 1.5 times in the HSV color space to simulate the appearance of the target damage area under different lighting conditions.

[0050] In this embodiment, the preset segmentation network model adopts the YOLO11n-Seg network. Step 124 may include: dividing the surface damage dataset into a training set, a validation set, and a test set according to a preset ratio; training the preset convolutional neural network model using the training set, with a maximum training epoch of 150 epochs; evaluating the performance on the validation set after each training epoch, and implementing early stopping judgment and optimal model weight selection based on the validation set evaluation results; the test set is only used for the final independent performance evaluation.

[0051] In this embodiment, such as Figure 2 As shown, the YOLO11n-Seg network includes a backbone network, a neck network, and a head network; as... Figure 3 As shown, the backbone network introduces the C3k2 module as the core feature extraction unit, and after the Spatial Pyramid Fast Pooling (SPPF) module, it connects to the C2PSA module to introduce a spatial attention mechanism. The C3k2 module is an improved version based on the C2f module in YOLOv8, serving as the core feature extraction unit of the network. The C3k2 module integrates the efficient splitting idea of ​​Cross-Stage Partial Network (CSPNet) and the variable convolution kernel strategy. The input features are split into two paths in the channel dimension. One path undergoes non-linear transformation through stacked C3k bottleneck blocks, and is finally concatenated and fused with the other path of features.

[0052] C3k2 exhibits high structural flexibility, supporting dynamic adjustment of the convolutional kernel size through parameter configuration. When the configuration parameter c3k=False, its internal C3k can degenerate into a standard Bottleneck, thus enabling flexible conversion between C3k2 and C2f structures under different computational constraints. The SPPF module serially connects three 5×5 max pooling layers to continuously perform fast pooling operations and concatenates feature maps of different scales along the channel dimension, thereby achieving deep fusion of multi-scale local features and global contextual information.

[0053] The C2PSA module is connected at the end of the backbone network. This module integrates a spatial attention mechanism, which suppresses background noise in complex industrial environments by enhancing the weighting of key feature regions. For damaged surfaces, C2PSA helps the network focus on the damaged area rather than interference areas with similar colors. The neck network achieves bidirectional fusion of deep semantic features and shallow detail features through a top-down upsampling path and a bottom-up convolutional path. The output of this structure contains feature maps of three different scales, providing multi-scale features for subsequent target region segmentation. Training uses the AdamW optimizer with β1=0.9, an initial learning rate of 0.001, a batch size of 16, and a maximum number of training epochs of 150. After each training epoch, performance is evaluated on the validation set, and early stopping and optimal model weight selection are performed based on the validation set evaluation results.

[0054] In this embodiment, step 12 inputs the color image of the surface to be tested into a preset single-class instance segmentation model to obtain one or more target damage instance masks corresponding to the target damage region. In a specific implementation, the segmentation head of the single-class instance segmentation model generates mask prediction information based on input features. The mask prediction information may include mask prototype features and mask coefficients corresponding to each candidate target damage instance. Mask decoding is performed based on the mask coefficients corresponding to each target damage instance and the mask prototype features to obtain the prediction mask corresponding to each target damage instance. Thresholding and size restoration processing are performed on the prediction mask to obtain N target damage instance masks that establish a pixel position correspondence with the color image of the surface to be tested, where N is the number of identified target damage instances. Each target damage instance mask is used to mark the pixel position covered by a target damage instance. Pixels in the target damage region and pixels in the background region use different pixel labels. For example, the background region pixel label is 0, and the target damage region pixel label is 1.

[0055] When the preset single-class instance segmentation model outputs a target damage instance mask, the target damage instance mask can be directly used as the target pixel-level mask of the target damage region; when the single-class instance segmentation model outputs multiple target damage instance masks, each target damage instance mask corresponds to a different target damage instance; the physical area of ​​each target damage instance can be determined according to each target damage instance mask, or the physical areas of each target damage instance can be accumulated to obtain the total physical area of ​​the target damage region.

[0056] In an optional embodiment of the present invention, step 12, which involves segmenting the color image of the surface to be tested to obtain a target pixel-level mask of the target damage area, may further include:

[0057] The pre-defined multi-class segmentation model identifies the coating damage area and the corrosion area respectively, and outputs the target pixel-level mask corresponding to the coating damage area and the corrosion area respectively.

[0058] In this embodiment, when using a multi-class segmentation model, a multi-class labeled dataset needs to be constructed, in which the coating damage area and the corrosion area are labeled with different category labels. In an optional mutually exclusive labeling method, pixels with substrate corrosion are preferentially labeled as the corrosion category, and the coating damage category is used to label coating damage areas without obvious substrate corrosion. The multi-class labeled dataset is used to train a preset segmentation network model to obtain the multi-class segmentation model. After the color image of the surface to be tested is input into the multi-class segmentation model, the multi-class segmentation model identifies the coating damage area and the corrosion area respectively, and outputs the target pixel-level masks corresponding to the coating damage area and the corrosion area respectively. When it is necessary to determine the area of ​​the coating damage area and the area of ​​the corrosion area separately, the target pixel-level masks corresponding to each category can be retained separately. When it is necessary to determine the total target damage area of ​​the coating damage area and the corrosion area, the target pixel-level masks corresponding to different categories can be deduplicated and merged at the pixel level to obtain the joint pixel-level mask of the target damage area.

[0059] In an optional embodiment of the present invention, step 13, which involves extracting the target depth dataset corresponding to the target damage region from the surface depth image to be tested based on the target pixel-level mask, may include:

[0060] Step 131: Determine the position coordinates of the pixels corresponding to the target damage area in the target pixel-level mask;

[0061] Step 132: Based on the corresponding position coordinates, extract the depth value at the corresponding position of the target damage area from the aligned depth image of the surface to be tested to obtain the initial depth dataset of the target damage area.

[0062] Step 133: Perform a preliminary validity determination on the initial depth data, determine the depth data with a depth value of zero or exceeding the preset effective range as initial invalid depth data, and determine the remaining depth data as candidate valid depth data;

[0063] Step 134: Determine a local reference depth value based on the target pixel-level mask and candidate effective depth data; specifically, perform connected component analysis on the target pixel-level mask, and determine a target damage connected region by mutually connected target damage pixels according to a preset pixel adjacency rule; for each candidate effective depth data, determine a preset neighborhood in the target damage instance mask or the same target damage connected region as the current candidate effective depth data, and determine a local reference depth value based on the candidate effective depth data in the preset neighborhood;

[0064] Step 135: Local anomaly determination is performed based on the absolute difference between the candidate valid depth data and the local reference depth value to obtain a determination result; specifically, for pixel coordinates... The candidate valid depth data, wherein the absolute difference is determined by the following formula: ,in, pixel coordinates The absolute difference between the candidate valid depth data and the local reference depth value. pixel coordinates Candidate effective depth values ​​at the location, pixel coordinates The corresponding local reference depth value;

[0065] Step 136: Based on the determination result, candidate valid depth data whose absolute difference exceeds a preset depth difference threshold are identified as local abnormal depth data. Specifically, when... At that time, the pixel coordinates The corresponding candidate valid depth data was determined to be local abnormal depth data. The preset depth difference threshold is used;

[0066] Step 137: The initial invalid depth data and the local abnormal depth data are jointly determined as invalid depth data, the remaining candidate valid depth data are determined as valid depth data, and the valid depth data and their corresponding position coordinates together constitute a subset of valid depth data;

[0067] Step 138: Perform neighborhood interpolation on the valid depth data within a preset neighborhood of the location coordinates corresponding to the invalid depth data to obtain the estimated depth value corresponding to the invalid depth data; specifically, for each invalid depth data, perform neighborhood interpolation on the valid depth data within a preset neighborhood of the location coordinates corresponding to the invalid depth data to obtain the estimated depth value corresponding to each invalid depth data.

[0068] Step 139: Replace the corresponding invalid depth data with the estimated depth value to obtain the target depth dataset; simultaneously, retain data markers in the target depth dataset to characterize each depth data as valid depth data or estimated depth value, and maintain the correlation between the position coordinates of each target pixel in the target damage region and the corresponding depth value; wherein, the pixel corresponding to the candidate valid depth data used to determine the local reference depth value is located in the same target damage instance mask or the same target damage connected region as the pixel corresponding to the currently to-be-determined candidate valid depth data; the pixel corresponding to the valid depth data used for the neighborhood interpolation process is located in the same target damage instance mask or the same target damage connected region as the pixel corresponding to the invalid depth data; the valid depth data used for the neighborhood interpolation process is the original valid depth data obtained from the initial depth data through the preliminary validity determination and the local anomaly determination, excluding the estimated depth value obtained through the neighborhood interpolation process.

[0069] In this embodiment, step 131 can specifically be as follows: using the target pixel-level mask as a spatial filter, only retaining pixels with a mask value of foreground (i.e., target damage region pixels marked as 1), and masking background pixels (i.e., background pixels marked as 0). Since the color image of the surface to be tested and the depth image of the surface to be tested have already been spatially registered, the corresponding pixel position in the depth image of the surface to be tested can be determined according to the target damage region pixel coordinates in the target pixel-level mask; for example: let the target pixel-level mask matrix be... Its size and the depth image of the surface to be measured The same formula applies: image height in pixels × image width in pixels. For any pixel coordinate... ,like If , it means that the pixel belongs to the target damage area; if This indicates that the pixel belongs to the background region. According to the mapping relationship, in You can find it in The corresponding position coordinates.

[0070] In this embodiment, step 132 involves extracting the depth value of the corresponding position of the target damage region from the aligned depth image of the surface to be tested, based on the corresponding position coordinates, to form an initial depth dataset of the target damage region. Specifically, this can be achieved by mapping the corresponding pixel position of each target damage region pixel in the depth image based on the spatial index of the target pixel-level mask region, i.e., the corresponding position coordinates of the target damage region, and the pixel position correspondence between the color image and the registered depth image, thereby separating the depth information belonging only to the target damage region. For example, for the target pixel-level mask matrix... All pixel positions with a value of 1 In the depth image of the surface to be measured Read depth values ​​at the same coordinates After traversing the entire target pixel-level mask region, the depth values ​​corresponding to all pixels in the target damage region are extracted to form a one-dimensional depth array. That is, the initial depth dataset, where For the initial depth dataset, the first The depth value corresponding to each target pixel. ; The total number of pixels covered by the target damage area is the number of target pixels.

[0071] In this embodiment, since there may be missing depth values, exceeding the effective range, or local measurement anomalies during the hardware acquisition process, the extracted depth data needs to be processed. Step 133 specifically involves determining the validity of the data in the extracted initial depth dataset. Specifically, this can be done by first performing a range screening on the initial depth data and setting a lower limit for the effective range. millimeters, upper limit millimeters; for the initial depth dataset Each depth value in ,like (Indicates that the depth value is invalid or missing) or or If the depth value and its corresponding position coordinates are associated and marked as initial invalid depth data, then depth data that does not meet any of the above conditions are determined as candidate valid depth data.

[0072] In this embodiment, step 134, determining the local reference depth value based on the target pixel-level mask and candidate valid depth data, may include:

[0073] Connectivity analysis is performed on the target pixel-level mask. According to a preset pixel adjacency rule, interconnected target damage region pixels are defined as a target damage connected region. Local anomaly determination is performed on each candidate valid depth data: for positions with coordinates... The candidate effective depth data is used to determine a preset neighborhood centered on the location coordinates within the same target damage instance mask or the same target damage connected region as the candidate effective depth data. Candidate effective depth data is then filtered from the preset neighborhood: when the number of candidate effective depth data is less than a preset neighborhood data quantity threshold, the neighborhood radius of the preset neighborhood is expanded by a preset step size, and the filtering continues within the same target damage instance mask or the same target damage connected region; when the neighborhood radius reaches a preset maximum neighborhood radius, and still no candidate effective depth data meeting the preset quantity requirement is obtained, the current candidate effective depth data is determined as local abnormal depth data; when the number of candidate effective depth data within the preset neighborhood reaches the preset neighborhood data quantity threshold, a local reference depth value is determined based on the median of the candidate effective depth data. ;

[0074] In this embodiment, steps 135 to 136 may specifically involve determining the absolute difference between the current candidate valid depth data and the local reference depth value according to the following formula: Set the depth difference threshold to :when When, the current candidate valid depth data is identified as local abnormal depth data; when When the current candidate valid depth data is determined, it is identified as valid depth data.

[0075] In this embodiment, steps 137 to 139 can specifically involve: first, the initial invalid depth data obtained from the range screening, the local abnormal depth data obtained from the local anomaly determination, and the local abnormal depth data determined due to insufficient neighborhood data are collectively considered as invalid depth data; the remaining candidate valid depth data and their corresponding position coordinates constitute a subset of valid depth data; the depth data in the subset of valid depth data are all the original measurement values ​​collected by the imaging device, excluding the estimated depth values ​​obtained subsequently through neighborhood interpolation; for example, assuming the initial depth data includes 5000 depth data points, after the range screening and the local anomaly determination, 4980 depth data points are marked as valid depth data, and the remaining 20 depth data points are marked as invalid depth data. If the data is invalid depth data, then the subset of valid depth data includes the 4980 valid depth data and their corresponding position coordinates. For each invalid depth data, neighborhood interpolation is performed on the valid depth data within a preset neighborhood of the position coordinates of the invalid depth data to obtain an estimated depth value corresponding to each invalid depth data. The pixel corresponding to the valid depth data used for the neighborhood interpolation is located within the same target damage instance mask or the same target damage connected region as the invalid depth data. The estimated depth values ​​are used to replace the corresponding invalid depth data, and the replaced depth data and the valid depth data together constitute the target depth dataset. Specifically, let the pixel coordinates corresponding to any invalid depth data be... Filter within the preset neighborhood corresponding to the pixel coordinates to obtain If there is valid depth data, then it can be expressed as follows: Determine the estimated depth value corresponding to the invalid depth data: where, coordinates The estimated depth value corresponding to invalid depth data. For the first in the preset neighborhood One valid depth data, The number of valid depth data within the preset neighborhood; the valid depth data participating in the neighborhood interpolation process only includes the valid depth data directly acquired by the depth acquisition device after validity determination in step 133; the estimated depth value obtained through the neighborhood interpolation process is not used as valid depth data when other invalid depth data is used for neighborhood interpolation; when the number of valid depth data within the preset neighborhood is less than the preset interpolation data number threshold, the neighborhood radius of the preset neighborhood is expanded by 1 pixel, and the search for valid depth data is continued to be limited to the same target damage instance mask or the same target damage connected region; when the neighborhood radius reaches the preset maximum radius and still cannot obtain enough valid depth data, or the proportion of invalid depth pixels exceeds the preset threshold, the area measurement is determined to be invalid and a re-acquisition prompt is output; for example: assuming that a preset neighborhood in the initial depth dataset includes four adjacent pixels, namely the first pixel. Second pixel The third pixel and the fourth pixel The depth values ​​corresponding to the four adjacent pixels are 1200 mm, 1204 mm, 1199 mm, and 0 mm, respectively. Since the depth value of the fourth pixel is 0, the depth data corresponding to the fourth pixel is marked as invalid depth data, and the depth data corresponding to the remaining three pixels are marked as valid depth data. Neighborhood mean interpolation is performed based on the valid depth data corresponding to the first, second, and third pixels, using the formula... Determine the estimated depth value corresponding to the fourth pixel; after replacing the invalid depth data corresponding to the fourth pixel with the estimated depth value of 1201 mm, the target depth dataset corresponding to the preset neighborhood includes: {(100, 200, 1200), (101, 200, 1204), (100, 201, 1199), (101, 201, 1201)}, where each set of data represents the pixel horizontal coordinate, pixel vertical coordinate, and corresponding depth value in sequence; after performing neighborhood interpolation on the 20 invalid depth data in the above manner, replace the 20 invalid depth data with the corresponding estimated depth value, and retain the 4980 valid depth data to obtain a target depth dataset containing 5000 depth data; the subset of valid depth data still includes the original 4980 valid depth data, and the 20 estimated depth data are not added to the subset of valid depth data.

[0076] In subsequent processing, the target depth dataset is used to determine the uncorrected single-pixel projected physical area corresponding to each target pixel within the target damage region, and the uncorrected single-pixel projected physical areas are accumulated. The effective depth data subset is used to extract the spatial geometric features of the surface where the target damage region is located, including calculating the discrete coefficient of the effective depth data, or performing plane fitting on the three-dimensional point set corresponding to the effective depth data to determine the normal vector of the fitting plane and the corresponding surface tilt angle. The depth data corresponding to the target damage region can be extracted from the spatially registered surface depth image using the target pixel-level mask index. By performing validity judgment and neighborhood interpolation on missing or abnormal depth data, and replacing the corresponding invalid depth data with estimated depth values, the pixel positions and pixel counts in the target depth dataset can be kept consistent with the target pixel-level mask of the target damage region, avoiding omissions in the area of ​​the target damage region due to directly removing invalid depth data. Meanwhile, by retaining the data source labels of effective depth data and estimated depth values ​​in the target depth dataset, and by ensuring that the estimated depth values ​​obtained through neighborhood interpolation do not participate in the determination of spatial geometric features, the smoothing effect of interpolation on the depth data distribution can be reduced, thereby reducing the possibility of distortion of discrete coefficients and deviations in the fitted plane normal vector or surface tilt angle.

[0077] In an optional embodiment of the present invention, step 14, determining the initial projected physical area of ​​the target damage region based on the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device, may include:

[0078] Step 141: Determine the number of target pixels in the target damage region based on the target pixel-level mask;

[0079] Step 142, based on the target depth dataset, determine the first... Depth value corresponding to each target pixel ;

[0080] Step 143, based on the depth value And the lateral focal length parameter in the calibration parameters of the shooting device. and longitudinal focal length parameters Determine the first The uncorrected projected physical area of ​​each target pixel ,in and These are the horizontal pixel coordinate increments and the vertical pixel coordinate increments, respectively.

[0081] Step 144: Accumulate the uncorrected single-pixel projected physical area corresponding to each target pixel in the target damage region to obtain the uncorrected projected physical area of ​​the target damage region. ;in, The number of target pixels in the target damage area;

[0082] Step 145, using the preset geometric scaling factor The uncorrected projected physical area is scaled to obtain the initial projected physical area. .

[0083] like Figure 4 As shown, in this embodiment, the specific implementation principle of the initial projected physical area is as follows: the imaging principle of the shooting device, i.e., the camera, can be simplified to a pinhole imaging model; Figure 4 In the image, O represents the optical center of the camera. Z represents a spatial point on the surface to be measured, where Z represents the axial depth of the spatial point relative to the optical center O along the optical axis. Representing a spatial point The projection point in the pixel plane; This represents the principal point coordinates, i.e., the position of the intersection of the camera's optical axis and the pixel plane in the pixel coordinate system. The x-coordinate of the principal point The ordinate of the principal point; , , and The camera intrinsic parameters corresponding to the shooting device shown in Table 1 at the current aligned output resolution can be used; according to the pinhole imaging model, the pixel coordinate increment... and In axial depth The corresponding lateral physical dimensions of the location and longitudinal physical dimensions They are respectively: , Therefore, the first The uncorrected projected physical area of ​​each target pixel is: The uncorrected projected physical area of ​​each pixel is accumulated to obtain the uncorrected projected physical area of ​​the target damage region. Then, using geometric scaling factors... Correcting for area scale deviations caused by the imaging equipment under the corresponding imaging configuration yields the initial projected physical area of ​​the target damage region: Since the depth values ​​obtained by the imaging device used in this embodiment are in millimeters, the initial projected physical area of ​​the target damage region is... The unit is square millimeters. The required area is obtained through unit conversion. It should be noted that the initial projected physical area is the projected area of ​​the pixel on a reference plane perpendicular to the camera's optical axis. When the surface of the target damage area is approximately flat, and the optical axis of the imaging device is perpendicular to the surface, the initial projected physical area... The target surface area is approximately equal to the target damage area; when there is a certain angle between the camera and the surface to be measured, it is necessary to use the spatial geometric features of the surface where the target damage area is located for correction.

[0084] In this embodiment, the focal length parameter in the calibration parameters of the shooting device can be directly obtained from the intrinsic parameters of the shooting device under a preset imaging configuration. For the intrinsic parameters of the Gemini336 camera at a resolution of 640×400, =311.38 pixels, =311.38 pixels; the preset geometric scaling factor can be determined by referring to Table 1, the in-camera calibration parameters of the Gemini336 camera under different imaging configurations. The value can be obtained through experimental calibration using a standard-size calibration plate; the geometric scaling factor is used to compensate for stable area scale deviations caused by at least one of spatial registration, image resampling, depth scale error, camera intrinsic parameter deviation, calibration template region segmentation error, and other imaging system errors in the preset imaging configuration and calibration processing link.

[0085] When the geometric scaling factor needs to be obtained through experimental calibration, the specific calibration method is as follows: Prepare multiple standard square calibration templates of different sizes (reference areas of 9cm², 25cm², 100cm², and 144cm², respectively); make the camera optical axis perpendicular to the plane where the calibration template is located, and keep the camera imaging configuration unchanged, the camera imaging configuration including image resolution and camera intrinsic parameters; adjust the distance between the camera and the calibration template, and acquire multiple frames of color images and depth images at multiple measurement positions; input the color images into a pre-trained calibration template instance segmentation model to obtain the target pixel-level mask corresponding to the calibration template area; based on the target pixel-level mask, extract the depth data corresponding to the calibration template area from the spatially registered depth image, and calculate the uncorrected projected physical area of ​​each set of calibration data according to the target pixel-level mask and the corresponding depth data;

[0086] Let the first The uncorrected projected physical area corresponding to the group calibration data is The initial projected physical area obtained after geometric scaling correction of this set of calibration data is: ;

[0087] The geometric scaling factor under the current camera imaging configuration is determined with the optimization objective of minimizing the sum of squared relative errors between the initial projected physical area corresponding to each set of calibration data and the calibration template reference area. : ;in The geometric scaling factor obtained from the calibration; The number of valid calibration data; For the first The uncorrected projected physical area corresponding to the group calibration data; For the first The reference area of ​​the calibration template corresponding to the set of calibration data; the optimization problem is solved using the least squares method; since the optimization objective is about the geometric scaling factor. It is a quadratic function in one variable, therefore its optimal calibration value is... Calculate according to the following formula: ; calculate the This serves as the geometric scaling factor corresponding to the current camera imaging configuration; in subsequent area measurements, the uncorrected projected physical area of ​​the target region is multiplied by this geometric scaling factor. The initial projected physical area of ​​the target region is obtained; for different camera imaging configurations, the above experimental calibration process is performed to obtain the geometric scaling factor corresponding to each camera imaging configuration.

[0088] The calibration template instance segmentation model can be trained using pre-acquired calibration template images and their pixel-level instance annotations. It is used to automatically extract the target pixel-level mask of the calibration template area during offline parameter calibration. The calibration template instance segmentation model is only used for calibration template area extraction, and can be trained and configured separately from the target area segmentation model used to identify coating damage areas and corrosion areas during actual detection.

[0089] Table 1. In-camera calibration parameters of the Gemini 336 camera under different imaging configurations.

[0090]

[0091] In this embodiment, the image acquisition device used in Table 1 is a Gemini336 depth camera, the camera firmware version is 1.5.55, and the software development kit version is pyorbbecsdk-2.0.13. The camera intrinsic parameters and related calibration parameters listed in Table 1 are the embodiment parameters obtained under the above-mentioned device, firmware version, software development kit version, and camera imaging configuration. They are only used to illustrate the specific implementation process of the method described in this application and should not be construed as general fixed parameters applicable to all Gemini336 depth cameras. When the camera firmware version, software development kit version, imaging resolution, image alignment method, or other imaging configuration changes, the camera intrinsic parameters and related calibration parameters may change accordingly. Different cameras of the same model may also have different parameters due to individual device differences. Therefore, in practical applications, the camera intrinsic parameters of the current camera under the corresponding imaging configuration should be obtained, and the corresponding calibration parameters should be determined according to the calibration method described in this application.

[0092] The resolutions in Table 1 are the alignment output resolutions set by the Gemini 336 depth camera when the depth image and color image alignment function is enabled; after alignment, the color image and depth image have the same image width and image height, and are established in the same pixel coordinate system; Table 1 The horizontal position of the intersection of the camera's optical axis and the imaging plane in the pixel coordinate system is the abscissa of the principal point, which represents the horizontal position of the optical axis in the image. The vertical position of the intersection of the camera's optical axis and the imaging plane in the pixel coordinate system is the principal point's ordinate, representing the vertical position of the optical axis in the image.

[0093] In an optional embodiment of the present invention, step 15, which corrects the initial projected physical area based on the spatial geometric features of the surface where the target damaged region is located, as characterized by the target depth dataset, to obtain the target surface area of ​​the target damaged region, may include:

[0094] Step 151: Based on the discrete coefficients or surface tilt angles of the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damage region.

[0095] In this embodiment, the spatial geometric features may include at least one of the following:

[0096] The discrete coefficients are determined based on the effective depth data subset; and a three-dimensional point set is constructed based on the position coordinates, depth values, and calibration parameters of the imaging device corresponding to each effective depth data in the effective depth data subset, and a fitting plane normal vector or surface tilt angle is obtained by fitting the three-dimensional point set to the plane; the discrete coefficients are used to characterize the statistical dispersion of the depth data corresponding to the target damage area, and the statistical dispersion may be affected by at least one of the following factors: surface tilt, local undulations, target area scale, and depth measurement noise; the fitting plane normal vector and the surface tilt angle can reflect the spatial attitude of the surface where the target damage area is located relative to the optical axis direction of the imaging device; the initial projected physical area can be empirically corrected based on the discrete coefficients, or the initial projected physical area can be projected corrected based on the fitting plane normal vector or the surface tilt angle; correcting the initial projected physical area based on the spatial geometric features is beneficial to compensate for the area calculation deviation under non-ideal observation conditions and improve the accuracy of determining the physical area of ​​the target damage area.

[0097] In an optional embodiment of the present invention, the discrete coefficients in step 151 are determined by the following steps:

[0098] Step 1511: Determine the average value of the effective depth data subset based on the effective depth data subset. and standard deviation ;

[0099] Step 1512, based on the standard deviation With the average value The ratio determines the discrete coefficients. The effective depth data subset consists of the effective depth data obtained from the initial depth data through the preliminary validity determination and the local anomaly determination, excluding the estimated depth values ​​obtained through neighborhood interpolation.

[0100] In this embodiment, step 1511 specifically involves determining the coordinate information and corresponding depth value of each pixel in the effective depth data subset based on the effective depth data subset in the target depth dataset; and then, based on the coordinate information and corresponding depth value of each pixel in the effective depth data subset, through... Determine the average depth data in the effective depth data subset. ,in, The number of depth data points in the effective depth data subset. For the effective depth data subset, the first One effective depth value; through Determine the standard deviation of depth data in the effective depth data subset. ;

[0101] In this embodiment, step 152 can be, through Determine the discrete coefficients .

[0102] In this embodiment, when the target surface is flat and parallel to the camera plane, the depth data dispersion is extremely low. The coefficient of variation of depth data tends to be close to 0; conversely, under similar conditions of imaging equipment configuration, working distance, and target area scale, as the degree of inclination or local undulation of the surface under test increases, the coefficient of variation of depth data usually increases accordingly. (The value increases); the coefficient of variation can characterize the degree of dispersion of depth data caused by the tilt or local undulation of the surface to be measured, and can be used as the basis for area correction.

[0103] In an optional embodiment of the present invention, step 15, which corrects the initial projected physical area based on the discrete coefficients of the spatial geometric features of the surface where the target damaged region is located, as represented by the target depth dataset, to obtain the target surface area of ​​the target damaged region, may include:

[0104] Step 1521, when the discrete coefficients When the image is within the preset discrete coefficient calibration range corresponding to the current imaging configuration of the shooting device, the discrete coefficients and the preset empirical discrete correction coefficients are used. ,pass Determine the area correction factor ;in, These are the preset empirical discrete correction coefficients. These are the discrete coefficients; the empirical discrete correction coefficients Color and depth images of a calibration template with a known physical area are acquired by an imaging device in multiple relative postures, and offline calibration is performed based on the relationship between the area calculation results in each relative posture and the known physical area of ​​the calibration template.

[0105] Step 1522, according to the area correction coefficient ,pass For the initial projected physical area Correction processing is performed to obtain the target surface area of ​​the target damage region in the image to be detected. The target surface area is the surface area to be measured obtained by empirically correcting the initial projected physical area using the spatial geometric features of the target depth data.

[0106] In this embodiment, the empirical discrete correction coefficient The empirical discrete correction coefficients can be obtained through experimental calibration or by directly looking up the values ​​in Table 1. Calibration experiments and geometric scaling factors The calibration experiments are independent of each other, and they do not use the same batch of calibration data; empirical discrete correction coefficients The calibration experiment requires the use of the calibrated geometric scaling factor. Calculate the initial projected physical area, and therefore the empirical discrete correction coefficient. The calibration experiment needs to be performed after the geometric scaling factor is determined; the specific calibration experiment is as follows: prepare a standard square calibration template with an area of ​​100cm² and a side length of 100mm; while keeping the camera imaging configuration unchanged, change the relative attitude between the camera and the calibration template to simulate the non-ideal observation condition that the optical axis of the imaging device is not perpendicular to the surface to be measured; in this embodiment, the calibration experiment collects multiple sets of color images and depth images within a measurement distance of approximately 300-1000mm between the imaging device and the calibration template, and ensures that the number of target pixels in the calibration template area in the target pixel-level mask is not less than 50; for each set of calibration data, input the collected color images into a pre-trained calibration template instance segmentation model to obtain the target pixel-level mask corresponding to the calibration template; based on the target pixel-level mask, extract the depth data corresponding to the calibration template area from the spatially registered depth image; for the first Calculate the coefficient of variation of a set of valid calibration data. Uncorrected projected physical area And based on the geometric scaling factor Obtain the initial projected physical area Based on empirical discrete correction coefficients The initial projected physical area is corrected to obtain the target surface area corresponding to the i-th set of calibration data: It should be noted that the 300-1000mm range is the actual calibration distance range used to obtain the empirical discrete correction coefficient in this embodiment, and does not constitute a limitation on the detection distance of the present invention. When the imaging device can obtain color images and depth images that meet the preset image resolution requirements and depth data quality requirements, the calibration distance can be extended beyond the range, and the empirical discrete correction coefficient under the corresponding imaging conditions can be obtained again by following the aforementioned calibration method to determine the area under different measurement distance conditions.

[0107] The optimization objective is to minimize the sum of squared errors of the target surface area relative to the corresponding calibration template reference area for each set of valid calibration data. The empirical discrete correction coefficients corresponding to the current camera imaging configuration are then determined as follows: ;in, The empirical discrete correction coefficients obtained from calibration; To participate The number of valid calibration data points calibrated; For the first The reference area of ​​the calibration sample corresponding to the set of valid calibration data; the above optimization objective is about The quadratic function in one variable is solved using the least squares method. For different camera imaging configurations, the above calibration process is performed to obtain the empirical discrete correction coefficients corresponding to each camera imaging configuration.

[0108] Under the imaging conditions that meet the preset resolution requirements for close-up examination images, this embodiment selects a square calibration template with an area of ​​100 cm² and a side length of 100 mm as the empirical discrete correction coefficient. The preferred calibration benchmark is a template whose feature size is larger than the millimeter-level and sub-millimeter-level detail features involved in the close-up inspection resolution verification. This allows the template to form a continuous target area with sufficient pixel coverage in color and depth images, which is beneficial for the stable extraction of template boundaries and effective depth data, and reduces the impact of a small number of boundary pixel errors and local depth noise on the calibration results. At the same time, the template is easy to set up, photograph, and change its relative attitude with the photographing equipment in the close-up inspection environment of a UAV in a ship cabin.

[0109] The 100cm² is a preferred parameter calibration area and does not limit the target damage area that can be processed by pixel-by-pixel area calculation; when using empirical discrete correction coefficients When performing area correction, the number of target pixels in the target pixel-level mask for the target damage area should be no less than 50. The imaging configuration of the current imaging device should match the calibration configuration corresponding to the empirical discrete correction coefficient, and the discrete coefficient of the target depth data should be within the preset discrete coefficient calibration range under the corresponding imaging configuration. If the above conditions are not met, parameter calibration, data acquisition, local region division, or other spatial geometric correction methods can be performed.

[0110] The number of target pixels is no less than 50 to ensure that the target area has the minimum pixel coverage required for area calculation and depth statistical processing. In this embodiment, no upper limit is set on the number of target pixels. As the area of ​​the target damaged area increases or the image resolution increases, the number of target pixels corresponding to the target area can be increased accordingly.

[0111] Table 2 shows the calibration results of the Gemini336 depth camera on a 100cm² calibration template under different preset imaging configurations, where MAE is the mean absolute error. This is used to characterize the average deviation of each group's area measurement results from the standard area; RMSE is the root mean square error. This is used to characterize the overall dispersion of area measurement errors, and assigns higher weights to larger measurement errors; when calculating the MAE and RMSE before correction, Take the initial projected physical area When calculating the corrected MAE and RMSE, Take the target surface area After correction of the coefficient of variation, the MAE and RMSE of the area calculation results were reduced on the calibration data used for parameter fitting.

[0112] When calibrating the parameters, the boundaries and details in the color image can be verified by close inspection of the image resolution to confirm that they meet the target region segmentation requirements. For depth data, the current depth image can be further judged to meet the parameter calibration requirements based on the effective depth pixel ratio, local depth fluctuation, or other depth data quality indicators.

[0113] Table 2. Gemini 336 camera performance on a 100cm² calibration template under different imaging configurations. Calibration results

[0114]

[0115] The MAE and RMSE before and after correction in Table 2 are the empirical discrete correction coefficients. Error statistics on the calibration data of parameter fitting are used to evaluate the fitting effect of the modified model on the calibration data;

[0116] Figure 5 The data shown was obtained using a Gemini 336 depth camera in a 640×400 imaging configuration. = =311.38 pixels, calibration template area is 100cm², collect depth data at different tilt angles and calculate area; Figure 5 Data points a2 represent the geometric scaling factor. The initial projected physical area after scale correction but without dispersion coefficient correction; data point a1 represents the area after dispersion coefficient and empirical dispersion correction coefficient. The corrected target surface area. Experimental data shows that as the relative tilt between the imaging device and the calibration template increases, the squared coefficient of variation of the depth data increases. As the area gradually increases, the uncorrected area calculation result gradually decreases, showing an approximately linear negative correlation between the two. Based on this statistical relationship, an empirical discrete correction coefficient is introduced. The initial projected physical area is empirically compensated, and parameters are optimized with the actual area of ​​the template as the target, ultimately determining... The optimal value; illustrated with a specific example: assuming the current camera intrinsic parameters correspond to... =311.38 pixels, =311.38 pixels, then =106.87 (as shown in Table 1, =311.38, The calibration result of =311.38 corresponds to =106.87);

[0117] In this embodiment, by introducing a correction model based on discrete coefficients, the area measurement error caused by non-ideal observation conditions can be reduced within the calibration range. This method uses the statistical discrete features of depth data to characterize the degree of depth change on the surface where the target damage area is located, and within the corresponding calibration range, empirical compensation is made for the area calculation deviation related to the statistical discrete features of the depth data, without the need for additional attitude sensors or complex three-dimensional reconstruction calculations.

[0118] It should be noted that, in this embodiment where a planar calibration template is used for parameter calibration, the empirical discrete correction model is mainly used to compensate for the projected area deviation caused when the surface under test is tilted relative to the imaging device. For surfaces under test with significant local undulations or that cannot be approximated as planes within the target damage area, local region division, recalibration of parameters for the corresponding surface morphology, or other area determination methods based on three-dimensional spatial information can be used.

[0119] The empirical discrete correction coefficients obtained using the 100cm² calibration template This method is applicable to imaging configurations that match the imaging equipment, image resolution, depth imaging mode, spatial registration method, and close-up examination image resolution during calibration. When the imaging configuration changes, the offline calibration process can be re-executed to obtain empirical discrete correction coefficients corresponding to the current imaging configuration.

[0120] When the number of target pixels in the target pixel-level mask of the target damage area is less than the preset minimum pixel number threshold, or the dispersion coefficient of the target depth data exceeds the calibration range corresponding to the current imaging configuration, or the imaging conditions of the current scene under test are significantly different from the calibration conditions of the empirical discrete correction coefficient, data can be re-acquired, parameter calibration can be re-performed, or other spatial geometric correction methods can be adopted. For target damage areas with large areas or significant differences in local geometry, the target damage area can be divided into multiple local sub-regions, the correction area of ​​each local sub-region can be determined, and the correction areas of each local sub-region can be accumulated.

[0121] In an optional embodiment of the present invention, step 15, which corrects the initial projected physical area based on the surface tilt angle of the spatial geometric features of the surface where the target damaged region is located, as represented by the target depth dataset, to obtain the target surface area of ​​the target damaged region, may include:

[0122] Step 1531: Based on the position coordinates of the pixels in the target damage area in the effective depth data subset, the depth value corresponding to each pixel in the target damage area, and the calibration parameters of the imaging device, back-project each pixel in the target damage area to the three-dimensional coordinate system of the imaging device to obtain the three-dimensional point set corresponding to the target damage area.

[0123] Step 1532: Perform plane fitting on the three-dimensional point set to obtain the fitting plane of the surface where the target damage area is located and the normal vector of the fitting plane;

[0124] Step 1533: Determine the surface tilt angle based on the angle between the normal vector and the optical axis of the imaging device. ;

[0125] Step 1534, according to For the initial projected physical area The target surface area of ​​the target damaged region is obtained by making corrections. .

[0126] This embodiment is applicable to situations where the surface of the target damage area can be approximately planar within the target damage area; in this embodiment, step 1531 may include:

[0127] Step 15311: Based on the effective depth data subset in the target depth dataset, determine the first [value] in the effective depth data subset. The position coordinates of each effective target pixel and its corresponding depth value ;

[0128] Step 15312, based on the position coordinates of the target pixel The depth value The target pixel is back-projected onto the three-dimensional coordinate system of the shooting device using the calibration parameters of the shooting device to obtain the three-dimensional point set corresponding to the target pixel. The calibration parameters of the shooting device include the lateral focal length parameter. Longitudinal focal length parameters x-coordinate of principal point and principal point ordinate The three-dimensional points 3D coordinates It can be determined by the following formulas: , , ; Traverse each target pixel in the effective depth data subset to obtain the effective three-dimensional point set corresponding to the target damage region. ,in, The number of valid depth data in the subset of valid depth data.

[0129] In this embodiment, step 1532 involves performing plane fitting on the three-dimensional point set to obtain a fitting plane and its normal vector. The fitting plane can be represented as: The unit normal vector of the fitted plane It can be represented as: The plane fitting can be any one of least squares plane fitting, weighted least squares plane fitting, or random sample consensus plane fitting. When using random sample consensus plane fitting, three-dimensional points whose distance from the candidate fitting plane is less than a preset distance threshold can be determined as inliers, and the fitting plane can be re-determined based on the inliers to reduce the influence of abnormal depth data or local surface defects on the fitting results.

[0130] In this embodiment, step 1533 is based on the unit normal vector of the fitted plane. The angle between the optical axis of the imaging device and the surface tilt angle of the surface where the target damaged area is located is determined. ;

[0131] Let the unit vector of the optical axis direction in the three-dimensional coordinate system of the imaging device be... Then the surface tilt angle It can be determined by the following formulas: , Specifically, by taking the absolute value of the inner product result, the influence of the direction selection of the fitting plane normal vector on the surface tilt angle calculation result is eliminated. The range of values ​​is .

[0132] In this embodiment, the initial projected physical area is the projected area of ​​the target damaged region on a reference plane perpendicular to the optical axis of the imaging device. When the surface where the target damaged region is located is tilted relative to the reference plane, the initial projected physical area and the target surface area of ​​the target damaged region approximately satisfy a cosine projection relationship. Therefore, the initial projected physical area can be corrected according to the surface tilt angle.

[0133] When the target damage region includes multiple target damage instances or multiple mutually separated target damage connected regions, the three-dimensional point sets corresponding to each target damage instance or each target damage connected region can be fitted with a plane to determine the corresponding surface tilt angle, and the corresponding initial projected physical area can be corrected. The corrected areas of each target damage instance or each target damage connected region are accumulated to obtain the target surface area of ​​the target damage region.

[0134] In this embodiment, when the number of effective 3D points participating in plane fitting is less than a preset threshold, or the fitting residual of the fitted plane exceeds a preset residual threshold, or If the result is less than the preset cosine threshold, it can be determined that the current plane fitting result does not meet the preset valid conditions, and a prompt to re-acquire can be output, or the initial projected physical area can be corrected using a correction method based on the discrete coefficient.

[0135] This embodiment directly estimates the spatial pose of the surface where the target damage area is located by using the depth data corresponding to the target damage area, and performs projection correction on the initial projected physical area according to the surface tilt angle. It does not require pre-calibration of empirical discrete correction coefficients and is suitable for detection scenarios where the surface where the target damage area is located can be approximated as a plane.

[0136] In one implementation, when the number of effective 3D points corresponding to the target damage area reaches a preset number, and the plane fitting residual does not exceed a preset residual threshold, and When the value is not lower than the preset threshold, the area correction method based on the fitting plane normal vector and the surface tilt angle is preferred; when the plane fitting condition is not met or the computing resources are limited, and the discrete coefficient is within the calibration range corresponding to the current imaging configuration, the area correction method based on the discrete coefficient and the empirical discrete correction coefficient is adopted; when the applicable conditions of both correction methods are not met, a prompt to re-acquire is output.

[0137] like Figure 6 As shown, the specific implementation process of the method for determining the area of ​​the damaged area of ​​the ship steel plate according to the present invention, which corrects the initial projected physical area based on the discrete coefficient, is as follows:

[0138] The method for determining the area of ​​the damaged area of ​​the ship steel plate described in this embodiment can be applied in practice by using a model deployment system to achieve quantitative calculation of the area of ​​the damaged area of ​​the ship steel plate through the collaborative work of a preset single-class instance segmentation model and a depth camera. The depth camera is connected to a Raspberry Pi via a USB interface to acquire synchronized RGB images (color images of the surface to be measured) and depth images (depth images of the surface to be measured) in real time. In the processing flow, the RGB image is first input into the preset single-class instance segmentation model for inference. The generated segmentation result is used as a mask to filter the depth image, thereby extracting the depth information of the target damaged area. Then, the aforementioned depth calculation method is applied to calculate the target surface area of ​​the target damaged area based on the extracted depth data. Specifically:

[0139] Step 1: Connect the Orbbec Gemini336 depth camera to the Raspberry Pi 5 via USB 3.0 interface, start real-time data stream acquisition, point the camera at the coating area to be measured on the ship's bulkhead, and acquire synchronized color and depth images in real time;

[0140] Step 2: Input the acquired color image (640×400) into the YOLO11n-Seg single-class instance segmentation model (NCNN format, FP16 inference) deployed on the Raspberry Pi; the model outputs the target pixel-level mask of the target damage region. The segmentation result marks all target damage regions in the image in the form of binary masks. Different instance masks or connected regions can correspond to different target damage instances.

[0141] Step 3: Using the segmentation mask as a spatial index, apply it to the spatially registered depth image to extract the depth values ​​corresponding to each target pixel in the mask, forming the initial depth data of the target damage region. First, depth data with a depth value of zero or exceeding the preset depth data filtering range are identified as initial invalid depth data, and the remaining depth data are identified as candidate valid depth data. Then, for each candidate valid depth data, a preset neighborhood is determined within the same target damage instance mask or the same target damage connected region. A local reference depth value is determined based on the median of the candidate valid depth data in the preset neighborhood, and local anomaly is determined based on the absolute difference between the candidate valid depth data and the local reference depth value. Data with an absolute difference exceeding a preset depth difference threshold are identified as local abnormal depth data. The initial invalid depth data and the local reference depth data are then compared and contrasted. Abnormal depth data are collectively identified as invalid depth data. The remaining candidate valid depth data are identified as valid depth data and form a subset of valid depth data. Neighborhood interpolation is performed on each invalid depth data using the valid depth data to obtain the target depth dataset. In this embodiment, the preset depth data filtering range is set to 300 to 5000 mm. In this example data, the initial depth data includes 5000 depth data. After validity determination, 4980 valid depth data and 20 invalid depth data are obtained. The 4980 valid depth data constitute the subset of valid depth data. Neighborhood interpolation is performed on each invalid depth data to obtain the target depth dataset. The number of depth data in the target depth dataset is s=5000, and the number of depth data in the subset of valid depth data is n=4980.

[0142] Step 4: Read the depth values ​​of each point in the target depth dataset and read the camera calibration parameters. =311.38 pixels =311.38 pixels, geometric scaling factor =0.468, calculate the uncorrected single-pixel projected physical area corresponding to each pixel, and perform cumulative processing to obtain the uncorrected projected physical area. Then, the initial projected physical area is obtained by correcting using geometric scaling factors. .

[0143] Step 5: Calculate the average depth data using a subset of valid depth data from the target depth dataset. Standard deviation Calculate the coefficients of variation .

[0144] Step 6: Read the empirical discrete correction coefficients Calculate the area correction factor The target surface area is calculated. .

[0145] It should be noted that the specific values ​​in this embodiment are only used to illustrate the complete calculation process from target pixel depth value, uncorrected projected physical area, geometric scale correction, spatial geometric feature calculation to target surface area, and are not used as area measurement accuracy verification data, nor do they constitute a limitation on the actual shooting distance for close-up inspection by UAV; the actual shooting distance can be determined based on camera imaging configuration, on-site environment and close-up inspection image resolution verification results.

[0146] In this embodiment, the method for determining the area of ​​the damaged region of the ship's steel plate achieves automatic location of the target damage and rapid estimation of its physical area. The lightweight nature of the system allows it to be deployed on embedded platforms such as Raspberry Pi, indicating that the solution is feasible for deployment on embedded computing platforms. After acquiring time-synchronized and spatially registered color and depth images of the surface to be tested, the target region segmentation, depth data processing, area calculation, and result output can be completed automatically. Under the condition that the imaging equipment and imaging configuration remain unchanged and offline calibration has been completed, there is no need for manual selection or repeated parameter calibration during the on-site detection stage, thus realizing the automated quantitative calculation of the area of ​​the damaged region of the ship's steel plate.

[0147] like Figure 7 As shown, an embodiment of the present invention also provides a device 70 for determining the area of ​​damaged areas in ship steel plates, comprising:

[0148] The acquisition module 71 is used to acquire the image to be detected of the surface to be tested collected by the imaging device. The image to be detected includes a time-synchronized and spatially registered color image of the surface to be tested and a depth image of the surface to be tested.

[0149] Processing module 72 is used to perform target region segmentation processing on the color image of the surface to be tested to obtain a target pixel-level mask of the target damage region, wherein the target pixel-level mask includes at least one target damage instance mask; the target damage region includes at least one of a coating damage region and a corrosion region; based on the target pixel-level mask, extract the target depth dataset corresponding to the target damage region from the depth image of the surface to be tested; determine the initial projected physical area of ​​the target damage region according to the target pixel-level mask, the target depth dataset and the calibration parameters of the imaging device; and correct the initial projected physical area according to the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, to obtain the target surface area of ​​the target damage region.

[0150] Optionally, the target region segmentation process is performed on the color image of the surface to be tested to obtain the target pixel-level mask of the target damage region, including: mapping one or more categories of coating damage category and corrosion category in the color image of the surface to be tested to the target damage category through a preset single-class instance segmentation model, identifying the target damage region corresponding to the target damage category, and outputting the target pixel-level mask corresponding to the target damage region.

[0151] Optionally, the training process of the single-class instance segmentation model includes: acquiring multiple target damage images with pixel-level target damage instance annotations, wherein the target damage instance annotations include target damage category labels and target damage instance masks used for training; merging the target damage category labels used for training into a unified category label while keeping the pixel range of each target damage instance mask unchanged, to obtain a single-class annotation dataset; expanding the single-class annotation dataset using data augmentation strategies to obtain a surface damage dataset, wherein the data augmentation strategies include at least one of random rotation, random scaling, and color space enhancement; dividing the surface damage dataset into a training set and a validation set, training a preset segmentation network model using the training set, and selecting hyperparameters, early stopping judgment, and optimal model weights based on the performance evaluation results on the validation set, to obtain the single-class instance segmentation model.

[0152] Optionally, based on the target pixel-level mask, extracting the target depth dataset corresponding to the target damage region from the depth image of the surface to be tested includes: determining the position coordinates of the pixels corresponding to the target damage region in the target pixel-level mask; extracting the depth value at the corresponding position from the depth image of the surface to be tested according to the position coordinates to obtain initial depth data; performing a preliminary validity determination on the initial depth data, determining depth data with a depth value of zero or exceeding a preset effective range as initial invalid depth data, and determining the remaining depth data as candidate valid depth data; determining a local reference depth value according to the target pixel-level mask and the candidate valid depth data; and determining the local reference depth value according to the candidate valid depth data and the local reference depth value. The absolute difference between the data points is used to determine local anomalies, resulting in a determination result. Based on the determination result, candidate valid depth data whose absolute difference exceeds a preset depth difference threshold are identified as locally anomalous depth data. The initial invalid depth data and the locally anomalous depth data are jointly identified as invalid depth data, and the remaining candidate valid depth data are identified as valid depth data. The valid depth data and their corresponding position coordinates together constitute a subset of valid depth data. Neighborhood interpolation is performed on the valid depth data within a preset neighborhood of the position coordinates corresponding to the invalid depth data to obtain an estimated depth value corresponding to the invalid depth data. The corresponding invalid depth data is replaced with the estimated depth value to obtain the target depth dataset.

[0153] Optionally, determining the initial projected physical area of ​​the target damaged region based on the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device includes: determining the number of target pixels in the target damaged region based on the target pixel-level mask; determining the depth value corresponding to each target pixel based on the target depth dataset; determining the uncorrected single-pixel projected physical area corresponding to each target pixel based on the depth value and the lateral and longitudinal focal length parameters in the calibration parameters of the imaging device; accumulating the uncorrected single-pixel projected physical areas corresponding to each target pixel in the target damaged region to obtain the uncorrected projected physical area of ​​the target damaged region; and determining the initial projected physical area of ​​the target damaged region based on the uncorrected projected physical area.

[0154] Optionally, determining the initial projected physical area of ​​the target damage region based on the uncorrected projected physical area includes: according to a preset geometric scaling factor. For the uncorrected projected physical area Scale correction is performed to obtain the initial projected physical area. The geometric scaling factor is obtained by acquiring color and depth images of a calibration template with a known physical area using an imaging device, and by performing offline calibration based on the ratio between the calculated area of ​​the calibration template and the known physical area.

[0155] Optionally, the initial projected physical area is corrected based on the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, to obtain the target surface area of ​​the target damage region. This includes: correcting the initial projected physical area based on the discrete coefficients or surface tilt angles of the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, to obtain the target surface area of ​​the target damage region.

[0156] Optionally, the coefficient of variation is determined by the following steps: determining the mean and standard deviation of the effective depth data subset based on the effective depth data subset; and determining the coefficient of variation based on the ratio of the standard deviation to the mean.

[0157] Optionally, the initial projected physical area is corrected based on the discrete coefficients of the spatial geometric features of the surface where the target damaged region is located, as represented by the target depth dataset, to obtain the target surface area of ​​the target damaged region, including: when the discrete coefficients... When the image is within the preset discrete coefficient calibration range corresponding to the current imaging configuration of the shooting device, according to the discrete coefficient and preset empirical discrete correction coefficients Determine the area correction factor According to the area correction factor For the initial projected physical area The target surface area is obtained by making corrections. .

[0158] Optionally, the initial projected physical area is corrected based on the surface tilt angle of the spatial geometric features of the surface where the target damaged area is located, as represented by the target depth dataset, to obtain the target surface area of ​​the target damaged area. This includes: back-projecting each target damaged area pixel onto the three-dimensional coordinate system of the imaging device based on the position coordinates of the target damaged area pixels in the effective depth data subset, the depth value corresponding to each target damaged area pixel, and the calibration parameters of the imaging device, to obtain a three-dimensional point set corresponding to the target damaged area; performing plane fitting on the three-dimensional point set to obtain a fitting plane of the surface where the target damaged area is located and the normal vector of the fitting plane; and determining the surface tilt angle based on the angle between the normal vector and the optical axis direction of the imaging device. ;according to For the initial projected physical area The target surface area of ​​the target damaged region is obtained by making corrections. The processing module can also be used to execute any of the implementation methods in the foregoing method embodiments regarding single-class instance segmentation, multi-class segmentation, depth data validity determination and neighborhood interpolation, projected physical area calculation, area correction based on discrete coefficients, and area correction based on plane fitting and surface tilt angle. The relevant technical features, parameter limitations, and technical effects in the foregoing method embodiments are applicable to this device embodiment and will not be repeated here.

[0159] It should be noted that this device is the same as the method described above. All implementations of the method described above are applicable to the embodiments of this device and can achieve the same technical effect.

[0160] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the above-described method. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0161] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for determining the area of ​​damaged zones in ship steel plates, characterized in that, include: Acquire the image to be detected of the surface to be tested captured by the imaging device. The image to be detected includes a time-synchronized and spatially registered color image of the surface to be tested and a depth image of the surface to be tested. The color image of the surface to be tested is subjected to target region segmentation processing to obtain a target pixel-level mask of the target damage region. The target pixel-level mask includes at least one target damage instance mask. The target damage area includes at least one of a coating damage area and a corrosion area; Based on the target pixel-level mask, the target depth dataset corresponding to the target damage area is extracted from the depth image of the surface to be tested; The initial projected physical area of ​​the target damage region is determined based on the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device. Based on the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damage region.

2. The method for determining the area of ​​damaged steel plate in a ship according to claim 1, characterized in that, The color image of the surface under test is segmented to obtain a target pixel-level mask of the target damaged area, including: A pre-defined single-class instance segmentation model maps one or more categories, such as coating damage and corrosion, in the color image of the surface under test to a target damage category. It then identifies the target damage region corresponding to the target damage category and outputs a pixel-level mask for that target damage region. The training process of the single-class instance segmentation model includes: Acquire multiple target damage images with pixel-level target damage instance annotations, wherein the target damage instance annotations include target damage category labels for training and target damage instance masks for training. While keeping the pixel range of each target damage instance mask used for training unchanged, the target damage category labels used for training are merged into a unified category label to obtain a single-class labeled dataset. The single-class labeled dataset is expanded using data augmentation strategies to obtain a surface damage dataset. The data augmentation strategies include at least one of random rotation, random scaling, and color space enhancement. The surface damage dataset is divided into a training set and a validation set. The preset segmentation network model is trained using the training set, and hyperparameter selection, early stopping judgment, and optimal model weight selection are performed based on the performance evaluation results on the validation set to obtain the single-class instance segmentation model.

3. The method for determining the area of ​​damaged steel plate in a ship according to claim 1, characterized in that, Based on the target pixel-level mask, the target depth dataset corresponding to the target damage region is extracted from the surface depth image to be tested, including: Determine the position coordinates of the pixels corresponding to the target damage area in the target pixel-level mask; Based on the location coordinates, the depth value at the corresponding location is extracted from the depth image of the surface to be measured to obtain the initial depth data; The initial depth data is initially evaluated for validity. Depth data with a depth value of zero or exceeding the preset effective range is identified as initial invalid depth data, while the remaining depth data is identified as candidate valid depth data. Based on the target pixel-level mask and candidate valid depth data, a local reference depth value is determined; Local anomalies are determined based on the absolute difference between the candidate valid depth data and the local reference depth value, and a determination result is obtained. Based on the determination result, candidate valid depth data whose absolute difference exceeds the preset depth difference threshold are identified as local abnormal depth data; The initial invalid depth data and the local abnormal depth data are jointly determined as invalid depth data, and the remaining candidate valid depth data are determined as valid depth data. The valid depth data and their corresponding position coordinates together constitute a subset of valid depth data. Neighborhood interpolation is performed on the valid depth data within a preset neighborhood of the location coordinates corresponding to the invalid depth data to obtain the estimated depth value corresponding to the invalid depth data. The target depth dataset is obtained by replacing the corresponding invalid depth data with the estimated depth values.

4. The method for determining the area of ​​damaged steel plate in a ship according to claim 1, characterized in that, Based on the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device, the initial projected physical area of ​​the target damage region is determined, including: The number of target pixels in the target damage region is determined based on the target pixel-level mask. Based on the target depth dataset, determine the depth value corresponding to each target pixel; Based on the depth value and the horizontal and vertical focal length parameters in the calibration parameters of the shooting device, determine the uncorrected single-pixel projected physical area corresponding to each target pixel; The uncorrected projected physical area of ​​the target damage region is obtained by accumulating the uncorrected single-pixel projected physical area corresponding to each target pixel in the target damage region. Based on the uncorrected projected physical area, the initial projected physical area of ​​the target damage region is determined.

5. The method for determining the area of ​​damaged zones in ship steel plates according to claim 4, characterized in that, Based on the uncorrected projected physical area, the initial projected physical area of ​​the target damage region is determined, including: Based on the preset geometric scaling factor For the uncorrected projected physical area Scale correction is performed to obtain the initial projected physical area. ; The geometric scaling factor is obtained by acquiring color and depth images of a calibration template with a known physical area using an imaging device, and by performing offline calibration based on the ratio between the calculated area of ​​the calibration template and the known physical area.

6. The method for determining the area of ​​damaged steel plate in a ship according to claim 3, characterized in that, Based on the spatial geometric features of the surface where the target damaged region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damaged region, including: Based on the discrete coefficients or surface tilt angles of the spatial geometric features of the surface where the target damaged region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damaged region.

7. The method for determining the area of ​​damaged steel plate in a ship according to claim 6, characterized in that, The discrete coefficients are determined through the following steps: Based on the effective depth data subset, determine the mean and standard deviation of the effective depth data subset; The coefficient of variation is determined based on the ratio of the standard deviation to the mean.

8. The method for determining the area of ​​damaged steel plate in a ship according to claim 6, characterized in that, Based on the discrete coefficients of the spatial geometric features of the surface where the target damaged region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damaged region, including: When the discrete coefficients When the image is within the preset discrete coefficient calibration range corresponding to the current imaging configuration of the shooting device, according to the discrete coefficient and preset empirical discrete correction coefficients Determine the area correction factor ; According to the area correction factor For the initial projected physical area The target surface area is obtained by making corrections. .

9. The method for determining the area of ​​damaged zones in ship steel plates according to claim 6, characterized in that, Based on the surface tilt angle of the surface of the target damaged region, as characterized by the spatial geometric features of the surface represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damaged region, including: Based on the position coordinates of the pixels in the target damage area in the effective depth data subset, the depth value corresponding to each pixel in the target damage area, and the calibration parameters of the imaging device, each pixel in the target damage area is back-projected onto the three-dimensional coordinate system of the imaging device to obtain the three-dimensional point set corresponding to the target damage area. The three-dimensional point set is fitted with a plane to obtain the fitting plane of the surface where the target damage area is located and the normal vector of the fitting plane; The surface tilt angle is determined based on the angle between the normal vector and the optical axis of the imaging device. ; according to For the initial projected physical area The target surface area of ​​the target damaged region is obtained by making corrections. .

10. A device for determining the area of ​​damaged regions in ship steel plates, characterized in that, include: The acquisition module is used to acquire the image of the surface to be tested captured by the imaging device. The image to be tested includes a time-synchronized and spatially registered color image of the surface to be tested and a depth image of the surface to be tested. The processing module is used to perform target region segmentation processing on the color image of the surface to be tested to obtain a target pixel-level mask of the target damage region, wherein the target pixel-level mask includes at least one target damage instance mask. The target damage area includes at least one of a coating damage area and a corrosion area; based on the target pixel-level mask, the target depth dataset corresponding to the target damage area is extracted from the depth image of the surface to be tested; The initial projected physical area of ​​the target damage region is determined based on the target pixel-level mask, the target depth dataset, and the calibration parameters of the imaging device. Based on the spatial geometric features of the surface where the target damage region is located, as represented by the target depth dataset, the initial projected physical area is corrected to obtain the target surface area of ​​the target damage region.

Images