A container damage automatic detection method and system based on multi-view stitching

Abstract

The present application relates to the technical fields of computer vision and intelligent detection, and particularly relates to a kind of container damage automatic detection method and system based on multi-view splicing;When the present application carries out image preprocessing operation to multi-view image, by adopting optical character recognition, the box number area is eliminated, and finally only the remaining non-box number strong texture area is subjected to directional Gaussian smoothing filtering, the interference features leading to splicing misplacement are eliminated at the source, while the damage edge and box number information are completely retained.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent inspection technology, and in particular to an automatic container damage detection method and system based on multi-view stitching. Background Technology

[0002] As the core carrier of cargo transportation, the appearance integrity of containers directly affects transportation safety and cargo quality. Traditional container damage detection mainly relies on manual visual inspection, which suffers from low efficiency, strong subjectivity, and high missed detection rates. In recent years, automatic detection methods based on monocular vision have gradually emerged, but they are limited by the limited shooting range of a single view, only able to detect a partial area, requiring multiple adjustments to the camera angle, resulting in a cumbersome inspection process and easy omission of damage due to blind spots. To solve the problem of insufficient coverage of a single view, some technologies adopt multi-view stitching methods, expanding the detection range by fusing images from multiple perspectives. In existing container detection methods based on multi-view stitching, image preprocessing and feature extraction typically employ classic algorithms such as ORB. These algorithms perform well in ordinary natural scenes, but in container surface detection, the outer surface of containers often has strong texture elements such as container numbers printed on it. These elements appear as high-frequency grayscale changes in the image and are easily misjudged as matchable feature points by feature extraction algorithms. During cross-view matching, the same container number or repeated patterns in different locations are incorrectly associated, resulting in ghosting, misalignment, or artifacts in the stitching results. This affects the subsequent location of damaged areas and may even completely cover up real dents or scratches, causing non-damaged textures to be misused for feature matching and reducing stitching and detection accuracy.

[0003] Meanwhile, strong texture suppression typically employs standard two-dimensional Gaussian filtering for smoothing. However, strong textures on container surfaces often exhibit clear regularity. For instance, container patterns are generally diagonal lines at a 45° angle. Since the standard Gaussian kernel is strictly isotropic, smoothing these directional textures averages the pixels in each direction, resulting in excessive blurring of the actual damaged edges perpendicular to the texture direction. Furthermore, while suppressing container patterns, the standard Gaussian filter significantly attenuates the gradient amplitude of weak edges such as dents and shallow scratches, increasing the difficulty of subsequent damage detection. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an automatic container damage detection method and system based on multi-view splicing, which solves the problems existing in the prior art.

[0005] This invention provides an automatic container damage detection method based on multi-view splicing, the method comprising the following steps: S1: Acquire multi-view images of the container to be inspected; S2: Perform image preprocessing on the multi-view image to obtain a preprocessed image; S2 specifically comprises: S2.1: performing grayscale conversion and local variance calculation on the multi-view image; S2.2: marking strong texture candidate regions on the multi-view image based on the calculated local variance, generating a strong texture candidate mask; S2.3: removing box number textures from the strong texture candidate mask using optical character recognition, obtaining an interference region mask that needs to be smoothed and suppressed; S2.4: performing local directional smoothing on the interference region mask that needs to be smoothed and suppressed based on an improved adaptive elliptic Gaussian kernel, obtaining a preprocessed image; S3: Perform feature extraction and matching on the preprocessed image to obtain a set of matching point pairs; S4: Based on the matching points, perform multi-view image stitching on the set to obtain a panoramic image; S5: Detect damaged areas in the panoramic image.

[0006] Preferably, in step S2.1, the multi-view image is first converted to grayscale image G(x,y), wherein the conversion formula is: ; In the formula, R(x,y), G(x,y) and B(x,y) are the red, green and blue channel pixel values ​​of the multi-view image at coordinates (x,y), respectively; Then, the local variance map V(x,y) of the converted grayscale image is calculated. The formula for calculating the local variance map V(x,y) of the converted grayscale image is as follows: ; In the formula, N is the side length of the local window, k is the window radius, and μ(x,y) is the average gray value in the 5×5 neighborhood centered at (x,y).

[0007] Preferably, in step S2.2, a local variance threshold is set to mark strong texture candidate regions in the multi-view image; The local variance threshold is set based on the statistical characteristics of the multi-view images, wherein the local variance threshold for each image in the multi-view images is: ; In the formula, Let mean(V) be the local variance threshold of image V, std(V) be the average local variance of image V, and λ be the adjustment coefficient. Generated strong texture candidate masks The expression is: ; In the formula, V(x,y) is the local variance map V(x,y) of the converted grayscale image.

[0008] Preferably, S2.3 specifically includes: For the strong texture candidate mask Perform morphological dilation to obtain dilated candidate masks. The structural element of the morphological dilation operation is a 5×5 rectangular kernel; The dilated candidate mask is used as the input image and fed into the Tesseract-OCR model for container number region recognition. Semantic verification is then performed to obtain the container number region result set R. valid ; Each element of the result set output by the Tesseract-OCR model contains: a text string s i Confidence score c i and bounding box coordinates (x (i) min ,y (i) min ,x (i) max ,y (i) max ); Semantic verification was performed on the result set output by the Tesseract-OCR model to obtain the box number region result set R. valid ; Let R be the set of results that pass the verification. valid ; Then, a morphological closing operation is performed on the text strings in the result set of the container number region to obtain the container number protection mask: Remove the box number protection mask from the strong texture candidate mask to obtain the interference region mask that needs to be smoothed and suppressed; The expression is: ; In the formula, For the mask of the interference region that needs to be smoothly suppressed, As a candidate mask after dilation, This represents the result of the logical NOT operation of the box number protection mask after the morphological closing operation.

[0009] Preferably, the semantic verification rule is: Length filtering: Filter results where the string length is between 10 and 11 characters; Format validation: Verify that the first 4 characters are a valid container owner code; Confidence level filtering: Remove confidence level c i Low-quality test results <0.7.

[0010] Preferably, S2.4 specifically includes: Sa: Calculate the local texture principal direction of each connected component region of the mask of the interference region that needs to be smoothed and suppressed; Sb: Construct an improved adaptive elliptic Gaussian kernel based on the local texture principal direction of each connected component region of the interference region mask that needs to be smoothed and suppressed. Sc: Based on the improved adaptive elliptic Gaussian kernel, the mask of the interference region that needs to be smoothed and suppressed is locally oriented smoothed to obtain the preprocessed image.

[0011] Preferably, in Sa, the local texture principal direction of each connected component region of the mask of the interference region to be smoothed and suppressed is calculated using a structure tensor; specifically: Calculate the gradient of the mask of the interference region that needs to be smoothed and suppressed at pixel (x,y), and then calculate its structural tensor level component. Vertical component of structure tensor and structural tensor coupling components ; Wherein, the structural tensor horizontal component The calculation formula is: ; In the formula, Let be the first-order gradient value along the horizontal direction at pixel position (x, y) of the mask of the interference region that needs to be smoothed and suppressed. Use a Gaussian window; The vertical component of the structural tensor The calculation formula is: ; In the formula, The first-order gradient value along the vertical direction at the pixel position (x,y) of the mask of the interference region that needs to be smoothed and suppressed. The structural tensor coupling components for: ; Calculate the local principal direction angle of the local texture principal direction. ; .

[0012] Preferably, the improved adaptive elliptic Gaussian kernel The expression is: ; In the formula, (u,v) are the coordinates relative to the center of the kernel. Let σ be the rotated coordinates.u σ is the standard deviation along the principal direction. v The standard deviation is along the vertical direction. The local principal direction angle is the principal direction of the local texture.

[0013] Preferably, S3 specifically comprises: S3.1: Calculate the dynamic scale factor of each of the preprocessed images; S3.2: Construct an adaptive scale pyramid based on the dynamic scale factor of the preprocessed image; S3.3: At each scale layer, use the FAST algorithm to detect corner points; S3.4: Use Hamming distance to perform coarse matching on the corner points, then perform fine matching with geometric constraints, and output a set of matching point pairs.

[0014] According to another aspect of the present invention, an automatic container damage detection system based on multi-view splicing is provided. The system employs the aforementioned automatic container damage detection method based on multi-view splicing. The system includes: The image acquisition module is used to acquire multi-view images of the container to be inspected; The image preprocessing module is used to perform image preprocessing operations on the multi-view image to obtain the preprocessed image; The feature matching module is used to extract and match features from the preprocessed image to obtain a set of matching point pairs; The image stitching module is used to stitch multiple view images of the set according to the matching points to obtain a panoramic image; The detection module is used to detect damaged areas in the panoramic image.

[0015] The embodiments of the present invention have the following technical effects: In the image preprocessing operation of multi-view images, this invention uses optical character recognition to remove box number areas, and finally performs directional Gaussian smoothing filtering only on the remaining non-box number strong texture areas, thereby eliminating interference features that cause splicing misalignment at the source, while completely preserving damaged edges and box number information.

[0016] This invention constructs an elliptical Gaussian kernel with its major axis parallel to the direction of the interfering texture. This kernel can perform strong smoothing along the pattern direction to eliminate periodic interference, while maintaining weak smoothing in the direction perpendicular to the texture. By eliminating periodic noise that leads to feature mismatch at the source, this invention effectively avoids ghosting and misalignment of box numbers and pattern areas in the stitched panoramic image. This provides a highly geometrically consistent image basis for the final damage detection and significantly reduces the false detection rate caused by stitching artifacts.

[0017] This invention provides a dynamic scale space construction method based on the saliency of damaged features during multi-view stitching. Instead of using a fixed scale factor, it adaptively adjusts the parameters of the scale pyramid according to the gradient magnitude distribution of the damaged region in the preprocessed image. This ensures dense sampling at the scale layer that best highlights the damaged features, thereby increasing the quantity and quality of feature points in the damaged region. Furthermore, it ensures that the damaged region has sufficient features to support stitching. At the same time, the dynamic scale strategy enables feature points to have better scale consistency across multiple views, providing high-precision control points for subsequent panoramic image stitching. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart of an automatic container damage detection method based on multi-view splicing provided by an embodiment of the present invention; Figure 2 This is a flowchart of image preprocessing operations on the multi-view image provided in an embodiment of the present invention; Figure 3 This is a flowchart of a local directional smoothing process for the mask of the interference region that needs to be smoothed and suppressed, based on an improved adaptive elliptic Gaussian kernel, provided in an embodiment of the present invention. Figure 4 This is a flowchart of feature extraction and matching of the preprocessed image provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] Example 1, Figure 1 A flowchart of an automatic container damage detection method based on multi-view splicing is shown, as follows: Figure 1 As shown, an automatic container damage detection method based on multi-view splicing includes the following steps: S1: Acquire multi-view images of the container to be inspected; Multiple CCD industrial cameras are used to acquire multi-view images of the container under inspection. The CCD industrial cameras have a resolution of 2592×1944 pixels, a pixel size of 2.2μm×2.2μm, and a maximum frame rate of 50fps. The CCD industrial cameras are mounted on supports that can move horizontally along a track, forming a three-view layout of main-left-right. The main camera C1 is located directly in front of the container, with its optical axis at an angle ≤5° to the normal of the container's front, and is used to acquire the front image of the container, i.e., the main view, covering a width of approximately 80%~90% of the container's length. The side camera C2 is mounted at a position 45° above and to the left of the main camera, with a pitch angle of approximately 45°, facing the left side of the container, covering the left side wall and the weld area at the upper left corner. The side camera C3 is mounted at a position 45° above and to the right of the main camera, symmetrical to the main camera, facing the right side of the container, covering the right side wall and the weld area at the upper right corner; the side camera is used to acquire the side view.

[0022] Each of the aforementioned CCD industrial cameras has a calculated field of view of approximately 60° × 45°. Within a working distance of 1.5m to 2.5m, it can ensure that a single image covers approximately 1 / 3 of the container's surface area, and the three cameras together cover more than 90% of the container's outer surface area.

[0023] S2: Perform image preprocessing on the multi-view image to obtain a preprocessed image; In existing container detection methods based on multi-view stitching, image preprocessing and feature extraction typically employ classic algorithms such as ORB. These algorithms perform well in ordinary natural scenes, but in container surface detection, the outer surface of containers often has strong texture elements such as container numbers printed on it. These elements appear as high-frequency grayscale changes in the image and are easily misjudged as matchable feature points by feature extraction algorithms. During cross-view matching, the same container number or repeated patterns in different locations are incorrectly associated, resulting in ghosting, misalignment, or artifacts in the stitching results. This affects the subsequent location of damaged areas and may even completely cover up real dents or scratches, causing non-damaged textures to be misused for feature matching and reducing stitching and detection accuracy.

[0024] Based on this, in this embodiment, when performing image preprocessing on multi-view images, optical character recognition is used to remove the box number area, and finally, only the remaining non-box number strong texture areas are subjected to directional Gaussian smoothing filtering to remove the interference features that cause splicing misalignment at the source, while completely preserving the damaged edges and box number information.

[0025] Specifically, such as Figure 2 As shown, S2 specifically refers to: S2.1: Perform grayscale conversion and local variance calculation on the multi-view image; Basically, the images captured by the industrial camera are RGB color images. In this step, the multi-view images are first converted to grayscale images G(x,y) to reduce computational complexity. The conversion formula is as follows: ; In the formula, R(x,y), G(x,y) and B(x,y) are the red, green and blue channel pixel values ​​of the multi-view image at coordinates (x,y), respectively.

[0026] Then, the local variance map V(x,y) of the converted grayscale image is calculated to quantify the degree of grayscale change in the neighborhood of each pixel; the formula for calculating the local variance map V(x,y) of the converted grayscale image is as follows: ; In the formula, N is the side length of the local window, which is set to N=5 in this embodiment; k is the window radius, which is set to 2 in this embodiment; and μ(x,y) is the average gray value in the 5×5 neighborhood centered at (x,y).

[0027] S2.2: Based on the calculated local variance, mark the strong texture candidate regions of the multi-view image and generate a strong texture candidate mask; In this step, in order to distinguish between strong texture candidate regions and ordinary regions, a local variance threshold is set to mark the strong texture candidate regions of the multi-view image. To improve the robustness of strong texture candidate region labeling, the local variance threshold is set based on the statistical characteristics of the multi-view images, wherein the local variance threshold for each image in the multi-view images is: ; In the formula, Let mean(V) be the local variance threshold of image V, std(V) be the average local variance of image V, and λ be the standard deviation of local variance of image V. In this step, λ = 2.0 is taken.

[0028] Generated strong texture candidate masks The expression is: ; In the formula, V(x,y) is the local variance map V(x,y) of the converted grayscale image, and the strong texture candidate mask includes box number, pattern and some high-contrast damaged areas.

[0029] S2.3: The strong texture candidate mask is subjected to box number texture removal by optical character recognition to obtain the interference region mask that needs to be smoothed and suppressed; In order to extract from the strong texture candidate mask In this step, the box number information that needs to be retained is removed, and only interfering patterns are suppressed. Optical character recognition technology is introduced to identify all areas that conform to the box number rules and remove them from the strong texture areas to be suppressed. Only decorative patterns without semantic meaning are retained for subsequent smoothing processing.

[0030] Specifically, S2.3 is as follows: For the strong texture candidate mask Perform morphological dilation to obtain dilated candidate masks. The morphological dilation operation uses a 5×5 rectangular kernel as the structural element to expand the boundary of the strong texture area and prevent subsequent optical character recognition from missing detection due to character edge breakage. The dilated candidate mask is used as the input image and fed into the Tesseract-OCR model for container number region recognition. Semantic verification is then performed to obtain the container number region result set R. valid ; The Tesseract-OCR model was trained using a training dataset specifically designed for containers. The page segmentation mode was set to PSM6, and the character set for recognition was limited to uppercase English letters and Arabic numerals 0-9 to exclude interference from punctuation marks and low-frequency characters. Each element of the result set output by the Tesseract-OCR model contains: a text string s i Confidence score c i and bounding box coordinates (x (i) min ,y (i) min ,x (i) max ,y (i) max ); Semantic verification was performed on the result set output by the Tesseract-OCR model to obtain the box number region result set R. valid ; The semantic verification rule is as follows: Length filtering: Filter results where the string length is between 10 and 11 characters; Format validation: Verify that the first 4 characters are a valid container owner code; Confidence level filtering: Remove confidence level c i Low-quality test results <0.7.

[0031] Let R be the set of results that pass the verification. valid .

[0032] Then, a morphological closing operation is performed on the text string in the box number region result set to fill the holes inside the characters and ensure that the mask completely covers the character region, thus obtaining the box number protection mask: Remove the box number protection mask from the strong texture candidate mask to obtain the interference region mask that needs to be smoothed and suppressed; The expression is: ; In the formula, For the mask of the interference region that needs to be smoothly suppressed, As a candidate mask after dilation, This represents the result of the logical NOT operation of the box number protection mask after the morphological closing operation.

[0033] This step achieves pixel-level intelligent decision-making by combining optical character recognition with high-level semantic information. This not only solves the problem of ghosting in the box number area caused by traditional methods, but also avoids the loss of damaged edges due to over-smoothing, laying the foundation for high-precision stitching.

[0034] S2.4: Based on the improved adaptive elliptic Gaussian kernel, the mask of the interference region that needs to be smoothed and suppressed is locally smoothed to obtain the preprocessed image; Strong texture suppression typically employs standard two-dimensional Gaussian filtering for smoothing. However, strong textures on container surfaces often exhibit clear regularities. For instance, container patterns are generally diagonal stripes at a 45° angle. Standard Gaussian kernels are strictly isotropic, and when smoothing these directional textures, they average pixels across all directions, resulting in excessive blurring of genuine damaged edges perpendicular to the texture direction. Simultaneously, while suppressing container patterns, standard Gaussian filtering significantly attenuates the gradient amplitude of weak edges such as dents and shallow scratches, increasing the difficulty of subsequent damage detection. Based on this, this embodiment proposes an adaptive elliptical Gaussian kernel based on the main direction of local texture. This improved kernel no longer maintains a circular shape but dynamically adjusts the shape and rotation angle of the Gaussian kernel according to the texture direction of the current suppression region, making its major axis parallel to the direction of the interfering texture. This allows for strong smoothing along the direction of the interfering texture while maintaining weak smoothing or even no smoothing in the direction perpendicular to the texture, thereby completely eliminating interference while preserving the lateral or diagonal damaged edge features to the maximum extent.

[0035] Specifically, such as Figure 3 As shown, S2.4 specifically includes: Sa: Calculate the local texture principal direction of each connected component region of the mask of the interference region that needs to be smoothed and suppressed; In this step, the local texture principal direction of each connected component region of the mask for the interference region that needs to be smoothed and suppressed is calculated using the structure tensor; specifically: Calculate the gradient of the mask of the interference region that needs to be smoothed and suppressed at pixel (x,y), and then calculate its structural tensor level component. Vertical component of structure tensor and structural tensor coupling components ; Wherein, the structural tensor horizontal component The calculation formula is: ; In the formula, The first-order gradient value along the horizontal direction at the pixel position (x,y) of the mask of the interference region that needs to be smoothed and suppressed is used to reflect the intensity of the horizontal grayscale change of the image at that position. Use a Gaussian window; The vertical component of the structural tensor The calculation formula is: ; In the formula, The first-order gradient value along the vertical direction at the pixel position (x,y) of the mask of the interference region that needs to be smoothed and suppressed is used to reflect the intensity of the vertical grayscale change of the image at that position. The structural tensor coupling components for: ; Calculate the local principal direction angle of the local texture principal direction. ; ; This angle represents the direction of the strongest texture in this area.

[0036] Sb: Construct an improved adaptive elliptic Gaussian kernel based on the local texture principal direction of each connected component region of the interference region mask that needs to be smoothed and suppressed. The improved adaptive elliptic Gaussian kernel The expression is: ; In the formula, (u,v) are the coordinates relative to the center of the kernel. Let σ be the rotated coordinates. u In this embodiment, the standard deviation along the principal direction is set as σ. u =2.5, used to smooth out directional patterns; σ v In this embodiment, the standard deviation along the vertical direction is set as σ. v =0.8, used to slightly smooth the edges while preserving the vertical direction. The local principal direction angle is the principal direction of the local texture.

[0037] Sc: Based on the improved adaptive elliptic Gaussian kernel, the mask of the interference region that needs to be smoothed and suppressed is locally smoothed to obtain the preprocessed image; According to the interference region mask that needs to be smoothed and suppressed. Perform a convolution operation on the grayscale image G(x,y) to generate a smoothed image. ; Due to the interference region mask that needs to be smoothed and suppressed The content may contain multiple strongly textured regions in different directions. In this step, a block-based convolution method is used for local directional smoothing; specifically: Traverse the mask of the interference regions that need to be smoothed and suppressed. For each pixel (x, y) in the dataset, if the pixel belongs to the suppression region, then the main direction of its connected component is used as the basis for the determination. Generate the corresponding G adapt Convolution is performed in the neighborhood of that pixel; if it does not belong to the suppression region, the original value is retained.

[0038] The expression is: ; In the formula, It represents the local principal direction of the suppression region where pixel (x,y) is located.

[0039] This embodiment constructs an elliptical Gaussian kernel with its major axis parallel to the direction of the interfering texture. This kernel can perform strong smoothing along the pattern direction to eliminate periodic interference, while maintaining weak smoothing in the direction perpendicular to the texture. By eliminating periodic noise that leads to feature mismatch at the source, the embodiment effectively avoids ghosting and misalignment of box numbers and pattern areas in the stitched panoramic image. This provides a highly geometrically consistent image basis for the final damage detection and significantly reduces the false detection rate caused by stitching artifacts.

[0040] S3: Perform feature extraction and matching on the preprocessed image to obtain a set of matching point pairs; In existing multi-view stitching technologies, the standard ORB algorithm is commonly used for feature extraction. The standard ORB algorithm uses a fixed-scale pyramid to detect FAST corner points. However, after the above processing steps, the damaged features on the container surface often appear as low-contrast, fine edges, while residual weak textures may exhibit similar forms at different scales. Fixed-scale methods struggle to uniformly capture these subtle damaged features across all levels, resulting in insufficient coverage of feature points in the actual damaged areas.

[0041] This embodiment provides a dynamic scale space construction method based on the saliency of damaged features. Instead of using a fixed scale factor, it adaptively adjusts the parameters of the scale pyramid according to the gradient magnitude distribution of the damaged region in the preprocessed image, ensuring dense sampling on the scale layer that best highlights the damaged features, thereby improving the quantity and quality of feature points in the damaged region.

[0042] Specifically, such as Figure 4 As shown, S3 specifically includes: S3.1: Calculate the dynamic scale factor of each of the preprocessed images; S3.1 specifically involves: calculating the gradient magnitude value M(x,y) of each preprocessed image; and calculating the mean gradient magnitude μ of the entire image. M Then, the dynamic scaling factor is calculated based on the mean. The dynamic scaling factor The calculation formula is: ; In the formula, μ M This represents the mean of the gradient magnitude.

[0043] S3.2: Construct an adaptive scale pyramid based on the dynamic scale factor of the preprocessed image; In this step, a dynamic scaling factor is used. Instead of the fixed factor of 1.2 in the standard ORB, an adaptive scaling pyramid with n=8 levels is constructed: The expression for the adaptive scaling pyramid is: ; In the formula, This represents the image at coordinate (x, y) on the k-th scale spatial layer. The image after preprocessing. is the dynamic scale factor of the k-th scale spatial layer.

[0044] S3.3: At each scale layer, use the FAST algorithm to detect corner points; In this step, when using the FAST algorithm to detect corner points, the corner point response threshold is adjusted to improve the sensitivity to damaged edges; The formula for calculating the corner response threshold is: ; In the formula, T f T is the adjusted corner response threshold. default This is the default value for the FAST algorithm; in this step, lowering the threshold helps extract more low-contrast damaged edge points.

[0045] S3.4: Use Hamming distance to perform coarse matching on the corner points, then perform fine matching with geometric constraints, and output a set of matching point pairs; The coarse matching of the corner points using Hamming distance is specifically as follows: Hamming distance is used as a similarity metric for a corner point P in the main view image. m Find the nearest neighbor P in the corner point set of the side view. s and the next nearest neighbor P s The matching rule is: ; The geometric constraint fine matching specifically involves: using a perspective transformation model to refine the coarse matching results, setting the interior point threshold of the Random Sample Consensus (RANSAC) algorithm to t=3.0 pixels and the number of iterations N=2000. Matching point pairs filtered by RANSAC are considered to be high-precision corresponding points, thus obtaining a set of matching point pairs.

[0046] S4: Based on the matching points, perform multi-view image stitching on the set to obtain a panoramic image; In this step, based on the set of matching point pairs filtered by random sampling consistency, an affine transformation model is used to estimate the transformation matrix between multiple views. Multi-view images are then stitched together according to the transformation matrix. During image stitching, a weighted average fusion method is used to eliminate stitching gaps, thereby outputting a panoramic image.

[0047] S5: Detect damaged areas in the panoramic image; Canny edge detection is performed on the panoramic image to obtain an edge map; morphological dilation is used to connect the broken edges of the edge map E(x,y) to form an edge map E(x,y) with continuous contours; the edge map E(x,y) with continuous contours is input into a pre-trained YOLOv5s model to output the coordinate boxes of the damaged areas.

[0048] The pre-trained YOLOv5s model has an input size of 640×640, and the training set contains 100,000 labeled images of container damage, categorized as dents, scratches, rust, and holes.

[0049] Example 2: The present invention also provides an automatic container damage detection system based on multi-view splicing. The system employs an automatic container damage detection method based on multi-view splicing as described in Example 1. The system includes: The image acquisition module is used to acquire multi-view images of the container to be inspected; The image preprocessing module is used to perform image preprocessing operations on the multi-view image to obtain the preprocessed image; The feature matching module is used to extract and match features from the preprocessed image to obtain a set of matching point pairs; The image stitching module is used to stitch multiple view images of the set according to the matching points to obtain a panoramic image; The detection module is used to detect damaged areas in the panoramic image.

[0050] Example 3: The present invention also provides an electronic device, including one or more processors and a memory.

[0051] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0052] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processor may execute the program instructions to implement the automatic container damage detection method based on multi-view splicing described in any embodiment of this application, and / or other desired functions. Various contents such as initial extrinsic parameters and thresholds may also be stored in the computer-readable storage medium.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. An automatic container damage detection method based on multi-view splicing, characterized in that, The method includes the following steps: S1: Acquire multi-view images of the container to be inspected; S2: Perform image preprocessing on the multi-view image to obtain a preprocessed image; S2 specifically comprises: S2.1: performing grayscale conversion and local variance calculation on the multi-view image; S2.2: marking strong texture candidate regions on the multi-view image based on the calculated local variance, generating a strong texture candidate mask; S2.3: removing box number textures from the strong texture candidate mask using optical character recognition, obtaining an interference region mask that needs to be smoothed and suppressed; S2.4: performing local directional smoothing on the interference region mask that needs to be smoothed and suppressed based on an improved adaptive elliptic Gaussian kernel, obtaining a preprocessed image; S3: Perform feature extraction and matching on the preprocessed image to obtain a set of matching point pairs; S4: Based on the matching points, perform multi-view image stitching on the set to obtain a panoramic image; S5: Detect damaged areas in the panoramic image.

2. The automatic container damage detection method based on multi-view splicing according to claim 1, characterized in that, In step S2.1, the multi-view image is first converted to grayscale image G(x,y), where the conversion formula is: ； In the formula, R(x,y), G(x,y) and B(x,y) are the red, green and blue channel pixel values ​​of the multi-view image at coordinates (x,y), respectively; Then, the local variance map V(x,y) of the converted grayscale image is calculated. The formula for calculating the local variance map V(x,y) of the converted grayscale image is as follows: ； In the formula, N is the side length of the local window, k is the window radius, and μ(x,y) is the average gray value in the 5×5 neighborhood centered at (x,y).

3. The automatic container damage detection method based on multi-view splicing according to claim 2, characterized in that, In step S2.2, a local variance threshold is set to mark strong texture candidate regions in the multi-view image; The local variance threshold is set based on the statistical characteristics of the multi-view images, wherein the local variance threshold for each image in the multi-view images is: ； In the formula, Let mean(V) be the local variance threshold of image V, std(V) be the average local variance of image V, and λ be the adjustment coefficient. Generated strong texture candidate masks The expression is: ； In the formula, V(x,y) is the local variance map V(x,y) of the converted grayscale image.

4. The automatic container damage detection method based on multi-view splicing according to claim 3, characterized in that, Specifically, S2.3 is as follows: For the strong texture candidate mask Perform morphological dilation to obtain dilated candidate masks. The structural element of the morphological dilation operation is a 5×5 rectangular kernel; The dilated candidate mask is used as the input image and fed into the Tesseract-OCR model for container number region recognition. Semantic verification is then performed to obtain the container number region result set R. valid ; Each element of the result set output by the Tesseract-OCR model contains: a text string s i Confidence score c i and bounding box coordinates (x (i) min ,y (i) min ,x (i) max ,y (i) max ); Semantic verification is performed on the result set output by the Tesseract-OCR model to obtain the box number region result set R. valid ; Let R be the set of results that pass the verification. valid ; Then, a morphological closing operation is performed on the text strings in the result set of the container number region to obtain the container number protection mask: Remove the box number protection mask from the strong texture candidate mask to obtain the interference region mask that needs to be smoothed and suppressed; The expression is: ; In the formula, For the mask of the interference region that needs to be smoothly suppressed, As a candidate mask after dilation, This represents the result of the logical NOT operation of the box number protection mask after the morphological closing operation.

5. The automatic container damage detection method based on multi-view splicing according to claim 4, characterized in that, The semantic verification rule is as follows: Length filtering: Filter results where the string length is between 10 and 11 characters; Format validation: Verify that the first 4 characters are a valid container owner code; Confidence level filtering: Remove confidence level c i Low-quality test results <0.

7.

6. The automatic container damage detection method based on multi-view splicing according to claim 4, characterized in that, Specifically, S2.4 is as follows: Sa: Calculate the local texture principal direction of each connected component region of the mask of the interference region that needs to be smoothed and suppressed; Sb: Construct an improved adaptive elliptic Gaussian kernel based on the local texture principal direction of each connected component region of the interference region mask that needs to be smoothed and suppressed. Sc: Based on the improved adaptive elliptic Gaussian kernel, the mask of the interference region that needs to be smoothed and suppressed is locally oriented smoothed to obtain the preprocessed image.

7. The automatic container damage detection method based on multi-view splicing according to claim 6, characterized in that, In Sa, the local texture principal direction of each connected component region of the mask of the interference region to be smoothed and suppressed is calculated using the structure tensor; specifically: Calculate the gradient of the mask of the interference region that needs to be smoothed and suppressed at pixel (x,y), and then calculate its structural tensor level component. Vertical component of structure tensor and structural tensor coupling components ; Wherein, the structural tensor horizontal component The calculation formula is: ； In the formula, Let be the first-order gradient value along the horizontal direction at pixel position (x, y) of the mask of the interference region that needs to be smoothed and suppressed. Use a Gaussian window; The vertical component of the structural tensor The calculation formula is: ； In the formula, The first-order gradient value along the vertical direction at the pixel position (x,y) of the mask of the interference region that needs to be smoothed and suppressed. The structural tensor coupling components for: ； Calculate the local principal direction angle of the local texture principal direction. ; 。 8. The automatic container damage detection method based on multi-view splicing according to claim 7, characterized in that, The improved adaptive elliptic Gaussian kernel The expression is: ； In the formula, (u,v) are the coordinates relative to the center of the kernel. Let σ be the rotated coordinates. u Let σ be the standard deviation along the principal direction. v Standard deviation along the vertical direction The local principal direction angle is the principal direction of the local texture.

9. The automatic container damage detection method based on multi-view splicing according to claim 1, characterized in that, Specifically, S3 is: S3.1: Calculate the dynamic scale factor of each of the preprocessed images; S3.2: Construct an adaptive scale pyramid based on the dynamic scale factor of the preprocessed image; S3.3: At each scale layer, use the FAST algorithm to detect corner points; S3.4: Use Hamming distance to perform coarse matching on the corner points, then perform fine matching with geometric constraints, and output a set of matching point pairs.

10. An automatic container damage detection system based on multi-view splicing, characterized in that, The automatic container damage detection system based on multi-view splicing adopts the automatic container damage detection method based on multi-view splicing as described in any one of claims 1-9, and the automatic container damage detection system based on multi-view splicing includes: The image acquisition module is used to acquire multi-view images of the container to be inspected; The image preprocessing module is used to perform image preprocessing operations on the multi-view image to obtain the preprocessed image; The feature matching module is used to extract and match features from the preprocessed image to obtain a set of matching point pairs; The image stitching module is used to stitch multiple view images of the set according to the matching points to obtain a panoramic image; The detection module is used to detect damaged areas in the panoramic image.

Images