A method and system for rapid identification of multi-dimensional features of containers

Abstract

This invention relates to the fields of computer vision and intelligent detection technology, and particularly to a method and system for rapid recognition of multi-dimensional features of containers. The invention constructs a bidirectional closed-loop dynamic weighted multi-scale feature pyramid that includes top-down propagation and bottom-up calibration. Through the top-down path, higher-level features provide semantic priors indicating the high probability of character presence, guiding shallower networks to focus on container number regions rather than background noise. Combined with dynamic weights, the contrast of characters can be adaptively adjusted under different lighting conditions, solving the problem of character breakage or adhesion caused by uneven lighting in existing technologies. Through the bottom-up calibration path, the precise edge information contained in the multi-scale features is fed back to the deep aggregated features, enabling the deep aggregated features to achieve feature aggregation and calibration, thereby improving the positioning accuracy of subsequent detection heads.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent detection technology, and in particular to a method and system for rapid identification of multi-dimensional features of containers. Background Technology

[0002] The efficiency of customs clearance and the intelligent management of container yards are pain points for the container port industry. Traditional container feature recognition mainly relies on manual inspection or OCR technology. Among existing technologies, identifying containers through artificial intelligence models is a major research direction. Current technologies typically use a pipelined model architecture to extract multi-dimensional features from images. For example, the YOLO model is used to locate the container number, crop the ROI features, feed it into an OCR model to recognize characters, and then feed it into a classification model to determine the size. In this multi-model chain approach, the output quality of the previous stage model directly affects the input of the next stage during the feature extraction process, making it impossible to achieve accurate feature fusion and difficult to meet the time requirements of high-speed gantry crane operations.

[0003] Meanwhile, existing technologies employ feature pyramids to achieve feature fusion. However, traditional feature pyramids combine high-level semantics with low-level spatial information through top-down paths and lateral connections, but their fusion method is a simple linear addition. This approach treats all input images the same and lacks flexibility. Furthermore, when fusing top-down high-level features with bottom-up low-level features, a severe semantic gap exists. Existing technologies typically use 1x1 convolution to reduce dimensionality and then directly add the features, ignoring the inconsistency in spatial attention between the two, resulting in noise interference in the fused features. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for rapid identification of multi-dimensional features of containers, which solves the problems existing in the prior art.

[0005] This invention provides a method for rapid identification of multi-dimensional features of containers, the method comprising the following steps: S1: Acquire image data of the container to be inspected; S2: Perform image preprocessing on the image data to obtain preprocessed image data; S3: The preprocessed image data is subjected to multi-dimensional feature extraction and fusion operations using an improved multi-scale feature fusion recognition model to generate fused features; the improved multi-scale feature fusion recognition model consists of the following three parts: a backbone network, a dynamic feature pyramid structure, and a feature fusion module; the dynamic feature pyramid structure adopts a bidirectional closed-loop structure including: a top-down path and a bottom-up feature calibration path; S4: Perform rapid identification of multi-dimensional features of containers based on the fused features.

[0006] Preferably, the backbone network uses the EfficientNet-B4 model as the basic model skeleton. After the preprocessed image data is input into the EfficientNet-B4 model, the EfficientNet-B4 model outputs four levels of features. , of which F i Features with different resolutions, the resolution of which is 1 / 2 of the preprocessed image data. i+1 .

[0007] Preferably, the operation flow of the top-down dynamic semantic enhancement path is as follows: First, a 1×1 convolution is applied to the deepest feature F4 to perform channel dimensionality reduction, generating the highest-level feature P4; then, in order to generate the third-level feature P3, the highest-level feature P4 is upsampled and then spliced ​​and fused with the deepest feature F4 through the tensor splicing layer. The operation of upsampling the highest-level feature P4 and then fusing it with the deepest-level feature F4 through the tensor concatenation layer specifically involves: performing bilinear upsampling on the upsampled highest-level feature P4 to generate a feature. A 1×1 convolution operation is performed on the deepest feature F4 to achieve channel alignment, generating feature L4. Then, the feature... The feature L4 is concatenated along the channel dimension, and dynamic weights are generated to produce a spatial weight map α4; wherein the expression of the spatial weight map α4 is: σ is the Sigmoid function, and then, for the features... Weigh and fuse feature L4 to generate feature P3. Then, following the steps above, we generate features P2 and P1, thus generating multi-scale features. .

[0008] Preferably, the dynamic feature pyramid further includes a bottom-up feature calibration path; its operation process is as follows: the generated feature P2 is downsampled through a 3×3 convolution with a stride of 2 to obtain the feature... Then the features By fusing with feature P3, we obtain The expression is: Repeat the above steps until the top-level feature is reached, thereby generating deep aggregated features. .

[0009] Preferably, the feature fusion module is used to fuse the hierarchical features, multi-scale features, and deep aggregation features to generate fused features.

[0010] Preferably, in step S4, the fused features are input to the parallel detection head to achieve rapid identification of multi-dimensional features of the container.

[0011] Preferably, the parallel detection head includes an OCR branch, a classification branch, and a segmentation branch; the OCR branch uses the CTC loss function to decode the container number sequence; the classification branch outputs the size type corresponding to the container ISO code through a fully connected layer; and the segmentation branch uses a lightweight U-Net structure to output a pixel-level damage mask.

[0012] Preferably, in step S2, the image preprocessing operation includes region of interest filtering, perspective geometry correction based on vanishing point, and image normalization.

[0013] Preferably, in step S1, a high frame rate industrial area scan camera combined with a narrow band filter is used to acquire image data of the container to be inspected; the high frame rate industrial area scan camera is a CMOS industrial camera with a resolution of 2448×2048 and a frame rate set to 50fps. A near-infrared bandpass filter with a center wavelength of 850nm is installed in front of the lens of the CMOS industrial camera.

[0014] According to another aspect of the present invention, a rapid identification system for multi-dimensional features of containers is provided. The system employs the aforementioned rapid identification method for multi-dimensional features of containers, and the system includes: The data acquisition module is used to acquire image data of the container to be inspected; The data preprocessing module is used to perform image preprocessing operations on the image data to obtain preprocessed image data; The feature extraction module is used to perform multi-dimensional feature extraction and fusion operations on the preprocessed image data using an improved multi-scale feature fusion recognition model to generate fused features; The identification module is used to quickly identify multi-dimensional features of containers based on the fused features.

[0015] The embodiments of the present invention have the following technical effects: This invention constructs a bidirectional closed-loop dynamic weighted multi-scale feature pyramid that includes top-down propagation and bottom-up calibration. Addressing the characteristics of container number characters, which typically have large aspect ratios and thin strokes, making them easily diluted in deep networks, the top-down path provides semantic priors indicating the high probability of character presence in these areas, guiding shallower networks to focus on the container number region rather than background noise. Combined with dynamic weights, the contrast of characters can be adaptively adjusted under different lighting conditions, solving the problem of character breakage or adhesion caused by uneven lighting in existing technologies. The bottom-up calibration path feeds back the precise edge information contained in the multi-scale features to the deep aggregated features, enabling feature aggregation and calibration, thereby improving the positioning accuracy of subsequent detection heads.

[0016] The bidirectional structure of this invention establishes a feedback loop, ensuring that the deep network is always constrained by the original signals from the shallow layer. This structure ensures that even the top-level features used for global classification are not overly smoothed, preserving the necessary spatial high-frequency components, which is crucial for distinguishing containers with similar appearances.

[0017] Meanwhile, the dynamic weights of this invention allow the fusion network to adjust the fusion ratio based on the content of the input image. In clear images, the fusion network tends to reduce weights to utilize more realistic textures; in blurry images, the fusion network increases weights to utilize high-level semantics for restoration, significantly improving the model's robustness in complex environments. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a method for rapid identification of multi-dimensional features of containers provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] Example 1, Figure 1A flowchart of a method for rapid identification of multi-dimensional features of containers is shown, such as... Figure 1 As shown, a method for rapid identification of multi-dimensional features of containers includes the following steps: S1: Acquire image data of the container to be inspected; In this step, a high frame rate industrial area scan camera, combined with a narrow band filter, is used to acquire image data of the container to be inspected. The high frame rate industrial area scan camera is a CMOS industrial camera with a resolution of 2448×2048 and a frame rate of 50fps. A near-infrared bandpass filter with a center wavelength of 850nm is installed in front of the lens of the CMOS industrial camera. The near-infrared light source is minimally affected by ambient visible light, which can effectively overcome the problems of overexposure or underexposure of images caused by strong backlight at dusk or night, ensuring that clear container number and texture information can be obtained under all weather conditions.

[0022] S2: Perform image preprocessing on the image data to obtain preprocessed image data; The image preprocessing operations include region of interest filtering, perspective geometry correction based on vanishing points, and image normalization.

[0023] Due to the large area of ​​the container body, directly processing full-resolution images would be computationally intensive. This embodiment employs a two-stage region of interest (ROI) filtering method. Specifically, a lightweight YOLOv5-nano model is used to perform real-time detection on the image data, quickly locating the bounding box of the container body. Then, based on the located bounding box, a rectangular region containing the container number and the container outline is cropped from the image data as the ROI. These steps reduce invalid background pixels by more than 60%, significantly reducing the computational complexity of subsequent steps.

[0024] Because industrial cameras are typically installed above the top of the container and the shooting angles are not perfectly orthogonal, the acquired images suffer from trapezoidal distortion, severely affecting the geometric proportions of the container number. This embodiment employs a perspective transformation algorithm based on vanishing point detection to achieve geometric correction of the image data. Specifically, the Canny edge detection operator is used to extract straight line segments from the region of interest; the top and bottom border lines of the container are fitted using Hough transform, and their intersection points, i.e., perspective vanishing points, are calculated. A perspective transformation matrix is ​​then calculated based on the vanishing points and a preset standard container aspect ratio. Finally, bilinear interpolation is used to reproject the distorted image onto a standard orthogonal plane to obtain the corrected image. In this embodiment, for a 40-foot standard container, the preset standard container aspect ratio is 40:8.

[0025] The image normalization process specifically involves normalizing the pixel values ​​in the corrected image to [ ]. The interval is [1, 1].

[0026] S3: The preprocessed image data is subjected to multi-dimensional feature extraction and fusion operations using an improved multi-scale feature fusion recognition model to generate fused features; Existing technologies typically employ a pipelined model architecture to extract multi-dimensional features from images. For example, a YOLO model is used to locate the box number, ROI features are cropped, the data is fed into an OCR model to recognize characters, and then fed into a classification model to determine the size. In this multi-model chaining approach, the output quality of the previous stage directly affects the input of the next stage during feature extraction, making it impossible to achieve accurate feature fusion and difficult to meet the time requirements of high-speed gantry crane operations.

[0027] To address the aforementioned shortcomings, the improved multi-scale feature fusion recognition model in this embodiment is an end-to-end architecture with a single backbone network connecting a dynamic feature pyramid and multiple heads outputting in parallel. Specifically, the improved multi-scale feature fusion recognition model consists of the following three parts: a backbone network, a dynamic feature pyramid structure, and a feature fusion module.

[0028] Specifically, the backbone network uses the EfficientNet-B4 model as its basic model skeleton. Compared to traditional models such as ResNet and EfficientNet, EfficientNet-B4 adjusts the network's depth, width, and resolution uniformly through composite scaling factors, achieving higher accuracy with the same computational load. It is worth emphasizing that after the preprocessed image data is input into the EfficientNet-B4 model, the model outputs four levels of features. , of which F i Features with different resolutions, the resolution of which is 1 / 2 of the preprocessed image data. i+1 .

[0029] Traditional feature pyramids combine high-level semantics with low-level spatial information through top-down paths and lateral connections. However, their fusion method is a simple linear addition, which treats all input images equally and lacks flexibility. Furthermore, when fusing top-down high-level features with bottom-up low-level features, a significant semantic gap exists. Existing techniques typically use 1x1 convolutional dimensionality reduction followed by direct addition, ignoring the inconsistency in spatial attention between the two, leading to noise interference in the fused features. In this embodiment, the dynamic feature pyramid structure includes an input layer, a tensor concatenation layer, a feature compression layer, an activation layer, and an output layer. The input layer receives the four levels of features output by the EfficientNet-B4 model. The tensor splicing layer is used to splice and fuse the features in the channel dimension. The feature compression layer is used to compress the number of channels of the features from 2C to 1 to obtain the original weight score map S. The activation layer is normalized by the Sigmoid function σ(S) to obtain the final spatial weight matrix α. The output layer is used to output the feature set, which includes high-resolution features, multi-scale features and deep aggregation features.

[0030] Unlike the unidirectional propagation of traditional feature pyramid structures, the dynamic feature pyramid structure adopts a bidirectional closed-loop structure, which not only includes the traditional top-down path, but also adds a bottom-up feature calibration path.

[0031] The input layer of the dynamic feature pyramid structure receives four levels of features output by the EfficientNet-B4 model. The dynamic feature pyramid includes a top-down dynamic semantic enhancement path and a bottom-up adaptive detail calibration path. The top-down dynamic semantic enhancement path is the main construction path of the feature pyramid, responsible for injecting high-level semantic features into low-level features. The process is as follows: First, a 1×1 convolution is applied to the deepest feature F4 to perform channel dimensionality reduction, generating the highest-level feature P4. Then, in order to generate the third-level feature P3, the highest-level feature P4 is upsampled and then spliced ​​and fused with the deepest feature F4 through the tensor splicing layer. The operation of upsampling the highest-level feature P4 and then fusing it with the deepest-level feature F4 through the tensor concatenation layer specifically involves: performing bilinear upsampling on the upsampled highest-level feature P4 to generate a feature. A 1×1 convolution operation is performed on the deepest feature F4 to achieve channel alignment, generating feature L4. Then, the feature... The feature L4 is concatenated along the channel dimension, and dynamic weights are generated to produce a spatial weight map α4; wherein the expression of the spatial weight map α4 is: σ is the Sigmoid function, and then, for the features... Weigh and fuse feature L4 to generate feature P3. Then, following the steps above, we generate features P2 and P1, thus generating multi-scale features. .

[0032] Traditional feature pyramid structures stop generating features at this point, but while the deeper features are semantically strong, they may lose the new details brought about by the dynamic fusion in the first step. Therefore, this embodiment adds a reverse path, that is, the dynamic feature pyramid also includes a bottom-up feature calibration path; the process is as follows: the generated feature P2 is downsampled through a 3×3 convolution with a stride of 2 to obtain the feature... Then the features By fusing with feature P3, we obtain The expression is: This process involves back-injecting high-resolution detailed features into deeper features, repeating the above steps until the top-level features are reached, thereby generating deep aggregated features. .

[0033] This embodiment of the invention constructs a bidirectional closed-loop dynamic weighted multi-scale feature pyramid that includes top-down propagation and bottom-up calibration. Addressing the characteristic that container number characters typically have large aspect ratios and thin strokes, making them easily diluted in deep networks, the top-down path provides semantic priors indicating the high probability of character presence in these areas, guiding shallower networks to focus on the container number region rather than background noise. Combined with dynamic weights, the contrast of characters can be adaptively adjusted under different lighting conditions, solving the problem of character breakage or adhesion caused by uneven lighting in existing technologies. The bottom-up calibration path feeds back the precise edge information contained in the multi-scale features to the deep aggregated features, enabling feature aggregation and calibration, thereby improving the positioning accuracy of subsequent detection heads.

[0034] The bidirectional structure in this embodiment establishes a feedback loop, ensuring that the deep network is always constrained by the original signals from the shallow layer. This structure ensures that even the top-level features used for global classification are not overly smoothed, preserving the necessary spatial high-frequency components, which is crucial for distinguishing containers with similar appearances.

[0035] Meanwhile, the dynamic weights in this embodiment allow the fusion network to adjust the fusion ratio based on the content of the input image. In clear images, the fusion network tends to reduce weights to utilize more realistic textures; in blurry images, the fusion network increases weights to utilize high-level semantics for restoration, significantly improving the model's robustness in complex environments.

[0036] The feature fusion module is used to fuse the hierarchical features, multi-scale features, and deep aggregation features to generate fused features.

[0037] S4: Rapidly identify multi-dimensional features of containers based on the fused features; The fused features are input into a parallel detection head to achieve rapid identification of multi-dimensional features of containers.

[0038] Furthermore, the parallel detection head includes an OCR branch, a classification branch, and a segmentation branch; the OCR branch uses the CTC loss function to decode the container number sequence; the classification branch outputs the size type corresponding to the container ISO code through a fully connected layer; and the segmentation branch uses a lightweight U-Net structure to output a pixel-level damage mask.

[0039] Example 2: The present invention also provides a rapid identification system for multi-dimensional features of containers. The system employs a rapid identification method for multi-dimensional features of containers as described in Example 1. The system includes: The data acquisition module is used to acquire image data of the container to be inspected; The data preprocessing module is used to perform image preprocessing operations on the image data to obtain preprocessed image data; The feature extraction module is used to perform multi-dimensional feature extraction and fusion operations on the preprocessed image data using an improved multi-scale feature fusion recognition model to generate fused features; The identification module is used to quickly identify multi-dimensional features of containers based on the fused features.

[0040] Example 3: The present invention also provides an electronic device, including one or more processors and a memory.

[0041] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0042] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processor may execute the program instructions to implement the fast identification method for multi-dimensional features of containers described in any embodiment of this application above, and / or other desired functions. Various contents such as initial extrinsic parameters and thresholds may also be stored in the computer-readable storage medium.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for rapid identification of multi-dimensional features of containers, characterized in that, The method includes the following steps: S1: Acquire image data of the container to be inspected; S2: Perform image preprocessing on the image data to obtain preprocessed image data; S3: The preprocessed image data is subjected to multi-dimensional feature extraction and fusion operations using an improved multi-scale feature fusion recognition model to generate fused features. The improved multi-scale feature fusion recognition model consists of three parts: a backbone network, a dynamic feature pyramid structure, and a feature fusion module. The dynamic feature pyramid structure adopts a bidirectional closed-loop structure, including a top-down path and a bottom-up feature calibration path. The backbone network uses the EfficientNet-B4 model as the basic model skeleton. After the preprocessed image data is input into the EfficientNet-B4 model, the EfficientNet-B4 model outputs four levels of features. , of which F i Features with different resolutions, the resolution of which is 1 / 2 of the preprocessed image data. i+1 The operation flow of the top-down dynamic semantic enhancement path is as follows: First, apply a 1×1 convolution to the deepest feature F4 to perform channel dimensionality reduction and generate the highest feature P4; then, in order to generate the third layer feature P3, the highest feature P4 is upsampled and then spliced ​​and fused with the deepest feature F4 through a tensor splicing layer. The operation of upsampling the highest-level feature P4 and then fusing it with the deepest-level feature F4 through a tensor concatenation layer specifically involves: performing bilinear upsampling on the upsampled highest-level feature P4 to generate a feature... A 1×1 convolution operation is performed on the deepest feature F4 to achieve channel alignment, generating feature L4. Then, the feature... The feature L4 is concatenated along the channel dimension, and dynamic weights are generated to produce a spatial weight map α4; wherein the expression of the spatial weight map α4 is: σ is the Sigmoid function, and then, for the features... Weigh and fuse feature L4 to generate feature P3. Then, following the steps above, we generate features P2 and P1, thus generating multi-scale features. ; S4: Perform rapid identification of multi-dimensional features of containers based on the fused features.

2. The method for rapid identification of multi-dimensional features of a container according to claim 1, characterized in that, The dynamic feature pyramid also includes a bottom-up feature calibration path; Its operation process is as follows: the generated feature P2 is downsampled through a 3×3 convolution with a stride of 2 to obtain the feature. Then the features By fusing with feature P3, we obtain The expression is: Repeat the above steps until the top-level feature is reached, thereby generating deep aggregated features. .

3. The method for rapid identification of multi-dimensional features of a container according to claim 2, characterized in that, The feature fusion module is used to fuse the hierarchical features, multi-scale features, and deep aggregation features to generate fused features.

4. The method for rapid identification of multi-dimensional features of a container according to claim 1, characterized in that, In step S4, the fused features are input to the parallel detection head to achieve rapid identification of multi-dimensional features of the container.

5. The method for rapid identification of multi-dimensional features of a container according to claim 4, characterized in that, The parallel detection head includes an OCR branch, a classification branch, and a segmentation branch; the OCR branch uses the CTC loss function to decode the container number sequence; the classification branch outputs the size type corresponding to the container ISO code through a fully connected layer; and the segmentation branch uses a lightweight U-Net structure to output a pixel-level damage mask.

6. The method for rapid identification of multi-dimensional features of a container according to claim 1, characterized in that, In step S2, the image preprocessing operations include region of interest filtering, perspective geometry correction based on vanishing points, and image normalization.

7. The method for rapid identification of multi-dimensional features of a container according to claim 1, characterized in that, In step S1, a high frame rate industrial area array camera is used in conjunction with a narrow band filter to acquire image data of the container to be inspected. The high frame rate industrial area array camera is a CMOS industrial camera with a resolution of 2448×2048 and a frame rate of 50fps. A near-infrared bandpass filter with a center wavelength of 850nm is installed in front of the lens of the CMOS industrial camera.

8. A rapid identification system for multi-dimensional features of containers, characterized in that, The system employs a rapid identification method for multi-dimensional features of containers as described in any one of claims 1-7, and the system comprises: The data acquisition module is used to acquire image data of the container to be inspected; The data preprocessing module is used to perform image preprocessing operations on the image data to obtain preprocessed image data; The feature extraction module is used to perform multi-dimensional feature extraction and fusion operations on the preprocessed image data using an improved multi-scale feature fusion recognition model to generate fused features; The identification module is used to quickly identify multi-dimensional features of containers based on the fused features.

Images