A ship's name plate detection method combined with ship hull structure analysis

Abstract

The application discloses a ship nameplate detection method combined with ship body structure analysis, and relates to the technical fields of image processing, target detection and artificial intelligence. The steps are as follows: S1, inputting an image and detecting and positioning a ship in the image; S2, obtaining an image mask of a ship body part based on an adaptive threshold segmentation algorithm; S3, dividing the ship body into a bow / tail part and a middle part according to a proportion; S4, detecting the ship body as a whole first, and if no text information is detected, then detecting the bow / tail part, and if no text information is detected in the bow / tail part, then detecting the middle part last, and calling a text detection model to obtain a ship nameplate position. The application is aimed at the problems of small ship nameplate area, complex background image and great detection and positioning difficulty, fuses prior knowledge that a ship nameplate placement position is mainly in a bow / tail part, and through processes of adding ship body extraction, ship body structure segmentation and setting a detection sequence, the accuracy of a general text detection model in a ship nameplate detection scene is improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing, object detection, and artificial intelligence applications, and specifically to a method for detecting ship license plates that combines hull structure analysis. Background Technology

[0002] China is one of the world's richest countries in fishery resources, possessing vast sea areas and abundant aquatic flora and fauna. According to statistics released by the Ministry of Agriculture in 2022, the total output value of the fishery economy was 3,087.314 billion yuan, with a fishing population of 16.1945 million and a total of 511,000 fishing vessels. With the strengthening of environmental protection efforts and the implementation of fishing ban policies, the sheer number of fishing vessels and their dispersed operating areas have significantly increased the difficulty for relevant departments in monitoring fishing vessels and illegal transport vessels.

[0003] With the development of artificial intelligence, deep learning-based automatic text region localization technology has been widely applied in industries such as finance and land transportation, greatly improving work efficiency. However, due to numerous problems such as small target area, complex background, and lack of datasets in fishing boat license plate images, text detection algorithms have not been widely applied in vessel license plate detection scenarios. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a vessel license plate detection method that incorporates ship structure analysis, specifically addressing the challenges of small license plate area, complex background images, and high detection and localization difficulty. This invention integrates prior knowledge that the license plate is primarily located at the bow / stern of the vessel, and improves the accuracy of general text detection models in license plate detection scenarios by incorporating processes such as hull extraction, hull structure segmentation, and setting a detection order.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for detecting vessel license plates that incorporates hull structure analysis includes the following steps:

[0007] S1: Acquire video images of the river channel, detect and locate the vessels in the images, and obtain the vessel's location information;

[0008] S2: Based on the acquired ship position information, obtain the image mask of the ship hull using an adaptive threshold segmentation algorithm;

[0009] S3: Divide the ship's hull proportionally into two parts: the bow, the stern, and the middle, and overlay an image mask to retain only the image of the hull.

[0010] S4: Following the detection order of bow and stern first, then the middle of the boat, call the text detection model to perform detection and obtain the location of the boat license plate.

[0011] Furthermore, the specific steps of S1 are as follows:

[0012] S101: Acquire river monitoring video and load the video stream into image frames;

[0013] S102: Use the YOLOv5 network to perform target detection on the image frame and obtain the position information of the ship in the current image.

[0014] Furthermore, the specific steps of S2 are as follows:

[0015] S201: Based on the ship position information obtained in step S1, crop the image frames to obtain the ship target image block;

[0016] S202: Using the maximum inter-class variance thresholding algorithm in the OpenCV vision library, the ship target image patch is segmented into two parts: the foreground ship hull and the background water.

[0017] S203: Using morphological operation algorithms in the OpenCV vision library, slender protrusions are eliminated, and gaps inside the hull are connected. Finally, the largest connected component in the image is extracted as the hull mask image.

[0018] Furthermore, the specific steps for S3 are as follows:

[0019] S301: Divide the hull into five equal parts according to its length, and mark them with numbers a to e from left to right on the picture;

[0020] S302: The portions of the hull a to b and the portions of the hull d to e are respectively designated as the bow and stern, and the portion of the hull b to d is designated as the midship.

[0021] S303: Overlay the image mask obtained in S2 onto the hull to remove the image information of the background water area and retain only the image of the hull.

[0022] Furthermore, the specific steps of S4 are as follows:

[0023] S401: Use the DBNet text detection model to perform text detection on the overall image of the ship hull with a mask overlaid on the ship hull portion image;

[0024] S402: If no text information is detected in the overall image of the ship, continue to detect the bow and stern. If no text information is detected in the bow and stern images, continue to detect text in the image of the middle section of the ship to obtain the location of the ship's license plate.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. This invention uses a hull segmentation extraction algorithm to reduce interference from background waters and irrelevant information such as ship railings and flags, retaining only the image of the hull part for subsequent detection and recognition processes, making the detection algorithm more focused on locating the ship's nameplate text.

[0027] 2. This invention integrates prior knowledge that the placement of ship license plates is mainly at the bow and stern of the ship. By segmenting the ship's hull structure and setting the detection order, it improves the detection accuracy of the general text detection model in ship license plate detection scenarios. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the method flow in an embodiment of the present invention.

[0029] Figure 2 This is a flowchart illustrating the ship positioning and hull extraction process in an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the DBNet text detection network structure in an embodiment of the present invention.

[0031] Figure 4 This is a flowchart illustrating the hull structure analysis and vessel registration inspection process in an embodiment of the present invention. Detailed Implementation

[0032] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0033] A method for detecting vessel license plates that combines hull structure analysis, such as... Figure 1 As shown, the specific process is as follows:

[0034] S1: Ship positioning. Input image, detect and locate the ship in the image, and obtain the ship's position information.

[0035] The specific method is as follows:

[0036] S101: Acquire river monitoring video and load the video stream into image frames.

[0037] S102: Use the YOLOv5 network to perform target detection on the image frame and obtain the position information of the ship in the current image.

[0038] YOLOv5 is an open-source one-stage object detection algorithm that can directly regress the location and category of bounding boxes at the output layer. The network structure mainly consists of four parts: input, backbone, neck, and prediction. The input uses the Mosaic data augmentation algorithm, which concatenates training data by randomly scaling, cropping, and rearranging it to enhance data and improve object detection performance. The backbone network, composed of Focus, CBL, CSP, and SPP modules, is used to abstract and extract image features. The neck uses an FPN+PAN structure to construct a feature pyramid, fusing detailed information from shallow features with global semantic information from deep features, enabling better detection of targets of different sizes. Finally, the prediction module outputs the final feature mapping vector, including target location, category, and confidence level.

[0039] S2: Ship hull segmentation and extraction. Based on the obtained ship position information, an image mask of the ship hull is obtained using an adaptive threshold segmentation algorithm.

[0040] The specific method is as follows:

[0041] S201: Based on the ship position information obtained in step S1, crop the image frames to obtain the ship target image block.

[0042] S202: Using the maximum inter-class variance thresholding algorithm in the OpenCV vision library, the ship target image patch is segmented into two parts: the foreground ship hull and the background water.

[0043] Thresholding segmentation is a classic method in image segmentation. It utilizes the difference in grayscale between the target and the background in an image, and divides pixels into several classes by setting a threshold, thereby achieving separation of the target and the background. Otsu's method is a method that automatically determines the threshold using the maximum inter-class variance. Variance is a measure of the uniformity of grayscale distribution. The larger the inter-class variance between the background and the foreground, the greater the difference between the two parts of the image. When part of the foreground is misclassified as background or part of the background is misclassified as foreground, the difference between the two parts decreases. Therefore, segmentation that maximizes the inter-class variance means minimizing the probability of misclassification.

[0044] The code part is implemented by calling the OpenCV library function Threshold().

[0045] S203: Using morphological operation algorithms in the OpenCV vision library, slender protrusions such as ship railings are eliminated, small gaps inside the hull are connected, and finally the largest connected component in the image is extracted as the hull mask image.

[0046] Morphology is a crucial research area in image processing. It primarily extracts component information from an image, which is often essential for representing and depicting the image's shape, representing the most fundamental shape features used in image understanding. Erosion and dilation operations are fundamental to morphological operations. Erosion eliminates image boundary points, causing the image to shrink inwards along the boundaries, and can also remove portions smaller than a specified structural element. Dilation expands image boundaries, merging background points that contact the current object into the current object, thus expanding the image boundary points outwards.

[0047] A connected region is an image region consisting of foreground pixels with the same pixel value and adjacent positions. Connected region analysis involves identifying and labeling each connected region in an image. After performing thresholding and erosion / dilation operations on the ship image, the contour of the largest connected region is extracted and filled into its interior to obtain the ship hull mask image.

[0048] The code implements this by calling OpenCV library functions such as dilate(), erode(), connectedComponents(), findContours(), and fillPoly().

[0049] The algorithm flow and results of S1 and S2 are shown in [link to algorithm description]. Figure 2 .

[0050] S3: Ship structure analysis, the ship body is divided into two parts according to the proportion: bow, stern and midship, and an image mask is superimposed to retain only the image of the ship body.

[0051] The specific method is as follows:

[0052] S301: Divide the hull into five equal parts according to its length, and mark them with numbers a to e from left to right on the picture.

[0053] S302: Sections a to b and d to e of the hull are the bow and stern, respectively, while section b to d of the hull is the midship.

[0054] It should be noted that there is overlap between the middle of the ship and the bow and stern. This is to avoid omissions when performing text detection at the edges of each image block, and also to facilitate the embedding and fusion of detection boxes.

[0055] S303: Overlay the hull mask image obtained in S2 onto the hull, eliminating the image information of the background water area and retaining only the image of the hull portion.

[0056] To make the detection algorithm more focused on locating the ship's license plate text and reduce interference from background waters, ship railings, flags, and other irrelevant information, a ship hull mask image can be used, retaining only the image of the ship hull for subsequent detection and recognition processes.

[0057] S4: Ship license plate text detection. Following the detection order of bow and stern first, then the middle of the ship, the text detection model is called to obtain the position of the ship license plate.

[0058] The specific method is as follows:

[0059] S401: Use the DBNet text detection model to perform text detection on the overall image of the ship's hull with an overlaid hull mask.

[0060] DbNet (Deep Bidirectional Network) is a deep learning model for text detection. It can detect text regions in images and output their location and size information. Addressing the time-consuming post-processing issue caused by thresholding binarization in segmentation-based methods, it proposes a learnable threshold and cleverly designs a binarization function approximating a step function. This allows the segmentation network to learn the text segmentation threshold end-to-end during training. Automatic threshold adjustment not only improves accuracy but also simplifies post-processing, enhancing the performance of text detection.

[0061] The implementation process of DBnet is as follows:

[0062] (1) Preprocessing: The input image is cropped, normalized and other preprocessing operations to make it suitable for input into the CNN for training and prediction.

[0063] Specifically: randomly crop a 640*640 pixel image block from the input image, and then perform normalization processing.

[0064] (2) Feature extraction: Convolutional neural network is used as the backbone network to extract image features from the input image, and multi-scale features are obtained by using FPN structure. Feature maps at different scales are extracted and stitched together, and finally the feature map (F) is output for the next step of processing.

[0065] Specifically: The image is processed by a ResNet34 network to extract downsampled feature maps of sizes 1 / 4, 1 / 8, 1 / 16, and 1 / 32, which are then upsampled to 1 / 4 size and stitched together to obtain feature map F.

[0066] (3) Detection head output: The detection head consists of three parts: threshold feature map (T), prediction probability map (P), and approximate binary map (B).

[0067] Specifically: The input to the detection head is the feature map F; the generation network for the threshold feature map T includes Conv, BatchNorm, ReLU, and other network layers, and then upsamples to obtain a feature vector with the same size as the original image. Finally, a Sigmoid function operation is performed on it to map the threshold feature map to the range (0,1). The prediction probability map P is calculated through multiple convolutional and deconvolutional layers, and finally, the density feature map is also mapped to the range (0,1) using the Sigmoid function. The approximate binary map B is calculated based on the probability map P and the threshold map T using a differentiable binary equation. Its calculation formula is:

[0068]

[0069] Where k represents the magnification factor, set to 50, P i,j To predict the value of point (i,j) in the probability graph, T i,j Let be the value of point (i,j) in the threshold feature map.

[0070] DBNet text detection network structure is as follows: Figure 3 As shown. Taking 3x3 Conv-BatchNorm-ReLU, / 4, 64*160*160 as an example, 3x3 conv means that the filter of the convolutional layer is 3x3 in size, / 4 means that the generated feature map is downsampled by a factor of 4 relative to the original image, and 64*160*160 means that the output feature map is 64 channels and 160*160 in size. And so on for other convolutional layer parameters.

[0071] (4) Prediction results: The approximate binary image B is binarized using a fixed threshold of 0.2. Pixels larger than the threshold are set to 1. The finally obtained connected region is the text prediction region.

[0072] S402: If no text information is detected in the overall image of the ship, continue to detect text in the bow and stern sections. If no text information is detected in the bow and stern sections, continue to detect text in the images of the middle section of the ship.

[0073] It's worth noting that because the image needs to be scaled to a fixed size before being input into the text detection model for prediction and inference, and the license plate area occupies a very limited proportion of the entire ship's area, smaller license plates may be difficult to detect. By dividing the ship's hull into regions and detecting each region separately, the model's detection performance can be greatly improved.

[0074] The S3 and S4 algorithm flowcharts and effects are shown in [link to algorithm description]. Figure 4 .

Claims

1. A method for detecting vessel license plates based on hull structure analysis, characterized in that, Includes the following steps: S1: Acquire video images of the river channel, detect and locate the vessels in the images, and obtain the vessel's location information; S2: Based on the acquired ship position information, obtain the image mask of the ship hull using an adaptive threshold segmentation algorithm; S3: Divide the ship's hull proportionally into two parts: the bow, the stern, and the middle, and overlay an image mask to retain only the image of the hull. S4: Following the detection order of bow and stern first, then middle of the boat, call the text detection model to perform detection and obtain the location of the boat license plate; The specific steps of S2 are as follows: S201: Based on the ship position information obtained in step S1, crop the image frames to obtain the ship target image block; S202: Using the maximum inter-class variance thresholding algorithm in the OpenCV vision library, the ship target image patch is segmented into two parts: the foreground ship hull and the background water. S203: Use morphological operation algorithms in the OpenCV vision library to eliminate slender protrusions, connect the gaps inside the hull, and finally extract the largest connected component in the image as the hull mask image. The specific steps in S3 are as follows: S301: Divide the hull into five equal parts according to its length, and mark them with numbers a~e from left to right on the picture; S302: The portions of the hull a~b and hull d~e are respectively designated as the bow and stern, and the portion of the hull b~d is designated as the midship; S303: Overlay the image mask obtained in S2 onto the hull to eliminate image information of the background water area and retain only the image of the hull; The specific steps of S4 are as follows: S401: Use the DBNet text detection model to perform text detection on the overall image of the ship hull with a mask overlaid on the ship hull portion image; S402: If no text information is detected in the overall image of the ship, continue to detect the bow and stern. If no text information is detected in the bow and stern images, continue to detect text in the image of the middle section of the ship to obtain the location of the ship's license plate.

2. The method for detecting vessel license plates based on hull structure analysis according to claim 1, characterized in that, The specific steps for S1 are as follows: S101: Acquire river monitoring video and load the video stream into image frames; S102: Use the YOLOv5 network to perform target detection on the image frame and obtain the position information of the ship in the current image.

Images