A viT-based in-and-out port fishing boat name number area extraction method, medium and system

Abstract

The application provides a ViT-based port entry and exit fishing boat name number area extraction method, medium and system, belonging to the technical field of machine vision, and realizes progressive positioning from the whole ship body to the accurate position of the name number through a three-level pyramid feature extraction structure. First, the self-attention mechanism is used to determine the first-level interest area containing the ship body contour, then the second-level interest area where the name number is located is extracted through position coding and multi-head attention operation, and finally the fine-grained feature extraction network is applied to accurately locate the name number area. The method introduces an optical imaging degradation equation and a ship body posture light reflection model, constructs a distortion matrix and a bias matrix, generates an identifiable matrix through a multi-feature fusion function, and guides the targeted enhancement processing on the low-identifiability area. Finally, the Marine IDNet model with a double-branch structure is used to verify the extraction result, so as to realize high-precision extraction of the name number area in a complex marine environment.

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, specifically, it relates to a method, medium and system for extracting the vessel name region of fishing vessels entering and leaving ports based on ViT. Background Technology

[0002] The vessel name and registration number are crucial information for identifying fishing vessels and are of great significance in fisheries resource management, port security monitoring, and the supervision of illegal fishing activities. Traditional fishing vessel name and registration number recognition technologies mainly rely on object detection algorithms based on convolutional neural networks (CNNs), such as Faster R-CNN and YOLOv4, treating the vessel name and registration number area as a regular target for detection and localization. These methods have a certain recognition effect under ideal conditions and are widely used in scenarios such as port monitoring, maritime law enforcement, and fisheries resource management.

[0003] However, traditional CNN-based ship name region extraction techniques face numerous challenges in real-world marine environments. First, the constantly changing attitudes of fishing vessels at sea cause severe perspective distortion in the ship name region. Second, significant differences in image resolution exist at different viewing distances, with details in the ship name region becoming blurred when photographed from a distance. Furthermore, complex sea surface lighting conditions, weather factors, and the reflective properties of ship materials further increase the difficulty of identification. These factors result in generally low accuracy of traditional methods in practical applications, especially under adverse weather conditions.

[0004] Existing technologies struggle to effectively extract the names and markings of fishing vessels in complex and ever-changing environments. In particular, they lack the ability to adapt to varying observation distances, lighting conditions, and changes in vessel attitude, resulting in low accuracy in vessel name extraction and limited subsequent identification accuracy. This fails to meet the practical needs of maritime law enforcement and fisheries management. In other words, existing technologies suffer from low accuracy in automatically extracting the names and markings of fishing vessels in complex marine environments. Summary of the Invention

[0005] In view of this, the present invention provides a method, medium and system for extracting the vessel name area of ​​fishing vessels entering and leaving ports based on ViT, which can solve the technical problem of low accuracy of automatic extraction of vessel name area in complex marine environments in the prior art.

[0006] The present invention is implemented as follows: The first aspect of the present invention provides a method for extracting the vessel name region of fishing vessels entering and leaving ports based on ViT, comprising:

[0007] A three-level pyramid feature extraction structure based on the ViT (Vision Transformer model) is designed for fishing vessel image analysis. A self-attention mechanism is used to determine the first-level region of interest (ROI). A second-level ROI is extracted based on the first-level ROI. A fine-grained feature extraction network is applied to determine the third-level ROI. A distortion matrix is ​​constructed for the third-level ROI. A deviation matrix for the third-level ROI is calculated. A recognizable matrix is ​​generated based on the combined calculation of the distortion and deviation matrices. An adaptive region enhancement algorithm is designed to perform targeted image enhancement processing on low-recognizable regions according to the recognizable matrix. The final optimized image of the vessel name region is output, along with the coordinate information of the vessel name region for subsequent text recognition and verification systems.

[0008] The first-level region of interest (ROI) refers to a large-scale area containing the main structure of the hull, determined after preliminary analysis of the overall image of the fishing vessel using ViT. The first-level ROI covers the main outline of the fishing vessel and its surrounding area. The second-level ROI refers to the hull structure area determined by further narrowing the scope based on the first-level ROI. The second-level ROI includes key parts such as the hull side and the outer wall of the bridge where the ship's name is marked. The third-level ROI refers to the precise location area of ​​the ship's name after fine processing. The third-level ROI only contains the ship's name text and its adjacent background.

[0009] The distortion matrix is ​​a mathematical model that quantifies the impact of different observation distances on image sharpness. It is constructed by analyzing the attenuation of high-frequency information in the image and is used to assess the recognition difficulty caused by distance factors. The deviation matrix is ​​a mathematical model that characterizes the degree of geometric deformation of the ship name area caused by changes in the ship's attitude angle. It mainly considers the difficulties brought by perspective projection transformation to character recognition. The identifiable matrix is ​​an evaluation index generated by combining the distortion matrix and the deviation matrix. It uses numerical values ​​to quantify the recognition difficulty of the ship name area under the current observation conditions.

[0010] This includes simulating the image quality variation of the ship name area at different observation distances using an optical imaging degradation equation. The inputs to the optical imaging degradation equation include the observation distance obtained from image metadata, atmospheric visibility obtained from a meteorological database, camera optical parameters obtained from a camera parameter file, the actual size of the ship name area measured from image analysis, and the imaging time obtained from image metadata. The outputs of the optical imaging degradation equation are the distance-dependent point spread function and the signal-to-noise ratio attenuation matrix.

[0011] This includes analyzing the impact of hull attitude light reflection characteristics on ship name recognition using a hull attitude light reflection model. The inputs of the hull attitude light reflection model include the hull surface normal vector calculated from image analysis, the incident light angle obtained from the illumination estimation algorithm, the hull material reflection coefficient obtained from the material recognition network, the hull surface roughness obtained from texture analysis, and the coating aging degree obtained from image degradation assessment. The output of the hull attitude light reflection model is an evaluation matrix of effective contrast and text edge sharpness in the ship name region.

[0012] This includes applying a multi-feature fusion function to determine the recognizability of the ship name area by integrating multiple factors. The inputs of the multi-feature fusion function include the distortion matrix, the deviation matrix, the ambient brightness matrix obtained from image analysis, the ship texture complexity obtained from texture analysis, and the ship name font feature vector obtained from the font recognition network. The output of the multi-feature fusion function is the area recognizability score and optimization strategy suggestions.

[0013] The method further includes using the MarineIDNet neural network for ship name region verification and boundary fine adjustment. The specific structure of the MarineIDNet model is a dual-branch network architecture, which includes a feature extraction branch and a region verification branch. The feature extraction branch adopts an improved ViT structure, while the region verification branch consists of a four-layer convolutional neural network and two fully connected layers.

[0014] The steps for establishing the training dataset during the pre-training process of the MarineIDNet model specifically include collecting high-resolution images of different types of fishing vessels under various lighting conditions, weather environments, and observation distances; using a semi-automatic annotation tool to label the vessel name area with rectangular boxes and transcribe the text content; performing stratified sampling and equalization processing based on vessel size, observation distance, and weather conditions; and finally constructing a pre-training dataset containing 50,000 images.

[0015] A second aspect of the present invention provides a computer-readable storage medium storing program instructions, which, when executed in a computer, are used to perform the above-described method for extracting the vessel name region of fishing vessels entering and leaving ports based on ViT.

[0016] A third aspect of the present invention provides a ViT-based system for extracting the vessel name and region of fishing vessels entering and leaving ports, comprising the aforementioned computer-readable storage medium. The system can be any one of a computer, a server, or a microcontroller. The computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor that executes the program instructions stored in the computer-readable storage medium.

[0017] This invention utilizes a recognizable matrix evaluation mechanism constructed by combining distortion and bias matrices to achieve high-precision extraction of fishing vessel name regions. This method progressively and precisely locates the vessel name position through multi-level regions of interest, effectively overcoming the limitations of traditional CNN methods in adapting to complex marine environments.

[0018] By introducing an optical imaging degradation equation and a ship attitude light reflection model, this invention can accurately quantify the impact of different observation distances and changes in ship attitude on the identification of ship name regions. Furthermore, based on a recognizable matrix, it guides adaptive region enhancement processing, significantly improving the extraction quality of ship name regions under adverse conditions. In particular, the application of the MarineIDNet neural network, through a dual-branch network architecture combined with a local receptive field constraint mechanism, further ensures the accurate localization of the ship name region boundaries.

[0019] This invention solves the technical problem of low accuracy in automatically extracting the name and number area of ​​fishing vessels in complex marine environments, and improves the robustness of recognition under different observation distances, lighting conditions and vessel attitudes. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0022] like Figure 1 The diagram shown is a flowchart of a method for extracting the vessel name region of fishing vessels entering and leaving ports based on ViT, provided by the first aspect of this invention. This method includes the following steps:

[0023] S01. A three-level pyramid feature extraction structure based on ViT is designed for fishing vessel image analysis, and multiple feature representations are constructed at different scales.

[0024] S02. Use the self-attention mechanism to determine the first-level region of interest in the input image, focusing on the region in the image that has ship outline features and occupies a large visual area.

[0025] S03. Based on the first-level region of interest, the second-level region of interest is extracted through position coding enhancement and multi-head attention operation to accurately locate the area where the upper and lower structures of the hull and the ship's name are located.

[0026] S04. A fine-grained feature extraction network is applied to determine the third-level region of interest, which precisely corresponds to the location of the ship's name number. Precise boundary positioning is achieved through token segmentation technology.

[0027] S05. Construct a distortion matrix for the third-level region of interest, quantify the change in image sharpness at different observation distances using the optical imaging degradation equation, and establish a mapping function between distance and image quality degradation.

[0028] S06. Calculate the deviation matrix of the third-level region of interest, apply the ship attitude light reflection model to analyze the degree of perspective distortion in the ship name area caused by changes in ship attitude, and establish a function of attitude angle and text distortion degree.

[0029] S07. Based on the distortion matrix and the deviation matrix, a recognizable matrix is ​​generated, and the difficulty coefficient of ship name area recognition under the current conditions is quantitatively evaluated by a multi-feature fusion function.

[0030] S08. Design an adaptive region enhancement algorithm to perform targeted image enhancement processing on low-recognition regions based on the recognizable matrix.

[0031] S09. Output the final optimized image of the ship name region and provide the coordinate information of the ship name region for subsequent text recognition and verification system. The MarineIDNet neural network is used for ship name region verification and fine boundary adjustment.

[0032] The first-level region of interest refers to the large-scale area containing the main structure of the hull, determined after preliminary analysis of the overall image of the fishing vessel using ViT. The first-level region of interest covers the main outline of the fishing vessel and the surrounding area.

[0033] The second-level area of ​​interest refers to the hull structure area determined by further narrowing the scope based on the first-level area of ​​interest. The second-level area of ​​interest typically includes key parts such as the ship's side and the outer wall of the bridge where the ship's name is marked.

[0034] The third-level region of interest refers to the precise location of the ship's name after fine processing. The third-level region of interest only includes the ship's name text and its adjacent background.

[0035] The distortion matrix is ​​a mathematical model that quantifies the impact of different observation distances on image sharpness. It is constructed by analyzing the attenuation of high-frequency information in the image and is used to assess the recognition difficulty caused by distance factors.

[0036] Among them, the deviation matrix is ​​a mathematical model that characterizes the degree of geometric deformation of the ship name area caused by changes in the ship's attitude angle. The deviation matrix mainly considers the difficulties brought about by perspective projection transformation to character recognition.

[0037] The identifiable matrix is ​​an evaluation index generated by combining the distortion matrix and the deviation matrix. The identifiable matrix uses numerical values ​​to quantify the difficulty of identifying the ship name area under the current observation conditions, providing an optimization basis for subsequent processing.

[0038] The optical imaging degradation equation is used to simulate the image quality variation of the ship name area at different observation distances. The inputs of the optical imaging degradation equation include the observation distance obtained from image metadata, the atmospheric visibility obtained from the meteorological database, the camera optical parameters obtained from the camera parameter file, the actual size of the ship name area measured from image analysis, and the imaging time obtained from the image metadata. The outputs of the optical imaging degradation equation are a distance-dependent point spread function and a signal-to-noise ratio attenuation matrix. The point spread function is used for subsequent image sharpening processing, and the signal-to-noise ratio attenuation matrix is ​​used to evaluate the recognition difficulty of different areas.

[0039] The ship attitude light reflection model is used to analyze the impact of the light reflection characteristics of the ship surface on the recognition of the ship name. The input of the ship attitude light reflection model includes the ship surface normal vector calculated from image analysis, the incident light angle obtained from the illumination estimation algorithm, the ship material reflection coefficient obtained from the material recognition network, the ship surface roughness obtained from texture analysis, and the coating aging degree obtained from image degradation assessment. The output of the ship attitude light reflection model is the effective contrast of the ship name region and the text edge sharpness evaluation matrix. The effective contrast of the ship name region is used for subsequent image enhancement processing, and the text edge sharpness evaluation matrix is ​​used to determine the edge enhancement strategy.

[0040] The multi-feature fusion function is used to determine the recognizability of the ship name area by integrating multiple factors. The inputs of the multi-feature fusion function include the distortion matrix generated in the above steps, the deviation matrix calculated in the above steps, the ambient brightness matrix obtained from image analysis, the ship texture complexity obtained from texture analysis, and the ship name font feature vector obtained from the font recognition network. The output of the multi-feature fusion function is the regional recognizability score and optimization strategy suggestions. The regional recognizability score is used to determine whether enhancement processing is needed, and the optimization strategy suggestions are used to guide the selection of enhancement algorithms.

[0041] The MarineIDNet model has a dual-branch network architecture, consisting of a feature extraction branch and a region verification branch. The feature extraction branch uses an improved ViT structure, dividing the input image into 16×16 image block sequences as the basic processing unit. The embedding dimension is 768, and it contains 12 Transformer encoder layers, each with 12 self-attention heads. The feedforward neural network has a hidden layer dimension of 3072. A local receptive field constraint mechanism is introduced to improve sensitivity to small text regions, making the attention mechanism focus more on the edge details of the ship name region. The region verification branch consists of four convolutional neural network layers and two fully connected layers, with convolutional kernel sizes of 7×7, 5×5, 3×3, and 3×3, and channel numbers of 64, 128, 256, and 256 respectively. This branch is used to verify whether candidate regions contain valid ship name information.

[0042] The steps for establishing the training dataset during the pre-training process of the MarineIDNet model specifically include collecting high-resolution images of different types of fishing vessels under various lighting conditions, weather environments, and observation distances; using a semi-automatic annotation tool to label the vessel name regions with rectangular boxes and transcribe the text content; performing stratified sampling and equalization processing based on vessel size, observation distance, and weather conditions; and finally constructing a pre-training dataset containing 50,000 images, of which 80% is used for training, 10% for validation, and 10% for testing. At the same time, data augmentation techniques are used to expand data diversity, and key sampling is performed on difficult examples to be identified in complex environments to improve the robustness of the model under harsh conditions.

[0043] The pre-training steps of the MarineIDNet model specifically include: first, basic visual feature learning on the ImageNet dataset; then, transfer learning on a general marine vessel dataset to obtain hull structure-related feature representation capabilities; and finally, fine-tuning on a fishing vessel name recognition dataset. The AdamW optimizer is used for training, with an initial learning rate of 0.0001, weight decay of 0.05, and a cosine annealing learning rate scheduling strategy. The batch size is 32, and the training epochs are 200. An early stopping mechanism is introduced during training to avoid overfitting. In the key parameter adaptive adjustment strategy, the number of attention heads is automatically adjusted according to the size of the vessel name region in the input image. When the region size is less than 1 / 20 of the input image, the number of heads is increased to 16 to improve detail perception. Simultaneously, the attention weight decay coefficient is dynamically adjusted according to image contrast; a smaller decay coefficient is applied to low-contrast images to enhance weak feature extraction capabilities.

[0044] The specific implementation methods of the above steps are described in detail below.

[0045] The specific implementation of step S01 is based on a three-level pyramid feature extraction structure constructed using ViT. This structure employs a multi-scale feature representation strategy to capture hierarchical information in fishing boat images through feature maps at different resolutions. First, the input image is resized to 224×224 pixels and divided into 16×16 image blocks, generating 196 basic image blocks. Then, each image block undergoes a linear projection transformation to a 768-dimensional feature space, and positional encoding is added to preserve spatial location information. Next, these feature representations are processed by three Transformer encoders of different depths: a shallow encoder with four attention layers to capture local texture features, a mid-level encoder with eight attention layers to learn moderately complex structural features, and a deep encoder with twelve attention layers to understand global semantic information. Finally, the feature pyramid network structure fuses the three different levels of feature representations to generate a multi-scale feature map. The purpose of this step is to construct a comprehensive representation of fishing boat images, providing rich visual feature information for subsequent region of interest localization, particularly improving adaptability to different large and small boat names through multi-scale representation.

[0046] The specific implementation of step S02 involves using a self-attention mechanism to determine the first-level region of interest (ROI). This step first calculates an attention weight matrix for each token in the input image and normalizes the weight distribution using the Softmax function. Then, it analyzes the attention heatmap, identifying regions with weight values ​​higher than a threshold of 0.65 as candidate ROIs. Next, it applies a connected component analysis algorithm to merge adjacent high-attention regions into a single region. Subsequently, it calculates the visual area percentage of each region, retaining regions whose area percentage exceeds 15% of the total image area. Finally, it applies a geometric shape analysis algorithm to calculate the perimeter-to-area ratio, contour complexity, and other features of the regions, filtering out regions with ship-like characteristics, i.e., regions with a perimeter-to-area ratio between 0.08 and 0.25 and a contour complexity index below 1.8. The purpose of this step is to quickly locate the main regions containing the ship's hull in the global image, reducing the search space for subsequent processing, improving computational efficiency, and simultaneously eliminating background interference to ensure that the focus is concentrated on the ship's structure.

[0047] The specific implementation of step S03 involves determining a second-level region of interest (ROI) based on the first-level ROI. This step first applies enhanced positional coding to the first-level ROI, combining sine and cosine positional coding with learnable positional coding to strengthen spatial location perception. Then, a multi-head attention mechanism is used to perform fine-grained analysis of this region, setting up eight attention heads, each with a 64-dimensional attention matrix, to capture features from different semantic subspaces. Next, attention response maps are calculated and threshold segmentation is applied, extracting regions with attention response values ​​greater than 0.78 as candidate regions. Subsequently, prior knowledge constraints are introduced, limiting the search range to the forward region of the hull and the outer wall of the ship based on statistical information from the ship type database. Finally, morphological operations are applied to optimize the region boundaries, using a combination of opening and closing operations to eliminate noise and smooth edges, with the structuring element size set to 3×3 pixels. The purpose of this step is to further narrow the recognition range and accurately locate key areas on the hull that may contain the ship's name, such as the outer wall of the ship and the surface of the bridge, laying the foundation for subsequent fine-grained recognition.

[0048] The specific implementation of step S04 involves using a fine-grained feature extraction network to determine the third-level region of interest. This step first applies a high-resolution feature extraction module to the second-level region of interest, dividing the region into smaller 4×4 pixel blocks for detailed analysis. Then, a fine-grained attention network is used for feature enhancement. This network contains four attention heads, each focusing on character-level visual features. Next, a text region detection algorithm is applied to identify potential text regions based on stroke density maps and text orientation fields, with a text region probability threshold set to 0.82. Subsequently, token segmentation technology is used to accurately locate the boundaries of the text regions, and an adaptive thresholding method is used to separate foreground text from the background. The threshold calculation is based on dynamic adjustment of local region contrast, with a contrast weight coefficient of 0.7. Finally, a boundary optimization algorithm is applied to finely adjust the detected text regions, including minimum bounding rectangle calculation, boundary expansion reservation, and boundary regularization processing. The boundary expansion ratio is set to 10% of the text region width. The purpose of this step is to accurately locate the specific region containing the ship's name, achieving pixel-level precision in extracting the ship's name region, ensuring that subsequent processing can focus on the minimum necessary region containing the ship's name.

[0049] The specific implementation of step S05 involves constructing a distortion matrix for the third-level region of interest. This step first extracts key parameters such as observation distance from image metadata, atmospheric visibility from a meteorological database, optical parameters from camera parameter files, the actual size of the ship name area from image analysis, and imaging time from metadata. Then, a mathematical model is established based on the optical imaging degradation equation, considering the combined effects of atmospheric scattering, optical system diffraction limits, and focus shift on image quality. Next, the point spread function (PSF) is calculated at different observation distances to quantify the degree of image blur. When the observation distance exceeds 50 meters, the PSF radius increase should not be less than 0.5 pixels / 10 meters. Subsequently, a signal-to-noise ratio (SNR) attenuation matrix is ​​generated, analyzing the ratio of signal intensity to noise level in each sub-region. Regions below 6 dB are marked as high-distortion-risk areas. Finally, normalization is performed to generate a standardized distortion matrix, with values ​​ranging from 0 to 1, where 0 represents no distortion and 1 represents complete distortion. The purpose of this step is to quantitatively evaluate the impact of observation distance on the image quality of the ship name area, providing a scientific basis for subsequent image enhancement processing and ensuring targeted improvement of low-quality areas.

[0050] The specific implementation of step S06 involves calculating the third-level region of interest deviation matrix. This step first applies a ship attitude light reflection model to analyze the interaction between light and the ship's surface. It extracts the ship's surface normal vector from the image, obtains the incident light angle using a lighting estimation algorithm, acquires the ship's material reflection coefficient using a material recognition network, obtains the surface roughness through texture analysis, and assesses the coating aging degree based on image degradation. Then, it establishes a model of the relationship between ship attitude and perspective distortion, calculating the tilt angle of the ship's surface relative to the camera. When the tilt angle exceeds 60 degrees, the text distortion coefficient should be greater than 1.5. Next, it analyzes the degree of perspective distortion in the text area under different attitude angles, establishing a quantitative index for the degree of distortion. For every 10-degree increase in the perspective distortion angle, the text recognition difficulty coefficient increases by 0.2. Subsequently, it generates an effective contrast matrix for the ship's name area, evaluating the change in text and background contrast under different lighting conditions. Areas with an effective contrast below 0.3 are marked as high-risk areas. Finally, it calculates a text edge sharpness evaluation matrix, analyzing the clarity of the text outline. Areas with edge sharpness below 0.4 require special enhancement processing. The purpose of this step is to quantitatively assess the degree of geometric distortion in the ship's name area caused by changes in the ship's attitude, providing accurate parameters for subsequent perspective correction and image enhancement.

[0051] The specific implementation of step S07 is to generate a recognizable matrix based on the comprehensive calculation of the distortion matrix and the deviation matrix. This step first applies a multi-feature fusion function to integrate various influencing factors, including the distortion matrix generated in the previous steps, the calculated deviation matrix, the ambient brightness matrix obtained from image analysis, the ship texture complexity obtained from texture analysis, and the ship name font feature vector obtained from the font recognition network. Then, a comprehensive score is calculated using a weighted summation method, with the weights of each factor being 0.35 for distortion, 0.3 for deviation, 0.15 for brightness, 0.1 for texture complexity, and 0.1 for font features. Next, a non-linear mapping function is applied to convert the original score into a standardized recognition difficulty coefficient, using the Sigmoid function for mapping, with an output range of 0 to 1, where 0 represents easy recognition and 1 represents extremely difficult recognition. Subsequently, a recognizable heatmap is generated based on the recognition difficulty coefficient, visually displaying the distribution of recognition difficulty in different parts of the ship name area, with areas having a recognition difficulty coefficient higher than 0.7 marked as red high-risk areas. Finally, optimization strategy suggestions are determined based on the recognizable score, including recommended image processing methods and parameter settings. When the overall recognizable score is lower than 0.5, adaptive enhancement processing is recommended. The purpose of this step is to comprehensively assess the difficulty of identifying the ship name area under the current conditions, and to provide a basis for decision-making for subsequent targeted enhancement processing.

[0052] The specific implementation of step S08 involves designing an adaptive region enhancement algorithm. This step first identifies the regions requiring enhancement based on the recognizability matrix, prioritizing regions with a recognizability coefficient greater than 0.6. Then, it designs differentiated enhancement strategies for different problems. For distortion-dominated problems, a deconvolution recovery algorithm is applied with a 5×5 kernel and a regularization parameter of 0.02. For deviation-dominated problems, a perspective correction algorithm is applied, performing perspective transformation based on a four-point correspondence, with control points selected at the four corners of the text. For insufficient contrast problems, local contrast enhancement is applied, with the enhancement coefficient adaptively adjusted based on local brightness, and the maximum gain not exceeding 200% of the original contrast. Subsequently, edge enhancement processing is applied to improve the clarity of text edges using a nonlinear gradient enhancement method, where the enhancement intensity is inversely proportional to the edge sharpness matrix. Finally, noise suppression filtering is applied, based on a guided filtering algorithm to protect text edges while suppressing noise, with a filtering radius of 3 pixels and a regularization coefficient of 0.1. The purpose of this step is to perform targeted enhancement on low-recognition regions based on the aforementioned analysis results, improving the overall recognizability of the ship name region and providing high-quality input for subsequent text recognition systems.

[0053] The specific implementation of step S09 involves outputting the optimized ship name region image and coordinate information. This step first integrates the aforementioned processing results to generate a high-quality ship name region image; then, it calculates the precise coordinate information of the ship name region, including the rectangular bounding box coordinates, rotation angle, and confidence score; next, it transmits the processed ship name region image and coordinate information to the verification system; subsequently, it applies the MarineIDNet neural network for ship name region verification. This network employs a dual-branch structure, including feature extraction and region verification branches, and verifies the validity of the region through multi-layer feature extraction and comparative analysis; finally, it performs fine-tuning of the boundaries of the verified regions, using a text edge precision localization algorithm to fine-tune the boundary positions, with the adjustment accuracy required to be controlled within 2 pixels of the original boundary. The purpose of this step is to provide the final optimized ship name region image and precise location information, providing standardized input for the subsequent text recognition system, while ensuring the accuracy and reliability of the extraction results through the verification process.

[0054] Furthermore, the detailed structure of the MarineIDNet model is a dual-branch network architecture, including a feature extraction branch and a region validation branch. The feature extraction branch adopts an improved ViT structure, which segments the input image into a sequence of 16×16 image patches. Each image patch is transformed into a 768-dimensional feature vector through a linear projection layer, while learningable positional encoding is added. This branch contains 12 Transformer encoder layers, each consisting of a multi-head self-attention module and a feedforward neural network. The multi-head self-attention module contains 12 attention heads, each with a query, key, and value matrix of dimension 64. The feedforward neural network contains two fully connected layers and a hidden layer dimension of 3072. This branch also introduces a local receptive field constraint mechanism, which improves the sensitivity to small-sized text by limiting the attention calculation range to a local region (radius of 3 tokens). Finally, the features are converted into feature vectors by a classification head for downstream tasks. The region validation branch consists of a four-layer convolutional neural network and two fully connected layers. The first convolutional layer has a kernel size of 7×7 and 64 output channels, the second convolutional layer has a kernel size of 5×5 and 128 output channels, and the third and fourth convolutional layers both have a kernel size of 3×3 and 256 output channels. Each convolutional layer is followed by batch normalization and ReLU activation function. The second and fourth layers are followed by max pooling layers with a pooling kernel size of 2×2. The output dimensions of the fully connected layers are 512 and 2, respectively. Finally, the probability that the region contains a valid ship name is output through the Softmax function.

[0055] Furthermore, the steps for establishing the MarineIDNet model pre-training dataset specifically include the following processes: First, high-resolution images of different types of fishing vessels under various environmental conditions are collected, including various lighting conditions (sunny, cloudy, dusk, night, etc.), weather conditions (sunny, cloudy, rainy, foggy, etc.), and samples at different observation distances (close distance within 5 meters, medium distance 5-30 meters, and long distance more than 30 meters); then, a semi-automatic annotation tool is used to annotate the vessel name area in each image with rectangular boxes, while recording the text content as ground truth labels, with annotation accuracy required to be controlled at the pixel level; next, based on vessel size (small vessels less than 12 meters in length, medium vessels 12-24 meters in length, and large vessels greater than 24 meters in length), observation distance, and weather conditions... Stratified sampling was performed to ensure a balanced sample size under various conditions. A pre-training dataset of 50,000 images was then constructed, with 80% (40,000) used for training, 10% (5,000) for validation, and 10% (5,000) for testing. Finally, data augmentation techniques were used to enhance data diversity, including random rotation (±15 degrees), scaling (0.8–1.2 times), brightness adjustment (±20%), contrast variation (±15%), adding noise (Gaussian noise standard deviation 0.01–0.05), and simulating adverse weather conditions (rain, fog, uneven lighting, etc.). Priority sampling and augmentation were performed on difficult-to-identify examples in complex environments to improve sample coverage in edge cases, thereby enhancing the model's robustness under adverse conditions.

[0056] Furthermore, the model's pre-training process specifically includes: firstly, basic visual feature learning on the ImageNet dataset; then, transfer learning on a general ocean vessel dataset to obtain hull structure-related feature representation capabilities; and finally, fine-tuning on a fishing vessel name recognition dataset. The AdamW optimizer is used for training, with an initial learning rate of 0.0001, weight decay of 0.05, a cosine annealing learning rate scheduling strategy, a batch size of 32, and 200 training epochs. An early stopping mechanism is introduced during training to avoid overfitting. In the key parameter adaptive adjustment strategy, the number of attention heads is automatically adjusted according to the size of the vessel name region in the input image. When the region size is less than 1 / 20 of the input image, the number of heads is increased to 16 to improve detail perception. At the same time, the attention weight decay coefficient is dynamically adjusted according to the image contrast. A smaller decay coefficient is applied to low-contrast images to enhance the ability to extract weak features.

[0057] A second aspect of the present invention provides a computer-readable storage medium storing program instructions, which, when executed in a computer, are used to perform the above-described method for extracting the vessel name region of fishing vessels entering and leaving ports based on ViT.

[0058] A third aspect of the present invention provides a ViT-based system for extracting the vessel name and region of fishing vessels entering and leaving ports, comprising the aforementioned computer-readable storage medium. The system can be any one of a computer, a server, or a microcontroller. The computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor that executes the program instructions stored in the computer-readable storage medium.

[0059] The mathematical model or calculation process involved in this invention will be described in detail below.

[0060] The three-level pyramid feature extraction structure in step S01 involves multiple computational processes, as detailed below:

[0061] The process of segmenting an input image into image patches can be represented as:

[0062] P = {p i , j |i = 1, 2, ..., N h j = 1, 2, ..., N w};

[0063] In the formula, P is the set of image patches; p i，j Represents the image patch at position (i, j); N h and N w N represents the number of blocks in the height and width directions of the image, respectively. For a 224×224 image, it is divided into 16×16 image blocks. h =N w =14.

[0064] The linear projection transformation process of an image patch is represented as follows:

[0065] z0 = [x class ;p1E;p2E;...;p N E]+E pos ;

[0066] In the formula, z0 represents the initial feature representation of the image patch sequence; x class It is a special marker used for classification; p n Represents the nth image patch; E is a linear projection matrix that maps the image patch to a D-dimensional feature space, where D = 768; E pos It is a positional encoding matrix with the same dimensions as z0. E is obtained through pre-training, and E pos Calculated using the following formula:

[0067] E pos (pos, 2i) = sin(pos / 10000) 2i / D );

[0068] E pos(pos, 2i+1) = cos(pos / 10000) 2i / D );

[0069] In the formula, pos represents the position in the sequence; i represents the feature dimension index, which is 0 ≤ i <D / 2。

[0070] The multi-head self-attention calculation process is represented as follows:

[0071]

[0072] In the formula, Q, K, and V represent the query, key, and value matrices, respectively; d k The dimension of the key vector is used to scale the dot product result and prevent gradient vanishing. k =D / h=64, where h=12 is the number of attention heads.

[0073] The output feature fusion process of encoders at different levels can be represented as:

[0074] F multi =α1F shallow +α2F middle +α3F deep ;

[0075] In the formula, F multi F represents the multi-scale features after fusion. shallow F middle F deep α1, α2, and α3 represent the output features of the shallow, medium, and deep encoders, respectively; α1, α2, and α3 are fusion weight coefficients, which are obtained through backpropagation optimization, and are generally initially set to α1 = 0.2, α2 = 0.3, and α3 = 0.5.

[0076] The self-attention mechanism in step S02, which determines the first-level region of interest, involves the following computational process:

[0077] The calculation of the attention heatmap is represented as follows:

[0078]

[0079] In the formula, A map A represents an average attention heatmap; i Let represent the attention matrix of the i-th attention head; h represents the number of attention heads, h = 12.

[0080] The process of recognizing high-attention regions is represented as follows:

[0081] M roi1 ={(x, y)|A map (x, y)>τ1};

[0082] In the formula, Mroi1 τ1 represents the first-level region of interest mask; (x, y) represents the pixel coordinates in the image; τ1 represents the attention threshold, τ1 = 0.65.

[0083] The calculation of the area percentage is expressed as follows:

[0084]

[0085] In the formula, R area Indicates the area percentage; Area(M) roi1 Area(I) represents the area of ​​the region of interest; total () represents the total area of ​​the image. R is retained. area Areas >0.15.

[0086] The complexity of the contour is expressed as:

[0087]

[0088] In the formula, C complexity Let P represent the contour complexity index; let A represent the perimeter of the region; and let A represent the area of ​​the region. Reserve 0.08. And C complexity Regions with an index of <1.8.

[0089] Determining the second-level region of interest in step S03 involves the following calculation process:

[0090] Enhanced positional coding calculation is represented as:

[0091]

[0092] In the formula, Indicates enhanced positional coding; Indicates the position code for sine and cosine; This represents the learnable location code; λ is the fusion coefficient, λ = 0.6.

[0093] The multi-head attention response calculation is represented as follows:

[0094]

[0095] In the formula, A response Represents an attention response map; w i The weight of the i-th attention head is represented by its symbol; Attention i This represents the output of the i-th attention head. i Obtained through training, with initial values ​​set to equal weights.

[0096] The second-level area of ​​interest is defined as follows:

[0097] Mroi2 ={(x, y)|A response (x, y)>τ2}∩R prior ;

[0098] In the formula, M roi2 τ represents the second-level region of interest mask; τ2 represents the attention response threshold, τ2 = 0.78; R prior This represents the constraint region based on prior knowledge, determined according to the ship type database.

[0099] Determining the third-level region of interest in step S04 involves the following calculation process:

[0100] Fine-grained feature extraction is represented as:

[0101]

[0102] In the formula, F fine represents fine-grained features; Conv represents convolution operation; Resize represents resampling operation, which shrinks the second-level region of interest to one-quarter of its original size to obtain a more refined feature representation.

[0103] Text region detection calculation is represented as:

[0104] P text =σ(W2·ReLU(W1·F) fine +b1)+b2);

[0105] In the formula, R text This represents the probability map of the text region; W1, W2, b1, and b2 are model parameters; σ represents the Sigmoid activation function; and ReLU represents the modified linear unit activation function.

[0106] The adaptive threshold calculation is expressed as:

[0107]

[0108] In the formula, T adaptive σ(x,y) represents the adaptive threshold; μ(x,y) represents the mean of the local region; σ(x,y) represents the standard deviation of the local region; γ represents the contrast weighting coefficient, γ=0.7.

[0109] The third-level area of ​​interest is defined as follows:

[0110] M roi3 ={(x, y)|P text (x, y) > τ3};

[0111] In the formula, M roi3 τ3 represents the third-level region of interest mask; τ3 represents the text region probability threshold, τ3 = 0.82.

[0112] Boundary extension processing is represented as:

[0113] B expanded =Expand(B original r expand );

[0114] In the formula, B expanded Indicates the expanded boundary; B original Indicates the original boundary; r expand Indicates the expansion ratio, r expand =0.1 means the expanded width is 10% of the original text area width.

[0115] The construction of the distortion matrix in step S05 involves the following calculation process:

[0116] The optical imaging degradation equation is expressed as:

[0117] g(x, y)=h(x, y)*f(x, y)+n(x, y);

[0118] In the formula, g(x,y) represents the observed image; h(x,y) represents the point spread function; f(x,y) represents the ideal distortion-free image; n(x,y) represents additive noise; and * represents the convolution operation.

[0119] The point spread function is calculated as follows:

[0120] h(x, y) = h system (x, y)*h atmospheric (x, y)*h defocus (x, y);

[0121] In the formula, h system h represents the point spread function of an optical system. atmospheric h represents the atmospheric scattering point spread function; defocus This represents the out-of-focus spread function.

[0122] The point spread function of the optical system is expressed as:

[0123]

[0124] In the formula, J1 represents the first-order Bessel function; r represents the radial distance; and r0 represents the Airy disk radius. Where λ represents the wavelength of light, f represents the focal length, and D represents the aperture diameter.

[0125] The atmospheric scattering point spread function is expressed as:

[0126]

[0127] In the formula, r represents the radial distance; r0 represents the atmospheric coherence length. Where L represents the observation distance, C n This represents the atmospheric refractive index structure constant, which is related to meteorological conditions.

[0128] The divergence function is expressed as:

[0129] $h_{defocus}(r) =

[0130] $;

[0131] In the formula, R defocus Indicates the defocus radius. Where s represents the actual object distance.

[0132] The increase in the radius of the diffusion function at the distance to the relevant point is expressed as follows:

[0133]

[0134] In the formula, ΔR PSF α represents the increase in the point spread function radius; d represents the observation distance (meters); d0 represents the reference distance, d0 = 50 meters; PSF α represents the radius increase coefficient. PSF = 0.5 pixels / 10 meters.

[0135] The signal-to-noise ratio attenuation matrix is ​​calculated as follows:

[0136]

[0137] In the formula, SNR(x,y) represents the signal-to-noise ratio (dB) at position (x,y); S(x,y) represents the signal strength.

[0138] N(x, y) represents the noise intensity. Areas with a signal-to-noise ratio below 6 dB are marked as high-distortion risk zones.

[0139] The normalized distortion matrix is ​​calculated as follows:

[0140]

[0141] In the formula, D norm Represents the normalized distortion matrix; SNR threshold Represents the signal-to-noise ratio threshold, SNR threshold = 20 decibels. D norm The value ranges from 0 to 1, where 0 represents no distortion and 1 represents complete distortion.

[0142] The calculation of the deviation matrix in step S06 involves the following calculation process:

[0143] The calculation of the hull surface normal vector is expressed as follows:

[0144]

[0145] In the formula, Represents the surface normal vector; Represents the image gradient; This represents the gradient magnitude.

[0146] The incident light angle estimate is expressed as:

[0147]

[0148] In the formula, represents the normalized incident light direction vector; S represents the set of pixels in the highlight region. This represents the half-vector of the i-th pixel; This represents the surface normal vector of the i-th pixel.

[0149] The relationship between hull attitude and perspective distortion can be represented by the following model:

[0150] D perspective (θ)=1+β·max(0, θ-θ0);

[0151] In the formula, D perspective θ represents the text distortion coefficient; θ represents the angle (in degrees) between the hull surface and the camera plane; θ0 represents the critical angle, θ0 = 30 degrees; β represents the distortion coefficient, β = 0.02 / degree, meaning that for every 1 degree increase in angle, the distortion coefficient increases by 0.02. When θ exceeds 60 degrees, D... perspective The result is greater than 1 + 0.02·(60-30) = 1.6, which meets the requirements.

[0152] The impact of perspective distortion angle on recognition difficulty is expressed as follows:

[0153]

[0154] In the formula, D difficulty The symbol represents the recognition difficulty coefficient; θ represents the perspective distortion angle (degrees). For every 10-degree increase in the perspective distortion angle, the recognition difficulty coefficient increases.

[0155] Effective contrast ratio is calculated as follows:

[0156]

[0157] In the formula, C effective Indicates effective contrast; I text Indicates the brightness of the text area; I background Indicates the brightness of the background area; I max and I min These represent the maximum and minimum brightness of the image, respectively; γ represents the perspective effect coefficient, γ = 0.3; D perspective(θ) represents the text distortion coefficient. Areas with an effective contrast ratio below 0.3 are marked as high-risk areas.

[0158] The text edge sharpness calculation is expressed as:

[0159]

[0160] In the formula, S edge Indicates edge sharpness; This represents the gradient magnitude at position (x, y). δ represents the maximum gradient magnitude; δ represents the perspective effect coefficient, δ = 0.4; D perspective (θ) represents the text distortion coefficient. Areas with an edge sharpness below 0.4 require special enhancement processing.

[0161] The deviation matrix is ​​calculated as follows:

[0162] B(x, y) = 1 - min(C effective (x, y), S edge (x, y));

[0163] In the formula, B(x, y) represents the deviation matrix; C effective Indicates effective contrast; S edge This represents the edge sharpness. The value of B(x, y) ranges from 0 to 1, where 0 represents no deviation and 1 represents the maximum deviation.

[0164] Step S07, which involves the comprehensive calculation to generate the identifiable matrix, includes the following calculation process:

[0165] The multi-feature fusion function is expressed as:

[0166] R(x, y) = w1·D norm (x, y)+w2·B(x,y)+w3·L(x,y)+w4·T(x,y)+w5·F(x,y);

[0167] In the formula, R(x, y) represents the original comprehensive score; D norm denoted as the normalized distortion matrix; B represents the bias matrix; L represents the ambient brightness matrix; T represents the hull texture complexity; F represents the feature vector of the ship name font; w1, w2, w3, w4, and w5 represent the weights of each factor, with w1 = 0.35, w2 = 0.3, w3 = 0.15, w4 = 0.1, and w5 = 0.1 respectively.

[0168] The ambient brightness matrix is ​​calculated as follows:

[0169]

[0170] In the formula, L(x, y) represents the ambient brightness deviation; I(x, y) represents the brightness value at position (x, y); I max This represents the maximum brightness value. The closer the value of L(x,y) is to 0, the more moderate the brightness is; the closer it is to 0.5, the more excessive or insufficient the brightness is.

[0171] Texture complexity calculation is expressed as:

[0172]

[0173] In the formula, T(x, y) represents the texture complexity; N represents the number of pixels in the local region; Indicates position (x) i y i The gradient magnitude at ().

[0174] The standardized recognition difficulty coefficient is calculated as follows:

[0175]

[0176] In the formula, R norm R represents the standardized recognition difficulty coefficient; R represents the original comprehensive score; k represents the steepness parameter of the Sigmoid function, k=5; R0 represents the center point parameter, R0=0.5. norm The value ranges from 0 to 1, where 0 indicates easy identification and 1 indicates extremely difficult identification.

[0177] The recognizable matrix generation representation is as follows:

[0178] $RM(x, y) =

[0179] $;

[0180] In the formula, RM represents the recognizable matrix; R norm τ represents the standardized identification difficulty coefficient; τ4 represents the identification difficulty threshold, τ4 = 0.7. Areas with an identification difficulty coefficient higher than 0.7 are marked as high-risk areas.

[0181] The design of the adaptive region enhancement algorithm in step S08 involves the following computational process:

[0182] The deconvolution recovery algorithm is expressed as:

[0183]

[0184] In the formula, denoted as the restored image; h represents the point spread function; g represents the observed image; D represents the difference operator; θ represents the regularization parameter, θ = 0.02.

[0185] The perspective correction algorithm is expressed as:

[0186]

[0187] In the formula, H represents the perspective transformation matrix; A1 represents the coordinate matrix of the four corner points of the source quadrilateral; and A2 represents the coordinate matrix of the four corner points of the target rectangle. The transformed image is calculated using the following formula:

[0188]

[0189] In the formula, I corrected Indicates the corrected image; I original Represents the original image; h ij Represents the elements of the perspective transformation matrix H.

[0190] The local contrast enhancement algorithm is represented as:

[0191] I enhanced (x,y)=I(x,y)+α(x,y)·(I(x,y)-I blur (x, y));

[0192] In the formula, I enhanced I represents the enhanced image; I represents the original image; I blur This represents the image after Gaussian blurring; α represents the enhancement coefficient. Ensure that the maximum gain does not exceed 200% of the original contrast.

[0193] Edge enhancement processing is represented as:

[0194] I edge (x,y)=I(x,y)+β(x,y)·(I(x,y)-I blur (x, y)) 2 ·sign(I(x,y)-I blur (x, y));

[0195] In the formula, I edge I represents the image after edge enhancement; I represents the original image; I blur This represents the image after Gaussian blurring; β represents the nonlinear enhancement coefficient. Where β0 represents the basic enhancement coefficient, β0 = 10, and ∈ represents a small constant to prevent division by zero, ∈ = 0.01.

[0196] The guided filtering algorithm is expressed as:

[0197] I filtered (x, y)=a(x, y)·G(x, y)+b(x, y);

[0198] In the formula, I filteredLet G represent the filtered image; G represents the guiding image, typically the original image; a and b are the coefficients of the local linear model, obtained by minimizing the cost function.

[0199]

[0200] In the formula, ω (x，y) This represents a window centered at (x, y); ∈ represents the regularization coefficient, ∈ = 0.1. The filter radius is set to 3 pixels.

[0201] The principles and significance of the equations involved in the above formula are explained as follows: The optical imaging degradation equation simulates the physical phenomena in the real imaging process, including the effects of factors such as the diffraction limit of the optical system, atmospheric scattering, and defocusing on image quality. The point spread function converts the ideal image into the actual observed image, and additive noise simulates sensor noise and quantization error. This equation takes into account the principles of physical optics and can accurately describe the variation of image quality at different observation distances.

[0202] The point spread function (PSF) calculation considers three key factors: the diffraction limit of the optical system, atmospheric scattering, and defocusing. The optical system PSF uses the Airy disk model, based on wave optics theory; the atmospheric scattering PSF considers the influence of atmospheric turbulence on light wave propagation, based on Kolmogorov's turbulence theory; and the defocusing PSF is based on geometric optics principles. This comprehensive consideration of these three factors allows the PSF to fully reflect changes in image quality under different observation conditions.

[0203] A model relating ship attitude and perspective distortion establishes a quantitative relationship between the tilt angle of the ship's surface and the degree of text distortion. A linear growth function is used to describe the impact of angle increases on distortion, and a critical angle is set to account for the relatively small impact of small angle changes on recognition in practical applications. This model simplifies complex perspective transformation calculations and provides a method for rapidly assessing the impact of perspective distortion.

[0204] Effective contrast calculation comprehensively considers the brightness difference between text and background, as well as the effect of perspective distortion. Normalization is used to ensure contrast values ​​are between 0 and 1, facilitating subsequent processing. A perspective influence coefficient is introduced to account for the fact that perspective distortion reduces text edge sharpness, thus lowering effective contrast; this design aligns with human visual perception characteristics.

[0205] The multi-feature fusion function comprehensively considers various factors affecting the difficulty of ship name recognition through weighted summation. The weight allocation reflects the different degrees of influence of each factor on the recognition difficulty, with the distortion matrix and bias matrix having higher weights, indicating that image quality and geometric deformation are the main factors affecting recognition. This multi-feature fusion method makes the assessment of recognition difficulty more comprehensive and accurate.

[0206] The standardized recognition difficulty coefficient is non-linearly mapped using the Sigmoid function, converting the original comprehensive score into a recognition difficulty coefficient ranging from 0 to 1. The Sigmoid function is chosen based on its smoothness and sensitivity to intermediate values, making it suitable for representing continuously changing difficulty levels. The parameter k controls the steepness of the function, and R0 controls the position of the center point; these two parameters can be used to adjust the sensitivity of the mapping.

[0207] Adaptive enhancement algorithms select different processing strategies based on image characteristics. Deconvolution restoration algorithms, based on regularization theory, restore degraded images by minimizing a cost function; perspective correction algorithms, based on projective geometry principles, achieve geometric correction of the image through a perspective transformation matrix; local contrast enhancement and edge enhancement are based on nonlinear image processing techniques, adjusting the enhancement intensity through adaptive parameters; and guided filtering, based on edge-preserving filtering theory, can retain edge details while suppressing noise. The combined use of these algorithms can provide effective solutions for different types of image degradation problems.

[0208] The hull attitude light reflection model is used to analyze the impact of the light reflection characteristics of the hull surface on ship name recognition. Its calculation process is as follows:

[0209] Optionally, the hull surface reflection model can be represented as follows:

[0210]

[0211] In the formula, I reflect Indicates the intensity of reflected light; k d This represents the diffuse reflectance coefficient, obtained from the material recognition network. This represents the surface normal vector, calculated from image analysis. The direction of the incident light is represented by k, obtained from the illumination estimation algorithm. s The specular reflection coefficient is obtained from the material recognition network. Indicates the direction of the reflected light; The direction of the line of sight is indicated; n represents the roughness index, which is obtained from texture analysis and generally ranges from 1 to 200. The larger the value, the smoother the surface.

[0212] Optionally, surface roughness calculation is expressed as:

[0213]

[0214] In the formula, R roughness σ represents surface roughness. gradient σ represents the local gradient standard deviation; gradient，max This represents the global maximum gradient standard deviation. The roughness value ranges from 0 to 1, with a larger value indicating a smoother surface.

[0215] Optionally, the coating aging degree can be calculated as follows:

[0216]

[0217] In the formula, A coating Indicates the degree of coating aging; CV local Represents the local color variation coefficient; CV reference This represents the reference color variation coefficient. A higher aging value indicates more severe coating aging.

[0218] Optionally, the effective contrast matrix calculation for the ship name area is represented as follows:

[0219]

[0220] In the formula, C effective Represents the effective contrast matrix; I text Indicates the brightness of the text area; I background Indicates the brightness of the background area; A coating This indicates the degree of coating aging; α represents the influence coefficient of the incident angle, α = 0.6.

[0221] Optionally, the text edge sharpness evaluation matrix is ​​calculated as follows:

[0222]

[0223]

[0224] In the formula, S edge This represents the edge sharpness evaluation matrix; R represents the gradient magnitude; roughness Indicates surface roughness; A coating β represents the degree of coating aging; β represents the roughness influence coefficient, β = 0.4; γ represents the aging influence coefficient, γ = 0.5.

[0225] Optionally, the output of the optimized ship name area image and coordinate information in step S09 involves the following calculation process:

[0226] Optionally, the coordinate information of the ship name area is calculated and represented as follows:

[0227] B final ={x min y min x max y max ,θ,cnof};

[0228] In the formula, B final Indicates the final bounding box information; x min y min x max ymax θ represents the coordinates of the top left and bottom right corners of the bounding box, respectively; θ represents the rotation angle; and conf represents the confidence score.

[0229] Optionally, the confidence score is calculated as follows:

[0230]

[0231] In the formula, conf represents the confidence score; |M roi3 | Indicates the number of pixels in the third-level region of interest; P text represents the probability of a text region; RM represents the recognizable matrix.

[0232] Optionally, the boundary fine-tuning algorithm is represented as:

[0233] B adjusted =B final +ΔB;

[0234] In the formula, B adjusted Indicates the adjusted boundary; B final The initial boundary is represented by ΔB; ΔB represents the boundary adjustment amount, ΔB = f adjustment (G text ), where f adjustment G represents the boundary adjustment function. text This represents the gradient map of the text edges. The adjustment precision is controlled within 2 pixels of the original boundary.

[0235] It should be noted that token segmentation technology is a key method for precise boundary localization in ViT-based extraction of fishing vessel name regions. Based on the characteristics of the Vision Transformer, it decomposes the image into a token sequence and then performs fine-tuning to achieve accurate segmentation of the target region.

[0236] The core of token segmentation lies in using the basic unit (Token) processed by the Vision Transformer for image segmentation tasks. In the traditional ViT model, the input image is segmented into fixed-size (typically 16×16 pixels) image blocks, and each image block is converted into a Token vector. These Token vectors are then processed by the Transformer encoder to form feature representations. Token segmentation innovates on this by treating each Token as a classifiable semantic unit, and determining whether they belong to the same target region by analyzing the correlation and semantic similarity between Tokens.

[0237] Specifically, the implementation steps of the token segmentation technology in determining the third-level interest area are as follows:

[0238] High-resolution token mapping: Divide the image of the second-level region of interest into smaller image blocks (e.g., 8×8 or 4×4 pixels) to generate a high-density token sequence and improve spatial resolution.

[0239] Token semantic feature extraction: Deep feature representation is extracted for each token through positional encoding enhancement and multi-head self-attention mechanism to capture local texture, edge and text features.

[0240] Token Clustering and Grouping: Based on the similarity calculation of token features, an adaptive threshold clustering algorithm is used to group semantically similar tokens to determine the set of tokens that may contain ship names.

[0241] Boundary precision processing: Construct a connected graph using the association strength between tokens, apply graph partitioning algorithms (such as normalized cut) to determine the precise boundary of the ship name area, and eliminate background interference at the same time.

[0242] Multi-scale Token Consistency Verification: By comparing the consistency of token segmentation results at different scales, the stability and accuracy of boundary localization are improved, and the impact of noise is reduced.

[0243] In this way, token segmentation technology can break through the limitations of rectangular bounding boxes in traditional object detection, and achieve accurate description of irregular shapes in ship name regions, making it particularly suitable for processing text regions on curved surfaces of ship hulls.

[0244] The reason why token segmentation technology can play an important role in precise boundary positioning is mainly reflected in the following aspects:

[0245] First, the token segmentation technique fully leverages the global contextual awareness of the Transformer model. In ship hull image processing, the ship name and number region often exhibits subtle visual differences from the surrounding background and may present irregular shapes due to the curvature of the hull. Traditional CNN-based bounding box detection methods struggle to accurately capture these features, while the token segmentation technique, through its self-attention mechanism, can establish connections between distant pixels, analyze global contextual information, and more accurately identify the true boundaries of the ship name and number region.

[0246] Secondly, token segmentation technology achieves pixel-level fine-grained classification. In the third-level region of interest determination process, each token actually represents a small region of the image. By classifying these tokens (belonging to the ship's name or background) and combining the token's positional information, a precise mask for the ship's name region can be constructed. This pixel-level classification is more accurate than traditional bounding boxes and can adapt to the deformation and perspective effects of the ship's name on the curved surface of the hull.

[0247] Furthermore, token segmentation technology exhibits excellent robustness. Through in-depth extraction and analysis of each token feature, it maintains high boundary localization accuracy even under poor lighting conditions, with soiled or partially occluded hulls. Particularly when processing low-contrast or blurry images, token segmentation technology can integrate contextual information, compensate for insufficient local features, and improve the reliability of boundary localization.

[0248] Finally, token segmentation technology can be closely integrated with subsequent recognition modules (such as MarineIDNet). By providing an accurate mask of the ship name region, background interference can be eliminated, improving the accuracy of text recognition. Simultaneously, the token segmentation results can also serve as important input for calculating the recognizable matrix, helping the system assess the recognition difficulty and guiding subsequent augmentation processing.

[0249] This token-based fine-grained segmentation method enables the system to accurately locate ship name regions in complex marine environments, providing a solid foundation for subsequent ship identification and verification.

[0250] Specifically, the principle of this invention is: the technical principle of this invention is based on the organic combination of ViT's global feature modeling capability and multi-level interest region fine extraction strategy. By introducing a quantitative evaluation mechanism, the accurate extraction of the fishing vessel name region in complex environments can be achieved.

[0251] First, the three-level pyramid feature extraction structure based on ViT leverages its powerful global context awareness to overcome the limitations of traditional CNNs' reliance on local receptive fields. By segmenting the input image into sequential image patches, ViT can capture the overall structural features of the hull and effectively identify key regions within the hull contour through a self-attention mechanism. The strategy of progressively refining the three-level regions of interest, from the overall hull to the precise location of the ship's name, avoids the difficulty of directly searching for small targets across the entire image, thus improving localization accuracy and computational efficiency.

[0252] Secondly, the introduction of the distortion matrix and the bias matrix is ​​a key technological innovation of this invention. The distortion matrix, established through the optical imaging degradation equation, can quantitatively describe the impact of observation distance on image quality, while the bias matrix, constructed based on the ship's attitude light reflection model, accurately characterizes the perspective distortion of text areas caused by changes in ship attitude. The recognizable matrix generated by combining the two provides a scientific basis for subsequent enhancement processing, achieving adaptive optimization of the ship name area under different observation conditions and solving the problem that traditional methods cannot cope with complex environmental changes.

[0253] The MarineIDNet neural network is designed with full consideration of the specific needs of ship name recognition. The feature extraction branch in its dual-branch architecture inherits the global modeling advantages of ViT while introducing local receptive field constraints to enhance sensitivity to small text regions. The region validation branch ensures that the extracted region contains valid ship name information, preventing false detections. During pre-training, stratified sampling and key sampling techniques are employed to specifically enhance the model's robustness under harsh conditions, addressing the problem of low recognition accuracy in complex environments found in existing technologies.

[0254] In summary, this invention solves the problem of accurately extracting the name region of fishing vessels in complex environments by combining ViT with a multi-level region of interest strategy, a quantitative evaluation mechanism, and adaptive enhancement processing.

[0255] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0256] The specific implementation of step S01 is based on constructing a three-level pyramid feature extraction structure using ViT. This structure employs a multi-scale feature representation strategy to capture hierarchical information in fishing boat images through feature maps at different resolutions. First, the input image is adjusted to a size of 224×224 pixels and divided into 16×16 image blocks. The set of image blocks is represented as P = {p i，j |i = 1, 2, ..., N h j = 1, 2, ..., N w}, where P is a set of image patches, p i，j N represents the image patch at position (i, j). h and N w N represents the number of blocks in the height and width directions of the image, respectively. For a 224×224 image, it is divided into 16×16 image blocks. h =N w =14. Then, a linear projection transformation is performed on each image patch to a 768-dimensional feature space, and positional encoding is added to preserve spatial location information. This process is represented as z0 = [x class ;p1E;p2E;...;p N E]+E pos Where z0 represents the initial feature representation of the image patch sequence, x class It is a special marker used for classification, p n Let E represent the nth image patch, and E be a linear projection matrix that maps the image patch to a D-dimensional feature space, where D = 768. pos It is a positional encoding matrix with the same dimensions as z0. The positional encoding is achieved through E. pos (pos, 2i) = sin(pos / 10000) 2i / D ) and Epos (pos, 2i + 1) = cos(pos / 10000 2i / D ) is calculated, where pos represents the position in the sequence, and i represents the feature dimension index, with the range 0 ≤ i < D / 2. Then, these feature representations are processed by three Transformer encoders with different depths. The shallow encoder contains 4 attention layers for capturing local texture features, the middle encoder contains 8 attention layers for learning medium-complexity structural features, and the deep encoder contains 12 attention layers for understanding global semantic information. The multi-head self-attention calculation process is expressed as where Q, K, and V represent the query, key, and value matrices respectively, and d k represents the dimension of the key vector, which is used to scale the dot product result to prevent gradient vanishing, and d k = D / h = 64, where h = 12 is the number of attention heads. Finally, the feature pyramid network structure is used to fuse the feature representations at three different levels to generate a multi-scale feature map. The fusion process is expressed as F multi = α1F shallow + α2F middle + α3F deep , where F multi represents the fused multi-scale feature, and F shallow , F middle , F deep represent the output features of the shallow, middle, and deep encoders respectively. α1, α2, and α3 are fusion weight coefficients, which are optimized through backpropagation and are generally set to the initial values α1 = 0.2, α2 = 0.3, and α3 = 0.5. The purpose of this step is to construct a comprehensive representation of the fishing boat image and provide rich visual feature information for subsequent region of interest localization, especially to improve the adaptability to ship name areas of different sizes through multi-scale representation.

[0257] The specific implementation of step S02 is to use the self-attention mechanism to determine the first-level region of interest. This step first calculates the attention weight matrix for each Token in the input image and normalizes the weight distribution through the Softmax function; then analyzes the attention heat map, and the calculation of the attention heat map is expressed as where A map represents the average attention heat map, A i represents the attention matrix of the i-th attention head, and h represents the number of attention heads, h = 12. The regions with weight values higher than the 0.65 threshold are identified as candidate regions of interest. The identification process of high-attention regions is expressed as M roi1 = {(x, y)|A map (x, y) > τ1}, where M roi1Let (x, y) represent the first-level region of interest mask, (x, y) represent the pixel coordinates in the image, and τ1 represent the attention threshold, τ1 = 0.65. Next, a connected component analysis algorithm is applied to merge adjacent high-attention regions into a single region; subsequently, the visual area percentage of each region is calculated, expressed as: Where R area Area(M) represents the percentage of a region's area. roi1 Area(I) represents the area of ​​the region of interest. total The area represents the total image area, and regions exceeding 15% of the total image area are retained. Finally, a geometric shape analysis algorithm is applied to calculate features such as the perimeter-to-area ratio and contour complexity of the regions. The contour complexity is expressed as... Where C complexity Let P represent the perimeter of the region and A represent the area of ​​the region. Regions with a perimeter-to-area ratio between 0.08 and 0.25 and a contour complexity index below 1.8 are retained. The purpose of this step is to quickly locate the main region containing the hull in the global image, reduce the search space for subsequent processing, improve computational efficiency, and at the same time eliminate background interference to ensure that the focus is on the hull structure.

[0258] The specific implementation of step S03 involves determining the second-level region of interest (ROI) based on the first-level ROI. This step first applies enhanced positional coding to the first-level ROI, employing a combination of sine and cosine positional coding and learnable positional coding. The enhanced positional coding is calculated as follows: in This indicates enhanced positional coding. Indicates the position code for sine and cosine. Let λ represent the learnable location encoding, and λ be the fusion coefficient, where λ = 0.6. Then, a multi-head attention mechanism is used for fine-grained analysis of this region. Eight attention heads are set, each with a 64-dimensional attention matrix, to capture features from different semantic subspaces. The multi-head attention response calculation is represented as follows: Where A response Represents the attention response map, w i The weight of the i-th attention head is represented by the attention head. i w represents the output of the i-th attention head. i Obtained through training, with initial values ​​set to equal weights. Next, the attention response map is calculated and threshold segmentation is applied. Regions with an attention response value greater than 0.78 are extracted as candidate regions, and the second-level interest region is determined and represented as M. roi2 ={(x, y)|A response (x, y)>τ2}∩R prior M roi2Let R represent the second-level region of interest mask, τ2 represent the attention response threshold, τ2 = 0.78, and R prior The first step involves defining a constrained region based on prior knowledge, determined using a ship type database. Then, prior knowledge constraints are introduced, and statistical information from the database is used to limit the search area to the forward region of the hull and the outer walls of the ship. Finally, morphological operations are applied to optimize the region boundaries, using a combination of opening and closing operations to eliminate noise and smooth edges. The structuring element size is set to 3×3 pixels. The purpose of this step is to further narrow down the recognition range and accurately locate key areas on the hull that may contain the ship's name, such as the outer walls of the ship and the surface of the bridge, laying the foundation for subsequent fine-grained recognition.

[0259] The specific implementation of step S04 involves applying a fine-grained feature extraction network to determine the third-level region of interest. This step first applies a high-resolution feature extraction module to the second-level region of interest, dividing the region into smaller 4×4 pixel blocks for detailed analysis. The fine-grained feature extraction is represented as follows: Where F fine The expression represents fine-grained features. `Conv` represents the convolution operation, and `Resize` represents the resampling operation, reducing the second-level region of interest to one-quarter of its original size to obtain a more refined feature representation. Then, feature enhancement is performed using a fine-grained attention network, which contains four attention heads, each focusing on character-level visual features. Next, a text region detection algorithm is applied, identifying potential text regions based on stroke density maps and text orientation fields. The text region detection computation is represented as P. text =σ(W2·ReLU(W1·F) fine +b1)+b2), where P text The graph represents the probability map of the text region. W1, W2, b1, and b2 are model parameters, σ represents the Sigmoid activation function, ReLU represents the modified linear unit activation function, and the text region probability threshold is set to 0.82. Subsequently, token segmentation technology is used to accurately locate the boundaries of the text region, and an adaptive thresholding method is used to separate the foreground text from the background. The adaptive threshold calculation is expressed as follows: Where T adaptive Let μ(x, y) represent the adaptive threshold, μ(x, y) represent the local region mean, σ(x, y) represent the local region standard deviation, and γ represent the contrast weighting coefficient, where γ = 0.7. The third-level region of interest is represented as M. roi3 ={(x, y)|P text (x, y) > τ3}, where M roi3 Let τ represent the third-level region of interest mask, and τ3 represent the text region probability threshold, where τ3 = 0.82. Finally, a boundary optimization algorithm is applied to fine-tune the detected text regions, including minimum bounding rectangle calculation, boundary expansion reservation, and boundary regularization. The boundary expansion processing is represented by B. expanded=Expand(B original r expand ), where B expanded B represents the expanded boundary. original Represents the original boundary, r expand Indicates the expansion ratio, r expsand =0.1 indicates that the expanded width is 10% of the original text area width. The purpose of this step is to accurately locate the specific area where the ship name is located, to achieve pixel-level precision in extracting the ship name area, and to ensure that subsequent processing can focus on the minimum necessary area containing the ship name.

[0260] The specific implementation of step S05 involves constructing a distortion matrix for the third-level region of interest. This step first extracts key parameters such as observation distance from image metadata, atmospheric visibility from a meteorological database, optical parameters from a camera parameter file, the actual size of the ship's name area from image analysis, and imaging time from metadata. Then, a mathematical model is established based on the optical imaging degradation equation. This model considers the combined effects of atmospheric scattering, the diffraction limit of the optical system, and focus shift on image quality. The optical imaging degradation equation is expressed as g(x,y)=h(x,y)*

[0261] f(x, y) + n(x, y), where g(x, y) represents the observed image, h(x, y) represents the point spread function, f(x, y) represents the ideal distortion-free image, n(x, y) represents additive noise, and * represents the convolution operation. The point spread function is calculated as h(x, y) = h system (x, y)*h atmospheric (x, y)*h defocus (x, y), where h system h represents the point spread function of the optical system. atmospheric h represents the atmospheric scattering point spread function. defocus This represents the point spread function away from the focal point. Specifically, the point spread function of an optical system is expressed as: Where J1 represents the first-order Bessel function, r represents the radial distance, and r0 represents the Airy disk radius. Where λ represents the wavelength of light, f represents the focal length, and D represents the aperture diameter; the atmospheric scattering point spread function is expressed as... Where r0 represents the atmospheric coherence length. Where L represents the observation distance, C n The atmospheric refractive index structure constant is represented by [insert value here], which is related to meteorological conditions; the out-of-focus diffusion function is expressed as [insert value here]. Where R defocus Indicates the defocus radius. Where s represents the actual object distance. Next, the point spread function (PSF) is calculated for different observation distances to quantify the degree of image blur. The increase in the distance-related PSF radius is expressed as: Where ΔR PSF α represents the increase in the point spread function radius, d represents the observation distance (meters), d0 represents the reference distance, d0 = 50 meters, and α PSF α represents the radius increase coefficient. PSF = 0.5 pixels / 10 meters. When the observation distance exceeds 50 meters, the increase in the point spread function radius should not be less than 0.5 pixels / 10 meters. Subsequently, a signal-to-noise ratio (SNR) attenuation matrix is ​​generated, and the signal strength to noise level ratio of each sub-region is analyzed. The SNR attenuation matrix is ​​calculated as follows: Where SNR(x, y) represents the signal-to-noise ratio (dB) at location (x, y), S(x, y) represents the signal strength, and N(x, y) represents the noise strength. Regions below 6 dB are marked as high-distortion risk areas. Finally, normalization is performed to generate a normalized distortion matrix, which is calculated as follows: Where D norm SNR represents the normalized distortion matrix. threshold Represents the signal-to-noise ratio threshold, SNR threshold =20 decibels, D norm The value ranges from 0 to 1, where 0 represents no distortion and 1 represents complete distortion. The purpose of this step is to quantitatively evaluate the impact of observation distance on the image quality of the ship name area, providing a scientific basis for subsequent image enhancement processing and ensuring targeted improvement of low-quality areas.

[0262] The specific implementation of step S06 involves calculating the third-level region of interest deviation matrix. This step first applies a hull attitude light reflection model to analyze the interaction between light and the hull surface. It extracts the hull surface normal vector from the image, obtains the incident light angle using a lighting estimation algorithm, acquires the hull material reflection coefficient using a material recognition network, obtains the surface roughness through texture analysis, and assesses the coating aging degree based on image degradation. The calculation of the hull surface normal vector is expressed as follows: in Represents the surface normal vector. Represents the image gradient. The gradient modulus is represented as; the incident light angle estimate is expressed as... in Let S represent the normalized incident light direction vector, and let S represent the set of pixels in the highlight region. Let represent the half-vector of the i-th pixel. Let represent the surface normal vector of the i-th pixel. The hull surface reflection model is represented as... Among them I reflect k represents the intensity of reflected light. dThe diffuse reflectance coefficient is obtained from the material recognition network. This represents the surface normal vector, calculated from image analysis. The direction of the incident light is represented by k, obtained from the illumination estimation algorithm. s The specular reflection coefficient is obtained from the material recognition network. Indicates the direction of the reflected light. The line of sight indicates the viewing direction, and 'n' represents the roughness index, obtained from texture analysis. The index typically ranges from 1 to 200; a higher value indicates a smoother surface. Surface roughness is calculated as follows: Where R roughness σ represents surface roughness. gradient σ represents the local gradient standard deviation. gradient，max This represents the global maximum gradient standard deviation, with roughness values ​​ranging from 0 to 1; a larger value indicates a smoother surface. The coating aging degree is calculated as follows: Where A coating Indicates the degree of coating aging, CV local Represents the local color variation coefficient, CV reference The value represents the reference color variation coefficient; a higher aging value indicates more severe coating aging. Then, a model of the relationship between the hull attitude and perspective distortion is established, and the tilt angle of the hull surface relative to the camera is calculated. This model is represented by D. perspective (θ)=1+β·max(0,θ-θ0), where D perspective The text distortion coefficient is represented by θ, which represents the angle (in degrees) between the hull surface and the camera plane. θ0 represents the critical angle, where θ0 = 30 degrees. β represents the distortion coefficient, where β = 0.02 / degree, meaning the distortion coefficient increases by 0.02 for every 1 degree increase in angle. When θ exceeds 60 degrees, D... perspective The value is greater than 1 + 0.02 * (60 - 30) = 1.6, which meets the requirements. Next, the degree of perspective distortion in the text area under different postures and angles is analyzed, and a quantitative index for the degree of distortion is established. The impact of the perspective distortion angle on the recognition difficulty is expressed as... Where D difficulty The difficulty level is indicated by θ, where θ represents the perspective distortion angle (in degrees). For every 10-degree increase in the perspective distortion angle, the difficulty level increases. Subsequently, an effective contrast matrix for the ship name area is generated, and the changes in text and background contrast under different lighting conditions are evaluated. The effective contrast matrix for the ship name area is calculated and represented as follows:

[0263] Where C effective I represents the effective contrast matrix. text Indicates the brightness of the text area, I bakcground A represents the brightness of the background area. coatingThe degree of coating aging is indicated by α, which represents the influence coefficient of the incident angle (α = 0.6). Areas with an effective contrast ratio below 0.3 are marked as high-risk areas. Finally, a text edge sharpness evaluation matrix is ​​calculated to analyze the text outline clarity. The text edge sharpness evaluation matrix is ​​calculated and represented as follows: Where S edge This represents the edge sharpness evaluation matrix. R represents the gradient magnitude. roughness A represents surface roughness. coating The coating aging degree is represented by β, which is the roughness influence coefficient (β = 0.4), and γ, which is the aging influence coefficient (γ = 0.5). Areas with edge sharpness below 0.4 require special reinforcement treatment. The deviation matrix is ​​calculated as B(x, y) = 1 - min(C effective (x, y), S edge (x, y)), where B(x, y) represents the deviation matrix, C effective Indicates effective contrast, S edge The edge sharpness is represented by B(x, y), which ranges from 0 to 1, where 0 indicates no deviation and 1 indicates maximum deviation. The purpose of this step is to quantitatively assess the degree of geometric distortion in the ship's name area caused by changes in hull attitude, providing precise parameters for subsequent perspective correction and image enhancement.

[0264] The specific implementation of step S07 is to generate a recognizable matrix based on the comprehensive calculation of the distortion matrix and the deviation matrix. This step first applies a multi-feature fusion function to integrate various influencing factors, including the distortion matrix generated in the previous steps, the calculated deviation matrix, the ambient brightness matrix obtained from image analysis, the ship hull texture complexity obtained from texture analysis, and the ship name font feature vector obtained from the font recognition network. The multi-feature fusion function is expressed as R(x, y) = w1·D norm (x, y) + w2·B(x, y) + w3·L(x, y) + w4·T(x, y) + w5·F(x, y), where R(x, y) represents the original comprehensive score, D norm Let B represent the normalized distortion matrix, B the bias matrix, L the ambient brightness matrix, T the hull texture complexity, F the feature vector of the ship's name font, and w1, w2, w3, w4, and w5 represent the weights of each factor, with w1 = 0.35, w2 = 0.3, w3 = 0.15, w4 = 0.1, and w5 = 0.1. The ambient brightness matrix is ​​calculated as follows: Where L(x, y) represents the ambient brightness deviation, and I(x, y) represents the brightness value at position (x, y). max This represents the maximum brightness value. A value closer to 0 indicates moderate brightness, while a value closer to 0.5 indicates excessively high or low brightness. Texture complexity calculation is expressed as... Where T(x, y) represents the texture complexity, and N represents the number of pixels in the local region. Indicates position (x) i y i The gradient magnitude at point () is then calculated. A weighted summation method is then used to calculate the overall score, with the weights of each factor being: distortion factor 0.35, bias factor 0.3, brightness factor 0.15, texture complexity 0.1, and font feature 0.1. Next, a nonlinear mapping function is applied to convert the original score into a standardized recognition difficulty coefficient, which is calculated as follows: Where R norm R represents the standardized recognition difficulty coefficient, R represents the original comprehensive score, k represents the sigmoid function steepness parameter, k=5, R0 represents the center point parameter, R0=0.5, R norm The value ranges from 0 to 1, where 0 indicates easy recognition and 1 indicates extremely difficult recognition. Subsequently, a recognition difficulty heatmap is generated based on the recognition difficulty coefficient, visually displaying the distribution of recognition difficulty for different parts of the ship name area. The recognition matrix is ​​then generated as follows: R norm The standardization recognition difficulty coefficient is represented by τ4, which represents the recognition difficulty threshold. τ4 = 0.7, and areas with a recognition difficulty coefficient higher than 0.7 are marked as high-risk areas. Finally, optimization strategy recommendations are determined based on the recognizability score, including recommended image processing methods and parameter settings. When the overall recognizability is below 0.5, adaptive enhancement processing is recommended. The purpose of this step is to comprehensively evaluate the recognition difficulty of the ship name region under the current conditions, providing a decision-making basis for subsequent targeted enhancement processing.

[0265] The specific implementation of step S08 involves designing an adaptive region enhancement algorithm. This step first identifies the regions requiring enhancement based on the identifiability matrix, prioritizing regions with an identifiability coefficient greater than 0.6. Then, it designs differentiated enhancement strategies for different problems. For distortion-dominated problems, it applies a deconvolution recovery algorithm, expressed as follows: in Let represent the restored image, h represent the point spread function, g represent the observed image, D represent the difference operator, and λ represent the regularization parameter, λ = 0.02. The convolution kernel size is 5×5. For bias-dominated problems, a perspective correction algorithm is applied, which is expressed as: Where H represents the perspective transformation matrix, A1 represents the coordinate matrix of the four corner points of the source quadrilateral, and A2 represents the coordinate matrix of the four corner points of the target rectangle. The transformed image is obtained through... Calculate, where I corrected I represents the corrected image. original h represents the original image. ijThe elements of the perspective transformation matrix H represent perspective transformations based on a four-point correspondence, with control points selected at the four corners of the text. For insufficient contrast, local contrast enhancement is applied; the local contrast enhancement algorithm is represented as I. enhanced (x,y)=I(x,y)+α(x,y)·(I(x,y)-I blur (x, y)), where I enhanced I represents the enhanced image, and I represents the original image. blur This represents the image after Gaussian blurring, where α represents the enhancement coefficient. To ensure the maximum gain does not exceed 200% of the original contrast, the enhancement factor is adaptively adjusted based on local brightness, with the maximum gain not exceeding 200% of the original contrast. Then, edge enhancement processing is applied to improve text edge sharpness; this edge enhancement processing is represented as I. edge (x,y)=I(x,y)+β(x,y)·(I(x,y)-I blur (x, y)) 2 ·sign(I(x,y)-I blur (x, y)), where I edge I represents the image after edge enhancement, and I represents the original image. blur This represents the image after Gaussian blurring, where β represents the nonlinear enhancement coefficient. Where β0 represents the base enhancement coefficient, β0 = 10, and ∈ represents a small constant to prevent division by zero, ∈ = 0.01. A nonlinear gradient enhancement method is used, and the enhancement intensity is inversely proportional to the edge sharpness matrix. Finally, noise suppression filtering is applied, based on a guided filtering algorithm to protect text edges while suppressing noise. The guided filtering algorithm is expressed as I. fltered (x, t) = a (x, y) G (x, y) + b (x, y), where I filtered The image represents the filtered image, G represents the guide image (usually the original image), and a and b are the coefficients of the local linear model, obtained by minimizing the cost function. Solve for ω. (x，y) This represents a window centered at (x, y), where ∈ represents the regularization coefficient, ∈ = 0.1, and the filter radius is set to 3 pixels. The purpose of this step is to specifically enhance the low-recognition area based on the aforementioned analysis results, thereby improving the overall recognizability of the ship name area and providing high-quality input for the subsequent character recognition system.

[0266] The specific implementation of step S09 is to output the optimized ship name region image and coordinate information. This step first integrates the aforementioned processing results to generate a high-quality ship name region image; then, it calculates the precise coordinate information of the ship name region, which is represented as B. final ={x min y min xmax y max ,θ,conf}, where B final Indicates the final bounding box information, x min y min x max y max These represent the coordinates of the top-left and bottom-right corners of the bounding box, respectively; θ represents the rotation angle; and conf represents the confidence score. The confidence score is calculated as follows: Where conf represents the confidence score, |M roi3 | Represents the number of pixels in the third-level region of interest, P text Let represent the probability of the text region, and RM represent the recognizable matrix. Next, the processed ship name region image and coordinate information are transmitted to the verification system. Then, a MarineIDNet neural network is applied for ship name region verification. This network employs a dual-branch structure, including feature extraction and region verification branches, and verifies the validity of the region through multi-layer feature extraction and comparative analysis. Finally, the verified regions undergo fine-tuning of their boundaries. The fine-tuning boundary algorithm is represented as B0. adjusted =B final +ΔB, where B adjusted Indicates the adjusted boundary, B final Let f represent the initial boundary, ΔB represent the boundary adjustment amount, and ΔB = f adjustment (G text ), where f adjustment G represents the boundary adjustment function. text This step represents the gradient map of the text edges, with the adjustment precision controlled within 2 pixels of the original boundary. The purpose of this step is to provide the final optimized image of the ship name region and its precise location information, providing standardized input for subsequent text recognition systems, while also ensuring the accuracy and reliability of the extraction results through a verification process.

[0267] To better understand and implement this invention, a specific application scenario is provided below as Example 2: In an intelligent ship monitoring project at a marine research institution, researchers used a ViT-based method for extracting the vessel names of fishing vessels entering and leaving a port to automate the monitoring and identification of such vessels. This project aims to improve the efficiency of fishing vessel management, reduce labor costs, and ensure accurate identification of vessel names under various complex environmental conditions. Researchers selected 50 different types of fishing vessels and collected a total of 500 high-definition images under different lighting, weather, and observation distance conditions for testing, to verify the effectiveness and robustness of the method. The test images were divided into three groups: a close-range group (5–10 meters), a medium-range group (10–30 meters), and a long-range group (30–100 meters), each group containing images under different weather and lighting conditions.

[0268] During the system initialization phase, the researchers first performed parameter settings and pre-training on ViT. The model parameter settings are shown in Table 1:

[0269] Table 1 ViT Parameter Settings Table

[0270]

[0271]

[0272] For a typical test image (resolution 3840×2160, mid-range group, sunny lighting, distance approximately 15 meters), step S01 is used to construct a three-level pyramid feature extraction structure. After processing by three encoders of different depths, the fused feature weight coefficients are set to α1 = 0.2 (shallow layer), α2 = 0.3 (mid-range layer), and α3 = 0.5 (deep layer), respectively. The fused multi-scale features F multi It has 768 dimensions and contains global and local feature information.

[0273] In step S02, a self-attention mechanism is applied to determine the first-level region of interest, and the resulting average attention heatmap A is calculated. map The dimensions are 224×224. An attention threshold τ1 = 0.65 was set to extract high-attention regions. After connected component analysis, three candidate regions were obtained, with area percentages of 23.6%, 18.7%, and 9.4%, respectively. Geometric shape analysis yielded contour complexity indices of 1.24, 1.76, and 2.31, with perimeter-to-area ratios of 0.11, 0.19, and 0.28, respectively. The first and second regions were retained as first-level regions of interest.

[0274] In step S03, enhanced positional coding is applied to the first-level region of interest, followed by fine-grained analysis using eight attention heads (each with a dimension of 64). An attention response threshold τ2 = 0.78 is set, and combined with prior knowledge constraints from the ship type database, the area on the starboard side of the hull is ultimately determined as the second-level region of interest, with a size of 512 × 256 pixels.

[0275] In step S04, a fine-grained feature extraction network is applied to process the second-level region of interest, dividing it into 4×4 pixel blocks for detailed analysis. Using a text region detection algorithm with a probability threshold τ3 = 0.82, the ship's name and number area on the hull were successfully detected. An adaptive thresholding algorithm (contrast weight coefficient γ = 0.7) is applied to accurately segment the text region, ultimately determining the third-level region of interest, with a size of 124×42 pixels. The boundary expansion ratio is set to 0.1, resulting in an expanded region size of 136×46 pixels.

[0276] In step S05, the key parameters extracted when constructing the distortion matrix are shown in Table 2:

[0277] Table 2 Image Quality Parameters

[0278] Parameter name Parameter value Observation distance 15.3 meters Atmospheric visibility 23.5 kilometers Camera focal length 85 mm Aperture diameter 17 mm Actual dimensions of the ship name area 0.9 x 0.3 meters Imaging time 14:25:36 Atmospheric refractive index structure constant <![CDATA[3.5×10 -14 ]]>

[0279] Based on the above parameters, the calculated point spread function radius is 1.2 pixels, and the lowest value in the signal-to-noise ratio attenuation matrix is ​​12.6 dB, which is higher than the 6 dB threshold, indicating good image quality at this observation distance. Normalized distortion matrix D norm The average value is 0.37.

[0280] In step S06, when calculating the deviation matrix, the key parameters extracted from the image are shown in Table 3:

[0281] Table 3. Parameters of Light Reflection from Ship Hull

[0282]

[0283]

[0284] The coefficient of text distortion D caused by the ship's attitude perspective =1.24, the influence coefficient D of perspective distortion angle on recognition difficulty difficulty =1.84, effective contrast matrix C effective The average value is 0.52, which is higher than the 0.3 threshold. (Text edge sharpness matrix S) edge The average value is 0.61, which is higher than the 0.4 threshold. The final calculated average value of the deviation matrix B is 0.48.

[0285] In step S07, various factors are integrated using a multi-feature fusion function. The average value of the environmental brightness matrix L is 0.12, the texture complexity T is 0.28, and the similarity of the font feature vector F is 0.83. The original comprehensive score R is calculated using weights w1 = 0.35 (distortion), w2 = 0.3 (bias), w3 = 0.15 (brightness), w4 = 0.1 (texture), and w5 = 0.1 (font). This score is then mapped to a standardized recognition difficulty coefficient R using a Sigmoid function (parameter k = 5, R0 = 0.5). norm The average value is 0.56, which is lower than the 0.7 threshold, but enhancement is still needed.

[0286] In step S08, since the recognition difficulty coefficient of 0.56 is greater than 0.5, adaptive enhancement processing is required. Based on the recognizable matrix analysis, the ship name area is mainly affected by perspective distortion; therefore, a perspective correction algorithm is applied. The perspective transformation matrix H is calculated to correct the text area, and then a local contrast enhancement algorithm is applied with adaptively adjusted enhancement coefficients, averaging 1.36. In edge enhancement processing, the average non-linear enhancement coefficient is 16.39. Finally, guided filtering is applied for noise suppression, with a filtering radius of 3 pixels and a regularization coefficient of 0.1. The enhancement processing results are shown in Table 4.

[0287] Table 4 Comparison of Image Enhancement Effects

[0288] Evaluation indicators Before processing After processing Improvement rate Effective contrast 0.52 0.78 50.0％ Edge sharpness 0.61 0.85 39.3％ Signal-to-noise ratio 12.6 decibels 18.9 decibels 50.0％ Text distortion coefficient 1.24 1.03 16.9％ Difficulty level of recognition 0.56 0.28 50.0％

[0289] In step S09, the final optimized image and coordinate information of the ship name region are output. The bounding box coordinates are (1562, 873, 1698, 919), the rotation angle is 2.7 degrees, and the confidence score is 0.92. MarineIDNet is used for verification, confirming that the extraction results are accurate and effective, and the final deviation after fine-tuning of the boundary is less than 1.8 pixels.

[0290] Based on the analysis of the processing results of all 500 test images, the accuracy of this method in extracting the ship name region at different distance groups is shown in Table 5:

[0291] Table 5. Statistical table of accuracy rate for ship name region extraction under different conditions.

[0292] Test conditions Sample size Accurately extract numbers accuracy Sunny day up close 65 64 98.5％ Cloudy at close range 55 53 96.4％ Close-up Dusk 35 33 94.3％ Medium distance sunny 70 67 95.7％ Mid-range cloudy 60 56 93.3％ Mid-distance dusk 40 36 90.0％ Sunny at a distance 80 70 87.5％ Cloudy at a distance 60 51 85.0％ distant dusk 35 28 80.0％ total 500 458 91.6％

[0293] Traditional methods for extracting the name and logo regions of fishing vessels mainly employ target detection algorithms based on convolutional neural networks, such as YOLO, SSD, or Faster R-CNN. These methods have several technical limitations: First, they have limited ability to detect small targets at long distances and under varying lighting conditions, with detection accuracy typically below 70% at distances beyond 30 meters. Second, traditional methods lack modeling for perspective distortion and image quality degradation, leading to a sharp decline in performance in complex environments. Third, they do not consider the material properties of the hull and the characteristics of light reflection, resulting in insufficient accuracy in recognizing text in the hull area. Finally, traditional methods typically employ image enhancement strategies with fixed parameters, failing to provide targeted processing based on different degradation types. In contrast, the three-level pyramid feature extraction structure based on ViT in this invention can better capture global and local features, significantly improving detection capabilities under long-distance conditions (by approximately 25%). By accurately modeling the imaging process through optical imaging degradation equations and ship attitude light reflection models, the system can understand and cope with various complex environments. The introduction of multi-feature fusion functions and recognizable matrices enables a quantitative assessment of recognition difficulty. The adaptive region enhancement algorithm can select the optimal processing strategy based on different types of image degradation. Overall, this invention achieves an average accuracy of 91.6% under various complex environmental conditions, an improvement of approximately 20 percentage points compared to traditional methods, particularly demonstrating significant advantages under long-distance and low-light conditions.

[0294] It should be noted that the variables involved in this invention are explained in detail in Tables 6, 7, 8, and 9 below.

[0295] Table 6. Variable Explanation Table (Part 1)

[0296]

[0297] Table 7. Variable Explanation Table (Part Two)

[0298]

[0299]

[0300] Table 8. Variable Explanation Table (Part 3)

[0301]

[0302]

[0303] Table 9. Variable Explanation Table (Part Four)

[0304]

[0305] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for extracting the vessel name region of fishing vessels entering and leaving ports based on ViT, characterized in that, include: A three-level pyramid feature extraction structure based on ViT was designed for fishing vessel image analysis. A first-level region of interest (ROI) is determined using a self-attention mechanism. A second-level ROI is extracted based on the first-level ROI. A fine-grained feature extraction network is applied to determine the third-level ROI. A distortion matrix is ​​constructed for the third-level ROI, and the changes in image sharpness at different observation distances are quantified using an optical imaging degradation equation. A mapping function between distance and image quality degradation is established. The optical imaging degradation equation is used to simulate the image quality changes of the ship name area at different observation distances. The deviation matrix of the third-level ROI is calculated, and a ship attitude light reflection model is applied to analyze the degree of perspective distortion in the ship name area caused by changes in ship attitude. A function between attitude angle and text distortion degree is established. The ship attitude light reflection model is used to analyze the impact of the light reflection characteristics of the ship surface on ship name recognition. A recognizable matrix is ​​generated based on the distortion matrix and the deviation matrix. An adaptive region enhancement algorithm is designed to perform targeted image enhancement processing on low-recognizable areas according to the recognizable matrix. The final optimized ship name area image is output, and the coordinate information of the ship name area is provided for subsequent text recognition and verification systems. The first-level region of interest (ROI) refers to a large-scale area containing the main structure of the hull, determined after preliminary analysis of the overall image of the fishing vessel using ViT. The first-level ROI covers the main outline of the fishing vessel and its surrounding area. The second-level ROI refers to the hull structure area determined by further narrowing the scope based on the first-level ROI. The second-level ROI includes key parts such as the hull side and the outer wall of the bridge where the ship's name is marked. The third-level ROI refers to the precise location area of ​​the ship's name after fine processing. The third-level ROI only contains the ship's name text and its adjacent background.

2. The method according to claim 1, characterized in that, The distortion matrix is ​​a mathematical model that quantifies the impact of different observation distances on image sharpness. The distortion matrix is ​​constructed by analyzing the attenuation of high-frequency information in the image and is used to assess the recognition difficulty caused by distance factors. The deviation matrix is ​​a mathematical model that characterizes the degree of geometric deformation of the ship name area caused by changes in the ship's attitude angle. The deviation matrix mainly considers the difficulties brought by perspective projection transformation to text recognition. The identifiable matrix is ​​an evaluation index generated by combining the distortion matrix and the deviation matrix. The identifiable matrix uses numerical values ​​to quantify the difficulty of identifying the ship name area under the current observation conditions.

3. The method according to claim 2, characterized in that, It also includes applying an optical imaging degradation equation to simulate the image quality variation of the ship name area at different observation distances. The inputs of the optical imaging degradation equation include the observation distance obtained from image metadata, the atmospheric visibility obtained from the meteorological database, the camera optical parameters obtained from the camera parameter file, the actual size of the ship name area measured from image analysis, and the imaging time obtained from the image metadata. The output of the optical imaging degradation equation is the distance-dependent point spread function and the signal-to-noise ratio attenuation matrix.

4. The method according to claim 3, characterized in that, It also includes the application of a hull attitude light reflection model to analyze the impact of hull surface light reflection characteristics on ship name recognition. The input of the hull attitude light reflection model includes the hull surface normal vector calculated from image analysis, the incident light angle obtained from the illumination estimation algorithm, the hull material reflection coefficient obtained from the material recognition network, the hull surface roughness obtained from texture analysis, and the coating aging degree obtained from image degradation assessment. The output of the hull attitude light reflection model is the effective contrast and text edge sharpness evaluation matrix of the ship name region.

5. The method according to claim 4, characterized in that, It also includes applying a multi-feature fusion function to determine the recognizability of the ship name area by integrating multiple factors. The input of the multi-feature fusion function includes the distortion matrix, the deviation matrix, the ambient brightness matrix obtained from image analysis, the ship texture complexity obtained from texture analysis, and the ship name font feature vector obtained from the font recognition network. The output of the multi-feature fusion function is the area recognizability score and optimization strategy suggestions.

6. The method according to claim 5, characterized in that, The method also includes using the MarineIDNet model neural network for ship name area verification and boundary fine adjustment. The specific structure of the MarineIDNet model is a dual-branch network architecture, which includes a feature extraction branch and a region verification branch. The feature extraction branch adopts an improved ViT structure, and the region verification branch is composed of a four-layer convolutional neural network and two fully connected layers.

7. The method according to claim 6, characterized in that, The steps for establishing the training dataset during the pre-training process of the MarineIDNet model specifically include collecting high-resolution images of different types of fishing vessels under various lighting conditions, weather environments, and observation distances; using a semi-automatic annotation tool to label the vessel name area with rectangular boxes and transcribe the text content; performing stratified sampling equalization processing based on vessel size, observation distance, and weather conditions; and finally constructing a pre-training dataset containing 50,000 images.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions, which, when executed in a computer, are used to perform the ViT-based method for extracting the vessel name region of fishing vessels entering and leaving ports, as described in any one of claims 1-7.

9. A system for extracting the vessel name region of fishing vessels entering and leaving ports based on ViT, characterized in that, The system includes the computer-readable storage medium of claim 8, wherein the system is any one of a computer, a server, or a microcontroller, the computer-readable storage medium is disposed within the system, and the system is provided with a microprocessor that executes the program instructions stored in the computer-readable storage medium.

Images