A trademark recognition method and system based on deep learning technology

Abstract

The application provides a trademark identification method and system based on deep learning technology, and belongs to the technical field of computer vision and image data processing; the method comprises the following steps: constructing a detection data set by uniformly marking all brand trademarks as a single trademark category; a trademark detection model is composed of a double-branch main network in parallel with a self-supervised visual branch and a convolutional neural network branch; the features of the two branches are projected into the same feature space through a feature alignment module, and the feature alignment loss in the form of mean square error is used for explicit supervision and alignment; the features are adaptively fused through a convolution block attention module, multi-scale features are extracted by a neck network, and a decoupled detection head is used to decode and output a horizontal bounding box according to the distance distribution expectation; during identification, the cosine similarity of the prototype features is calculated by taking the mean value of the brand trademark features and the trademark feature graph library; high-precision positioning and identification of trademarks in complex scenes, automatic rejection of unknown trademarks outside the graph, and low-cost expansion of the graph prototype by adding new brands are achieved.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image data processing technology, and in particular to a trademark recognition method and system based on deep learning technology. Background Technology

[0002] Trademarks are important identifiers that distinguish the source of goods or services. In applications such as commodity circulation supervision, brand protection, advertising content monitoring, and intellectual property infringement evidence collection, there is a need for the automatic location and recognition of trademarks appearing in images or videos. Therefore, automatic trademark recognition technology has a wide range of applications.

[0003] Traditional trademark recognition methods primarily employ manually designed local feature descriptors (such as SIFT and SURF) to extract image features, combined with template matching and other methods to achieve trademark location and recognition. These methods rely on manually designed features, which are less robust to changes in lighting, scale, partial occlusion, and appearance deformation, making them ill-suited to the complex and varied imaging conditions of real-world scenarios.

[0004] In recent years, deep learning-based object detection technology has been gradually applied to trademark recognition tasks. By training deep neural networks on large-scale labeled data to automatically learn image features, it has achieved good detection results in conventional scenarios. However, existing deep learning-based trademark recognition schemes still have the following shortcomings in practical applications: First, brand expansion is costly. Existing solutions generally treat each brand as an independent detection or classification category, with the number of categories in the model's output layer corresponding one-to-one with the brands appearing in the training set. The range of brands that the model can recognize is fixed during the training phase. When a new brand appears, it is necessary to re-collect training samples for that brand, re-label them, and retrain the entire model. The whole process is time-consuming and requires significant labeling investment, making it difficult to meet the application needs of rapid trademark and brand updates.

[0005] Secondly, the recognition performance is significantly affected by the sample distribution. In real trademark data, the frequency of occurrence of different brands varies greatly, and the number of samples for some long-tail brands is very limited. The number of samples between categories is seriously unbalanced, resulting in lower recognition accuracy for brands with fewer samples.

[0006] Third, the localization accuracy of difficult samples is insufficient. In real-world scenarios, trademarks vary greatly in size, and small-sized trademarks, trademarks with low contrast to the background, and trademarks with blurred boundaries are common. Existing detection methods have limited accuracy in locating such targets, which can easily lead to missed detections and false detections, thus affecting the overall recognition performance.

[0007] It is evident that existing trademark recognition technologies struggle to balance brand expansion costs with detection and recognition accuracy in complex scenarios. How to adapt to dynamic changes in brand types at a lower cost while maintaining detection and recognition accuracy in complex scenarios is a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0008] To address the aforementioned technical challenges, this invention provides a trademark recognition method and system based on deep learning technology. The method utilizes a dual-branch backbone network—combining a convolutional neural network branch and a self-supervised visual Transformer branch—to extract local detail features and global semantic features of trademarks. Feature alignment loss is used to explicitly supervise the alignment of the feature distributions of the two branches, which are then adaptively fused via a convolutional block attention module. Multi-scale features are extracted by a feature pyramid network and a path aggregation network. A decoupled detection head decodes the horizontal bounding box using distance distribution expectation decoding. Simultaneously, all brands are uniformly labeled as a single trademark category for detection. Open-set recognition is performed using a prototype feature map library obtained by averaging by brand, combined with cosine similarity and a similarity threshold. This achieves high-precision trademark localization and recognition in complex scenarios, automatic rejection of unknown trademarks outside the map library, and low-cost brand expansion where adding new brands only requires adding prototypes to the map library without retraining the detection model.

[0009] To achieve the above objectives, this application proposes a trademark recognition method based on deep learning technology, comprising the following steps: S1. For the image to be recognized, the trademark detection model detects trademarks as a single category, outputs the horizontal bounding box of the trademark, and removes redundant bounding boxes through non-maximum suppression. The trademark detection model includes a dual-branch backbone network consisting of parallel convolutional neural network branches and a self-supervised visual Transformer branch, a feature alignment module, a neck network, and a decoupled detection head. The convolutional neural network branch extracts local detail features, and the self-supervised visual Transformer branch extracts global semantic features. The local detail features and the global semantic features are projected to the same feature space by the feature alignment module, and their distribution alignment is explicitly supervised by a feature alignment loss in the form of mean square error between the projected features before fusion. The fused features are then processed by the neck network to extract multi-scale features, and the horizontal bounding box is output by the decoupled detection head. S2. After cropping the corresponding regions of each horizontal bounding box in the original image, input them into the self-supervised visual Transformer branch to extract depth features and perform L2 normalization. Calculate the cosine similarity with the prototype features of each brand in the trademark feature library, and take the brand with the highest similarity as the candidate recognition result. The prototype features are obtained by L2 normalizing the depth features of each trademark region, averaging them according to the brand category, and then L2 normalizing them again. S3. Compare the maximum similarity with a preset similarity threshold; if it is not less than the similarity threshold, output the corresponding brand name; if it is less than the similarity threshold, determine it as an unknown trademark outside the trademark feature library and output the unknown trademark mark.

[0010] As a further solution, before detecting the image to be identified, the method further includes: acquiring an image containing the trademark and marking the location of the trademark with a horizontal bounding box, uniformly labeling all brand trademarks as a single trademark category, constructing a detection dataset, while retaining the brand category label of each trademark, and training the trademark detection model based on the detection dataset.

[0011] As a further solution, when constructing the trademark feature library, each trademark region is cropped from the training set images according to the brand category label, and the deep features are extracted by the self-supervised visual Transformer branch. The deep features are then L2 normalized and mean-valued according to the brand category, and then L2 normalized again to obtain the prototype features of each brand. When expanding into a new brand, only the sample of the new brand needs to be cut. After cropping the trademark area, the self-supervised visual Transformer branch is used to extract deep features and perform L2 normalization. The mean value is calculated according to the brand category and then L2 normalized again to obtain its prototype features, which are then appended to the trademark feature library without retraining the trademark detection model.

[0012] As a further solution, the convolutional neural network branch extracts shallow features at 1 / 8 resolution and mid-level features at 1 / 16 resolution through multi-level convolution and cross-stage partial connection modules. The self-supervised visual Transformer branch divides the input image into image blocks, and after linear projection, extracts global semantic features at a resolution of 1 / 16 through a multi-layer self-attention module; wherein, rotational position encoding is applied to the query matrix and the key matrix in the self-attention calculation.

[0013] As a further solution, the feature alignment module projects the features extracted by the convolutional neural network branches and the global semantic features into the same feature space through 1×1 convolutions, calculates the mean square error between the two projected features as the feature alignment loss, and explicitly supervises the alignment of the feature distributions of the two branches; and then concatenates the two projected features in the channel dimension and fuses them with the convolutional block attention module to obtain the aligned features; the total loss of training the trademark detection model is composed of the box regression loss based on perfect intersection-union ratio, the classification loss based on binary cross-entropy, the distribution focus loss based on distance distribution, and the feature alignment loss, weighted and summed according to preset weights.

[0014] As a further solution, when training the trademark detection model, the parameters of the image patch embedding layer, rotation position encoding, and the first few self-attention modules in the self-supervised visual Transformer branch are frozen, and only the parameters of the last few self-attention modules and the final normalization layer are fine-tuned. The learning rate of the self-supervised visual Transformer branch is 0.1 times that of the rest of the model.

[0015] As a further solution, the decoupled detection head adopts an anchorless design, including independent bounding box regression branches and classification branches, which operate on feature maps at multiple scales respectively. The bounding box regression branch outputs a distance distribution consisting of multiple discrete probability values ​​for the distances of the left, top, right, and bottom edges of the horizontal bounding box located at the center of the grid cell of each scale feature map. The expectation of the probability distribution after softmax normalization is first calculated to obtain continuous distance values, which are then decoded into the horizontal bounding box of the trademark by combining the anchor point coordinates. The classification branch outputs the confidence level of a single trademark category.

[0016] As a further solution, the neck network adopts a structure that combines a feature pyramid network and a path aggregation network, generating feature maps of multiple scales from top to bottom, backpropagating the shallow features from bottom to top, and introducing convolutional block attention modules at each level of the feature pyramid network.

[0017] As a further solution, the similarity threshold is adaptively determined by traversing candidate thresholds on the validation set, and the candidate threshold that best combines the correct recognition rate of known brands with the rejection rate of unknown trademarks is selected.

[0018] On the other hand, the present invention also provides a trademark recognition system based on deep learning technology, for performing a trademark recognition method based on deep learning technology as described in any of the preceding claims, comprising: The trademark detection model includes a dual-branch backbone network consisting of parallel convolutional neural network branches and a self-supervised visual Transformer branch, a feature alignment module, a neck network, and a decoupled detection head. It is used to detect trademarks as a single category. The convolutional neural network branches extract local detail features, and the self-supervised visual Transformer branches extract global semantic features. The features from the two branches are projected onto the same feature space by the feature alignment module and then fused. The neck network extracts multi-scale features, and the decoupled detection head outputs the horizontal bounding box of the trademark. Redundant bounding boxes are removed by non-maximum suppression. The trademark recognition image library is used to crop trademark regions from training set images based on brand category labels, input them into the self-supervised visual Transformer branch to extract depth features, perform L2 normalization on the depth features, calculate the mean value according to brand category, and perform L2 normalization again to obtain the prototype features of each brand, thus forming the trademark feature image library; it is also used to crop the detected trademark regions from the original image, input them into the self-supervised visual Transformer branch to extract depth features and perform L2 normalization, calculate the cosine similarity with the prototype features of each brand in the trademark feature image library, and select the brand with the highest similarity as the candidate recognition result; The recognition result output module is used to compare the maximum similarity of the candidate recognition results with a preset similarity threshold; if it is not less than the similarity threshold, the corresponding brand name is output; if it is less than the similarity threshold, an unknown trademark mark is output.

[0019] Compared with related technologies, the trademark recognition method and system based on deep learning technology provided by this invention has the following advantages: 1. This invention constructs a dual-branch backbone network by connecting a convolutional neural network branch and a self-supervised visual Transformer branch in parallel. The local detail features and global semantic features of the trademark are extracted respectively. The feature distributions of the two branches are explicitly supervised and aligned using feature alignment loss, and then adaptively fused by the convolutional block attention module. This effectively reduces the semantic gap between heterogeneous feature spaces, enabling the fused features to have both local detail perception and global semantic understanding capabilities.

[0020] 2. The decoupled detection head of the present invention models the frame regression as the distance distribution from the anchor point to the four sides of the horizontal bounding box in an anchorless manner. After softmax expectation decoding, it is converted into continuous distance values. Compared with direct regression of a single value, it can more accurately express the uncertainty of the horizontal bounding box position. Combined with the distribution focus loss for joint supervision of adjacent discrete positions, it improves the positioning accuracy of small-sized trademarks and trademarks with blurred boundaries and low contrast with the background.

[0021] 3. This invention decouples trademark positioning from brand recognition. The detection model detects only trademarks as a single category, with all brand samples participating in training. This avoids the category imbalance problem caused by the scarcity of long-tail brand samples, thus improving the recall rate of trademark detection. Brand recognition is completed through trademark feature library matching. Adding a new brand only requires feature extraction and mean normalization of the brand sample and appending the prototype features to the library. The entire process is completed in seconds, and the mechanism supports single-sample registration without retraining the detection model. Brand expansion time is reduced from hours to days to seconds, and the number of samples required for expansion is reduced from hundreds to thousands per brand to just 1, with 5 or more recommended, significantly reducing the cost of brand expansion.

[0022] 4. This invention uses a similarity threshold to determine the candidate recognition results using an open set. It automatically rejects unknown trademarks outside the image library and outputs unknown trademark markers. The similarity threshold can be adaptively determined on the validation set, taking into account both the acceptance of known brands and the rejection of unknown trademarks. The rejection rate of unknown trademarks on the test set reaches 91.98%, enabling the system to reject new brands that have not appeared in the training set. It can be applied to scenarios such as new brand discovery and infringement trademark monitoring. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram illustrating the steps of a trademark recognition method based on deep learning technology provided by this invention; Figure 2 This is a schematic diagram of the trademark detection model structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the feature alignment module structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the decoupled detection head and distance distribution decoding structure provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the trademark feature image library construction and open set recognition process provided in the embodiments of the present invention; Figure 6 A schematic diagram of a trademark recognition system based on deep learning technology provided by the present invention; The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0027] Please see Figure 1This invention provides a trademark recognition method based on deep learning technology, comprising the following steps: S1. For the image to be recognized, the trademark detection model detects trademarks as a single category, outputs the horizontal bounding box of the trademark, and removes redundant bounding boxes through non-maximum suppression. The trademark detection model includes a dual-branch backbone network consisting of parallel convolutional neural network branches and a self-supervised visual Transformer branch, a feature alignment module, a neck network, and a decoupled detection head. The convolutional neural network branch extracts local detail features, and the self-supervised visual Transformer branch extracts global semantic features. The local detail features and the global semantic features are projected to the same feature space by the feature alignment module, and their distribution alignment is explicitly supervised by a feature alignment loss in the form of mean square error between the projected features before fusion. The fused features are then processed by the neck network to extract multi-scale features, and the horizontal bounding box is output by the decoupled detection head. S2. After cropping the corresponding regions of each horizontal bounding box in the original image, input them into the self-supervised visual Transformer branch to extract depth features and perform L2 normalization. Calculate the cosine similarity with the prototype features of each brand in the trademark feature library, and take the brand with the highest similarity as the candidate recognition result. The prototype features are obtained by L2 normalizing the depth features of each trademark region, averaging them according to the brand category, and then L2 normalizing them again. S3. Compare the maximum similarity with a preset similarity threshold; if it is not less than the similarity threshold, output the corresponding brand name; if it is less than the similarity threshold, determine it as an unknown trademark outside the trademark feature library and output the unknown trademark mark.

[0028] The present invention will be described in detail below with reference to various embodiments: Example 1 Trademarks are crucial identifiers for distinguishing the source of goods or services. In scenarios such as commodity circulation supervision, brand protection, advertising content monitoring, and intellectual property infringement evidence collection, automatic location and recognition of trademarks appearing in images or videos are essential. Existing technologies typically construct trademark recognition as a closed-set classification task. The range of brands the model can recognize is fixed during the training phase. When new brands appear, it is necessary to re-collect samples, re-label, and retrain the entire model, resulting in high brand expansion costs. Multi-class detection schemes that treat each brand as an independent category are affected by the scarcity of long-tail brand samples, leading to a severe imbalance in the number of samples between categories, and lower accuracy for brands with fewer samples. Furthermore, traditional template matching methods using manually designed features have poor robustness under conditions of lighting changes, scale changes, partial occlusion, and appearance deformation.

[0029] To address the aforementioned issues, this embodiment decouples trademark positioning from brand recognition. The method first completes three preparatory steps: constructing the detection dataset, training the trademark detection model, and building the trademark feature library, before performing open-set recognition on the image to be identified.

[0030] Specifically, images containing trademarks are collected, and their locations are marked with horizontal bounding boxes. All trademarks from all brands are uniformly labeled as a single trademark category to construct a detection dataset, while retaining the brand category label for each trademark. A trademark detection model is trained based on the detection dataset, and the model detects trademarks as a single category. A trademark feature map library is constructed based on the brand category labels. During open-set recognition, the trademark detection model outputs the horizontal bounding boxes of the trademarks in the image to be recognized, and redundant bounding boxes are removed by non-maximum suppression. The regions corresponding to each horizontal bounding box are cropped from the original image and input into the same self-supervised visual Transformer branch to extract depth features, which are then L2 normalized. Cosine similarity is calculated with the prototype features of each brand in the trademark feature map library, and the brand with the highest similarity is selected as the candidate recognition result. The maximum similarity is compared with a preset similarity threshold. If it is not less than the threshold, the corresponding brand name is output; otherwise, it is determined to be an unknown trademark outside the trademark feature map library, and an unknown trademark mark is output. The following provides a detailed explanation of each step.

[0031] In the dataset construction phase, this embodiment collects an image dataset containing trademarks. The dataset uses the publicly available OpenLogo dataset, annotated in PASCAL VOC format. The annotation information includes the brand category name and horizontal bounding box coordinates. Since the detection process only needs to distinguish whether a trademark exists in the image, not the specific brand, this embodiment uniformly labels all brand trademarks as a single trademark category. That is, only the trademark location is labeled, without distinguishing the brand. This constructs the detection dataset and generates the label files required for training the trademark detection model. Specifically, the horizontal bounding box coordinates are converted into a normalized form represented by the center point coordinates and width and height. This allows trademark samples from all brands to participate in the training of the trademark detection model, avoiding the class imbalance problem caused by the scarcity of long-tail brand samples, which is beneficial to improving the recall rate of trademark detection. It also lays the foundation for expanding new brands in the future using a trademark feature map library. At the same time, the brand category labels of each trademark are retained for subsequent construction of the trademark feature map library. Approximately 27,000 images are randomly divided into training, validation, and test sets in an 8:1:1 ratio. The trademark detection model is trained only on the training set, the optimal weights are selected on the validation set, and the final performance is reported on the test set.

[0032] The following combination Figure 2The structure of the trademark detection model is explained. Trademark images contain both local details such as edges and textures, and global semantic information such as the overall brand layout and graphic structure. A single type of feature cannot simultaneously capture both. Therefore, the trademark detection model includes a dual-branch backbone network, a feature alignment module, a neck network, and a decoupled detection head. The dual-branch backbone network consists of parallel convolutional neural network branches and a self-supervised visual Transformer branch. The convolutional neural network branch extracts local details such as edges and textures, while the self-supervised visual Transformer branch extracts global semantic features such as the overall brand layout and graphic structure. The features from both branches are projected onto the same feature space by the feature alignment module and then fused. The neck network then extracts multi-scale features, and finally, the decoupled detection head outputs the horizontal bounding box of the trademark. The model input image size is 800×800 pixels.

[0033] The convolutional neural network branch adopts a spatial prior module structure, extracting shallow features at 1 / 8 resolution and mid-level features at 1 / 16 resolution through multi-level convolution and cross-stage partial connection modules, in order to preserve local details such as the edges and textures of the trademark. The layer-by-layer configuration is as follows: First, a Stem convolution with a 3×3 kernel and a stride of 2 transforms the 3-channel input to 1 / 2 resolution and 64 channels. Then, a Stem convolution with a 3×3 kernel and a stride of 1 maintains the 1 / 2 resolution and 64 channels. Next, a Stem convolution with a 3×3 kernel and a stride of 2 downsamples to 1 / 4 resolution and 128 channels. This is then passed through a cross-stage partial connection module containing two bottleneck blocks to obtain features at 1 / 4 resolution and 256 channels. A downsampled convolution with a 3×3 kernel and a stride of 2 obtains features at 1 / 8 resolution and 256 channels. This is then passed through a cross-stage partial connection module containing two bottleneck blocks to output shallow features C3 at 1 / 8 resolution. This is followed by a downsampled convolution with a 3×3 kernel and a stride of 2 to obtain features at 1 / 16 resolution and 512 channels. Finally, a cross-stage partial connection module containing two bottleneck blocks outputs mid-level features C4 at 1 / 16 resolution. All convolutional units are combinations of convolutional layers, batch normalization, and the SiLU activation function.

[0034] The internal structure of the cross-stage partial connection module is as follows: the input is first processed by a 1×1 convolution and output, and then divided into two sub-features. Then, n bottleneck blocks are applied to the last sub-feature in sequence. Each bottleneck block is two 3×3 convolutions with residual connections. Finally, the 2+n sub-features are concatenated in the channel dimension and output by a 1×1 convolution, realizing the multi-path gradient flow across stages.

[0035] Considering that supervised training relying on manually labeled data is limited by the scale and quality of the labeled data, resulting in a bottleneck in feature representation capabilities, the self-supervised visual Transformer branch in this embodiment adopts the DINOv3 ViT model, specifically the DINOv3-ViT-H / 16+, which has been pre-trained in a self-supervised manner on large-scale image data. The ViT-H / 16+ model used in the self-supervised visual Transformer branch is obtained by self-distillation from the ViT-7B teacher model, which has been pre-trained in a self-supervised manner on the LVD-1689M dataset (approximately 1.689 billion unlabeled images). The entire pre-training and distillation process does not rely on manual annotation, and rich general visual feature representations can be learned without brand-labeled data. This is particularly advantageous in long-tail brand recognition scenarios where training samples are scarce.

[0036] The self-supervised visual Transformer branch divides the input image into 16×16 pixel image blocks. After linear projection, the image blocks enter the multi-layer self-attention module. In the self-attention calculation, rotational position encoding is applied to the query matrix and key matrix according to the two-dimensional grid coordinates of the image blocks. The multi-layer self-attention module extracts global semantic features at a resolution of 1 / 16.

[0037] The self-attention weights are obtained by multiplying the dot product of the query matrix and the key matrix by the value matrix after scaling by the square root of the key feature dimension and normalizing by softmax, i.e., softmax(QK). T / √d k )V; where X is the feature matrix of the input image patch token sequence, n is the number of image patch tokens, d is the embedding dimension, and Q is the query matrix, which is equal to X and the learnable projection matrix W. Q The product of X and W, where K is the key matrix and equal to the product of X and W. K The product of X and W, where V is a value matrix and equal to the product of X and W. V The product of d k The key feature dimension is equal to the ratio of the embedding dimension to the number of attention heads. The attention weights are obtained by normalizing along the key dimension using softmax, with a scaling factor of 1 / √d. k This is used to prevent the gradient from vanishing due to an excessively large dot product. The actual implementation uses multi-head attention, which splits Q, K, and V into multiple heads for parallel computation, and the outputs of each head are concatenated before being linearly projected.

[0038] The self-supervised visual Transformer branch is configured with an image patch size of 16×16 pixels, an embedding dimension of 1280, 32 self-attention module layers, 20 attention heads (each with a dimension of 64), a feedforward network using a SwiGLU structure with a scaling ratio of 6.0, and a rotation position encoding fundamental frequency of 100. The token sequence consists of one class token, four register tokens, and an image patch token. The register tokens only participate in self-attention calculations and do not participate in the rearrangement of the spatial feature map. The self-supervised visual Transformer branch outputs the features of the last layer of image patch tokens after processing by the final normalization layer, with a resolution of 1 / 16 and 1280 channels.

[0039] The self-supervised visual Transformer branch does not employ learnable absolute position embedding. Instead, the rotation position encoding independently normalizes the row and column coordinates of the image patch before scaling them according to a preset ratio, calculating the rotation phase using single-precision floating-point. Therefore, the same model can directly infer at input sizes beyond the training resolution without interpolating the position encoding. During training, a partial fine-tuning strategy is employed, fine-tuning only the last four self-attention modules and the final normalization layer. All other parameters, including the image patch embedding layer, rotation position encoding, and the first 28 self-attention modules, are frozen. Furthermore, the learning rate of the self-supervised visual Transformer branch is set to 0.1 times that of the rest of the model, thus adapting to the trademark detection task while preserving the feature representation capabilities of the self-supervised pre-training.

[0040] The features of the convolutional neural network branch and the self-supervised visual Transformer branch belong to heterogeneous feature spaces, and direct concatenation and fusion are insufficient to eliminate the semantic gap between them. Therefore, this embodiment includes a feature alignment module, the structure of which is as follows: Figure 3 As shown, the feature alignment module projects the mid-layer feature C4 and the global semantic feature into the same feature space through 1×1 convolutions. Specifically, one side of the convolutional neural network branch uses 512-to-512 channel 1×1 convolutions with batch normalization, while the other side of the self-supervised vision Transformer branch uses 1280-to-512 channel 1×1 convolutions with batch normalization. Before projection, the global semantic feature is interpolated to the same spatial size as the mid-layer feature; in this embodiment, both have a resolution of 1 / 16.

[0041] Let p be the projected feature of the middle layer feature C4 after 1×1 convolution and batch normalization. cnn The global semantic features, after 1×1 convolution and batch normalization, result in projected features p. vitThe mean of the squared element-wise differences between two projected features is calculated as the feature alignment loss to guide the alignment of the feature distributions of the two branches. This feature alignment loss is weighted into the total training loss and participates in backpropagation, so that the feature alignment of the two branches is achieved under explicit supervision during the training phase, rather than relying solely on the implicit implementation of the network structure. This effectively reduces the semantic gap between heterogeneous feature spaces, enabling the fused features to possess both local detail perception and global semantic understanding capabilities, which is beneficial for improving the detection accuracy in complex backgrounds and small-sized trademark scenarios.

[0042] The two projected features are then concatenated along the channel dimension to obtain 1024-channel features, which are then fused to 512 channels via a 3×3 convolution. This fusion is further performed by the convolutional block attention module to obtain aligned features. The convolutional block attention module consists of channel attention and spatial attention connected in series, prioritizing channel attention over spatial attention. Features are adaptively weighted in both the channel and spatial dimensions, allowing the model to focus on more discriminative channels and regions. Channel attention performs global average pooling and global max pooling on the input, which is then processed by a multilayer perceptron consisting of two shared 1×1 convolutional layers. The channel reduction ratio is 16, with SiLU activation used in between. The two outputs are summed and then processed by a sigmoid function to obtain the channel weights. Spatial attention calculates the mean and maximum maps along the channel dimension and concatenates them to obtain 2 channels. This is then processed by a 7×7 convolution followed by a sigmoid function to obtain the spatial weights.

[0043] In real-world scenarios, trademarks vary greatly in size, with small-sized trademarks being common. A single-scale feature cannot fully cover targets of all sizes. To enhance the expressive power of multi-scale features, the neck network employs a structure combining a feature pyramid network and a path aggregation network. The feature pyramid network transmits deep semantic features to the shallow layers from top to bottom, generating feature maps P5, P4, and P3 at three resolutions: 1 / 32, 1 / 16, and 1 / 8. The path aggregation network transmits shallow features back from bottom to top, allowing shallow details to be incorporated into the deeper layers.

[0044] Specifically, P5 is obtained by applying global semantic features through a 3×3 convolutional kernel, learnable downsampling with a stride of 2, a cross-stage partial connection module, and a convolutional block attention module, with a resolution of 1 / 32 and 1280 channels. P4 is obtained by concatenating aligned features with the features upsampled from the nearest neighbors of P5 along the channel dimension, compressing 1792 channels to 512 channels through a 1×1 convolution, and then applying the cross-stage partial connection module and a convolutional block attention module, with a resolution of 1 / 16 and 512 channels. P3 is obtained by concatenating shallow features C3 with the features upsampled from the nearest neighbors of P4 along the channel dimension, compressing 768 channels to 256 channels through a 1×1 convolution, and then applying the cross-stage partial connection module and a convolutional block attention module, with a resolution of 1 / 8 and 256 channels.

[0045] In the path aggregation network, P3 is processed by a 3×3 convolutional kernel, a downsampling convolution with a stride of 2, and a cross-stage partial connection module, then added to P4 to obtain enhanced P4. Enhanced P4 is then processed by a 3×3 convolutional kernel, a downsampling convolution with a stride of 2, and a cross-stage partial connection module, then added to P5 to obtain enhanced P5. The convolutional block attention module is embedded in four places: after the fusion convolution of the feature alignment module, and after the fusion convolution of the three layers of the feature pyramid network (P5, P4, and P3). Paths in the path aggregation network are no longer embedded individually.

[0046] Considering the significant uncertainty in the horizontal bounding box positions of small-sized trademarks and trademarks with blurred boundaries and low contrast with the background, probabilistic modeling of the distance distribution can more precisely express this positional uncertainty compared to directly regressing a single distance value. Therefore, the decoupled detection head in this embodiment employs a distance distribution probabilistic decoding method, the structure of which is as follows: Figure 4 As shown. The decoupled detection head adopts an anchorless design, including independent bounding box regression branches and classification branches, which operate on feature maps at three scales: P3, P4, and P5, with corresponding strides of 8, 16, and 32 pixels, respectively. The anchor point is located at the center of each grid cell in the feature map at each scale, with a relative grid offset of 0.5.

[0047] The bounding box regression branch outputs the distance distribution from the anchor point to the left, top, right, and bottom edges of the trademark's horizontal bounding box. This distribution consists of 16 discrete probability values, with 4×16 output channels, or 64. These 16 discrete distance values ​​are fixed and uniformly discretized, representing an integer sequence from 0, 1, 2 to 15, in grid units. These are preset fixed values ​​and do not participate in network learning; the network only predicts the probability distribution above them. During decoding, the probability distribution is first normalized using softmax to obtain continuous distance values, which are then combined with the anchor point coordinates to decode the horizontal bounding box. Since the maximum distance in each of the four directions can be 15 grid units, and the width of the horizontal bounding box is the sum of the distances in the left and right directions, and the height is the sum of the distances in the top and bottom directions, the maximum horizontal bounding box size that can be expressed at each scale under this configuration is approximately 30 times the step size in pixels: approximately 240 pixels at scale P3, approximately 480 pixels at scale P4, and approximately 960 pixels at scale P5. This multi-scale collaborative approach covers trademarks of any size. The classification branch outputs the confidence score of a single trademark category, i.e., the probability that a trademark exists at the anchor position, with 1 output channel. Both branches have a structure of two levels of 3×3 convolution followed by 1×1 convolution, with the middle channel being one-quarter of the input channel and no less than 128; specifically, P3 and P4 are 128, and P5 is 320.

[0048] In the decoding calculation, let l, t, r, and b be the distances from the anchor point to the left, top, right, and bottom edges of the horizontal bounding box, respectively, and let ax and ay be the x and y coordinates of the anchor point. Then, the center coordinate cx of the horizontal bounding box is equal to ax plus (r... l) / 2, cy equals ay plus (b) t) / 2, width w equals l plus r, height h equals t plus b. Each distance is first calculated in the grid coordinate system, and then multiplied by the step size of the corresponding feature layer to convert it into pixel coordinates.

[0049] After determining the above structure, the training process of the trademark detection model is described below. To ensure that positive sample allocation takes into account both classification and localization quality, the trademark detection model adopts a task-aligned label allocation strategy to assign positive sample anchors to each real trademark box. The alignment metric m is the product of the α power of the classification score after sigmoid mapping and the β power of the intersection-union ratio of the anchor predicted box and the real box. In this embodiment, α is 0.5 and β is 6.0. α less than 1 can compress the difference in classification scores and avoid easily classified samples dominating the positive sample allocation, while β greater than 1 can amplify the contribution of localization quality, making positive samples biased towards anchors with high localization accuracy. Among the candidate anchors whose center falls within the real box, the top 10 are selected as positive samples according to the alignment metric. The classification training target, i.e., the soft label, is generated by normalizing the maximum value of the alignment metrics of each positive sample within the same real box.

[0050] The total training loss consists of a weighted sum of four components: bounding box regression loss based on perfect intersection-union ratio (MIRR), classification loss based on binary cross-entropy, distribution focus loss based on distance distribution, and feature alignment loss. The weights are 7.5, 0.5, 1.5, and 50, respectively. The distribution focus loss provides joint supervision between two adjacent discrete locations, which is beneficial for improving the localization accuracy of small-sized trademarks and trademarks with blurred boundaries and low contrast with the background. The MIRR loss has a relatively small magnitude, hence a large weight of 7.5. The first three weights follow the classic loss balancing configuration of the mature YOLOv8 detector series. The feature alignment loss, after batch normalization, has a mean squared error of approximately 0.01 to 0.001, and a weight of 50 is chosen to balance its gradient contribution with other loss terms. This value is determined by observing the magnitudes of each loss term on the validation set.

[0051] The training implementation details are as follows: Input images are scaled and padded proportionally to a uniform 800×800. The AdamW optimizer is used with an initial learning rate of 0.001 and weight decay of 0.0005. Cosine annealing is used for learning rate scheduling, with a minimum decay to 1% of the initial value. The batch size for each iteration is 16, with 8 images handled by each of two GPUs, and gradient accumulation is used to achieve an equivalent batch size of 64. Training is performed for a total of 30 epochs. The first 3 epochs are linear warm-up, and the last 5 epochs disable mosaic stitching and randomized augmentation to allow the model to converge on samples that closely approximate the real data distribution. The training is terminated early if the validation set metrics show no improvement for 15 consecutive epochs.

[0052] Data augmentation includes mosaic stitching, random blending, horizontal flipping, HSV color dithering, random translation, and random scaling. These augmentations improve the model's adaptability to complex imaging conditions. Specifically, mosaic stitching is applied with a probability of 1.0, random blending with a probability of 0.15, horizontal flipping with a probability of 0.5, HSV color dithering perturbing hue, saturation, and brightness by 0.015, 0.7, and 0.4 respectively, random translation by 0.1, and random scaling by 0.5. Mixed-precision training is employed, and the gradient norm is clipped to 5.0.

[0053] Example 2 After the trademark detection model is trained, a trademark feature map library can be built, and brand recognition can be performed on the detected trademarks. Since detection and recognition are decoupled, brand recognition is completed through feature map library matching; adding new brands does not require modification to the trademark detection model. This embodiment illustrates the construction of the trademark feature map library and the open set recognition process, such as... Figure 5 As shown.

[0054] The process of building a trademark feature library is as follows: Figure 5 The upper part is shown. The finely tuned self-supervised visual Transformer branch is loaded from the trained trademark detection model, set to inference mode, and frozen. Trademark regions are cropped from the training set images based on brand category labels, skipping regions with a width or height less than 32 pixels to avoid insufficient feature information in small regions affecting the quality of prototype features. The cropped regions are scaled to 256×256, normalized using ImageNet mean and standard deviation, and input into the self-supervised visual Transformer branch. The category token (CLS token) feature from the last output layer, processed by the final normalization layer, is taken as the depth feature of this sample, with a dimension of 1280, consistent with the embedding dimension of the self-supervised visual Transformer branch.

[0055] L2 normalization is applied to deep features. Then, the average of all sample features for each brand category is calculated, followed by another L2 normalization, yielding the prototype features for that brand. These prototype features from all brands constitute the trademark feature library. By averaging by brand, imaging differences between different samples of the same brand are fused and absorbed, allowing the prototype features to more stably represent the overall appearance of the brand. The library is organized by storing one prototype feature vector for each brand. In addition to the mapping between brands and prototype features, the library file also stores the source image identifiers for each sample of each brand, the number of samples for each brand, feature dimensions, and backbone network source information for traceability and consistency verification.

[0056] The image library construction and online recognition use the same fine-tuned self-supervised visual Transformer branch to extract features. This system directly loads the self-supervised visual Transformer branch from the trained trademark detection model, which ensures that the image library features and query features are distributed in the same way, avoiding the failure of similarity calculation due to inconsistent feature caliber in the two stages.

[0057] The trademark feature library supports incremental updates. Traditional closed-set classification schemes require re-collecting training samples, re-labeling, and retraining the entire model when adding a new brand, typically taking hours to days and requiring hundreds to thousands of labeled samples for each brand. In this embodiment, when adding a new brand, only cropping, feature extraction, and mean normalization of the brand's samples are performed, and the new prototype features are appended to the library. The entire process is completed in seconds, without recalculating the prototype features of existing brands or retraining the trademark detection model.

[0058] The mechanism supports single-sample registration. When a brand provides only one sample, its prototype feature becomes the normalized feature of that sample, allowing for immediate brand recognition without any model training. The robustness of the prototype feature is positively correlated with the number and diversity of samples. In practice, it is recommended to collect five or more samples covering different viewpoints, lighting, and backgrounds for each brand to obtain a more stable prototype. Deleting a brand or updating its prototype is merely a write operation to the image library entry and does not affect the trademark detection model.

[0059] For trademarks with multiple variations, various variant samples of the same brand, such as different colors and layouts, are all included in the calculation of the prototype mean for that brand. Each brand corresponds to one prototype feature, and the differences between variations are absorbed through feature fusion. When the appearance differences between variants of the same brand are too large, causing a decrease in the distinguishability of a single prototype, the brand can be split into multiple prototypes for registration. Each variant group corresponds to one prototype feature. During recognition, the cosine similarity between the candidate region features and each prototype of the brand is calculated, and the maximum value is taken as the matching similarity of the brand. This mechanism does not require modification to the trademark detection model.

[0060] Open set recognition process in Figure 5 The lower half is shown. For the image to be recognized, the trademark detection model detects trademarks as a single category, outputting the horizontal bounding boxes of the trademarks. Redundant bounding boxes are then removed using non-maximum suppression to prevent the same trademark from being detected repeatedly. During non-maximum suppression, candidate boxes with a classification confidence score below 0.25 are first eliminated, and then overlapping redundant boxes are suppressed using an intersection-over-union (IoU) threshold of 0.45. A maximum of 300 detection boxes are retained per image.

[0061] After cropping the regions corresponding to each horizontal bounding box in the original image, the data are input into the same self-supervised visual Transformer branch to extract depth features, which are then L2 normalized. Cosine similarity is calculated between these features and the prototype features of each brand in the trademark feature library. The brand with the highest similarity is selected as the candidate recognition result. Cosine similarity is calculated as the dot product of the depth feature vector f of the trademark region to be identified and the prototype feature vector p of the brand, divided by the product of their L2 norms. Since both are L2 normalized, cosine similarity equals the inner product of the two vectors, with a value ranging from -1 to 1. When the trademark to be identified does not belong to any brand in the library, its overall similarity with each prototype feature is low, making it clearly distinguishable from brands within the library.

[0062] To address this, this embodiment sets a similarity threshold for open-set judgment. The maximum similarity is compared with the preset similarity threshold. If the similarity is not less than the threshold, the corresponding brand name is output; otherwise, it is determined to be an unknown trademark outside the trademark feature library and an unknown trademark mark is output. The recognition result output module simultaneously overlays the detection box, recognition result, and similarity onto the original image and outputs a structured recognition result. Thus, the system possesses the ability to reject new brands not appearing in the training set. This capability can be applied to scenarios such as new brand discovery and infringing trademark monitoring.

[0063] The similarity threshold can be set manually or adaptively determined on the validation set. When adaptively determined, candidate thresholds are iterated within the range of 0.30 to 0.70 with a step size of 0.02. The correct recognition rate of known brands and the correct rejection rate of unknown trademarks outside the image library are calculated separately. The harmonic mean of these two rates is then calculated, and the candidate value with the largest harmonic mean is selected as the similarity threshold to balance the acceptance of known brands with the rejection of unknown trademarks. In this embodiment, a value of 0.40 is selected.

[0064] Example 3 This embodiment is a variant implementation when the training data size is small. When the training data size is small, such as about 1300 images, the model requires more training epochs to converge fully, and the augmentation strategy at the end of training has a greater impact on the convergence quality. Therefore, the number of training epochs can be increased to 100 epochs, the batch size per iteration can be adjusted to 8, and mosaic stitching and randomized mixed augmentation can be turned off in the last 10 epochs, so that the model can converge on samples that are close to the real data distribution. The remaining configuration is the same as in Embodiment 1.

[0065] Example 4 This embodiment provides a trademark recognition system based on deep learning technology to implement the trademark recognition methods described in Embodiments 1 and 2. Its overall architecture is as follows: Figure 6 As shown, the system includes a trademark detection model, a trademark recognition library, and a recognition result output module.

[0066] The trademark detection model is used to detect trademarks as a single category. It outputs horizontal bounding boxes of trademarks from the input image, and removes redundant bounding boxes through non-maximum suppression. The trademark detection model includes a dual-branch backbone network consisting of parallel convolutional neural network branches and a self-supervised visual Transformer branch, a feature alignment module, a neck network, and a decoupled detection head. Its specific structure and training method are as described in Example 1. Because the trademark detection model only detects a single trademark category without distinguishing specific brands, all brand samples participate in the training, avoiding the category imbalance problem caused by the scarcity of long-tail brand samples.

[0067] The trademark recognition image library is used to construct a trademark feature image library and perform brand recognition on detected trademark regions. During library construction, trademark regions are cropped from the training set images based on brand category labels. Depth features are extracted from the self-supervised visual Transformer branch. After L2 normalization of the depth features, the mean value is calculated according to the brand category, and then L2 normalized again to obtain the prototype features of each brand.

[0068] During recognition, the detected trademark regions are cropped from the original image and input into the same self-supervised visual Transformer branch to extract depth features, which are then L2 normalized. Cosine similarity is calculated between these features and the prototype features of each brand in the trademark feature library. The brand with the highest similarity is selected as the candidate recognition result. The specific process is as described in Example 2. Since the library construction and online recognition use the same self-supervised visual Transformer branch for feature extraction, the mechanism ensures that the library features and query features are distributed in the same way. When adding a new brand, only the prototype features of that brand need to be appended to the trademark feature library; there is no need to retrain the trademark detection model.

[0069] The recognition result output module is used to compare the maximum similarity of the candidate recognition results with a preset similarity threshold. If the similarity is not less than the threshold, the corresponding brand name is output; if it is less than the threshold, the unknown trademark mark is output. The detection box, recognition result and similarity can be superimposed on the original image to output a structured recognition result.

[0070] Before deploying the recognition, it is necessary to complete the construction of the detection dataset, the training of the trademark detection model, and the construction of the trademark feature library, as described in Examples 1 and 2.

[0071] Example 5 To verify the technical effectiveness of this invention, this embodiment reports the system's performance evaluation results. The dataset used is the OpenLogo public dataset, randomly divided into a training set of 21,666 images, a validation set of 2,708 images, and a test set of 2,709 images in an 8:1:1 ratio, with zero overlap between the test and training sets. The trademark feature image library is constructed from 1,280 brand-labeled images, covering 34 brands and a total of 1,802 trademark regions. The test set contains 5,012 labeled trademark frames, including 2,053 known brands and 2,959 unknown brands outside the image library, providing a sufficient statistical base for open-set rejection evaluation.

[0072] The recognition and rejection evaluations were conducted on a test subset of images excluding those from the image library. This subset contained 2579 images, including 1867 known brands and 2959 unknown brands with bounding boxes. Recognition accuracy was calculated based on the premise that the trademark was successfully detected (i.e., the intersection-union ratio of the detected bounding box to the ground truth bounding box was not less than 0.5). Therefore, the number of known brands and unknown brands included in the recognition statistics were 1730 and 2495, respectively. The average detection precision was evaluated on the complete test set of 2709 images.

[0073] The experimental environment consisted of an NVIDIA GeForce RTX 5090 graphics card with 32GB of VRAM, PyTorch 2.8.0 deep learning framework, and CUDA 12.8. The evaluation protocol was that a correct match was defined as a detection bounding box with an intersection-union ratio (IU) of at least 0.5 with the ground truth bounding box. The similarity threshold was adaptively determined on the validation set, and the final performance was reported on the test set.

[0074] The overall performance of the system on the test set is shown in Table 1.

[0075] Table 1 Overall System Performance To verify the advantages of this scheme over existing technologies, two sets of comparative experiments were set up. First, for the scheme based on supervised training convolutional neural network feature matching, the detector and detection boxes were kept identical, only the feature extractor was replaced with a ResNet50 pre-trained on ImageNet with a feature dimension of 2048, and this was used to reconstruct a prototype image library of the same brand. Recognition performance was compared under the same test images and the same similarity threshold, thus eliminating the interference of differences in detection boxes and separately measuring the impact of feature extraction methods on recognition performance. Second, the general single-branch detector YOLOv8m was selected, trained on the same data partition, and the detection accuracy was compared. The comparison results are shown in Table 2.

[0076] Table 2 Comparison with existing technologies As shown in Table 2, under fair conditions where the detection boxes are exactly the same, the Top-1 accuracy of this scheme for known brand recognition is 29.65 percentage points higher than that of the supervised convolutional neural network feature scheme, and the rejection rate for unknown trademarks is 47.21 percentage points higher. This indicates that the self-supervised visual Transformer features have a significant recognition advantage over supervised convolutional neural network features, which stems from the self-supervised pre-training mechanism used in feature extraction. Furthermore, the detection accuracy of this scheme is comparable to that of the general detector YOLOv8m, and it additionally possesses the ability to recognize open-set brands and reject unknown trademarks. The optimal weights for this scheme are selected based on mAP@0.5. Under a relaxed intersection-union ratio (IUU), mAP@0.5 is slightly better. YOLOv8m has more refined localization under a strict IUU, but it lacks the ability to recognize open-set brands and reject unknown trademarks.

[0077] To verify the technical contributions of each key module, under the premise of maintaining a training limit of 30 rounds, the same early stopping criteria, the same data partitioning and data augmentation, and an equivalent batch size of 64, specific modules were removed or replaced one by one for ablation experiments. The results are shown in Table 3.

[0078] Table 3 Ablation Experiment Results As shown in Table 3, regarding the dual-branch design, retaining only the convolutional neural network branch significantly reduced mAP@0.5 by 30.83 percentage points to 49.91%, while retaining only the self-supervised visual Transformer branch resulted in a 2.10 percentage point decrease. This indicates that both branches in the parallel structure are indispensable, and the self-supervised visual Transformer branch is the main contributor to detection capability. Regarding the feature alignment mechanism, removing only the explicit supervision of the alignment loss while keeping the network structure completely unchanged resulted in a 1.12 percentage point decrease in mAP@0.5. Further removing the entire alignment module resulted in a 2.19 percentage point decrease. These two levels of ablation demonstrate that the alignment contribution comes from both the explicit supervision in the training mechanism and the module structure itself, rather than simply structural stacking. The removal of the convolutional block attention module and the distributed focus loss supervision led to a 1.30 and 0.52 percentage point decrease in mAP@0.5, respectively, both positive contributions.

[0079] Except for the single-branch convolutional neural network variant, the Top-1 accuracy of all other variants remained within a narrow range of 93% to 95%, indicating that the open-set recognition capability mainly comes from the pre-trained features of the self-supervised visual Transformer itself. Feature alignment supervision acts on the feature fusion of the detection side, and the small fluctuations on the recognition side do not conflict with the ablation conclusions on the detection side. It should be noted that the optimal weights for each variant were selected according to mAP@0.5, and mAP@0.75 is not the modulus selection target. Therefore, the values ​​of some variants under strict intersection-union ratio may have normal fluctuations in a single run.

[0080] To verify the determination method and generalization of the similarity threshold, candidate thresholds in the range of 0.30 to 0.70 were traversed on the validation set with a step size of 0.02. The acceptance rate of known brands and the rejection rate of unknown trademarks were statistically analyzed under each threshold. The threshold with the largest harmonic mean of the two was selected as the working threshold. Some scanning results are shown in Table 4.

[0081] Table 4. Similarity threshold scanning results (validation set) As shown in Table 4, the harmonic mean reaches its maximum value of 0.8823 at 0.40, therefore, an adaptive similarity threshold of 0.40 is selected. Applying this threshold to the test set, the acceptance rate for known brands is 85.32%, the rejection rate for unknown trademarks is 91.98%, and the harmonic mean is 0.8853, consistent with the validation set results. This indicates that the threshold determination method has good generalization ability and no parameter tuning was required on the test set.

[0082] The above results demonstrate that the synergy between the dual-branch parallel backbone and the explicit supervision of feature alignment loss makes a decisive contribution to detection accuracy. The distance distribution probability decoding method improves the localization accuracy of small-sized trademarks and low-contrast trademarks. Self-supervised pre-trained features bring significant recognition advantages. Single-class detection avoids the class imbalance problem caused by the scarcity of long-tail brand samples. The similarity threshold rejection mechanism enables automatic rejection of unknown trademarks. The decoupled framework of detection and recognition allows new brands to be added only by adding prototypes to the image library without retraining the detection model.

[0083] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A trademark recognition method based on deep learning technology, characterized in that, Includes the following steps: S1. For the image to be recognized, the trademark detection model detects trademarks as a single category, outputs the horizontal bounding box of the trademark, and removes redundant bounding boxes through non-maximum suppression. The trademark detection model includes a dual-branch backbone network consisting of parallel convolutional neural network branches and a self-supervised visual Transformer branch, a feature alignment module, a neck network, and a decoupled detection head. The convolutional neural network branch extracts local detail features, and the self-supervised visual Transformer branch extracts global semantic features. The local detail features and the global semantic features are projected to the same feature space by the feature alignment module, and their distribution alignment is explicitly supervised by a feature alignment loss in the form of mean square error between the projected features before fusion. The fused features are then processed by the neck network to extract multi-scale features, and the horizontal bounding box is output by the decoupled detection head. S2. After cropping the corresponding regions of each horizontal bounding box in the original image, input them into the self-supervised visual Transformer branch to extract depth features and perform L2 normalization. Calculate the cosine similarity with the prototype features of each brand in the trademark feature library, and take the brand with the highest similarity as the candidate recognition result. The prototype features are obtained by L2 normalizing the depth features of each trademark region, averaging them according to the brand category, and then L2 normalizing them again. S3. Compare the maximum similarity with a preset similarity threshold; if it is not less than the similarity threshold, output the corresponding brand name; if it is less than the similarity threshold, determine it as an unknown trademark outside the trademark feature library and output the unknown trademark mark.

2. The trademark recognition method based on deep learning technology according to claim 1, characterized in that, Before detecting the image to be identified, the process also includes: acquiring images containing trademarks and marking the trademark locations with horizontal bounding boxes, uniformly labeling all brand trademarks as a single trademark category, constructing a detection dataset, while retaining the brand category labels of each trademark, and training the trademark detection model based on the detection dataset.

3. The trademark recognition method based on deep learning technology according to claim 2, characterized in that, When constructing the trademark feature library, each trademark region is cropped from the training set images according to the brand category label, and the depth features are extracted by the self-supervised visual Transformer branch. The depth features are then L2 normalized and mean-valued according to the brand category, and then L2 normalized again to obtain the prototype features of each brand. When expanding to new brands, it is only necessary to crop the trademark area of ​​the new brand's sample, input the self-supervised visual Transformer branch to extract the depth features and perform L2 normalization, calculate the mean by brand category and perform L2 normalization again to obtain its prototype features, and append them to the trademark feature library, without retraining the trademark detection model.

4. The trademark recognition method based on deep learning technology according to claim 1, characterized in that, The convolutional neural network branch extracts shallow features at 1 / 8 resolution and mid-level features at 1 / 16 resolution through multi-level convolution and cross-stage partial connection modules. The self-supervised visual Transformer branch divides the input image into image blocks, and after linear projection, extracts global semantic features at a resolution of 1 / 16 through a multi-layer self-attention module; wherein, rotational position encoding is applied to the query matrix and the key matrix in the self-attention calculation.

5. The trademark recognition method based on deep learning technology according to claim 1, characterized in that, The feature alignment module projects the features extracted by the convolutional neural network branch and the global semantic features into the same feature space through 1×1 convolution, and calculates the mean square error between the two projected features as the feature alignment loss to explicitly supervise the alignment of the feature distributions of the two branches. The two projection features are then concatenated along the channel dimension and fused with a convolutional block attention module to obtain aligned features. The total loss of the trademark detection model is composed of the box regression loss based on perfect intersection-union ratio, the classification loss based on binary cross-entropy, the distribution focus loss based on distance distribution, and the feature alignment loss, which are weighted and summed according to preset weights.

6. The trademark recognition method based on deep learning technology according to claim 4, characterized in that, When training the trademark detection model, the parameters of the image patch embedding layer, rotation position encoding, and the first few self-attention modules in the self-supervised visual Transformer branch are frozen, and only the parameters of the last few self-attention modules and the final normalization layer are fine-tuned. The learning rate of the self-supervised visual Transformer branch is 0.1 times that of the rest of the model.

7. A trademark recognition method based on deep learning technology according to claim 1, characterized in that, The decoupled detection head adopts an anchorless design, including independent bounding box regression and classification branches, which operate on feature maps at multiple scales respectively. The bounding box regression branch outputs a distance distribution consisting of multiple discrete probability values ​​for the distances of the left, top, right, and bottom edges of the horizontal bounding box located at the center of the grid cell of each scale feature map. The expected value of the probability distribution after softmax normalization is first obtained to obtain continuous distance values, and then the anchor point coordinates are combined to decode the horizontal bounding box of the trademark. The classification branch outputs the confidence level of a single trademark category.

8. A trademark recognition method based on deep learning technology according to claim 4, characterized in that, The neck network adopts a structure that combines a feature pyramid network and a path aggregation network. It generates feature maps of multiple scales from top to bottom, backpropagates the shallow features from bottom to top, and introduces convolutional block attention modules at each level of the feature pyramid network.

9. A trademark recognition method based on deep learning technology according to claim 1, characterized in that, The similarity threshold is adaptively determined by traversing candidate thresholds on the validation set, and the candidate threshold that best combines the correct recognition rate of known brands with the rejection rate of unknown trademarks is selected.

10. A trademark recognition system based on deep learning technology, used to execute a trademark recognition method based on deep learning technology as described in any one of claims 1 to 9, characterized in that, include: The trademark detection model includes a dual-branch backbone network consisting of parallel convolutional neural network branches and a self-supervised visual Transformer branch, a feature alignment module, a neck network, and a decoupled detection head. It is used to detect trademarks as a single category. The convolutional neural network branches extract local detail features, and the self-supervised visual Transformer branches extract global semantic features. The features from the two branches are projected onto the same feature space by the feature alignment module and then fused. The neck network extracts multi-scale features, and the decoupled detection head outputs the horizontal bounding box of the trademark. Redundant bounding boxes are removed by non-maximum suppression. The trademark recognition image library is used to crop trademark regions from training set images based on brand category labels, input them into the self-supervised visual Transformer branch to extract depth features, perform L2 normalization on the depth features, calculate the mean value according to brand category, and perform L2 normalization again to obtain the prototype features of each brand, thus forming the trademark feature image library; it is also used to crop the detected trademark regions from the original image, input them into the self-supervised visual Transformer branch to extract depth features and perform L2 normalization, calculate the cosine similarity with the prototype features of each brand in the trademark feature image library, and select the brand with the highest similarity as the candidate recognition result; The recognition result output module is used to compare the maximum similarity of the candidate recognition results with a preset similarity threshold; if it is not less than the similarity threshold, the corresponding brand name is output. If the similarity is less than the threshold, then an unknown trademark mark is output.

Images