Image recognition method, system, electronic device and storage medium

Abstract

The application discloses an image recognition method and system, an electronic device and a storage medium, and relates to the technical field of large models and image processing. The method comprises the following steps: acquiring a to-be-detected image, wherein the to-be-detected image comprises a to-be-recognized mark; performing instance segmentation on the to-be-detected image by using a target instance segmentation model to obtain a plurality of candidate instances; filtering the plurality of candidate instances according to a multi-dimensional filtering strategy to obtain a plurality of target instances; and performing mark recognition on the plurality of target instances based on a target image matching model to obtain a recognition result of the to-be-recognized mark. The application solves the technical problems of low recognition efficiency and poor accuracy in image mark recognition in the related art.

Description

Technical Field

[0001] This application relates to the fields of large model technology and image processing technology, and more specifically, to an image recognition method, system, electronic device and storage medium. Background Technology

[0002] In the digital age, logo detection and recognition technology has become a cornerstone of content security management, widely used in brand protection, misinformation tracking, and specific content screening. Related technologies primarily rely on target detection models for initial localization during logo detection and recognition, followed by classification models or image similarity matching techniques for precise identification. While this approach performs well in scenarios with abundant samples, it often struggles to accurately identify new logos lacking sufficient data. This necessitates significant time and resources for additional data annotation and model training, severely limiting response speed and applicability, and further reducing image recognition efficiency and accuracy.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides an image recognition method, system, electronic device, and storage medium to at least solve the technical problems of low recognition efficiency and poor accuracy in image mark recognition in related technologies.

[0005] According to one aspect of the embodiments of this application, an image recognition method is provided, comprising: acquiring an image to be detected, wherein the image to be detected includes: a mark to be identified; performing instance segmentation on the image to be detected using a target instance segmentation model to obtain multiple candidate instances; filtering the multiple candidate instances according to a multidimensional filtering strategy to obtain multiple target instances; and performing mark recognition on the multiple target instances based on a target image matching model to obtain a recognition result of the mark to be identified.

[0006] According to another aspect of the embodiments of this application, an image recognition method is also provided, comprising: acquiring an advertising brand image, wherein the advertising brand image includes: a brand logo to be identified; performing instance segmentation on the advertising brand image using a target instance segmentation model to obtain multiple candidate instances; filtering the multiple candidate instances according to a multi-dimensional filtering strategy to obtain multiple target instances; and performing logo recognition on the multiple target instances based on a target image matching model to obtain the recognition result of the brand logo to be identified.

[0007] According to another aspect of the embodiments of this application, an image recognition method is also provided, comprising: obtaining an image recognition request through a first application programming interface, wherein the request data carried in the image recognition request includes: an image to be detected, the image to be detected including: a mark to be recognized; and returning an image recognition response through a second application programming interface, wherein the response data carried in the image recognition response includes: the recognition result of the mark to be recognized, the recognition result being generated according to any one of the image recognition methods in the embodiments of this application.

[0008] According to another aspect of the embodiments of this application, an image recognition method is also provided, comprising: acquiring a currently input image recognition dialogue request, wherein the request data carried in the image recognition dialogue request includes: an image to be detected, the image to be detected including: a mark to be recognized; responding to the image recognition dialogue request, returning an image recognition dialogue response, wherein the information carried in the image recognition dialogue response includes: the recognition result of the mark to be recognized, the recognition result being generated according to any one of the image recognition methods in the embodiments of this application; and displaying the recognition result in a graphical user interface.

[0009] According to another aspect of the embodiments of this application, an image recognition method is also provided, comprising: displaying an image to be detected on the operation interface in response to an input command applied to the operation interface; and displaying a recognition result of a mark to be recognized on the operation interface in response to a processing command applied to the operation interface; wherein the image to be detected includes a mark to be recognized, and the recognition result is generated according to any one of the image recognition methods in the embodiments of this application.

[0010] According to another aspect of the embodiments of this application, an image recognition system is also provided, including: a client for sending an image to be detected, wherein the image to be detected includes: a mark to be recognized; a server connected to the client for generating a recognition result of the mark to be recognized according to any one of the image recognition methods in the embodiments of this application; the client is also used to output the recognition result.

[0011] According to another aspect of the embodiments of this application, a computing device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0012] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor connected to the memory via a bus for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0013] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0014] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0015] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0016] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0017] In this embodiment, an image to be detected containing a mark to be identified is acquired, and then a target instance segmentation model is used to segment the image to be detected to obtain multiple candidate instances. Subsequently, multiple candidate instances are filtered according to a multi-dimensional filtering strategy to obtain multiple target instances. Finally, the multiple target instances are identified based on a target image matching model to obtain the identification result of the mark to be identified. Thus, by combining instance segmentation, multi-dimensional filtering and target image matching model, rapid adaptation and accurate identification of the mark to be identified are achieved, while improving the identification efficiency and accuracy. This solves the technical problems of low identification efficiency and poor accuracy in image mark recognition in related technologies.

[0018] It is worth noting that the general description above and the detailed description that follow are merely for illustrative purposes and do not constitute a limitation on this application. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a schematic diagram illustrating an application scenario of an image recognition method according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of an image recognition method according to an embodiment of this application;

[0022] Figure 3This is a schematic diagram of an image recognition method according to an embodiment of this application;

[0023] Figure 4 This is a flowchart of another image recognition method according to an embodiment of this application;

[0024] Figure 5 This is a flowchart of another image recognition method according to an embodiment of this application;

[0025] Figure 6 This is a flowchart of another image recognition method according to an embodiment of this application;

[0026] Figure 7 This is a flowchart of another image recognition method according to an embodiment of this application;

[0027] Figure 8 This is a structural block diagram of an image recognition device according to an embodiment of this application;

[0028] Figure 9 This is a structural block diagram of another image recognition device according to an embodiment of this application;

[0029] Figure 10 This is a structural block diagram of another image recognition device according to an embodiment of this application;

[0030] Figure 11 This is a structural block diagram of another image recognition device according to an embodiment of this application;

[0031] Figure 12 This is a structural block diagram of another image recognition device according to an embodiment of this application;

[0032] Figure 13 This is a structural block diagram of a computing device according to an embodiment of this application;

[0033] Figure 14 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0035] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] The technical solution provided in this application is mainly implemented using large-scale model technology. Here, large-scale model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. Large-scale models can also be called foundation models. They are pre-trained on large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include large-scale language models (LLM) and multi-modal pre-training models.

[0037] It should be noted that, in practical applications, large models can be fine-tuned using a small number of samples to adapt them to different tasks. For example, large models can be widely used in Natural Language Processing (NLP), computer vision, and speech processing. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. Therefore, the main application scenarios for large models include, but are not limited to, digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design. In this embodiment, the data processing using a target instance segmentation model and a target image matching model in an image recognition scenario is used as an example for explanation.

[0038] According to an embodiment of this application, an image recognition method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] Considering the large number of model parameters in large models and the limited computing resources of mobile terminals, the method provided in this application embodiment can be applied to, for example, Figure 1 The application scenarios shown are not limited to these. In, for example... Figure 1 In the application scenario shown, the large model is deployed on server 10. Server 10 can connect to one or more client devices 20 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Client devices 20 can interact with users through a graphical user interface to access the large model, thereby implementing the method provided in this embodiment.

[0040] In this embodiment, the system consisting of a client device and a server can perform the following steps: the server obtains the currently input image recognition dialogue request from the client device. The request data carried in the image recognition dialogue request includes: an image to be detected, which includes: a mark to be recognized. In response to the image recognition dialogue request, the server returns an image recognition dialogue response to the client device. The information carried in the image recognition dialogue response includes: the recognition result of the mark to be recognized. The recognition result is generated according to the image recognition method of any one of the embodiments of this application. The client device can display the recognition result in a graphical user interface.

[0041] It should be noted that with the rapid development of high-performance computing units, the methods provided in this application embodiment can also be applied to model-in-machine systems in other application scenarios. In one optional embodiment, the model-in-machine system has multiple built-in models, and users can select one model to adjust as needed to obtain their own model. The high-performance computing unit built into the model-in-machine system can then directly call the adjusted model to execute the methods provided in this application embodiment. In another optional embodiment, the large model-in-machine system has a pre-trained model built-in, and the high-performance computing unit built into the model-in-machine system can then directly call that model to execute the methods provided in this application embodiment.

[0042] Furthermore, when users need to train their own models, they can upload their own datasets via the client. These datasets are then sent to the server, allowing the server to adjust the pre-trained model using the dataset to obtain the user's customized model, which can then be deployed to the production environment. To facilitate users' model adjustment needs, the server provides complete adjustment tools, development frameworks, and processes, supporting multiple adjustment strategies. This allows the adjusted model to better adapt to different application domains and achieve a high degree of customization.

[0043] Under the aforementioned operating environment, this application provides the following: Figure 2 The image recognition method shown. Figure 2 This is a flowchart of an image recognition method according to an embodiment of this application, such as... Figure 2 As shown, the method may include the following steps:

[0044] Step S21: Obtain the image to be detected, wherein the image to be detected includes: a mark to be identified;

[0045] Step S22: Use the target instance segmentation model to segment the image to be detected into multiple candidate instances;

[0046] Step S23: Filter multiple candidate instances according to the multi-dimensional filtering strategy to obtain multiple target instances;

[0047] Step S24: Based on the target image matching model, perform marker recognition on multiple target instances to obtain the recognition result of the marker to be recognized.

[0048] The images to be detected can originate from multiple sources, including the internet, social media, user uploads, and real-time video streams, and contain the identifier to be identified. Specifically, images containing the identifier can be downloaded automatically or semi-automatically from online resources such as web pages, forums, and blogs. Users can also submit images containing the identifier for personal queries, brand detection, etc., and keyframes containing the identifier can be extracted from real-time video streams or pre-recorded videos for detection.

[0049] The aforementioned marks to be identified are graphic or symbolic elements contained in the image to be inspected, whose organizational or product identity needs to be determined through image recognition. These marks can be known and widely recognized, or they can be newly emerging and uniquely designed. Accurate detection and identification of these marks are crucial in areas such as content security, brand monitoring, and advertising compliance, helping to identify potential infringements, monitor brand exposure, and screen for unauthorized use of brand resources. By rapidly and accurately identifying these marks in the image to be inspected, automated content security management and brand protection can be achieved.

[0050] The aforementioned target instance segmentation model is a pre-trained full instance segmentation network capable of segmenting instances in the image to be detected. Unlike traditional image classification or object detection tasks, instance segmentation not only identifies the presence and location of candidate instances in the image to be detected, but also accurately separates each candidate instance from the background and other candidate instances, generating a pixel-level mask corresponding to each candidate instance. The target instance segmentation model can accurately segment multiple candidate instances of different categories when handling complex scenes, while maintaining the clear boundaries and intact shape of each candidate instance.

[0051] Instance segmentation is a computer vision technique that aims to identify and segment each candidate instance in an image to be detected. Each candidate instance is assigned a classification label and a corresponding pixel-level mask representing its precise location and shape within the image. Instance segmentation provides finer-grained segmentation results, which is crucial for applications requiring precise understanding of image content, such as autonomous driving, medical image analysis, and sign detection and recognition.

[0052] The aforementioned candidate instances are image regions containing the target marker, initially identified and segmented by the target instance segmentation model. After processing the image to be detected, the target instance segmentation model generates a series of candidate instances, each containing a pixel mask and classification information of an independent target. Multiple candidate instances form the basis for subsequent more refined identification and analysis, such as further feature extraction, classification confirmation, or quality filtering. In the marker detection and recognition scenario, candidate instances are image regions that the target instance segmentation model initially determines may be the target marker. These image regions will undergo subsequent multi-dimensional filtering and image matching processes to determine whether they are truly the target marker.

[0053] Furthermore, multiple candidate instances are filtered according to a multidimensional filtering strategy to obtain multiple target instances. The multidimensional filtering strategy is a key step in selecting target instances from multiple candidate instances. By applying a series of predefined rules and conditions, non-target instances in multiple candidate instances are eliminated, thereby improving the efficiency of subsequent processing and reducing the possibility of misidentification.

[0054] For example, multidimensional filtering strategies include, but are not limited to: score threshold filtering, size filtering, position filtering, shape and proportion filtering, intersection over union (IOU) filtering, background analysis filtering, visual similarity filtering, and deep feature filtering.

[0055] Specifically, during score threshold filtering, the target instance segmentation model assigns a score to each candidate instance, representing the probability that the instance is a target instance. By setting a threshold, such as 0.5 or higher, candidate instances with scores below the threshold can be excluded to reduce noise. During size filtering, since the target sign typically has an inherent size range, candidate instances that are too small or too large may not be true target instances. Therefore, a size threshold can be set to retain candidate instances whose size is within a reasonable range. During location filtering, filtering is based on the possible locations of the target sign in the image to be detected, such as a target sign appearing in a specific area of ​​product packaging, the header or footer of a website, etc. By excluding candidate instances appearing in uncommon locations, false positives can be further reduced. During shape and scale filtering, for signs of known brands, shape and scale rules can be set to exclude candidate instances whose shapes do not conform or whose scales are distorted. During IOU filtering, when the bounding boxes of multiple candidate instances overlap, the intersection-union ratio (IU) between the multiple candidate instances is calculated. Candidate instances with high IU scores or those close to the expected shape are retained. In IOU filtering, the Intersection over Union (IoU) score measures the degree of overlap between two bounding boxes. Proximity to Expected Shape (PES) is an independent strategy, designed to retain candidate instances that more closely match the shape of the target sign. IoU focuses on positional and size consistency, while shape fit focuses on the similarity between the candidate instance and the outline of the target sign, together ensuring that the filtered candidate instances are both accurate in position and conform to the expected shape. Background analysis filtering can be performed by analyzing the background information around the candidate instance to exclude instances whose background does not match the sign's usage scenario; for example, signs typically do not appear in the center of natural landscapes. Visual similarity filtering involves a preliminary visual similarity comparison with a pre-defined sign registry to exclude candidate instances that clearly do not match known signs. Deep feature filtering utilizes deep learning models to extract high-level features of candidate instances, such as texture, color, and edges, and compares them with features of known signs for further filtering.

[0056] By comprehensively applying the above multi-dimensional filtering strategies, multiple target instances can be selected from multiple candidate instances. These target instances will then undergo a more detailed image matching and recognition process to ultimately determine whether they are the identifier to be identified. This filtering process can significantly reduce the computational load of subsequent processing, improve the speed and accuracy of recognition, and simultaneously reduce the false recognition rate.

[0057] Furthermore, based on the target image matching model, multiple target instances are subjected to marker recognition to obtain the recognition results of the markers to be recognized. During marker recognition, for each target instance that passes the multi-dimensional filtering strategy, the target image matching model can extract its features and perform matching analysis with each known marker in the preset marker registration library to obtain the recognition results of the markers to be recognized, thereby confirming whether the target instance is one of the registered markers.

[0058] Based on the above steps S21 to S24, by acquiring the image to be detected containing the mark to be identified, and then using the target instance segmentation model to segment the image to be detected to obtain multiple candidate instances, the multiple candidate instances are then filtered according to the multi-dimensional filtering strategy to obtain multiple target instances. Finally, the multiple target instances are identified based on the target image matching model to obtain the identification result of the mark to be identified. Thus, by combining instance segmentation, multi-dimensional filtering and target image matching model, the rapid adaptation and accurate identification of the mark to be identified are achieved, while improving the identification efficiency and accuracy. This solves the technical problems of low identification efficiency and poor accuracy in image mark recognition in related technologies.

[0059] The image recognition method in the embodiments of this application will be further described below.

[0060] In an optional embodiment, the image recognition method in this application further includes:

[0061] Obtain the image segmentation dataset, which includes: multiple sample images and the segmentation masks corresponding to the multiple sample images;

[0062] The initial instance segmentation model is trained using an image segmentation dataset to generate the target instance segmentation model.

[0063] Specifically, when acquiring the image segmentation dataset, a large number of sample images need to be collected. These images cover various scenes and targets, including but not limited to natural landscapes, urban environments, and indoor settings, ensuring the diversity and representativeness of the image segmentation dataset. For the sign detection and recognition scenario, images of signs from different brands and styles, as well as the presentation of signs against various backgrounds, should be included. Each sample image is meticulously annotated to generate a segmentation mask for each target. The segmentation mask is a binary image of the same size as the original image, where the pixel value of the target is marked as 1, and the pixel value of the background is 0. Furthermore, incorrectly labeled or poorly quality samples can be removed, and the image segmentation dataset can be divided into training, validation, and test sets according to a preset ratio to ensure the effectiveness of model training, performance evaluation, and final testing.

[0064] After obtaining the image segmentation dataset, a suitable neural network model is selected as the initial instance segmentation model. This begins with model initialization and loading pre-trained weights. These pre-trained weights are typically trained on large-scale datasets, providing rich low-level feature representation capabilities and accelerating subsequent fine-tuning. The image segmentation dataset is then loaded into the training framework, and key model parameters such as the loss function, batch size, and training epochs are set to guide the initial instance segmentation model's training process. Using the image segmentation dataset, the initial instance segmentation model learns how to assign pixels from different regions to the target instance or background category based on the segmentation mask. During training, the initial instance segmentation model continuously adjusts its parameters to minimize the loss function, gradually improving segmentation accuracy, ultimately resulting in the target instance segmentation model.

[0065] Based on the above optional embodiments, by acquiring an image segmentation dataset and then using the image segmentation dataset to train an initial instance segmentation model, a target instance segmentation model can be generated for instance segmentation of the image to be detected, thereby further improving the efficiency of marker recognition in the image.

[0066] In an optional embodiment, the target instance segmentation model includes: a fully convolutional encoding / decoding module, a multi-scale feature fusion module, and a dynamic kernel generation module. In step S22, the target instance segmentation model is used to segment the image to be detected, resulting in multiple candidate instances, including:

[0067] Step S221: Use a fully convolutional encoder-decoder module to extract features from the image to be detected, and obtain the feature extraction results;

[0068] Step S222: Use a multi-scale feature fusion module to perform feature fusion on the feature extraction results to obtain the instance localization result;

[0069] Step S223: Use the dynamic kernel generation module to perform mask prediction on the instance localization result to obtain the instance segmentation mask;

[0070] Step S224: Determine multiple candidate instances based on the instance segmentation mask.

[0071] The aforementioned fully convolutional encoding / decoding module is a convolutional neural network-based architecture designed to extract multi-level feature representations from images to be detected. The module consists of an encoder and a decoder. The encoder compresses image information and extracts high-level semantic features, while the decoder reconstructs these features, generating low-level, spatially relevant information. The target instance segmentation model avoids the spatial information loss caused by fully connected layers, thus allowing it to accept input images of arbitrary size and generate feature maps of corresponding sizes. The output of the fully convolutional encoding / decoding module is multi-level feature maps, i.e., feature extraction results, containing image information of different scales and details, providing a rich feature base for subsequent localization and segmentation.

[0072] The multi-scale feature fusion module described above is used to merge feature maps from different scales from the fully convolutional encoding and decoding module to construct a more comprehensive target representation. This module enables the target instance segmentation model to understand both global and local features of the target, improving segmentation accuracy. The fused feature maps can be used to locate each candidate instance in the image to be detected, i.e., to determine the approximate location and size of the instance, thus obtaining the instance localization result.

[0073] The aforementioned dynamic kernel generation module can dynamically adjust the convolutional kernel of each target based on the instance localization results to better fit the shape and edges of the target. This allows the target instance segmentation model to generate customized mask predictions based on the specific circumstances of each instance. The dynamic kernel generation module can generate an accurate instance segmentation mask, i.e., a binary image, for each target in the instance localization results, where 1 represents a target pixel and 0 represents a background pixel. The mask prediction results form the basis for accurate localization and marker recognition in subsequent steps.

[0074] Multiple candidate instances can be identified based on instance segmentation masks. These candidate instances are regions segmented from the image to be detected that are considered to potentially contain the marker to be identified. Candidate instances not only include visually coherent objects but also come with instance segmentation masks, facilitating subsequent multidimensional filtering and marker recognition.

[0075] Based on the above optional embodiments, a fully convolutional encoding / decoding module is used to extract features from the image to be detected, obtaining the feature extraction results. Then, a multi-scale feature fusion module is used to fuse the feature extraction results to obtain the instance localization results. Subsequently, a dynamic kernel generation module is used to predict the mask of the instance localization results to obtain the instance segmentation mask. Finally, multiple candidate instances are determined based on the instance segmentation mask. This enables instance segmentation in complex backgrounds, and can make correct judgments even when faced with new or slightly deformed signs. Through the combined use of the fully convolutional encoding / decoding module, the multi-scale feature fusion module, and the dynamic kernel generation module, the target instance segmentation model can efficiently process large-scale image data while maintaining high accuracy, thus significantly improving the practicality and user experience of sign detection and recognition.

[0076] In an optional embodiment, in step S23, filtering multiple candidate instances according to a multi-dimensional filtering strategy to obtain multiple target instances includes:

[0077] Step S231: Filter multiple candidate instances based on the confidence level of the marked region according to the threshold filtering strategy to obtain the first filtering result;

[0078] Step S232: Filter the multiple candidate instances by the marked region location according to the location filtering strategy to obtain the second filtering result;

[0079] Step S233: Perform duplicate detection filtering on multiple candidate instances based on the duplicate detection filtering strategy to obtain the third filtering result;

[0080] Step S234: Determine multiple target instances based on at least some of the filtering results from the first filtering result, the second filtering result, and the third filtering result.

[0081] Specifically, when filtering multiple candidate instances based on the threshold filtering strategy using the confidence score of the marked region, a threshold is set according to the confidence score of the marked region for each candidate instance output by the target instance segmentation model. This threshold excludes candidate instances whose confidence scores are below the threshold, thereby reducing the number of candidate instances processed subsequently and improving process efficiency. By comparing the confidence scores of the marked regions of all candidate instances with the threshold, candidate instances with confidence scores higher than the set threshold are retained, thus forming the first filtering result.

[0082] When filtering multiple candidate instances based on a location filtering strategy, the distribution of the target sign in the image to be detected is considered. For example, brand logos are usually located in specific positions on product packaging or in the central area of ​​an advertisement. Therefore, when filtering the location of the target sign, it is possible to check whether the position of the candidate instance conforms to common distribution locations. By eliminating candidate instances whose positions do not conform to common distribution locations of the target sign area, only candidate instances with reasonable positions are retained, thus forming the second filtering result, further improving the accuracy of detection and recognition.

[0083] When performing duplicate detection filtering on multiple candidate instances based on a duplicate detection filtering strategy, since the same target identifier may be identified multiple times, especially when there are multiple identical identifiers in the image to be detected, duplicate candidate instances can be detected and filtered according to certain merging or exclusion rules. By calculating the similarity between candidate instances, such as the intersection-union ratio, almost completely overlapping candidate instances can be merged, or only one representative candidate instance can be retained, thereby avoiding repeated calculations of identical identifiers in subsequent processing. The merged or selected candidate instances constitute the third filtering result.

[0084] After obtaining the first, second, and third filtering results, it is necessary to comprehensively consider these results to determine the final target instances. For example, candidate instances that simultaneously exist in the first, second, and third filtering results can be used as target instances; that is, candidate instances that simultaneously satisfy high confidence, reasonable location, and no duplicate detection can be retained as target instances.

[0085] Based on the above optional embodiments, a first filtering result is obtained by filtering multiple candidate instances based on the confidence of the marker region according to the threshold filtering strategy. Then, a second filtering result is obtained by filtering multiple candidate instances based on the location of the marker region according to the location filtering strategy. Subsequently, a third filtering result is obtained by filtering multiple candidate instances based on the same marker repetition detection according to the repetition detection filtering strategy. Finally, multiple target instances are determined based on at least some of the filtering results of the first filtering result, the second filtering result, and the third filtering result. This can effectively reduce false alarms in the marker recognition process, improve the recognition accuracy, and ensure the efficiency of the detection process.

[0086] In an optional embodiment, the target image matching model includes a visual encoding module and a semantic decoupling module. In step S24, based on the target image matching model, multiple target instances are identified to obtain the identification result of the identifier to be identified, including:

[0087] Step S241: Use the visual encoding module to perform visual feature encoding on multiple target instances to obtain the visual encoding result;

[0088] Step S242: Use the semantic decoupling module to perform dual-path feature decoupling on the visual encoding result to obtain the recognition result of the mark to be recognized.

[0089] The aforementioned visual encoding module is used to extract the visual features of target instances, thereby obtaining visual encoding results. Visual features include, but are not limited to, information such as color, shape, texture, and structure, which are the basis for recognizing the appearance of identifiers. Each target instance, after being processed by the visual encoding module, can be converted into a fixed-length visual encoding vector. The visual encoding vector contains the visual features of the target instance, and the visual encoding results will be used for subsequent comparison and recognition.

[0090] The aforementioned semantic decoupling module can decouple the visual encoding results through two-way feature decoupling, thereby decomposing the visual encoding results into symbolic semantic elements and artistic style elements. Symbolic semantic elements capture information related to the core elements of the brand identity, such as the special meaning of a particular logo to the brand; artistic style elements focus on non-core visual attributes, such as color and font style. After decoupling the visual encoding results through two-way feature decoupling, the recognition result of the logo to be identified is determined based on the symbolic semantic elements and artistic style elements.

[0091] Based on the above optional embodiments, by using a visual encoding module to perform visual feature encoding processing on multiple target instances to obtain visual encoding results, and then using a semantic decoupling module to perform dual-path feature decoupling on the visual encoding results to obtain the recognition result of the target identifier, the recognition result of the target identifier can be obtained from multiple target instances through the collaborative work of the visual encoding module and the semantic decoupling module, thereby further improving the recognition accuracy.

[0092] In an optional embodiment, in step S242, the semantic decoupling module performs dual-path feature decoupling on the visual encoding result to obtain the recognition result of the target symbol, including:

[0093] Step S2421: The semantic decoupling module is used to decouple the visual encoding results in two ways, and decompose the visual encoding results into symbolic semantic elements and artistic style elements. The symbolic semantic elements are used to determine the meaning conveyed by the logo to be identified, and the artistic style elements are used to determine the visual decorative effect of the logo to be identified.

[0094] Step S2422: Determine the recognition result of the mark to be identified based on the semantic elements and artistic style elements of the symbol.

[0095] Specifically, after the visual features of the target instance are encoded into visual encoding results, the semantic decoupling module is used to decouple the visual encoding results in two ways, thereby separating the symbolic semantic elements that represent the core information of the logo, as well as the artistic style elements that reflect the appearance style of the logo.

[0096] The aforementioned symbolic semantic elements focus on the most recognizable parts of a logo, such as the initials of the brand name, unique patterns, or shapes. These elements are key to understanding the meaning conveyed by the logo. In contrast, artistic style elements focus on the visual decorative effects of the logo, such as color, font, line thickness, and texture. While artistic style elements do not directly determine the semantics of a logo, they can help target instance segmentation models determine the presentation form and version of the logo, such as distinguishing between a white logo on a dark background and a black logo on a light background.

[0097] After separating the symbolic semantic elements and artistic style elements, a comprehensive judgment is made on the target instance based on these two elements. First, the meaning conveyed is determined by comparing the symbolic semantic elements of the target instance with those in a pre-defined logo registration database. Second, the visual effect of the logo is judged to match the records in the pre-defined logo registration database in conjunction with the artistic style elements. Based on the results of the comprehensive analysis, the recognition result of the logo to be identified is determined. If the symbolic semantic elements of the target instance are highly consistent with a known logo, and its artistic style elements also conform to the common expression of that brand, then the corresponding brand information can be output as the recognition result. If the symbolic semantic elements of the target instance are inconsistent with any known logo, or if the symbolic semantic elements of the target instance are highly consistent with a known logo but the artistic style elements do not conform to the common expression of that brand, the recognition result will indicate that the logo is an unknown brand or an unregistered brand logo.

[0098] By decoupling dual-path features, not only can the meaning of a logo be identified, but also visually similar but substantially different logos can be distinguished, such as differentiating the letter X in abstract art from the X logo of a cryptocurrency trading platform. This significantly improves the accuracy of recognition and reduces false alarms caused by visual similarity. The above recognition process is applicable to various complex image environments, including but not limited to multiple variations of brand logos, logo presentation under different design styles, and high-density logo scenes, ensuring accurate identification of brand affiliation even when there are subtle changes in the logo's appearance.

[0099] Based on the above optional embodiments, by using a semantic decoupling module to decouple the visual encoding results into dual-path features, the visual encoding results are decomposed into symbolic semantic elements and artistic style elements. Then, based on the symbolic semantic elements and artistic style elements, the recognition result of the sign to be recognized is determined, thereby further improving the accuracy of sign detection and recognition, and also enhancing its applicability and robustness in different scenarios.

[0100] In an optional embodiment, the image recognition method in this application further includes:

[0101] The matching results are obtained by searching for matching signs in the preset sign registration library using a feature search method. The preset sign registration library is used to store sign data that have been registered in advance.

[0102] The matching results are cross-validated using a visual multimodal model to obtain the target validation result.

[0103] The aforementioned pre-registered logo database stores a large amount of pre-registered logo data. Each logo can be encoded into a specific set of visual feature vectors during registration, including symbolic semantic elements and artistic style elements. These feature vectors constitute the core data of the pre-registered logo database.

[0104] After obtaining the recognition result of the target mark, a feature search method is used to search for matching marks in a pre-set mark registration database to obtain the matching result. The feature search process typically utilizes an efficient approximate nearest neighbor search algorithm, which calculates the similarity between the feature vector of the recognition result and the feature vectors of all marks in the pre-set mark registration database to obtain the matching result. The matching result usually includes the mark image, brand name, and related detailed information, such as the brand's history and mark usage guidelines.

[0105] To further verify the accuracy of the matching results, a visual multimodal model can be used to perform multidimensional cross-validation of the matching results. Based on the visual features of the matching results, the brand name, contextual information, etc., a multidimensional comparison and analysis can be performed to ensure the visual and semantic consistency between the logo to be identified and the matching logo.

[0106] Based on the above optional embodiments, matching signs of the recognition results are searched from the preset sign registration library by using feature search to obtain matching results. Then, the matching results are cross-validated in multiple dimensions using a visual multimodal model to obtain target verification results. This further ensures the accuracy of image sign recognition and avoids misidentification caused by visual similarity or contextual misleading.

[0107] In one optional embodiment, a visual multimodal model is used to perform multidimensional cross-validation on the matching results to obtain target validation results, including:

[0108] The visual multimodal model is used to perform visual and semantic correlation analysis on the matching results to obtain the analysis results;

[0109] A visual multimodal model is used to assess the contextual risks of the matching results, and the assessment results are obtained.

[0110] A visual multimodal model is used to detect style anomalies in the matching results, and the detection results are obtained.

[0111] The target verification results are determined based on at least some of the analysis results, judgment results, and detection results.

[0112] Specifically, in visual and semantic association analysis, the visual multimodal model can process image and text data in the matching results and map them to the same semantic space, thereby achieving cross-modal feature alignment. The visual multimodal model can not only understand the visual form of a logo but also analyze the semantic information behind it, such as brand name and philosophy. The visual features in the matching results are compared with preset brand semantic information to confirm whether the proportions, angles, etc., of the matched logo conform to brand specifications. If the similarity between the visual features of the matching results and the brand semantic information in the multimodal space is higher than a preset threshold, the matching results are considered to have passed the association analysis; otherwise, the matching results are considered to have failed the association analysis.

[0113] When conducting contextual risk assessment, the aim is to evaluate the compliance and security of the logo's usage scenarios and prevent potential risky behaviors. Visual multimodal models can analyze the contextual information of the image containing the logo in the matching results, including surrounding text, scene descriptions, and other visible symbols, to determine the reasonableness of the authorized use of the logo, thus obtaining the assessment result. Based on contextual analysis, it can be determined whether the scenario in which the logo is used is consistent with the brand's regular application scenarios. If inconsistencies or potential risks exist, the matching result needs to be marked as a case requiring further review in the assessment results.

[0114] Style anomaly detection aims to identify deviations in visual style from the matched logos, such as color changes, shape distortions, and the addition of extra graphic elements. A visual multimodal model is used to compare the style consistency of the matched results with logos in a pre-defined logo registration library, assessing whether the visual differences exceed the normal range of variation. Specifically, firstly, artistic style elements for each brand are extracted from the pre-defined logo registration library, including typical colors, textures, and design elements. Then, the style features of the matched results are compared with the brand style features in the pre-defined logo registration library in the model's semantic space, calculating the style distance between the two. If the style distance of the matched results exceeds a threshold of the normal range of variation, it is determined that the style is inconsistent, potentially involving variants or counterfeiting.

[0115] Furthermore, the final target verification result is determined by comprehensively considering the results of visual and semantic association analysis, contextual risk assessment, and style anomaly detection. Only when a tag instance passes the checks at multiple verification levels will it be confirmed as a correct recognition result.

[0116] Based on the above optional embodiments, the matching results are analyzed by visual and semantic association using a visual multimodal model to obtain analysis results. Then, the matching results are assessed for contextual risk using a visual multimodal model to obtain assessment results. Subsequently, the matching results are detected for style anomalies using a visual multimodal model to obtain detection results. Finally, the target verification results are determined based on at least some of the analysis results, assessment results, and detection results. This not only accurately identifies the identity of the identifier in the image but also assesses the risk of its usage scenario, ensuring the comprehensiveness and reliability of the recognition process.

[0117] In an optional embodiment, the image recognition method in this application further includes:

[0118] Iterative samples are determined based on the target verification results, where the iterative samples are markers of errors identified by the target image matching model.

[0119] The model parameters of the target image matching model are updated based on the iterative samples to generate the updated image matching model.

[0120] Specifically, the matching results are validated through visual and semantic association analysis, contextual risk assessment, and style anomaly detection using a visual multimodal model to obtain the target validation result. If the recognition result is found to be inconsistent with reality during the validation process, i.e., there are cases of misidentification or omission, the erroneous recognition result can be used as an iterative sample. Types of misidentification in the target validation result include, but are not limited to, inaccurate visual feature matching, semantic understanding deviation, contextual misunderstanding, or failure to correctly identify style anomalies.

[0121] Furthermore, during the update of the model parameters of the target image matching model, Low-Rank Adaptation (LoRA) is used to fine-tune only the model parameters. LoRA allows training a small number of newly added low-rank matrices on top of the pre-trained model. This lightweight update method can significantly reduce computational resources and update time, enabling the target image matching model to quickly adapt to new knowledge, such as newly emerging logo patterns or corrected misidentification cases.

[0122] Determined iterative samples are injected into the update process of the target image matching model as fine-tuning input. These iterative samples guide the target image matching model to learn how to correctly handle recognition tasks in specific scenarios. Guided by the iterative samples, the target image matching model can adjust its parameters through backpropagation to reduce similar errors in future recognition processes. This process involves adjusting weights, biases, or other parameters in the model structure to improve the target image matching model's sensitivity and recognition ability to specific features in the iterative samples. After updating the model parameters, an updated image matching model is generated. The updated model incorporates the results learned from the iterative samples and can more accurately identify similar or related markers, reducing false positives and false negatives.

[0123] Based on the above optional embodiments, by determining iterative samples based on the target verification results, and then updating the model parameters of the target image matching model according to the iterative samples, an updated image matching model is generated. This can significantly reduce the model iteration cycle and resource consumption, and further improve the efficiency of sign recognition.

[0124] In one optional embodiment, updating the model parameters of the target image matching model based on iterative samples to generate the updated image matching model includes:

[0125] Based on iterative samples, some layer parameters of the visual encoding module in the target image matching model are updated to generate the updated image matching model.

[0126] The aforementioned visual encoding module is responsible for extracting visual features from the image to be detected, while updates to some layer parameters are focused on layers that are particularly important for sign recognition. For example, only the last few layers of the Transformer module can be fine-tuned, as this part of the high-level network focuses on learning the sign-specific detail patterns, such as edge sharpness and shape topology. Alternatively, all layer parameters of the visual encoding module can be updated.

[0127] LoRA technology is used to fine-tune the selected visual coding module layer, thereby preserving the general visual understanding ability of the target image matching model, while specifically correcting the recognition ability for specific iterative samples, thus achieving lightweight and efficient model updates.

[0128] Based on the above optional embodiments, by updating some layer parameters of the visual coding module in the target image matching model according to iterative samples, an updated image matching model is generated. This enables dynamic learning and correction of misidentification situations, and continuously improves the detection capability for newly emerging or difficult-to-identify signs.

[0129] In an optional embodiment, the image recognition method in this application further includes:

[0130] A fine-tuning pipeline approach is used to perform hot knowledge updates on the preset tag registration library over a preset time dimension, resulting in an updated tag registration library.

[0131] Specifically, before implementing hot knowledge updates, an efficient and flexible fine-tuning pipeline needs to be designed to ensure that the image recognition system can quickly respond to newly added or changed logo data. The fine-tuning pipeline includes key steps such as data preprocessing, model fine-tuning, validation, and deployment to achieve an automated and standardized update process. The image recognition system can automatically collect new logo samples or updated logo information from various sources periodically or according to preset time dimensions (such as hourly, daily, weekly, etc.), specifically from brand official updates, market testing, user feedback, and other channels.

[0132] By fine-tuning the pipeline, the logo registration library can be hot-updated within a preset time frame. This means that the latest logo knowledge can be integrated into the preset logo registration library without downtime or long waiting times. The dynamic update mechanism ensures that the system can respond to logo data changes in minutes or even less, further improving the efficiency and flexibility of identifying new logos.

[0133] Figure 3 This is a schematic diagram of an image recognition method according to an embodiment of this application, such as... Figure 3 As shown, an image to be detected is acquired, which includes a marker to be identified. A target instance segmentation model is then used to segment the image, resulting in multiple candidate instances. Subsequently, a multi-dimensional filtering strategy is applied to filter these candidate instances, yielding multiple target instances. A threshold filtering strategy is used to filter the marker region confidence of these candidate instances, resulting in a first filtering result. A positional filtering strategy is used to filter the marker region position of these candidate instances, resulting in a second filtering result. A duplicate detection filtering strategy is used to filter the duplicate detection of the same marker in these candidate instances, resulting in a third filtering result. Multiple target instances are determined based on at least a portion of the first, second, and third filtering results. A target image matching model is used to identify the markers in these multiple target instances, resulting in the identification result of the marker to be identified. A feature search method is then used to search for matching markers from a pre-defined marker registration library, resulting in a matching result. The pre-defined marker registration library stores pre-registered marker data. A visual multimodal model is then used to perform multi-dimensional cross-validation on the matching result, resulting in a target verification result. Based on the target verification results, iterative samples are determined. These iterative samples are markers of errors identified by the target image matching model. The model parameters of the target image matching model are updated based on the iterative samples, generating an updated image matching model. A fine-tuning pipeline approach is used to perform knowledge-based hot updates on the preset marker registration library over a preset time dimension, resulting in the updated marker registration library.

[0134] Figure 4 This is a flowchart of another image recognition method according to an embodiment of this application, such as... Figure 4 As shown, the method may include the following steps:

[0135] Step S41: Obtain the advertising brand image, wherein the advertising brand image includes: the brand logo to be identified;

[0136] Step S42: Use the target instance segmentation model to segment the advertising brand image to obtain multiple candidate instances;

[0137] Step S43: Filter multiple candidate instances according to the multi-dimensional filtering strategy to obtain multiple target instances;

[0138] Step S44: Based on the target image matching model, perform logo recognition on multiple target instances to obtain the recognition result of the brand logo to be recognized.

[0139] Based on steps S41 to S44 above, by acquiring the advertising brand image, and then using a target instance segmentation model to segment the advertising brand image to obtain multiple candidate instances, the multiple candidate instances are then filtered according to a multi-dimensional filtering strategy to obtain multiple target instances. Finally, based on a target image matching model, the multiple target instances are used for logo recognition to obtain the recognition result of the brand logo to be recognized. Thus, by combining instance segmentation, multi-dimensional filtering, and target image matching model, rapid adaptation and accurate recognition of the brand logo to be recognized are achieved, while improving recognition efficiency and accuracy. This solves the technical problems of low recognition efficiency and poor accuracy in image logo recognition in related technologies.

[0140] Figure 5 This is a flowchart of another image recognition method according to an embodiment of this application, such as... Figure 5 As shown, the method may include the following steps:

[0141] Step S51: Obtain an image recognition request through the first application programming interface, wherein the request data carried in the image recognition request includes: an image to be detected, and the image to be detected includes: a mark to be recognized;

[0142] Step S52: Return an image recognition response through the second application programming interface. The response data carried in the image recognition response includes: the recognition result of the mark to be recognized, which is generated according to the image recognition method of any one of the embodiments of this application.

[0143] Based on the above steps S51 to S52, an image recognition request is obtained through the first application programming interface. The request data carried in the image recognition request includes: an image to be detected, which includes: a mark to be recognized. Then, an image recognition response is returned through the second application programming interface. The response data carried in the image recognition response includes: the recognition result of the mark to be recognized. The recognition result is generated according to any one of the image recognition methods in the embodiments of this application. Thus, by combining instance segmentation, multidimensional filtering and target image matching models, rapid adaptation and accurate recognition of the mark to be recognized are achieved, while improving recognition efficiency and accuracy. This solves the technical problems of low recognition efficiency and poor accuracy in image mark recognition in related technologies.

[0144] Figure 6 This is a flowchart of another image recognition method according to an embodiment of this application, such as... Figure 6 As shown, the method may include the following steps:

[0145] Step S61: Obtain the currently input image recognition dialogue request, wherein the request data carried in the image recognition dialogue request includes: the image to be detected, and the image to be detected includes: the symbol to be recognized;

[0146] Step S62: In response to the image recognition dialogue request, return an image recognition dialogue response, wherein the information carried in the image recognition dialogue response includes: the recognition result of the mark to be recognized, and the recognition result is generated according to any one of the image recognition methods in the embodiments of this application;

[0147] Step S63: Display the recognition results within the graphical user interface.

[0148] Based on steps S61 to S63 above, by acquiring the currently input image recognition dialogue request, the request data carried in the image recognition dialogue request includes: the image to be detected, the image to be detected includes: the mark to be recognized, and then responding to the image recognition dialogue request, returning an image recognition dialogue response, the information carried in the image recognition dialogue response includes: the recognition result of the mark to be recognized, the recognition result is generated according to any one of the image recognition methods in the embodiments of this application, and finally the recognition result is displayed in the graphical user interface. Thus, by combining instance segmentation, multidimensional filtering and target image matching model, rapid adaptation and accurate recognition of the mark to be recognized are achieved, while improving recognition efficiency and accuracy, thereby solving the technical problems of low recognition efficiency and poor accuracy in image mark recognition in related technologies.

[0149] Figure 7 This is a flowchart of another image recognition method according to an embodiment of this application, such as... Figure 7 As shown, the method may include the following steps:

[0150] Step S71: In response to the input command applied to the operation interface, display the image to be detected on the operation interface;

[0151] Step S72: In response to the processing command applied to the operation interface, the recognition result of the mark to be recognized is displayed on the operation interface; wherein, the image to be detected includes: the mark to be recognized, and the recognition result is generated according to any one of the image recognition methods in the embodiments of this application.

[0152] Based on the above steps S71 to S72, by responding to the input command applied to the operation interface, the image to be detected is displayed on the operation interface, and then by responding to the processing command applied to the operation interface, the recognition result of the mark to be identified is displayed on the operation interface. The image to be detected includes: the mark to be identified. The recognition result is generated according to any one of the image recognition methods in the embodiments of this application. Thus, by combining instance segmentation, multidimensional filtering and target image matching model, rapid adaptation and accurate recognition of the mark to be identified are achieved, while improving recognition efficiency and accuracy. This solves the technical problems of low recognition efficiency and poor accuracy in image mark recognition in related technologies.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0154] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0156] According to an embodiment of this application, an image recognition apparatus for implementing the above-described image recognition method is also provided. Figure 8 This is a structural block diagram of an image recognition device according to an embodiment of this application, such as... Figure 8 As shown, the device includes:

[0157] The acquisition module 801 is used to acquire the image to be detected, wherein the image to be detected includes: a mark to be identified;

[0158] The segmentation module 802 is used to perform instance segmentation on the image to be detected using a target instance segmentation model to obtain multiple candidate instances;

[0159] The filtering module 803 is used to filter multiple candidate instances according to a multi-dimensional filtering strategy to obtain multiple target instances;

[0160] The recognition module 804 is used to perform mark recognition on multiple target instances based on the target image matching model to obtain the recognition result of the mark to be recognized.

[0161] Optionally, the acquisition module 801 is further configured to acquire an image segmentation dataset, wherein the image segmentation dataset includes: multiple sample images and segmentation masks corresponding to the multiple sample images; the image recognition device further includes: a training module 805, configured to train an initial instance segmentation model using the image segmentation dataset to generate a target instance segmentation model.

[0162] Optionally, the segmentation module 802 is further configured to: use a fully convolutional encoding and decoding module to extract features from the image to be detected, and obtain feature extraction results; use a multi-scale feature fusion module to perform feature fusion on the feature extraction results, and obtain instance localization results; use a dynamic kernel generation module to perform mask prediction on the instance localization results, and obtain instance segmentation mask; and determine multiple candidate instances based on the instance segmentation mask.

[0163] Optionally, the filtering module 803 is further configured to: perform marker region confidence filtering on multiple candidate instances according to a threshold filtering strategy to obtain a first filtering result; perform marker region location filtering on multiple candidate instances according to a location filtering strategy to obtain a second filtering result; perform duplicate detection filtering on multiple candidate instances according to a duplicate detection filtering strategy to obtain a third filtering result; and determine multiple target instances based on at least some of the filtering results in the first filtering result, the second filtering result, and the third filtering result.

[0164] Optionally, the recognition module 804 is further configured to: use a visual encoding module to perform visual feature encoding processing on multiple target instances to obtain a visual encoding result; and use a semantic decoupling module to perform dual-path feature decoupling on the visual encoding result to obtain the recognition result of the mark to be recognized.

[0165] Optionally, the recognition module 804 is further configured to: use a semantic decoupling module to perform dual-path feature decoupling on the visual encoding result, decompose the visual encoding result into symbolic semantic elements and artistic style elements, wherein the symbolic semantic elements are used to determine the meaning conveyed by the sign to be recognized, and the artistic style elements are used to determine the visual decorative effect of the sign to be recognized; and determine the recognition result of the sign to be recognized based on the symbolic semantic elements and the artistic style elements.

[0166] Optionally, the image recognition device further includes: a search module 806, used to search for matching signs of the recognition result from a preset sign registration library using a feature search method to obtain a matching result, wherein the preset sign registration library is used to store sign data that has been registered in advance; and a verification module 807, used to perform multi-dimensional cross-verification of the matching result using a visual multimodal model to obtain a target verification result.

[0167] Optionally, the verification module 807 is further configured to: perform visual and semantic association analysis on the matching results using a visual multimodal model to obtain analysis results; perform contextual risk assessment on the matching results using a visual multimodal model to obtain assessment results; perform style anomaly detection on the matching results using a visual multimodal model to obtain detection results; and determine the target verification result based on at least some of the analysis results, assessment results, and detection results.

[0168] Optionally, the image recognition device further includes: a determination module 808, used to determine iterative samples based on the target verification result, wherein the iterative samples are signs of errors in the target image matching model; and an update module 809, used to update the model parameters of the target image matching model according to the iterative samples, and generate an updated image matching model.

[0169] Optionally, the update module 809 is also used to: update some layer parameters of the visual coding module in the target image matching model based on the iterative samples, and generate an updated image matching model.

[0170] Optionally, the update module 809 is also used to: perform a knowledge hot update of the preset tag registration library in a preset time dimension using a fine-tuning pipeline method to obtain the updated tag registration library.

[0171] It should be noted that the acquisition module 801, segmentation module 802, filtering module 803, and identification module 804 correspond to steps S21 to S24 in the above embodiments. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. These modules can also run as part of a device in the server provided in the above embodiments.

[0172] Figure 9 This is a structural block diagram of another image recognition device according to an embodiment of this application, such as... Figure 9 As shown, the device includes:

[0173] The acquisition module 901 is used to acquire the advertising brand image, wherein the advertising brand image includes: the brand logo to be identified;

[0174] The segmentation module 902 is used to segment the advertising brand image into multiple candidate instances using a target instance segmentation model.

[0175] The filtering module 903 is used to filter multiple candidate instances according to a multi-dimensional filtering strategy to obtain multiple target instances;

[0176] The recognition module 904 is used to perform logo recognition on multiple target instances based on the target image matching model to obtain the recognition result of the brand logo to be recognized.

[0177] It should be noted that the acquisition module 901, segmentation module 902, filtering module 903, and identification module 904 correspond to steps S41 to S44 in the above embodiments. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. These modules can also run as part of a device in the server provided in the above embodiments.

[0178] Figure 10 This is a structural block diagram of another image recognition device according to an embodiment of this application, such as... Figure 10 As shown, the device includes:

[0179] The acquisition module 1001 is used to acquire an image recognition request through a first application programming interface, wherein the request data carried in the image recognition request includes: an image to be detected, and the image to be detected includes: a mark to be recognized;

[0180] The return module 1002 is used to return an image recognition response through a second application programming interface, wherein the response data carried in the image recognition response includes: the recognition result of the mark to be recognized, and the recognition result is generated according to any one of the image recognition methods in the embodiments of this application.

[0181] It should be noted that the acquisition module 1001 and return module 1002 correspond to steps S51 to S52 in the above embodiments. The two modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. The above modules can also be part of a device and run in the server provided in the above embodiments.

[0182] Figure 11 This is a structural block diagram of another image recognition device according to an embodiment of this application, such as... Figure 11 As shown, the device includes:

[0183] The acquisition module 1101 is used to acquire the currently input image recognition dialogue request, wherein the request data carried in the image recognition dialogue request includes: the image to be detected, and the image to be detected includes: the symbol to be recognized;

[0184] Return module 1102 is used to respond to an image recognition dialogue request and return an image recognition dialogue response, wherein the information carried in the image recognition dialogue response includes: the recognition result of the mark to be recognized, and the recognition result is generated according to any one of the image recognition methods in the embodiments of this application;

[0185] The display module 1103 is used to display the recognition results within the graphical user interface.

[0186] It should be noted that the acquisition module 1101, return module 1102, and display module 1103 correspond to steps S6 to S63 in the above embodiments. The three modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware or software components stored in memory and processed by one or more processors. The above modules can also be part of the device and run in the server provided in the above embodiments.

[0187] Figure 12This is a structural block diagram of another image recognition device according to an embodiment of this application, such as... Figure 12 As shown, the device includes:

[0188] The first display module 1201 is used to respond to input commands applied to the operation interface and display the image to be detected on the operation interface;

[0189] The second display module 1202 is used to respond to processing instructions applied to the operation interface and display the recognition result of the mark to be recognized on the operation interface; wherein, the image to be detected includes: the mark to be recognized, and the recognition result is generated according to any one of the image recognition methods in the embodiments of this application.

[0190] It should be noted that the first display module 1201 and the second display module 1202 mentioned above correspond to steps S71 to S72 in the above embodiments. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also be part of a device and run in the server provided in the above embodiments.

[0191] It should be noted that the preferred embodiments involved in the above embodiments of this application are the same as the solutions, application scenarios and implementation processes provided in the above embodiments, but are not limited to the solutions provided in the above embodiments.

[0192] Embodiments of this application may provide an image recognition system, comprising: a client for sending an image to be detected, wherein the image to be detected includes: a mark to be recognized; a server connected to the client for generating a recognition result of the mark to be recognized according to any one of the image recognition methods in the embodiments of this application; the client is also used to output the recognition result.

[0193] Embodiments of this application may provide a computing device. Figure 13 This is a structural block diagram of a computing device according to an embodiment of this application. Figure 13 As shown, the computing device may include: one or more (only one is shown in the figure) processors 132, memory 134, memory controller, and peripheral interfaces.

[0194] The aforementioned computing device can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, PCs (Personal Computers), all-in-one model machines, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.

[0195] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing API calling capabilities, allowing models to be called into created applications through API interfaces, and providing application management tools for application management and monitoring.

[0196] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0197] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0198] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.

[0199] Embodiments of this application may provide an electronic device. Figure 14 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 14 As shown, the electronic device may include: an input / output device 142; a memory 144; and a processor 146, wherein the processor 146 is connected to the input / output device 142 and the memory 144 via a bus 148.

[0200] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the methods in the above embodiments. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0201] The processor can invoke an executable program stored in memory via a transmission device to execute the method described in any of the above embodiments.

[0202] It will be understood by those skilled in the art that the structure shown in the figure is merely illustrative, and the computing device may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. This figure does not limit the structure of the aforementioned computing device. For example, the computing device may include more or fewer components (such as network interfaces, display devices, etc.) than shown in the figure, or may have a different configuration than that shown in the figure.

[0203] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0204] Embodiments of this application also provide a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store program code executed by the method provided in the above embodiments.

[0205] Optionally, in this embodiment, the storage medium may be located in a computing device.

[0206] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, which, when the executable program is running, controls the device where the computer-readable storage medium is located to execute the method described in any of the above embodiments.

[0207] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods provided in the embodiments described above.

[0208] Embodiments of this application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which can be used to store a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0209] Embodiments of this application also provide a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, it implements the method provided in the above embodiments.

[0210] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0214] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0215] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image recognition method, characterized in that, include: Acquire an image to be detected, wherein the image to be detected includes: a marker to be identified; The target instance segmentation model is used to segment the image to be detected, resulting in multiple candidate instances; Multiple candidate instances are filtered according to a multi-dimensional filtering strategy to obtain multiple target instances; Based on the target image matching model, the multiple target instances are identified to obtain the identification result of the target to be identified.

2. The image recognition method according to claim 1, characterized in that, The image recognition method further includes: Obtain an image segmentation dataset, wherein the image segmentation dataset includes: multiple sample images and segmentation masks corresponding to the multiple sample images; The initial instance segmentation model is trained using the image segmentation dataset to generate the target instance segmentation model.

3. The image recognition method according to claim 1, characterized in that, The target instance segmentation model includes a fully convolutional encoding / decoding module, a multi-scale feature fusion module, and a dynamic kernel generation module. The target instance segmentation model is used to segment the image to be detected, resulting in the following candidate instances: The fully convolutional encoder-decoder module is used to extract features from the image to be detected, and the feature extraction results are obtained. The multi-scale feature fusion module is used to fuse the feature extraction results to obtain the instance localization result; The dynamic kernel generation module is used to perform mask prediction on the instance localization results to obtain the instance segmentation mask; The plurality of candidate instances are determined based on the instance segmentation mask.

4. The image recognition method according to claim 1, characterized in that, The multiple candidate instances are filtered according to the multidimensional filtering strategy to obtain the multiple target instances, including: The multiple candidate instances are filtered by the confidence level of the marked region according to the threshold filtering strategy to obtain the first filtering result; Based on the location filtering strategy, the multiple candidate instances are filtered by the marked region location to obtain a second filtering result; Based on the duplicate detection filtering strategy, the multiple candidate instances are subjected to duplicate detection filtering with the same flag to obtain a third filtering result; The plurality of target instances are determined based on at least a portion of the first filtering result, the second filtering result, and the third filtering result.

5. The image recognition method according to claim 1, characterized in that, The target image matching model includes a visual encoding module and a semantic decoupling module. Based on the target image matching model, the model performs marker recognition on the multiple target instances to obtain the recognition result of the marker to be recognized, including: The visual encoding module is used to perform visual feature encoding on the multiple target instances to obtain the visual encoding result; The semantic decoupling module is used to perform dual-path feature decoupling on the visual encoding result to obtain the recognition result of the target symbol.

6. The image recognition method according to claim 5, characterized in that, The semantic decoupling module is used to perform dual-path feature decoupling on the visual encoding result to obtain the recognition result of the target symbol, including: The semantic decoupling module is used to perform dual-path feature decoupling on the visual encoding result, decomposing the visual encoding result into symbolic semantic elements and artistic style elements. The symbolic semantic elements are used to determine the meaning conveyed by the sign to be identified, and the artistic style elements are used to determine the visual decorative effect of the sign to be identified. The recognition result of the mark to be identified is determined based on the symbol semantic elements and the artistic style elements.

7. The image recognition method according to claim 1, characterized in that, The image recognition method further includes: The matching flags of the recognition result are searched from the preset flag registration library using a feature search method to obtain the matching result. The preset flag registration library is used to store flag data that has been registered in advance. The matching results are cross-validated using a visual multimodal model to obtain the target validation result.

8. The image recognition method according to claim 7, characterized in that, The target verification result is obtained by performing multidimensional cross-validation on the matching result using the visual multimodal model, including: The visual multimodal model is used to perform visual and semantic correlation analysis on the matching results to obtain the analysis results; The visual multimodal model is used to perform contextual risk assessment on the matching results to obtain the assessment results; The visual multimodal model is used to detect style anomalies in the matching results to obtain the detection results; The target verification result is determined based on at least a portion of the analysis results, the judgment results, and the detection results.

9. The image recognition method according to claim 7, characterized in that, The image recognition method further includes: Based on the target verification results, iterative samples are determined, wherein the iterative samples are markers of errors identified by the target image matching model; The model parameters of the target image matching model are updated based on the iterative samples to generate an updated image matching model.

10. The image recognition method according to claim 9, characterized in that, The updated image matching model is generated by updating the model parameters of the target image matching model based on the iterative samples, including: Based on the iterative samples, some layer parameters of the visual encoding module in the target image matching model are updated to generate the updated image matching model.

11. The image recognition method according to claim 7, characterized in that, The image recognition method further includes: The preset flag registration library is updated by performing a knowledge hot update over a preset time dimension using a fine-tuning pipeline approach, resulting in an updated flag registration library.

12. An image recognition method, characterized in that, include: Obtain an advertising brand image, wherein the advertising brand image includes: a brand logo to be identified; The advertising brand image is segmented using a target instance segmentation model to obtain multiple candidate instances; Multiple candidate instances are filtered according to a multi-dimensional filtering strategy to obtain multiple target instances; Based on the target image matching model, the multiple target instances are identified to obtain the identification result of the brand logo to be identified.

13. An image recognition method, characterized in that, include: An image recognition request is obtained through a first application programming interface, wherein the request data carried in the image recognition request includes: an image to be detected, and the image to be detected includes: a mark to be recognized; The image recognition response is returned through the second application programming interface, wherein the response data carried in the image recognition response includes: the recognition result of the mark to be recognized, and the recognition result is generated according to the image recognition method according to any one of claims 1 to 11.

14. An image recognition method, characterized in that, include: Obtain the currently input image recognition dialogue request, wherein the request data carried in the image recognition dialogue request includes: the image to be detected, and the image to be detected includes: the symbol to be recognized; In response to the image recognition dialogue request, an image recognition dialogue response is returned, wherein the information carried in the image recognition dialogue response includes: the recognition result of the mark to be recognized, and the recognition result is generated according to the image recognition method according to any one of claims 1 to 11; The recognition results are displayed within a graphical user interface.

15. An image recognition method, characterized in that, include: In response to input commands applied to the operation interface, the image to be detected is displayed on the operation interface; In response to the processing instructions applied to the operation interface, the recognition result of the mark to be recognized is displayed on the operation interface; The image to be detected includes the mark to be identified, and the identification result is generated according to the image recognition method according to any one of claims 1 to 11.

16. An image recognition system, characterized in that, include: A client is used to send an image to be detected, wherein the image to be detected includes: a marker to be identified; The server, connected to the client, is used to generate the recognition result of the target mark according to the image recognition method according to any one of claims 1 to 11; The client is also used to output the recognition results.

17. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the image recognition method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the image recognition method according to any one of claims 1 to 15.

19. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the image recognition method according to any one of claims 1 to 15.

Images