Advertisement intelligent inspection method, device, equipment, medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides an advertisement intelligent inspection method, device and equipment, a medium and a program product, and the method comprises the steps: carrying out the multi-modal feature extraction of an advertisement periodical image, and obtaining a multi-modal feature; matching the multi-modal features of the advertisement on-periodical image with the multi-modal features of the advertisement mother set material to obtain an initial matching result; and under the condition that the confidence coefficient of the initial matching result is not enough, calling the multi-modal large model to carry out consistency verification, and generating a final matching conclusion according to a consistency verification result. According to the intelligent advertisement inspection method provided by the invention, the multi-modal feature extraction technology and the interference suppression technology are combined, the influence of environmental interference on feature matching is reduced, then the initial matching result is quickly obtained through multi-modal feature matching, the multi-modal large model is called to perform consistency verification when the confidence coefficient is insufficient, and the accuracy of advertisement inspection is improved. And a more accurate matching conclusion is generated, intelligence, precision and high efficiency of advertisement inspection are finally realized, and the manual intervention cost is remarkably reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an intelligent advertising inspection method, device, equipment, medium, and program product. Background Technology

[0002] Advertising inspection is a process in which advertisers or monitoring agencies systematically check the execution of placed advertisements through manual verification, technical monitoring, and other methods. The core is to ensure that the advertising is carried out as agreed.

[0003] Currently, the advertising inspection field mainly relies on manual methods. The main process involves providing clients with original advertising materials; inspectors taking on-site photos of the advertisements and categorizing them by building name; staff manually comparing each published image with multiple master materials to determine their attribution; and finally, a designated person delivering the matching results to the client. In practical applications, the large volume of inspection data makes manual comparison not only inefficient but also prone to misjudgments and omissions. Furthermore, it suffers from high costs and long turnaround times, making it difficult to meet the needs of large-scale advertising inspections. Summary of the Invention

[0004] This application provides an intelligent advertising inspection method, device, equipment, medium, and program product to solve the problems of low accuracy, low efficiency, and high cost of manual advertising inspection methods in the prior art.

[0005] Firstly, this application provides an intelligent advertising inspection method, including: Multimodal feature extraction is performed on the advertisement image to obtain multimodal features; wherein, the local feature features in the multimodal features are extracted based on the reflection region suppression strategy; the global visual features in the multimodal features are extracted based on the reflection region suppression strategy and / or illumination invariance processing; The multimodal features of the advertisement image are matched with the multimodal features of the advertisement master material to obtain an initial matching result; the multimodal features of the advertisement master material are obtained by pre-extracting multimodal features from the advertisement master material; If the confidence level of the initial matching result does not meet the preset confidence level requirement, a multimodal large model is invoked to verify the consistency between the advertisement image and the initial matching result, and a final matching conclusion is generated based on the consistency verification result.

[0006] In one embodiment, the reflection region suppression strategy includes a first suppression strategy; the first suppression strategy refers to applying a negative weight to the reflection region during the global visual feature extraction process; The global visual features were obtained through the following methods: The image of the advertisement is input into the visual feature extraction model, and the global visual features of the advertisement image are output by the visual feature extraction model. The visual feature extraction model is used for: Based on the first suppression strategy, the semantic information of the advertisement image is extracted layer by layer; After semantic information is extracted and a feature map is output at the intermediate feature layer, the feature map is fused with the reflection component of the advertisement image to obtain a fused feature map. The fused feature map is input into the subsequent feature layers of the intermediate feature layer for layer-by-layer semantic information extraction until the last feature layer is reached for semantic information extraction and output of the global visual features.

[0007] In one embodiment, the local feature features include local visual features, text features, and facial features; the reflection region suppression strategy includes a second suppression strategy, a third suppression strategy, and a fourth suppression strategy; the second suppression strategy refers to filtering key points and their descriptors in the reflection region during the local visual feature extraction process; the third suppression strategy refers to filtering text features in the reflection region during the text feature extraction process; and the fourth suppression strategy refers to filtering facial features in the reflection region during the facial feature extraction process. The local feature characteristics are obtained through the following methods: Based on the second suppression strategy, key points and their descriptors are extracted from the image of the advertisement to obtain local visual features; Based on the third suppression strategy, the text content in the advertisement image is extracted to obtain text features; Based on the fourth suppression strategy, facial information is extracted from the image published in the advertisement to obtain facial features.

[0008] In one embodiment, matching the multimodal features of the advertisement image with the multimodal features of the advertisement master material to obtain an initial matching result includes: For each of the modal features, the modal features of the advertisement image and the modal features of the advertisement master material are matched to obtain a matching score for the modal features; The matching scores of each modality feature are weighted and summed to obtain the comprehensive matching score between the advertisement image and the advertisement master material; The ad master creative with the highest overall match score is selected as the initial match result.

[0009] In one embodiment, if the confidence level of the initial matching result does not meet the preset confidence level requirement, a multimodal large model is invoked to perform consistency verification between the advertisement image and the initial matching result, and a final matching conclusion is generated based on the consistency verification result, including: If the confidence level of the initial matching result does not meet the preset confidence level requirement, the advertisement image and its multi-dimensional semantic information, the initial matching result and its multi-dimensional semantic information are input into the multimodal large model. The multimodal large model is used to perform semantic consistency verification on the advertisement image and the initial matching result to obtain the consistency verification result. Based on the consistency verification results, a final matching conclusion is generated.

[0010] In one embodiment, the advertisement image is obtained through the following method: Obtain images of the advertisement after it has been published; The image captured of the advertisement screen is cropped to obtain a foreground enhanced image; If a QR code area exists in the foreground enhancement image, the QR code area is whitened to obtain the advertisement image after interference removal.

[0011] In one embodiment, the intelligent advertising inspection method further includes: Obtain at least one target ad image that matches the target ad master material provided by the target customer; Based on a preset report template, the inspection data of the target advertisement images and media resource information are automatically integrated to generate an inspection report; After the inspection report is generated, it will be automatically delivered to the target customer.

[0012] Secondly, this application also provides an intelligent advertising inspection device, comprising: A multimodal feature extraction module is used to extract multimodal features from the advertisement image to obtain multimodal features; wherein, the local feature features in the multimodal features are extracted based on the reflection region suppression strategy; the global visual features in the multimodal features are extracted based on the reflection region suppression strategy and / or illumination invariance processing; The multimodal feature matching module is used to match the multimodal features of the advertisement image with the multimodal features of the advertisement master material to obtain an initial matching result; the multimodal features of the advertisement master material are obtained by pre-extracting multimodal features from the advertisement master material; The multimodal large model consistency verification module is used to call the multimodal large model to verify the consistency between the advertisement image and the initial matching result when the confidence of the initial matching result does not meet the preset confidence requirement, and generate the final matching conclusion based on the consistency verification result.

[0013] Thirdly, this application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of any of the above-described intelligent advertising inspection methods.

[0014] Fourthly, this application also provides a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described intelligent advertising inspection methods.

[0015] Fifthly, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by the processor, it implements the steps of any of the above-described intelligent advertising inspection methods.

[0016] The intelligent advertising inspection method, device, equipment, medium, and program products provided in this application effectively improve the accuracy of image feature extraction and reduce the impact of environmental interference on feature matching by using multimodal feature extraction technology combined with interference suppression techniques such as reflection region suppression strategy and illumination invariance processing. Then, the initial matching result is quickly obtained through multimodal feature matching. When the confidence of the matching result is insufficient, a multimodal large model is introduced as an arbitrator to verify the consistency between the initial matching result and the image of the advertisement, generating a more accurate matching conclusion. This not only ensures efficient inspection in conventional scenarios, but also solves the matching problem in complex scenarios through the semantic understanding capability of the multimodal large model. Ultimately, it realizes intelligent, precise, and efficient advertising inspection, and significantly reduces the cost of manual intervention. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts of the intelligent advertising inspection method provided in this application.

[0019] Figure 2 This is the second flowchart of the intelligent advertising inspection method provided in this application.

[0020] Figure 3 This is a structural schematic diagram of the intelligent advertising inspection device provided in this application.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein.

[0024] The following is combined Figures 1-4 This application describes the intelligent advertising inspection method, apparatus, equipment, media, and program products provided.

[0025] The intelligent advertising inspection method provided in this application embodiment can be implemented based on an intelligent advertising inspection device. Therefore, this application embodiment uses the intelligent advertising inspection device as the execution subject to describe the intelligent advertising inspection method.

[0026] Combination Figure 1 and Figure 2 , Figure 1 This is one of the flowcharts illustrating the intelligent advertising inspection method provided in this application. Figure 2 This is the second flowchart of the intelligent advertising inspection method provided in this application.

[0027] like Figure 1 As shown, the intelligent advertising inspection method includes the following steps: Step 101: Extract multimodal features from the advertisement image to obtain multimodal features; wherein, the local feature features in the multimodal features are extracted based on the reflection region suppression strategy; the global visual features in the multimodal features are extracted based on the reflection region suppression strategy and / or illumination invariance processing.

[0028] Specifically, this involves acquiring images of the advertisement after it has been displayed. These images include at least the advertisement content projected onto the screen, and may also include environmental information surrounding the screen. This environmental information is affected by various factors, such as shooting angle, lighting conditions, and reflections. Therefore, preprocessing of the advertisement images is necessary to actively eliminate environmental interference. The preprocessed images focus on the advertisement content on the screen, providing a clean image foundation for subsequent intelligent inspection of the advertisement.

[0029] After obtaining the pre-processed advertisement images, in order to accurately extract advertisement-related features from the complex images, multimodal feature extraction technology is adopted to extract multimodal features from the advertisement images, including local feature features and global visual features. Local feature features mainly focus on key elements in the advertisement, while global visual features focus more on the overall visual effect of the advertisement.

[0030] During the extraction process, local feature extraction is performed based on a reflection region suppression strategy. By identifying and suppressing reflection regions in the image, interference from reflections on local feature characteristics is avoided, resulting in more accurate and clearer local feature characteristics. For global visual features, extraction is performed simultaneously based on both the reflection region suppression strategy and illumination invariance processing. This suppresses the influence of reflection regions and eliminates the impact of different lighting conditions on image features, making the extracted global visual features more stable and reliable. Alternatively, only the influence of reflection regions can be suppressed, or only illumination invariance processing can be performed to extract global visual features; the choice can be flexible and tailored to the specific application scenario and requirements. Ultimately, the extracted multimodal features provide an accurate data foundation for subsequent intelligent advertising inspection.

[0031] Step 102: Match the multimodal features of the published advertisement image with the multimodal features of the advertisement master material to obtain the initial matching result.

[0032] Specifically, it should be noted that when clients have advertising needs, they provide the original master advertising materials to the advertising platform. The provided format can be images or videos; if it is a video, keyframes are extracted. Similarly, in order to accurately extract advertising-related features from complex images, multimodal feature extraction technology is used to extract multimodal features from the master advertising materials. The multimodal features corresponding to the master advertising materials are uniformly stored in the master material library, which facilitates subsequent multimodal feature matching with the images to be published in the advertisement.

[0033] During the matching process, an efficient feature matching algorithm is used to compare the multimodal features of the advertisement image with the multimodal features of the advertisement master material stored in the master material library one by one, and generate a matching score between the advertisement image and the advertisement master material.

[0034] Furthermore, the recall results are reordered based on the matching scores, and the top-K master ad creatives are used as the initial matching results, where K is a positive integer greater than 1, providing an important basis for subsequent analysis and judgment.

[0035] Step 103: If the confidence level of the initial matching result does not meet the preset confidence level requirement, call the multimodal large model to perform consistency verification on the advertisement image and the initial matching result, and generate the final matching conclusion based on the consistency verification result.

[0036] Specifically, the confidence level of the initial matching result is an important indicator for measuring the degree of matching between the image published in the advertisement and the master material of the advertisement. The matching score can be used as the confidence level of the matching result to reflect the degree of similarity between the two.

[0037] If the confidence level of the initial matching result meets the preset confidence level requirement, the initial matching result can be considered credible. The final matching conclusion is generated based on the initial matching result, which is the conclusion that the image in the advertisement "matches" the initial matching result.

[0038] If the confidence level of the initial matching result does not meet the preset confidence level requirement, it can be considered that the uncertainty of the initial matching result is high. At this time, the multimodal large model consistency verification process is automatically triggered. The multimodal large model has powerful image and text understanding capabilities and can jointly analyze image content and text semantics to perform multi-dimensional matching analysis on the advertisement image and the initial matching result, realize end-to-end consistent inference, and output the final matching conclusion, that is, output the conclusion that the advertisement image and the initial matching result are "matched", "not matched" or "require manual review".

[0039] In addition, after obtaining the advertising inspection results through the above-mentioned multimodal feature extraction and matching technology, the automated report generation technology can be used, combined with template engine and scripting program, to realize the automatic layout and report generation of mixed text and image content, supporting multiple output formats such as PPT and PDF, and improving the standardization and efficiency of information presentation.

[0040] Therefore, based on multimodal feature extraction, feature comprehensive matching, contextual understanding and intelligent decision-making, an end-to-end automated inspection analysis and report generation framework was constructed to achieve closed-loop processing from "image input" to "insight output", with intelligent processing throughout the process, greatly reducing manual intervention.

[0041] The intelligent advertising inspection method provided in this application effectively improves the accuracy of image feature extraction and reduces the impact of environmental interference on feature matching by combining multimodal feature extraction technology with interference suppression techniques such as reflection region suppression strategy and illumination invariance processing. Then, it quickly obtains the initial matching result through multimodal feature matching. When the confidence of the matching result is insufficient, a multimodal large model is introduced as an arbitrator to verify the consistency between the initial matching result and the published advertising image, generating a more accurate matching conclusion. This method not only ensures efficient inspection in conventional scenarios, but also solves the matching problem in complex scenarios through the semantic understanding capability of the multimodal large model. Ultimately, it realizes intelligent, precise and efficient advertising inspection, and significantly reduces the cost of manual intervention.

[0042] In one embodiment, before acquiring the advertisement image, the method further includes: Obtain images of the advertisement after it has been published; Multi-target detection is performed on the image captured of the advertisement to obtain the multi-target detection results.

[0043] Specifically, the process involves capturing images of the advertisement after it has been published, and then converting the color space of the image using a color space conversion tool. For example, OpenCV's cv2.cvtColor can be used instead of manual color space conversion, thus improving the efficiency of image processing.

[0044] In PyTorch, explicitly calling torch.tensor(...).contiguous() ensures data continuity and reduces the overhead of explicit copying.

[0045] In actual testing, the preprocessing time on the accelerated graphics card was reduced by about half, significantly improving the overall system response speed. The final output of the image before and after optimization was the same, but the processing efficiency was significantly improved, especially in high-concurrency scenarios.

[0046] After image optimization, object detection technology is used to perform multi-object detection on the advertising images, identify specific categories of objects in the images, accurately locate their positions, output their bounding boxes, and finally generate the detection results of these objects.

[0047] In multi-object detection, it is necessary to detect multiple objects such as advertising screens, QR codes, faces, and reflection regions. For advertising screen detection, a deep learning model (such as YOLOv11) is used to detect advertising screen regions in the image and obtain their bounding boxes. This process aims to separate valid content regions from invalid background. For QR code detection, a deep learning model is used to detect QR code regions in the image and obtain their bounding boxes. This process is a prerequisite for subsequent cleanup operations. For face detection, a dedicated face detection algorithm (such as MTCNN) is used to identify face regions in the image, providing localization for subsequent face feature extraction. For reflection region detection, for scenes that are prone to specular reflection, such as glass curtain walls and human shadows, a reflection structure perception technology is adopted. This involves vertically dividing the image into upper and lower parts, calculating structural similarity and edge direction consistency, using a lightweight convolutional neural network (such as SynNet) to predict reflection confidence, and outputting a reflection region mask. This supports the distinction between "real content" and "reflected content," and specifically models "reflections of surrounding buildings" and "reflections of the photographer / pedestrians" to avoid these reflections interfering with and being mistaken for advertising content.

[0048] Of course, under ideal shooting conditions, there might be no invalid backgrounds, reflections, QR code data interference, or light interference. In such cases, no preprocessing methods like active interference removal are needed. However, in most real-world shooting scenarios, images are often affected by these factors, which are difficult to avoid. Therefore, in practical applications, preprocessing images to eliminate or reduce these interferences is particularly important, effectively reducing missed and false detections.

[0049] This application embodiment uses multi-target detection technology to accurately locate and distinguish different types of targets such as advertising screens, QR codes, faces, and reflection areas, providing a data foundation for subsequent feature extraction and interference elimination.

[0050] In one embodiment, the advertisement image is obtained through the following method: Obtain images of the advertisement after it has been published; The image captured of the advertisement screen is cropped to obtain a foreground enhanced image; If a QR code area exists in the foreground enhancement image, the QR code area is whitened to obtain the advertisement image after interference removal.

[0051] Specifically, it involves acquiring images of the advertisement after it has been published and their multi-target detection results.

[0052] Based on the bounding box detected by the advertising screen, the advertising screen area is cropped from the image captured by the advertising screen, and the interference of complex background is removed to obtain a foreground enhanced image.

[0053] Advertising content often includes QR code information. These QR codes consist of alternating black and white rectangular modules, densely packed and with high contrast overlaid on local areas of an image. When QR codes are superimposed on an image, their regular geometric patterns disrupt the natural texture continuity of the original image. Simultaneously, the high contrast of the QR code masks weak signal features of the original image; for example, subtle shadows and color gradients may be obscured by the strong edges of the QR code, preventing algorithms from extracting true local information. In other words, the strongly structured pattern of the QR code interferes with the local texture and edges of the image, leading to the loss of crucial details during local feature extraction.

[0054] When a QR code region exists in the foreground enhancement image, the pixel values ​​of this region are uniformly set to white pixel values ​​([255,255,255]) based on the bounding box of the QR code detection. This can eliminate the interference of the high-frequency texture of the QR code on subsequent local feature extraction, prevent the generation of a large number of false key points, thereby improving the purity and reliability of local features at the source, and finally obtaining a pure advertisement image.

[0055] This application embodiment suppresses interference sources at the source by eliminating invalid backgrounds and whiteing out QR codes, which can effectively improve the quality of advertisement images and more accurately reflect the advertisement placement status. This provides reliable data support for subsequent feature extraction and matching analysis, thereby significantly reducing the mismatch rate in complex scenarios.

[0056] In one embodiment, based on step 101, the reflection region suppression strategy includes a first suppression strategy; the first suppression strategy refers to applying a negative weight to the reflection region during the global visual feature extraction process; The global visual features were obtained through the following methods: The image of the advertisement is input into the visual feature extraction model, and the global visual features of the advertisement image are output by the visual feature extraction model. The visual feature extraction model is used for: Based on the first suppression strategy, the semantic information of the advertisement image is extracted layer by layer; After semantic information is extracted and a feature map is output at the intermediate feature layer, the feature map is fused with the reflection component of the advertisement image to obtain a fused feature map. The fused feature map is input into the subsequent feature layers of the intermediate feature layer for layer-by-layer semantic information extraction until the last feature layer is reached for semantic information extraction and output of the global visual features.

[0057] Specifically, the image of the advertisement is input into the visual feature extraction model, which maps the entire image into fixed-dimensional global visual features. To address the semantic shift caused by strong light and glass reflections in the image, an illumination-invariant feature enhancement network is introduced into the visual feature extraction model.

[0058] To mitigate semantic shift caused by strong light or glass reflection, the model structure adopts a dual-branch decoupled feature extraction architecture, which includes a main branch and an auxiliary branch.

[0059] The advertisement image is input into the main branch, where a pre-trained deep model backbone network (such as the DINOv2 backbone network) is used to extract multi-layer semantic information, obtaining feature maps for each feature layer. Furthermore, during the extraction of semantic information at each feature layer, a negative weight suppression strategy is applied to the reflection region. This negative weight suppression strategy effectively reduces the interference of the reflection region on the overall semantic understanding, improving the accuracy of feature extraction.

[0060] The advertisement image is input into the auxiliary branch, where a physical imaging model (e.g., Retinex) is used to decompose the image into two components: a reflection component and an illumination component. The reflection component reflects the material properties and color information of the object's surface and possesses strong semantic stability; the illumination component characterizes the intensity and direction of ambient light and is easily affected by external conditions. The decomposition formula is as follows: I(x) = R(x) × L(x) Where x represents a pixel, I represents the input image (i.e., the image in the advertisement), R represents the reflection component, and L represents the illumination component.

[0061] The reflection component, decoupled from the illumination component, is unaffected by changes in illumination. Integrating it into the semantic information extraction process can effectively improve the accuracy of semantic information extraction and reduce semantic shifts caused by changes in illumination.

[0062] To fuse the semantic information of the reflection components, a feature fusion module is introduced into the intermediate feature layer of the backbone network. The specific layer number of this intermediate feature layer is not limited here and can be set according to the actual network structure of the model; for example, for the DINOv2 network card, layer 6 can be set as the intermediate feature layer. First, the reflection component R is encoded using a lightweight convolutional neural network to generate the corresponding reflection feature map FR. Then, the feature map F output from the intermediate feature layer of the backbone network... mainA Convolutional Block Attention Module (CBAM) is introduced between the FR and the feature map. The attention mechanism enables the weighted fusion of the reflection feature map in both channel and spatial dimensions to obtain the attention feature map. Finally, the feature map output by the intermediate feature layer is fused with the attention feature map to obtain the fused feature map.

[0063] The calculation process of the fused feature map is as follows: F fused =F main +CBAM(FR) Among them, F fused F represents the fused feature map. main FR represents the feature map output by the intermediate feature layer.

[0064] The fused feature map F fused The input continues to be fed into subsequent network layers for high-level semantic modeling until the final feature layer is reached for semantic information extraction, at which point the global visual features are output.

[0065] By introducing an illumination-invariant feature enhancement network, we can enhance only the semantic region responses related to the intrinsic properties of the object, effectively suppressing aberrant activation caused by illumination abrupt changes or reflective regions, thereby improving the illumination invariance of features.

[0066] During model training, illumination invariance constraints can be employed. To ensure semantic consistency of the model under different illumination conditions, an illumination invariance loss function is introduced. During the training phase, pairs of identical objects under different illuminations are constructed to achieve data augmentation.

[0067] The illumination invariance loss function is as follows: in, Represents the illumination invariance loss function. and These represent sample images of the same semantic content under different lighting conditions. and sample images Extracted feature vectors, express The square of the norm is used to measure the Euclidean distance between two eigenvectors.

[0068] The overall loss function of the model is as follows: in, Indicates the total loss. Indicates comparative loss, Represents classification loss. This indicates the loss of illumination invariance.

[0069] Experimental results show that, on a test set that includes verification of glass reflection and strong light interference, this design significantly improves the mAP of DINOv2, outperforming the baseline model.

[0070] In one embodiment, based on step 101, the local feature features include local visual features, text features, and facial features; the reflection region suppression strategy includes a second suppression strategy, a third suppression strategy, and a fourth suppression strategy; the second suppression strategy refers to filtering key points and their descriptors in the reflection region during the local visual feature extraction process; the third suppression strategy refers to filtering text features in the reflection region during the text feature extraction process; and the fourth suppression strategy refers to filtering facial features in the reflection region during the facial feature extraction process. The local feature characteristics are obtained through the following methods: Based on the second suppression strategy, key points and their descriptors are extracted from the image of the advertisement to obtain local visual features; Based on the third suppression strategy, the text content in the advertisement image is extracted to obtain text features; Based on the fourth suppression strategy, facial information is extracted from the image published in the advertisement to obtain facial features.

[0071] Specifically, local feature extraction is performed on the images in the advertisement, including local visual features, text features, and facial features.

[0072] In the process of local visual feature extraction, key point detection algorithms (such as the DISK algorithm) are used to extract key points and their descriptions in the advertisement image. During feature extraction, the calculation of key points and their descriptors in these areas is skipped directly, and key point detection and descriptor generation are only performed on non-reflection areas, thereby obtaining pure local visual features.

[0073] In the text feature extraction process, the text region in the advertisement image is first located using image recognition algorithms (such as optical character recognition technology). Then, combined with the position information of the reflection region, the text content belonging to the reflection region is excluded, and only the effective text region is used for feature extraction, thereby obtaining accurate text features.

[0074] In the process of facial feature extraction, a face detection algorithm (such as the FaceNet model) is used to find the location of faces in the advertisement image. Then, based on the reflection area information, faces in the reflection area are filtered out, and only the real faces are used for feature extraction, thereby obtaining reliable facial features.

[0075] The two embodiments described above introduce illumination invariance processing into global feature extraction, constructing an end-to-end, trainable closed-loop architecture of "preprocessing—feature extraction—fusion decision." This not only improves the semantic stability of single-modal images but also provides a high-quality, interference-resistant visual representation foundation for subsequent multimodal branches. Simultaneously, combined with a reflection region suppression mechanism, it achieves collaborative suppression and dynamic fusion of multimodal features. The overall system demonstrates significantly better matching accuracy and robustness than the general model direct stacking scheme in complex real-world scenarios.

[0076] In one embodiment, based on step 102, matching the multimodal features of the advertisement image with the multimodal features of the advertisement master material to obtain an initial matching result includes: For each of the modal features, the modal features of the advertisement image and the modal features of the advertisement master material are matched to obtain a matching score for the modal features; The matching scores of each modality feature are weighted and summed to obtain the comprehensive matching score between the advertisement image and the advertisement master material; The ad master creative with the highest overall match score is selected as the initial match result.

[0077] Specifically, for each modal feature, such as global visual features, local visual features, facial features, and text features, an efficient feature matching algorithm is adopted. For example, a specific similarity measurement method is used to calculate the similarity between each modal feature of the advertisement image and the corresponding modal feature in the advertisement master material, and use it as the matching score of that modal feature.

[0078] Among them, global visual feature matching can quickly locate the similarity of the overall layout of the advertisement, local visual feature matching can accurately identify the positional deviation of key elements, facial feature matching can verify the consistency of the advertising spokesperson, and text feature matching can ensure the accuracy of the advertising slogan and brand information.

[0079] Furthermore, a weighted fusion strategy is employed to calculate the comprehensive matching score between the published image and the master advertising material. The weight values ​​can be dynamically adjusted according to the actual application scenario. The matching scores of each modal feature are weighted and summed according to their corresponding weight values ​​to obtain the comprehensive matching score between the two, which can comprehensively reflect the degree of similarity between them.

[0080] Using the above method, we can obtain the comprehensive matching score between the image in the advertisement and each master material in the master material library. Based on the size of these comprehensive matching scores, the recall results are reordered, and the TOP-1 master material is used as the initial matching result.

[0081] This application's embodiments introduce a multimodal feature matching mechanism, which can not only efficiently obtain matching results to meet the needs of large-scale advertising inspection, but also comprehensively evaluate advertisements from multiple dimensions. This effectively solves the problem of insufficient matching accuracy caused by relying on only a single visual feature in traditional advertising inspection, and significantly enhances robustness in complex scenarios.

[0082] In one embodiment, based on step 103, when the confidence level of the initial matching result does not meet the preset confidence level requirement, a multimodal large model is invoked to perform consistency verification between the advertisement image and the initial matching result, and a final matching conclusion is generated based on the consistency verification result, including: If the confidence level of the initial matching result does not meet the preset confidence level requirement, the advertisement image and its multi-dimensional semantic information, the initial matching result and its multi-dimensional semantic information are input into the multimodal large model. The multimodal large model is used to perform semantic consistency verification on the advertisement image and the initial matching result to obtain the consistency verification result. Based on the consistency verification results, a final matching conclusion is generated.

[0083] Specifically, in practical applications, a pre-set reliability threshold can be set according to specific needs and scenarios. When the confidence of the initial matching result is lower than the pre-set reliability threshold, it is considered that it does not meet the pre-set reliability requirements. Alternatively, a pre-set similarity threshold can be set according to specific needs and scenarios. If the difference in similarity scores between the TOP-1 and TOP-2 ad master materials in the initial matching results is less than the pre-set similarity threshold, it is considered that it does not meet the pre-set reliability requirements. In both cases, the system can determine that the matching result has high uncertainty and low confidence, and there is a risk of misjudgment. In this case, the semantic-level consistency verification process of the multimodal large model is automatically triggered. In particular, if the current ad instance is marked by the system as a high-value, high-risk category (such as well-known brand collaboration ads, ads in key target areas, etc.), even if the system determines that the matching result has high credibility, the semantic-level consistency verification process of the multimodal large model can be automatically triggered to ensure the accuracy and comprehensiveness of the matching, and to avoid significant losses or adverse effects caused by misjudgment of high-value, high-risk ads.

[0084] To achieve fine-grained, cross-modal semantic consistency judgment, a multimodal large model is introduced as the core arbitration module. For example, the Qwen2.5-VL-7B-Instruct model is adopted, which has powerful image and text understanding capabilities and can jointly analyze image content and text semantics to achieve end-to-end semantic consistency reasoning.

[0085] The following three types of information are jointly encoded and input into the multimodal large model to form a comprehensive judgment basis: 1. Images used in advertisements; 2. Initial matching results (Top-1 ad master materials matched from the master material library); 3. Structured, multi-dimensional semantic information, including: (1) Image features: Global visual features: The overall semantic representation extracted by a pre-trained visual feature extraction model, reflecting macroscopic information such as image theme, style, and color distribution; Local visual features: Focus on the detailed features of key areas, such as the text area, the face of a person, and the brand logo area, for refined comparison; (2) Text information: Image-to-text recognition results, including confidence levels; Associate metadata text, such as ad titles, descriptions, tags, etc.; Text semantic embedding vectors are used to measure semantic rather than literal similarity; (3) Structural information: The spatial layout of image elements, including the position coordinates, size ratio, and hierarchical relationship of each component; Layout similarity scoring, such as algorithms based on Bounding Box IoU or graph structure alignment; (4) Brand element characteristics: The test results of the brand logo and the consistency of its position, size, and color; The degree of matching with the brand's proprietary color system; Brand visual language characteristics such as font style and typography standards.

[0086] The above information is input into the multimodal large model in a multimodal fusion format, supporting joint input of multiple modalities such as raw image pixels, feature vectors, and structured text. The multimodal large model acts as a "semantic arbitrator," analyzing the semantic consistency between two images in terms of image features, text content, people, brand elements, and overall layout, and outputs a judgment of "match," "mismatch," or "requires manual review." This judgment is used to finally confirm or correct the machine matching result, solving the problem of "similar text content but semantic discrepancies" that traditional methods cannot handle.

[0087] This application introduces a multimodal large model as a semantic arbitrator to perform semantic-level reasonableness judgment on machine matching results, realizing a technical leap from "visual matching" to "semantic verification". The multimodal large model can identify and correct errors such as "text fine-tuning" and "character changes" that result in "similar in form but not in spirit", realize dynamic calibration of matching results, and solve the fundamental technical problem that "visual similarity is not equivalent to content compliance". The credibility of matching results has reached a new level.

[0088] In one embodiment, after the inspection and matching of the advertisement images are achieved in the above manner, the intelligent advertisement inspection method further includes: Obtain at least one target ad image that matches the target ad master material provided by the target customer; Based on a preset report template, the inspection data of the target advertisement images and media resource information are automatically integrated to generate an inspection report; After the inspection report is generated, it will be automatically delivered to the target customer.

[0089] Specifically, based on the master data provided by a client, the media locations where the client needs to publish ads can be identified. These media locations can then be located, and actual photos of the ads published at these locations (i.e., the images of the aforementioned ad scenes) can be obtained. All the aforementioned implementation methods are used to inspect and match each actual photo of the ad publication. After final confirmation of the ad publication matching results, based on a preset report template, the inspection data of the actual photos of the ad publication (or ad publication images) and media resource information are automatically integrated to generate a standardized and deliverable inspection report. This process uses a template-driven, data-filled automated generation mechanism, supporting output of PPT (.pptx) or PDF (.pdf) format reports. The report file automatically generates a unique filename according to the naming rules of the ad master material, ensuring the traceability and standardized management of the report.

[0090] For the design of the preset report template, the system supports customers to customize the report template. The template file is, for example, a standard PPT format, which includes the following key design elements: (1) Master page design: Define the overall style, including theme color, font style, header and footer, company logo position and other visual elements; (2) Slide layout: Preset multiple layout types, such as cover page, table of contents page, graphic display page, summary page, etc.; (3) Placeholder mechanism: Set structured placeholders in the template for dynamic content injection, such as: image placeholder (such as "{{ad_image}}"), used to insert real photos of advertisements; text placeholder (such as "{{building_name}}"), used to fill in structured fields such as building name; (4) Naming rule binding: Supports combining master data fields such as project number, media batch, and regional code to automatically generate standardized report file names according to naming rules, realizing intelligent naming and unified management of template-related files.

[0091] For the report generation process: (1) Template loading: The system reads the PPT template file specified by the customer, parses its slide structure, layout and placeholder configuration, and builds the template metadata model. (2) Report object initialization: A new PPT document object is created (based on Apache POI or python-pptx document processing library, etc.), and the style information such as master slide, theme, and layout in the template is copied to the new document to ensure consistent output style. (3) Dynamic content injection: Based on the inspection data of the completed advertisement placement real shot images, the template content is filled item by item. For example: Image insertion: The advertisement placement real shot images in the matching results are inserted into the image placeholder area according to the specified size and position, and automatic scaling and center alignment are supported; Text filling: The structured fields (such as building name, location number, media type, matching score, inspection time, etc.) are replaced with the corresponding text placeholders; Multi-page generation logic: If a single project contains multiple media locations, multiple graphic pages can be automatically created in a loop, and each page displays the image and attribute information of one location to realize batch content generation. (4) Format conversion and export: After generating a complete PPT document, the system calls the document conversion service to convert it to PDF format (optional); it supports outputting both PPT and PDF versions at the same time, or outputting only one version according to the configuration; the output file is automatically saved to the specified file path or cloud storage directory according to the preset naming rules, and a downloadable link is generated.

[0092] It should be noted that the dynamic content injection part prioritizes inserting successfully matched images and filling in the remaining content; secondly, it selects whether to insert images that require manual review based on the actual situation. If inserted, it will also fill in the remaining content and mark the relevant information that requires manual review so that the customer can make their own judgment; while images that fail to match are not inserted.

[0093] After the inspection report is generated, the delivery process is automatically triggered. Notifications can be sent to customers or downstream systems via email, API callbacks, or message queues, along with a report download link, enabling unattended delivery.

[0094] This application embodiment deeply integrates high-precision matching with automated report generation. The generated report can be directly delivered to the customer, realizing full-process automation from image input to report output. It completely eliminates the dependence on manual review and typesetting, and the degree of automation and efficiency far exceeds the existing technology, while significantly reducing the cost of manual intervention.

[0095] In summary, this application constructs an end-to-end intelligent inspection mechanism from image uploading to inspection report generation, achieving a high recognition rate and low false recognition rate on customer datasets. Furthermore, this application constructs a system-level innovative architecture with synergistic effects, namely "perception-purification-fusion-verification." This architecture is not a simple aggregation of processes, but rather forms a positive feedback loop of "interference elimination → feature optimization → matching improvement → semantic arbitration," with overall technical performance far exceeding the expectations of simply combining individual modules.

[0096] The following test results for different schemes illustrate the high accuracy of this application: Option 1: No QR code or reflection interference removal was performed: 25,000 test samples, of which 7,056 samples had a master copy, 30 samples were missed, and 216 samples were falsely detected. Option 2: Eliminate interference such as QR codes and reflections: 25,000 test samples, including 7,056 samples with master images, 1 missed test sample, and 1 false positive test sample. Analysis revealed that the missed detections were all composed of reflected images, while the false positives had similar background colors and their local feature matching points mostly fell on the QR code. By whitewashing the QR code area and eliminating the reflected areas, an extremely high recognition rate and an extremely low false positive rate were achieved on the client's dataset. Since the final output is a PPT or similar document, manual intervention was minimized, aiming for an accuracy rate as close to 100% as possible in terms of missed and false detections. The existence of missed and false detections would incur some manual intervention costs.

[0097] Figure 3 This is a structural schematic diagram of the intelligent advertising inspection device provided in this application.

[0098] like Figure 3 As shown, the intelligent advertising inspection device includes: The multimodal feature extraction module 310 is used to extract multimodal features from the advertisement image to obtain multimodal features; wherein, the local feature features in the multimodal features are extracted based on the reflection region suppression strategy; the global visual features in the multimodal features are extracted based on the reflection region suppression strategy and / or illumination invariance processing; The multimodal feature matching module 320 is used to match the multimodal features of the advertisement image with the multimodal features of the advertisement master material to obtain an initial matching result; the multimodal features of the advertisement master material are obtained by pre-extracting multimodal features from the advertisement master material; The multimodal large model consistency verification module 330 is used to call the multimodal large model to verify the consistency between the advertisement image and the initial matching result when the confidence of the initial matching result does not meet the preset confidence requirement, and generate the final matching conclusion based on the consistency verification result.

[0099] The intelligent advertising inspection device provided in this application effectively improves the accuracy of image feature extraction and reduces the impact of environmental interference on feature matching by using multimodal feature extraction technology combined with reflection region suppression strategy and illumination invariance processing and other interference suppression techniques. Then, it quickly obtains the initial matching result through multimodal feature matching. When the confidence of the matching result is insufficient, a multimodal large model is introduced as an arbitrator to verify the consistency between the initial matching result and the image of the advertisement, generating a more accurate matching conclusion. This not only ensures efficient inspection in conventional scenarios, but also solves the matching problem in complex scenarios through the semantic understanding capability of the multimodal large model. Finally, it realizes intelligent, precise and efficient advertising inspection and significantly reduces the cost of manual intervention.

[0100] In one embodiment, the reflection region suppression strategy includes a first suppression strategy; the first suppression strategy refers to applying a negative weight to the reflection region during the global visual feature extraction process; The multimodal feature extraction module 310 is also used for: The image of the advertisement is input into the visual feature extraction model, and the global visual features of the advertisement image are output by the visual feature extraction model. The visual feature extraction model is used for: Based on the first suppression strategy, the semantic information of the advertisement image is extracted layer by layer; After semantic information is extracted and a feature map is output at the intermediate feature layer, the feature map is fused with the reflection component of the advertisement image to obtain a fused feature map. The fused feature map is input into the subsequent feature layers of the intermediate feature layer for layer-by-layer semantic information extraction until the last feature layer is reached for semantic information extraction and output of the global visual features.

[0101] In one embodiment, the local feature features include local visual features, text features, and facial features; the reflection region suppression strategy includes a second suppression strategy, a third suppression strategy, and a fourth suppression strategy; the second suppression strategy refers to filtering key points and their descriptors in the reflection region during the local visual feature extraction process; the third suppression strategy refers to filtering text features in the reflection region during the text feature extraction process; and the fourth suppression strategy refers to filtering facial features in the reflection region during the facial feature extraction process. The multimodal feature extraction module 310 is also used for: Based on the second suppression strategy, key points and their descriptors are extracted from the image of the advertisement to obtain local visual features; Based on the third suppression strategy, the text content in the advertisement image is extracted to obtain text features; Based on the fourth suppression strategy, facial information is extracted from the image published in the advertisement to obtain facial features.

[0102] In one embodiment, the multimodal feature matching module 320 is further configured to: For each of the modal features, the modal features of the advertisement image and the modal features of the advertisement master material are matched to obtain a matching score for the modal features; The matching scores of each modality feature are weighted and summed to obtain the comprehensive matching score between the advertisement image and the advertisement master material; The ad master creative with the highest overall match score is selected as the initial match result.

[0103] In one embodiment, the multimodal large model consistency verification module 330 is further configured to: If the confidence level of the initial matching result does not meet the preset confidence level requirement, the advertisement image and its multi-dimensional semantic information, the initial matching result and its multi-dimensional semantic information are input into the multimodal large model. The multimodal large model is used to perform semantic consistency verification on the advertisement image and the initial matching result to obtain the consistency verification result. Based on the consistency verification results, a final matching conclusion is generated.

[0104] In one embodiment, the intelligent advertising inspection device is also used for: Obtain images of the advertisement after it has been published; The image captured of the advertisement screen is cropped to obtain a foreground enhanced image; If a QR code area exists in the foreground enhancement image, the QR code area is whitened to obtain the advertisement image after interference removal.

[0105] In one embodiment, the intelligent advertising inspection device is also used for: Obtain at least one target ad image that matches the target ad master material provided by the target customer; Based on a preset report template, the inspection data of the target advertisement images and media resource information are automatically integrated to generate an inspection report; After the inspection report is generated, it will be automatically delivered to the target customer.

[0106] It should be noted that the intelligent advertising inspection device provided in this application can execute the intelligent advertising inspection method described in any of the above embodiments during actual operation, which will not be elaborated in this embodiment.

[0107] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logical instructions in the memory 430 to execute an intelligent advertising inspection method. This method includes: extracting multimodal features from the advertisement image to obtain multimodal features; wherein the local feature features in the multimodal features are extracted based on a reflection region suppression strategy; the global visual features in the multimodal features are extracted based on a reflection region suppression strategy and / or illumination invariance processing; matching the multimodal features of the advertisement image with the multimodal features of the advertisement master material to obtain an initial matching result; the multimodal features of the advertisement master material are obtained by pre-extracting multimodal features from the advertisement master material; if the confidence level of the initial matching result does not meet a preset confidence level requirement, calling a multimodal large model to verify the consistency between the advertisement image and the initial matching result, and generating a final matching conclusion based on the consistency verification result.

[0108] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] On the other hand, this application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by the computer, the computer can execute the intelligent advertising inspection method provided in the above embodiments. The method includes: extracting multimodal features from an advertisement image to obtain multimodal features; wherein, the local feature features in the multimodal features are extracted based on a reflection region suppression strategy; the global visual features in the multimodal features are extracted based on a reflection region suppression strategy and / or illumination invariance processing; matching the multimodal features of the advertisement image with the multimodal features of the advertisement master material to obtain an initial matching result; the multimodal features of the advertisement master material are obtained by pre-extracting multimodal features from the advertisement master material; if the confidence of the initial matching result does not meet the preset confidence requirement, calling a multimodal large model to verify the consistency between the advertisement image and the initial matching result, and generating a final matching conclusion based on the consistency verification result.

[0110] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the intelligent advertising inspection method provided in the above embodiments. The method includes: extracting multimodal features from an advertisement image to obtain multimodal features; wherein, the local feature features in the multimodal features are extracted based on a reflection region suppression strategy; the global visual features in the multimodal features are extracted based on a reflection region suppression strategy and / or illumination invariance processing; matching the multimodal features of the advertisement image with the multimodal features of the advertisement master material to obtain an initial matching result; the multimodal features of the advertisement master material are obtained by pre-extracting multimodal features from the advertisement master material; if the confidence level of the initial matching result does not meet the preset confidence level requirement, calling a multimodal large model to perform consistency verification on the advertisement image and the initial matching result, and generating a final matching conclusion based on the consistency verification result.

[0111] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0112] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An intelligent advertising inspection method, characterized in that, The advertisement intelligent inspection method comprises: Multi-modal feature extraction is performed on the advertisement published image to obtain multi-modal features; wherein the local element features in the multi-modal features are extracted based on a reflection area suppression strategy; and the global visual features in the multi-modal features are extracted based on the reflection area suppression strategy and / or illumination invariance processing; The multi-modal features of the advertisement published image are matched with the multi-modal features of the advertisement master material to obtain an initial matching result; the multi-modal features of the advertisement master material are obtained by pre-extracting the multi-modal features of the advertisement master material; In the case that the confidence of the initial matching result does not satisfy the pre-set confidence requirement, a multi-modal large model is called to verify the consistency of the advertisement published image and the initial matching result, and a final matching conclusion is generated according to the consistency verification result.

2. The method of claim 1, wherein, The reflection area suppression strategy comprises a first suppression strategy; the first suppression strategy refers to adopting a negative weight for the reflection area in the global visual feature extraction process; The global visual features are obtained by the following way: The advertisement published image is input into a visual feature extraction model to obtain global visual features of the advertisement published image output by the visual feature extraction model; The visual feature extraction model is used for: Based on the first suppression strategy, the advertisement published image is subjected to layer-by-layer semantic information extraction; After reaching the intermediate feature layer for semantic information extraction and outputting a feature map, the feature map is fused with the reflection component of the advertisement published image to obtain a fused feature map; The fused feature map is input into the subsequent feature layer of the intermediate feature layer for layer-by-layer semantic information extraction until the endmost feature layer for semantic information extraction and output of the global visual features.

3. The method of claim 2, wherein, The visual feature extraction model is trained based on an illumination invariance loss function; the illumination invariance loss function is determined based on the difference between sample image pairs with the same semantic content but different illumination conditions.

4. The method of claim 1, wherein, The local element features comprise local visual features, text features and face features; the reflection area suppression strategy comprises a second suppression strategy, a third suppression strategy and a fourth suppression strategy; the second suppression strategy refers to filtering the key points and their descriptors of the reflection area in the local visual feature extraction process; the third suppression strategy refers to filtering the text features of the reflection area in the text feature extraction process; and the fourth suppression strategy refers to filtering the face features of the reflection area in the face feature extraction process; The local element features are obtained by the following way: Based on the second suppression strategy, key points and their descriptors in the advertisement published image are extracted to obtain local visual features; Based on the third suppression strategy, text content in the advertisement published image is extracted to obtain text features; Based on the fourth suppression strategy, face information in the advertisement published image is extracted to obtain face features.

5. The method of claim 1, wherein, The matching of the multi-modal features of the advertisement published image with the multi-modal features of the advertisement master material to obtain an initial matching result comprises: The modal feature of the advertisement image is matched with the modal feature of the advertisement master material, and a matching score of the modal feature is obtained; The matching scores of the modal features are weighted and summed to obtain a comprehensive matching score between the advertisement image and the advertisement master material; The advertisement master material with the highest comprehensive matching score is determined as an initial matching result.

6. The method of claim 1, wherein, In a case where the confidence of the initial matching result does not satisfy a preset confidence requirement, a multi-modal large model is called to perform consistency verification on the advertisement image and the initial matching result, and a final matching conclusion is generated according to the consistency verification result, including: In a case where the confidence of the initial matching result does not satisfy a preset confidence requirement, the advertisement image and its multi-dimensional semantic information, the initial matching result and its multi-dimensional semantic information are input into a multi-modal large model, semantic consistency verification is performed on the advertisement image and the initial matching result by the multi-modal large model, and a consistency verification result is obtained; According to the consistency verification result, a final matching conclusion is generated.

7. The method of claim 1, wherein, The advertisement image is obtained by the following method: An advertisement screen shooting image is obtained after an advertisement is published; An advertisement screen region of the advertisement screen shooting image is cropped to obtain a foreground enhanced image; If a two-dimensional code region exists in the foreground enhanced image, the two-dimensional code region is set to white to obtain an advertisement image after interference elimination.

8. The method of claim 1 to 7, wherein, The advertisement intelligent inspection method further includes: At least one target advertisement image matched by a target advertisement master material provided by a target customer is obtained; Based on a preset report template, inspection data of the target advertisement image and media resource information are automatically integrated to generate an inspection report; After the inspection report is generated, the inspection report is automatically delivered to the target customer.

9. An advertising intelligent inspection device, characterized in that, The advertisement intelligent inspection device includes: A multi-modal feature extraction module is configured to extract multi-modal features from an advertisement image, wherein local element features in the multi-modal features are extracted based on a reflection region suppression strategy, and global visual features in the multi-modal features are extracted based on the reflection region suppression strategy and / or illumination invariance processing; A multi-modal feature matching module is configured to match the multi-modal features of the advertisement image with multi-modal features of an advertisement master material to obtain an initial matching result, wherein the multi-modal features of the advertisement master material are obtained by pre-extracting multi-modal features from the advertisement master material; A multi-modal large model consistency verification module is configured to call a multi-modal large model to perform consistency verification on the advertisement image and the initial matching result in a case where a confidence of the initial matching result does not satisfy a preset confidence requirement, and generate a final matching conclusion according to a consistency verification result.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein The processor executes the computer program to implement the steps of the advertisement intelligent inspection method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein The computer program is executed by the processor to implement the steps of the advertisement intelligent inspection method according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of intelligently patrolling the advertisement as claimed in any one of claims 1 to 8.

Images