A large model-based multi-modal illegal advertising content identification method and system

Abstract

The application provides a kind of multi-modal rule-breaking advertisement content identification method and system based on large model, the method comprises: extracting speech recognition text and subtitle recognition text in the audio and video stream to be processed, and aligning fusion is carried out to construct joint text;The joint text is segmented according to the first granularity, the first granularity segmented text is obtained, and the first granularity segmented text is preliminarily screened based on the first semantic analysis model, to obtain the suspected rule-breaking advertisement period;The text corresponding to the suspected rule-breaking advertisement period is segmented according to the second granularity, the second granularity segmented text is obtained, and the second granularity segmented text is analyzed based on the second semantic analysis model, to obtain the rule-breaking advertisement text paragraph;The rule-breaking advertisement text paragraph is merged, and the rule-breaking advertisement analysis result is generated and output, the dispersed local feature is associated and restored at the overall semantic level, to prevent the misjudgment caused by context fragmentation, finally realize the target content identification of high precision and automation.

Description

Technical Field

[0001] This invention relates to the field of audio and video information processing technology, and in particular to a method and system for identifying multimodal illegal advertising content based on a large model. Background Technology

[0002] As a crucial channel for the public to access information, radio and television broadcasts pharmaceutical advertisements whose content directly impacts the health, safety, and legitimate rights of the people. In recent years, the state has continuously strengthened its supervision of radio and television programs related to medical and health care, as well as pharmaceutical advertisements, severely cracking down on various illegal activities such as false advertising, exaggerated efficacy claims, and disguised marketing. However, with the increasing length of radio and television programs and the concurrent broadcasting of multiple channels around the clock, regulatory authorities are facing massive amounts of continuous audio and video data. Given this enormous data base, achieving large-scale, comprehensive monitoring of illegal pharmaceutical advertisements has become a core challenge in the field of intelligent radio and television supervision.

[0003] Currently, the conventional methods for detecting false medical advertisements in radio and television can be divided into two categories: The first category is based on cataloging and initial screening and manual judgment. This approach typically uses audio and video template comparison technology to compare the content consistency of the program stream to generate an advertisement playlist, and then relies on manual browsing and screening to subjectively judge whether it violates regulations. The second category is based on keyword retrieval and manual review. This involves manually setting a database of sensitive words suspected of violating regulations, and the system extracts the audio and subtitle text of the radio and television broadcasts for shallow string retrieval and matching. Once a match is found, it is handed over to manual review for qualitative assessment.

[0004] While the aforementioned existing technologies have assisted in relevant regulatory work, they suffer from a major deficiency: a lack of deep semantic understanding capabilities. Current sensitive word matching methods remain at a superficial string comparison level. Faced with increasingly complex disguised marketing and hidden violations, simple surface feature matching is easily evaded by synonym substitution or contextual changes, resulting in a high rate of missed detections of false pharmaceutical advertisements. Therefore, how to overcome the interference of complex contexts in massive amounts of audio and video data and achieve accurate and automated identification of deep-seated violations in disguised false pharmaceutical advertisements is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This invention provides a multimodal method and system for identifying illegal advertising content based on a large model. It addresses the shortcomings of existing technologies, such as the lack of deep semantic understanding capabilities and the fact that they only rely on shallow string comparison, which makes them easily evaded by synonym substitution or contextual changes, resulting in a high rate of missed detection for disguised and implicit illegal content. This invention achieves accurate and automated identification of deep target semantic intent by overcoming complex contextual interference in massive audio and video data.

[0006] This invention provides a method for identifying multimodal illegal advertising content based on a large model, including: Extract speech recognition text and subtitle recognition text from the audio and video streams to be processed, and align and fuse them to construct joint text; The joint text is segmented according to a first granularity to obtain first granularity segmented text, and the first granularity segmented text is initially screened based on a first semantic analysis model to obtain suspected illegal advertising periods. The text corresponding to the suspected illegal advertising period is segmented according to the second granularity to obtain the second granularity segmented text. The second granularity segmented text is then subjected to feature analysis based on the second semantic analysis model to obtain the illegal advertising text paragraphs. The first granularity is greater than the second granularity. The semantics of the illegal advertising text segments are semantically merged to generate and output the illegal advertising analysis results.

[0007] According to the multimodal illegal advertising content identification method based on a large model provided by the present invention, the step of extracting speech recognition text and subtitle recognition text from the audio and video stream to be processed, and aligning and fusing them to construct joint text, includes: Obtain the timestamps corresponding to the speech recognition text and the subtitle recognition text; based on the timestamps, match and align the speech recognition text and the subtitle recognition text in the time dimension; analyze the semantic logical association between the aligned speech recognition text and the subtitle recognition text; perform cross-modal feature fusion based on the semantic logical association to generate the joint text with contextual information.

[0008] According to the multimodal illegal advertising content identification method based on a large model provided by the present invention, the step of segmenting the joint text according to a first granularity to obtain first-granularity segmented text, and performing preliminary screening of the first-granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising time periods, includes: Based on a first preset time length, the joint text is dynamically segmented to generate the first granular segmented text; The first granular segmented text is input into the first semantic analysis model to analyze the overall narrative logic of the first granular segmented text; Based on the overall narrative logic, non-target texts that do not contain target intent features are filtered out, and time periods are extracted from the first-granularity segmented texts that contain target intent features, which are then used as the suspected illegal advertising periods.

[0009] According to the multimodal illegal advertising content identification method based on a large model provided by the present invention, the text corresponding to the suspected illegal advertising period is segmented according to a second granularity to obtain second-granularity segmented text, and feature analysis is performed on the second-granularity segmented text based on a second semantic analysis model to obtain illegal advertising text segments, including: Based on the second preset text length, the first granularity segmented text corresponding to the suspected illegal advertising period is segmented to generate the second granularity segmented text; The second granular segmented text is input into the second semantic analysis model to extract specific semantic feature points representing the target intent in the second granular segmented text; The specific semantic feature points are timestamped to obtain the illegal advertisement text paragraph.

[0010] According to the multimodal illegal advertising content identification method based on a large model provided by the present invention, the step of semantically merging the illegal advertising text segments to generate and output the illegal advertising analysis results includes: The step of semantically merging the illegal advertising text segments to generate and output the illegal advertising analysis results includes: Calculate the semantic similarity and temporal proximity among the various segments of the illegal advertisement text; Based on the semantic similarity and temporal proximity, multiple related illegal advertising text segments are combined into an illegal advertising text segment set and then merged. Based on the merged set of illegal advertisement text segments, the illegal advertisement analysis results are generated and output.

[0011] According to the multimodal illegal advertising content identification method based on a large model provided by the present invention, the step of generating and outputting the illegal advertising analysis results based on the merged set of illegal advertising text segments includes: Extract the timestamps corresponding to each illegal advertisement text segment in the set of illegal advertisement text segments; By combining the set of illegal advertisement text segments and the timestamp, structured data is constructed, and the structured data is output as the analysis result of the illegal advertisement.

[0012] According to the multimodal illegal advertising content identification method based on a large model provided by the present invention, the step of dynamically segmenting the joint text based on a first preset time length to generate the first granularity segmented text includes: The joint text is initially segmented according to the first preset time length to obtain a first initial boundary point; speech silence features or punctuation features are extracted within a preset time window centered on the first initial boundary point; based on the speech silence features or punctuation features, the first initial boundary point is slid to an adjacent semantic pause position for segmentation to generate the first granular segmented text.

[0013] According to the multimodal illegal advertising content identification method based on a large model provided by the present invention, the step of segmenting the first-granularity segmented text corresponding to the suspected illegal advertising time period based on a second preset text length to generate the second-granularity segmented text includes: The first-granularity segmented text corresponding to the suspected illegal advertising period is initially segmented according to the second preset text length to obtain the second initial boundary point; Extract speech silence features or punctuation features within a preset character window centered on the second initial boundary point; Based on the speech silence feature or punctuation feature, the second initial boundary point is slid to the adjacent semantic pause position for segmentation to generate the second granular segmented text.

[0014] This invention also provides a multimodal illegal advertising content identification system based on a large model, comprising: The module is used to extract speech recognition text and subtitle recognition text from the audio and video streams to be processed, and then align and fuse them to construct joint text; The initial screening module is used to segment the joint text according to a first granularity to obtain the first granularity segmented text, and to perform initial screening on the first granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising periods. The analysis module is used to segment the text corresponding to the suspected illegal advertising period according to the second granularity to obtain the second granularity segmented text, and to perform feature analysis on the second granularity segmented text based on the second semantic analysis model to obtain the illegal advertising text paragraph, wherein the first granularity is greater than the second granularity; The merging module is used to semantically merge the illegal advertising text segments, generate and output the illegal advertising analysis results.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multimodal illegal advertising content identification method based on a large model as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multimodal illegal advertising content identification method based on a large model as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multimodal illegal advertising content identification method based on a large model as described above.

[0018] The present invention provides a method, system, and electronic device for identifying multimodal illegal advertising content based on a large model. First, it extracts speech recognition text and subtitle recognition text from the audio / video stream to be processed, aligns and fuses them to construct a joint text. Then, it segments the joint text according to a first granularity to obtain first-granularity segmented text. Based on a first semantic analysis model, it performs initial screening on the first-granularity segmented text to identify suspected illegal advertising periods. Next, it segments the text corresponding to these suspected illegal advertising periods according to a second granularity smaller than the first granularity to obtain second-granularity segmented text. Finally, it performs feature analysis on the second-granularity segmented text based on a second semantic analysis model. The analysis yielded illegal advertising text segments. This processing method overcomes the limitations of simple surface feature matching, enabling a deep understanding of the overall narrative logic within a larger text range at the first granularity level to overcome interference from complex contextual changes. Furthermore, it accurately analyzes and extracts deeper, specific intentions within a smaller text range at the second granularity level, effectively reducing the false negative rate of implicit content. Finally, the illegal advertising text segments are semantically merged to generate and output the illegal advertising analysis results. This process correlates and restores scattered local features at the overall semantic level, preventing misjudgments caused by contextual fragmentation, and ultimately achieving high-precision and automated target content recognition. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts of the multimodal illegal advertising content identification method based on a large model provided by the present invention.

[0021] Figure 2 This is the second flowchart of the multimodal illegal advertising content identification method based on a large model provided by the present invention.

[0022] Figure 3 This is a schematic diagram of the structure of the multimodal illegal advertising content recognition system based on a large model provided by the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] To more clearly illustrate the improvements of this invention compared to the prior art, a more detailed description of the prior art related to this solution is first provided. Currently, in the regulatory practice of addressing false advertising of pharmaceutical products on radio and television, the following two types of technical means are mainly employed: The first type is based on advertising cataloging and manual review. This approach typically begins with manual annotation of advertising segments in radio and television programs, including advertising names, types, and templates (such as opening and closing frames). Building on this, the system uses audio-visual template comparison technology (such as fingerprint matching or feature point comparison) to automatically identify and mark advertising segments matching the annotated templates by performing content consistency searches on the program stream, thus generating a broadcast television advertising playlist. After the playlist is constructed, qualified reviewers manually browse the advertising list of the target channel, filter out pharmaceutical advertisements, and, based on the standards for judging false advertising in pharmaceuticals issued by the State Administration of Radio and Television, judge each advertisement to determine whether it involves exaggerated efficacy claims, disguised marketing, or other violations. Finally, the system manually fills in the violation type and specific judgment opinion. The core of this method is to delegate the task of identifying repetitive advertisements to the system, but the core violation judgment process still relies entirely on manual intervention.

[0026] The second method is based on keyword retrieval using voice and subtitles. This method first involves technicians manually pre-setting a batch of potentially illegal sensitive keywords according to the relevant regulations of the State Administration of Radio and Television, such as absolute terms like "cure," "miracle cure," and "traditional remedy," or words implying therapeutic effects. Then, the system processes the audio and video streams of the broadcast television in real time, extracting the voice text using Automatic Speech Recognition (ASR) technology and extracting the on-screen subtitle text using Optical Character Recognition (OCR) technology. The system performs high-speed string matching retrieval on these two text streams; once a pre-set keyword is matched, the content corresponding to that time point is marked as potentially illegal. Afterwards, reviewers need to verify all matched results one by one to confirm whether they are genuine false advertising for pharmaceuticals, and mark the confirmed illegal advertisements with the advertisement name and type, filling in the type of violation and their assessment. This method attempts to improve the efficiency of initial screening through automated retrieval, but its core judgment still requires human intervention.

[0027] While the aforementioned existing technologies possess certain application value under specific conditions, their limitations are becoming increasingly apparent. Taking the first type of method as an example, the process of manually browsing advertising playlists is constrained by the subjective experience, work fatigue, and processing speed of the reviewers. Faced with massive amounts of broadcast television data occurring 24 / 7 across multiple channels, it is difficult to achieve large-scale, comprehensive real-time monitoring, resulting in a high risk of missed detections. Taking the second type of method as an example, its essence is based on character-level pattern matching, lacking a deep understanding of the text's semantics. With the continuous evolution of false advertising methods, violators increasingly employ synonym substitution (e.g., changing "cure" to "get rid of pain"), context avoidance (e.g., embedding efficacy promises within patient narratives), and packaging advertisements as health lectures or expert interviews, making simple keyword searches easily bypassed. This leads to a large amount of hidden violations failing to be effectively identified, resulting in a persistently high rate of missed detections.

[0028] To address the shortcomings of existing technologies, this invention provides a multimodal method for identifying illegal advertising content based on a large model. This method no longer limits violation identification to shallow keyword matching or full manual judgment, but innovatively integrates multimodal information extraction with a two-level large model cascaded reasoning mechanism. Specifically, firstly, the dual-channel text obtained from broadcast television audio and video streams through ASR and OCR processing is time-aligned and semantically fused to construct a joint text representation rich in contextual information. Then, a two-level progressive granularity strategy is adopted: firstly, a first semantic analysis model quickly screens text blocks with larger temporal granularity to locate time periods containing suspected illegal intent; then, a second semantic analysis model performs refined semantic analysis on smaller text blocks within the initially screened time periods to accurately extract specific illegal semantic feature points. Finally, semantic similarity and temporal proximity analysis are performed on scattered illegal text fragments, automatically merging complete evidence chains belonging to the same illegal advertising campaign, and outputting a structured result containing timestamps and judgment criteria. The following will provide a detailed description of this technical solution with reference to the accompanying drawings.

[0029] The execution entity of the method provided in this invention can be any computing device or system with audio and video data processing capabilities and large model inference capabilities. For example, the execution entity can be a standalone server, a server cluster, a cloud computing platform, a dedicated hardware processing device deployed inside a broadcast television monitoring center, or a functional module or software component integrated into an existing intelligent broadcast television monitoring system. In practical applications, the execution entity can access broadcast television signal sources (such as cable television, satellite television, terrestrial digital television, or IPTV streams), acquire the audio and video stream data to be monitored in real time or offline, and execute the processing steps described in this invention. For ease of description, this invention uses "this system" as a general term for the execution entity, but this does not constitute a limitation on the specific implementation form of the execution entity. Any hardware or software entity capable of implementing the method described in this invention falls within the protection scope of this invention.

[0030] Before describing the technical solutions of the embodiments of the present invention, the technical terms of the embodiments of the present invention will be explained.

[0031] Unprocessed audio and video stream: refers to raw audio and video data collected from broadcast television signals that has not yet undergone violation detection. Its sources may include, but are not limited to, live streams or recorded files from cable television, satellite television, terrestrial digital television, IPTV, and internet audiovisual programs. This audio and video stream typically contains continuous audio tracks and video frame sequences, and is the basic data source for this invention to detect illegal advertisements.

[0032] Speech-recognized text: This refers to the text content obtained after processing the audio track in the audio / video stream using automatic speech recognition technology. This text typically exhibits conversational characteristics, may contain repetitions, pauses, slips of the tongue, and other natural language phenomena, and carries corresponding timestamps to identify the start and end times of each segment of text within the original audio / video stream.

[0033] Subtitle recognition text: This refers to the text content obtained after detecting and recognizing video frames in the audio and video stream being processed using optical character recognition technology. This text usually originates from subtitles, scrolling subtitles, or bullet comments superimposed on the television screen, has written characteristics, and also carries corresponding timestamp information.

[0034] The first semantic analysis model refers to a large-scale language model that has been pre-trained or fine-tuned, used for semantic understanding and intent recognition of text blocks with a large temporal granularity (e.g., more than 5 minutes). This model has the ability to comprehensively judge the overall narrative logic of the text, potentially misleading expression patterns, and illegal intent, and can quickly determine at a coarse-grained level whether a text block contains semantic features that may indicate false advertising for pharmaceuticals.

[0035] First-level granularity text segmentation: This refers to a series of text blocks obtained after segmenting the joint text according to the first granularity. The first granularity is based on time length (e.g., 5 minutes, 10 minutes), and each text block covers the entire joint text content within a continuous time period. This segmented text serves as input to the first semantic analysis model for preliminary violation screening.

[0036] The second semantic analysis model refers to a large-scale language model that has been pre-trained or fine-tuned, used to extract semantic features and locate violations in text paragraphs with smaller text granularity (e.g., more than 100 characters). Compared with the first semantic analysis model, the second semantic analysis model focuses on the accurate identification and timestamping of specific violations in local text (e.g., efficacy promises, expert testimonials, patient testimonials, etc.).

[0037] Suspected Illegal Advertising Periods: These refer to the time intervals identified as potentially containing false advertising content in the first-level segmented text after initial screening using the first semantic analysis model. These periods are marked with their start and end times and serve as candidate regions for subsequent refined analysis.

[0038] Second-level granularity text segmentation: This refers to a series of text blocks obtained by further segmenting the first-level granularity text corresponding to the suspected illegal advertising period. The second granularity uses text length (e.g., 100 characters, 200 characters) as the dividing benchmark, and each text block is smaller in scale, which facilitates the second semantic analysis model to extract refined semantic features.

[0039] Violational advertising text segments: These refer to text fragments containing specific violation semantic features identified by the second-granularity segmented text from the second-level semantic analysis model. Each violational advertising text segment corresponds to a clear violation category (such as exaggerating efficacy, impersonating experts, disguised marketing, etc.) and is associated with its accurate timestamp in the original audio and video stream.

[0040] Analysis results of illegal advertisements: This refers to the structured output data generated after semantically merging the text segments of illegal advertisements. This result includes at least the original audio and video timestamps corresponding to the illegal content, the illegal text fragments, the violation type determination, and the analysis process text generated by the large model, used to support subsequent manual review, evidence preservation, and law enforcement actions.

[0041] Speech silence feature: refers to the time period feature in an audio signal where no effective speech energy is detected. In this invention, speech silence feature can serve as an indicator signal of natural semantic boundaries to optimize text segmentation strategies and avoid splitting a complete semantic unit (such as a complete advertising paragraph) into two different segments.

[0042] Punctuation features refer to the semantic boundary information contained in punctuation marks (such as periods, question marks, exclamation marks, semicolons, etc.) appearing in text. In this invention, punctuation features are also used to assist in adjusting text segmentation boundaries, making the segmentation results closer to natural sentence or paragraph divisions, thereby improving the input quality of the semantic analysis model.

[0043] Figure 1 This is a flowchart illustrating the multimodal illegal advertising content identification method based on a large model provided by the present invention. The method includes: Step 101: Extract the speech recognition text and subtitle recognition text from the audio and video stream to be processed, and align and fuse them to construct a joint text.

[0044] In this step, the system first acquires the broadcast television audio and video stream to be monitored. This stream can be a live broadcast signal acquired in real time or a pre-stored recording file. The system performs two types of processing on the audio and video stream: firstly, it uses automatic speech recognition technology to convert the audio track into speech-to-text; secondly, it uses optical character recognition technology to extract the subtitle recognition text corresponding to the overlaid subtitles from the video frames. Since both the speech-to-text and the subtitle recognition text carry their own timestamp information, the system aligns them on the timeline based on the timestamps, ensuring that the audio content and subtitle content within the same time period correspond to each other. Based on this, the system merges the aligned dual-stream text to construct a joint text that is complementary and coherent. This joint text retains the colloquial expressions in the speech while supplementing the more standardized textual information in the subtitles, providing a rich data foundation for subsequent semantic analysis.

[0045] Step 102: Segment the joint text according to the first granularity to obtain the first granularity segmented text, and perform preliminary screening on the first granularity segmented text based on the first semantic analysis model to obtain suspected illegal advertising time periods.

[0046] After obtaining the joint text, this system segments it according to a preset first granularity. The first granularity uses time length as the dividing basis, for example, it can be set to a segment unit of 5 minutes or 10 minutes. Based on this time length, the system cuts the continuous joint text into multiple non-overlapping text blocks, each corresponding to all text content within a continuous time interval. These text blocks are the first-granularity segmented text. Subsequently, the system inputs each first-granularity segmented text into a pre-trained first semantic analysis model. This model is a large-scale language model with the ability to understand the overall semantics and narrative logic of the text. The first semantic analysis model analyzes the input text blocks to determine whether they contain semantic features suspected of false advertising for pharmaceuticals, such as narrative patterns that disguise drug promotion through health lectures or patient interviews, or expressions of intent to exaggerate efficacy or promote pseudoscience.

[0047] For text blocks deemed to contain suspected illegal intent, this system extracts the corresponding time interval as the suspected illegal advertising period. For text blocks deemed normal (such as news, weather forecasts, TV series, etc.), this system does not perform any further processing, thus focusing computing resources.

[0048] Step 103: Segment the text corresponding to the suspected illegal advertising period according to the second granularity to obtain the second granularity segmented text, and perform feature analysis on the second granularity segmented text based on the second semantic analysis model to obtain the illegal advertising text segment, wherein the first granularity is greater than the second granularity.

[0049] After identifying the suspected period of illegal advertising, this system further segments the combined text within that period. Unlike step 102, this step uses a second granularity as the segmentation basis, with text length as the dividing standard. For example, it can be set to 100 or 200 characters as a segment unit. Because the second granularity is measured by the number of characters, the length of the text blocks it divides is significantly smaller than the time block length of the first granularity, thus enabling more precise positioning. This system cuts the text corresponding to the suspected period of illegal advertising into multiple second-granularity segmented texts according to the second granularity.

[0050] Subsequently, the system inputs each second-granularity segment of text into the second semantic analysis model. This model is also a large-scale language model, focusing on the precise extraction of specific violation semantic features from local text. The second semantic analysis model can identify which statements or phrases in the text constitute exaggerated claims about therapeutic effects, which are fabricated expert testimonies or patient testimonies, and which fall under the specific category of suggestive advertising, etc.

[0051] For text fragments identified as containing specific semantic features of violations, this system marks them as illegal advertising text segments and records the accurate timestamp of the segment in the original audio and video stream.

[0052] Step 104: Semantically merge the illegal advertisement text segments to generate and output the illegal advertisement analysis results.

[0053] In actual broadcast television programs, the same false advertising medical advertisement may be divided into multiple segments, broadcast at different times, or separated from normal content within a continuous program stream. Therefore, there may be multiple segments of the illegal advertising text, and they may belong to the same illegal advertising campaign.

[0054] In this step, the system performs a comprehensive analysis of all illegal advertising text segments. Specifically, the system calculates the semantic similarity and temporal proximity between each illegal advertising text segment. If multiple illegal advertising text segments have highly similar semantic content (such as promoting the same drug or using the same core promotional slogan) and are close in time, the system classifies them as belonging to the same illegal advertising campaign and performs semantic merging. The merged result restores the complete chain of evidence for the illegal advertisement, avoiding judgment bias caused by information fragmentation. Finally, the system generates and outputs structured illegal advertising analysis results. These results include at least the original audio / video timestamps corresponding to the illegal content, the illegal text fragments, and the type of violation, facilitating subsequent manual review, law enforcement evidence collection, and data archiving.

[0055] The multimodal illegal advertising content identification method based on a large model provided in this invention first extracts speech recognition text and subtitle recognition text from the audio and video stream to be processed, and aligns and fuses them to construct a joint text. Then, the joint text is segmented according to a first granularity to obtain first-granularity segmented text, and a first semantic analysis model is used to initially screen the first-granularity segmented text to obtain suspected illegal advertising time periods. Then, the text corresponding to the suspected illegal advertising time periods is segmented according to a second granularity smaller than the first granularity to obtain second-granularity segmented text, and a second semantic analysis model is used to perform feature analysis on the second-granularity segmented text to obtain illegal advertising text segments. This processing method breaks through the limitations of simple surface feature matching, and can deeply understand the overall narrative logic on a larger range of text at the first granularity to overcome the interference of complex context changes, and accurately analyze and extract deep specific intentions on a smaller range of text at the second granularity, thereby effectively reducing the false negative rate of implicit content. Finally, the illegal advertising text segments are semantically merged to generate and output the illegal advertising analysis results, and the scattered local features are associated and restored at the overall semantic level, preventing misjudgments caused by context fragmentation, and finally achieving high-precision and automated target content identification.

[0056] In step 101, the speech recognition text and subtitle recognition text are extracted from the audio and video stream to be processed and aligned and fused to construct the joint text. This can be achieved in the following way: First, this system obtains the timestamps corresponding to the speech-recognized text and the subtitle-recognized text. In actual processing, when the Automatic Speech Recognition (ASR) engine converts audio to text, it simultaneously records the start and end times of each segment of text in the original audio, forming the timestamp of the speech-recognized text. Similarly, when the Optical Character Recognition (OCR) engine extracts subtitles from video frames, it also records the time point of each frame or segment of subtitle appearance, forming the timestamp of the subtitle-recognized text. This system reads these two text streams and their accompanying timestamp information respectively.

[0057] Secondly, based on the aforementioned timestamps, this system matches and aligns the speech-recognized text with the subtitle-recognized text along the time dimension. Specifically, using the timeline as a benchmark, this system establishes a correspondence between speech-recognized text segments and subtitle-recognized text segments that fall within the same time period or near a specific time point. For example, for speech-recognized text with timestamps from 00:01:00 to 00:01:30, this system searches for subtitle-recognized text that also falls within this time period and associates the two. Through this alignment operation, this system can determine what the system is saying (speech) and what text is being displayed on the screen (subtitles) at any given moment in the audio / video stream.

[0058] Next, this system analyzes the semantic logical relationship between the aligned speech recognition text and the subtitle recognition text. Since the speech recognition text originates from the audio track, it typically exhibits colloquial characteristics and may contain filler words, repetitions, word order inversions, and other natural language phenomena. In contrast, the subtitle recognition text originates from overlaid subtitles on the screen and is usually edited, exhibiting formal and standardized characteristics. The two may present different language styles when expressing the same content, and may even have complementary information. For example, the speech may describe the efficacy of a medicine in a more colloquial way, while the subtitles may summarize the key information in concise formal language; or the speech may mention a product name, while the subtitles may provide information on the product's contraindications. This system uses large-scale models or semantic analysis algorithms to compare the aligned dual-path text, identifying whether they are semantically repetitive, complementary, contradictory, or unrelated, thereby grasping the inherent logical relationship between cross-modal information.

[0059] Finally, based on the aforementioned semantic logical connections, this system performs cross-modal feature fusion to generate joint text with contextual information. The fusion strategy varies depending on the type of semantic connection: when the speech recognition text and the subtitle recognition text have semantic overlap, the system can choose to retain the one that is clearer or more complete, avoiding redundancy; when the two are semantically complementary, the system splices the two texts in chronological order to form a more comprehensive description; when the two are semantically contradictory (e.g., the speech claims "no side effects" while the subtitles display "contraindications: see instruction manual"), the system retains the contradictory information and marks it for subsequent analysis models to focus on. Through this fusion, the generated joint text not only contains all the effective information from both texts but also eliminates ambiguity and supplements missing information through semantic logical analysis, thus possessing stronger contextual coherence and semantic richness.

[0060] The above alignment and fusion process ensures the correspondence between speech and subtitles in the time dimension through timestamp matching, laying the foundation for subsequent cross-modal analysis. Through semantic logical association analysis, it reveals the inherent connection between spoken and written text, enabling the system to understand different expressions of the same content in different modalities. Through cross-modal feature fusion, the generated joint text combines the spoken richness of ASR text with the standardization of OCR text, greatly improving the depth of understanding of the content by the subsequent semantic analysis model.

[0061] Further, in step 102, the joint text is segmented according to a first granularity to obtain first-granularity segmented text, and the first-granularity segmented text is initially screened based on a first semantic analysis model to obtain suspected illegal advertising periods. This can be achieved in the following ways: First, this system dynamically segments the joint text based on a first preset time length, generating first-granularity segmented text. Specifically, the system initially segments the joint text according to the first preset time length (e.g., 5 minutes), obtaining multiple first initial boundary points. Then, the system extracts speech silence features or punctuation features within a preset time window centered on each first initial boundary point (e.g., 5 seconds before and after the boundary point). Speech silence features refer to the time period in the audio signal where there is no effective speech energy, and punctuation features refer to punctuation marks in the text, such as periods, exclamation marks, and semicolons, which indicate semantic pauses. Based on these features, the system determines whether the current boundary point is located at a natural semantic pause position. If the boundary point falls in the middle of a word or sentence, the system slides the boundary point forward or backward to an adjacent semantic pause position (e.g., a silence gap or a period), and then segments at that position. Through this dynamic adjustment, the boundaries of the first-granularity segmented text generated by this system more closely fit complete semantic units, avoiding the splitting of a complete advertising paragraph or health lecture segment into two different segments.

[0062] Secondly, the system inputs the generated first-granularity segmented text into the first semantic analysis model. This model is a finely tuned large-scale language model with a deep understanding of the overall narrative logic of long texts. The first semantic analysis model analyzes each first-granularity segmented text to determine whether its overall narrative logic conforms to the typical pattern of false advertising in pharmaceuticals. For example, the first semantic analysis model can identify whether the text uses a disguised marketing narrative structure of "patient stories + expert recommendations + product efficacy implications," or whether it is using health education as a guise for drug promotion.

[0063] Finally, based on the analysis results output by the first semantic analysis model, this system filters out non-target text (such as news, TV dramas, weather forecasts, and other normal program content) that does not contain target intent features. It only extracts the corresponding time periods from the first-granularity segmented text that contains target intent features, outputting them as suspected illegal advertising periods. For example, if a 5-minute segmented text is determined by the model to have the semantic features of "pseudo-science + implying therapeutic effects," this system marks the start and end times of that segment as suspected illegal advertising periods for subsequent analysis.

[0064] This step utilizes dynamic segmentation based on speech silence and punctuation features to ensure that segment boundaries more closely resemble natural semantic units. This guarantees that the text blocks input to the model have complete context and avoids semantic breaks and recognition errors caused by improper boundary segmentation. Through in-depth analysis of the overall narrative logic using the first semantic analysis model, it can quickly identify disguised false advertisements masquerading as health lectures or patient interviews at a macro level, effectively filtering a large amount of legitimate program content and achieving precise focus of computational resources. Compared to traditional methods, this step no longer relies on manual browsing or keyword matching. Instead, it leverages the semantic understanding capabilities of a large model to efficiently screen massive amounts of data at a coarse-grained level, significantly improving the intelligence and processing efficiency of the initial screening.

[0065] Further, in step 103, the text corresponding to the suspected illegal advertising period is segmented according to the second granularity to obtain the second granularity segmented text. Then, feature analysis is performed on the second granularity segmented text based on the second semantic analysis model to obtain the illegal advertising text segments. This can be achieved in the following way: First, this system segments the text at the first granularity based on a second preset text length, generating second granularity segmented text. Specifically, the system initially segments the combined text within the suspected illegal advertising period according to the second preset text length (e.g., 100 characters), obtaining multiple second initial boundary points. For example, suppose a combined text has a period at the 95th character and the word "efficacy" at the 102nd character (between "treatment" and "effect"). Since the second preset text length is 100 characters, the initial boundary point falls at the 100th character, precisely in the middle of the word "efficacy." At this point, the system extracts punctuation features within a preset character window centered on this initial boundary point (e.g., 10 characters before and after the boundary point), detecting a semantic pause marker—a period—at the 95th character. Therefore, the system slides this boundary point forward to the period position at the 95th character for segmentation, ensuring the segment boundary falls at the end of the complete sentence. Through this dynamic adjustment, the boundaries of the second-granularity segmented text generated by this system are closer to complete sentences or phrases, ensuring that each text block has relatively independent semantic integrity, which facilitates subsequent fine feature extraction.

[0066] Secondly, the system inputs each generated second-granularity segmented text into the second semantic analysis model. This model is also a finely tuned large-scale language model, but its focus is on accurately extracting specific violation semantic features from local texts. The second semantic analysis model can identify which statements or phrases in the text exaggerate therapeutic effects (e.g., radical cure, never relapse), which are fabricated expert testimonies or patient testimonies (e.g., Professor Zhang's recommendation, patient Li's personal account), which use unverified concepts for suggestion (e.g., quantum energy, nanotechnology), and which are implicit advertisements that subtly guide consumption (e.g., call to order now). The second semantic analysis model outputs these specific semantic feature points and provides the violation category and confidence score for each feature point.

[0067] Finally, the system timestamps each extracted semantic feature point. Since the second-granularity segmented text is derived from timestamped joint text, each segment corresponds to a time interval, and the positional offset of the feature point within the segment can be converted into a precise time point. The system records the start and end times corresponding to the feature points as timestamps and combines these timestamped feature points into segments of illegal advertising text. Each segment of illegal advertising text corresponds to a clear point of evidence of violation, such as "At 00:15:23-00:15:28, the audio text contains an exaggerated claim of 'curing diabetes'."

[0068] This step uses dynamic segmentation with a second preset text length to ensure that the text blocks in each input model are of moderate length and their boundaries closely resemble natural semantic units. This guarantees that the second semantic analysis model can focus on the fine-grained feature extraction of local text, avoiding distraction due to excessively long segments or missing context due to excessively short segments. Through the extraction of specific semantic feature points by the second semantic analysis model, precise localization from coarse-grained suspected time periods to fine-grained evidence of violations is achieved, solving the problem that traditional methods cannot identify specific violating statements. Timestamp marking allows each piece of evidence of violations to be traced back to its corresponding position in the original audio and video stream, providing a reliable temporal basis for subsequent merging, review, and evidence collection.

[0069] In step 104, semantic merging is performed on the illegal advertising text segments, and the illegal advertising analysis results are generated. This can be achieved in the following ways: First, this system calculates the semantic similarity and temporal proximity between the various illegal advertising text segments. In actual processing, the illegal advertising text segments output in step 103 may come from different locations within the same suspected illegal advertising time period, or even span multiple different suspected illegal advertising time periods. Some of these illegal advertising text segments may belong to different fragments of the same false advertising medical product, while others may correspond to completely different illegal advertisements. To correctly aggregate evidence of violations belonging to the same advertising campaign, this system calculates the semantic similarity between every two illegal advertising text segments. For example, it uses large models or vectorization techniques to convert the text segments into semantic vectors, and then calculates the cosine similarity between the vectors. Simultaneously, this system compares the timestamps corresponding to each illegal advertising text segment and calculates their distance on the time axis (i.e., temporal proximity). For example, if the timestamps of two segments are 00:15:20-00:15:25 and 00:15:30-00:15:35 respectively, and the time interval between them is only 5 seconds, then they have high temporal proximity.

[0070] Then, based on the aforementioned semantic similarity and temporal proximity, this system combines multiple related illegal advertising text segments into a set of illegal advertising text segments and merges them. A semantic similarity threshold (e.g., 0.75) and a temporal proximity threshold (e.g., 30 seconds) can be set. When the similarity of two segments exceeds the threshold and the time interval is less than the threshold, they are determined to belong to the same illegal advertising activity and are grouped into the same set. For multiple segments in the set, they are spliced ​​together in chronological order to form a complete and continuous evidence text of the violation. For example, the first segment of a false advertisement claims "Mr. Zhang has had diabetes for many years," the second segment claims "His blood sugar returned to normal after taking this product," and the third segment mentions "Ordering hotline 400-XXX." These three segments are highly semantically related and occur at similar times. This system merges them into a complete violation record, restoring the complete narrative chain from setup and promotion to guiding consumption.

[0071] Finally, this system generates analysis results for illegal advertisements based on the merged set of illegal advertisement text segments. It extracts the timestamps (start and end times) corresponding to each illegal advertisement text segment in the set, and combines the merged text with the individual timestamps to construct structured data. This structured data includes at least the illegal advertisement identifier, the merged illegal text content, a list of timestamps for each original segment, a violation type determination (e.g., "exaggerated efficacy," "impersonating an expert"), a confidence score, and text describing the analysis process generated by the large model. This structured data is output as the final result, for example, pushed to a manual review platform in JSON format or stored in a monitoring database.

[0072] By combining semantic similarity and temporal proximity, the system automatically identifies and merges fragments of the same illegal advertisement from different time periods, restoring a complete chain of evidence. This avoids judgment bias caused by fragmented information and reduces manual data piecing together during review. The constructed structured data is both machine-readable and human-traceable, allowing reviewers to quickly locate the original violation based on timestamps. This significantly improves the efficiency of review and law enforcement evidence collection, achieving a closed loop from automated discovery to assisted accurate judgment.

[0073] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following is combined with... Figure 2 The provided flowchart, using a 30-minute health lecture segment broadcast on a certain TV station as an example, illustrates the method flow of this invention in detail.

[0074] It should be noted that, Figure 2 The specific application scenario parameters shown (such as "original audio and video data of broadcast television", "duration ≥ 5 minutes", "number of words ≥ 100 words" and "false advertising of medical products") are only a preferred application example of the technical solution of the present invention in a specific business scenario. They are mainly used to help understand the core system architecture and data flow logic of the present invention, and do not limit the scope of protection of the audio and video target content recognition method that is generally applicable to the present invention.

[0075] First, this system acquires the raw audio and video data of the health lecture segment from the broadcast television signal, simultaneously activating both auditory and visual perception channels. The auditory channel uses Automatic Speech Recognition (ASR) technology to convert the audio content into text, recognizing phrases such as the host introducing "Today we have invited Professor Wang to explain diabetes knowledge," and subsequent statements from patients such as "My blood sugar really dropped after taking this product." The visual channel uses Optical Character Recognition (OCR) technology to extract subtitles from the video screen, such as the scrolling text at the bottom of the screen: "Consultation Hotline: 400-XXX-XXXX" and "Product Name: XX Blood Sugar Lowering Capsules." Both text streams carry timestamp information.

[0076] Next, the system performs segmentation processing on the merged joint text using two different granularities. One is based on duration, performing coarse-grained segmentation with a minimum length of 5 minutes. For example, a 30-minute program is divided into 6 text blocks of 5 minutes each. The other is based on word count, performing fine-grained segmentation with a minimum length of 100 words, for subsequent precise positioning.

[0077] The first level employs a coarse-grained analysis, with a first-level AI model concurrently detecting text blocks of at least 5 minutes in length. Taking the second 5-minute block as an example, it includes content such as "Professor Wang introducing product ingredients" and "patients recounting the effects of taking the product." The model analyzes the overall narrative logic of this text block, identifying a disguised marketing model that "uses expert lectures as a shell, patient testimonials as a means, and ultimately implies product efficacy." This block is determined to contain semantic features suggestive of false advertising for pharmaceutical products. Therefore, this block is marked as a suspected illegal advertising segment and proceeds to the second level of processing. Other normal news or weather forecast blocks are directly excluded, no longer consuming subsequent computing resources.

[0078] The second level employs fine-grained analysis, executed by a second-level AI model, focusing on a more refined text granularity (no less than 100 characters). The system segments the text within the suspected illegal advertising period into several fine-grained text blocks of approximately 100 characters each. The large model analyzes these blocks one by one, accurately locating specific semantic features of the violation: for example, from 00:12:30 to 00:12:45, it identifies exaggerated claims of "blood sugar returns to normal, stop using insulin"; from 00:14:20 to 00:14:35, it identifies purported expert testimony from "Professor Wang, a retired expert from a top-tier hospital"; and from 00:16:00 to 00:16:15, it identifies disguised consumer inducement statements such as "Call the number below the screen to order now." Each feature point is marked with an accurate timestamp, forming a segment of the illegal advertising text.

[0079] For the multiple illegal advertising text segments output by the secondary analysis, the system further merges advertising content with the same semantics. Although the three segments are scattered in different locations in time, they all point to the same product (XX hypoglycemic capsules) and are highly semantically related. By calculating semantic similarity (all involving "diabetes", "product efficacy", and "purchase guidance") and temporal proximity (all within 2 minutes), the system automatically aggregates them into a complete chain of evidence for the same false pharmaceutical advertisement, restoring the complete narrative from expert endorsements and patient testimonies to purchase appeals.

[0080] Finally, the system outputs the analysis results in a formatted manner. The output is structured data, including: the time period of the false medical advertisement (00:12:30-00:16:15), the merged content of the violation text, a list of timestamps for each violation, the type of violation (such as exaggerating efficacy, impersonating experts, or misleading consumers), the confidence score, and the analysis process text generated by the large model. This result can be directly used for manual review, law enforcement evidence collection, and monitoring data archiving.

[0081] Through the above process, this system demonstrates, using a health lecture as an example, the fully automated processing from raw audio and video input to structured evidence of violations, achieving a unified approach of efficient screening and accurate identification.

[0082] The following describes the multimodal illegal advertising content recognition system based on a large model provided in the embodiments of the present invention. The multimodal illegal advertising content recognition system based on a large model described below can be referred to in correspondence with the multimodal illegal advertising content recognition method based on a large model described above.

[0083] This invention provides a multimodal illegal advertising content identification system based on a large model. See [link to relevant documentation]. Figure 3 ,include: Module 310 is used to extract speech recognition text and subtitle recognition text from the audio and video stream to be processed, and to align and fuse them to construct joint text; The initial screening module 320 is used to segment the joint text according to a first granularity to obtain the first granularity segmented text, and to perform initial screening on the first granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising periods. Analysis module 330 is used to segment the text corresponding to the suspected illegal advertising period according to the second granularity to obtain the second granularity segmented text, and to perform feature analysis on the second granularity segmented text based on the second semantic analysis model to obtain the illegal advertising text segment, wherein the first granularity is greater than the second granularity; The merging module 340 is used to semantically merge the illegal advertising text segments, generate and output the illegal advertising analysis results.

[0084] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a multimodal illegal advertising content identification method based on a large model. The method includes: extracting speech recognition text and subtitle recognition text from the audio and video stream to be processed, and aligning and fusing them to construct a joint text; segmenting the joint text according to a first granularity to obtain first-granularity segmented text, and performing preliminary screening of the first-granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising periods; segmenting the text corresponding to the suspected illegal advertising periods according to a second granularity to obtain second-granularity segmented text, and performing feature analysis of the second-granularity segmented text based on a second semantic analysis model to obtain illegal advertising text segments, wherein the first granularity is greater than the second granularity; performing semantic merging on the illegal advertising text segments, generating and outputting the illegal advertising analysis results.

[0085] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multimodal illegal advertising content identification method based on a large model provided by the above methods. The method includes: extracting speech recognition text and subtitle recognition text from the audio and video stream to be processed, and aligning and fusing them to construct a joint text; segmenting the joint text according to a first granularity to obtain first-granularity segmented text, and performing preliminary screening of the first-granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising periods; segmenting the text corresponding to the suspected illegal advertising periods according to a second granularity to obtain second-granularity segmented text, and performing feature analysis of the second-granularity segmented text based on a second semantic analysis model to obtain illegal advertising text segments, wherein the first granularity is greater than the second granularity; performing semantic merging on the illegal advertising text segments, and generating and outputting illegal advertising analysis results.

[0087] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the multimodal illegal advertising content identification method based on a large model provided by the above methods. The method includes: extracting speech recognition text and subtitle recognition text from the audio and video stream to be processed, and aligning and fusing them to construct a joint text; segmenting the joint text according to a first granularity to obtain first-granularity segmented text, and performing preliminary screening of the first-granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising time periods; segmenting the text corresponding to the suspected illegal advertising time periods according to a second granularity to obtain second-granularity segmented text, and performing feature analysis of the second-granularity segmented text based on a second semantic analysis model to obtain illegal advertising text segments, wherein the first granularity is greater than the second granularity; performing semantic merging on the illegal advertising text segments, and generating and outputting illegal advertising analysis results.

[0088] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying multimodal illegal advertising content based on a large model, characterized in that, include: Extract speech recognition text and subtitle recognition text from the audio and video streams to be processed, and align and fuse them to construct joint text; The joint text is segmented according to a first granularity to obtain first granularity segmented text, and the first granularity segmented text is initially screened based on a first semantic analysis model to obtain suspected illegal advertising periods. The text corresponding to the suspected illegal advertising period is segmented according to the second granularity to obtain the second granularity segmented text. The second granularity segmented text is then subjected to feature analysis based on the second semantic analysis model to obtain the illegal advertising text paragraphs. The first granularity is greater than the second granularity. The semantics of the illegal advertising text segments are semantically merged to generate and output the illegal advertising analysis results.

2. The method according to claim 1, characterized in that, The process of extracting speech recognition text and subtitle recognition text from the audio and video stream to be processed, and then aligning and fusing them to construct joint text, includes: Obtain the timestamp corresponding to the speech-recognized text and the timestamp corresponding to the subtitle-recognized text; Based on the timestamp, the speech recognition text and the subtitle recognition text are matched and aligned in the time dimension; Analyze the semantic logical relationship between the aligned speech recognition text and the subtitle recognition text; Cross-modal feature fusion is performed based on the semantic logical association to generate the joint text with contextual information.

3. The method according to claim 1, characterized in that, The step of segmenting the joint text according to a first granularity to obtain first-granularity segmented text, and performing preliminary screening of the first-granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising periods, includes: Based on a first preset time length, the joint text is dynamically segmented to generate the first granular segmented text; The first granular segmented text is input into the first semantic analysis model to analyze the overall narrative logic of the first granular segmented text; Based on the overall narrative logic, non-target texts that do not contain target intent features are filtered out, and time periods are extracted from the first-granularity segmented texts that contain target intent features, which are then used as the suspected illegal advertising periods.

4. The method according to claim 1, characterized in that, The text corresponding to the suspected illegal advertising period is segmented according to a second granularity to obtain second-granularity segmented text. Then, feature analysis is performed on the second-granularity segmented text based on a second semantic analysis model to obtain illegal advertising text segments, including: Based on the second preset text length, the first granularity segmented text corresponding to the suspected illegal advertising period is segmented to generate the second granularity segmented text; The second granular segmented text is input into the second semantic analysis model to extract specific semantic feature points representing the target intent in the second granular segmented text; The specific semantic feature points are timestamped to obtain the illegal advertisement text paragraph.

5. The method according to claim 1, characterized in that, The step of semantically merging the illegal advertising text segments to generate and output the illegal advertising analysis results includes: The step of semantically merging the illegal advertising text segments to generate and output the illegal advertising analysis results includes: Calculate the semantic similarity and temporal proximity among the various segments of the illegal advertisement text; Based on the semantic similarity and temporal proximity, multiple related illegal advertising text segments are combined into an illegal advertising text segment set and then merged. Based on the merged set of illegal advertisement text segments, the illegal advertisement analysis results are generated and output.

6. The method according to claim 5, characterized in that, The process of generating and outputting the analysis results of the illegal advertisements based on the merged set of illegal advertisement text segments includes: Extract the timestamps corresponding to each illegal advertisement text segment in the set of illegal advertisement text segments; By combining the set of illegal advertisement text segments and the timestamp, structured data is constructed, and the structured data is output as the analysis result of the illegal advertisement.

7. The method according to claim 3, characterized in that, The step of dynamically segmenting the joint text based on a first preset time length to generate the first granularity segmented text includes: The joint text is initially segmented according to the first preset time length to obtain the first initial boundary point; Extract speech silence features or punctuation features within a preset time window centered on the first initial boundary point; Based on the speech silence feature or the punctuation mark feature, the first initial boundary point is slid to the adjacent semantic pause position for segmentation to generate the first granular segmented text.

8. The method according to claim 4, characterized in that, The step of segmenting the first-granularity segmented text corresponding to the suspected illegal advertising period based on the second preset text length to generate the second-granularity segmented text includes: The first-granularity segmented text corresponding to the suspected illegal advertising period is initially segmented according to the second preset text length to obtain the second initial boundary point; Extract speech silence features or punctuation features within a preset character window centered on the second initial boundary point; Based on the speech silence feature or punctuation feature, the second initial boundary point is slid to the adjacent semantic pause position for segmentation to generate the second granular segmented text.

9. A multimodal illegal advertising content identification system based on a large model, characterized in that, include: The module is used to extract speech recognition text and subtitle recognition text from the audio and video streams to be processed, and then align and fuse them to construct joint text; The initial screening module is used to segment the joint text according to a first granularity to obtain the first granularity segmented text, and to perform initial screening on the first granularity segmented text based on a first semantic analysis model to obtain suspected illegal advertising periods. The analysis module is used to segment the text corresponding to the suspected illegal advertising period according to the second granularity to obtain the second granularity segmented text, and to perform feature analysis on the second granularity segmented text based on the second semantic analysis model to obtain the illegal advertising text paragraph, wherein the first granularity is greater than the second granularity; The merging module is used to semantically merge the illegal advertising text segments, generate and output the illegal advertising analysis results.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the multimodal illegal advertising content identification method based on a large model as described in any one of claims 1 to 8.

Images