Marketing jargon violation segment detection method and device, electronic equipment and storage medium

Abstract

This application relates to a method, apparatus, electronic device, and storage medium for detecting marketing copy violation fragments. It is applied in the field of text recognition, such as in finance, consumer goods, and advertising. Through preprocessing, semantic compression, feature fusion, and backpropagation steps, it performs token-level violation fragment detection and localization, outputting the detection results. This scheme adds a semantic compression branch to refine core semantics and guides feature fusion through summary representation, outputting accurate violation fragment intervals. The semantic compression branch and the violation detection task have a positive promoting effect, significantly improving the convergence efficiency of the joint loss and enabling model training convergence within a short period. A global encoder parameter sharing mechanism is adopted, eliminating extra redundant parameters and effectively improving inference efficiency while ensuring consistency of training representations.

Description

Technical Field

[0001] This application belongs to the field of natural language processing, and in particular relates to a method, device, electronic device and storage medium for detecting illegal segments of marketing rhetoric. Background Technology

[0002] Taking the financial industry as an example, there is a prominent problem of hidden violations in the actual review of marketing texts: In order to circumvent compliance review, some illegal marketing entities will disguise themselves by splitting illegal keywords and illegal paragraphs, inserting irrelevant characters at intervals, and semantically separating them, so as to hide the illegal content in the lengthy marketing texts in a fragmented way. Conventional detection models are unable to accurately capture hidden illegal information.

[0003] Currently, existing technologies for detecting violations in financial marketing texts still have several core technical flaws, making it difficult to adapt to the high-precision, high-real-time compliance review requirements. They suffer from one or more of the following defects:

[0004] 1. The system is redundant and has low detection efficiency;

[0005] 2. Task association is missing, and there is no effective positive guidance mechanism.

[0006] 3. Multi-stage training (training the summarization model first, then the detection model) leads to poor parameter consistency, slow convergence of training loss, and difficulty in deployment. Summary of the Invention

[0007] The technical problem to be solved by this invention is to address the issues of coarse detection granularity, inaccurate violation location, or difficulty in identifying hidden violations in the existing technology for marketing script compliance detection scenarios, thereby providing a method, device, electronic device, and storage medium for detecting marketing script violation segments.

[0008] The technical solution adopted by this invention to solve its technical problem is:

[0009] A method for detecting marketing script violations includes the following steps:

[0010] The marketing script was preprocessed and then input into a shared encoder to obtain the original text encoding features. R is the set of real numbers, n is the length of the text token, and d is the dimension of the encoder's hidden layer;

[0011] Will The input is processed by the decoder, semantic compression is performed to generate a summary of the marketing message, and the summary features of the last layer output of the decoder are extracted. m is the length of the digest token;

[0012] right Feature purification and cross-attention fusion are performed, including: based on a dictionary of marketing rhetoric violation keywords... Perform token-level violation matching and reduce the weight of neutral and compliant tokens. After purification, the purified abstract features are obtained. After normalization ,by For query vectors, Cross-attention calculation is performed on the key vector and value vector to obtain the violation guidance feature. ,Will and The fusion resulted in characteristics of illegal guidance during the fusion process. and will The reverse transmission is then passed to the intermediate layer of the shared encoder;

[0013] Based on the original features, through the shared encoder Characteristics of illegal guidance Perform token-level violation fragment detection and location, and output the detection results.

[0014] In an optional embodiment, the method employs end-to-end joint training of a single model, simultaneously optimizing semantic compression and token-level violation detection tasks through a single shared encoder and a joint loss function, and directly outputting the detection result in a single forward inference.

[0015] In an optional embodiment, the joint loss function is formulated as follows: ,in: For dual task weights, Weighting for attention penalty Loss is generated for summarizing. Losses due to detection of illegal segments This is a penalty term for the uniformity of attention weights.

[0016] In an optional embodiment, the dictionary of marketing script violation keywords is used to... Token-level violation matching is performed, and the weight of neutral and compliant tokens is attenuated. This includes: the violation keyword dictionary is only used to identify clearly violating semantic units and is not used for compliance determination, resulting in a set of neutral and compliant tokens. ;

[0017] right Weight decay is applied while the characteristics of the illegitimate token remain unchanged. The weight decay formula is as follows: ,in: , This is the attenuation coefficient.

[0018] In an optional embodiment, the cross-attention calculation step includes:

[0019] Construct a cross-attention layer, number of attention heads Consistent with the number of attention heads in the shared encoder, Mapped to Q, The mapping is K and V, and the mapping matrix is ​​shared with the attention layer of the shared encoder;

[0020] The formula for attention score is: ;

[0021] Dynamic attention weight calculation formula: ;

[0022] Formula for calculating the characteristics of illegal guidance:

[0023] Formula for calculating the characteristics of illegal guidance and fusion: ;

[0024] Where: Q is the query matrix composed of original text features, and K and V are the key matrix and value matrix composed of cleaned summary features, respectively. For single-head attention feature dimensions, satisfying , For the number of attention heads; This is the dynamic semantic attention weight matrix.

[0025] In an optional embodiment, the semantic compression step includes: The input is a Transformer decoder, which uses an autoregressive generation method and a masked attention mechanism to prevent future token information leakage, generating a semantically compressed summary that focuses on core semantics, filters redundant information, and highlights key illegal content; the decoder uses the summary generation loss. Supervised training, The weighted sum of cross-entropy loss and ROUGE loss is used, as shown in the formula. , ( (Default 0.6) enables redundant information filtering and violation information reinforcement; among which, cross-entropy loss , For real digest tokens, To predict the summary token, the ROUGE loss is used to measure the consistency between the summary and the core semantics of the original text.

[0026] In an optional embodiment, the step of performing token-level violation fragment detection and location, and outputting the detection result includes:

[0027] Shared encoder will The input violation detection head employs a cascaded structure of a linear transform layer and a softmax activation layer for token-level classification. The classification labels utilize the BIO annotation system: O for normal tokens, B-Toxic for the start token of a violation fragment, and I-Toxic for the duration token of a violation fragment. The Viterbi algorithm decodes the token classification results, accurately locating the violation's start position (start_idx), end position (end_idx), content of the violation fragment, and violation type (promises of guaranteed profits, misleading advertising, exaggerated statements, etc.). Violation detection loss is calculated. Using cross-entropy loss, the formula is as follows: , For real token labels, To predict token labels.

[0028] A device for detecting marketing script violation segments includes:

[0029] The preprocessing module is used to preprocess the marketing script and input it into the shared encoder to obtain the raw text encoding features. R is the set of real numbers, n is the length of the text token, and d is the dimension of the encoder's hidden layer;

[0030] The semantic compression module is used to compress... The input is processed by the decoder, semantic compression is performed to generate a summary of the marketing message, and the summary features of the last layer output of the decoder are extracted. m is the length of the digest token;

[0031] The fusion module is used for... Feature purification and cross-attention fusion are performed, including: based on a dictionary of marketing rhetoric violation keywords... Perform token-level violation matching and reduce the weight of neutral and compliant tokens to achieve the following: For query vectors, cleaned-up summary features Cross-attention calculation is performed on the key vector and value vector to obtain the violation guidance feature. ,Will and The fusion resulted in characteristics of illegal guidance during the fusion process. and will The reverse transmission is then passed to the intermediate layer of the shared encoder;

[0032] The detection module, based on the original features of the shared encoder Characteristics of illegal guidance Perform token-level violation fragment detection and location, and output the detection results.

[0033] An electronic device includes a processor, a memory, and an executable program stored in the memory and executable by the processor, wherein the processor executes the steps of the marketing script violation detection method when running the executable program.

[0034] A storage medium having an executable program stored thereon, wherein the executable program, when executed by a processor, implements the steps of the marketing script violation segment detection method.

[0035] The beneficial effects of this invention are as follows: By performing preprocessing, semantic compression, feature fusion, and backpropagation steps, token-level violation fragment detection and localization are performed, and the detection results are output. This scheme adds a semantic compression branch to refine core semantics, guides feature fusion through summary representation, and outputs accurate violation fragment intervals; the semantic compression branch and the violation detection task have a positive promoting effect, significantly improving the convergence efficiency of the joint loss, and enabling model training convergence to be completed in a short period; a global encoder parameter sharing mechanism is adopted, with no additional redundant parameters, effectively improving inference efficiency while ensuring the consistency of training representations. Attached Figure Description

[0036] The technical solution of this application will be further described below with reference to the accompanying drawings and embodiments.

[0037] Figure 1 This is a schematic diagram of the marketing script violation detection method architecture in this embodiment;

[0038] Figure 2 This is a schematic diagram of the marketing script violation detection method in this embodiment;

[0039] Figure 3 This is a single-model inference structure diagram of this embodiment;

[0040] Figure 4 This is a schematic diagram of the attention fusion optimization process of the semantic summarization reverse guidance module in this embodiment. Detailed Implementation

[0041] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0042] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the scope of protection of this application. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. For example, if the description object is "field," then the ordinal numbers preceding "field" in "first field" and "second field" do not restrict the position or order of the "fields." "First" and "second" do not restrict whether the "fields" they modify are in the same message, nor do they restrict the order of "first field" and "second field." Similarly, if the description object is "level," then the ordinal numbers preceding "level" in "first level" and "second level" do not restrict the priority between "levels." Furthermore, if the description object is "information," then "first information" and "second information" can be the same information or different information, and their content can be the same or different.

[0043] The technical solution of this application will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] This embodiment provides a method for detecting prohibited segments of marketing scripts, such as... Figures 1-4 As shown, it includes the following steps:

[0045] The marketing script was preprocessed and then input into a shared encoder to obtain the original text encoding features. R is the set of real numbers, n is the length of the text token, and d is the dimension of the encoder's hidden layer;

[0046] Will The input is processed by the decoder, semantic compression is performed to generate a summary of the marketing message, and the summary features of the last layer output of the decoder are extracted. m is the length of the digest token;

[0047] right Feature purification and cross-attention fusion are performed, including: based on a dictionary of marketing rhetoric violation keywords... Perform token-level violation matching and reduce the weight of neutral and compliant tokens. After purification, the purified abstract features are obtained. After normalization ,by For query vector Query, Cross-attention is performed on the key vector (Key) and value vector (Value) to obtain the violation guidance feature. ,Will and The fusion resulted in characteristics of illegal guidance during the fusion process. and will The reverse transmission is then passed to the intermediate layer of the shared encoder;

[0048] Based on the original features, through the shared encoder Characteristics of illegal guidance Perform token-level violation fragment detection and location, and output the detection results.

[0049] The marketing script violation detection method in this embodiment is applied to the field of text recognition, such as finance, consumer goods, and advertising. It performs token-level violation detection and localization through preprocessing, semantic compression, feature fusion, and backpropagation steps, outputting the detection results. This scheme adds a semantic compression branch to refine core semantics and guides feature fusion through summary representation, outputting accurate violation segment intervals. The semantic compression branch and the violation detection task have a positive promoting effect, significantly improving the convergence efficiency of the joint loss and enabling model training convergence within a short period. A global encoder parameter sharing mechanism is adopted, eliminating redundant parameters and effectively improving inference efficiency while ensuring consistency in training representations.

[0050] The following is a comparison between this method and existing technologies:

[0051] In the prior art:

[0052] 1. The dual-model separation architecture results in system redundancy and low detection efficiency. The two independent models have a large number of redundant basic semantic understanding parameters, causing parameter resource redundancy; at the same time, the serial inference of the two models lengthens the computation chain, increases the inference process and time consumption, and the overall model has a large number of parameters and low computational efficiency, which cannot meet the business application requirements of real-time review and rapid risk control of financial marketing scripts.

[0053] 2. Lack of task correlation and effective positive guidance mechanism. The semantic compression task and the violation detection task operate independently, and the two tasks cannot complement each other or generate positive gains. The semantic compression module cannot provide the violation detection module with accurate core semantic features and effective violation clues, making it difficult for the model to identify fragmented and disguised hidden violations. The model has poor ability to distinguish and judge difficult violation samples, and its overall detection performance has obvious shortcomings.

[0054] 3. Existing violation detection technologies are generally limited by coarse-grained judgment paradigms, supporting only overall judgment at the paragraph or sentence level. When faced with long text marketing rhetoric, implicit violations, and soft borderline expressions, they are prone to missed detections and false detections due to a lack of local fine-grained perception.

[0055] The method implemented here is as follows:

[0056] 1. The encoder is shared by both the semantic compression and violation detection tasks throughout the process, requiring no additional redundant parameters. Its parameter updates are constrained by the losses of both tasks. This effectively improves inference efficiency while ensuring consistency in training representations.

[0057] 2. The abstract uses reverse semantic guidance to... and The fusion resulted in characteristics of illegal guidance during the fusion process. and will The signal is passed back to the intermediate layer of the shared encoder as a strong guiding signal for violation fragment detection. It participates in the subsequent feature iteration optimization and semantic representation update of the encoder, realizing deep interaction between the semantic information of the decoder summary and the original text features of the encoder. This further enhances the representation capability of violation features and improves the accuracy of violation fragment localization. The semantic compression branch and the violation detection task form a positive promoting effect, and the convergence efficiency of the joint loss is significantly improved, enabling the model training to converge within a short period of time.

[0058] Based on the optimized violation feature guidance module, the traditional fragmented mode of independent training for dual tasks or one-way empowerment has been completely abandoned, and a two-way correlation constraint mechanism between semantic compression generation and violation fragment detection has been established.

[0059] In terms of its specific mechanism of action, on the one hand, the feature purification mechanism accurately filters redundant information in the text, highlighting core content such as illegal semantic fragments. This provides high-purity, highly targeted, and strong feature guidance signals for downstream illegal fragment detection, significantly improving the model's ability to perceive and locate implicit violations and local illegal fragments in long texts, effectively reducing the probability of missed and false detections. On the other hand, the high-precision violation detection results serve as a powerful supervisory signal, which can conversely constrain the semantic summarization generation logic, prompting the summarization model to prioritize illegal fragments and eliminate redundant information, thereby improving the quality of semantic summarization generation. Through this closed-loop iterative architecture of "positive guidance and reverse constraint," the interaction of features and the collaborative updating of parameters for both tasks are realized. Ultimately, under the effect of the positive cyclic gain of the two tasks, the F1 score of illegal fragment detection is steadily improved by 5%~8%, while the ROUGE-L index of summarization generation is simultaneously improved by 3%~5%, achieving synchronous optimization and bidirectional upgrade of the performance of the two core tasks of semantic compression and illegal fragment detection.

[0060] 3. A semantic compression branch is added to refine the core semantics. Feature fusion is guided by summary representation to perform token-level violation fragment detection and localization, outputting accurate violation fragment intervals. By relying on semantic compression capabilities, disguised and fragmented violation content can be identified, overcoming the problem of concealed violations.

[0061] By leveraging the strong reverse guidance mechanism of semantic summarization and combining feature purification and cross-attention fusion optimization, we have achieved accurate extraction of violation features from coarse-grained to fragment-level fine-grained features.

[0062] Its core principle lies in: removing redundant background noise from the text through a feature purification mechanism, significantly increasing the distribution distance between the illegal semantic fragments and normal text in the feature space; based on this, using a cross-attention fusion mechanism, high-weight illegal clues in the semantic summary are accurately mapped to specific token positions in the original text, thereby achieving "targeted locking" and feature enhancement of implicit illegal keywords in long sequences. This mechanism not only effectively overcomes the limitations of traditional methods in perceiving local illegal features, but also fully adapts to various complex scenarios of detecting long and short financial marketing scripts, significantly improving the reliability and scenario adaptability of the model in implicit illegal identification.

[0063] 4. Feature cleanup and cross-attention dynamic fusion replace the traditional fixed linear fusion, dynamically allocate attention weights, significantly reduce the interference of non-violation semantic noise, and significantly improve the robustness of the model.

[0064] 5. Dictionary adaptation for prohibited words

[0065] By inputting and updating the dictionary, it meets the needs of specific industries. Taking the financial industry as an example, it can support common types of violations in financial marketing: illegal profit promises (such as "guaranteed principal and interest" or "guaranteed profit without loss"), misleading publicity (such as "expected return = fixed return"), exaggerated statements (such as "doubled returns"), illegal guarantees (such as "bank guarantee"), inappropriate analogies (such as "safer than deposits"), and regulatory circumvention (such as "circumventing interest rate restrictions"). It has a built-in dictionary of illegal keywords in financial marketing language, and the dictionary content can be flexibly expanded according to financial regulatory policies.

[0066] In an alternative embodiment, such as Figure 2 As shown, the method employs end-to-end joint training of a single model, simultaneously optimizing semantic compression and token-level violation detection tasks through a single shared encoder and a joint loss function, directly outputting the detection result in a single forward inference. Based on the single-model end-to-end training and single-inference paradigm, it is suitable for marketing script compliance detection scenarios.

[0067] In an optional embodiment, the joint loss function is formulated as follows: ,in: The weight is set to dual task weight (default 0.5). The attention penalty weight (default 0.1). Loss is generated for summarizing. Losses due to detection of illegal segments This is a penalty term for the uniformity of attention weights, used to guide the model to focus on the illegal token. The formula is as follows: W attn [i][j] represents the self-attention weights of the decoder's i-th output token with respect to the j-th input token in the original text (after Softmax normalization, W). attn [i] is the sum of the model's attention weights for the violation tokens. This loss design forces the model to focus its attention on the violation tokens in the original text, achieving "detecting where the violation is, and extracting content from the violation fragment in the summary," thus solving the pain points of model attention drift and missed detection of violation content in risk control summaries.

[0068] In an optional embodiment, the dictionary of marketing script violation keywords is used to... Token-level violation matching is performed, and the weight of neutral and compliant tokens is attenuated. This includes: the violation keyword dictionary is only used to identify clearly violating semantic units and is not used for compliance determination, resulting in a set of neutral and compliant tokens. ;

[0069] right Weight decay is applied while the characteristics of the illegitimate token remain unchanged. The weight decay formula is as follows: ,in: , This is the attenuation coefficient.

[0070] In this solution, the dictionary of prohibited keywords only locks down prohibited units and does not reverse-check compliance, thus avoiding the mistaken elimination of compliant words by the rule dictionary and solving the flaw of the traditional one-size-fits-all judgment in sensitive word libraries. It reduces the feature weight of compliant / irrelevant tokens in the decoder generation process and strengthens the feature expression weight of prohibited keywords, allowing the model to accurately focus on the prohibited semantics for constraint optimization during training, thereby reducing the probability of large models generating prohibited marketing rhetoric.

[0071] In an optional embodiment, the purified summary features Perform L2 normalization, the formula is as follows Ensure that it matches the original text encoding features. Consistent norms (all norms are 1). This step uniformly scales the cleaned Decoder features to the unit norm, ensuring that the encoded and decoded features fall within the same feature metric space, so that subsequent feature fusion and cross-attention calculation steps are not affected by amplitude.

[0072] In an optional embodiment, the cross-attention calculation step includes:

[0073] Construct a cross-attention layer, number of attention heads Consistent with the number of attention heads in the encoder, Mapped to Q (Query). The mapping is K (Key) and V (Value), and the mapping matrix is ​​shared with the attention layer of the encoder, reducing the number of new parameters;

[0074] The formula for attention score is: ;

[0075] Dynamic attention weight calculation formula: ;

[0076] Formula for calculating the characteristics of illegal guidance:

[0077] Formula for calculating the characteristics of illegal guidance and fusion: ;

[0078] Where: Q is the query matrix composed of original text features, and K and V are the key matrix and value matrix composed of cleaned summary features, respectively. For single-head attention feature dimensions, satisfying , For the number of attention heads; This is the dynamic semantic attention weight matrix.

[0079] This scheme uses Softmax normalization to obtain dynamic attention weights. . During the model generation phase, priority is given to the semantics of prohibited keywords, and during training, targeted constraints are applied to the generation of extreme words / false advertising rhetoric.

[0080] In an optional embodiment, dynamic guidance features are used. Compared with the original text encoding features Perform linear fusion, followed by ReLU activation function ( After that, high-quality guided fusion characteristics were obtained. This embodiment uses ReLU as the activation function for the fusion layer. One reason is that it can truncate the negative invalid components in the fusion features, retaining only the positive features that carry violation guidance information, thus focusing on the semantics of risk control objectives.

[0081] In an optional embodiment, the semantic compression step includes: The input is a Transformer decoder, which uses an autoregressive generation method and a masked attention mechanism to prevent future token information leakage, generating a semantically compressed summary that focuses on core semantics, filters redundant information, and highlights key illegal content; the decoder uses the summary generation loss. Supervised training, The weighted sum of cross-entropy loss and ROUGE loss is used, as shown in the formula. , ( (Default 0.6) enables redundant information filtering and violation information reinforcement; among which, cross-entropy loss , For real digest tokens, To predict the summary token, the ROUGE loss is used to measure the consistency between the summary and the core semantics of the original text.

[0082] This scheme employs a dual-loss complementary approach: balancing local word generation with global key semantic retention, it accurately achieves the semantic compression goal of filtering redundancy and highlighting illegal content. The decoder uses an autoregressive generation method, constructing the output sequence through iterative steps. When generating each word in the semantic summary, the model uses the previously generated word sequence as a contextual constraint, combined with the violation features of the source text as guiding signals, to predict and expand the current summary content word by word. This autoregressive mechanism allows the decoder to dynamically plan subsequent expressions based on the preceding context, much like human writing. This not only effectively avoids logical gaps in the generated content but also closely aligns with the fine-grained features provided by the violation detection task, achieving high-quality, highly focused semantic summary output.

[0083] In an optional embodiment, the step of performing token-level violation fragment detection and location, and outputting the detection result includes:

[0084] Shared encoder will The input violation detection head employs a cascaded structure of a linear transform layer and a softmax activation layer for token-level classification. The classification labels utilize the BIO annotation system: O for normal tokens, B-Toxic for the start token of a violation fragment, and I-Toxic for the duration token of a violation fragment. The Viterbi algorithm decodes the token classification results, accurately locating the violation's start position (start_idx), end position (end_idx), content of the violation fragment, and violation type (promises of guaranteed profits, misleading advertising, exaggerated statements, etc.). Violation detection loss is calculated. Using cross-entropy loss, the formula is as follows: , For real token labels, To predict token labels.

[0085] This solution leverages the fusion features of the shared encoder to classify BIO system tokens and decodes them using Viterbi. It can accurately locate the position and content of various types of advertising violation segments at low cost, and ensures detection accuracy by relying on cross-entropy loss.

[0086] This embodiment provides an operational example:

[0087] Text type: Bank wealth management marketing script, fund promotion copy, financial investment advisor copy (Example: "Our bank's newly launched wealth management product guarantees principal and interest, with an expected return of up to 8%, a sure-fire way to make money, and even safer than deposits").

[0088] Step 1: Financial text input and shared encoder encoding

[0089] Preprocessing: Encode using Hugging Face's Bert Tokenizer, padding / truncation to 512 tokens, removal of special characters and invalid characters, and output of a text tensor. (For batch processing) Batch size is 32 by default.

[0090] Shared encoder: Employs a BERT-base-chinese pre-trained model (12 layers, 768 hidden layers, 12 attention heads, dropout probability set to 0.1). Through a multi-layer attention mechanism and a feedforward neural network, it outputs the original text encoding features. The module's parameters are shared by both semantic summarization and violation fragment detection tasks, and parameter updates are subject to joint loss. constraint.

[0091] Step 2: Semantic Compression Generation

[0092] Decoder: A 6-layer Transformer decoder is used (768 hidden layer dimensions, 12 attention heads, and dropout probability set to 0.1), employing an autoregressive generation method;

[0093] Objective: Generate a digest of 64-128 tokens (default 96 tokens), retaining the core marketing message and key points of violation (example digest: "Bank wealth management products advertise guaranteed principal and interest, expected return of 8%, guaranteed profit, safer than deposits").

[0094] loss: , The default value is 0.6, where For cross-entropy loss, For ROUGE-L loss, during training Convergence to below 0.15. A built-in abstract quality assessment unit evaluates abstract quality in real time using the ROUGE-L value, ensuring the integrity of the core semantics and highlighting any violations.

[0095] Step 3: Abstract semantic reverse guidance

[0096] 1. Feature purification

[0097] A dictionary of prohibited financial marketing tactics: It includes prominent prohibited keywords such as "guaranteed principal and interest," "guaranteed profit," "expected returns achievable," and "safer than deposits" (containing 2000+ core words of financial violations, covering 8 types of violations including misleading marketing, illegal promises and guarantees of principal, exaggerated statements and extreme words, risk downplaying and concealment, emotional incitement and a sense of urgency, unfair competition and comparisons, non-compliant qualifications and endorsements, and illegal lead generation and qualification confusion).

[0098] Neutral and compliant token weight decay Values ​​range from 0.1 to 0.3, with a default of 0.2. In the example, neutral or compliant token characteristics such as "bank, wealth management, product, and promotion" are multiplied by 0.2.

[0099] L2 normalization: This method is used to normalize the features of the purified abstract. implement The normalized norm is 1.

[0100] 2. Cross-attention fusion

[0101] Number of attention heads =12, =768 / 12=64;

[0102] →Query( ), purification summary →Key( Value );

[0103] Attention score calculation: The score matrix is ​​obtained. ;

[0104] Softmax normalization: This yields dynamic attention weights;

[0105] Dynamic guidance features: Output ;

[0106] Fusion and Activation: Output after ReLU activation .

[0107] Output The data is fed into the encoder's intermediate layers (layers 5-7, with layer 6 as the default) for detecting illegal segments.

[0108] Step 4: Detection of Violational Fragments

[0109] The token-level multi-classification of infringing texts adopts the BIO annotation system:

[0110] Tags: O (normal), B-Toxic (violation start), I-Toxic (violation continuation);

[0111] Detection head: Linear layer (768 inputs, 3 outputs) + Softmax, Viterbi decoding;

[0112] Output results (example): Illegal fragment 1 "Guaranteed principal and interest" (start_idx=8, end_idx=11, type=illegal profit promise, confidence level=0.92); Illegal fragment 2 "Guaranteed profit" (start_idx=20, end_idx=23, type=illegal profit promise, confidence level=0.91); Illegal fragment 3 "Safer than bank deposits" (start_idx=28, end_idx=33, type=misleading advertising, confidence level=0.88).

[0113] Step 5: End-to-end joint training

[0114] Training environment: GPU (NVIDIA RTX 3090), PyTorch framework, batch_size=32, learning rate 1e-5, AdamW optimizer;

[0115] Total loss: ,in This is a penalty item for attention-related issues;

[0116] Training rhythm: The cross-attention layer is frozen for the first 10 epochs, and only the encoder and decoder are trained; after 10 epochs, the layer is unfrozen and updated in full; a built-in training monitoring unit monitors the training loss, validation set F1 score and ROUGE-L score in real time. When the validation set F1 score does not improve for 5 consecutive epochs, training is automatically stopped and the optimal model parameters are saved.

[0117] Reasoning: Single forward reasoning, simultaneously outputting an example summary and the detection results of the aforementioned violation fragments. Supports JSON format output for easy integration with financial institution compliance review systems; a built-in result visualization unit can intuitively display the location and violation type of the violation fragment in the original text.

[0118] This method achieves inference time of ≤3.0s for 100 texts, meeting the real-time review requirements of financial marketing scripts (processing 30+ texts per second). The results include a compressed summary of the financial marketing scripts (96 tokens in length, ROUGE-L value ≥0.75) and the location of the toxic span (including the toxic span text, start_idx, end_idx, violation type, and confidence level, with a confidence level ≥0.85).

[0119] This embodiment provides a marketing script violation detection system, configured as follows:

[0120] System input: Financial marketing scripts / copywriting / advertising slogans / promotional materials (supports batch input of txt, excel, and json formats, with a maximum of 128 entries per batch);

[0121] System output:

[0122] Concise compliance summary (96 characters per entry, ROUGE-L value ≥ 0.75);

[0123] List of prohibited clips:

[0124] Excerpt: Original text of the violation;

[0125] Position (start / end): The starting / ending position of the token in the original text of the offending segment;

[0126] Type of violation: Corresponding financial violation category (illegal profit promises, misleading advertising, etc.);

[0127] Confidence level: Model prediction confidence level (≥0.85).

[0128] Deployment methods: Supports GPU (NVIDIA series) and CPU deployment, compatible with TensorRT and ONNX formats, provides API interface (RESTful API) and can be directly connected to the existing risk control and review system of financial institutions; supports real-time review (response time ≤3.0s / 100 records) and batch review (128 records / batch, processing time ≤4.0s), and has a built-in back-end management interface to view review records and export test results (Excel / JSON format).

[0129] This embodiment also provides a device for detecting marketing script violation segments, including:

[0130] The preprocessing module is used to preprocess the marketing script and input it into the shared encoder to obtain the raw text encoding features. R is the set of real numbers, n is the length of the text token, and d is the dimension of the encoder's hidden layer;

[0131] The semantic compression module is used to compress... The input is processed by the decoder, semantic compression is performed to generate a summary of the marketing message, and the summary features of the last layer output of the decoder are extracted. m is the length of the digest token;

[0132] The fusion module is used for... Feature purification and cross-attention fusion are performed, including: based on a dictionary of marketing rhetoric violation keywords... Perform token-level violation matching and reduce the weight of neutral and compliant tokens to achieve the following: For query vectors, cleaned-up summary features Cross-attention calculation is performed on the key vector and value vector to obtain the violation guidance feature. ,Will and The fusion resulted in characteristics of illegal guidance during the fusion process. and will The reverse transmission is then passed to the intermediate layer of the shared encoder;

[0133] The detection module, based on the original features of the shared encoder Characteristics of illegal guidance Perform token-level violation fragment detection and location, and output the detection results.

[0134] This embodiment also provides an electronic device, including a processor, a memory, and an executable program stored in the memory and capable of being run by the processor. When the processor runs the executable program, it performs the steps of the marketing script violation segment detection method.

[0135] This embodiment also provides a storage medium on which an executable program is stored, wherein the executable program, when executed by a processor, implements the steps of the marketing script violation segment detection method.

[0136] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0137] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0138] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0139] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting prohibited segments of marketing rhetoric, characterized in that, Includes the following steps: The marketing language is preprocessed and input into a shared encoder to obtain original text coding features , R is a real number set, n is the length of a text token, and d is the dimension of a hidden layer of the encoder Will Input decoder, perform semantic compression generation to obtain the summary of marketing language, and extract the summary features of the last layer output of the decoder , m is the length of the summary Token right Feature purification and cross-attention fusion are performed, including: based on a dictionary of marketing rhetoric violation keywords... Perform token-level violation matching and reduce the weight of neutral and compliant tokens. After purification, the purified abstract features are obtained. After normalization ,by For query vectors, Cross-attention calculation is performed on the key vector and value vector to obtain the violation guidance feature. ,Will and After fusion, the characteristics of illegal fusion guidance were obtained. and will The reverse transmission is then passed to the intermediate layer of the shared encoder; By the shared encoder, based on the original features With the violation guide features , execute Token-level violation segment detection and positioning, output detection results.

2. The marketing-speech violation segment detection method of claim 1, wherein, The method employs end-to-end joint training of a single model, simultaneously optimizing semantic compression and token-level violation detection tasks through a single shared encoder and a joint loss function, and directly outputting the detection result with a single forward inference.

3. The method for detecting marketing script violation segments according to claim 2, characterized in that, The formula for the joint loss function is: ,in: For dual task weights, Weighting for attention penalty Loss is generated for summarizing. Losses due to detection of illegal segments This is a penalty term for the uniformity of attention weights.

4. The method for detecting marketing script violation segments according to any one of claims 1-3, characterized in that, The dictionary of marketing rhetoric violation keywords Token-level violation matching is performed, and the weight of neutral and compliant tokens is attenuated. This includes: the violation keyword dictionary is only used to identify clearly violating semantic units and is not used for compliance determination, resulting in a set of neutral and compliant tokens. ; right Weight decay is applied while the characteristics of the illegitimate token remain unchanged. The weight decay formula is as follows: ,in: , This is the attenuation coefficient.

5. The method for detecting marketing script violation segments according to any one of claims 1-3, characterized in that, The steps for calculating cross-attention include: Construct cross attention layer, number of attention heads Consistent with the number of shared encoder attention heads, Map to Q, Map to K and V, the mapping matrix is shared with the attention layer of the shared encoder; The attention score formula is: ; Dynamic attention weight calculation formula: ; Misconduct guiding feature calculation formula: The violation guidance fusion feature calculation formula is: ; Where: Q is the query matrix composed of original text features, and K and V are the key matrix and value matrix composed of cleaned summary features, respectively. For single-head attention feature dimensions, satisfying , For the number of attention heads; This is the dynamic semantic attention weight matrix.

6. The method for detecting marketing script violation segments according to any one of claims 1-3, characterized in that, The The steps for generating semantic compression by inputting the decoder include: The input is a Transformer decoder, which uses an autoregressive generation method and a masked attention mechanism to prevent future token information leakage; the decoder generates loss using a digest. Supervised training, The weighted sum of cross-entropy loss and ROUGE loss is used, as shown in the formula. Among them, cross-entropy loss , For real digest tokens, To predict the summary token, the ROUGE loss is used to measure the consistency between the summary and the core semantics of the original text.

7. The method for detecting marketing script violation segments according to any one of claims 1-3, characterized in that, The steps for performing token-level violation fragment detection and location, and outputting the detection results include: Shared encoder will The input violation detection head employs a cascaded structure of a linear transform layer and a softmax activation layer for token-level classification. The classification labels utilize the BIO annotation system: O for normal tokens, B-Toxic for the start of a violation, and I-Toxic for the continuation of a violation. The Viterbi algorithm decodes the token classification results to accurately pinpoint the start and end positions of the violation, the content of the violation, and the violation type. The violation detection loss is calculated accordingly. Using cross-entropy loss, the formula is as follows: , For real token labels, To predict token labels.

8. A marketing script violation snippet detection apparatus characterized by, include: The preprocessing module is used to preprocess the marketing script and input it into the shared encoder to obtain the raw text encoding features. R is the set of real numbers, n is the length of the text token, and d is the dimension of the hidden layer of the shared encoder; a semantic compression module, configured to compress an input decoder, configured to perform semantic compression generation to obtain an abstract of the marketing language, and extract abstract features output by a last layer of the decoder , m is the length of the abstract Token; The fusion module is used for... Feature purification and cross-attention fusion are performed, including: based on a dictionary of marketing rhetoric violation keywords... Perform token-level violation matching and reduce the weight of neutral and compliant tokens to achieve the following: For query vectors, cleaned-up summary features Cross-attention calculation is performed on the key vector and value vector to obtain the violation guidance feature. ,Will and The fusion resulted in characteristics of illegal guidance during the fusion process. and will The reverse transmission is then passed to the intermediate layer of the shared encoder; The detection module, based on the original features of the shared encoder Characteristics of illegal guidance Perform token-level violation fragment detection and location, and output the detection results.

9. An electronic device comprising a processor, a memory, and an executable program stored in the memory and executable by the processor, characterized in that, When the processor runs the executable program, it performs the steps of the marketing script violation segment detection method as described in any one of claims 1-7.

10. A storage medium having an executable program stored thereon, characterized in that, When the executable program is executed by the processor, it implements the steps of the marketing script violation segment detection method as described in any one of claims 1-7.

Images