A multi-modal content review method and system for an internet platform

Abstract

The application provides a kind of multi-modal content auditing method and system for internet platform, it is related to internet platform content auditing technical field, first receive the multi-modal mixed publishing unit containing visual picture frame sequence, audio sampling point sequence and text character sequence and with same time stamp mark, then the multi-modal mixed publishing unit is carried out cross-channel content conflict detection, generates conflict identification set, according to conflict degree hierarchical identification and position index information call auditing rule scheduling library, extract auditing strategy programming instruction set, according to auditing strategy programming instruction set start distributed auditing processor cluster to specific interval content compliance analysis, generate non-compliance event positioning label sequence, bind non-compliance event positioning label sequence with publisher account identifier, generate intercept response instruction to prevent content from entering public display area.The application improves the accuracy and comprehensiveness of auditing, and safeguards platform content safety compliance.

Description

Technical Field

[0001] This invention relates to the field of content moderation technology for internet platforms, and more specifically, to a multimodal content moderation method and system for internet platforms. Background Technology

[0002] Internet platforms have become crucial venues for information dissemination and exchange, with massive amounts of content being published and disseminated daily. This content takes various forms, encompassing visual images, audio, and text. Traditional content moderation methods on internet platforms often focus on reviewing a single modality of content independently. For example, for video content, only the video itself might be reviewed, ignoring the accompanying audio and subtitles; for audio content, the focus is solely on the audio itself, disregarding any related visual or textual information.

[0003] However, single-modal content moderation has significant drawbacks. In practice, multimodal content exhibits complex relationships and mutual influences. Visuals, audio, and text can collectively convey a complete message; reviewing only a portion of them can easily overlook some violations. For example, some inappropriate information might evade review by cleverly combining visuals and audio, or mislead users by exploiting inconsistencies between text and visuals. Moreover, with the increasing complexity and diversity of internet content, the presentation of violations has become more covert and varied, making traditional moderation methods insufficient to meet the current accuracy and comprehensiveness requirements of internet platforms for content moderation. Summary of the Invention

[0004] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a multimodal content review method for internet platforms, the method comprising: The system receives a stream of content to be reviewed transmitted from an internet platform. The stream of content to be reviewed includes a sequence of visual frames, an audio sampling point sequence, and a text character sequence that have the same timestamp. The visual frame sequence, audio sampling point sequence, and text character sequence together constitute a multimodal hybrid publishing unit. The multimodal hybrid publishing unit is subjected to cross-channel content conflict detection processing to generate a cross-channel content conflict identifier set. The cross-channel content conflict identifier set is used to record the location index information and conflict degree classification identifier of the content matching anomaly between the visual channel and the auditory channel. The pre-built audit rule scheduling library is invoked based on the conflict level classification identifier and location index information recorded in the cross-channel content conflict identifier set. The audit strategy orchestration instruction set corresponding to the conflict level classification identifier is extracted. The audit strategy orchestration instruction set includes an activation order description of at least one audit processor unit and the compliance parsing parameter configuration that the audit processor unit needs to load. The activation order description in the set of audit strategy orchestration instructions is used to sequentially start multiple target audit processor units in the distributed audit processor cluster to perform content compliance parsing processing on the frame interval, audio sampling interval and text character interval corresponding to the location index information in the multimodal hybrid publishing unit, and generate a non-compliance event location tag sequence. The non-compliance event location tag sequence includes the start timestamp, end timestamp and non-compliance type code of the non-compliance phenomenon. The non-compliant event location tag sequence is bound to the publisher account identifier associated with the multimodal hybrid publishing unit to generate a platform interception response instruction. The platform interception response instruction is then sent to the platform content distribution gateway to prevent the multimodal hybrid publishing unit from entering the public content display area. The platform interception response instruction carries the publisher account identifier and the non-compliant event location tag sequence.

[0005] Furthermore, embodiments of the present invention also provide a multimodal content moderation system for internet platforms, comprising: A processor; a machine-readable storage medium for storing machine-executable instructions of the processor; wherein the processor is configured to perform the aforementioned multimodal content moderation method for an internet platform by executing the machine-executable instructions.

[0006] Based on the above, by receiving a multimodal hybrid publishing unit containing a sequence of visual frames, audio sampling points, and text characters with the same timestamp, cross-channel content conflict detection processing is performed on the multimodal hybrid publishing unit to generate a set of cross-channel content conflict identifiers. This allows for the timely detection of abnormal content matching between the visual and auditory channels, effectively identifying potential combinations of non-compliant content. Based on the set of cross-channel content conflict identifiers, the review rule scheduling library is invoked to extract the corresponding review strategy orchestration instruction set, achieving dynamic and accurate matching of review strategies. The target review processor units in the distributed review processor cluster are sequentially activated through the review strategy orchestration instruction set, performing compliance parsing processing on content within specific intervals and generating a sequence of non-compliant event location tags. This allows for the location of the time, scope, and type of non-compliant phenomena, significantly improving the accuracy and efficiency of the review process. Finally, the non-compliant event location tag sequence is bound to the publisher's account identifier to generate a platform interception response instruction, preventing non-compliant content from entering the public content display area. This ensures the security and compliance of internet platform content from the source, improving the quality and level of internet platform content review. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the execution flow of the multimodal content review method for Internet platforms provided in an embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of exemplary hardware and software components of a multimodal content moderation system for an internet platform provided in an embodiment of the present invention. Detailed Implementation

[0009] Figure 1 This is a flowchart illustrating a multimodal content review method for internet platforms provided in one embodiment of the present invention, which will be described in detail below.

[0010] Step S110: Receive the content stream to be reviewed transmitted from the Internet platform. The content stream to be reviewed includes a sequence of visual frames, an audio sampling point sequence, and a text character sequence with the same timestamp. The visual frame sequence, audio sampling point sequence, and text character sequence together constitute a multimodal hybrid publishing unit.

[0011] In a content moderation scenario applied to an internet content platform, a user publishes a comprehensive post containing video footage, background music, and text description via a mobile client. Step S110 receives the content stream to be reviewed from the internet platform. This content stream contains a sequence of visual frames, an audio sampling point sequence, and a text character sequence, all with the same timestamp. The visual frame sequence consists of consecutive image frames extracted by a video codec at a fixed frame rate, with each frame having a red, green, and blue three-channel pixel matrix. The audio sampling point sequence consists of time-domain waveform data stored in pulse code modulation format, with a sampling frequency of 44,100 sampling points per second. The text character sequence consists of user-inputted text content such as titles, descriptions, and comments, stored using Unicode encoding. The above visual frame sequence, audio sampling point sequence, and text character sequence together constitute a multimodal hybrid publishing unit. All data collection and processing comply with relevant laws and regulations, and user privacy information is anonymized.

[0012] Step S120: Perform cross-channel content conflict detection processing on the multimodal hybrid publishing unit to generate a cross-channel content conflict identifier set. The cross-channel content conflict identifier set is used to record the location index information and conflict degree classification identifier of the content matching anomaly between the visual channel and the auditory channel.

[0013] Step S121: Perform a screen region segmentation operation on the visual image frame sequence in the multimodal hybrid publishing unit. Divide each visual image frame into multiple non-overlapping screen segmentation region units based on the color similarity and spatial adjacency of each pixel block in the image. Statistically analyze the texture distribution within the screen segmentation region units. Calculate the direction consistency parameter and texture thickness uniformity parameter within each screen segmentation region unit. Combine the direction consistency parameter and texture thickness uniformity parameter into a region visual description tuple for that screen segmentation region unit.

[0014] For each frame in the visual image frame sequence, a simple linear iterative clustering superpixel segmentation algorithm is used. The Euclidean distance of pixel color values ​​in the red-green-blue color space is used as a similarity measure, while constraining the adjacency relationship of pixel spatial positions. The image is divided into multiple non-overlapping superpixel blocks, each superpixel block serving as a screen segmentation region unit. For each screen segmentation region unit, the gradient direction of each pixel within that unit is calculated. The gradient direction is obtained by calculating the arctangent of the horizontal and vertical gradients using the Sobel operator. A direction histogram is generated for all pixel gradient directions, and the direction with the highest frequency in the histogram is taken as the dominant texture direction. The absolute value of the angular deviation between the texture direction and the dominant texture direction for each pixel within the screen segmentation region unit is calculated, and the arithmetic mean of all absolute angular deviations is used as the direction consistency parameter. For the texture thickness uniformity parameter, the local standard deviation of the grayscale value of each pixel within each screen segmentation region unit and the grayscale values ​​of its neighboring pixels are calculated, and the variance of all local standard deviations is used as the texture thickness uniformity parameter. The directional consistency parameter and the uniformity of change parameter are combined into a two-dimensional vector, which serves as the region visual description tuple for the segmented region unit of the image.

[0015] Step S122: Perform frame-segmentation and windowing processing on the audio sampling point sequence in the multimodal hybrid publishing unit, dividing the continuous audio sampling point sequence into multiple short-time audio analysis frames with a preset overlap ratio. For each short-time audio analysis frame, extract the spectral energy fluctuation envelope curve along the frequency axis. Based on the fluctuation envelope curve, identify the peak prominence and valley depth in each short-time audio analysis frame. Use the ratio between the peak prominence and valley depth as the spectral fluctuation change parameter of the short-time audio analysis frame, and arrange the spectral fluctuation change parameters of each short-time audio analysis frame along the time axis to form a spectral fluctuation change pattern sequence.

[0016] The audio sampling point sequence is framed, with a frame length of L sampling points and an overlap of M sampling points between adjacent frames. A Hamming window function is applied to each frame of audio data, expressed as w(n) = 0.54 - 0.46 * cos(2πn / (L-1)), where n is the sampling point index. A Fast Fourier Transform is performed on each windowed frame of audio data to obtain the spectral amplitude distribution. The spectral amplitude distribution is smoothed using a moving average filter to remove high-frequency noise. In the smoothed spectral amplitude distribution, local maxima are extracted as spectral peaks, and local minima are extracted as spectral valleys. For each spectral peak, its prominence is calculated, which is the amplitude value of the peak minus the average amplitude of its two adjacent valleys. For each spectral valley, its depth is calculated, which is the average amplitude of its two adjacent peaks minus the amplitude value of the valley. The arithmetic mean of the prominence of all spectral peaks is taken as the average prominence of the frame, and the arithmetic mean of the depth of all spectral valleys is taken as the average depth of the frame. The ratio of the average prominence of the peaks to the average depth of the valleys is taken as the spectral fluctuation parameter of the short-time audio analysis frame. All short-time audio analysis frames are arranged in chronological order to form a sequence of spectral fluctuation patterns.

[0017] Step S123: Perform word segmentation on the text character sequence in the multimodal hybrid publishing unit to obtain multiple semantic lexical units, and count the frequency and position of each semantic lexical unit in the text character sequence.

[0018] Chinese word segmentation is performed on the text character sequence using a Conditional Random Field (CRF) model based on a bidirectional Long Short-Term Memory (LSTM) network. This CRF model comprises a bidirectional LSM layer and a CRF output layer. The bidirectional LSM layer processes the character sequence in both forward and backward directions, capturing contextual dependencies. The CRF output layer performs sequence labeling on the output of the bidirectional LSM layer, marking each character as the beginning, middle, end, or single-character word. Based on the labeling results, the continuous character sequence is segmented into independent lexical units. After segmentation, stop words and punctuation marks are filtered out, retaining nouns, verbs, adjectives, and other semantic lexical units. All semantic lexical units are traversed, and the total number of occurrences of each semantic lexical unit in the entire text character sequence is counted as its frequency. The character offset of each occurrence position in the text character sequence is also recorded.

[0019] Step S124: Use a pre-trained word embedding mapping network to convert each semantic word unit into a fixed-length semantic feature vector, calculate the spatial distance metric between the semantic feature vectors corresponding to different semantic word units, merge semantic word units with spatial distance metric values ​​less than a preset threshold into the same semantic cluster, calculate the average vector of the semantic feature vectors of all members of each semantic cluster, and take the semantic word unit with the closest spatial distance between the average vector and the semantic feature vectors of each member as the semantic representation word of the semantic cluster, and generate a list of semantic representation words.

[0020] A pre-trained word embedding mapping network is loaded. This network adopts a continuous bag-of-words model architecture, containing an input layer, a projection layer, and an output layer. The input layer receives the one-hot encoded representation of the real-meaning vocabulary units, and the dimension of the one-hot encoded vector is equal to the vocabulary size. The projection layer multiplies the one-hot encoded vector of the input layer with an embedding matrix to obtain a low-dimensional dense semantic feature vector. The embedding matrix has a dimension of V x D, where V is the vocabulary size and D is the embedding dimension, with D set to 300. The output layer is used during the training phase and not during the inference phase. The training process of this word embedding mapping network is as follows: the training data comes from a large-scale Chinese corpus containing approximately 1 billion vocabulary units. During training, the context words of the target word are used as input to predict the target word. A negative sampling method is used to optimize the loss function, and the optimizer uses the stochastic gradient descent algorithm. The initial learning rate is set to 0.025, and the training epochs are 5. During the inference phase, the one-hot encoded representation of each real-meaning vocabulary unit is input into the word embedding mapping network, and the corresponding 300-dimensional semantic feature vector is directly extracted from the projection layer. For all semantic units obtained in step S123, after acquiring their respective semantic feature vectors, calculate the Euclidean distance between the semantic feature vectors of any two semantic units. Initialize a semantic cluster set, treating each semantic unit as an independent semantic cluster. Traverse all pairs of semantic units; if the Euclidean distance between two semantic units is less than a preset clustering threshold, merge the semantic clusters containing these two semantic units. Repeat the merging process until no new merges occur. For each final semantic cluster, calculate the arithmetic mean of the semantic feature vectors of all semantic units within that semantic cluster to obtain the average vector. Within that semantic cluster, calculate the Euclidean distance between the semantic feature vector of each semantic unit and the average vector, and select the semantic unit with the smallest distance as the semantic representation word of that semantic cluster. Sort the semantic representation words of all semantic clusters in descending order of the number of members within the cluster to generate a list of semantic representation words.

[0021] Step S125: Based on the timestamp of the visual frame and the timestamp of the audio sampling point sequence in the multimodal hybrid publishing unit, establish the correspondence between the frame interval and the audio sampling interval. For any frame segmentation unit appearing in any frame interval, extract the abnormal fluctuation segment whose fluctuation amplitude exceeds the average fluctuation level in the spectral fluctuation change pattern sequence in the corresponding audio sampling interval of the frame interval. Determine whether the time span of the audio sampling interval where the abnormal fluctuation segment is located overlaps with the continuous dwell time period of the frame segmentation unit in the frame. If they overlap, associate and pair the regional visual description tuple of the frame segmentation unit with the spectral fluctuation change parameter of the abnormal fluctuation segment in the overlapping time period to generate a set of frame audio association pairing records.

[0022] The timestamps of the visual frame sequence and the audio sampling point sequence are read from the metadata of the multimodal hybrid publishing unit, and a mapping relationship from frame index to absolute time coordinate is established. For each frame in the visual frame sequence, the audio sampling point interval corresponding to the frame is determined according to its timestamp. Each screen segmentation unit is traversed, and the frame index of the first and last occurrence of the screen segmentation unit is recorded. The frame index range is converted into a time interval as the continuous dwell time period of the screen segmentation unit. Within the audio sampling interval corresponding to the screen segmentation unit, the spectral fluctuation change pattern sequence generated in step S122 is obtained. The arithmetic mean of all spectral fluctuation change parameters within the audio sampling interval is calculated as the average fluctuation level. Continuous segments with spectral fluctuation change parameter values ​​greater than the average fluctuation level are marked as abnormal fluctuation segments. It is determined whether the time interval of the abnormal fluctuation segment overlaps with the continuous dwell time period of the screen segmentation unit. If there is an overlap, the regional visual description tuple of the screen segmentation unit and the spectral fluctuation change parameters of the abnormal fluctuation segment in the overlapping time period are extracted, and the two are associated and paired to form a screen-audio association pairing record.

[0023] Step S126: Extract the most frequently occurring semantic word unit from the semantic representation word list within the text character interval corresponding to the overlapping time period. Perform cross-channel content consistency calculation between the semantic feature vector of the semantic word unit and the regional visual description tuple in the video-audio association pairing record. When the correlation between the semantic feature vector and the regional visual description tuple is lower than a preset threshold, generate a cross-channel content conflict entry containing the starting position index of the overlapping time period, the ending position index of the overlapping time period, and the conflict degree classification identifier.

[0024] Based on the time coordinates of the overlapping time period, locate the text character interval corresponding to the overlapping time period in the text character sequence. Within this text character interval, find the frequency of occurrence of the semantic lexical units counted in step S123, and extract the semantic lexical unit with the highest frequency. Obtain the semantic feature vector of this semantic lexical unit from the semantic representation lexical list generated in step S124. Perform cross-channel content consistency calculation on the regional visual description tuples and semantic feature vectors in the image-audio association pairing record. The consistency calculation uses the cosine similarity method, combining the direction consistency parameter and change uniformity parameter in the regional visual description tuple into a two-dimensional vector, reducing the dimensionality of the semantic feature vector to a two-dimensional space through principal component analysis, and calculating the cosine value of the angle between the two two-dimensional vectors as the degree of association. When the degree of association is lower than a preset threshold, it is determined that there is a content conflict between the visual channel and the auditory channel within the overlapping time period. According to the different degrees of conflict, the conflict degree classification is divided into three levels: when the degree of association is lower than the first threshold but higher than the second threshold, it is a mild conflict; when it is lower than the second threshold but higher than the third threshold, it is a moderate conflict; and when it is lower than the third threshold, it is a severe conflict. Generate a cross-channel content conflict entry, which includes the start position index of the overlapping time period, the end position index of the overlapping time period, and the corresponding conflict level classification identifier.

[0025] Step S127: After traversing all video frame intervals and audio sampling intervals, summarize the cross-channel content conflict entries obtained to generate the cross-channel content conflict identifier set.

[0026] Repeat steps S125 and S126 for each frame in the visual image frame sequence and each audio interval in the audio sampling point sequence. Arrange all generated cross-channel content conflict entries in ascending order according to the starting position index to form a cross-channel content conflict identifier set. Each entry records the location information and severity of the content matching anomaly between the visual channel and the auditory channel.

[0027] Step S130: Based on the conflict level classification identifier and location index information recorded in the cross-channel content conflict identifier set, call the pre-built review rule scheduling library to extract the review strategy arrangement instruction set corresponding to the conflict level classification identifier.

[0028] Step S131: Parse each cross-channel content conflict entry in the cross-channel content conflict identifier set to obtain the overlapping time period start position index, overlapping time period end position index, and conflict degree classification identifier recorded therein.

[0029] Iterate through each cross-channel content conflict entry in the cross-channel content conflict identifier set generated in step S127. Read three fields from each entry: the start position index of the overlapping period, which indicates the time coordinate at which the conflict begins; the end position index of the overlapping period, which indicates the time coordinate at which the conflict ends; and the conflict severity rating identifier, which takes the values ​​of mild conflict, moderate conflict, or severe conflict.

[0030] Step S132: Compare the conflict level classification identifier with the conflict level mapping table pre-stored in the audit rule scheduling library, and determine the target scheduling strategy number corresponding to the conflict level classification identifier based on the comparison result. Each record in the conflict level mapping table contains a correspondence between a conflict level classification identifier and a scheduling strategy number.

[0031] A pre-built audit rule scheduling library is constructed, which contains a conflict level mapping table. The conflict level mapping table is a key-value pair data structure, where the key is the conflict severity level identifier and the value is the scheduling strategy number. The conflict severity level identifier obtained in step S131 is used as the query key to perform an exact match search in the conflict level mapping table to find the corresponding scheduling strategy number. For example, a minor conflict identifier maps to the scheduling strategy number S001, a moderate conflict identifier maps to the scheduling strategy number S002, and a severe conflict identifier maps to the scheduling strategy number S003.

[0032] Step S133: Based on the target scheduling strategy number, find the corresponding strategy orchestration instruction storage address in the strategy orchestration index table in the audit rule scheduling library. The strategy orchestration index table records the mapping relationship between each scheduling strategy number and the offset of its corresponding strategy orchestration instruction set in the storage medium.

[0033] The audit rule scheduling library also includes a policy orchestration index table. This index table records the physical storage location information of each scheduling policy number and its corresponding policy orchestration instruction set in the storage medium, specifically including the starting sector number, offset in bytes, and data block length. Using the target scheduling policy number obtained in step S132, a search is performed in the policy orchestration index table to obtain the storage address of the policy orchestration instruction corresponding to that policy number, i.e., the starting read position and data length of the policy orchestration instruction set in the storage medium.

[0034] Step S134: Read the original data of the policy orchestration instruction set, which consists of multiple audit processor control instructions, from the audit rule scheduling library according to the policy orchestration instruction storage address. The audit processor control instructions include the type identifier field of the audit processor unit, the activation order field of the audit processor unit, and the execution parameter configuration field of the audit processor unit.

[0035] Based on the starting read position and data length obtained from the policy orchestration instruction storage address in step S133, a binary data block of a specified length is read from the storage medium of the audit rule scheduling library. This binary data block is deserialized and parsed according to a predefined instruction format to obtain the original data of a policy orchestration instruction set consisting of multiple audit processor control instructions. Each audit processor control instruction contains three fields: a type identifier field, used to specify the type of audit processor unit to be activated; an activation order field, used to define the order in which each audit processor unit is awakened; and an execution parameter configuration field, used to store the parameters required for the execution of this audit processor unit.

[0036] Step S135: Extract the type identifier field of each audit processor control instruction from the original data of the policy orchestration instruction set, and confirm whether there is a corresponding physical audit processor unit entity in the distributed audit processor cluster for the audit processor unit type represented by the type identifier field.

[0037] Iterate through each audit processor control instruction in the original data of the policy orchestration instruction set parsed in step S134. Extract the type identifier field of the instruction. Query the registry of the distributed audit processor cluster, which maintains metadata for all deployed physical audit processor units, including unit type, health status, and current load. Match the extracted type identifier field with the unit type in the registry to confirm whether there exists at least one physical audit processor unit in an available state. If not, trigger an alarm process and wait for resource release or call a backup unit.

[0038] Step S136: Parse the activation order field of each audit processor control instruction in the original data of the policy orchestration instruction set to determine the wake-up order of the target audit processor unit and the waiting time interval parameter between adjacent activation steps.

[0039] Extract the activation sequence field from the raw data of each audit processor control instruction in the strategy orchestration instruction set. The activation sequence field is a composite structure containing two parts: a wake-up sequence number, indicating which audit processor unit is activated; and a wait time interval parameter, indicating the time required between completing the current activation step and executing the next activation step. Sort all audit processor control instructions in ascending order of their wake-up sequence numbers to obtain the activation sequence of the target audit processor units. Simultaneously, record the wait time interval parameter between each activation step and its subsequent steps.

[0040] Step S137: Extract the execution parameter configuration field of each review processor control instruction from the original data of the strategy orchestration instruction set. The execution parameter configuration field defines the frame interval selection rule, audio sampling interval selection rule, and text character interval selection rule used by the corresponding review processor unit when performing content compliance parsing processing.

[0041] Extract the execution parameter configuration field for each audit processor control instruction from the raw data of the strategy orchestration instruction set. This execution parameter configuration field is a structured object containing three sub-rules: a frame interval selection rule, which defines how to extract the frame interval to be processed from the visual frame sequence, including the calculation formulas for the start and end frame offsets; an audio sampling interval selection rule, which defines how to extract the audio interval to be processed from the audio sampling point sequence, including the calculation formulas for the start and end sampling point offsets; and a text character interval selection rule, which defines how to extract the character interval to be processed from the text character sequence, including the calculation formulas for the start and end character offsets.

[0042] Step S138: Fill the overlapping time period start position index and overlapping time period end position index in the cross-channel content conflict entry into the frame interval selection rule, audio sampling interval selection rule and text character interval selection rule of the execution parameter configuration field to generate a concrete compliance parsing parameter configuration for the cross-channel content conflict entry.

[0043] Extract the overlapping time period start index S and overlapping time period end index E from the cross-channel content conflict entry parsed in step S131. Substitute these start index S and end index E into the frame interval selection rules, audio sampling interval selection rules, and text character interval selection rules extracted in step S137, respectively. Specifically, fill in the frame interval selection rules with S as the start frame offset and E as the end frame offset; fill in the audio sampling interval selection rules with S as the start sampling point offset and E as the end sampling point offset; fill in the text character interval selection rules with S as the start character offset and E as the end character offset. After the rules are filled in, generate a concrete compliance parsing parameter configuration for the cross-channel content conflict entry.

[0044] Step S139: Sort the concrete compliance parsing parameter configurations after filling according to the wake-up order defined by the activation order field, generate an audit processor unit parameter configuration queue corresponding to each activation step, and combine the audit processor unit parameter configuration queue with the corresponding activation order field and type identifier field to generate a set of audit strategy orchestration instructions for the cross-channel content conflict items.

[0045] The concrete compliance parsing parameter configuration generated in step S138 is associated with the type identifier field extracted in step S135 and the activation order field extracted in step S136. Following the wake-up sequence number defined in the activation order field from smallest to largest, the type identifier field, activation order field, and their corresponding concrete compliance parsing parameter configurations for each review processor control instruction are arranged sequentially to form a parameter configuration queue. Each element in this parameter configuration queue corresponds to an activation step. This parameter configuration queue is then output as a set of review strategy orchestration instructions for the current cross-channel content conflict item.

[0046] Step S140: Based on the activation order description in the audit strategy orchestration instruction set, multiple target audit processor units in the distributed audit processor cluster are sequentially activated to perform content compliance parsing processing on the screen frame interval, audio sampling interval, and text character interval corresponding to the location index information in the multimodal hybrid publishing unit, and generate a non-compliance event location tag sequence.

[0047] Step S141: Parse the type identifier field and execution parameter configuration field corresponding to the first activation step in the set of review strategy orchestration instructions. Send an activation signal to the visual content review processor unit in the distributed review processor cluster according to the type identifier field. At the same time, pass the part of the image frame interval selection rule in the execution parameter configuration field to the visual content review processor unit so that after receiving the activation signal, the visual content review processor unit can extract the corresponding image frame interval subsequence from the visual image frame sequence of the multimodal hybrid publishing unit according to the overlapping time period start position index and overlapping time period end position index specified in the image frame interval selection rule.

[0048] The instruction element corresponding to the first activation step is retrieved from the set of orchestration instructions generated in step S139. The type identifier field in this instruction element is parsed to identify the target review processor unit to be activated as the visual content review processor unit. An activation signal is sent to the visual content review processor unit via the message bus of the distributed review processor cluster. The activation signal contains the concrete compliance parsing parameter configuration corresponding to the instruction element. After receiving the activation signal, the visual content review processor unit extracts the frame interval selection rule from the parameter configuration. This frame interval selection rule includes the overlapping time period start position index S_frame and the overlapping time period end position index E_frame. Based on S_frame and E_frame, the visual content review processor unit extracts all visual images from the visual image frame sequence of the multimodal hybrid publishing unit from the S_frame to the E_frame, forming a frame interval subsequence.

[0049] Step S142: Call the pre-loaded violation visual pattern matching network inside the visual content review processor unit to perform sensitive visual element retrieval for each frame visual image in the subsequence of the frame interval. The violation visual pattern matching network consists of multiple parallel convolution extraction channels, and each convolution extraction channel corresponds to a different category of visual violation feature template.

[0050] Step S1421: Perform a color space conversion operation on each visual image in the subsequence of the image frame interval, convert the original red, green and blue color representation into hue, saturation and luminance color representation, and perform uniform quantization processing on the hue channel to compress the total number of color categories. Then, input the quantized hue channel, saturation channel and luminance channel into the three independent input branches of the illegal visual pattern matching network. The three independent input branches each contain a sequence of convolutional filter layers with convolutional receptive regions of different sizes.

[0051] For each visual frame in the subsequence of image frames, a color space transformation operation is first performed, converting the pixel values ​​of the red, green, and blue channels into pixel values ​​of the hue, saturation, and brightness channels. The hue channel represents the type of color, the saturation channel represents the purity of the color, and the brightness channel represents the lightness or darkness of the color. The hue channel is then uniformly quantized, discretizing continuous hue values ​​into a fixed number of intervals, each represented by a representative value, to compress the total number of color categories. The quantized hue, saturation, and brightness channels are then input into three independent input branches of the violation visual pattern matching network. The first input branch receives the hue channel, and its convolutional filter layer sequence uses small-sized convolutional kernels to capture subtle changes in color boundaries. The second input branch receives the saturation channel, and its convolutional filter layer sequence uses medium-sized convolutional kernels to capture transition features in color regions. The third input branch receives the brightness channel, and its convolutional filter layer sequence uses large-sized convolutional kernels to capture the overall pattern of brightness changes.

[0052] Step S1422: In the three independent input branches, the input features are extracted using the convolutional filter layer sequence of the first convolutional receptive region to extract local edge direction and corner distribution features, resulting in a local detail feature mapping group. The input features are also extracted using the convolutional filter layer sequence of the second convolutional receptive region to extract regional color transition gradient and texture density distribution features, resulting in a regional structure feature mapping group. The size of the second convolutional receptive region is larger than the size of the first convolutional receptive region.

[0053] In the hue and saturation input branches, a sequence of convolutional filtering layers with small kernels is first used to scan the input features, extracting edge direction information and corner position information of local regions in the image. Each convolutional kernel slides on the input feature map, and the dot product of the kernel weight and the pixel value of the covered area is calculated to obtain the feature response value at that location. Through the stacking of multiple layers of small convolutional kernels, the network can capture hierarchical features from simple edges to complex corners, and organize these feature response values ​​into a set of local detail feature maps. In the luminance input branch, a sequence of convolutional filtering layers with large kernels is used to scan the input features, extracting color transition gradient information and the density and arrangement of texture primitives over a large area of ​​the image. Large convolutional kernels have a larger receptive field, which can aggregate contextual information from the surrounding area and output response values ​​that reflect regional structural features. These response values ​​are organized into a set of regional structural feature maps.

[0054] Step S1423: Connect and merge the local detail feature mapping group and the regional structure feature mapping group along the feature channel direction to generate a multi-level visual feature descriptor of the frame. Perform sliding window similarity calculation on the multi-level visual feature descriptor and multiple illegal feature templates stored in the illegal visual pattern matching network to obtain a set of similarity response values ​​of each illegal feature template at different spatial positions in the multi-level visual feature descriptor.

[0055] The image local detail feature maps and image region structure feature maps obtained from the three independent input branches are concatenated along the feature channel dimension. The merged feature body contains multi-level visual information from local edges to region structures, forming a multi-level visual feature descriptor for the frame. The violation visual pattern matching network pre-stores multiple violation feature templates, each corresponding to a type of violation visual content to be detected. The template itself is also a feature body with the same format as the multi-level visual feature descriptor. For each violation feature template, a sliding window scan is performed on the multi-level visual feature descriptor. The similarity between the feature descriptor sub-block and the template is calculated at each window position, using dot product or cosine distance. After traversing all spatial positions, a set of similarity response values ​​corresponding to the violation feature template is obtained. The level of the response value indicates the probability that the corresponding violation visual content exists at that position.

[0056] Step S1424: For each violation feature template, select the peak response position and peak response value that exceed the preset response activation threshold from the corresponding similarity response value set, and use the peak response value as the violation confidence metric value of the violation feature template in the visual image of the frame. Arrange the violation confidence metric values ​​of the same violation feature template in multiple consecutive visual images of the frame interval subsequence along the time axis to form the single template time-series response waveform of the violation feature template.

[0057] For each set of similarity response values ​​corresponding to a violation feature template, a response activation threshold is set. All values ​​in the response value set are traversed, and local maxima exceeding the threshold are selected. Each local maximum corresponds to a peak response position and its peak response value. This peak response value is used as the violation confidence metric for the violation feature template in the current frame's visual image. The above process is repeated for each frame in the frame interval subsequence to obtain a sequence of violation confidence metrics for the violation feature template across multiple consecutive frames. This sequence is arranged frame-by-frame along the time axis to form the single-template temporal response waveform of the violation feature template. Peaks in the waveform indicate a high probability of the violation visual content occurring within that time period, and the height of the peaks represents the degree of confidence in the violation.

[0058] Step S1425: Perform a weighted summation on the single-template time-series response waveforms corresponding to all violation feature templates in the violation visual pattern matching network. The weights are determined by the violation severity coefficient associated with each violation feature template. The summation result forms the first time-series response waveform.

[0059] In the violation visual pattern matching network, each violation feature template is associated with a preset violation severity coefficient, which reflects the severity of the violation. For example, the severity coefficient for violent content is higher than that for vulgar content. The temporal response waveforms of all violation feature templates are weighted and summed according to their respective severity coefficients. For each time point, the violation confidence value of all templates at that time point is multiplied by their respective severity coefficients and then summed to obtain the comprehensive violation response value for that time point. The comprehensive violation response values ​​of all time points are arranged in chronological order to form the first temporal response waveform. This first temporal response waveform integrates the detection results of multiple violation types, and can more comprehensively reflect the overall situation of visual violations in the subsequence of frame intervals.

[0060] Step S143: The violation feature response heat values ​​output by the violation visual pattern matching network for each frame visual image in the subsequence of the frame interval are fused to obtain a first time-series response waveform describing the time of occurrence and duration of visual violations in the subsequence of the frame interval. The start and end timestamps of the visual violation are determined according to the time of the peak and the half-width of the peak in the first time-series response waveform, and the category code of the visual violation is recorded as the first non-compliance type code.

[0061] For each frame in the subsequence of frame intervals, the five parallel convolutional extraction channels of the violation visual pattern matching network output a violation confidence value. These five confidence values ​​are weighted and summed according to the violation severity weights of their respective channels to obtain the comprehensive violation feature response heatmap value for that frame. The comprehensive violation feature response heatmap values ​​of all frames are arranged in frame order to form the first temporal response waveform. Peak detection is performed on this waveform to find local maxima as peaks. Each peak corresponds to a visual violation event. For each peak, the timestamp of its corresponding frame is recorded as the start timestamp of the visual violation. Extending outwards from the peak position, when the waveform value drops to 50% of the peak value, the timestamp at that position is recorded as the end timestamp of the visual violation. The violation category code corresponding to the peak (obtained from the channel with the highest output confidence) is recorded as the first non-compliance type code.

[0062] Step S144: Parse the type identifier field and execution parameter configuration field corresponding to the second activation step in the set of review strategy orchestration instructions. Send an activation signal to the auditory content review processor unit in the distributed review processor cluster according to the type identifier field. At the same time, pass the part of the audio sampling interval selection rule in the execution parameter configuration field to the auditory content review processor unit so that after receiving the activation signal, the auditory content review processor unit can extract the corresponding audio sampling interval subsequence from the audio sampling point sequence of the multimodal hybrid publishing unit according to the overlapping time period start position index and overlapping time period end position index specified in the audio sampling interval selection rule.

[0063] The instruction element corresponding to the second activation step is retrieved from the instruction set of the review strategy orchestration. The type identifier field in this instruction element is parsed to identify the target review processor unit to be activated as the auditory content review processor unit. An activation signal is sent to the auditory content review processor unit via the message bus of the distributed review processor cluster. This activation signal contains the concrete compliance parsing parameter configuration corresponding to the instruction element. Upon receiving the activation signal, the auditory content review processor unit extracts the audio sampling interval selection rule from the parameter configuration. This rule includes the overlapping time period start position index S_audio and the overlapping time period end position index E_audio. Based on S_audio and E_audio, the auditory content review processor unit extracts audio data from the audio sampling point sequence of the multimodal hybrid publishing unit from the S_audio-th sampling point to the E_audio-th sampling point, forming an audio sampling interval subsequence.

[0064] Step S145: Call the illegal acoustic pattern recognition network pre-loaded inside the auditory content review processor unit to perform sensitive acoustic event detection on the audio sampling interval subsequence. The illegal acoustic pattern recognition network determines whether there are pre-labeled illegal audio segments by analyzing the Mel frequency cepstral coefficient sequence of the audio sampling interval subsequence.

[0065] Step S1451: Pre-emphasize the subsequence of the audio sampling interval to enhance the spectral resolution of the high-frequency part of the audio. Perform short-time framing and windowing operations on the audio sampling point sequence after pre-emphasis processing to obtain an audio analysis frame group. For each audio analysis frame in the audio analysis frame group, calculate its logarithmic energy value after passing through the Mel frequency filter bank to obtain the Mel frequency cepstral coefficient feature vector. Arrange the Mel frequency cepstral coefficient feature vectors of all audio analysis frames in chronological order to form the Mel frequency cepstral coefficient feature matrix.

[0066] A pre-emphasis filter is applied to the subsequence of the audio sampling interval. This pre-emphasis filter enhances the energy of the high-frequency components of the signal, compensates for the attenuation of high-frequency components during propagation, and enhances spectral resolution. The pre-emphasis processed audio sampling point sequence is divided into multiple short-time audio analysis frames according to a fixed frame length and frame shift. Adjacent frames overlap to ensure continuity between frames. A window function is applied to each audio analysis frame to reduce spectral leakage. A short-time Fourier transform is performed on each windowed audio analysis frame to transform it from the time domain to the frequency domain, obtaining the spectral amplitude distribution of that frame. This spectral amplitude distribution is passed through a set of Mel frequency filters, with each filter outputting the accumulated energy value within its passband. The logarithm of the output energy value of each filter is taken to obtain the logarithmic energy spectrum. A discrete cosine transform is performed on the logarithmic energy spectrum to obtain the Mel frequency cepstral coefficient eigenvectors. The Mel frequency cepstral coefficient eigenvectors of all audio analysis frames are stacked in chronological order to form a two-dimensional matrix, namely the Mel frequency cepstral coefficient feature matrix, where rows correspond to time frames and columns correspond to feature dimensions.

[0067] Step S1452: Input the Mel frequency cepstral coefficient feature matrix into the first layer convolutional temporal processing module of the illegal acoustic pattern recognition network, and use a one-dimensional convolutional filter kernel sliding along the time axis to extract the changing trend features of the Mel frequency cepstral coefficient feature vector between adjacent audio analysis frames.

[0068] The first layer of the violation acoustic pattern recognition network is a convolutional temporal processing module, which uses a one-dimensional convolutional filter kernel that slides along the time axis. The Mel-frequency cepstral coefficient feature matrix is ​​treated as a time series, with each time step corresponding to a feature vector. The one-dimensional convolutional kernel slides along the time dimension, each kernel covering the feature vectors of several consecutive time steps. The dot product of the kernel weights and the feature values ​​within the covered region is calculated, outputting a scalar value. Through multiple different one-dimensional convolutional kernels, it can extract the changing trend features of the Mel-frequency cepstral coefficient feature vectors between adjacent audio analysis frames, such as the rate of increase, rate of decrease, and fluctuation patterns of feature values. The output after convolutional processing still retains the temporal structure, forming a temporal feature map.

[0069] Step S1453: Pass the output of the first layer convolutional timing processing module to the second layer cyclic timing processing module. The second layer cyclic timing processing module includes a memory update unit with a gating mechanism. The memory update unit calculates the hidden state vector output at the current time based on the input features at the current time and the hidden state vector at the previous time.

[0070] The output of the first convolutional temporal processing module is passed to the second recurrent temporal processing module, which consists of multiple cascaded gated recurrent units. Each gated recurrent unit maintains a hidden state vector that encodes all historical information from the beginning of the sequence to the current time step. At each time step, the gated recurrent unit receives the input features of the current time step and the hidden state vector of the previous time step. It determines how much historical information to retain through the update gate and how much historical information to ignore through the reset gate. Then, it calculates the candidate hidden states for the current time step and finally combines the output of the update gate and the candidate hidden states to generate the final hidden state vector for the current time step. Through this gating mechanism, the recurrent temporal processing module can capture long-range dependencies in the audio sequence, that is, the correlation patterns between the current audio event and the preceding and following context.

[0071] Step S1454: The output of the second-layer cyclic timing processing module is passed to the third-layer attention focusing module. The third-layer attention focusing module generates a focusing weighting coefficient sequence for the hidden state vector at each time step by calculating the inner product similarity between the hidden state vector and the trainable query vector at each time step of the second-layer cyclic timing processing module.

[0072] The output of the second-layer recurrent temporal processing module is a sequence of hidden state vectors, where each hidden state vector encodes the acoustic information of that moment and its context. The third-layer attention focusing module introduces a trainable query vector, which is continuously optimized during training to ultimately learn the feature patterns most relevant to the inappropriate audio segment. The attention focusing module calculates the inner product similarity between the hidden state vector and the trainable query vector at each moment, and then normalizes the similarity values ​​across all moments into a probability distribution using the Softmax function, resulting in a focus-weighted coefficient sequence. Each coefficient in this focus-weighted coefficient sequence represents the importance of the corresponding hidden state vector in determining whether the entire audio segment contains inappropriate content.

[0073] Step S1455: Use the focus weighting coefficient sequence to perform weighted summation on the hidden state vectors at each time step of the second layer cyclic temporal processing module to obtain an aggregated hidden state vector representing the overall acoustic content of the audio sampling interval subsequence. Input the aggregated hidden state vector into the classification output layer of the violation acoustic pattern recognition network. The classification output layer consists of a fully connected operation structure and an activation function. Calculate the probability distribution of the aggregated hidden state vector belonging to each pre-labeled violation audio segment category.

[0074] Each coefficient in the weighted coefficient sequence is multiplied by the corresponding hidden state vector at that time step. Then, all weighted hidden state vectors are summed to obtain an aggregated hidden state vector. This aggregated hidden state vector integrates information from all time steps within the entire audio sampling interval subsequence and highlights acoustic features within key time periods based on attention weights, providing a compact representation of the overall content of the audio segment. This aggregated hidden state vector is input into a classification output layer, which consists of one or more fully connected layers. Each fully connected layer undergoes a linear transformation followed by an activation function to introduce non-linearity. The number of output nodes in the final fully connected layer is the same as the number of categories of violating audio segments. The output values ​​are converted into a probability distribution using a Softmax activation function, where the probability value for each category represents the likelihood that the audio segment belongs to that category.

[0075] Step S1456: Determine the sensitive acoustic event type of the audio sampling interval subsequence according to the category of the illegal audio segment corresponding to the maximum probability value in the probability distribution, and backtrack to the peak position of the focus weighting coefficient sequence output by the third-layer attention focusing module on the time axis.

[0076] The system identifies the category index corresponding to the highest probability value from the probability distribution output by the classification output layer. This category index is then mapped to a specific name of the offending audio segment type, such as violent audio, pornographic audio, or politically sensitive audio, which serves as the sensitive acoustic event type for that audio sampling interval subsequence. Simultaneously, it traces back to the focus weighting coefficient sequence output by the third-layer attention focusing module, finding the position with the highest coefficient value. This position corresponds to the time region within the audio sampling interval subsequence that contributes the most to the classification decision.

[0077] Step S1457: Based on the peak position of the focus weighting coefficient sequence on the time axis and the time span covered by its decay to half peak, determine the start and end boundaries of the sensitive acoustic event in the subsequence of the audio sampling interval, map the start and end boundaries back to the absolute time coordinates of the audio sampling point sequence, obtain the start and end timestamps of the auditory violation, and encode and record the sensitive acoustic event type as the second non-compliance type code.

[0078] Centered on the peak position of the weighted coefficient sequence, a search is performed to the left and right to find the position where the coefficient value decays to half of the peak value. The left half-peak position corresponds to the start boundary of the sensitive acoustic event, and the right half-peak position corresponds to the end boundary of the sensitive acoustic event. These start and end boundaries are mapped from the sequence number of the audio analysis frame back to the sampling point positions in the original audio sampling point sequence. Then, based on the sampling rate, the sampling point positions are converted into absolute time coordinates to obtain the start and end timestamps of the auditory violation. The sensitive acoustic event type determined in step S1456 is converted into an encoded value according to a preset encoding rule and recorded as the second non-compliant type code.

[0079] Step S146: Smooth the acoustic event probability curve obtained by passing the audio sampling interval subsequence through the illegal acoustic pattern recognition network, extract the time interval in the acoustic event probability curve where the probability value exceeds the preset judgment threshold as the start and end timestamps of the auditory violation, and record the category code of the auditory violation as the second non-compliance type code.

[0080] The output layer of the violation acoustic pattern recognition network outputs the probability distribution for each category at each time step. For each violation category, a curve showing the probability value of that category changing over time is extracted. This curve is then smoothed using median filtering to remove isolated noise points. A decision threshold is set; when the probability curve value exceeds this threshold, an auditory violation is considered to exist at that moment. The time intervals that continuously exceed the decision threshold are extracted, and the start time of this interval is recorded as the start timestamp of the auditory violation, and the end time as the end timestamp. The violation category with the highest probability value within this time interval is encoded and recorded as the second non-compliance type code.

[0081] Step S147: Fuse the first time-series response waveform and the acoustic event probability curve, calculate the ratio of the overlapping area of ​​the first time-series response waveform and the acoustic event probability curve on the time axis to the total duration. When the ratio is higher than a preset fusion threshold, take the smaller of the start timestamp of the visual violation and the start timestamp of the auditory violation as the joint non-compliance start timestamp, take the larger of the end timestamp of the visual violation and the end timestamp of the auditory violation as the joint non-compliance end timestamp, and merge the first non-compliance type code and the second non-compliance type code into a joint non-compliance type code to generate a non-compliance event location tag containing the joint non-compliance start timestamp, the joint non-compliance end timestamp, and the joint non-compliance type code. The non-compliance event location tags collected after all activation steps are completed constitute the non-compliance event location tag sequence.

[0082] Align the first temporal response waveform obtained in step S143 with the acoustic event probability curve obtained in step S146 on the same time axis. Calculate the time region where both curve values ​​simultaneously exceed their respective judgment thresholds; this time region is the overlapping region. Calculate the ratio R_overlap, the total duration of the overlapping region to the total duration of the first temporal response waveform. Compare R_overlap with a preset fusion threshold. If R_overlap is greater than the preset fusion threshold, then the visual violation and auditory violation are determined to belong to the same joint violation event. Take the smaller value between the visual violation start timestamp and the auditory violation start timestamp as the joint non-compliance start timestamp, and take the larger value between the visual violation end timestamp and the auditory violation end timestamp as the joint non-compliance end timestamp. Concatenate the first non-compliance type code and the second non-compliance type code into a composite code as the joint non-compliance type code. Generate a non-compliance event location label, containing the joint non-compliance start timestamp, the joint non-compliance end timestamp, and the joint non-compliance type code. After all audit steps are completed in the activation order, all generated non-compliance event location tags are collected, arranged in ascending order by start timestamp, to form a non-compliance event location tag sequence.

[0083] Step S150: Bind the non-compliant event location tag sequence with the publisher account identifier associated with the multimodal hybrid publishing unit, generate a platform interception response instruction, and send the platform interception response instruction to the platform content distribution gateway to prevent the multimodal hybrid publishing unit from entering the public content display area. The platform interception response instruction carries the publisher account identifier and the non-compliant event location tag sequence.

[0084] Extract the publisher account identifier from the metadata of the content stream to be reviewed. This publisher account identifier is a unique identifier for the user on the internet platform. Bind the non-compliance event location tag sequence generated in step S147 to this publisher account identifier, storing them as a linked data pair. Generate a platform interception response instruction, which contains two fields: a publisher account identifier field to identify the publisher of the violating content; and a non-compliance event location tag sequence field to describe the specific location and type of the violating content. Send the platform interception response instruction to the platform content distribution gateway via the network. Upon receiving the instruction, the platform content distribution gateway locates the violating segment in the multimodal hybrid publishing unit based on the non-compliance event location tag sequence in the instruction, prevents the entire multimodal hybrid publishing unit from entering the platform's public content display area, and simultaneously records the violation against the publisher based on the publisher account identifier.

[0085] Step S160: Perform time-axis extension processing on each non-compliant event location tag in the non-compliant event location tag sequence. Extend the protection time forward based on the joint non-compliance start timestamp in the non-compliant event location tag and extend the protection time backward based on the joint non-compliance end timestamp to generate an extended location interval containing the extended portion. Based on the extended location interval, extract the visual review frame sub-sequence, auditory review audio sampling point sequence, and text review character sub-sequence from the visual image frame sequence, audio sampling point sequence, and text character sequence of the multimodal hybrid publishing unit, respectively.

[0086] Extract the joint non-compliance start timestamp Ts and joint non-compliance end timestamp Te from the non-compliance event location tag sequence generated in step S147. Set a preset protection duration parameter T_guard. Calculate the start time T_ext_start = Ts - T_guard and the end time T_ext_end = Te + T_guard for the extended location interval. Based on T_ext_start and T_ext_end, extract all frames within this time range from the visual image frame sequence of the multimodal hybrid publishing unit to form a visual verification image frame subsequence. Similarly, extract the auditory verification audio sampling point subsequence from the audio sampling point sequence and the text verification character subsequence from the text character sequence.

[0087] Step S161: Perform a screen integrity segmentation verification operation on the visual verification screen frame subsequence, calculate the structural similarity offset of the screen content between each adjacent frame in the visual verification screen frame subsequence, and record the frame position where the structural similarity offset changes as the screen content switching node.

[0088] For each pair of adjacent frames in the visual verification frame subsequence, the structural similarity index is calculated. The structural similarity index is calculated considering three components: luminance contrast, contrast contrast, and structural contrast. Let the two adjacent frames be frame A and frame B. First, the means μ_A and μ_B, standard deviations σ_A and σ_B, and covariance σ_AB of the two frames are calculated. The luminance contrast component is l(A,B)=(2μ_Aμ_B+C1) / (μ_A^2+μ_B^2+C1), the contrast contrast component is c(A,B)=(2σ_Aσ_B+C2) / (σ_A^2+σ_B^2+C2), and the structural contrast component is s(A,B)=(σ_AB+C3) / (σ_Aσ_B+C3). The structural similarity index SSIM(A,B)=l(A,B)*c(A,B)*s(A,B). Calculate the structural similarity offset D_ssim = 1 - SSIM(A,B) by subtracting the structural similarity index from 1. Set a transition detection threshold; when D_ssim exceeds this threshold, mark the current frame position as a content switching node. Record the frame indices of all content switching nodes.

[0089] Step S162: Perform an acoustic environment consistency verification operation on the auditory verification audio sampling point sub-sequence, extract the background voiceprint feature contour of the auditory verification audio sampling point sub-sequence, calculate the degree of drift of the background voiceprint feature contour during the duration of the entire auditory verification audio sampling point sub-sequence, and record the moment when the degree of drift exceeds the preset drift tolerance range as the audio scene transformation node.

[0090] The audio sampling point subsequence for auditory verification is processed by framing, with each frame containing 2048 sampling points and a frame shift of 512 sampling points. Mel-frequency cepstral coefficients are extracted from each frame to obtain a 39-dimensional feature vector. A Gaussian mixture model (GMM) is used to model the feature vectors of the first N frames, obtaining the background acoustic signature contour. The GMM contains 16 Gaussian components, each including a weight, mean vector, and covariance matrix. For the feature vectors of each subsequent frame, the Mahalanobis distance relative to the background acoustic signature contour is calculated. The Mahalanobis distances are arranged along the time axis to obtain the drift curve. A drift tolerance range is set; when the drift continuously exceeds the upper limit of this range, the corresponding time position is marked as an audio scene transition node.

[0091] Step S163: Perform semantic coherence verification on the text verification character subsequence, use the context semantic prediction model to calculate the reasonable occurrence probability of the next character relative to the previous character in the text verification character subsequence at each character position, and record the character connection points where the reasonable occurrence probability is lower than the preset probability threshold as semantic break nodes.

[0092] A pre-trained contextual semantic prediction model is loaded. This model employs a Transformer decoder architecture and contains six decoder layers. Each decoder layer includes a masked multi-head self-attention sublayer, an encoder-decoder attention sublayer, and a feedforward neural network sublayer. Each sublayer is followed by normalization and residual connections. The model is pre-trained on a large Chinese corpus, employing a causal language modeling task during training: predicting the next character based on the preceding context. During inference, the contextual semantic prediction model inputs a subsequence of key characters from the text, and it outputs the probability distribution of the next character character by character. For each character position, the probability value of the actual character predicted by the contextual semantic prediction model is extracted as the reasonable occurrence probability. A probability threshold is set; when the reasonable occurrence probability falls below this threshold, the connection point between that character and the previous character is marked as a semantic breakpoint. The character positions of all semantic breakpoints are recorded.

[0093] Step S164: Project the timestamps of the screen content switching nodes, the audio scene transformation nodes, and the semantic break nodes onto a unified time axis, perform node alignment and merging processing, and obtain a review time segment boundary point sequence. Based on the review time segment boundary point sequence, divide the extended positioning interval into multiple review sub-segments. Each review sub-segment contains a corresponding visual review screen frame sub-interval, an auditory review audio sampling point sub-interval, and a text review character sub-interval.

[0094] The frame index of each screen content switching node recorded in step S161 is converted into a timestamp, and multiplied by the frame rate to obtain a time value in seconds. The timestamps of each audio scene change node recorded in step S162 and each semantic break node recorded in step S163 are aggregated onto the same timeline. All timestamps are sorted in ascending order. For adjacent timestamps with a time interval less than a preset merging threshold (e.g., 1 second), they are merged into a boundary point, and the midpoint between the two is taken as the merged boundary point timestamp. After merging, a review time segmentation boundary point sequence is obtained. Based on this boundary point sequence, the extended positioning interval is divided into multiple continuous and non-overlapping review sub-segments. Each review sub-segment corresponds to a time interval, and the corresponding visual review screen frame sub-interval, auditory review audio sampling point sub-interval, and text review character sub-interval are extracted from this time interval.

[0095] Step S165: Perform the same operation process as the cross-channel content conflict detection process again for each verification sub-segment, and generate a set of verification cross-channel content conflict identifiers with the visual verification frame sub-interval, auditory verification audio sampling point sub-interval and text verification character sub-interval within the verification sub-segment as input.

[0096] For each verification sub-segment, its visual verification frame sub-interval, auditory verification audio sampling point sub-interval, and text verification character sub-interval are used as input, and the cross-channel content conflict detection processing flow described in steps S121 to S127 is re-executed. Specifically, this includes: performing image region segmentation on the visual verification frame sub-interval and calculating the region visual description tuple; performing frame-by-frame windowing processing on the auditory verification audio sampling point sub-interval and calculating the spectral fluctuation change pattern sequence; performing word segmentation processing on the text verification character sub-interval and generating a semantic representation vocabulary list; establishing image-audio association pairing records, performing cross-channel content consistency calculation, and finally generating a set of verification cross-channel content conflict identifiers.

[0097] Step S166: Mark the review sub-segments in the cross-channel content conflict identifier set that have reached the severe level of conflict severity as confirmed violation review segments. Update the start and end timestamps of the confirmed violation review segments to the joint non-compliance start and end timestamp fields of the corresponding tags in the non-compliance event location tag sequence. Regenerate a platform interception response instruction carrying the publisher account identifier and the updated non-compliance event location tag sequence based on the updated non-compliance event location tag sequence, and send the platform interception response instruction to the platform content distribution gateway.

[0098] Iterate through each entry in the cross-channel content conflict identifier set generated in step S165 and check its conflict severity level identifier. If the level identifier indicates a severe conflict, mark the corresponding review sub-segment as a confirmed violation review segment. Extract the start and end timestamps of the confirmed violation review segment. Search for the tag corresponding to the review sub-segment in time in the original non-compliance event location tag sequence, update the joint non-compliance start timestamp of the tag to the start timestamp of the review sub-segment, and update the joint non-compliance end timestamp to the end timestamp of the review sub-segment. After all confirmed violation review segments have been processed, the updated non-compliance event location tag sequence is obtained. Following the method in step S150, use the updated non-compliance event location tag sequence to regenerate the platform interception response instruction and send it to the platform content distribution gateway to more accurately prevent violating content from entering the public display area.

[0099] Step S170: Construct a publisher behavior tracking record storage space for the Internet platform. The publisher behavior tracking record storage space is used to store the publisher's historical violation sequence and publisher's content publishing habit parameters with the publisher account identifier as the primary key.

[0100] In the distributed storage system, a dedicated namespace is allocated as the storage space for publisher behavior tracking records. This publisher behavior tracking record storage space adopts a key-value pair storage model, where the key is the publisher account identifier and the value is a composite data structure. This composite data structure contains two fields: a historical violation sequence field, used to store the timestamp, type, and duration of each violation event of the publisher; and a content publishing habit parameter field, used to store the visual feature statistical distribution and text feature statistical distribution of the publisher's normal content.

[0101] Step S171: After receiving the non-compliance event location tag sequence, extract the joint non-compliance type code, joint non-compliance start timestamp, and joint non-compliance end timestamp of each non-compliance event location tag in the non-compliance event location tag sequence. Calculate the duration of the current non-compliance event based on the joint non-compliance start timestamp and joint non-compliance end timestamp, and write the joint non-compliance type code and the duration into the new record of the publisher's historical violation behavior sequence with the publisher account identifier as the primary key.

[0102] Traverse each tag from the non-compliance event location tag sequence generated in step S147. For each tag, extract the joint non-compliance start timestamp Ts and joint non-compliance end timestamp Te, and calculate the duration D_violation = Te - Ts. Extract the joint non-compliance type code. Using the publisher account identifier of the current multimodal hybrid publishing unit as the key, search for the corresponding record in the publisher behavior tracking record storage space constructed in step S170. Add the timestamp, duration D_violation, and joint non-compliance type code of this non-compliance event as a new record to the end of the historical violation behavior sequence field.

[0103] Step S172: Perform statistical analysis on the updated sequence of publishers' historical violations, and calculate the frequency parameter of the same joint non-compliance type code appearing in the publishers corresponding to the publisher account identifier within the preset backtracking period, as well as the average time interval parameter between two adjacent non-compliance events.

[0104] Retrieves a sequence of historical violations from the publisher behavior tracking record storage space for a specified publisher account identifier. A backtracking period T_window is set, for example, 30 days. Violation records within this backtracking period are filtered out. For each joint non-compliance type code, the number of times it occurs within the backtracking period is counted as the repetition frequency parameter F_repeat. The time interval between two adjacent non-compliance events is calculated, and the arithmetic mean of all intervals is taken as the average time interval parameter T_avg_interval.

[0105] Step S173: Abstract the content features of the compliant screen frame intervals that have not triggered cross-channel content conflict flags in the multimodal hybrid publishing unit, extract the distribution pattern of the regional visual description tuples of the screen segmentation area units within the compliant screen frame intervals, and record them as the publisher's content publishing habit parameters.

[0106] From the multimodal hybrid publishing unit, locate the visual frame intervals that have not triggered any cross-channel content conflict flags. Perform screen region segmentation on the above compliant frame intervals according to the method in step S121 to obtain multiple screen segmentation region units. For each screen segmentation region unit, extract its region visual description tuple. Calculate the mean and variance of the direction consistency parameter and the mean and variance of the change uniformity parameter in all region visual description tuples. Use these statistics as publisher content publishing habit parameters and store them in the record corresponding to the publisher account identifier in the publisher behavior tracking record storage space.

[0107] Step S174: Generate a dynamic review intensity adjustment factor for the publisher account identifier based on the repetition frequency parameter and the average time interval parameter. When the dynamic review intensity adjustment factor exceeds the preset adjustment limit, generate a publisher key monitoring mark instruction. Associate the publisher key monitoring mark instruction with the publisher account identifier and store it in the publisher behavior tracking record storage space. The dynamic review intensity adjustment factor is used to adjust the conflict judgment threshold in the cross-channel content conflict detection processing when the publisher account identifier is subsequently received as the content stream to be reviewed.

[0108] The dynamic review intensity adjustment factor F_adj is calculated as follows: F_adj = w_f * (F_repeat / F_base) + w_t * (T_base / T_avg_interval), where F_base is the baseline frequency value, T_base is the baseline time interval, and w_f and w_t are weighting coefficients. When F_adj exceeds the preset adjustment limit F_max, a publisher key monitoring mark instruction is generated. This publisher key monitoring mark instruction includes the publisher account identifier and the key monitoring mark. This publisher key monitoring mark instruction is stored in the publisher behavior tracking record storage space and associated with the publisher account identifier. The dynamic review intensity adjustment factor F_adj is used to adjust the conflict judgment threshold in subsequent review processes. Specifically, the correlation degree threshold in step S126 is multiplied by (1-α*F_adj), so that the content of high-risk publishers is reviewed more strictly.

[0109] Step S175: When receiving the content stream to be reviewed from the Internet platform, first parse the publisher account identifier associated with the content stream to be reviewed, and query whether there is a publisher key monitoring mark instruction corresponding to the publisher account identifier in the publisher behavior tracking record storage space.

[0110] When step S110 receives a new content stream to be reviewed, it first extracts the publisher account identifier from the metadata of the content stream. Using this identifier as the key, it queries the publisher behavior tracking record storage space. It then checks whether there is a key monitoring flag instruction associated with this identifier.

[0111] Step S176: If the publisher's key monitoring mark instruction exists, the multimodal hybrid publishing unit corresponding to the content stream to be reviewed is directly sent to the high-priority processing queue in the distributed review processor cluster. At the same time, the review strategy orchestration instruction set for the publisher account identifier in the review rule scheduling library is replaced with an enhanced review strategy orchestration instruction set. The enhanced review strategy orchestration instruction set includes a newly added sensitive word deep analysis review processor unit and a newly added screen detail secondary screening review processor unit. The number of review processor units in the activation order description of the enhanced review strategy orchestration instruction set is greater than the number of review processor units in the review strategy orchestration instruction set.

[0112] If the query result in step S175 shows the presence of a key monitoring marker instruction, the multimodal hybrid publishing unit corresponding to the current content stream to be reviewed is directly sent to a high-priority processing queue in the distributed review processor cluster, which is specifically used for handling high-priority tasks. Simultaneously, the enhanced review strategy orchestration instruction set corresponding to the publisher's account identifier is loaded from the review rule scheduling library. The enhanced review strategy orchestration instruction set adds two review processor units to the original set: a sensitive word deep analysis review processor unit, used for deeper semantic analysis of text character sequences; and a secondary screening review processor unit for magnified analysis of local detail areas in the visual image. The activation order description of the enhanced review strategy orchestration instruction set shows that the total number of review processor units is two more than that of the conventional strategy, and the priority of these two newly added units is set after visual review and auditory review, and before fusion analysis.

[0113] Step S180: When the multimodal hybrid publishing unit is successfully blocked from entering the public content display area by the platform's interception response command, the publisher account identifier and the non-compliance event location tag sequence corresponding to the intercepted multimodal hybrid publishing unit are extracted. Based on the joint non-compliance start timestamp and joint non-compliance end timestamp in the non-compliance event location tag sequence, a snapshot of the non-compliance frame interval is extracted from the visual frame sequence of the multimodal hybrid publishing unit.

[0114] After the platform interception response command generated in step S150 is successfully executed, the publisher account identifier and the non-compliance event location tag sequence are extracted from the command. For each tag in the non-compliance event location tag sequence, based on its joint non-compliance start timestamp Ts and joint non-compliance end timestamp Te, frames in the range of Ts to Te are extracted from the visual frame sequence of the original multimodal hybrid publishing unit as non-compliance frame interval snapshots.

[0115] Step S181: Blur the pixel information of the original visual frame of the non-compliant frame interval snapshot to hide the specific visual content, and retain the boundary coordinate sequence of the screen segmentation area unit obtained after the screen region segmentation operation of the non-compliant frame interval snapshot as the abstract feature of the screen structure.

[0116] For each frame in the snapshot of the non-compliant image frame range captured in step S180, a Gaussian blur operation is performed. A Gaussian kernel with a standard deviation of σ is used to convolve the red, green, and blue channels respectively to obtain the blurred image. Simultaneously, image region segmentation is performed on the original non-compliant image frames according to the method in step S121 to obtain the boundary coordinate sequence of the image segmentation region units. This boundary coordinate sequence is used as an abstract feature of the image structure and stored separately from the blurred image.

[0117] Step S182: Extract non-compliant audio sampling interval segments from the audio sampling point sequence of the multimodal hybrid publishing unit according to the joint non-compliant start timestamp and joint non-compliant end timestamp, perform acoustic feature extraction processing on the non-compliant audio sampling interval segments, and retain the spectral fluctuation change pattern sequence obtained after the non-compliant audio sampling interval segments are processed by frame segmentation and windowing as sound contour abstract features.

[0118] Audio data within the range of Ts to Te is extracted from the audio sampling point sequence of the original multimodal hybrid release unit as a non-compliant audio sampling interval segment. Following the method in step S122, this segment undergoes frame-by-frame windowing processing, and the spectral fluctuation change parameters of each short-time audio analysis frame are calculated to obtain a spectral fluctuation change pattern sequence. This sequence is then used as the sound contour abstract feature.

[0119] Step S183: Combine the abstract features of the image structure, the abstract features of the sound contour, and the joint non-compliance type encoding in the non-compliance event location tag sequence into an interception evidence digest data packet, and bind the interception evidence digest data packet with the publisher account identifier and the current system timestamp and store it in the cloud interception evidence archive area.

[0120] Create an intercepted evidence digest data packet data structure containing three fields: an abstract feature field for image structure, storing the sequence of boundary coordinates of the image segmentation region units obtained in step S181; an abstract feature field for sound contour, storing the sequence of spectral fluctuation patterns obtained in step S182; and a joint non-compliance type encoding field, storing the type of violation. Bind the publisher's account identifier and the current system timestamp to the intercepted evidence digest data packet to form a complete evidence record. Upload this evidence record to the cloud-based intercepted evidence archive area for persistent storage.

[0121] Step S184: Generate a violation notification message carrying the access identifier of the intercepted evidence archive area. Based on the publisher's account identifier, query the publisher's bound communication address in the platform user database. Send the violation notification message to the communication receiving terminal corresponding to the bound communication address. Simultaneously, broadcast the joint non-compliance type code and publisher's account identifier from the intercepted evidence digest data packet to the consensus node of the internet platform's audit consortium blockchain. The consensus node records the joint non-compliance type code and publisher's account identifier in a distributed tamper-proof ledger. The violation notification message includes templated text explaining that the published content violates platform regulations and the access identifier.

[0122] An access identifier is generated, pointing to the evidence record stored in step S183 in the cloud-based intercepted evidence archive area. A violation notification message is constructed, containing the templated text "Your published content violates platform regulations and has been blocked" along with the aforementioned access identifier. Using the publisher's account identifier as the query key, the platform's user database is searched for the publisher's bound communication address (phone number or email address). The violation notification message is sent to this communication address via SMS gateway or email server. Simultaneously, the combined non-compliance type code and publisher's account identifier are broadcast to all consensus nodes of the internet platform's audit consortium blockchain. After executing the consensus algorithm, the consensus nodes package this information into a block and append it to the distributed tamper-proof ledger, generating an on-chain evidence-based transaction identifier.

[0123] Step S185: Receive the on-chain evidence storage transaction identifier returned by the consensus node, and supplement the on-chain evidence storage transaction identifier into the metadata field of the corresponding intercepted evidence digest data packet in the cloud intercepted evidence archive area.

[0124] Receive the on-chain evidence storage transaction identifier returned by the consensus node in step S184. Locate the corresponding intercepted evidence summary data packet in the cloud intercepted evidence archive area, and add a new entry in the metadata field of this data packet to record the on-chain evidence storage transaction identifier. This on-chain evidence storage transaction identifier can be used to subsequently query and verify the authenticity and immutability of the violation record from the blockchain.

[0125] For example, the method may further include: step S190: during the idle computing period of the distributed audit processor cluster, a retrospective analysis training operation is performed on the historical multimodal hybrid publishing units and the corresponding historical non-compliance event location tag sequences stored in the audit log database.

[0126] The scheduler of the distributed audit processor cluster periodically checks the computational load of each processing unit in the cluster. When it detects that the load of a processing unit is lower than a preset idle threshold, the scheduler marks the processing unit as idle and triggers a retrospective analysis training task. This retrospective analysis training task reads the historical multimodal hybrid publishing units and their corresponding historical non-compliance event location tag sequences stored in the audit log database within a historical time period.

[0127] Step S191: Extract a set of historical multimodal hybrid release unit samples within a preset time window from the audit log database. Each sample in the set of historical multimodal hybrid release unit samples is associated with a final violation identification label confirmed by manual review.

[0128] Set a preset time window, such as the past 30 days. Extract all historical multimodal hybrid release units within this time window from the audit log database as a sample set. Each sample is associated with a final violation label confirmed by human audit experts, with the final violation label value being either "compliant" or "violation".

[0129] Step S192: Perform the cross-channel content conflict detection process on each sample in the historical multimodal hybrid release unit sample set to obtain a sample cross-channel content conflict identifier set. Compare the cross-channel content conflict entries in the sample cross-channel content conflict identifier set with the final violation identification label confirmed by manual review to identify false positive conflict entries in the cross-channel content conflict identifier set that have conflict degree classification labels but whose final violation identification label is compliant.

[0130] For each historical sample extracted in step S191, cross-channel content conflict detection processing is performed according to the methods in steps S121 to S127 to generate a set of cross-channel content conflict identifiers for the sample. Each cross-channel content conflict entry in this set is compared with the final violation classification label of the sample under manual review. If there is a conflict level classification label in the set of cross-channel content conflict identifiers (i.e., a conflict is detected), but the final violation classification label under manual review is "compliant," then the conflict entry is marked as a false positive conflict entry.

[0131] Step S193: Extract the subsequences of the image frame interval, the subsequence of the audio sampling interval, and the subsequence of the text character interval from the historical multimodal hybrid publishing unit corresponding to the false positive conflict entry, construct a false positive sample training dataset, and use the false positive sample training dataset to optimize and adjust the preset threshold parameters in the image-audio association pairing record generation logic used in the cross-channel content conflict detection processing, and update the preset threshold parameters through gradient descent to reduce the number of false positive conflict entries generated.

[0132] For each false positive conflict entry, extract the corresponding historical multimodal hybrid release unit's frame interval subsequence, audio sampling interval subsequence, and text character interval subsequence, and combine these subsequences into a false positive sample. All false positive samples constitute the false positive sample training dataset. In the frame-audio association pairing record generation logic of step S125, there is a preset threshold parameter θ_overlap used to determine the overlap between abnormal fluctuation segments and frame segmentation region units. Define the loss function L(θ) = the number of conflict entries still generated in the false positive samples. Use gradient descent to optimize θ, calculate the partial derivative of the loss function with respect to θ, update θ along the negative gradient direction, and repeat the iteration until the number of false positive conflict entries decreases below the preset target value.

[0133] Step S194: Extract false negative samples from the historical multimodal hybrid release unit sample set in the review log database that were determined to be non-conflicting by the cross-channel content conflict detection processing but confirmed as non-compliant by manual review. Construct a false negative sample training dataset. Use the false negative sample training dataset to optimize and update the adjustable parameters in the correlation degree calculation function of the region visual description tuple and semantic feature vector involved in the cross-channel content consistency calculation used in the cross-channel content conflict detection processing. Correct the adjustable parameters by minimizing the error between the correlation degree calculation value and the non-compliance score of manual review. Encapsulate the updated cross-channel content conflict detection processing logic into an incremental update patch file.

[0134] Samples meeting the following conditions are selected from the review log database: the cross-channel content conflict detection processing in step S192 determines there is no conflict (i.e., no conflict level classification is generated), but the final violation label in the manual review is "violation". These samples are marked as false negative samples, and a false negative sample training dataset is constructed. In the cross-channel content consistency calculation in step S126, there is a correlation degree calculation function containing an adjustable parameter vector φ. The loss function L(φ) = Σ(calculated correlation degree value - manual review violation degree score)^2 is defined. Gradient descent is used to optimize φ, the gradient of the loss function with respect to φ is calculated, and φ is updated along the negative gradient direction to minimize the error between the calculated value and the score. After optimization, the updated cross-channel content conflict detection processing logic (including the new threshold parameter θ and the new adjustable parameter φ) is packaged into an incremental update patch file.

[0135] Step S195: During the maintenance time window of the distributed review processor cluster, the reception of the content stream to be reviewed transmitted from the Internet platform is suspended, and the incremental update patch file is loaded into the cross-channel content conflict detection module of the distributed review processor cluster. After the loading is completed, the reception of the content stream to be reviewed transmitted from the Internet platform is resumed, and the updated cross-channel content conflict detection processing logic is used to perform cross-channel content conflict detection processing on the subsequent multimodal hybrid publishing units.

[0136] The distributed content processing cluster is configured with a daily maintenance window, such as 2:00 AM to 3:00 AM. During this window, the cluster's load balancer temporarily stores newly arriving content streams awaiting review in a buffer queue, pausing the distribution of new tasks to the content processing units. The incremental update patch file generated in step S194 is distributed to all nodes in the cluster where the cross-channel content conflict detection module resides. After each node verifies the integrity of the patch file, it dynamically loads the updated parameters from the patch, replacing the old parameters currently in memory. Once loading is complete, the load balancer is notified to resume receiving and distributing content streams awaiting review. Subsequently, all newly received multimodal hybrid publishing units will be reviewed using the updated cross-channel content conflict detection processing logic.

[0137] Step S200: After the cross-channel content conflict identifier set is generated, perform temporal correlation analysis on the start index and end index of the overlapping time period of each cross-channel content conflict entry in the cross-channel content conflict identifier set to identify multiple adjacent cross-channel content conflict entries with an interval of less than a preset merging threshold on the time axis.

[0138] Extract the start and end indexes of all cross-channel content conflict entries from the cross-channel content conflict identifier set generated in step S127. Sort the entries in ascending order of their start indexes. Traverse the sorted list of entries, and for two adjacent entries i and i+1, calculate the difference ΔT between the end index of entry i and the start index of entry i+1. If ΔT is less than the preset merging threshold T_merge, then these two entries are identified as adjacent entries that need to be merged.

[0139] Step S201: Merge multiple adjacent cross-channel content conflict entries with an interval less than a preset merging threshold into a conflict event cluster, and use the earliest overlapping time period start position index in the conflict event cluster as the conflict event cluster start position index, and the latest overlapping time period end position index in the conflict event cluster as the conflict event cluster end position index.

[0140] For each group of adjacent mergeable entries identified in step S200, they are grouped into a conflict event cluster. From all entries in this cluster, the smallest starting position index is found as the conflict event cluster starting position index S_cluster, and the largest ending position index is found as the conflict event cluster ending position index E_cluster.

[0141] Step S202: Extract the conflict severity classification identifiers of all cross-channel content conflict entries within the conflict event cluster, construct a conflict severity classification identifier distribution sequence, and calculate the conflict intensity evolution parameters of the conflict event cluster based on the frequency of occurrence and temporal distribution density of each classification identifier in the conflict severity classification identifier distribution sequence.

[0142] Conflict severity grading identifiers are extracted from each cross-channel content conflict entry within a conflict event cluster and arranged chronologically to form a conflict severity grading identifier distribution sequence. Each grading identifier is mapped to a numerical value: mild conflict = 1, moderate conflict = 2, severe conflict = 3. The arithmetic mean of the values ​​in the sequence is calculated as the baseline value for conflict intensity. The change in conflict severity between two adjacent entries in the sequence is calculated, and the sum of the squares of all changes is taken as the degree of conflict intensity fluctuation. The conflict intensity evolution parameter E_intensity = baseline value of conflict intensity * (1 + degree of conflict intensity fluctuation).

[0143] Step S203: Determine whether the conflict event cluster shows a gradually escalating evolution trend based on the conflict intensity evolution parameters. If the conflict intensity evolution parameters show a monotonically increasing characteristic in the time dimension, then mark the conflict event cluster as an escalating conflict event cluster.

[0144] The conflict event cluster is divided into two subsets, a front segment and a back segment, according to chronological order. The conflict intensity evolution parameters for the front segment and the back segment are calculated and denoted as E_front and E_back, respectively. If E_back > E_front, and the conflict severity classification indicator of each subsequent entry in the entire sequence is not less than the classification indicator of its preceding entry, then the conflict event cluster is determined to show a gradually escalating evolution trend, and the conflict event cluster is marked as an escalating conflict event cluster.

[0145] Step S204: Backtrack and extract the video-audio association pairing records corresponding to each cross-channel content conflict entry within the upgraded conflict event cluster, and obtain the regional visual description tuple and spectral fluctuation change parameters in the video-audio association pairing records.

[0146] Based on the temporal position of each cross-channel content conflict entry in the escalating conflict event cluster, the corresponding video-audio association pairing record generated in step S125 is traced back. From these records, regional visual description tuples (containing directional consistency parameters and variation uniformity parameters) and spectral fluctuation variation parameters are extracted.

[0147] Step S205: Input the sequence of regional visual description tuples arranged in chronological order within the upgraded conflict event cluster into a pre-constructed visual semantic drift tracking network, and output the visual semantic drift trajectory vector by comparing the change in the directional consistency parameter of the regional visual description tuples in adjacent time periods; and input the sequence of spectral fluctuation change parameters arranged in chronological order within the upgraded conflict event cluster into a pre-constructed auditory fluctuation pattern evolution analysis network, and output the auditory fluctuation evolution path vector by calculating the degree of spectral peak shift of the spectral fluctuation change parameters in adjacent time periods.

[0148] A visual semantic drift tracking network is constructed. This network employs a gated recurrent unit (GRU) architecture, containing two GRU layers, each with 128 hidden units. The network input is a temporally ordered sequence of region visual description tuples, where each tuple is a two-dimensional vector. The network output is a visual semantic drift trajectory vector, representing the trend of visual content changes over time. An auditory fluctuation pattern evolution analysis network is also constructed, employing the same GRU architecture. Its input is a temporally ordered sequence of spectral fluctuation change parameters, and its output is an auditory fluctuation evolution path vector. The sequence of region visual description tuples within escalating conflict event clusters is input into the visual semantic drift tracking network to obtain the visual semantic drift trajectory vector V_visual. The sequence of spectral fluctuation change parameters is input into the auditory fluctuation pattern evolution analysis network to obtain the auditory fluctuation evolution path vector V_audio.

[0149] Step S206: Perform cross-channel collaborative evolution analysis on the visual semantic drift trajectory vector and the auditory fluctuation evolution path vector, calculate the cosine value of the angle between the visual semantic drift trajectory vector and the auditory fluctuation evolution path vector in the evolution direction and the ratio parameter of the evolution rate. When the cosine value of the angle is lower than the preset collaborative threshold, it is determined that there is an evolutionary deviation between the visual channel and the auditory channel in the upgraded conflict event cluster. Generate a channel evolution deviation alarm sign and attach the channel evolution deviation alarm sign to the compliance parsing parameter configuration of the review strategy arrangement instruction set of the conflict event cluster. Based on the review strategy arrangement instruction set with the attached channel evolution deviation alarm sign, drive the target review processor unit in the distributed review processor cluster to perform enhanced content compliance parsing processing on the screen frame interval and audio sampling interval corresponding to the conflict event cluster.

[0150] Calculate the cosine of the angle between the visual semantic drift trajectory vector V_visual and the auditory fluctuation evolution path vector V_audio: cosθ = (V_visual·V_audio) / (|V_visual|*|V_audio|). Calculate the magnitude ratio of the two vectors: R_magnitude = |V_visual| / |V_audio|. When cosθ is lower than a preset coordination threshold (e.g., 0.5), an evolutionary deviation is identified. Generate a channel evolutionary deviation alarm flag. Attach this alarm flag to the compliance parsing parameter configuration of the review policy orchestration instruction set corresponding to the conflict event cluster. Based on the review policy orchestration instruction set with the alarm flag attached, drive the target review processor unit in the distributed review processor cluster to perform enhanced content compliance parsing processing on the frame interval and audio sampling interval corresponding to the conflict event cluster, i.e., enable a higher-precision violation detection model and finer-grained feature extraction.

[0151] Step S210: If the multimodal hybrid publishing unit passes the cross-channel content conflict detection process and is determined to have no content matching anomalies, perform a screen subject saliency detection operation on the visual screen frame sequence in the multimodal hybrid publishing unit to generate a saliency heat map describing the distribution of visual focus areas in the screen.

[0152] For multimodal hybrid publishing units determined to have no content matching anomalies in step S120, a graph-based saliency detection algorithm is used to process each frame in the visual image frame sequence. Each frame is segmented into superpixel blocks, and a graph model is constructed where nodes represent superpixel blocks and edges represent the relationships between adjacent superpixel blocks. The color feature contrast and spatial location prior weights of each superpixel block are calculated, and the saliency score of each superpixel block is obtained by solving the graph Laplacian linear system. The saliency scores of all superpixel blocks are mapped to the pixel locations in the original image, generating a saliency heatmap with the same size as the image, where higher pixel values ​​indicate greater attention at that location.

[0153] Step S211: Perform auditory attention focus localization operation on the audio sampling point sequence in the multimodal hybrid publishing unit. Determine the auditory attention focus segment sequence by analyzing the energy concentration frequency band and the intensity of energy fluctuation in the frequency domain fluctuation pattern of the audio sampling point sequence. Each auditory attention focus segment is defined by its start timestamp and end timestamp.

[0154] The audio sampling point sequence is framed according to the method in step S122, and the spectral energy distribution of each frame is calculated. For each frame, the frequency band with the highest energy is identified as the energy concentration band of that frame. The intensity of energy fluctuation in the energy concentration band is calculated, which is the absolute value of the difference between the energy value of that frame and the energy value of the adjacent frame. When the intensity of energy fluctuation exceeds a preset threshold, the frame is marked as an auditory attention candidate frame. Multiple consecutive marked candidate frames are merged into an auditory attention focus segment, and the start and end timestamps of the segment are recorded. All focus segments constitute an auditory attention focus segment sequence.

[0155] Step S212: Extract the boundary coordinate sequence of the visual focus area and the duration of the visual focus area based on the saliency heat map. Calculate the spatial overlap between the boundary coordinate sequence of the visual focus area and the boundary coordinate sequence of the image segmentation area unit. Select the image segmentation area units that have a spatial overlap with the visual focus area exceeding a preset overlap threshold. Each image segmentation area unit has a corresponding start and end timestamp in the image.

[0156] For highlighted regions in the saliency heatmap with saliency scores exceeding a threshold, the boundary coordinate sequence of their connected regions is extracted as the boundary of the visual focus region. The duration of the visual focus region's appearance in multiple consecutive frames is recorded as the dwell time. The spatial overlap is calculated by comparing the boundary coordinates of the visual focus region with the boundary coordinate sequences of the image segmentation region units obtained in step S121, where overlap = intersection area / union area. Image segmentation region units with overlap exceeding a preset overlap threshold (e.g., 0.6) are selected as focus image segmentation region units. Each focus image segmentation region unit has a corresponding start time stamp and end time stamp based on the timestamps of its first and last appearance frames.

[0157] Step S213: Extract the auditory focus audio segment from the audio sampling point sequence according to the auditory attention focus segment sequence, and extract the spectral energy concentration band range and the energy attenuation rate parameter within the spectral energy concentration band range of the auditory focus audio segment.

[0158] Based on the start and end timestamps in the auditory attention focus segment sequence determined in step S211, the corresponding auditory focus audio segment is extracted from the original audio sampling point sequence. A short-time Fourier transform is performed on this segment to obtain the time-frequency spectrum. In the time-frequency spectrum, for each time point, the frequency band with the highest energy is found, and the average value of this frequency band across all time points is taken as the concentrated energy frequency band range. Within this frequency band range, the energy decay trend is calculated along the time axis, and a linear fit is performed after taking the logarithm of the energy values. The absolute value of the slope of the fitted line is used as the energy decay rate parameter.

[0159] Step S214: Perform cross-channel attention focus correlation analysis on the region visual description tuple of the attention image segmentation unit and the spectral energy concentration band range and energy decay rate parameters of the auditory focus audio segment. Calculate the overlap time ratio between the occurrence time of the attention image segmentation unit and the auditory attention focus segment sequence. When the overlap time ratio exceeds a preset overlap threshold, construct a cross-channel attention focus pairing record by combining the region visual description tuple of the attention image segmentation unit and the spectral energy concentration band range and energy decay rate parameters of the auditory focus audio segment.

[0160] For each attention segmentation region, obtain its start time stamp Ts_visual and end time stamp Te_visual. For each auditory attention focus segment, obtain its start time stamp Ts_audio and end time stamp Te_audio. Calculate the overlap duration between the occurrence time of the attention segmentation region and the auditory attention focus segment: max(0, min(Te_visual, Te_audio) - max(Ts_visual, Ts_audio)). Calculate the overlap time ratio = overlap duration / (Te_visual - Ts_visual). When the overlap time ratio exceeds a preset overlap threshold (e.g., 0.5), associate the region visual description tuple of the attention segmentation region with the spectral energy concentration band range and energy decay rate parameters of the auditory attention focus audio segment to construct an attention focus cross-channel pairing record.

[0161] Step S215: Extract the text character intervals corresponding to the time periods of the attention focus cross-channel pairing records of the semantic representation vocabulary list of the text character sequence, and identify the content word units carrying the core semantics in the text character intervals as attention focus semantic anchor words. Match and compare the attention focus semantic anchor words with the preset platform key supervision vocabulary library. The platform key supervision vocabulary library stores supervision vocabulary entries that need to be subject to covert deep review and the covert review trigger conditions corresponding to each supervision vocabulary entry.

[0162] Based on the start and end timestamps of the segmented region units in the focus area, the text character interval corresponding to that time period is located from the text character sequence. The word segmentation process of step S123 is performed on this text character interval to obtain a list of semantic units. The importance score of each semantic unit within this interval is calculated using the term frequency-inverse document frequency method, and the word with the highest score is selected as the attention focus semantic anchor word. A platform-focused regulatory vocabulary library is pre-set, which stores regulatory vocabulary entries that require covert deep review. Each regulatory vocabulary entry includes the regulatory vocabulary itself and the corresponding covert review triggering conditions (such as frequency of occurrence, context, etc.). The attention focus semantic anchor words are then matched precisely and fuzzily with the regulatory vocabulary entries in the vocabulary library.

[0163] Step S216: If the attention focus semantic anchor word matches the regulatory term in the platform's key regulatory term library, a covert deep review instruction is generated based on the covert review triggering condition corresponding to the matched regulatory term. Without triggering the platform's interception response instruction, the covert deep review instruction copies and forwards the multimodal hybrid publishing unit to the covert review analysis queue. The covert review analysis queue then calls a covert content parsing processor unit with higher analysis accuracy to perform offline deep semantic profiling on the multimodal hybrid publishing unit.

[0164] If the matching result in step S215 is successful, the covert review trigger condition corresponding to the matched regulatory term entry is read. A covert deep review instruction is generated based on this trigger condition, which includes the following fields: a unique identifier for the multimodal hybrid publishing unit, a trigger condition code, and a priority. A key characteristic of the covert deep review instruction is that it does not trigger a platform interception response instruction; that is, the instruction is sent to a separate covert review analysis queue rather than the main review channel. Tasks in the covert review analysis queue are processed by a set of covert content parsing processor units with higher analysis accuracy. These processor units use computationally more complex and accurate deep learning models (such as large-scale visual transformer models and pre-trained language models) to perform offline deep semantic profiling of the multimodal hybrid publishing unit. The analysis results are for manual review or record retention, but do not affect the normal publishing status of the content on the current platform.

[0165] Based on the same inventive concept, please refer to Figure 2 This diagram illustrates a schematic block diagram of a multimodal content moderation system for an internet platform provided in an embodiment of this application. The system includes a central processing unit (CPU), a system memory comprising random access memory (RAM) and read-only memory (ROM), and a system bus connecting the system memory and the CPU. The multimodal content moderation system for internet platforms also includes a basic input / output system to facilitate information transfer between various devices within the computer, and a large-capacity storage device for storing the operating system, applications, and other program modules.

[0166] A basic input / output system includes a display for showing information and input devices such as a mouse and keyboard for user input. Both the display and the input devices are connected to the central processing unit via an input / output controller connected to the system bus. The basic input / output system may also include an input / output controller for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller also provides output to a display screen, printer, or other types of output devices.

[0167] Mass storage devices are connected to the central processing unit via a mass storage controller connected to the system bus. The mass storage devices and their associated computer-readable media provide non-volatile storage for multimodal content moderation for internet platforms. Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. According to various embodiments of this application, multimodal content moderation for internet platforms can also be performed via a network connection to a remote computer on the network, such as the Internet. That is, multimodal content moderation for internet platforms can be connected to a network via a network interface unit connected to the system bus, or a network interface unit can be used to connect to other types of networks or remote computer systems.

[0168] In addition, in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0169] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A method for multimodal content moderation on internet platforms, characterized in that, The method includes: The system receives a stream of content to be reviewed transmitted from an internet platform. The stream of content to be reviewed includes a sequence of visual frames, an audio sampling point sequence, and a text character sequence that have the same timestamp. The visual frame sequence, audio sampling point sequence, and text character sequence together constitute a multimodal hybrid publishing unit. The multimodal hybrid publishing unit is subjected to cross-channel content conflict detection processing to generate a cross-channel content conflict identifier set. The cross-channel content conflict identifier set is used to record the location index information and conflict degree classification identifier of the content matching anomaly between the visual channel and the auditory channel. The pre-built audit rule scheduling library is invoked based on the conflict level classification identifier and location index information recorded in the cross-channel content conflict identifier set. The audit strategy orchestration instruction set corresponding to the conflict level classification identifier is extracted. The audit strategy orchestration instruction set includes an activation order description of at least one audit processor unit and the compliance parsing parameter configuration that the audit processor unit needs to load. The activation order description in the set of audit strategy orchestration instructions is used to sequentially start multiple target audit processor units in the distributed audit processor cluster to perform content compliance parsing processing on the frame interval, audio sampling interval and text character interval corresponding to the location index information in the multimodal hybrid publishing unit, and generate a non-compliance event location tag sequence. The non-compliance event location tag sequence includes the start timestamp, end timestamp and non-compliance type code of the non-compliance phenomenon. The non-compliant event location tag sequence is bound to the publisher account identifier associated with the multimodal hybrid publishing unit to generate a platform interception response instruction, and the platform interception response instruction is sent to the platform content distribution gateway to prevent the multimodal hybrid publishing unit from entering the public content display area. The platform interception response instruction carries the publisher account identifier and the non-compliant event location tag sequence. The process of performing cross-channel content conflict detection on the multimodal hybrid publishing unit to generate a cross-channel content conflict identifier set includes: The visual image frame sequence in the multimodal hybrid publishing unit is subjected to image region segmentation operation. Based on the color similarity and spatial adjacency of each pixel block in the image, each visual image frame is divided into multiple non-overlapping image segmentation region units. The texture distribution in the image segmentation region units is statistically analyzed. The direction consistency parameter and texture thickness uniformity parameter of the texture direction inside each image segmentation region unit are calculated. The direction consistency parameter and texture thickness uniformity parameter are combined into the region visual description tuple of the image segmentation region unit. The audio sampling point sequence in the multimodal hybrid publishing unit is subjected to frame-by-frame windowing processing. The continuous audio sampling point sequence is divided into multiple short-time audio analysis frames with a preset overlap ratio. For each short-time audio analysis frame, the fluctuation envelope curve of its spectral energy along the frequency axis is extracted. Based on the fluctuation envelope curve, the spectral peak prominence and spectral valley depth in each short-time audio analysis frame are identified. The ratio between the spectral peak prominence and the spectral valley depth is used as the spectral fluctuation change parameter of the short-time audio analysis frame. The spectral fluctuation change parameters of each short-time audio analysis frame are arranged along the time axis to form a spectral fluctuation change pattern sequence. The text character sequence in the multimodal hybrid publishing unit is segmented to obtain multiple semantic lexical units, and the frequency and position of each semantic lexical unit in the text character sequence are counted. Each semantic unit is converted into a fixed-length semantic feature vector using a pre-trained word embedding mapping network. The spatial distance metric between the semantic feature vectors of different semantic units is calculated. Semantic units with spatial distance metrics less than a preset threshold are merged into the same semantic cluster. The average vector of the semantic feature vectors of all members of each semantic cluster is calculated. The semantic unit with the closest spatial distance between the average vector and the semantic feature vectors of each member is taken as the semantic representation vocabulary of the semantic cluster, and a list of semantic representation vocabulary is generated. Based on the timestamp of the visual frame and the timestamp of the audio sampling point sequence in the multimodal hybrid publishing unit, a correspondence between the frame interval and the audio sampling interval is established. For any frame segmentation unit appearing in any frame interval, abnormal fluctuation segments with fluctuation amplitudes exceeding the average fluctuation level are extracted from the spectral fluctuation change pattern sequence in the audio sampling interval corresponding to the frame interval. It is determined whether the time span of the audio sampling interval where the abnormal fluctuation segment is located overlaps with the continuous dwell time period of the frame segmentation unit in the frame. If they overlap, the regional visual description tuple of the frame segmentation unit is associated and paired with the spectral fluctuation change parameters of the abnormal fluctuation segment in the overlapping time period to generate a set of frame audio association pairing records. Extract the most frequently occurring semantic word unit from the semantic representation vocabulary list within the text character interval corresponding to the overlapping time period. Perform cross-channel content consistency calculation between the semantic feature vector of the semantic word unit and the regional visual description tuple in the video-audio association pairing record. When the correlation between the semantic feature vector and the regional visual description tuple is lower than a preset threshold, generate a cross-channel content conflict entry containing the starting position index of the overlapping time period, the ending position index of the overlapping time period, and the conflict degree classification identifier. After traversing all video frame intervals and audio sampling intervals, the cross-channel content conflict entries obtained are summarized to generate the cross-channel content conflict identifier set.

2. The multimodal content review method for internet platforms according to claim 1, characterized in that, The step of calling a pre-built review rule scheduling library based on the conflict severity classification identifier and location index information recorded in the cross-channel content conflict identifier set, and extracting the review strategy orchestration instruction set corresponding to the conflict severity classification identifier, includes: Each cross-channel content conflict entry in the cross-channel content conflict identifier set is parsed to obtain the overlapping time period start position index, overlapping time period end position index, and conflict degree classification identifier recorded therein; The conflict severity classification identifier is compared with the conflict level mapping table pre-stored in the audit rule scheduling library. Based on the comparison result, the target scheduling strategy number corresponding to the conflict severity classification identifier is determined. Each record in the conflict level mapping table contains a correspondence between a conflict severity classification identifier and a scheduling strategy number. Based on the target scheduling strategy number, the corresponding strategy orchestration instruction storage address is found in the strategy orchestration index table in the audit rule scheduling library. The strategy orchestration index table records the mapping relationship between each scheduling strategy number and the offset of its corresponding strategy orchestration instruction set in the storage medium. Based on the storage address of the policy orchestration instruction, the original data of the policy orchestration instruction set, which consists of multiple audit processor control instructions, is read from the audit rule scheduling library. The audit processor control instruction includes a type identifier field of the audit processor unit, an activation order field of the audit processor unit, and an execution parameter configuration field of the audit processor unit. Extract the type identifier field of each audit processor control instruction from the original data of the policy orchestration instruction set, and confirm whether there is a corresponding physical audit processor unit entity in the distributed audit processor cluster for the audit processor unit type represented by the type identifier field. The activation order field of each audit processor control instruction in the original data of the strategy orchestration instruction set is parsed to determine the wake-up order of the target audit processor unit and the waiting time interval parameter between adjacent activation steps. Extract the execution parameter configuration field of each review processor control instruction from the original data of the strategy orchestration instruction set. The execution parameter configuration field defines the frame interval selection rule, audio sampling interval selection rule, and text character interval selection rule used by the corresponding review processor unit when performing content compliance parsing and processing. The overlapping time period start index and overlapping time period end index in the cross-channel content conflict entry are filled into the frame interval selection rule, audio sampling interval selection rule and text character interval selection rule in the execution parameter configuration field to generate a concrete compliance parsing parameter configuration for the cross-channel content conflict entry. The concrete compliance parsing parameter configurations after filling are sorted according to the wake-up order defined by the activation order field, generating a parameter configuration queue of the audit processor unit corresponding to each activation step. The audit processor unit parameter configuration queue is combined with the corresponding activation order field and type identifier field to generate a set of audit strategy orchestration instructions for the cross-channel content conflict items. The set of audit strategy orchestration instructions is used to drive the distributed audit processor cluster to perform step-by-step compliance parsing of the overlapping time periods corresponding to the cross-channel content conflict items.

3. The multimodal content review method for internet platforms according to claim 1, characterized in that, The process involves sequentially activating multiple target review processor units in the distributed review processor cluster according to the activation order description in the review strategy orchestration instruction set. These units perform content compliance parsing processing on the frame intervals, audio sampling intervals, and text character intervals corresponding to the location index information in the multimodal hybrid publishing unit, generating a sequence of non-compliant event location tags, including: The type identifier field and execution parameter configuration field corresponding to the first activation step in the set of review strategy orchestration instructions are parsed. An activation signal is sent to the visual content review processor unit in the distributed review processor cluster according to the type identifier field. At the same time, the part of the execution parameter configuration field regarding the frame interval selection rule is passed to the visual content review processor unit, so that after receiving the activation signal, the visual content review processor unit extracts the corresponding frame interval subsequence from the visual frame sequence of the multimodal hybrid publishing unit according to the overlapping time period start position index and overlapping time period end position index specified in the frame interval selection rule. The pre-loaded violation visual pattern matching network inside the visual content review processor unit is invoked to retrieve sensitive visual elements for each frame of visual image in the subsequence of the frame interval. The violation visual pattern matching network consists of multiple parallel convolution extraction channels, and each convolution extraction channel corresponds to a different category of visual violation feature template. The violation feature response heat values ​​output by the violation visual pattern matching network for each visual image in the subsequence of the frame interval are fused to obtain a first time-series response waveform describing the time of occurrence and duration of visual violations in the subsequence of the frame interval. The start and end timestamps of the visual violation are determined based on the time of the peak and the half-width of the peak in the first time-series response waveform, and the category code of the visual violation is recorded as the first non-compliance type code. The type identifier field and execution parameter configuration field corresponding to the second activation step in the set of review strategy orchestration instructions are parsed. An activation signal is sent to the auditory content review processor unit in the distributed review processor cluster according to the type identifier field. At the same time, the part of the audio sampling interval selection rule in the execution parameter configuration field is passed to the auditory content review processor unit so that after receiving the activation signal, the auditory content review processor unit extracts the corresponding audio sampling interval subsequence from the audio sampling point sequence of the multimodal hybrid publishing unit according to the overlapping time period start position index and overlapping time period end position index specified in the audio sampling interval selection rule. The illegal acoustic pattern recognition network pre-loaded inside the auditory content review processor unit is invoked to perform sensitive acoustic event detection on the audio sampling interval subsequence. The illegal acoustic pattern recognition network determines whether there are pre-calibrated illegal audio segments by analyzing the Mel frequency cepstral coefficient sequence of the audio sampling interval subsequence. The acoustic event probability curve obtained by passing the subsequence of the audio sampling interval through the illegal acoustic pattern recognition network is smoothed, and the time interval in the acoustic event probability curve where the probability value exceeds the preset judgment threshold is extracted as the start and end timestamps of the auditory violation, and the category code of the auditory violation is recorded as the second non-compliance type code. By fusing the first time-series response waveform and the acoustic event probability curve, the ratio of the overlapping area of ​​the first time-series response waveform and the acoustic event probability curve on the time axis to the total duration is calculated. When the ratio is higher than a preset fusion threshold, the smaller of the start timestamp of the visual violation and the start timestamp of the auditory violation is taken as the joint non-compliance start timestamp, and the larger of the end timestamp of the visual violation and the end timestamp of the auditory violation is taken as the joint non-compliance end timestamp. The first non-compliance type code and the second non-compliance type code are merged into a joint non-compliance type code to generate a non-compliance event location tag containing the joint non-compliance start timestamp, the joint non-compliance end timestamp, and the joint non-compliance type code. The non-compliance event location tags collected after all activation steps are completed constitute the non-compliance event location tag sequence.

4. The method for multimodal content review on internet platforms according to claim 3, characterized in that, The step of invoking the pre-loaded violation visual pattern matching network within the visual content review processor unit to perform sensitive visual element retrieval for each frame in the subsequence of the image frame interval includes: A color space conversion operation is performed on each visual image in the subsequence of the image frame interval, converting the original red, green and blue color representation into a hue, saturation and brightness color representation. After uniform quantization processing of the hue channel to compress the total number of color categories, the quantized hue channel, saturation channel and brightness channel are respectively input into the three independent input branches of the illegal visual pattern matching network. The three independent input branches each contain a sequence of convolutional filter layers with convolutional receptive regions of different sizes. In the three independent input branches, the input features are extracted by the convolutional filter layer sequence of the first convolutional receptive region to extract local edge direction and corner distribution features, resulting in a local detail feature mapping group of the image; and the input features are extracted by the convolutional filter layer sequence of the second convolutional receptive region to extract regional color transition gradient and texture density distribution features, resulting in a regional structure feature mapping group of the image. The size of the second convolutional receptive region is larger than the size of the first convolutional receptive region. The image local detail feature mapping group and the image region structure feature mapping group are connected and merged along the feature channel direction to generate a multi-level visual feature descriptor of the image frame. The multi-level visual feature descriptor and multiple illegal feature templates stored in the illegal visual pattern matching network are subjected to sliding window similarity calculation to obtain a set of similarity response values ​​of each illegal feature template at different spatial positions in the multi-level visual feature descriptor. For each violation feature template, the peak response position and peak response value exceeding the preset response activation threshold are selected from the corresponding similarity response value set, and the peak response value is used as the violation confidence metric value of the violation feature template in the visual image of that frame. The violation confidence metric values ​​of the same violation feature template in multiple consecutive visual images of the subsequence of the image frame interval are arranged along the time axis to form the single template time-series response waveform of the violation feature template. The time-series response waveforms of each single template corresponding to all violation feature templates in the violation visual pattern matching network are weighted and summed. The weights are determined by the violation severity coefficient associated with each violation feature template, and the summation result forms the first time-series response waveform.

5. The method for multimodal content review on internet platforms according to claim 3, characterized in that, The step of invoking the pre-loaded illegal acoustic pattern recognition network within the auditory content review processor unit to perform sensitive acoustic event detection on the sub-sequence of the audio sampling interval includes: The audio sampling interval subsequence is pre-emphasized to enhance the spectral resolution of the high-frequency part of the audio. The audio sampling point sequence after pre-emphasis is subjected to short-time framing and windowing operations to obtain an audio analysis frame group. For each audio analysis frame in the audio analysis frame group, the logarithmic energy value after passing through the Mel frequency filter bank is calculated to obtain the Mel frequency cepstral coefficient feature vector. The Mel frequency cepstral coefficient feature vectors of all audio analysis frames are arranged in chronological order to form the Mel frequency cepstral coefficient feature matrix. The Mel frequency cepstral coefficient feature matrix is ​​input into the first layer of the convolutional temporal processing module of the illegal acoustic pattern recognition network, and the changing trend features of the Mel frequency cepstral coefficient feature vector between adjacent audio analysis frames are extracted using a one-dimensional convolutional filter kernel that slides along the time axis. The output of the first convolutional timing processing module is passed to the second cyclic timing processing module. The second cyclic timing processing module includes a memory update unit with a gating mechanism. The memory update unit calculates the hidden state vector output at the current time based on the input features at the current time and the hidden state vector at the previous time. The output of the second-layer cyclic timing processing module is passed to the third-layer attention focusing module. The third-layer attention focusing module generates a focusing weighting coefficient sequence for the hidden state vector at each time step by calculating the inner product similarity between the hidden state vector and the trainable query vector at each time step of the second-layer cyclic timing processing module. The weighted coefficient sequence is used to perform a weighted summation on the hidden state vectors at each time step of the second-layer cyclic temporal processing module to obtain an aggregated hidden state vector that represents the overall acoustic content of the subsequence of the audio sampling interval. The aggregated hidden state vector is then input into the classification output layer of the illegal acoustic pattern recognition network. The classification output layer consists of a fully connected operation structure and an activation function. The probability distribution of the aggregated hidden state vector belonging to each pre-labeled illegal audio segment category is calculated. The sensitive acoustic event type of the audio sampling interval subsequence is determined based on the category of the illegal audio segment corresponding to the maximum probability value in the probability distribution, and the peak position of the focus weighting coefficient sequence output by the third-layer attention focusing module on the time axis is traced back. Based on the peak position of the focus weighting coefficient sequence on the time axis and the time span covered by its decay to half peak, the start and end boundaries of the sensitive acoustic event in the subsequence of the audio sampling interval are determined. The start and end boundaries are mapped back to the absolute time coordinates of the audio sampling point sequence to obtain the start and end timestamps of the auditory violation. The sensitive acoustic event type is then encoded and recorded as the second non-compliance type code.

6. The multimodal content review method for internet platforms according to claim 1, characterized in that, The method further includes: Each non-compliant event location tag in the non-compliant event location tag sequence is subjected to time axis extension processing. The protection time is extended forward by a preset duration based on the joint non-compliance start timestamp in the non-compliant event location tag, and extended backward by a preset duration based on the joint non-compliance end timestamp, generating an extended location interval containing the extended portion. Based on the extended location interval, visual review frame sub-sequence, auditory review audio sampling point sequence, and text review character sub-sequence are respectively extracted from the visual image frame sequence, audio sampling point sequence, and text character sequence of the multimodal hybrid publishing unit. Perform a screen integrity segmentation verification operation on the visual verification screen frame subsequence, calculate the structural similarity offset of the screen content between each adjacent frame in the visual verification screen frame subsequence, and record the frame position where the structural similarity offset jumps as the screen content switching node. An acoustic environment consistency verification operation is performed on the auditory verification audio sampling point sub-sequence. The background voiceprint feature contour of the auditory verification audio sampling point sub-sequence is extracted. The drift degree of the background voiceprint feature contour during the duration of the entire auditory verification audio sampling point sub-sequence is calculated. The moment when the drift degree exceeds the preset drift tolerance range is recorded as the audio scene transformation node. The semantic coherence review operation is performed on the subsequence of the text review characters. The reasonable occurrence probability of the next character relative to the previous character in the subsequence of the text review characters is calculated character by character using the context semantic prediction model. The character connection points where the reasonable occurrence probability is lower than the preset probability threshold are recorded as semantic break nodes. The timestamps of the screen content switching nodes, the audio scene transformation nodes, and the semantic break nodes are projected onto a unified time axis, and node alignment and merging are performed to obtain a review time segment boundary point sequence. Based on the review time segment boundary point sequence, the extended positioning interval is divided into multiple review sub-segments. Each review sub-segment contains a corresponding visual review screen frame sub-interval, an auditory review audio sampling point sub-interval, and a text review character sub-interval. For each review sub-segment, the same operation process as the cross-channel content conflict detection process is performed again, using the visual review frame sub-interval, auditory review audio sampling point sub-interval, and text review character sub-interval within the review sub-segment as input to generate a set of cross-channel content conflict identifiers for review. The review sub-segments whose conflict severity classification in the cross-channel content conflict identifier set reaches the severe level are marked as confirmed violation review segments. The start and end timestamps of the confirmed violation review segments are updated to the joint non-compliance start and end timestamp fields of the corresponding tags in the non-compliance event location tag sequence. Based on the updated non-compliance event location tag sequence, a platform interception response instruction carrying the publisher account identifier and the updated non-compliance event location tag sequence is regenerated, and the platform interception response instruction is sent to the platform content distribution gateway.

7. The method for multimodal content review on an internet platform according to claim 1, characterized in that, The method further includes: Construct a publisher behavior tracking record storage space for the aforementioned Internet platform. The publisher behavior tracking record storage space is used to store the publisher's historical violation behavior sequence and publisher's content publishing habit parameters with the publisher account identifier as the primary key. Upon receiving the sequence of non-compliant event location tags, the joint non-compliance type code, joint non-compliance start timestamp, and joint non-compliance end timestamp of each non-compliant event location tag in the sequence are extracted. The duration of the current non-compliant event is calculated based on the joint non-compliance start timestamp and joint non-compliance end timestamp. The joint non-compliance type code and the duration are written into the new record of the publisher's historical violation behavior sequence with the publisher account identifier as the primary key. Statistical analysis is performed on the updated sequence of publishers’ historical violations to calculate the frequency of repetition of the same joint non-compliance type code for the publishers corresponding to the publishers’ account identifiers within a preset backtracking period and the average time interval between two adjacent non-compliance events. Content features are abstracted for compliant screen frame intervals that do not trigger cross-channel content conflict flags in the multimodal hybrid publishing unit. The distribution pattern of regional visual description tuples of screen segmentation units within the compliant screen frame intervals is extracted and recorded as publisher content publishing habit parameters. A dynamic review intensity adjustment factor is generated for the publisher account identifier based on the repetition frequency parameter and the average time interval parameter. When the dynamic review intensity adjustment factor exceeds the preset adjustment limit, a publisher key monitoring mark instruction is generated. The publisher key monitoring mark instruction is associated with the publisher account identifier and stored in the publisher behavior tracking record storage space. The dynamic review intensity adjustment factor is used to adjust the conflict judgment threshold in the cross-channel content conflict detection processing when the publisher account identifier is subsequently received as the content stream to be reviewed. When receiving content streams to be reviewed from the Internet platform, the publisher account identifier associated with the content stream to be reviewed is parsed first, and the publisher behavior tracking record storage space is queried to see if there is a publisher key monitoring mark instruction corresponding to the publisher account identifier. If the publisher's key monitoring mark instruction exists, the multimodal hybrid publishing unit corresponding to the content stream to be reviewed will be directly sent to the high-priority processing queue in the distributed review processor cluster. At the same time, the review strategy orchestration instruction set for the publisher account identifier in the review rule scheduling library will be replaced with an enhanced review strategy orchestration instruction set. The enhanced review strategy orchestration instruction set includes a newly added sensitive word deep analysis review processor unit and a newly added screen detail secondary screening review processor unit. The number of review processor units in the activation order description of the enhanced review strategy orchestration instruction set is greater than the number of review processor units in the review strategy orchestration instruction set.

8. The method for multimodal content review on an internet platform according to claim 1, characterized in that, The method further includes: When the multimodal hybrid publishing unit is successfully blocked from entering the public content display area by the platform's interception response command, the publisher account identifier and the non-compliance event location tag sequence corresponding to the intercepted multimodal hybrid publishing unit are extracted. Based on the joint non-compliance start timestamp and joint non-compliance end timestamp in the non-compliance event location tag sequence, a snapshot of the non-compliance frame interval is extracted from the visual frame sequence of the multimodal hybrid publishing unit. The original visual frames of the non-compliant frame interval snapshots are pixel information blurred to hide specific visual content, and the boundary coordinate sequence of the screen segmentation area units obtained after the non-compliant frame interval snapshots are processed by the screen region segmentation operation is retained as the abstract feature of the screen structure. Based on the joint non-compliance start timestamp and joint non-compliance end timestamp, non-compliant audio sampling interval segments are extracted from the audio sampling point sequence of the multimodal hybrid publishing unit. Acoustic feature extraction processing is performed on the non-compliant audio sampling interval segments, and the spectral fluctuation change pattern sequence obtained after the non-compliant audio sampling interval segments are processed by frame segmentation and windowing is retained as the sound contour abstract feature. The abstract features of the image structure, the abstract features of the sound contour, and the joint non-compliance type encoding in the non-compliance event location tag sequence are combined into an interception evidence summary data packet. The interception evidence summary data packet is then bound to the publisher account identifier and the current system timestamp and stored in the cloud interception evidence archive area. A violation notification message carrying the access identifier of the intercepted evidence archive area is generated. Based on the publisher's account identifier, the platform user database is queried for the publisher's bound communication address. The violation notification message is sent to the communication receiving terminal corresponding to the bound communication address. Simultaneously, the combined non-compliance type code and the publisher's account identifier in the intercepted evidence digest data packet are broadcast to the consensus node of the internet platform's audit consortium blockchain. The consensus node records the combined non-compliance type code and the publisher's account identifier in a distributed tamper-proof ledger. The violation notification message includes templated text explaining that the published content violates platform regulations and the access identifier. Receive the on-chain evidence storage transaction identifier returned by the consensus node, and supplement the on-chain evidence storage transaction identifier into the metadata field of the corresponding intercepted evidence digest data packet in the cloud intercepted evidence archive area.

9. A multimodal content moderation system for internet platforms, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to execute the multimodal content moderation method for an internet platform as described in any one of claims 1 to 8 by executing the machine-executable instructions.

Images