Digital marketing big data processing method and system
By processing marketing data using encrypted communication index tables, the isolated forest algorithm, and the BitFit algorithm, the problems of distribution imbalance and anomaly detection in multi-source data processing are solved, achieving efficient data mapping and intelligent scheduling, and improving the accuracy and response speed of marketing strategies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU BIG EVENT NETWORK TECH CO LTD
- Filing Date
- 2025-12-25
- Publication Date
- 2026-05-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In existing technologies, multi-source marketing data processing suffers from problems such as unbalanced distribution, low positioning efficiency, inaccurate bid value filtering, insufficient anomaly detection, lack of scheduling flexibility, and low level of intelligence, resulting in insufficient data reliability and processing efficiency.
The algorithm employs an encrypted communication index table, an isolated forest algorithm, and a BitFit algorithm to process advertising bid values, generating a secure aggregated gradient set. It then generates a locally adapted weight matrix through inner product calculation and matrix cutting. Combined with index rearrangement and scheduling task allocation, it achieves targeted data scheduling and anomaly detection.
It enables efficient mapping and allocation of multi-source heterogeneous data, improves data reliability and processing efficiency, enhances the intelligence and data-driven capabilities of marketing strategies, and shortens response time.
Smart Images
Figure CN121997190A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method and system for processing digital marketing big data. Background Technology
[0002] The purpose of the data processing technology field is to transform and utilize raw data in an orderly, efficient, and structured manner to support subsequent calculations, analysis, modeling, and decision-making. It aims to solve the processing flow of massive, multi-source, and heterogeneous data after collection, improve the availability, consistency, and computing performance of data, make the data analyzable and operable, and provide stable and accurate data support for upper-level algorithms, application systems, and business processes.
[0003] A digital marketing big data processing method aims to achieve accurate customer identification, marketing path evaluation, and campaign performance prediction. The goal is to process large-scale raw data in marketing scenarios in a systematic and structured manner, support the quantitative analysis and dynamic adjustment of marketing strategies, improve the accuracy of customer conversion rate identification, shorten data processing and response time, and enhance data-driven marketing automation capabilities.
[0004] In existing technologies, multi-source marketing data is often stored directly in the form of flattening and table joins, lacking mapping processing based on the total number of nodes or bit operations. This leads to distribution imbalance and low positioning efficiency. Bid values are only filtered based on fixed intervals or simple deviations, lacking a method to measure anomalies based on the performance of samples in multiple random tree depths. This results in misjudging and concealing anomalies due to short-term fluctuations, affecting the reliability of subsequent data. After matrix updates, column-wise average replacement is usually not performed, which easily leads to parameter gaps between shards. Task allocation is mostly executed according to the static table order, without retaining the original index mapping and dynamic paragraph rearrangement. This results in a lack of flexible traceability in scheduling, and the execution order is difficult to dynamically optimize according to data differences, further limiting the level of intelligent scheduling. Summary of the Invention
[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a digital marketing big data processing method and system.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: a digital marketing big data processing method, comprising the following steps: S1: Based on the identifiers of each marketing participant and the training round number, extract the channel distribution vector and click event field, split the spliced identifier into 8 bits, shift it to the left, and XOR them sequentially. Take the remainder of the merged result to get the total number of nodes and generate an encrypted communication index table. S2: Based on the encrypted communication index table, the isolated forest algorithm is used to convert the advertising bid value into fixed-point form according to the numerical precision. Then, the modulus multiplication and accumulation operations are performed on each bit value. The resulting ciphertext sequence is compared with the digest bit sequence of each participant bit by bit to perform consistency verification and generate a secure aggregate gradient set. S3: Based on the aforementioned secure aggregated gradient set, the BitFit algorithm is used to perform inner product calculation on the row vector and column vector of the advertising weight submatrix. The feature vector is subtracted from the atomic array item by item to obtain the difference vector. Each element of the difference vector is multiplied by the learning rate and then added to the corresponding element of the original submatrix to obtain the local adaptation weight matrix. S4: Based on the local adaptation weight matrix, the matrix is horizontally cut to extract sub-matrices. The sub-matrices are covered with the average gradient matrix of the corresponding column in the safe aggregated gradient set, and all sub-matrices are reassembled in the original order to generate a sequence of field identifier value vectors. S5: Based on the scheduling task allocation list, the sequence is divided into several position segments with a fixed length and the order is rearranged. The index mapping relationship of each position segment in the original sequence is extracted. The segment rearrangement and numbering update are performed in combination with the displacement difference of each segment to obtain the recombined position index table.
[0007] As a further embodiment of the present invention, the encrypted communication index table includes participant identifier mapping values, training round index values, XOR merging results, and modulo index values; the secure aggregation gradient set specifically includes each participant's encrypted gradient sequence, accumulated encrypted text result, digest bit sequence, and consistency check identifier; the local adaptation weight matrix includes updated row vector values, updated column vector values, difference accumulation result, and learning rate product matrix; the field identifier value vector sequence specifically includes covered submatrix fragments, splicing order index, vectorized numerical array, and sequence length marker; and the recombined positional index table includes original index mapping, new index mapping, shift difference list, and number update mapping.
[0008] As a further aspect of the present invention, the specific steps for generating the encrypted communication index table are as follows: Based on the identifiers of each marketing participant and the training round number, the channel distribution vector and click event field are extracted. The channel identifier and click value are read by field positioning and the sequence is aligned. The field values are concatenated and divided into 8-bit segments according to the byte boundaries to generate an identifier string segment set. Based on the segmented set of the identifier string, each segment is left-shifted and XORed together. The merged result is then divided by the number of nodes to obtain the remainder, generating an encrypted communication index table.
[0009] As a further aspect of the present invention, the specific steps for generating the secure aggregated gradient set are as follows: Based on the encrypted communication index table, extract the bid value of each advertisement, truncate the decimal places and multiply by a fixed multiple to convert it into an integer, verify the numerical range of the integer value sequence and clear overflow items, and generate a list of fixed-point numbers. Based on the fixed-point list, the isolated forest algorithm is used to evaluate the abnormal scores of each value in the list in real time, and remove bids with scores exceeding the threshold. The peak and valley values are extracted and multiplied with a specified modulus. The product is added bitwise and the sum is adjusted item by item to generate a cumulative encrypted text sequence. Based on the accumulated encrypted text sequence, each bit in the ciphertext sequence is compared one by one with the digest bits provided by the corresponding participant. The comparison results are marked as consistent and conflicting, and then statistical analysis is performed to obtain a secure aggregated gradient set.
[0010] As a further aspect of the present invention, the isolated forest is based on a list of fixed-point advertising bid values extracted from the encrypted communication index table. Each value is input into the isolated forest model, and an anomaly score is calculated based on the path length in the binary trees constructed from multiple random subsamples. Each tree is constructed by randomly selecting subsamples from the original dataset. Node partitioning is performed based on random selection of features and random partitioning of thresholds until the samples are completely isolated and the preset tree depth is reached. The anomaly score is derived from the difference between the average path length of the samples and the theoretically expected path length. All fixed-point values are judged according to the score, and samples with scores exceeding the set threshold are removed. Bids judged as normal are retained and enter the subsequent processing flow.
[0011] As a further aspect of the present invention, the specific steps for generating the local adaptation weight matrix are as follows: Based on the secure aggregated gradient set, read the row vectors of each gradient and the corresponding positions of the global weight matrix, match and verify the row length and record the valid index segments to generate a gradient row vector set. Based on the gradient row vector set, multiplication operations are performed on the corresponding elements of each group of vectors in index-aligned order and the product sequence is recorded. Row-level accumulation operation is performed on the product sequence and stored in the temporary storage area to obtain the row vector update segment. Based on the row vector update segment, the BitFit algorithm is used to add each updated value to the current column vector element and multiply it by a set scaling factor. The updated values are then merged into the corresponding area of the original matrix according to the column index to obtain the local adaptation weight matrix.
[0012] As a further aspect of the present invention, the BitFit algorithm selects the set of column vector elements corresponding to the update segment index in the original global weight matrix, adds each update value to the corresponding column vector element, and immediately multiplies the addition result by a preset scaling factor, which is a fixed scalar. Then, it writes the result to the corresponding matrix area according to the original column index position to complete the local fine-tuning update of the weight matrix.
[0013] As a further aspect of the present invention, the specific steps for generating the field identifier value vector sequence are as follows: Based on the local adaptation weight matrix, the matrix is first divided into segments from left to right according to the column width, and then the corresponding column blocks are extracted from each segment and stored in a list. The start and end column indices of each segment are marked to generate a list of sub-matrix blocks. Based on the submatrix block list, the values in the corresponding column positions of each segment are replaced with the average value of the same column in the safe aggregate gradient set, and the segments are concatenated to restore the complete matrix according to the original column index order, generating a sequence of field identifier value vectors.
[0014] As a further aspect of the present invention, the specific steps for generating the scheduling task allocation list are as follows: Based on the scheduling task allocation list, the list is split into several segments of fixed length starting from the beginning. Each segment is assigned a number and the index position of the elements in the segment in the original list is recorded to generate a position segment mapping set. Based on the position segment mapping set, the element order within each segment is rearranged according to the original index mapping result, and the segment number mapping table is updated. All segments are then output by number combination to generate a recombined positional index table.
[0015] A digital marketing big data processing system, wherein the digital marketing big data processing system is used to execute the above-mentioned digital marketing big data processing method, the system comprising: The identifier index construction module extracts the channel distribution vector and click event field based on the marketing participant identifier and training round number, splices and divides them into 8-bit segments, and then takes the remainder after left shifting and XORing the merged segments and taking the remainder after dividing by the total number of nodes to generate an encrypted communication index table. The bidding anomaly detection module extracts the advertising bid values based on the encrypted communication index table, truncates the decimals, multiplies them by a fixed multiple to convert them to integers, and removes items exceeding the threshold to form a fixed-point list. It uses isolated forest multi-tree split comparison to obtain the average path value to calculate the anomaly score, removes the anomaly values, selects the peak and valley values and multiplies them with the specified modulus, performs bit-by-bit adjustment to obtain the cumulative encrypted text sequence, compares and statistically analyzes it bit by bit with the digest bit sequence to obtain the secure aggregated gradient set. The weight matrix adaptation module extracts global matrix row vectors based on the secure aggregated gradient set, compares the length and records the index, multiplies each item and accumulates the rows to generate row vector update segments, extracts columns, performs addition and proportional product, updates the corresponding positions in the matrix using BitFit, and obtains the local adaptation weight matrix. Matrix gradient coverage module: Based on the local adaptation weight matrix, the sub-matrix block list is obtained by dividing it by column, and the average value of the corresponding column of the safe aggregate gradient set is replaced. The sub-matrix blocks are then concatenated according to the original index to form a vector sequence of field identifier values. Sequence Index Rearrangement Module: Based on the field identifier value vector sequence and the scheduling task allocation list, a position segment mapping set is formed by recording the index of the segment number with a fixed length. The rearrangement and combination of the mappings are used to output a recombined position index table.
[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by extracting channel distribution and click fields from marketing participant identifiers and training round numbers, and performing segmentation, left shift, XOR, and remainder calculations, the resulting index structure completes the mapping and allocation of multi-source heterogeneous inputs, realizing node-level data-oriented scheduling; In this invention, the advertising bid value is converted to an integer by truncating the decimal and multiplying by a multiple. After removing out-of-range values, the anomaly score is calculated based on the multi-tree split depth of the isolated forest. Anomalies are removed, and then bit addition and carry are performed using the peak and valley values and the modulus to generate an accumulated encrypted text sequence. The secure aggregated gradient set is obtained by comparing and statistically analyzing the bit sequence bit by bit, so that sensitive data can be verified and consistent among multiple parties. In this invention, the row vectors corresponding to the aggregated gradient set are multiplied by index and accumulated at the row level to obtain the update segment. Then, the segment is added to the corresponding column vectors and multiplied by a set scaling factor to perform local updates. The BitFit method is used to adjust the weights of key parameters of the submatrix only, avoiding global synchronization and reducing the computational load. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the workflow of the present invention; Figure 2 This is a system flowchart of the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0019] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0020] Example 1 Please see Figure 1 This invention provides a technical solution: a digital marketing big data processing method, comprising the following steps: S1: Based on the identifiers of each marketing participant and the training round number, extract the channel distribution vector and click event field, split the spliced identifier into 8 bits, shift it to the left, and XOR them sequentially. Take the remainder of the merged result to get the total number of nodes and generate an encrypted communication index table. S2: Based on the encrypted communication index table, the isolated forest algorithm is used to convert the advertising bid value into fixed-point form according to the numerical precision. Then, the modulus multiplication and accumulation operations are performed on each bit value. The resulting ciphertext sequence is compared with the digest bit sequence of each participant bit by bit to perform consistency verification and generate a secure aggregate gradient set. S3: Based on the secure aggregated gradient set, the BitFit algorithm is used to perform inner product calculation on the row vector and column vector of the advertising weight submatrix. The feature vector is subtracted from the atomic array item by item to obtain the difference vector. Each element of the difference vector is multiplied by the learning rate and added to the corresponding element of the original submatrix to obtain the local adaptation weight matrix. S4: Based on the local adaptation weight matrix, the matrix is horizontally cut to extract sub-matrices. The sub-matrices are then covered with the average gradient matrix of the corresponding column in the safe aggregated gradient set. All sub-matrices are then reassembled in their original order to generate a sequence of field identifier value vectors. S5: Based on the scheduling task allocation list, the sequence is divided into several position segments with a fixed length and the order is rearranged. The index mapping relationship of each position segment in the original sequence is extracted. Combined with the displacement difference of each segment, the segment rearrangement and numbering update are performed to obtain the recombined position index table.
[0021] The encrypted communication index table includes participant identifier mapping values, training round index values, XOR merging results, and modulo index values. The secure aggregated gradient set specifically includes the encrypted gradient sequence of each participant, the accumulated encrypted text result, the summary bit sequence, and the consistency check identifier. The local adaptation weight matrix includes the updated row vector values, the updated column vector values, the difference accumulation result, and the learning rate product matrix. The field identifier value vector sequence specifically includes the covered submatrix fragment, the splicing order index, the vectorized numerical array, and the sequence length marker. The recombined positional index table includes the original index mapping, the new index mapping, the shift difference list, and the number update mapping.
[0022] The specific steps for generating the encrypted communication index table are as follows: Based on the identifiers of each marketing participant and the training round number, the channel distribution vector and click event field are extracted. The channel identifier and click value are read by field positioning and the sequence is aligned. The field values are concatenated and divided into 8-bit segments according to the byte boundaries to generate an identifier string segment set. Based on the segmented set of the identifier string, each segment is left-shifted and XORed together. The merged result is then divided by the number of nodes and the remainder is taken to generate an encrypted communication index table. Based on the identifiers of multiple marketing participants and the training round number, the channel distribution vector and click event field are extracted. The channel identifier and click value are read by field positioning. The sequence is aligned using a dynamic time warping algorithm. The time window is set to 1 second. The elements in the channel distribution vector and click event field are compared pairwise according to the time order. The shortest path is matched based on the absolute difference. Abnormal data with an interval of more than 1 second is ignored during the matching process. The sequence alignment operation is completed. The field values are concatenated. Byte partitioning is used. Each 8-bit binary is set as the partitioning unit. The concatenation result is read bit by bit and sliced into 8-bit segments to generate the identifier string segment set. Based on the identifier string segment set, a combination of bit shift and XOR operation is used to perform a left shift operation on each segment value, with the left shift bit set to 3 bits. After the left shift, the result is immediately XORed with the next segment value. All segments are processed sequentially to finally obtain a single value merged result. For the merged result, a modulo operation is performed, with the number of nodes set to 256. The modulo result is used as the index value to generate an encrypted communication index table.
[0023] The specific steps for generating a secure aggregated gradient set are as follows: Based on the encrypted communication index table, extract the bid value of each advertisement, truncate the decimal places and multiply by a fixed multiple to convert it into an integer, verify the numerical range of the integer value sequence and remove overflow items, and generate a list of fixed-point numbers; Based on a fixed-point list, the isolated forest algorithm is used to evaluate the abnormal scores of each value in the list in real time, and remove bids with scores exceeding the threshold. The peak and valley values are extracted and multiplied with a specified modulus. The product is added bitwise and the sum is adjusted item by item to generate a cumulative encrypted text sequence. Based on the accumulator sequence, each bit in the ciphertext sequence is compared one by one with the digest bits provided by the corresponding participant. The comparison results are marked as consistent and conflicting, and statistical analysis is performed to obtain a secure aggregate gradient set. Based on the encrypted communication index table, the bid value of each advertisement is extracted. A decimal truncation operation is performed, with the truncation number set to four decimal places. The truncated value is multiplied by a fixed multiple, which is set to 10000. The product result is converted to an integer. A range verification operation is performed, with the lower limit of the verification range set to 0 and the upper limit set to 100000000. Values exceeding the upper limit are deleted. During deletion, index positioning is used to remove excess value items in order, generating a list of fixed-point numbers. Based on a list of fixed-point numbers, the Isolation Forest algorithm is used, with a subsample size of 256, a tree size of 100, and a tree height limit of 8. During the training process, 256 fixed-point numbers are sampled sequentially to construct 100 binary random partition trees. Each node is traversed sequentially, and a random attribute partitioning method is used to record the node partitioning range as a binary left-closed and right-open interval. Anomaly scores are calculated using path length, and the scores are compared with a set threshold of 50. Bids with anomaly scores higher than the threshold are deleted. During deletion, the bids are deleted one by one according to the index order of the fixed-point number list. The peak value is extracted from the remaining bids, which is the maximum value in the current fixed-point number list. The valley value is extracted, which is the minimum value in the current fixed-point number list. The peak value and valley value are multiplied by a specified modulus, which is set to 500. The product result is added bitwise, and binary addition is performed bit by bit. When a carry is encountered, it is carried over to the higher bit for adjustment. After all carry is completed, the sum is recorded to generate an accumulated encrypted text sequence. Based on the accumulator sequence, each bit in the ciphertext sequence is compared one by one with the digest bit provided by the corresponding participant. The comparison operation adopts a bit-by-bit scanning comparison command, and the scanning order is from low bit to high bit. When the bit values are the same, they are marked as a matching item, and when the bit values are different, they are marked as a conflict item. The number of matching items and the number of conflict items are counted to generate a secure aggregate gradient set. Isolation Forest is a model that uses a list of ad bid point values extracted from an encrypted communication index table. Each value is input into the Isolation Forest model, and anomaly scores are calculated based on the path lengths in binary trees constructed from multiple random subsamples. Each tree is constructed by randomly selecting subsamples from the original dataset. Node partitioning is performed based on random selection of features and random partitioning of thresholds until the samples are completely isolated and the preset tree depth is reached. The anomaly score is derived from the difference between the average path length of the samples and the theoretically expected path length. All point values are judged according to the score, and samples with scores exceeding the set threshold are removed. Bids judged as normal are retained and enter the subsequent processing flow. An isolated forest, according to the formula:
[0024] in: Indicates the first The fixed-point value of digital marketing bid data, This indicates the total length of the list of digital marketing bid data. Indicates the first The confidence probability density of each bid data point in the historical bid probability model. This represents the deviation adjustment factor used to adjust the intensity of the influence of the confidence probability density. Indicates the first The time weight of each bid data point within a local time window. This represents the total dimension of the features used for splitting nodes in an isolated forest model. Indicating the isolation of the forest for the first Estimation of the average path length of bid data. Representation and data size The relevant path length normalization constant, Indicates the first Corrected anomaly scores for bid data in the improved isolated forest model; Execution process: First, the isolated forest model is used to construct multiple binary tree structures with random splitting characteristics, and the bid data for each item is calculated. Average path length across all trees This is used to measure the degree of isolation, and then extract... The corresponding confidence probability density in the historical bid probability model This is achieved by fitting a Beta distribution to a large amount of historical bidding behavior data, reflecting the reasonableness of the data in the historical distribution. Then, a deviation adjustment factor is introduced. Cross-validation is used to automatically adjust parameters within a set range to optimize anomaly detection performance, thereby adjusting the exponential amplification effect of confidence density, and then based on bidding data. In contrast to the timeliness of the current time window, the next step is to record the total number of features participating in random splits in the isolated forest model. This is used to adjust the denominator of the scoring formula to take into account the complexity of the feature dimensions. Finally, the above parameters are substituted into the scoring formula to calculate the corrected anomaly score for each data point. It is used to identify and remove abnormal bids in real time, thereby improving the effectiveness and accuracy of subsequent peak and valley extraction and encryption calculations.
[0025] The specific steps for generating the local adaptation weight matrix are as follows: Based on the secure aggregated gradient set, read the row vector of each gradient and the corresponding position of the global weight matrix, match and verify the row length and record the valid index segment to generate a gradient row vector set; Based on the set of gradient row vectors, multiplication is performed on each element of each group of vectors in index-aligned order, and the product sequence is recorded. Row-level accumulation is performed on the product sequence and stored in the temporary storage area to obtain the row vector update segment. Based on the row vector update segment, the BitFit algorithm is used to add each updated value to the current column vector element and multiply it by a set scaling factor. The updated values are then merged into the corresponding area of the original matrix according to the column index to obtain the local adaptation weight matrix. Based on the secure aggregated gradient set, the row vectors corresponding to the positions of each gradient group and the global weight matrix are read. The index-by-index reading command is used, the reading step size is set to 1, and the reading order starts from the first row of the matrix. The number of elements in the gradient group is matched with the length of the corresponding row vector of the global weight matrix. The length verification command is used, and row vectors with inconsistent lengths are skipped. The index positions of row vectors with consistent lengths are recorded. The start and end positions of the valid index segment are set to generate a set of gradient row vectors. Based on the gradient row vector set, index order alignment is adopted, and one-to-one correspondence is matched between the gradient group index and the weight matrix row index. Multiplication operation is performed on each element of the corresponding vector group one by one. Element-wise multiplication command is adopted. The operation order is set to start from the first element of the vector and process all elements in turn. All products are recorded as a product sequence. Row-level accumulation operation is performed on the product sequence. Vector element summation is performed row by row. The accumulation result is stored in the temporary storage area. The index of the temporary storage area and the storage order are recorded to obtain the row vector update segment. Based on the row vector update segment, the BitFit algorithm is used to perform addition operations on each updated value and the current column vector element. The column processing order is set, and the column-by-column addition command is executed. The scaling factor is set to 0.01, and the multiplication command is executed. The addition result is multiplied by the scaling factor. After processing, the matrix is merged into the corresponding area of the original matrix according to the column index position. The matrix block overwrite command is used to maintain strict consistency of column index during the writing process, and a local adaptation weight matrix is generated. The BitFit algorithm selects the set of column vector elements in the original global weight matrix that correspond to the index of the updated segment, adds each updated value to the corresponding column vector element, and immediately multiplies the addition result by a preset scaling factor, which is a fixed scalar. Then, it writes the result to the corresponding matrix area according to the original column index position to complete the local fine-tuning update of the weight matrix. The BitFit algorithm, according to the formula:
[0026] in: Represents the first element in the original weight matrix. Line number The values in the column, Indicates the first segment in the update segment Line number Incremental value of column This represents the local learning rate, used to control the update magnitude. Indicates the first The row importance coefficient corresponding to the row, Indicates the first The column stability factor corresponding to the column. This represents the exponential adjustment parameter of the column stability factor. This indicates the total number of feature dimensions that participate in the BitFit update locally. This represents the global scaling factor for structural correlation offset correction. Indicates the first Line number Column structural similarity offset value, Indicates the th element in the corrected local adaptation weight matrix. Line number The final result of the column; Execution process: First, extract the first weight from the original weight matrix. Line number Column elements And combined with the incremental value calculated in the update segment Through local learning rate right After scaling and The values are added together to form an initial adjustment item, and then a row importance coefficient is assigned to each row. At the same time, a column stability factor is assigned to each column. This is used to reflect the parameter stability of each marketing feature dimension in the historical model, and then the parameters are adjusted using an exponential method. right Power-law scaling is applied to enhance the influence of highly stable features on the results, followed by updating the total number of feature dimensions locally. Calculate the normalization factor for the principal numerator term. To balance the biases caused by updates in different dimensions, structural correlation offsets are then introduced. It is obtained by calculating the cosine similarity between adjacent weighted units in the same sliding window and multiplying it by a normalized offset correction coefficient. The result is then superimposed into the main calculation result to obtain the final result. Line number Column-corrected local adaptation weight values It is used to build a personalized weight matrix that adapts to the current marketing environment.
[0027] The specific steps for generating the field identifier value vector sequence are as follows: Based on the local adaptation weight matrix, the matrix is first divided into segments from left to right according to the column width, and then the corresponding column blocks are extracted from each segment and stored in a list. The start and end column indices of each segment are marked to generate a list of sub-matrix blocks. Based on the list of submatrix blocks, the values in the corresponding column positions of each segment are replaced with the average value of the same column in the safe aggregate gradient set. The segments are then concatenated to restore the complete matrix according to the original column index order, generating a sequence of field identifier value vectors. Based on the locally adapted weight matrix, the matrix is segmented from left to right according to the column width. A fixed-length segmentation command is used, with each segment having a column width of 4 columns. The segmentation starts from the first column and is divided sequentially. Each segment extracts the corresponding column block using a column block reading command. The reading order is from the left side of the matrix to the right side, with a reading step of 4 columns. After extracting the column block, it is stored in a list. The list structure uses a sequential linked list to store the starting and ending column indices of each segment. The index record format is a continuous integer index, generating a submatrix block list. Based on the list of submatrix blocks, the values at the corresponding column positions in the secure aggregated gradient set are read according to the column index order of the submatrix block records. A column mean calculation command is used, and during the calculation process, the current column index is set, and all values in the current column are read one by one. After the values are summed, they are divided by the number of values in the current column to calculate the average value of the same column. All values at the current column position in the submatrix block are replaced with the calculated average value. The replacement is done using an element-wise overwrite command. After all column blocks are processed, a matrix concatenation operation is performed according to the original column index order. During the concatenation process, all column blocks are connected according to the column index order to generate a vector sequence of field identifier values.
[0028] The specific steps for generating the task allocation list are as follows: Based on the task allocation list, the list is split into several segments of fixed length starting from the beginning. Each segment is assigned a number and the index position of the elements in the segment in the original list is recorded to generate a position segment mapping set. Based on the position segment mapping set, the element order in each segment is rearranged according to the original index mapping result, and the segment number mapping table is updated at the same time. All segments are output by number combination to generate a reorganized positional index table. Based on the task allocation list, a fixed-length splitting command is used to split the list from the beginning into segments of a fixed length, with each segment having 10 elements. During splitting, the list elements are read sequentially, and each segment is divided into 10-element segments. The reading order starts from the first element of the list. After splitting, each segment is assigned a number, which is incremented sequentially, starting from 1. The index positions of all elements in each segment in the original list are recorded, and the index records are stored in a sequential format to generate a position segment mapping set. Based on the position segment mapping set, according to the original index mapping result, the index mapping rearrangement command is used to read segment by segment. The elements in each segment are reordered according to their index positions in the original list. The rearrangement adopts ascending order sorting, and the sorting algorithm adopts insertion sort. The initial step size of insertion sort is set to 1. Each time the current segment element is traversed, the current element is compared with the previous element. If the index of the current element is less than the index of the previous element, the positions are swapped until all elements in the segment are arranged in ascending order by index. After sorting, the segment number mapping table is updated. During the update, the mapping table content is rewritten according to the segment index order. All segments are output by number combination. The combination adopts the segment order concatenation command to generate a reorganized positional index table.
[0029] A digital marketing big data processing system, used to execute the aforementioned digital marketing big data processing method, the system comprising: The identifier index construction module extracts the channel distribution vector and click event field based on the marketing participant identifier and training round number, splices and divides them into 8-bit segments, and then takes the remainder after left shifting and XORing the merged segments and taking the remainder after dividing by the total number of nodes to generate an encrypted communication index table. The bid anomaly detection module extracts ad bid values from the encrypted communication index table, truncates the decimals, multiplies them by a fixed multiple to convert them to integers, and removes items exceeding the threshold to form a fixed-point list. It uses isolated forest multi-tree split comparison to obtain the average path value to calculate the anomaly score, removes anomaly values, selects peak and valley values and multiplies them with a specified modulus, performs bit-by-bit adjustment to obtain the cumulative encrypted text sequence, compares and statistically analyzes it bit by bit with the digest bit sequence to obtain a secure aggregated gradient set. The weight matrix adaptation module extracts global matrix row vectors based on the secure aggregated gradient set, compares the length and records the index, multiplies each item and accumulates the rows to generate row vector update segments, extracts columns, performs addition and proportional product and updates the corresponding positions in the matrix using BitFit to obtain the local adaptation weight matrix. Matrix gradient coverage module: Based on the locally adapted weight matrix, the sub-matrix block list is obtained by dividing it by column, and then replaced with the average value of the corresponding column of the safe aggregated gradient set. The sub-matrix blocks are then concatenated according to the original index to form a vector sequence of field identifier values. Sequence Index Reordering Module: Based on the field identifier value vector sequence and the scheduling task allocation list, the index of fixed-length segment number records forms a position segment mapping set, and the reordering combination according to the mapping outputs a reorganized position index table.
[0030] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.
Claims
1. A method for processing digital marketing big data, characterized in that, Includes the following steps: S1: Based on the identifiers of each marketing participant and the training round number, extract the channel distribution vector and click event field, split the spliced identifier into 8 bits, shift it to the left, and XOR them sequentially. Take the remainder of the merged result to get the total number of nodes and generate an encrypted communication index table. S2: Based on the encrypted communication index table, the isolated forest algorithm is used to convert the advertising bid value into fixed-point form according to the numerical precision. Then, the modulus multiplication and accumulation operations are performed on each bit value. The resulting ciphertext sequence is compared with the digest bit sequence of each participant bit by bit to perform consistency verification and generate a secure aggregate gradient set. S3: Based on the aforementioned secure aggregated gradient set, the BitFit algorithm is used to perform inner product calculation on the row vector and column vector of the advertising weight submatrix. The feature vector is subtracted from the atomic array item by item to obtain the difference vector. Each element of the difference vector is multiplied by the learning rate and then added to the corresponding element of the original submatrix to obtain the local adaptation weight matrix. S4: Based on the local adaptation weight matrix, the matrix is horizontally cut to extract sub-matrices. The sub-matrices are covered with the average gradient matrix of the corresponding column in the safe aggregated gradient set, and all sub-matrices are reassembled in the original order to generate a sequence of field identifier value vectors. S5: Based on the scheduling task allocation list, the sequence is divided into several position segments with a fixed length and the order is rearranged. The index mapping relationship of each position segment in the original sequence is extracted. The segment rearrangement and numbering update are performed in combination with the displacement difference of each segment to obtain the recombined position index table.
2. The digital marketing big data processing method according to claim 1, characterized in that, The encrypted communication index table includes participant identifier mapping values, training round index values, XOR merging results, and modulo index values. The secure aggregated gradient set specifically includes each participant's encrypted gradient sequence, accumulated encrypted text result, digest bit sequence, and consistency check identifier. The local adaptation weight matrix includes updated row vector values, updated column vector values, difference accumulation result, and learning rate product matrix. The field identifier value vector sequence specifically includes covered submatrix fragments, splicing order index, vectorized numerical array, and sequence length marker. The recombined positional index table includes original index mapping, new index mapping, shift difference list, and number update mapping.
3. The digital marketing big data processing method according to claim 1, characterized in that, The specific steps for generating the encrypted communication index table are as follows: Based on the identifiers of each marketing participant and the training round number, the channel distribution vector and click event field are extracted. The channel identifier and click value are read by field positioning and the sequence is aligned. The field values are concatenated and divided into 8-bit segments according to the byte boundaries to generate an identifier string segment set. Based on the segmented set of the identifier string, each segment is left-shifted and XORed together. The merged result is then divided by the number of nodes to obtain the remainder, generating an encrypted communication index table.
4. The digital marketing big data processing method according to claim 1, characterized in that, The specific steps for generating the secure aggregated gradient set are as follows: Based on the encrypted communication index table, extract the bid value of each advertisement, truncate the decimal places and multiply by a fixed multiple to convert it into an integer, verify the numerical range of the integer value sequence and clear overflow items, and generate a list of fixed-point numbers. Based on the fixed-point list, the isolated forest algorithm is used to evaluate the abnormal scores of each value in the list in real time, and remove bids with scores exceeding the threshold. The peak and valley values are extracted and multiplied with a specified modulus. The product is added bitwise and the sum is adjusted item by item to generate a cumulative encrypted text sequence. Based on the accumulated encrypted text sequence, each bit in the ciphertext sequence is compared one by one with the digest bits provided by the corresponding participant. The comparison results are marked as consistent and conflicting, and then statistical analysis is performed to obtain a secure aggregated gradient set.
5. The digital marketing big data processing method according to claim 4, characterized in that, The isolated forest model extracts a list of ad bid point values from the encrypted communication index table. Each value is input into the isolated forest model, and anomaly scores are calculated based on the path lengths in the binary trees constructed from multiple random subsamples. Each tree is constructed by randomly selecting subsamples from the original dataset. Node partitioning is performed based on random selection of features and random partitioning of thresholds until the samples are completely isolated and the preset tree depth is reached. The anomaly score is derived from the difference between the average path length of the samples and the theoretically expected path length. All point values are judged according to the score, and samples with scores exceeding the set threshold are removed. Bids judged as normal are retained and enter the subsequent processing flow.
6. The digital marketing big data processing method according to claim 1, characterized in that, The specific steps for generating the local adaptation weight matrix are as follows: Based on the secure aggregated gradient set, read the row vectors of each gradient and the corresponding positions of the global weight matrix, match and verify the row length and record the valid index segments to generate a gradient row vector set. Based on the gradient row vector set, multiplication operations are performed on the corresponding elements of each group of vectors in index-aligned order and the product sequence is recorded. Row-level accumulation operation is performed on the product sequence and stored in the temporary storage area to obtain the row vector update segment. Based on the row vector update segment, the BitFit algorithm is used to add each updated value to the current column vector element and multiply it by a set scaling factor. The updated values are then merged into the corresponding area of the original matrix according to the column index to obtain the local adaptation weight matrix.
7. The digital marketing big data processing method according to claim 6, characterized in that, The BitFit algorithm selects the set of column vector elements in the original global weight matrix that correspond to the index of the updated segment, adds each updated value to the corresponding column vector element, and immediately multiplies the addition result by a preset scaling factor, which is a fixed scalar. Then, it writes the result to the corresponding matrix area according to the original column index position to complete the local fine-tuning update of the weight matrix.
8. The digital marketing big data processing method according to claim 1, characterized in that, The specific steps for generating the field identifier value vector sequence are as follows: Based on the local adaptation weight matrix, the matrix is first divided into segments from left to right according to the column width, and then the corresponding column blocks are extracted from each segment and stored in a list. The start and end column indices of each segment are marked to generate a list of sub-matrix blocks. Based on the submatrix block list, the values in the corresponding column positions of each segment are replaced with the average value of the same column in the safe aggregate gradient set, and the segments are concatenated to restore the complete matrix according to the original column index order, generating a sequence of field identifier value vectors.
9. The digital marketing big data processing method according to claim 1, characterized in that, The specific steps for generating the scheduling task allocation list are as follows: Based on the scheduling task allocation list, the list is split into several segments of fixed length starting from the beginning. Each segment is assigned a number and the index position of the elements in the segment in the original list is recorded to generate a position segment mapping set. Based on the position segment mapping set, the element order within each segment is rearranged according to the original index mapping result, and the segment number mapping table is updated. All segments are then output by number combination to generate a recombined positional index table.
10. A digital marketing big data processing system, characterized in that, The digital marketing big data processing method according to any one of claims 1-9, wherein the system comprises: The identifier index construction module extracts the channel distribution vector and click event field based on the marketing participant identifier and training round number, splices and divides them into 8-bit segments, and then takes the remainder after left shifting and XORing the merged segments and taking the remainder after dividing by the total number of nodes to generate an encrypted communication index table. The bidding anomaly detection module extracts the advertising bid values based on the encrypted communication index table, truncates the decimals, multiplies them by a fixed multiple to convert them to integers, and removes items exceeding the threshold to form a fixed-point list. It uses isolated forest multi-tree split comparison to obtain the average path value to calculate the anomaly score, removes the anomaly values, selects the peak and valley values and multiplies them with the specified modulus, performs bit-by-bit adjustment to obtain the cumulative encrypted text sequence, compares and statistically analyzes it bit by bit with the digest bit sequence to obtain the secure aggregated gradient set. The weight matrix adaptation module extracts global matrix row vectors based on the secure aggregated gradient set, compares the length and records the index, multiplies each item and accumulates the rows to generate row vector update segments, extracts columns, performs addition and proportional product, updates the corresponding positions in the matrix using BitFit, and obtains the local adaptation weight matrix. Matrix gradient coverage module: Based on the local adaptation weight matrix, the sub-matrix block list is obtained by dividing it by column, and the average value of the corresponding column of the safe aggregate gradient set is replaced. The sub-matrix blocks are then concatenated according to the original index to form a vector sequence of field identifier values. Sequence Index Rearrangement Module: Based on the field identifier value vector sequence and the scheduling task allocation list, a position segment mapping set is formed by recording the index of the segment number with a fixed length. The rearrangement and combination of the mappings are used to output a recombined position index table.