A method, system, electronic device, and storage medium for offline-real-time fusion feature prediction of big data based on parameter passing.

Abstract

This invention proposes a method, system, electronic device, and storage medium for offline-real-time fusion feature prediction of big data based on parameter transfer. It belongs to the field of big data processing and artificial intelligence technology. The method includes: an offline model generating offline analysis results using historical data and extracting transfer parameters; transferring the transfer parameters to a real-time model; the real-time model using the transfer parameters to expand features of real-time data and make predictions; after the offline model completes its analysis of data from the same period, fusing the offline and real-time prediction results using a confidence model fusion method; and finally, the offline model updating the transfer parameters for use by the real-time model in the next period. This invention proposes a parameter transfer mechanism and "three principles," constructing an offline-real-time collaborative closed loop. Without compromising real-time performance, it utilizes the deep features of the offline model to improve the accuracy of real-time predictions, solving the problem of balancing timeliness and accuracy in big data analysis.

Description

Technical Field

[0001] This invention relates to the fields of big data processing, machine learning and data mining technology, and in particular to a method, system, electronic device and storage medium for offline-real-time fusion feature prediction of big data based on parameter passing. Background Technology

[0002] In the field of big data feature prediction, processing modes are generally divided into two categories: offline feature recognition (batch processing) and real-time feature recognition (stream processing).

[0003] Offline prediction models have ample computing time and complete historical raw data, enabling in-depth cleaning, processing, and feature extraction. Therefore, they typically extract richer feature dimensions and have higher model accuracy. However, their drawback is poor timeliness, often only able to process data from T-1 days or even earlier. Real-time prediction models, on the other hand, process data streams rapidly, offering extremely high timeliness and meeting millisecond or second-level business needs. However, limited by extremely low latency requirements, real-time processing struggles with complex multi-table joins or deep historical feature mining, resulting in limited feature dimensions and generally lower accuracy compared to offline analysis.

[0004] In existing technologies, offline and real-time systems are often separated (e.g., SpeedLayer and Batch Layer in Lambda architecture), or simply concatenate the results. Without parameter passing, relying solely on the inherent parameters of each system for optimization makes it difficult to significantly improve accuracy without sacrificing overall comprehensiveness. How to "inherit" the high accuracy advantage of offline models to real-time models while maintaining the high timeliness of real-time models is a pressing technical problem in the field of big data. Summary of the Invention

[0005] This invention aims to solve the above-mentioned technical problems and proposes a big data offline-real-time fusion feature prediction method based on parameter transfer. By constructing a parameter transfer mechanism and a confidence fusion model, the advantages of offline and real-time prediction are complemented.

[0006] The first aspect of this invention discloses a big data offline-real-time fusion feature prediction method based on parameter passing; the method includes:

[0007] Step S1: Obtain the offline model over time period T Offline feature recognition is performed on historical data from 1 to generate time period T. The offline analysis results of 1 are used to extract the transmission parameters based on the offline analysis results;

[0008] Step S2: Transfer the transmission parameters from the offline model to the real-time model;

[0009] Step S3: The real-time model acquires real-time data for time period T, expands the features of the real-time data using the transmission parameters, constructs a real-time feature vector, and performs real-time feature prediction to obtain the real-time prediction result and first confidence level for time period T.

[0010] Step S4: After the time period T ends, the offline model acquires the full data of time period T and performs offline feature recognition to obtain the offline prediction result and second confidence level of time period T.

[0011] Step S5: Using a confidence fusion model, fuse the real-time prediction results and offline prediction results for the time period T to obtain the final prediction result;

[0012] Step S6: The offline model updates the transmission parameters based on the offline prediction results of time period T, and transmits them to the real-time model that processes data of time period T+1, and repeats the above steps in a loop.

[0013] The purpose of updating the transmission parameters in step S6 is to eliminate the accumulated error caused by the time misalignment between the offline model and the real-time model in processing data.

[0014] Preferably, in step S1, the extraction of the transmission parameters follows the following principles:

[0015] The principle of weak time correlation: ensure that the difference between the transfer parameters generated by time period T and the transfer parameters generated by time period T+1 is less than a preset threshold, i.e., P(T+1)≈P(T).

[0016] Strong feature correlation principle: The transmission parameters include rules or thresholds that can be directly mapped to low-order features of real-time data;

[0017] Real-time computability principle: The data structure for transmitting parameters meets the low latency and resource consumption stability requirements of the real-time computing engine.

[0018] Preferably, the principle of strong feature correlation specifically means that the content of the transmitted parameter is an identifier list, numerical range or status bit obtained based on offline data statistics, and the real-time model can map the transmitted parameter to real-time features through field matching, range filtering or simple logical operations;

[0019] The principle of real-time computability specifically means that the transmitted parameters adopt a fixed-length or convergent data structure, and the calculation logic does not contain variable-length loops that require dynamic memory allocation.

[0020] Preferably, in step S3, the feature expansion of the real-time data using the transmission parameters includes:

[0021] Receive the x-dimensional transfer parameters P1 to P from the offline model. x ;

[0022] The x-dimensional transfer parameters are defined as the x-dimensional extended features R1 to R2 of the real-time model. x ;

[0023] Extract y-dimensional native real-time features from the real-time data;

[0024] The x-dimensional extended features are combined with the y-dimensional native real-time features to construct a real-time feature vector B[n] of dimension n, where n = x + y.

[0025] Preferably, in step S5, the fusion of the real-time prediction result and the offline prediction result for the time period T specifically includes:

[0026] When a validation set is unavailable, the arithmetic mean method is used for fusion:

[0027]

[0028] Among them, C final The final confidence level after fusion is represented by C1, which is the first confidence level, and C2, which is the second confidence level.

[0029] The final prediction result is based on the final confidence level C. final Sure.

[0030] Preferably, in step S5, the fusion of the real-time prediction result and the offline prediction result for the time period T specifically includes:

[0031] When a validation set is available, a weighted average method is used for fusion.

[0032] Calculate the accuracy ACC1 and recall RCC1 of the real-time model, and the accuracy ACC2 and recall RCC2 of the offline model based on the validation set.

[0033] The real-time model weight α and the offline model weight β are calculated based on the accuracy and recall, using the following formula:

[0034]

[0035] Calculate the final confidence level after fusion:

[0036] ;

[0037] The final prediction result is based on the final confidence level C. final Sure.

[0038] Preferably, the weights α and β are calculated in a manner proportional to the product of the model's performance metrics or accuracy on the validation set, and satisfy α + β = 1.

[0039] A second aspect of this invention discloses a big data offline-real-time fusion feature prediction system based on parameter passing; the system includes:

[0040] The offline analysis module is used to perform deep feature mining on historical periodic data, generate offline prediction results, and extract transfer parameters that meet the requirements of weak temporal correlation, strong feature correlation, and real-time computability.

[0041] The real-time analysis module is used to receive the transmitted parameters, combine them with the real-time data stream of the current period to perform feature expansion and real-time prediction, and output the real-time prediction results and confidence level.

[0042] The model fusion module is used to perform weighted fusion of real-time prediction results and offline prediction results based on confidence scores after the offline analysis module has completed the analysis of the current period data, and output the final prediction result.

[0043] The parameter transmission interface module is used to periodically send the transmission parameters updated by the offline analysis module to the real-time analysis module.

[0044] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the offline-real-time fusion feature prediction method for big data based on parameter passing, as described in any of the first aspects of this disclosure.

[0045] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a parameter-passing-based big data offline-real-time fusion feature prediction method according to any one of the first aspects of this disclosure.

[0046] The beneficial effects of this invention are as follows:

[0047] 1. Improve real-time prediction accuracy: By passing parameters, the deep mining capabilities of the offline model are given to the real-time model, enabling it to utilize historical high-value information (such as long-term behavioral features) even when the feature dimensions are limited, thus significantly improving the accuracy of real-time identification.

[0048] 2. Closed-loop self-optimization: A closed loop of "offline prediction - parameter extraction - real-time application - result fusion - parameter update" is constructed, which can eliminate the accumulation of time errors and enable the system to have the ability to continuously evolve.

[0049] 3. Balance between resources and efficiency: The parameter design follows the principle of real-time computability, ensuring that the introduction of external features will not bring uncontrollable resource burden or latency jitter to the real-time computing engine.

[0050] 4. More scientific fusion: Confidence fusion based on sample dimensions avoids the one-size-fits-all problem of missed judgments caused by traditional model fusion (such as intersection finding), and maximizes the retention of effective information. Attached Figure Description

[0051] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 A flowchart of a big data offline-real-time fusion feature prediction method based on parameter passing according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram illustrating the principle of the parameter passing mechanism in an embodiment of the present invention;

[0054] Figure 3 This is a flowchart of the big data offline-real-time fusion feature prediction process in an embodiment of the present invention;

[0055] Figure 4 This is a structural diagram of a big data offline-real-time fusion feature prediction system based on parameter passing according to an embodiment of the present invention;

[0056] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] The first aspect of this invention discloses a big data offline-real-time fusion feature prediction method based on parameter passing. Figure 1 This is a flowchart of a big data offline-real-time fusion feature prediction method based on parameter passing according to an embodiment of the present invention, such as... Figure 1As shown, the method includes:

[0059] Step S1: Obtain the offline model over time period T Offline feature recognition is performed on historical data from 1 to generate time period T. The offline analysis results of 1 are used to extract the transmission parameters based on the offline analysis results;

[0060] In step S1, the extraction of the transmission parameters follows the following principles:

[0061] The principle of weak time correlation: ensure that the difference between the transfer parameters generated by time period T and the transfer parameters generated by time period T+1 is less than a preset threshold, i.e., P(T+1)≈P(T).

[0062] Strong feature correlation principle: The transmission parameters include rules or thresholds that can be directly mapped to low-order features of real-time data;

[0063] Real-time computability principle: The data structure for transmitting parameters meets the low latency and resource consumption stability requirements of the real-time computing engine.

[0064] The principle of strong feature correlation specifically means that the content of the transmitted parameters is an identifier list, numerical range or status bit obtained based on offline data statistics, and the real-time model can map the transmitted parameters to real-time features through field matching, range filtering or simple logical operations.

[0065] The principle of real-time computability specifically means that the transmitted parameters adopt a fixed-length or convergent data structure, and the calculation logic does not contain variable-length loops that require dynamic memory allocation.

[0066] Step S2: Transfer the transmission parameters from the offline model to the real-time model;

[0067] Step S3: The real-time model acquires real-time data for time period T, expands the features of the real-time data using the transmission parameters, constructs a real-time feature vector, and performs real-time feature prediction to obtain the real-time prediction result and first confidence level for time period T.

[0068] In step S3, the feature expansion of the real-time data using the transmission parameters includes:

[0069] Receive the x-dimensional transfer parameters P1 to P from the offline model. x ;

[0070] The x-dimensional transfer parameters are defined as the x-dimensional extended features R1 to R2 of the real-time model. x ;

[0071] Extract y-dimensional native real-time features from the real-time data;

[0072] The x-dimensional extended features are combined with the y-dimensional native real-time features to construct a real-time feature vector B[n] of dimension n, where n = x + y.

[0073] Step S4: After the time period T ends, the offline model acquires the full data of time period T and performs offline feature recognition to obtain the offline prediction result and second confidence level of time period T.

[0074] Step S5: Using a confidence fusion model, fuse the real-time prediction results and offline prediction results for the time period T to obtain the final prediction result;

[0075] In step S5, the fusion of the real-time prediction results and the offline prediction results for the time period T specifically includes:

[0076] When a validation set is unavailable, the arithmetic mean method is used for fusion:

[0077]

[0078] Among them, C final The final confidence level after fusion is represented by C1, which is the first confidence level, and C2, which is the second confidence level.

[0079] The final prediction result is based on the final confidence level C. final Sure.

[0080] In step S5, the fusion of the real-time prediction results and the offline prediction results for the time period T specifically includes:

[0081] When a validation set is available, a weighted average method is used for fusion.

[0082] Calculate the accuracy ACC1 and recall RCC1 of the real-time model, and the accuracy ACC2 and recall RCC2 of the offline model based on the validation set.

[0083] The real-time model weight α and the offline model weight β are calculated based on the accuracy and recall, using the following formula:

[0084]

[0085] Calculate the final confidence level after fusion:

[0086]

[0087] The final prediction result is based on the final confidence level C. final Sure.

[0088] The weights α and β are calculated in a way that is proportional to the product of the model’s performance metrics or accuracy on the validation set, and satisfy α + β = 1.

[0089] Step S6: The offline model updates the transmission parameters based on the offline prediction results of time period T, and transmits them to the real-time model that processes data of time period T+1, and repeats the above steps in a loop.

[0090] The purpose of updating the transmission parameters in step S6 is to eliminate the accumulated error caused by the time misalignment between the offline model and the real-time model in processing data.

[0091] Based on the above method, in some specific embodiments, taking the identification of high-value users on e-commerce platforms as an example, such as... Figure 2 and Figure 3 As shown, assume the time period T is 1 day.

[0092] Phase 1: Offline Analysis and Parameter Generation (for T-1 day data)

[0093] Historical Feature Recognition: Offline Model O Obtains T All user data from the day before and prior to this date (including browsing, orders, returns, etc.) is collected. Due to the completeness of the data, an offline model constructs an m-dimensional feature vector A[m] (e.g., total spending over the past year, return rate, category preferences, etc.), and uses a machine learning model (e.g., XGBoost) to identify T. High-value user group O(T) on the 1st 1), and output the confidence level.

[0094] Generate transfer parameters: offline model from O(T) 1) Extract features suitable for real-time matching.

[0095] For example: parameter P1(T) 1) A whitelist of user IDs whose total spending in the past year exceeds 50,000 yuan. This satisfies the criteria of "strong feature correlation" (user IDs are present in the real-time stream) and "real-time computability" (extremely fast Bloom filter or Hash Map queries).

[0096] Parameter P2(T) 1) A list of potential target product IDs (such as a new luxury handbag) that high-value users have recently visited frequently. This also satisfies the "weak time relevance" requirement, because the lifecycle of a popular product is usually longer than one day.

[0097] These parameters are packaged and sent to the real-time computing system:

[0098] Phase Two: Real-time Analysis (for Day T data)

[0099] Real-time Feature Extension: The real-time model R processes user behavior streams over day T. The raw real-time data only contains simple features such as "user ID, currently viewed product ID, and dwell time".

[0100] Real-time model receives parameters P1(T) 1) and P2(T 1).

[0101] Feature extension: When user u accesses product g, the real-time model quickly determines: Is u in the P1 whitelist? Is g in the P2 product list?

[0102] This expands the originally scarce real-time features into an n-dimensional feature vector B[n] (containing real-time behavioral features + offline transmitted features).

[0103] Real-time feature recognition: The real-time model makes predictions based on B[n]. For example, if a user's current behavior is ordinary, but he / she is on the P1 whitelist (i.e., a historical high-spending user), the real-time model can increase the probability of him / her being a "high-value user," outputting the result R(T) and confidence level C1. This enables offline capabilities to empower real-time performance.

[0104] Phase 3: Model Fusion and Closed Loop (Executed on T+1 day)

[0105] Offline reanalysis: On day T+1, the offline model has complete data from day T. The algorithm in step 1 is run again to obtain the offline result O(T) and confidence level C2 for day T. At this point, the accuracy of O(T) is theoretically higher than that of R(T).

[0106] Model fusion: In order to obtain the final accurate label library I(T) for day T, R(T) and O(T) are fused.

[0107] Assuming the system has a validation set, the real-time model weights are measured to be α = 0.4. α =0.4, offline model weight β=0.6.

[0108] For a given user, the final confidence level C final =0.4×C1+0.6×C2.

[0109] The final result is determined based on CFinal. This fusion avoids simply discarding real-time results and also corrects real-time misjudgments.

[0110] Update parameters: The offline model recalculates the parameters P(T) based on O(T) (i.e. the analysis results of the latest day) (e.g., updating the whitelist of high-value users, removing churned users, and adding users who had an outbreak yesterday), and passes it to the real-time model on day T+1.

[0111] Through the above loop, parameter P is continuously iterated, eliminating the effect caused by using T. The cumulative time error caused by using data from day 1 to predict data from day T.

[0112] Key technical points explained:

[0113] Three principles of parameters:

[0114] Weak time relevance: If the parameter is "the user's current position in seconds", it is not suitable as a passed parameter because T The position of 1 is meaningless to T. However, "user's usual location" or "VIP level" changes slowly and are suitable for transmission.

[0115] As shown in Table 1, when the passed parameter is "time period t is located at location L", O(T) can use the parameter to extract features from the real-time streaming data. If the parameter is "the user was born between 2000 and 2008", then O(T) cannot use this parameter to extract features.

[0116] Table 1 Examples of the characteristic correlation of parameters

[0117]

[0118] Strong feature relevance: The rules transmitted must be triggered by fields that are "visible" in the real-time stream. For example, transmitting the "year of birth" parameter is useless because the real-time stream usually only contains user IDs and cookies, not birth years, unless it is linked to the database in real time (but this violates the low latency principle). Therefore, the "year of birth" should be converted into a "list of user IDs" on the offline end before being transmitted.

[0119] Real-time computability: Parameters should be designed as Bitmap, Bloom Filter, or simple threshold (Key-Value) to avoid complex tree or graph structures, in order to ensure the throughput of real-time engines (such as Flink / Spark Streaming).

[0120] This invention innovatively establishes a bridge between offline and real-time methods, providing a universal, low-cost, and high-precision fusion prediction solution.

[0121] In summary, the solution proposed in this invention can:

[0122] 1) The concept of parameter transfer was first proposed, and three principles for parameter design (weak temporal correlation, strong feature correlation, and real-time computability) were given. This provides an operational guide on how to build an effective offline-real-time "bridge", which has never been inspired by existing technologies.

[0123] 2) For the first time, a complete, cyclical, and self-optimizing technical closed loop was designed, consisting of "offline prediction -> parameter extraction -> transmission -> feature expansion -> real-time prediction -> result fusion". With perfect parameter design, this closed loop design can automatically achieve gradual convergence of accuracy to the limit value without human intervention. This is not just a simple optimization of the model, but an innovative ecological closed loop of offline-real-time co-evolution.

[0124] 3) This innovative design refines confidence calculation from the "model dimension" to the "sample dimension," binding confidence to sample feature dimensions. Through a process of "feature dimension splitting - model matching - confidence fusion," this approach avoids the indiscriminate discarding of features by traditional intersection methods. Even if the prediction results of different models do not completely overlap, as long as the prediction confidence of a single model for a certain feature dimension meets the standard, the effective information of that feature can be included in the fusion result.

[0125] 4) This approach reduces reliance on "model quality," resulting in wider applicability and lower costs. It represents a low-dependency, high-performance engineering innovation. Traditional multi-model fusion methods often require extensive experimentation to select the "optimal model combination" and adjust model weights, leading to high engineering costs. This solution focuses on sample feature confidence and outputs model quality as a weight in the final confidence fusion process. Even using conventional basic models (such as linear regression and common classification algorithms), as long as the accuracy and recall of the model can be obtained through the validation set, it can be included in the fusion process without relying on complex high-end algorithms. This is a more universal multi-model result fusion method with lower costs and easier application in big data analysis scenarios for SMEs.

[0126] This invention, without compromising real-time performance, empowers the real-time model with the "deep insights" of the offline model through parameters as an intermediary bridge, significantly improving the accuracy of real-time predictions. Simultaneously, through a fusion closed-loop mechanism, it minimizes the cumulative time error of parameter transmission, continuously enhancing the predictive capabilities of the entire system and resolving the long-standing pain point of the difficulty in simultaneously achieving "timeliness" and "accuracy" in big data processing.

[0127] The second aspect of this invention discloses a big data offline-real-time fusion feature prediction system based on parameter passing. Figure 4 This is a structural diagram of a big data offline-real-time fusion feature prediction system based on parameter passing according to an embodiment of the present invention; as follows: Figure 4 As shown, the system 100 includes:

[0128] The offline analysis module 101 is used to perform deep feature mining on historical periodic data, generate offline prediction results, and extract transfer parameters that meet the requirements of weak temporal correlation, strong feature correlation, and real-time computability.

[0129] The real-time analysis module 102 is used to receive the transmitted parameters, combine them with the real-time data stream of the current period to perform feature expansion and real-time prediction, and output the real-time prediction result and confidence level.

[0130] The model fusion module 103 is used to perform weighted fusion of real-time prediction results and offline prediction results based on confidence scores after the offline analysis module has completed the analysis of the current period data, and output the final prediction result.

[0131] The parameter transmission interface module 104 is used to periodically send the transmission parameters updated by the offline analysis module to the real-time analysis module.

[0132] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the parameter-passing-based big data offline-real-time fusion feature prediction method according to any one of the first aspects of this invention.

[0133] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 5 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0134] Those skilled in the art will understand that Figure 5 The structure shown is merely a structural diagram of the part related to the technical solution of this disclosure and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0135] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the parameter-passing-based big data offline-real-time fusion feature prediction method according to any one of the first aspects of this invention.

[0136] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for large data offline-real-time fusion feature prediction based on parameter transmission, characterized in that, The method includes: Step S1: Obtain the offline model over time period T Offline feature recognition is performed on historical data from 1 to generate time period T. The offline analysis results of 1 are used to extract the transmission parameters based on the offline analysis results; Step S2: Transfer the transmission parameters from the offline model to the real-time model; Step S3: The real-time model acquires real-time data for time period T, expands the features of the real-time data using the transmission parameters, constructs a real-time feature vector, and performs real-time feature prediction to obtain the real-time prediction result and first confidence level for time period T. Step S4: After the time period T ends, the offline model acquires the full data of time period T and performs offline feature recognition to obtain the offline prediction result and second confidence level of time period T. Step S5: Using a confidence fusion model, fuse the real-time prediction results and offline prediction results for the time period T to obtain the final prediction result; Step S6: The offline model updates the transmission parameters based on the offline prediction results of time period T, and transmits them to the real-time model that processes data of time period T+1, and repeats the above steps in a loop. In step S1, the extraction of the transmission parameters follows the following principles: The principle of weak time correlation: ensure that the difference between the transfer parameters generated by time period T and the transfer parameters generated by time period T+1 is less than a preset threshold, i.e., P(T+1)≈P(T). Strong feature correlation principle: The transmission parameters include rules or thresholds that can be directly mapped to low-order features of real-time data; Real-time computability principle: The data structure for transmitting parameters meets the low latency and resource consumption stability requirements of the real-time computing engine.

2. The big data offline-real-time fusion feature prediction method based on parameter transfer according to claim 1, characterized in that, The principle of strong feature correlation specifically means that the content of the transmitted parameters is an identifier list, numerical range or status bit obtained based on offline data statistics, and the real-time model can map the transmitted parameters to real-time features through field matching, range filtering or simple logical operations. The principle of real-time computability specifically means that the transmitted parameters adopt a fixed-length or convergent data structure, and the calculation logic does not contain variable-length loops that require dynamic memory allocation.

3. The big data offline-real-time fusion feature prediction method based on parameter transfer according to claim 1, characterized in that, In step S3, the feature expansion of the real-time data using the transmission parameters includes: Receiving the x-dimensional transfer parameters P1 to Pn transferred by the offline model transfer x ; The x-dimensional transfer parameter is defined as an x-dimensional extension feature R1 to R x of the real-time model Extract y-dimensional native real-time features from the real-time data; The x-dimensional extended features are combined with the y-dimensional native real-time features to construct a real-time feature vector B[n] of dimension n, where n = x + y.

4. The big data offline-real-time fusion feature prediction method based on parameter transfer according to claim 1, characterized in that, In step S5, the fusion of the real-time prediction results and the offline prediction results for the time period T specifically includes: When a validation set is unavailable, the arithmetic mean method is used for fusion: ； wherein C final is the final confidence after fusion, C1 is the first confidence, and C2 is the second confidence. The final prediction result is based on the final confidence C final determined.

5. The big data offline-real-time fusion feature prediction method based on parameter transfer according to claim 1, characterized in that, In step S5, the fusion of the real-time prediction results and the offline prediction results for the time period T specifically includes: When a validation set is available, a weighted average method is used for fusion. Calculate the accuracy ACC1 and recall RCC1 of the real-time model, and the accuracy ACC2 and recall RCC2 of the offline model based on the validation set. Calculate the real-time model weight α and the offline model weight β based on the accuracy and recall rates; The specific formula is as follows: The final confidence level after fusion is calculated using the following formula: The final prediction result is based on the final confidence C final determined.

6. The big data offline-real-time fusion feature prediction method based on parameter transfer according to claim 5, characterized in that, The weights α and β are calculated to be proportional to the product of the model's performance metrics or accuracy on the validation set, and satisfy α + β = 1.

7. A big data offline-real-time fusion feature prediction system based on parameter transfer, characterized in that, The system employs the method described in any one of claims 1-6, and the system comprises: The offline analysis module is used to perform deep feature mining on historical periodic data, generate offline prediction results, and extract transfer parameters that meet the requirements of weak temporal correlation, strong feature correlation, and real-time computability. The real-time analysis module is used to receive the transmitted parameters, combine them with the real-time data stream of the current period to perform feature expansion and real-time prediction, and output the real-time prediction results and confidence level. The model fusion module is used to perform weighted fusion of real-time prediction results and offline prediction results based on confidence scores after the offline analysis module has completed the analysis of the current period data, and output the final prediction result. The parameter transmission interface module is used to periodically send the transmission parameters updated by the offline analysis module to the real-time analysis module.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the steps of the big data offline-real-time fusion feature prediction method based on parameter passing as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the big data offline-real-time fusion feature prediction method based on parameter passing as described in any one of claims 1 to 6.

Images