Data processing method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a data processing method and device, computer equipment, a storage medium and a program product. The method comprises the steps of determining data information of to-be-processed data in response to a serialization interface calling instruction triggered for the to-be-processed data; obtaining each candidate serialization strategy covering each application scene and a strategy matching mode of each candidate serialization strategy; for each candidate serialization strategy, based on a strategy matching mode of the candidate serialization strategy, determining a strategy matching value of the candidate serialization strategy relative to the to-be-processed data according to the data information; determining a target serialization strategy of the to-be-processed data from the candidate serialization strategies based on the strategy matching values; and performing serialization processing on the to-be-processed data according to the target serialization strategy to obtain serialized data. By adopting the method, the matching between the serialization strategy and the data scene can be improved, and the pertinence and the high efficiency of the serialization strategy are further improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing communication technology, and in particular to a data processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the rapid advancement of Internet technology and the popularization of IoT technology, the amount of data has experienced an unprecedented surge, especially in key areas such as audio and video transmission, industrial automation, smart city management, telemedicine services and financial services. Serialization of data is a key technology to achieve data persistence, cross-platform communication, caching, version control, logging, backup and recovery, cross-language interoperability, and optimize storage and transmission efficiency.

[0003] Existing serialization solutions all use preset serialization strategies to perform serialization operations, which are not able to meet the specific needs of different application scenarios. For example, in audio and video scenarios, it is crucial to ensure the synchronization of video and audio, while in the field of financial services, the serialization process must be highly secure and data consistency must be strictly guaranteed. Therefore, exploring new data serialization technologies that are more efficient, flexible, and can accurately match the needs of changing scenarios has become a key direction for current technological progress. Summary of the invention

[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the matching of serialization strategies and data scenarios, thereby improving the pertinence and efficiency of serialization strategies, in response to the above technical problems.

[0005] In a first aspect, the present application provides a data processing method, the method comprising:

[0006] In response to a serialization interface call instruction triggered for the data to be processed, determining data information of the data to be processed;

[0007] Acquire each candidate serialization strategy covering each application scenario, and a strategy matching mode of each candidate serialization strategy; the strategy matching mode is related to a rule logical connection type of the candidate serialization strategy;

[0008] For each of the candidate serialization strategies, based on the strategy matching mode of the candidate serialization strategy, determine the strategy matching value of the candidate serialization strategy relative to the data to be processed according to the data information;

[0009] Determine a target serialization strategy for the data to be processed from each of the candidate serialization strategies based on each of the strategy matching values;

[0010] The data to be processed is serialized according to the target serialization strategy to obtain serialized data.

[0011] In a second aspect, the present application further provides a data processing device, the device comprising:

[0012] A data information acquisition module, configured to determine data information of the data to be processed in response to a serialization interface call instruction triggered for the data to be processed;

[0013] A candidate serialization strategy acquisition module is used to acquire each candidate serialization strategy covering each application scenario, and a strategy matching mode of each candidate serialization strategy; the strategy matching mode is related to the rule logical connection type of the candidate serialization strategy;

[0014] A strategy matching module, configured to determine, for each of the candidate serialization strategies, a strategy matching value of the candidate serialization strategy relative to the data to be processed based on the strategy matching mode of the candidate serialization strategy and according to the data information;

[0015] A target strategy determination module, configured to determine a target serialization strategy for the data to be processed from each of the candidate serialization strategies based on each of the strategy matching values;

[0016] The serialization processing module is used to perform serialization processing on the data to be processed according to the target serialization strategy to obtain serialized data.

[0017] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0019] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0020] The above-mentioned data processing method, apparatus, computer equipment, storage medium and computer program product, in response to the serialization interface call instruction triggered for the data to be processed, determine the data information of the data to be processed, obtain the candidate serialization strategies covering each application scenario, and the respective strategy matching mode of each candidate serialization strategy, wherein the strategy matching mode is related to the rule logical connection type of the candidate serialization strategy, for each candidate serialization strategy, based on the strategy matching mode of the candidate serialization strategy, with the data information of the data to be processed as a guide, determine the strategy matching value of each candidate serialization strategy relative to the data to be processed, determine the target serialization strategy of the data to be processed from each candidate serialization strategy based on each strategy matching value, can effectively improve the matching of the target serialization strategy with the data to be processed, serialize the data to be processed according to the target serialization strategy, and effectively improve the pertinence and efficiency of the serialization strategy in the data processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A diagram of an application environment of a data processing method in some embodiments;

[0022] Figure 2 Schematic diagram of a data processing method in some embodiments;

[0023] Figure 3 Schematic diagram of the flow of data processing methods in other embodiments;

[0024] Figure 4 Schematic diagram of the flow of data processing methods in other embodiments;

[0025] Figure 5 A schematic diagram of a process of performing compression analysis on serialized data based on various feature data to obtain a target compression algorithm for serialized data and compression usage resources in some embodiments;

[0026] Figure 6 Schematic diagram of the flow of data processing methods in other embodiments;

[0027] Figure 7 It is a structural block diagram of a data processing platform in some embodiments;

[0028] Figure 8 Schematic diagram of the flow of data processing methods in other embodiments;

[0029] Fig. 9 is a structural block diagram of a data processing device in some embodiments;

[0030] Fig.10 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] The data processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the data processing platform 102 communicates with the user terminal 104 through the network. The data storage system can store the data that the data processing platform 102 needs to process. The data storage system can be integrated on the data processing platform 102, or it can be placed on the cloud or other network servers. The user can trigger the serialization interface call instruction for the data to be processed based on the user terminal 104. The data processing platform 102 responds to the serialization interface call instruction triggered for the data to be processed. It can determine the data information of the data to be processed, obtain the candidate serialization strategies covering each application scenario, and the strategy matching method of each candidate serialization strategy, wherein the strategy matching method is related to the rule logical connection type of the candidate serialization strategy, for each candidate serialization strategy, based on the strategy matching method of the candidate serialization strategy, determine the strategy matching value of the candidate serialization strategy relative to the data to be processed according to the data information, determine the target serialization strategy of the data to be processed from each candidate serialization strategy based on each strategy matching value, and perform serialization processing on the data to be processed according to the target serialization strategy to obtain the serialized data of the data to be processed. The data processing platform may be integrated in the user terminal 104 or the service, the user terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers.

[0033] In some embodiments, Figure 2 As shown, a data processing method is provided, which is applied to Figure 1 The data processing platform 102 in the example is used as an example to illustrate, and the following steps are included:

[0034] S202: In response to a serialization interface calling instruction triggered for the data to be processed, determine data information of the data to be processed.

[0035] The data to be processed are data that have serialization requirements, such as data that need to be serialized for transmission or storage. It is understandable that the data content and data information of the data to be processed are different due to different data application scenarios.

[0036] Among them, serialization is the process of converting the state information of an object into a form that can be stored or transmitted, usually converting the object into a byte stream so that it can be saved in a file or transmitted on the network. The serialization interface usually refers to the java.io.Serializable interface in Java. This interface itself does not contain any methods, but it identifies that objects of a class can be serialized. When the user has a serialization requirement for the data to be processed, it can be used as a serialization object and a serialization interface call instruction can be triggered based on the user terminal. The serialization interface call instruction is a command signal used to instruct the data processing platform to implement serialization operations on the data to be processed.

[0037] The data information of the data to be processed is descriptive information of the data to be processed, which is used to describe the data to be processed. For example, the data information of the data to be processed may include at least one of metadata and tag information. Metadata is data that describes data and is used to provide information about the organization, structure, attributes and relationships of the data to be processed. Tag information is information data that adds labels and descriptions to the data. The tag information may include the length, type, encoding method, etc. of the data to be processed.

[0038] Specifically, the data processing platform responds to the serialization interface calling instruction triggered for the data to be processed, and the serialization interface calling instruction carries the data information of the data to be processed. The data processing platform can parse the serialization interface calling instruction to determine the data information of the data to be processed.

[0039] S204, obtaining candidate serialization strategies covering various application scenarios and respective strategy matching modes of the candidate serialization strategies.

[0040] Among them, each application scenario refers to each application scenario that may generate data to be processed, such as audio scenarios, video scenarios, financial business scenarios, etc.

[0041] Among them, the serialization strategy refers to the process and method for performing serialization operations on data. In order to meet the data serialization requirements in various application scenarios, the data processing platform is pre-configured with serialization strategies covering various application scenarios. When it is necessary to perform serialization processing on the data to be processed, each serialization strategy will become a candidate serialization strategy for the data processing platform to match and filter based on the data information of the data to be processed. It can be understood that each candidate serialization strategy can include a system-level strategy pre-formulated by the designer, and can also include a custom strategy configured by the user according to actual needs.

[0042] Among them, the strategy matching method is a preset method for performing matching operations between candidate serialization strategies and data to be processed. It can be understood that each candidate serialization strategy has a corresponding strategy matching method, and the strategy matching methods of each candidate serialization strategy may be the same or different.

[0043] For each candidate serialization strategy, the strategy matching mode is related to the rule logic connection type of the candidate serialization strategy, and the rule logic connection type refers to the logic connection type of each serialization rule in the candidate serialization strategy.

[0044] Specifically, the data processing platform can obtain various candidate serialization strategies covering various application scenarios, and respective strategy matching methods of each candidate serialization strategy.

[0045] In some of the embodiments, the data processing platform pre-assigns a corresponding strategy matching method to each candidate serialization strategy according to the rule logical connection type of each candidate serialization strategy. When it is necessary to obtain each candidate serialization strategy, the strategy matching method corresponding to each candidate serialization strategy can be directly obtained.

[0046] In some of the embodiments, the candidate serialization strategies in the data processing platform may be updated. In order to improve the correspondence accuracy between the candidate serialization strategies and the strategy matching methods, the data processing platform may also assign corresponding strategy matching methods to the candidate serialization strategies in real time according to the rule logical connection types of the candidate serialization strategies.

[0047] In some of the embodiments, the data processing platform may use a memory cache mechanism to preload each candidate serialization strategy to reduce the burden of strategy retrieval and avoid performance bottlenecks that may be caused by a large number of strategies.

[0048] S206 , for each candidate serialization strategy, based on the strategy matching mode of the candidate serialization strategy and according to the data information, determine a strategy matching value of the candidate serialization strategy relative to the data to be processed.

[0049] The strategy matching value is a parameter value used to characterize the matching degree between the candidate serialization strategy and the data to be processed.

[0050] Specifically, after the data processing platform obtains each candidate serialization strategy and the policy matching method of each candidate serialization strategy, for each candidate serialization strategy, the data processing platform can determine the policy matching value of the candidate serialization strategy relative to the data to be processed based on the policy matching method of the candidate serialization strategy and the data information.

[0051] In some of the embodiments, a policy matching model is pre-configured in the data processing platform. For each candidate serialization strategy, the policy matching method corresponding to the candidate serialization strategy and the data information of the data to be processed can be input into the policy matching model together to obtain the policy matching value of the candidate serialization strategy relative to the data to be processed.

[0052] S208: Determine a target serialization strategy for the data to be processed from each candidate serialization strategy based on each strategy matching value.

[0053] Specifically, after determining the policy matching value of each candidate serialization policy relative to the data to be processed, the data processing platform may determine the target serialization policy of the data to be processed from the candidate serialization policies based on the policy matching values.

[0054] In some of the embodiments, the data processing platform may determine the candidate serialization strategy corresponding to the highest matching value among the strategy matching values ​​as the target serialization strategy for the data to be processed.

[0055] In some of the embodiments, when determining that there are multiple parallel highest matching values, the data processing platform may randomly select one from the candidate serialization strategies corresponding to the highest matching values ​​as the target serialization strategy for the data to be processed.

[0056] In some of the embodiments, the data processing platform can also record the usage frequency of each candidate serialization strategy. When it is determined that there are multiple parallel highest matching values, the one with the highest usage frequency can be selected from the candidate serialization strategies corresponding to the highest matching values ​​and determined as the target serialization strategy for the data to be processed.

[0057] S210, serializing the data to be processed according to the target serialization strategy to obtain serialized data.

[0058] Specifically, after determining the target serialization strategy for the data to be processed, the data processing platform may perform serialization processing on the data to be processed according to the target serialization strategy to obtain serialized data.

[0059] In the above data processing method, in response to the serialization interface call instruction triggered for the data to be processed, the data information of the data to be processed is determined, and each candidate serialization strategy covering each application scenario and the respective strategy matching mode of each candidate serialization strategy are obtained, wherein the strategy matching mode is related to the rule logical connection type of the candidate serialization strategy, and for each candidate serialization strategy, based on the strategy matching mode of the candidate serialization strategy, the strategy matching value of each candidate serialization strategy relative to the data to be processed is determined by taking the data information of the data to be processed as a guide, and the target serialization strategy of the data to be processed is determined from each candidate serialization strategy based on each strategy matching value, which can effectively improve the matching of the target serialization strategy with the data to be processed, and serialize the data to be processed according to the target serialization strategy, thereby effectively improving the pertinence and efficiency of the serialization strategy in the data processing process.

[0060] In some embodiments, the candidate serialization strategy includes multiple serialization rules. In S206, based on the strategy matching method of the candidate serialization strategy, the strategy matching value of the candidate serialization strategy relative to the data to be processed is determined according to the data information, including: determining the rule matching value of each serialization rule in the candidate serialization strategy relative to the data to be processed according to the data information. Based on the strategy matching method of the candidate serialization strategy and each rule matching value, the strategy matching value of the candidate serialization strategy relative to the data to be processed is determined.

[0061] Among them, serialization rules can be considered as components of candidate serialization strategies. Each candidate serialization strategy can be composed of multiple serialization rules connected together through rule logic. For example, serialization rules can include trigger thresholds, compression levels, version control, compatibility modes, etc. of the strategy. Rule logic connection types can include strategies that only contain "and" connections, strategies that only contain "or" connections, and strategies that contain both "and" and "or" connections.

[0062] The rule matching value is the matching value between each rule and the data to be processed, which can represent the degree to which the corresponding serialization rule is satisfied under the condition of the data information possessed by the data to be processed.

[0063] Specifically, for each candidate serialization strategy, the data processing platform can first determine the rule matching values ​​of each serialization rule in the candidate serialization strategy relative to the data to be processed based on the data information, and then determine the policy matching value of the candidate serialization strategy relative to the data to be processed based on the policy matching method of the candidate serialization strategy and the matching values ​​of each rule.

[0064] In some embodiments, the data information includes the metadata and data tags of the data to be processed. The data processing platform needs to traverse each candidate serialization strategy, match it with the metadata and data tags of the data to be processed one by one, and calculate its matching value. Among them, each candidate serialization strategy is composed of a series of serialization rules, which can be connected by logical "or" and "and". For a more detailed explanation, several basic elements can be defined first: M represents the set of all features in the metadata. T represents a specific data tag. P=(R1, R2, ..., R n ) represents each candidate serialization strategy, where R i is the i-th rule. S(R i , M, T) is a function that represents the rule R i The degree to which the condition is satisfied given metadata M and data tag T, takes the value of "1" or "0", where "1" indicates a match and "0" indicates a mismatch.

[0065] The degree of match between the data to be processed and each candidate serialization strategy can be calculated by considering the combined impact of all rules. Specifically, if the serialization rules in the serialization strategy are connected by a logical "or", then the strategy is considered to match as long as at least one serialization rule is satisfied. If the serialization rules in the serialization strategy are connected by a logical "and", then all serialization rules must be satisfied. Assuming that the "or" rule in the serialization strategy is represented by ∨ and the "and" rule is represented by ∧, a strategy matching function F can be defined, that is, the strategy matching method is as follows:

[0066] For a serialization strategy whose rule logic connection type contains only "AND" connections, all its serialization rules must be satisfied:

[0067] ;

[0068] For a serialization strategy whose rule logic connection type contains only "OR" connections, it only needs at least one serialization rule to be satisfied:

[0069] ;

[0070] For a serialization strategy with a regular logic connection type that includes both "AND" and "OR" connections:

[0071] ;

[0072] The data processing platform may determine the policy matching value of the candidate serialization policy relative to the data to be processed based on the policy matching function F of the candidate serialization policy and the matching values ​​of each rule.

[0073] In the above embodiment, the policy matching value between the candidate serialization strategy and the data to be processed is determined by the rule matching value between each serialization rule constituting the candidate serialization strategy and the data to be processed. This can comprehensively consider the matching between each component of the serialization strategy and the data to be processed, effectively improve the accuracy of determining the policy matching value, and thus improve the matching between the target serialization strategy subsequently determined according to the policy matching value and the data to be processed.

[0074] In some embodiments, Figure 3 As shown, the data processing method also includes the following steps:

[0075] S302 , in response to a policy configuration request triggered by a policy configuration interface, generate and display a policy configuration page based on a preset policy configuration template; the policy configuration page includes configuration controls for various policy configuration parameters.

[0076] Among them, the policy configuration interface is a system API interface open to users. Users only need to implement the basic policy interface to easily register personalized serialization policies, achieve seamless docking with existing processes, and flexibly adapt to various specific needs.

[0077] The policy configuration request is request information sent by the user to the data processing platform based on the policy configuration interface, and is used to request a custom configuration serialization policy in the data processing platform.

[0078] Among them, the preset policy configuration template is a preset template file used to generate a policy configuration page. The preset policy configuration template can specify the configuration controls to be included in the policy configuration page, the page position of each configuration control in the policy configuration page, the triggering action of each configuration control, and the preset parameter configuration information of each configuration control.

[0079] Among them, the policy configuration parameters are necessary parameters for implementing the serialization interface and generating the serialization policy, such as the trigger threshold, compression level, version control, compatibility mode, etc. that constitute the serialization policy. Each configuration control specified in the preset policy configuration template corresponds to each policy configuration parameter. Therefore, the policy configuration page generated according to the preset policy configuration template will include configuration controls corresponding to each policy configuration parameter.

[0080] Specifically, the data processing platform responds to the policy configuration request triggered by the user on the policy configuration interface, determines the page generation information of the policy configuration page based on the preset policy configuration template, and renders the page according to the page generation information, thereby generating a policy configuration page including configuration controls corresponding to each policy configuration parameter, and displays the policy configuration page on the user terminal.

[0081] S304 , for each configuration control, in response to a control triggering operation on the configuration control, displaying a parameter configuration area of ​​the configuration control on a policy configuration page according to preset parameter configuration information of the configuration control.

[0082] Among them, the trigger operation of the configuration control is the behavior of the user interacting with the configuration control through the user terminal to activate the control to perform a specific function or event. It can be understood that different configuration controls can correspond to different control trigger operations. For example, the control trigger operation of the configuration control can include but is not limited to pause, drag, click, input, etc.

[0083] Among them, the preset parameter configuration information of the configuration control is the preset information used to specify the configuration method and / or configurable content used by the user to configure the policy configuration parameters corresponding to the configuration control. It can be understood that different preset parameter configuration information corresponds to different parameter configuration areas of the configuration control. For example, if the preset parameter configuration information of the configuration control is parameter filling, the parameter configuration area corresponding to the configuration control is the parameter filling area. If the preset parameter configuration information of the configuration control is parameter selection, the parameter configuration area corresponding to the configuration control is a selection area containing multiple selection items.

[0084] Specifically, when the user needs to configure the policy configuration parameters, he can interact with each configuration control in turn through the user terminal to perform the control trigger operation. For each configuration control, the data processing platform will respond to the control trigger operation for the configuration control and display the parameter configuration area of ​​the configuration control on the policy configuration page according to the preset parameter configuration information of the configuration control.

[0085] S306 , for each parameter configuration area, generating a new serialization strategy in response to a parameter configuration operation on the parameter configuration area.

[0086] The parameter configuration operation refers to the operation behavior of the user to configure the policy configuration parameters corresponding to the configuration control. By configuring the policy configuration parameters, a policy configuration file of the custom serialization policy can be obtained.

[0087] Specifically, the user can perform parameter configuration operations on the parameter configuration area of ​​the configuration control in sequence through the user terminal, such as filling in parameter information or selecting parameter information, etc. For each parameter configuration area, the data processing platform can respond to the parameter configuration operation on the parameter configuration area and obtain the parameter content of the policy configuration parameter corresponding to the parameter configuration area. After obtaining the parameter content of each policy configuration parameter, the data processing platform can generate a policy configuration file of the custom serialization policy according to the parameter content of each policy configuration parameter, and then perform policy assembly according to the policy configuration file to generate a user-defined newly added serialization policy.

[0088] S308: Store the newly added serialization strategy as a candidate serialization strategy.

[0089] Specifically, after generating a new serialization strategy, the data processing platform may store the new serialization strategy as a candidate serialization strategy so as to facilitate subsequent use for strategy matching with data to be processed.

[0090] In the above embodiment, the data processing platform can hand over the key of creation to the user by providing the user with a policy configuration interface, providing the user with more flexible choices and facilitating the user to complete customized functions according to the user's own scenarios.

[0091] In some of the embodiments, the data processing platform also has verification and evaluation functions, and can evaluate the serialization results. For example, after the user configures a new serialization strategy, the serialization interface call instruction can be triggered for the detection data of the new serialization strategy. The data processing platform will match the detection data with the new serialization strategy as the target serialization strategy for data serialization, and evaluate the completed serialization data to determine the serialization effect of the new serialization strategy, thereby covering multiple dimensions such as data integrity, performance, and consistency, to ensure that each round of serialization operations can be accurate and efficient, and provide solid data support and feedback for the continuous optimization and upgrading of user-defined strategies.

[0092] In some embodiments, Figure 4 As shown, the data processing method also includes the following steps:

[0093] S402 , in response to a data transmission instruction triggered for serialized data, determining a transmission address.

[0094] The data transmission instruction is instruction information for instructing the data processing platform to transmit the serialized data to the corresponding transmission address. The data transmission instruction can be triggered and generated by the user based on the user terminal for the serialized data. It can be understood that the data transmission instruction carries the transmission address, and the data processing platform can determine the transmission address of the serialized data by parsing the data transmission instruction.

[0095] Specifically, the data processing platform may determine a transmission address in response to a data transmission instruction triggered for serialized data.

[0096] S404, extracting data features from the data information and serialized data to obtain feature data corresponding to each feature type.

[0097] Among them, data feature extraction refers to the processing operation of extracting feature data corresponding to multiple feature types from the data information of the data to be processed and the serialized data. It can be understood that data feature extraction can be performed by a feature extraction component pre-set in the data processing platform.

[0098] The feature type is a type parameter used to set the type classification of each feature data. The specific type of feature type is related to the data content of the data information and serialized data. For example, the feature type can include format features, statistical features, structural features, content features, etc.

[0099] Among them, format features: use metadata to identify file types. Statistical features: such as entropy, mean, standard deviation, etc., reflect the randomness and concentration of data. Structural features: analyze the repetitive patterns and texture features of data. Content features: for images and videos, it may involve color distribution, edge information, etc. For text, it may be vocabulary richness, sentence length, etc.

[0100] Specifically, the data processing platform can call a preset feature extraction component to extract data features from data information and serialized data to obtain feature data corresponding to each feature type.

[0101] S406, performing compression analysis on the serialized data based on each feature data to obtain a target compression algorithm and compression usage resources for the serialized data.

[0102] The compression analysis is to determine the target compression algorithm for performing compression operations on serialized data according to the feature data and the analysis operation of the compression usage resources.

[0103] Among them, the target compression algorithm is the compression algorithm to be used when performing compression operations on serialized data. It can be understood that there are multiple compression algorithms preset in the data processing platform. The data processing platform can analyze the serialized data transmission requirements through various feature data, such as high real-time performance or high accuracy, and determine the target compression algorithm from multiple compression algorithms.

[0104] Among them, the compression usage resources refer to the memory resources and CPU core resources allocated for performing compression operations on serialized data. The data processing platform can determine the memory resources and CPU core resources allocated for performing compression operations on serialized data based on the target compression algorithm determined for the serialized data and various feature data.

[0105] Specifically, the data processing platform can perform compression analysis on the serialized data based on each feature data to obtain a target compression algorithm for the serialized data and compression usage resources.

[0106] In some of the embodiments, the data processing platform is pre-set with analysis rules, and each feature data can be analyzed according to the analysis rules to determine the target compression algorithm for performing compression operations on the serialized data and the compression usage resources.

[0107] S408, performing data compression on the serialized data according to the target compression algorithm and the compression usage resources to obtain data to be transmitted.

[0108] Specifically, the data processing platform can perform data compression operations on the serialized data according to the target compression algorithm and compression usage resources to obtain data to be transmitted after the serialized data is compressed.

[0109] S410, performing data transmission processing on the data to be transmitted according to the transmission address.

[0110] Specifically, the data processing platform may transmit the data to be transmitted to the transmission address according to the transmission address.

[0111] In the above embodiment, data features are extracted from data information and serialized data to obtain feature data corresponding to each feature type, and compression analysis is performed on the serialized data based on each feature data to obtain a target compression algorithm for the serialized data and compression usage resources. This can make the target compression algorithm and compression usage resources used when compressing the serialized data more closely match the actual data situation of the serialized data, greatly reduce data transmission time and improve serialization speed, and directly enhance the overall efficiency of data processing.

[0112] In some embodiments, Figure 5 As shown, S406, based on each feature data, compression analysis is performed on the serialized data to obtain a target compression algorithm for the serialized data and compression usage resources, including:

[0113] S502, input each feature data into the compression analysis model.

[0114] Among them, the compression analysis model is a preset model used to perform compression analysis on serialized data according to various feature data, and the compression analysis model can be trained based on historical feature data.

[0115] The compression analysis model includes a shared feature extraction layer, and an algorithm selection branch and an efficiency prediction branch respectively connected to the shared feature extraction layer, wherein the shared feature extraction layer is an extraction layer for extracting key features from each feature data. The shared feature extraction layer may include a convolution layer, a pooling layer, and a fully connected layer, which are used to process each feature data, extract key features therefrom, and perform operations such as feature integration.

[0116] The output end of the algorithm selection branch in the compression analysis model is connected to the input end of the efficiency prediction branch, and the input end of the algorithm selection branch and the input end of the efficiency prediction branch are also respectively connected to the output end of the shared feature extraction layer, and can receive the shared key features extracted by the shared feature extraction layer.

[0117] S504, extracting and integrating key features of each feature data based on the shared feature extraction layer to obtain key feature data of the serialized data.

[0118] The feature data may be vectorized feature data, and its data format is more suitable for machine learning model input. The key feature data is the feature data corresponding to the key feature parameters required for compression analysis of serialized data. The key feature parameters can be determined in advance by the designer according to the actual analysis requirements.

[0119] Specifically, after the data processing platform inputs each feature data into the compression analysis model, the shared feature extraction layer in the compression analysis model will extract key features of each feature data to obtain each key feature vector, and then perform integration processing on each key feature vector to obtain the key feature data of the serialized data.

[0120] S506, performing algorithm selection analysis on the key feature data through the algorithm selection branch to determine the target compression algorithm for the serialized data.

[0121] Among them, the algorithm selection branch is a model branch used to select a compression algorithm for serialized data. It can perform algorithm selection analysis based on key feature data to determine the target compression algorithm for serialized data.

[0122] Specifically, after the shared feature extraction layer extracts the key feature data, the key feature data will be input into the algorithm selection branch and the efficiency prediction branch as shared features. The algorithm selection branch can perform algorithm selection analysis based on the key feature data to determine the target compression algorithm for the serialized data.

[0123] S508, determining the algorithm execution prediction efficiency of the serialized data according to the target compression algorithm and the key feature data through the efficiency prediction branch.

[0124] Among them, the efficiency prediction branch is used for regression tasks, that is, a model branch used to predict the compression efficiency for serialized data. The compression efficiency of serialized data can be predicted through the target compression algorithm and key feature data to obtain the algorithm execution prediction efficiency when the serialized data uses the target compression algorithm to perform compression operations.

[0125] Specifically, since the output end of the algorithm selection branch is connected to the input end of the efficiency prediction branch, after the algorithm selection branch determines the target compression algorithm for the serialized data, it will output the target compression algorithm to the efficiency prediction branch. The efficiency prediction branch predicts the compression efficiency of the serialized data through the target compression algorithm and shared key feature data, and obtains the algorithm execution prediction efficiency when the serialized data uses the target compression algorithm to perform compression operations.

[0126] S510, performing a resource allocation operation on the serialized data according to the algorithm execution prediction efficiency, and determining the compression usage resources of the serialized data.

[0127] Among them, during the resource allocation operation, the compressed usage resources are allocated to the serialized data according to the algorithm execution prediction efficiency, such as memory resources and CPU core operation behavior.

[0128] Specifically, after obtaining the algorithm execution prediction efficiency, the data processing platform can perform resource allocation operations on the serialized data according to the algorithm execution prediction efficiency to determine the compression usage resources of the serialized data.

[0129] In some of the embodiments, the data processing platform is pre-configured with a correspondence between the algorithm execution efficiency and the compression usage resources. After obtaining the algorithm execution prediction efficiency, the compression usage resources that match the algorithm execution prediction efficiency can be searched from the correspondence between the algorithm execution efficiency and the compression usage resources.

[0130] In the above embodiment, the data processing platform can select the most suitable compression algorithm for the serialized data through the compression analysis model, predict the execution efficiency of the current compression algorithm, and allocate memory and CPU cores of appropriate size to it, which can improve the compression performance of the serialized data while reducing resource waste.

[0131] In some of the embodiments, the training method of the compression analysis model includes the following steps: first, the data processing platform can obtain a diverse data set, wherein the diverse data set covers different formats, such as text, images, videos, etc., different arrangement modes, such as ordered, disordered, repetitive, etc., and data of different content complexity.

[0132] The data processing platform performs feature extraction operations on each piece of data, including but not limited to format features: using metadata to identify file types. Statistical features: such as entropy, mean, standard deviation, etc., reflecting the randomness and concentration of data. Structural features: analyzing the repetitive patterns and texture features of data. Content features: for images and videos, it may involve color distribution, edge information, etc.; for text, it may be vocabulary richness, sentence length, etc. The extracted features are vectorized to form a data format suitable for machine learning model input, so as to obtain training feature sample data.

[0133] The data processing platform can also build a multi-task learning framework (MTL), which includes two branches, namely, the algorithm selection branch and the efficiency prediction branch. The algorithm selection branch is used for classification tasks, namely, selecting compression algorithms, and the efficiency prediction branch is used for regression tasks, namely, predicting compression efficiency.

[0134] The multi-task learning framework design includes: input layer, shared feature extraction layer, algorithm selection branch and efficiency prediction branch. Among them, the input layer is used to receive training feature sample data, that is, preprocessed data feature vectors. The shared feature extraction layer includes convolution layer, pooling layer, and fully connected layer, which are used to process training feature sample data, extract key features, perform feature integration and other operations. The algorithm selection branch and efficiency prediction branch further learn the features of specific patterns based on the shared key features and output the results.

[0135] Regarding the loss function design, considering the dual-task characteristics of the model, a joint loss function L can be designed total , combined with the classification loss L class and regression loss L reg , cross entropy loss is used as the loss of classification task, and mean square error is used as the loss of regression task. The formula is as follows:

[0136]

[0137]

[0138] ;

[0139] Among them, y ic is an indicator variable (one-hot encoding) that sample i actually belongs to category c, is the probability that the model predicts that sample i belongs to category c, c is the number of categories in the compression algorithm, N is the number of samples, is the actual compression efficiency indicator of sample i, is the compression efficiency value predicted by sample i, and α is a balancing factor used to adjust the importance of classification and regression tasks.

[0140] The data processing model can perform model training on the initial model corresponding to the multi-task learning framework based on the training feature sample data. After the training is completed, for any input data, the model first selects the most suitable compression algorithm category through the algorithm selection branch, and then predicts the execution efficiency of the current compression algorithm according to the efficiency prediction branch, and allocates the appropriate size of memory and CPU core to it, thereby improving performance while reducing resource waste.

[0141] In some embodiments, Figure 6As shown, the data processing method also includes the following steps:

[0142] S602, obtaining compression performance data when the serialized data is actually compressed.

[0143] The compression performance data of the serialized data during actual compression is information data that can characterize the compression performance of the serialized data, such as the compression ratio and compression time of the serialized data.

[0144] Specifically, after the serialized data is compressed, the data processing platform can obtain the compression performance data when the serialized data is actually compressed.

[0145] S604: Determine actual compression execution efficiency of the serialized data based on the compression performance data.

[0146] Specifically, after obtaining the compression performance data of the serialized data when it is actually compressed, the data processing platform determines the actual compression execution efficiency of the serialized data based on the compression performance data.

[0147] In some of the embodiments, an execution efficiency determination model is pre-configured in the data processing platform. After obtaining the compression performance data, the data processing platform inputs the compression performance data into the execution efficiency determination model to obtain the actual compression execution efficiency of the serialized data.

[0148] S606: According to the efficiency difference between the actual compression execution efficiency and the algorithm execution prediction efficiency, the compression analysis model is adjusted to update the compression analysis model.

[0149] Specifically, the data processing platform can compare the efficiency values ​​of the actual compression execution efficiency with the algorithm execution prediction efficiency to obtain the efficiency difference between the actual compression execution efficiency and the algorithm execution prediction efficiency, and adjust the compression analysis model based on the efficiency difference between the actual compression execution efficiency and the algorithm execution prediction efficiency to optimize the model parameters of the compression analysis model, update the compression analysis model to improve the prediction accuracy and the ability to select the compression strategy, implement an early stopping strategy or regularly evaluate the performance of the model on the validation set to prevent overfitting.

[0150] In some embodiments, a data processing method is provided, which is applied to Figure 7 The data processing platform shown is used as an example for explanation. The data processing platform includes a sequence strategy customization module, a serialization strategy identification module, a deep learning driven serialization compression module, and an execution module.

[0151] Among them, the serialization strategy customization module allows users to customize serialization strategies, providing users with more flexible choices and facilitating users to complete customized functions based on their own scenarios.

[0152] The serialization strategy identification module accurately diagnoses data characteristics and scenario requirements based on the preset advanced strategy matching logic, and selects and starts the most suitable serialization strategy.

[0153] The deep learning-driven serialization compression module uses advanced neural network technology to analyze the intrinsic structure and distribution characteristics of data in real time, intelligently allocate the best compression and decompression mechanism, and efficiently perform data volume reduction and recovery operations. The module innovatively integrates a compression performance estimation model to accurately estimate the task time, and flexibly allocates system resources based on this to ensure maximum resource utilization, injecting unprecedented efficiency and intelligence into data processing.

[0154] The execution module executes the serialization strategy and compression algorithm selected above, and performs serialization and deserialization processes on user data.

[0155] like Figure 8 As shown, the data processing method may specifically include the following steps:

[0156] First, the data processing platform can determine the metadata and data tags of the data to be processed in response to the serialization interface call instructions triggered for the data to be processed. Obtain each candidate serialization strategy covering each application scenario, and the strategy matching method of each candidate serialization strategy. Among them, each candidate serialization strategy integrates system-level and user-customized strategy sets. Specifically, the strategy set not only integrates a series of carefully pre-made system-level policy templates, but also provides an entry for user-defined strategy design. These preset system-level strategies cover a wide range of needs from heavy network I / O tasks, multimedia processing scenarios, financial-level service configuration, to multi-source data flow control. Users do not need to delve into the micro-structure and execution logic of the strategy. They only need to follow the preset system guidelines to easily deploy and achieve an efficient plug-and-play experience.

[0157] Customized strategy design puts the key of creation in the hands of users. Through the open system API interface, users only need to implement the basic strategy interface to easily register personalized serialization strategies, achieve seamless connection with existing processes, and flexibly adapt to various specific needs.

[0158] Among them, to build a custom strategy, you need to implement the following core interfaces: strategy configuration interface, serialization interface, and verification and evaluation interface. In the strategy configuration interface, users can set a series of conditional rules, and use the "or" and "and" logic to weave ingeniously to accurately define the trigger threshold of the strategy. It also supports a variety of configuration options such as compression level, version control, and compatibility mode. In the serialization interface, users customize the serialization process. The system supports common serialization algorithms by default. At the same time, users can also customize the serialization method, implement the Serializable interface, directly process multiple data streams, and output the serialized results. In the meantime, the framework intelligently integrates the compression and decompression processes to simplify the user's burden. The verification and evaluation interface is responsible for evaluating the serialization results, covering multiple dimensions such as data integrity, performance, and consistency, to ensure that each round of serialization operations can be accurate and efficient, and provide solid data support and feedback for the continuous optimization and upgrading of user strategies.

[0159] Specifically, when there is a need for user-defined serialization policies, the data processing platform can respond to a policy configuration request triggered by a policy configuration interface, call the serialization policy customization module to generate and display a policy configuration page based on a preset policy configuration template, wherein the policy configuration page includes configuration controls for each policy configuration parameter. For each configuration control, the serialization policy customization module responds to a control trigger operation for the configuration control, and displays the parameter configuration area of ​​the configuration control on the policy configuration page according to the preset parameter configuration information of the configuration control. For each parameter configuration area, the serialization policy customization module responds to the parameter configuration operation for the parameter configuration area, generates a new serialization policy, and stores the new serialization policy as a candidate serialization policy.

[0160] The data processing platform can use a memory cache mechanism to preload strategies to reduce the burden of strategy retrieval and avoid performance bottlenecks that may be caused by a large number of strategies.

[0161] The data processing platform can call the serialization strategy identification module, among which the core responsibilities of the serialization strategy identification module include: accurately identifying and enabling the most suitable strategy scheme based on the metadata and tags carried when calling the serialization interface. The strategy matching mechanism focuses on using user-defined metadata and tag information to evaluate the fit of each strategy and select the strategy implementation scheme with the highest matching degree.

[0162] In practice, for each candidate serialization strategy, the core responsibilities of the serialization strategy identification module include: accurately identifying and enabling the most suitable strategy solution based on the metadata and tags carried when calling the serialization interface. The strategy matching mechanism focuses on using user-defined metadata and tag information to evaluate the fit of each strategy and select the strategy implementation plan with the highest matching degree.

[0163] Therefore, the serialization strategy identification module can determine the rule matching values ​​of each serialization rule in the candidate serialization strategy relative to the data to be processed based on the metadata and data tags, and then determine the policy matching value of the candidate serialization strategy relative to the data to be processed based on the policy matching method of the candidate serialization strategy and the matching value of each rule. Summarize the matching scores of all strategies, sort the list from high to low by score, and select the candidate serialization strategy with the highest score as the target serialization strategy. If there is a special situation where the matching score is tied for the highest, that is, multiple strategies are equivalent and optimal, random selection can be used to determine the target serialization strategy to ensure fairness and flexibility in the selection.

[0164] After obtaining the target serialization strategy, the data processing platform can call the execution module to perform serialization processing on the data to be processed according to the target serialization strategy to obtain serialized data.

[0165] The data processing platform can then respond to the data transmission instruction triggered for the serialized data, determine the transmission address, call the serialization compression module to extract data features from the metadata and data tags of the data to be processed, and the serialized data, obtain the feature data corresponding to each feature type, input each feature data into the pre-trained compression analysis model, extract and integrate the key features of each feature data based on the shared feature extraction layer in the compression analysis model, obtain the key feature data of the serialized data, perform algorithm selection analysis on the key feature data through the algorithm selection branch, determine the target compression algorithm of the serialized data, and determine the algorithm execution prediction efficiency of the serialized data according to the target compression algorithm and the key feature data through the efficiency prediction branch. Perform resource allocation operations on the serialized data according to the algorithm execution prediction efficiency to determine the compression usage resources of the serialized data.

[0166] After obtaining the target compression algorithm and compression usage resources for the serialized data, the data processing platform can call the execution module to perform data compression on the serialized data according to the target compression algorithm and compression usage resources, obtain the data to be transmitted, and perform data transmission processing on the data to be transmitted according to the transmission address.

[0167] After the compression is completed, the serialization compression module can also obtain the compression performance data of the serialized data when it is actually compressed. Based on the compression performance data, the actual compression execution efficiency of the serialized data is determined. According to the efficiency difference between the actual compression execution efficiency and the algorithm execution prediction efficiency, the compression analysis model is adjusted and the compression analysis model is updated.

[0168] The data processing method in this embodiment uses an intelligent multimodal data serialization strategy, integrates scene characteristics and metadata guidance, and dynamically selects the serialization algorithm, which not only achieves wide adaptability to various application scenarios, but also significantly enhances the pertinence and efficiency of data processing, especially when processing diverse data types and complex requirements in the big data era, showing a high degree of flexibility and optimization potential. The data compression and decompression scheme customized by the deep learning model is used to intelligently identify and utilize data features, greatly reduce data transmission time and improve serialization speed, and directly enhance the overall efficiency of data processing. This innovation not only solves the transmission delay problem caused by the explosive growth of data volume, but also provides a faster and smoother data processing foundation for data-intensive applications such as audio and video streaming, telemedicine, and financial services. The innovatively integrated compression and decompression time prediction mechanism, combined with the intelligent resource scheduling strategy, dynamically optimizes resource allocation according to the estimated time consumption of the task, effectively avoids resource waste, significantly improves resource utilization efficiency and system response speed, and ensures that the system can maintain high-performance operation when facing large-scale data processing needs, laying a solid technical foundation for meeting future data processing challenges.

[0169] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0170] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for implementing the data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments provided below can refer to the limitations on the data processing method above, and will not be repeated here.

[0171] In some embodiments, Fig. 9 As shown, a data processing device 900 is provided, comprising: a data information acquisition module 901, a candidate serialization strategy acquisition module 902, a strategy matching module 903, a target strategy determination module 904 and a serialization processing module 905, wherein:

[0172] The data information acquisition module 901 is used to determine the data information of the data to be processed in response to the serialization interface calling instruction triggered for the data to be processed.

[0173] The candidate serialization strategy acquisition module 902 is used to acquire various candidate serialization strategies covering various application scenarios and respective strategy matching modes of various candidate serialization strategies; the strategy matching mode is related to the rule logic connection type of the candidate serialization strategy.

[0174] The strategy matching module 903 is used to determine, for each candidate serialization strategy, a strategy matching value of the candidate serialization strategy relative to the data to be processed based on the strategy matching mode of the candidate serialization strategy and according to the data information.

[0175] The target strategy determination module 904 is used to determine the target serialization strategy of the to-be-processed data from each candidate serialization strategy based on each strategy matching value.

[0176] The serialization processing module 905 is used to perform serialization processing on the data to be processed according to the target serialization strategy to obtain serialized data.

[0177] In some embodiments, the candidate serialization strategy includes multiple serialization rules, and the strategy matching module is used to: determine the rule matching value of each serialization rule in the candidate serialization strategy relative to the data to be processed based on the data information; determine the strategy matching value of the candidate serialization strategy relative to the data to be processed based on the strategy matching method of the candidate serialization strategy and the matching value of each rule.

[0178] In some embodiments, the data processing apparatus further comprises:

[0179] The request response module is used to respond to the policy configuration request triggered by the policy configuration interface, generate and display a policy configuration page based on a preset policy configuration template; the policy configuration page includes configuration controls for various policy configuration parameters.

[0180] The parameter configuration area display module is used to display the parameter configuration area of ​​each configuration control on the policy configuration page in response to a control trigger operation on the configuration control and according to preset parameter configuration information of the configuration control.

[0181] The serialization strategy adding module is used to generate a new serialization strategy for each parameter configuration area in response to a parameter configuration operation on the parameter configuration area.

[0182] The strategy storage module is used to store the newly added serialization strategy as a candidate serialization strategy.

[0183] In some embodiments, the data processing apparatus further comprises:

[0184] The data transmission instruction response module is used to determine a transmission address in response to a data transmission instruction triggered for serialized data.

[0185] The feature extraction module is used to extract data features from data information and serialized data to obtain feature data corresponding to each feature type.

[0186] The compression analysis module is used to perform compression analysis on the serialized data based on various feature data to obtain the target compression algorithm and compression usage resources of the serialized data.

[0187] The data compression module is used to perform data compression on the serialized data according to the target compression algorithm and compression usage resources to obtain data to be transmitted.

[0188] The data transmission module is used to perform data transmission processing on the data to be transmitted according to the transmission address.

[0189] In some embodiments, the compression analysis module is used to: input each feature data into a compression analysis model; the compression analysis model includes a shared feature extraction layer, and an algorithm selection branch and an efficiency prediction branch respectively connected to the shared feature extraction layer, and the output end of the algorithm selection branch is connected to the input end of the efficiency prediction branch; based on the shared feature extraction layer, key features of each feature data are extracted and integrated to obtain key feature data of serialized data; through the algorithm selection branch, an algorithm selection analysis is performed on the key feature data to determine the target compression algorithm of the serialized data; through the efficiency prediction branch, the algorithm execution prediction efficiency of the serialized data is determined according to the target compression algorithm and the key feature data; according to the algorithm execution prediction efficiency, a resource allocation operation is performed on the serialized data to determine the compression usage resources of the serialized data.

[0190] In some embodiments, the data processing apparatus further comprises:

[0191] The compression performance data acquisition module is used to obtain the compression performance data when the serialized data is actually compressed.

[0192] The actual compression execution efficiency determination module is used to determine the actual compression execution efficiency of the serialized data based on the compression performance data.

[0193] The model updating module is used to adjust the compression analysis model and update the compression analysis model according to the efficiency difference between the actual compression execution efficiency and the algorithm execution prediction efficiency.

[0194] Each module in the above data processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0195] In some embodiments, a computer device is provided. The computer device may be a terminal or server integrated with a data processing platform. The internal structure diagram thereof may be as follows: Fig.10 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data information, various candidate serialization strategies, strategy matching methods, strategy matching value serialization data and other data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0196] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0197] In some embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the specific steps of the above data processing method embodiment when executing the computer program.

[0198] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the specific steps of the above-mentioned data processing method embodiment are implemented.

[0199] In some embodiments, a computer program product is provided, including a computer program, which implements the specific steps of the above data processing method embodiment when executed by a processor.

[0200] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. In addition, the acquisition, storage, processing, and transmission of data are in compliance with relevant laws and regulations.

[0201] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0202] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0203] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: The method comprises: In response to a serialization interface call instruction triggered for the data to be processed, determining data information of the data to be processed; Acquire each candidate serialization strategy covering each application scenario, and a strategy matching mode of each candidate serialization strategy; the strategy matching mode is related to a rule logical connection type of the candidate serialization strategy; For each of the candidate serialization strategies, based on the strategy matching mode of the candidate serialization strategy, determine the strategy matching value of the candidate serialization strategy relative to the data to be processed according to the data information; Determine a target serialization strategy for the data to be processed from each of the candidate serialization strategies based on each of the strategy matching values; The data to be processed is serialized according to the target serialization strategy to obtain serialized data.

2. The method according to claim 1, characterized in that The candidate serialization strategy includes a plurality of serialization rules; The strategy matching method based on the candidate serialization strategy, determining the strategy matching value of the candidate serialization strategy relative to the data to be processed according to the data information, includes: Determine, according to the data information, a rule matching value of each serialization rule in the candidate serialization strategy relative to the data to be processed; Based on the policy matching mode of the candidate serialization policy and each of the rule matching values, a policy matching value of the candidate serialization policy relative to the data to be processed is determined.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In response to a policy configuration request triggered by a policy configuration interface, a policy configuration page is generated and displayed based on a preset policy configuration template; the policy configuration page includes configuration controls for various policy configuration parameters; For each of the configuration controls, in response to a control triggering operation on the configuration control, displaying a parameter configuration area of ​​the configuration control on the policy configuration page according to preset parameter configuration information of the configuration control; For each of the parameter configuration areas, in response to a parameter configuration operation on the parameter configuration area, generating a newly added serialization strategy; The newly added serialization strategy is stored as a candidate serialization strategy.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: In response to a data transmission instruction triggered for the serialized data, determining a transmission address; Extracting data features from the data information and the serialized data to obtain feature data corresponding to each feature type; Performing compression analysis on the serialized data based on each of the characteristic data to obtain a target compression algorithm and compression usage resources for the serialized data; Performing data compression on the serialized data according to the target compression algorithm and the compression usage resources to obtain data to be transmitted; Data transmission processing is performed on the data to be transmitted according to the transmission address.

5. The method according to claim 4, characterized in that The compressing and analyzing the serialized data based on each of the characteristic data to obtain a target compression algorithm and compression usage resources for the serialized data includes: Input each of the feature data into a compression analysis model; the compression analysis model includes a shared feature extraction layer, and an algorithm selection branch and an efficiency prediction branch respectively connected to the shared feature extraction layer, and the output end of the algorithm selection branch is connected to the input end of the efficiency prediction branch; Extracting and integrating key features of each of the feature data based on the shared feature extraction layer to obtain key feature data of the serialized data; Performing algorithm selection analysis on the key feature data through the algorithm selection branch to determine a target compression algorithm for the serialized data; Determining the algorithm execution prediction efficiency of the serialized data according to the target compression algorithm and the key feature data through the efficiency prediction branch; A resource allocation operation is performed on the serialized data according to the predicted efficiency of the algorithm to determine the compression usage resources of the serialized data.

6. The method according to claim 5, characterized in that The method further comprises: Obtaining compression performance data when the serialized data is actually compressed; determining an actual compression execution efficiency of the serialized data based on the compression performance data; According to the efficiency difference between the actual compression execution efficiency and the algorithm execution prediction efficiency, the compression analysis model is adjusted to update the compression analysis model.

7. A data processing device, characterized in that: The device comprises: A data information acquisition module, configured to determine data information of the data to be processed in response to a serialization interface call instruction triggered for the data to be processed; A candidate serialization strategy acquisition module is used to acquire each candidate serialization strategy covering each application scenario, and a strategy matching mode of each candidate serialization strategy; the strategy matching mode is related to the rule logical connection type of the candidate serialization strategy; A strategy matching module, configured to determine, for each of the candidate serialization strategies, a strategy matching value of the candidate serialization strategy relative to the data to be processed based on the strategy matching mode of the candidate serialization strategy and according to the data information; A target strategy determination module, configured to determine a target serialization strategy for the data to be processed from each of the candidate serialization strategies based on each of the strategy matching values; The serialization processing module is used to perform serialization processing on the data to be processed according to the target serialization strategy to obtain serialized data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images