A data processing method that can replace stored procedures

Abstract

This application relates to a data processing method that replaces stored procedures. The method includes: acquiring data processing request information, including a target script identifier, a target data source identifier, and request parameters; parsing the target script based on the target script identifier, determining the set of sensitive fields participating in branch determination, and generating a branch semantic fingerprint based on the sensitive field set; querying the original result set from the corresponding database based on the target data source identifier, and converting the original result set into a standardized result object according to a preset heterogeneous field mapping rule; comparing the actual semantic features corresponding to the sensitive field set in the standardized result object with the branch semantic fingerprint to obtain a branch release result; when the branch release result representation passes, inputting the standardized result object into the target script to execute the corresponding data processing logic; when the branch release result representation fails, outputting a semantic mismatch identifier. This application is applicable to data processing in a storage-compute separation scenario.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method that replaces stored procedures. Background Technology

[0002] In existing data processing systems, some business logic is typically implemented using stored procedures within the database. Data queries, conditional judgments, branch selections, and result processing all rely on database-side execution. While this approach allows business processing to be completed in a single database environment, the high degree of coupling between the business logic and the underlying database implementation hinders migration and reuse across different database environments.

[0003] With the adoption of storage-compute separation architectures, more and more data processing logic is being executed by scripts outside the database. This allows the database to primarily handle data storage, while the external computing engine handles data computation. In this scenario, Java objects can receive business requests, organize request parameters, and send the requests to a Python process via Py4J communication channels and socket communication. The Python script then queries the database and executes the data processing logic.

[0004] However, when using external scripts to replace internal database stored procedures to execute data processing logic, although the results returned by different databases can complete general field mapping and format conversion, for target scripts that rely on field position, null value semantics, time granularity, numerical precision, and the combination relationship between fields to make conditional branch judgments, even if the returned results can be read and converted, as long as the actual semantics of the relevant fields in the branch judgment drift, the same target script may still enter different processing branches in different database environments, thus resulting in inconsistent processing results for the same business request.

[0005] Existing technologies focus more on heterogeneous field type mapping, metadata conversion, and general data quality checks. They lack specific design for how to perform dedicated semantic checks on sensitive fields that affect branch paths before the target script is executed, and whether to allow script execution based on the check results. Summary of the Invention

[0006] This application provides a data processing method that replaces stored procedures, in order to solve the problem in the prior art where, when using external scripts to replace internal database stored procedures to execute data processing logic, the actual semantics of the relevant fields in the branch decision drifts, causing the same target script to enter different processing branches in different database environments.

[0007] On one hand, this application provides a data processing method that replaces stored procedures. This method is applied to a data processing management device. The method includes: acquiring data processing request information, which includes a target script identifier, a target data source identifier, and request parameters; parsing a target script based on the target script identifier to determine a set of sensitive fields participating in branch determination, and generating a branch semantic fingerprint based on the set of sensitive fields; querying an original result set from a corresponding database based on the target data source identifier, converting the original result set into a standardized result object according to a preset heterogeneous field mapping rule, comparing the actual semantic features in the standardized result object corresponding to the set of sensitive fields with the branch semantic fingerprint to obtain a branch release result; when the branch release result representation passes, inputting the standardized result object into the target script to execute corresponding data processing logic; and when the branch release result representation fails, outputting a semantic mismatch identifier.

[0008] In the above method, the data processing and management device does not only perform general format conversion on the database query results, but first establishes a branch semantic fingerprint around the sensitive fields that actually participate in the branch determination in the target script, and then performs a release determination based on the actual semantic features in the standardized result object. In this way, semantic drift situations that may lead to branch misjudgment can be identified before the target script is executed, thereby improving the branch stability of the same target script in different database environments in the scenario of replacing stored procedures.

[0009] In one possible design, the step of parsing the target script based on the target script identifier to determine the set of sensitive fields involved in branch decision-making includes: performing abstract syntax tree parsing on the target script to extract fields referenced in conditional statements, threshold statements, and null value statements; and determining the extracted fields as the set of sensitive fields. Through this design, the data processing and management device can extract fields from the statements within the target script that actually affect branch switching, rather than performing indiscriminate processing on all query fields, thereby improving the targeting of branch semantic modeling.

[0010] In one possible design, the branch semantic fingerprint includes at least one of field position dependency, null value semantic constraint, temporal granularity constraint, numerical precision constraint, and inter-field association constraint. Through this design, the branch semantic fingerprint can not only characterize the existence of a field, but also characterize the semantic boundary information that influences the determination of the target script branch, thereby enabling subsequent comparison results to truly reflect whether the standardized result object meets the requirements for stable execution of the target script branch.

[0011] In one possible design, converting the original result set into a standardized result object according to a preset heterogeneous field mapping rule includes: mapping the original fields in the original result set to standard field names according to the preset heterogeneous field mapping rule; uniformly processing the null value representation, time precision, and numerical precision in the original result set; and generating the standardized result object based on the uniformly processed result. Through this design, the data processing and management device can converge the structural and representational differences returned by different databases into a unified object form, providing a consistent data foundation for subsequent branch semantic comparisons.

[0012] In one possible design, comparing the actual semantic features corresponding to the sensitive field set in the standardized result object with the branch semantic fingerprint to obtain the branch release result includes: extracting at least one of the following from the standardized result object: field position features, null value semantic features, time granularity features, numerical precision features, and field association features corresponding to the sensitive field set; comparing the extracted actual semantic features with the branch semantic fingerprint item by item; and determining that the branch release result characterization fails when a mismatch item meets a preset mismatch condition. Through this design, the data processing and management device can upgrade the target script branch stability determination from general field verification to a release determination oriented towards actual branch execution risk.

[0013] In one possible design, when the branch release result fails to represent the semantic mismatch, the method further includes: invoking a preset correction process based on the semantic mismatch identifier, the preset correction process including at least one of field rearrangement, null value expression correction, time granularity correction, and numerical precision correction; after the preset correction process is completed, re-executing the comparison between the actual semantic features and the branch semantic fingerprint; if the re-comparison still fails to represent the semantic mismatch, blocking the execution of the target script. Through this design, the data processing management device can not only identify semantic mismatches but also provide a targeted correction opportunity before blocking execution, thus balancing execution stability and processing continuity.

[0014] In one possible design, after the branch release result characterization is passed and the target script is executed, the method further includes: recording the branch execution record corresponding to the target script, wherein the branch execution record is used to characterize the correspondence between the branch semantic fingerprint, the actual semantic features, and the actual execution branch path. This design provides a traceable basis for subsequent analysis of whether the target script branch has shifted under different data source conditions.

[0015] In one possible design, the branch execution record includes the target script identifier, the target data source identifier, the sensitive field set identifier, the actual semantic features, the branch release result, and the actual execution branch path.

[0016] In one possible design, before converting the original result set into a standardized result object according to a preset heterogeneous field mapping rule, the method further includes: obtaining metadata information and data source version information corresponding to the target data source; determining a target mapping rule version matching the target data source from multiple mapping rule versions based on the metadata information and the data source version information; and converting the original result set using the target mapping rule version. Through this design, the data processing management device can select the corresponding mapping rule version when the data source structure evolves or its version changes, improving the adaptability of the standardization conversion process.

[0017] In one possible design, the target script is a Python script, and the data processing request information is sent from a Java object to the Python process. The Java object and the Python process interact via Py4J communication channels and socket communication. Through this design, the method provided in this application can be applied to storage-compute separation scenarios where the database only handles data storage, and the data processing logic is handled by an external Python script, thereby achieving decoupling between business logic and the underlying database data types.

[0018] Compared with the prior art, the embodiments of this application determine the set of sensitive fields by parsing the target script and generating a branch semantic fingerprint. Before the target script is executed, the actual semantic features in the standardized result object are judged to allow the execution. This enables the identification of semantic drift that may cause branch deviation before the target script is executed, and avoids the same target script from entering the wrong processing branch due to the inconsistency of the semantics of sensitive fields in different database environments. Attached Figure Description

[0019] Figure 1 A schematic diagram of the architecture of a data processing system that provides an alternative to stored procedures, as provided in an embodiment of this application; Figure 2 A flowchart illustrating a data processing method for an alternative stored procedure provided in this application embodiment; Figure 3 This is a schematic diagram of a branch semantic fingerprint structure provided in an embodiment of this application; Figure 4 This is a schematic diagram of a branch release determination and correction process provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. It should be understood that the descriptions such as "first" and "second" in the embodiments of this application are only used to distinguish different objects and are not used to limit relative importance.

[0021] Figure 1 This is a schematic diagram of the architecture of a data processing system that provides an alternative to stored procedures, as an embodiment of this application. Figures 1-4 As shown, the system may include a Java caller side, a data processing and management device, a Python script execution side, and a database. The database is used to store basic data and business data. The Java caller side is used to receive data processing requests initiated by users and organize request parameters. The data processing and management device is used to perform request parsing, branch semantic fingerprint generation, result standardization, semantic comparison, and release control. The Python script execution side is used to execute pre-written target scripts.

[0022] In one implementation, a Java object can communicate with a Python process through a communication channel created by the Py4J service. The Java object sends the interface address and parameters to the Python process via a socket. The Python process initiates a database query based on the received target script identifier and request parameters, and executes the target script processing logic on the query results.

[0023] While existing methods of using external scripts to replace stored procedures can decouple business logic from the database, the same target script may enter different branches in different database environments when the returned results differ in field order, null value representation, time granularity, or numerical precision. To address these issues, this application provides a data processing method that replaces stored procedures.

[0024] Figure 2 A flowchart illustrating a data processing method for an alternative stored procedure provided in this application embodiment is shown below. Figure 2 The method may include the following steps: S101: The data processing management device obtains data processing request information; S102: The target script is parsed according to the target script identifier, the sensitive field set is determined, and a branch semantic fingerprint is generated; S103: The original result set is queried according to the target data source identifier, and the original result set is converted into a standardized result object; S104: The actual semantic features corresponding to the sensitive field set in the standardized result object are compared with the branch semantic fingerprint to obtain the branch release result; S105: When the branch release result representation is passed, the target script is input to execute the corresponding data processing logic; when the branch release result representation is not passed, a semantic mismatch identifier is output, and a preset correction process is called as needed; S106: After the target script is executed, the branch execution record is recorded.

[0025] In step S101, the data processing request information may include a target script identifier, a target data source identifier, and request parameters. The request parameters may include at least one of the following: query conditions, filtering conditions, business thresholds, branch control parameters, and result output parameters.

[0026] In one optional implementation, the data processing request information can be generated by a Java object. The Java object can obtain the interface address and request parameters based on the user-initiated interface call, and encapsulate the target script identifier, target data source identifier, and request parameters into data processing request information, which is then sent to the Python process. Upon receiving the data processing request information, the Python process forwards it to the processing module corresponding to the data processing management device.

[0027] In step S102, the data processing and management device parses the target script based on the target script identifier to determine the set of sensitive fields involved in the branch determination. It should be noted that, in this embodiment, the extraction of the sensitive field set is not a unified processing of all fields in the original result set, but rather focuses only on the statements in the target script that actually control the switching of the execution path.

[0028] In one optional implementation, the data processing and management device may first convert the target script into a corresponding abstract syntax tree (AST), and then locate the statement nodes related to branch decisions within the AST. The statement nodes related to branch decisions may include at least one of condition judgment nodes, threshold comparison nodes, null value judgment nodes, logical combination nodes, and branch jump nodes.

[0029] In one implementation, the data processing and management device can extract candidate fields from the following statements: if statements, elif statements, case statements, loop interruption judgment statements, exception branch judgment statements, and return result classification judgment statements used to control execution path switching. If the above statements directly reference query result fields, the referenced fields are determined as candidate fields; if the above statements reference intermediate variables, the assignment statements of the intermediate variables are further backtracked to generate them, the original fields on which the assignment statements depend are extracted, and the original fields are determined as candidate fields.

[0030] In one alternative implementation, the data processing and management device can perform recursive expansion processing on logical combination expressions. For compound judgment expressions composed of logical AND, logical OR, and logical NOT relations, the fields participating in each sub-judgment can be extracted separately and merged to form a candidate field set. For nested conditional statements, each level of condition nodes can be traversed layer by layer from the outside in until all fields that affect branch switching are extracted.

[0031] In one alternative implementation, not all fields referenced by the target script are included in the sensitive field set. If a field only participates in result display, log output, non-branching arithmetic operations, or result encapsulation after execution, and does not participate in branch determination, threshold comparison, missing value determination, or path switching, the data processing and management device can determine that field as a non-sensitive field and exclude it from the sensitive field set.

[0032] An exemplary extraction rule is as follows: When the target script contains the statement "Enter the first branch when field A is empty and the time granularity corresponding to field B is daily, otherwise enter the second branch," the data processing and management device can identify fields A and B as sensitive fields, instead of identifying field C, which only participates in the summation operation within the first branch but does not affect the branch selection, as a sensitive field. In this way, the set of sensitive fields can be concentrated to represent the range of fields that actually affect the branch path.

[0033] In one optional implementation, after determining the set of sensitive fields, the data processing and management device can also generate a field role identifier for each sensitive field. The field role identifier is used to indicate the type of function of the corresponding sensitive field in the target script, and the type of function may include at least one of null value judgment field, time judgment field, precision judgment field, position-dependent field, and association judgment field.

[0034] In step S102, after determining the set of sensitive fields, the data processing and management device can further generate a corresponding branch semantic fingerprint based on the set of sensitive fields. The branch semantic fingerprint is not a general description of the overall structure of the original result set, but rather a set of expected constraints on the actual semantic state of the sensitive fields before the target script enters the target branch.

[0035] In one optional implementation, the branch semantic fingerprint can be generated into fingerprint units at the field granularity and aggregated according to the branch nodes in the target script. Each fingerprint unit may include at least one or more of the following parameters: standard field identifier, field role identifier, expected position index, expected null value semantic type, expected time granularity level, expected numerical precision range, and associated field identifier.

[0036] The expected null value semantic type can be used to distinguish different null value meanings such as uncollected, invalid value, empty string, zero value substitution, and negligible missing value; the expected time granularity level can be used to represent at least one of grade, month, day, hour, level, or second; the expected numerical precision range can be used to represent the decimal place range, significant digit range, or rounding rule; and the associated field identifier can be used to represent that two or more fields should exist simultaneously in the branch decision, appear in a fixed order, or satisfy a preset combination relationship.

[0037] In one optional implementation, if the target script has multiple candidate branches, the data processing and management device can generate a corresponding branch semantic fingerprint for each candidate branch and establish an index relationship in the form of "script identifier - branch identifier - sensitive field set identifier". Subsequently, when performing the release decision, the comparison can be performed on the branch semantic fingerprint corresponding to the target branch, instead of using a single verification rule applicable to all branches.

[0038] An exemplary branch semantic fingerprint may include: Field A's field role identifier is a null value determination field, with an expected null value semantic type of "not collected"; Field B's field role identifier is a time determination field, with an expected time granularity level of daily; Field C's field role identifier is a precision determination field, with an expected numerical precision range of two decimal places; and the associated field identifiers of Fields B and C indicate that both should exist simultaneously. In this way, the branch semantic fingerprint can be stored in a rule base or configuration table as a structured object, facilitating retrieval and updates.

[0039] In one optional implementation, the branch semantic fingerprint can be generated when the target script is initially deployed, or it can be regenerated after the target script version changes. If the branch decision statements in the target script change, the data processing and management device can re-execute sensitive field extraction and semantic fingerprint generation to ensure that the constraints on which subsequent approval decisions are based are consistent with the target script.

[0040] In step S103, the data processing and management device queries the original result set from the corresponding database according to the target data source identifier, and converts the original result set into a standardized result object according to the preset heterogeneous field mapping rules.

[0041] In one optional implementation, before performing the transformation, the data processing management device can obtain metadata information and data source version information corresponding to the target data source. The metadata information may include at least one of field name, field type, field order, field length, field precision, and nullable attributes, and the data source version information may be used to characterize the corresponding database schema version, interface version, or data source structure version.

[0042] In one optional implementation, the data processing management device can determine a target mapping rule version that matches the target data source from multiple mapping rule versions based on the metadata information and the data source version information, and then use the target mapping rule version to transform the original result set. This method allows for the selection of the appropriate mapping rule version when the data source structure evolves or the interface version changes, improving the adaptability of the standardized transformation process.

[0043] In one optional implementation, the conversion process may include the following actions: mapping the original fields in the original result set to standard field names according to a preset heterogeneous field mapping rule; uniformly processing the null value representation, time precision, and numerical precision in the original result set; and generating a standardized result object based on the uniformly processed result.

[0044] For example, in cases of inconsistent null value representation, the data processing and management device can uniformly map empty strings, specific placeholders, and preset exception codes to standard null value identifiers; in cases of inconsistent time precision, it can uniformly convert second-level, grade-level, or hour-level time into preset time granularity; and in cases of inconsistent numerical precision, it can uniformly normalize data from different sources to preset decimal places or significant number ranges.

[0045] In step S104, the data processing and management device compares the actual semantic features corresponding to the sensitive field set in the standardized result object with the branch semantic fingerprint to obtain the branch release result. The actual semantic features may include at least one of the following: actual field location, actual null value semantics, actual time granularity, actual numerical precision, and actual field association relationship.

[0046] In one optional implementation, the data processing and management device can group the comparison process according to field role identifiers. For null value determination fields, the actual null value semantics are compared with the expected null value semantics type first; for time determination fields, the actual time granularity level is compared with the expected time granularity level; for precision determination fields, the actual numerical precision falls within the expected numerical precision range; for location-dependent fields, the actual location index is compared with the expected location index; for association determination fields, the related fields are compared to see if they exist simultaneously and satisfy a preset association relationship.

[0047] In one optional implementation, the data processing and management device can set different mismatch levels for different types of mismatches. The mismatch levels can include fatal mismatches, significant mismatches, and general mismatches. Fatal mismatches can include missing sensitive fields, incorrect field roles, fundamentally inconsistent null value semantics, and missing necessary related fields; significant mismatches can include time granularity being one level higher or lower than expected, and numerical precision exceeding the expected range; general mismatches can include positional index offsets that can be corrected by standard field names, etc.

[0048] In one optional determination method, when there is at least one fatal mismatch, the data processing management device can directly determine that the branch release result representation fails; when there is no fatal mismatch but the number of important mismatches is greater than a preset threshold, the branch release result representation also fails; when there are only general mismatches and the correction action corresponding to the general mismatch is already included in the preset correction process, the correction process can be entered first, and then the pass can be determined based on the comparison result after correction.

[0049] In one optional implementation, the data processing and management device can further assign corresponding mismatch weights to each mismatch item and determine the branch release result based on the relationship between the sum of the mismatch weights and a preset release threshold. If the sum of the mismatch weights is greater than the preset release threshold, the branch release result is determined to be unsuccessful; if the sum of the mismatch weights is not greater than the preset release threshold, the branch release result is determined to be successful or enter the correction and review state.

[0050] An exemplary judgment rule is as follows: When field A is a null value judgment field and the expected null value semantics is not collected, but field A is parsed as a zero value replacement in the standardized result object, the data processing and management device can determine it as a fatal mismatch; when the expected time granularity of field B is daily, but the actual time granularity is hourly, it can be determined as a significant mismatch; when the actual position of field C deviates from the expected position but the standard field name is the same, it can be determined as a general mismatch.

[0051] Through the above comparison rules, the formation of branch release results is no longer limited to principle descriptions, but has clear judgment basis, judgment level and passing conditions, enabling technicians in the relevant technical field to implement release control logic accordingly.

[0052] In step S105, when the branch release result representation passes, the data processing management device inputs the standardized result object into the target script to execute the corresponding data processing logic; when the branch release result representation fails, a semantic mismatch identifier is output. It should be noted that the semantic mismatch identifier can be used to represent at least one of the following: mismatch field, mismatch type, mismatch level, and whether correction is allowed.

[0053] In one optional implementation, if the branch release result fails, but the mismatch is a correctable mismatch, the data processing management device can invoke a preset correction procedure. The preset correction procedure can invoke the corresponding correction rule based on the mismatch type, rather than applying the same correction method to all mismatches.

[0054] For field position mismatches, the data processing and management device can perform field rearrangement based on the standard field names and field mapping relationships. This involves rearranging the fields in the standardized result object according to a preset standard order and regenerating the field position features. If the rearranged field position matches the expected position index, the corresponding position mismatch is eliminated.

[0055] For null value semantic mismatches, the data processing and management device can perform null value expression correction according to a preset null value semantic conversion table. This null value semantic conversion table can be used to convert empty strings, special placeholders, default zero values, and exception code values ​​into preset standard null value types. For example, an empty string can be converted into a missing value identifier, and a preset exception code can be converted into an uncollected identifier; however, if a value is explicitly defined as a valid zero value in the business rules, it will not be corrected to a missing value.

[0056] For time granularity mismatches, the data processing and management device can perform time granularity correction. This correction may include truncating second-level times to day-level times, merging hour-level times to day-level times, or padding date fields with missing time portions to a standard date format. If the target script's corresponding branch only requires day-level judgment, the time granularity features can be regenerated after the above corrections and used for comparison again.

[0057] For numerical precision mismatches, the data processing and management device can perform numerical precision correction. This correction may include rounding, truncation, or precision normalization according to preset rules. For example, when the target script requires numerical fields to retain two decimal places, the actual values ​​can be uniformly processed to two decimal places before comparison; if the processed values ​​still exceed the preset precision range, the mismatch status is retained.

[0058] In one optional implementation, the preset correction process can be executed sequentially in the order of position correction, null value correction, time granularity correction, and numerical precision correction, or it can be selectively executed only for the currently identified mismatch type. After each type of correction is executed, the data processing and management device can locally update the corresponding actual semantic features without having to query the database again.

[0059] In one optional implementation, after completing the preset correction process, the data processing and management device can re-execute the comparison between the actual semantic features and the branch semantic fingerprint. If no fatal mismatch is found after the re-comparison, and the important mismatch and general mismatch meet the preset release conditions, the branch release result is determined to be passed; if fatal mismatch still exists after the re-comparison, or the sum of mismatch weights is still greater than the preset release threshold, the execution of the target script is blocked, and the semantic mismatch identifier is output.

[0060] An exemplary correction process is as follows: If field A only has inconsistencies between empty strings and missing value identifiers, and field B only has inconsistencies between second-level and day-level times, the data processing and management device can first correct the empty strings in field A to missing value identifiers, then truncate the second-level times in field B to day-level times, and then re-execute the comparison; if the re-comparison result is satisfactory, the target script is input to execute the corresponding data processing logic; if the re-comparison still fails, the execution of the target script is blocked.

[0061] In one alternative implementation, for semantic mismatches that cannot be eliminated by the preset correction process, the data processing management device can directly block the execution of the target script and return the semantic mismatch flag to the calling side to avoid continuing to execute the target script when the semantic state does not meet the conditions for stable branch execution.

[0062] In step S106, the data processing and management device records a branch execution record after the target script has been executed. The branch execution record is used to characterize the correspondence between the branch semantic fingerprint, the actual semantic features, and the actual execution branch path.

[0063] In one optional implementation, the branch execution record may include a target script identifier, a target data source identifier, a sensitive field set identifier, actual semantic features, branch release result, and the actual execution branch path. By recording the above information, the semantic factors causing branch changes can be quickly located when analyzing the execution differences of the same target script under different data source conditions.

[0064] In one optional implementation, the data processing and management device can also statistically analyze the mismatch types that occur frequently under different data source conditions based on historical branch execution records, and update the preset heterogeneous field mapping rules, null value semantic conversion table, or branch semantic fingerprint accordingly, thereby improving the accuracy of subsequent release judgment and correction processes.

[0065] It should be noted that steps S101 to S106 above do not necessarily have to be executed in a strict order. For example, the determination of the target mapping rule version can be performed after the target script is parsed and before the database query; or, for example, when the semantic mismatch identifier indicates a fatal mismatch, the data processing management device can directly block the execution of the target script without calling the preset correction process. As long as stable control over the execution of the target script branch can be achieved, it falls within the protection scope of the embodiments of this application.

Claims

1. A data processing method for replacing a stored procedure, characterized by, Applied to data processing and management equipment, the method includes: Obtain data processing request information, which includes target script identifier, target data source identifier, and request parameters; The target script is parsed based on the target script identifier to determine the set of sensitive fields involved in the branch determination, and a branch semantic fingerprint is generated based on the set of sensitive fields. The original result set is queried from the corresponding database according to the target data source identifier. The original result set is converted into a standardized result object according to the preset heterogeneous field mapping rules. The actual semantic features in the standardized result object corresponding to the sensitive field set are compared with the branch semantic fingerprint to obtain the branch release result. When the branch release result representation passes, the standardized result object is input into the target script to execute the corresponding data processing logic. When the branch release result representation fails, a semantic mismatch identifier is output.

2. The data processing method of replacing a stored procedure according to claim 1, wherein, The step of parsing the target script based on the target script identifier to determine the set of sensitive fields participating in the branch determination includes: The target script is parsed using an abstract syntax tree to extract the fields referenced in conditional statements, threshold statements, and null statement statements. The extracted fields are identified as the set of sensitive fields.

3. The data processing method of replacing a stored procedure according to claim 1, wherein, The branch semantic fingerprint includes at least one of the following: field position dependency, null value semantic constraint, time granularity constraint, numerical precision constraint, and inter-field association constraint.

4. The data processing method of replacing a stored procedure according to claim 1, wherein, The step of converting the original result set into a standardized result object according to a preset heterogeneous field mapping rule includes: According to the preset heterogeneous field mapping rules, the original fields in the original result set are mapped to standard field names; The null value representation, time precision, and numerical precision of the original result set are uniformly processed. The standardized result object is generated based on the results of the unified processing.

5. The data processing method of replacing a stored procedure according to claim 4, wherein, The step of comparing the actual semantic features corresponding to the sensitive field set in the standardized result object with the branch semantic fingerprint to obtain the branch release result includes: Extract at least one of the following features from the standardized result object: field position feature, null value semantic feature, time granularity feature, numerical precision feature, and field association feature corresponding to the sensitive field set; The extracted actual semantic features are compared item by item with the branch semantic fingerprint; When the mismatch item meets the preset mismatch condition, the branch release result is determined to be unqualified.

6. The data processing method of replacing a stored procedure according to claim 5, wherein, When the branch release result fails to pass the characterization, the method further includes: A preset correction process is invoked based on the semantic mismatch identifier. The preset correction process includes at least one of field rearrangement, null value expression correction, time granularity correction, and numerical precision correction. After the preset correction process is completed, the comparison between the actual semantic features and the branch semantic fingerprint is re-executed; If the re-comparison still fails, the execution of the target script is blocked.

7. The data processing method of replacing a stored procedure according to claim 6, wherein, After the branch release result is deemed acceptable and the target script is executed, the method further includes: Record the branch execution record corresponding to the target script. The branch execution record is used to represent the correspondence between the branch semantic fingerprint, the actual semantic feature and the actual execution branch path.

8. The data processing method for replacing stored procedures according to claim 7, characterized in that, The branch execution record includes the target script identifier, the target data source identifier, the sensitive field set identifier, the actual semantic features, the branch release result, and the actual execution branch path.

9. The data processing method for replacing stored procedures according to claim 5, characterized in that, Before converting the original result set into a standardized result object according to a preset heterogeneous field mapping rule, the method further includes: Obtain the metadata information and data source version information corresponding to the target data source; Based on the metadata information and the data source version information, determine the target mapping rule version that matches the target data source from multiple mapping rule versions; The original result set is transformed using the version of the target mapping rule.

10. The data processing method for the alternative stored procedure according to any one of claims 1-9, characterized in that, The target script is a Python script, and the data processing request information is sent from a Java object to the Python process. The Java object and the Python process interact through the Py4J communication channel and socket communication.

Images