A software development cloud computing system with a big data processing module

Abstract

The application discloses a software development cloud computing system with a big data processing module, and relates to the technical fields of computer systems and data processing.The system comprises a development input collection module, which is used for receiving source code data, construction script data, dependency declaration data, configuration file data and development environment description data, and performing integrity check, structure analysis, dependency analysis, parameter extraction and environment element analysis processing, respectively, to form a standardized development input data set.In the application, unified feature abstraction and aggregation are performed on the source code, construction script, dependency declaration, configuration parameter and development environment element to form version fingerprint data capable of stably representing the software version state, thereby significantly enhancing the strictness of analysis, check and influence analysis in a high-difference version scene, and avoiding the misjudgment, omission or resource waste caused by the fixed threshold or artificial configuration parameter in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer system and data processing technology, and particularly relates to a software development cloud computing system with a big data processing module. BACKGROUND

[0002] With the rapid development of cloud computing technology and big data technology, software development activities gradually evolve from traditional local development mode to centralized and distributed development mode based on cloud platform. In the existing software development cloud computing environment, the development process usually involves source code management, build script configuration, dependency declaration management, configuration file maintenance and development environment configuration and other types of heterogeneous data. These data jointly determine the build result and running behavior of software products.

[0003] The existing software development cloud computing system generally has the problems of scattered development input data management, lack of unified standardized processing, version identification mainly focusing on code level and being difficult to cover multi-dimensional elements such as build script, dependency relationship, configuration parameter and development environment, build process analysis relying on dynamic execution and lacking static modeling capability for execution steps, dependency relationship and data input / output relationship, difficulty in timely and accurate evaluation of the influence range of changes in the overall build process, and limitation of the use of historical running record data to monitoring and troubleshooting, fixed strategy being used for resource scheduling and concurrent control, and difficulty in adaptive adjustment according to job characteristics and change risk, thereby leading to insufficient version traceability, high release risk, and limited resource utilization efficiency and system stability. SUMMARY

[0004] The present application relates to the field of computer system and data processing technology, and particularly relates to a software development cloud computing system with a big data processing module.

[0005] In order to achieve the above object, the application adopts the following technical scheme: A software development cloud computing system with a big data processing module, comprising: a development input collection module for receiving source code data, build script data, dependency declaration data, configuration file data and development environment description data, performing integrity check, structure analysis, dependency analysis, parameter extraction and environment element analysis processing respectively, and forming a standardized development input data set; a version fingerprint construction module for receiving the standardized development input data set, performing feature abstract calculation and aggregation on the source code data, build script data, dependency analysis data, configuration parameter data and environment element data respectively, and forming version fingerprint data; a job semantic modeling module for receiving the standardized development input data set and the version fingerprint data, performing feature unpacking and version feature index calculation based on the version fingerprint data, performing interval determination processing on the version feature index, determining the version feature index as one of four preset intervals, determining the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter and the version binding consistency threshold parameter based on the interval determination result, and then using a build script static analysis abstraction algorithm to perform step analysis, operator abstraction and dependency relationship modeling on the build script data, combining data input description and data output description to form job abstraction data, and performing operator semantic rule check, data contract consistency check and binding relationship consistency verification on the job abstraction data to form job semantic data; a change impact analysis module for receiving the job semantic data and the version fingerprint data, performing feature unpacking and version feature index calculation based on the version fingerprint data, performing interval determination processing on the version feature index, determining the version feature index as one of four preset intervals, determining the propagation depth control parameter, the edge weight weighting parameter, the propagation cutoff threshold parameter, the path merging strategy parameter and the change source confidence parameter based on the interval determination result, and then using a dependency propagation impact domain calculation algorithm to identify change nodes and perform impact propagation and impact range calculation based on operator dependency relationship, forming change impact evaluation data, and generating release control instruction data according to a preset release control rule; a big data processing module for receiving the job semantic data, the change impact evaluation data and the historical running record data, performing distributed statistical analysis and feature aggregation processing to form job running feature data; an adaptive resource concurrent control module for receiving the job running feature data and the release control instruction data, calculating resource load index and concurrent pressure index, and generating running control strategy data under the condition allowed by the release control instruction; a software product output module for receiving the version fingerprint data, the release control instruction data and the running control strategy data, controlling and deploying the build process for constructing the standardized development input data set into a software product according to the release control instruction, and writing the version fingerprint data into the software product meta information to obtain a developed software product.

[0006] As a further description of the above technical scheme:

[0007] The development input collection module receives source code data, build script data, dependency declaration data, configuration file data and development environment description data, performs integrity check processing on the source code data to generate checked source code data, performs structure analysis processing on the build script data to generate analyzed build script data, performs dependency item analysis processing on the dependency declaration data to generate dependency analysis data, performs parameter extraction processing on the configuration file data to generate configuration parameter data, performs environment element analysis processing on the development environment description data to generate environment element data, and performs merging processing on the checked source code data, the analyzed build script data, the dependency analysis data, the configuration parameter data and the environment element data to form a standardized development input data set.

[0008] As a further description of the above technical solution:

[0009] The version fingerprint construction module performs code feature abstract calculation processing on the source code data to obtain code feature abstract data, performs script feature abstract calculation processing on the analyzed build script data to obtain script feature abstract data, performs dependency feature abstract calculation processing on the dependency analysis data to obtain dependency feature abstract data, performs configuration feature abstract calculation processing on the configuration parameter data to obtain configuration feature abstract data, performs environment feature abstract calculation processing on the environment element data to obtain environment feature abstract data, and performs aggregation calculation processing on the code feature abstract data, the script feature abstract data, the dependency feature abstract data, the configuration feature abstract data and the environment feature abstract data to form version fingerprint data.

[0010] As a further description of the above technical solution:

[0011] The job semantic modeling module performs feature unpacking and feature index calculation based on the version fingerprint data to obtain version feature indexes, compares the version feature indexes with preset interval determination logic, determines the interval in which the version feature indexes fall according to the size relationship between the version feature indexes and three interval boundary values, matches the version feature indexes with a preset parameter mapping rule based on the interval to determine the parameter values corresponding to the current version features, and forms a mapping result including the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter, and the version binding consistency threshold parameter. The parsed construction script data is subjected to execution step analysis processing to generate a candidate execution step set, and the analysis confidence of each candidate execution step is calculated. The analysis confidence is compared with the analysis confidence threshold parameter, and the candidate execution steps that meet the threshold condition are retained and the candidate execution steps that do not meet the threshold condition are removed to obtain execution step description data. The execution step description data is subjected to operator abstraction processing to map each execution step to a corresponding operator semantic unit, and operator identification and operator attribute description are generated to obtain operator description data. The execution dependency relationship between the operators in the operator description data is subjected to directed relationship modeling processing. For each pair of candidate dependency relationships, a dependency clue is extracted and weighted calculation is performed according to the dependency determination weight parameter. The existence and direction of the dependency edge are determined based on the weighted result to obtain operator dependency relationship data. The data input description and data output description obtained in the execution step analysis process and the operator dependency relationship data are subjected to unified modeling processing. The data input description and the data output description are bound to the corresponding operator identification, and the operator dependency relationship data is organized into an executable topology structure to form job abstraction data. The job abstraction data is subjected to operator semantic rule verification processing. According to the semantic rule strength parameter, a set of semantic rules is selected and a rule triggering boundary is determined. The semantic consistency of each operator is determined and the constraint satisfaction is determined to obtain operator semantic verification result data. The data input description and the data output description in the job abstraction data are subjected to data contract consistency verification processing. According to the semantic rule strength parameter, a contract verification boundary is determined. The data structure constraint, field constraint, and dependency constraint of the input and output data are subjected to consistency determination to obtain data contract verification result data. The job abstraction data is subjected to binding relationship consistency verification processing in combination with the version fingerprint data. The target binding feature is determined based on the version fingerprint data, and the consistency determination boundary is determined according to the version binding consistency threshold parameter. The binding feature of the job abstraction data and the target binding feature are subjected to consistency matching and deviation determination to obtain version binding verification result data. The operator semantic verification result data, the data contract verification result data, and the version binding verification result data are subjected to summary processing. According to the semantic rule strength parameter and the version binding consistency threshold parameter, a summary decision boundary is determined to form consistency verification result data. The job abstraction data and the consistency verification result data are merged to form job semantic data.

[0012] As a further description of the above technical solutions:

[0013] The change impact analysis module performs feature unpacking and feature index calculation based on the version fingerprint data to obtain version feature indexes, and compares and processes the version feature indexes with preset interval determination logic, determines the interval in which the version feature indexes fall according to the size relationship between the version feature indexes and three interval boundary values; the change impact analysis module matches the version feature indexes with the preset parameter mapping rule based on the interval determination result, determines the parameter values corresponding to the current version features, and collects the parameter values to form a mapping result of the version fingerprint data, the mapping result including a propagation depth control parameter, an edge weight weighting parameter, a propagation cutoff threshold parameter, a path merging strategy parameter, and a change source confidence parameter; the job semantic data is analyzed and processed by operator node traversal to form node analysis result data, the change node set is identified based on the version fingerprint data, and the starting impact strength is configured for each change node in the change node set according to the change source confidence parameter; the node analysis result data is analyzed and processed by operator dependency relationship impact propagation, the propagation level is limited according to the propagation depth control parameter, the propagation edge is weighted according to the edge weight weighting parameter, the cumulative impact strength is threshold cut off according to the propagation cutoff threshold parameter, and the multi-path propagation result is merged according to the path merging strategy parameter to form impact propagation result data; the impact propagation result data is calculated and processed by the execution path impact range according to the version fingerprint data, and the impact range calculation result is uniformly constrained according to the propagation depth control parameter, the propagation cutoff threshold parameter, and the path merging strategy parameter to form change impact evaluation data; the change impact evaluation data is processed by impact degree determination, and the impact degree is compared and processed with the preset release control rule to generate a release control decision to obtain release control instruction data.

[0014] As a further description of the above technical solutions:

[0015] The big data processing module performs distributed statistical analysis processing on the historical running record data, extracts job history execution frequency features, resource usage distribution features, and execution time length statistical features, and forms historical running statistical feature data; the job semantic data is processed by job execution feature aggregation to extract operator quantity features, dependency level features, and execution path structure features, and forms job execution feature data; the change impact evaluation data is processed by association feature analysis to extract impact range features, impact path features, and impact degree features, and forms change impact feature data; the historical running statistical feature data, the job execution feature data, and the change impact feature data are merged to form job running feature data, and the job running feature data is output.

[0016] As a further description of the above technical solutions:

[0017] The adaptive resource concurrency control module performs resource load index calculation processing on the job running feature data to generate resource load index data, performs concurrency stress index calculation processing on the job running feature data to generate concurrency stress index data, performs adaptive adjustment processing on the calculation resource allocation parameter and the job concurrency degree parameter according to the resource load index data and the concurrency stress index data under the condition that the release control instruction data allows execution, forms running control strategy data, and outputs the running control strategy data to the software product output module for controlling the execution mode of the construction process and the deployment process.

[0018] Further description of the above technical solution:

[0019] The software product output module controls and processes the construction process for constructing the standardized development input data set into a software product according to the release control instruction data under the condition that the release control instruction data allows execution, obtains construction execution result data, performs deployment processing on the construction execution result data according to the running control strategy data, writes version fingerprint data into software product meta information, forms deployed software product data, performs result solidification processing on the deployed software product data, obtains a developed software product, and outputs the developed software product.

[0020] The present application has the following beneficial effects:

[0021] 1、In the present application, by unified feature extraction and aggregation of source code, build script, dependency declaration, configuration parameters and development environment elements, version fingerprint data capable of stably representing software version state is formed, and further calculation of version feature index, the degree of version difference is discretely divided into multiple intervals, based on the interval determination result, the system automatically determines the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter, the version binding consistency threshold parameter and the propagation control parameter, through the above mechanism, the analysis and verification overhead can be reduced for low difference version scene, and the strictness of analysis, verification and impact analysis can be significantly enhanced for high difference version scene, thereby avoiding the misjudgment, omission or resource waste problem caused by fixed threshold or manual configuration parameter in the prior art, improving the adaptability and stability of the system under different version change intensity, performing static analysis and multi-stage abstract modeling processing on the build script data, mapping the execution steps in the script into operator semantic units with clear semantics, and constructing a directed operator dependency relationship based on the dependency determination weight parameter, further combining data input description and data output description to form an executable topology structure, on this basis, operator semantic rule verification, data contract consistency verification and version binding consistency verification are performed on the job abstract data, so that the build process is upgraded from text-level script to job semantic data with structure semantics, data semantics and version semantics constraints, thereby effectively avoiding the problem that implicit dependencies, data inconsistency or environment binding errors in the build script are not discovered until the subsequent build or deployment stage, significantly improving the predictability and verifiability of the build process.

[0022] 2、In the present application, the change impact analysis module is based on job semantic data, identifies a set of change nodes corresponding to version changes, and takes this as the propagation source, performs directed impact propagation analysis based on operator dependency relationship, through joint control of propagation depth control parameter, edge weight weighting parameter, propagation truncation threshold parameter and path merging strategy parameter, the system can effectively suppress the diffusion of irrelevant or low confidence paths during propagation, and uniformly merge the multi-path propagation results, thereby forming change impact evaluation data that can accurately reflect the actual impact range of version changes on job execution path, operator node and overall job structure, avoiding the impact range expansion or omission problem caused by simple dependency file-level difference or manual experience judgment in the prior art, and providing a reliable basis for subsequent release control and resource scheduling.

[0023] 3、In the present application, based on the change impact evaluation data, the influence degree is determined by rules, and compared with the preset release control rule, the release control instruction data is automatically generated, so as to realize the dynamic control of the construction process and the deployment process, under the condition that the release control instruction allows execution, the system further combines the historical running record data, the job semantic data and the change impact evaluation data, forms the job running feature data, and calculates the resource load index and the concurrent pressure index according to the job running feature data, through the above processing, the concurrency and resource utilization can be improved when the version change risk is low, the resource allocation and concurrent control can be automatically tightened when the version change risk is high, and even the release process can be blocked, so as to ensure the system stability and release safety, avoid the problem of long-term conservative or excessive allocation of resource configuration, write the version fingerprint data into the software product meta information in the software product output stage, and perform consistency checking and result solidification processing on the construction result and the deployment result, so that the final output software product and its corresponding development input state, construction environment state and version difference feature form a stable binding relationship, thereby, the system can not only realize the automatic construction and deployment of software product, but also provide complete and reliable data basis for subsequent problem tracing, version rollback, impact review and compliance audit, which is significantly superior to the implementation mode of the construction result and the version state in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The system architecture diagram of the present application is shown in the figure.

[0025] Figure 2 The version feature index interval determination and parameter mapping flowchart of the present application is shown in the figure. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0027] REFERENCE Figure 1The application provides a software development cloud computing system with a big data processing module, which comprises a development input collection module, a version fingerprint construction module, a job semantic modeling module, a change impact analysis module, a big data processing module, an adaptive resource concurrent control module and a software product output module.

[0028] In this embodiment, the development input collection module receives source code data, build script data, dependency declaration data, configuration file data and development environment description data, first performs integrity check processing on the source code data, reads the file list and directory level information of the source code data and establishes a check object set of the source code data, extracts file size, file modification time and file content digest from the check object set one by one and forms a check feature set, performs missing file judgment, empty file judgment, duplicate file judgment and unreadable file judgment on the source code data based on the check feature set, further performs content consistency check on the files determined to be readable and not missing and generates check mark information, performs check pass object retention and check failure object elimination processing on the source code data according to the check mark information, and generates checked source code data;

[0029] Performing structural parsing processing on the build script data, reading the script type identifier and script content text of the build script data and forming script reading result data, performing statement segmentation processing on the script reading result data and generating a script statement sequence, performing key instruction recognition processing on the script statement sequence and generating an instruction fragment set, performing hierarchical merging processing on the instruction fragment set to form a script structure node set, and establishing node order relationship and node inclusion relationship for the script structure node set to form a script structure relationship set, generating parsed build script data based on the script structure node set and the script structure relationship set;

[0030] Performing dependency item parsing processing on the dependency declaration data, reading the dependency declaration text of the dependency declaration data and forming dependency declaration reading result data, performing dependency item entry segmentation processing on the dependency declaration reading result data and generating a dependency item entry set, performing field extraction processing on the dependency item entry set and generating a dependency item field set, performing normalization processing on the dependency item entry set based on the dependency item field set and generating a normalized dependency item set, and performing deduplication merging processing on the normalized dependency item set to form dependency parsing data;

[0031] Performing parameter extraction processing on the configuration file data, reading the configuration content of the configuration file data and forming configuration reading result data, performing key-value pair positioning processing on the configuration reading result data and generating a candidate parameter entry set, performing key name extraction and value extraction processing on the candidate parameter entry set to generate a parameter key set and a parameter value set, pairing the parameter key set and the parameter value set according to the entry correspondence relationship and generating a parameter entry set, and performing type judgment and value normalization processing on the parameter entry set to form configuration parameter data;

[0032] The environment element analysis processing is performed on the development environment description data, the environment description content of the development environment description data is read and environment read result data is formed, field positioning and field extraction processing is performed on the environment read result data to generate an environment element entry set, element deduplication and element normalization processing is performed on the environment element entry set to form environment element data;

[0033] The five types of data are encapsulated and merged by establishing a correlation mapping relationship within a same project scope and generating correlation index information according to a unified data identification rule, and the five types of data and the correlation index information are encapsulated and merged to form a standardized development input data set.

[0034] In the embodiment, the version fingerprint construction module receives the checked source code data, the parsed build script data, the dependency analysis data, the configuration parameter data and the environment element data,

[0035] First, code feature summary calculation processing is performed on the source code data, the source code content is read file by file according to the code file organization structure of the source code data during the code feature summary calculation processing, content normalization processing is performed on the read source code content to eliminate the influence of irrelevant format differences, content feature extraction processing is performed on the normalized source code content to obtain stable features that can represent the code logic and structure, and summary calculation and aggregation processing are performed on the stable features to form code feature summary data that has unique representation ability for the source code data;

[0036] Script feature summary calculation processing is performed on the parsed build script data, the script content is read segment by segment according to the script structure and instruction sequence in the parsed build script data during the script feature summary calculation processing, structure unification processing is performed on the script content to unify the sentence organization form, feature extraction processing is performed on the key build instructions, key parameter items and instruction interdependence in the script to obtain stable features that can represent the build behavior, and summary calculation and aggregation processing are performed on the stable features to form script feature summary data;

[0037] Dependency feature summary calculation processing is performed on the dependency analysis data, the dependency item entries are read iteratively according to the dependency item organization structure in the dependency analysis data during the dependency feature summary calculation processing, normalization processing is performed on the dependency item entries to unify the representation form of the dependency items, feature extraction processing is performed on the dependency identification information, version indication information and dependency relationship organization mode in the dependency item entries to obtain stable features that can represent the state of the dependency set, and summary calculation and aggregation processing are performed on the stable features to form dependency feature summary data;

[0038] The configuration parameter data is subjected to configuration feature abstract calculation processing. In the configuration feature abstract calculation processing, the parameter entries are read one by one according to the parameter entry organization structure in the configuration parameter data, and the key name and value of the parameter entries are unified to unify the parameter expression form. Then, the construction option, the running option and the environment option represented by the parameter entries are subjected to feature extraction processing to obtain stable features representing the configuration state. The stable features are subjected to abstract calculation and aggregation processing to form configuration feature abstract data.

[0039] The environment element data is subjected to environment feature abstract calculation processing. In the environment feature abstract calculation processing, the element entries are read one by one according to the element entry organization structure in the environment element data, and the element representation of the element entries is unified to unify the environment element expression form. Then, the development environment key element represented by the element entries is subjected to feature extraction processing to obtain stable features representing the environment state. The stable features are subjected to abstract calculation and aggregation processing to form environment feature abstract data.

[0040] The code feature abstract data, the script feature abstract data, the dependency feature abstract data, the configuration feature abstract data and the environment feature abstract data are subjected to aggregation calculation processing. In the aggregation calculation processing, the five types of feature abstracts are subjected to unified sorting and unified organization processing to ensure the stability of the aggregation order, and the five types of feature abstracts after organization are subjected to joint aggregation calculation to form version fingerprint data representing the source code state, the construction script state, the dependency state, the configuration state and the environment state.

[0041] In the embodiment, the job semantic modeling module performs feature unpacking and feature index calculation based on the version fingerprint data to obtain a version feature index. In the feature unpacking process, the version fingerprint identifier, the fingerprint abstract value and the fingerprint generation time identifier of the version fingerprint data are read, the content of the version fingerprint data is uniformly split to obtain the code feature abstract data, the script feature abstract data, the dependency feature abstract data, the configuration feature abstract data and the environment feature abstract data, the feature abstract data after splitting is subjected to comparability preparation and alignment organization to form a difference metric set, the difference metric set is subjected to aggregation calculation and scale normalization to generate the version feature index, the version feature index is compared with the preset interval determination logic, the interval into which the version feature index falls is determined, based on the interval determination result, the version feature index is matched with the preset parameter mapping rule, the parameter value corresponding to the current version feature is determined, the determined parameter value is collected to form the mapping result of the version fingerprint data, and it is ensured that the mapping result of the version fingerprint data includes the mapping result of the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter and the version binding consistency threshold parameter;

[0042] The parsed construction script data is subjected to execution step parsing processing. In the execution step parsing processing, the parsed construction script data is read according to the script structure and the instruction sequence, and construction-related instruction segments and control segments are identified. The identified segments are subjected to step boundary determination to generate a candidate execution step set. The statement structure features and context consistency features of each candidate execution step in the candidate execution step set are extracted to calculate a parsing confidence. The parsing confidence is compared with a parsing confidence threshold parameter to complete step retention determination and step elimination determination, so that the candidate execution steps that meet the threshold conditions are retained and the candidate execution steps that do not meet the threshold conditions are eliminated to obtain execution step description data.

[0043] The execution step description data is subjected to operator abstraction processing. In the operator abstraction processing, semantic classification and functional classification are performed on each execution step in the execution step description data to map the execution step to a corresponding operator semantic unit, and a unique operator identifier is generated for the operator semantic unit. Key attributes of the operator semantic unit are extracted to generate operator attribute descriptions, so as to obtain operator description data.

[0044] The execution dependency relationships between the operators in the operator description data are subjected to directed relationship modeling processing. In the directed relationship modeling processing, the operator description data is traversed according to the order of the operator identifiers to construct a candidate dependency pair set. For each pair of candidate dependency relationships, dependency clues are extracted and weighted calculation is performed on the dependency clues according to a dependency determination weight parameter. Based on the weighted results, the existence and direction of the dependency edges are determined, and the determined dependency edges are summarized and organized to obtain operator dependency relationship data.

[0045] The data input description and the data output description obtained in the execution step parsing process and the operator dependency relationship data are subjected to unified modeling processing. In the unified modeling processing, the data input description and the data output description are subjected to structure alignment and identifier alignment to maintain consistent expression of the input and output descriptions. The data input description and the data output description are bound to the corresponding operator identifiers, and the bound operator set is topologically organized according to the operator dependency relationship data to form an executable topology structure, thereby forming job abstraction data.

[0046] The job abstraction data is subjected to operator semantic rule verification processing. In the operator semantic rule verification processing, a set of semantic rules is selected according to a semantic rule strength parameter, and a rule triggering boundary is determined. Each operator in the job abstraction data is subjected to semantic consistency determination and constraint satisfaction determination according to the operator identifier, and the determination results are summarized to form operator semantic verification result data.

[0047] The data contract consistency checking process is performed on the data input description and the data output description in the job abstraction data. In the data contract consistency checking process, the contract checking boundary is determined according to the semantic rule strength parameter, the consistency determination is performed on the data structure constraint, the field constraint and the dependency constraint in the data input description and the data output description item by item, and the data contract checking result data is formed by aggregating the determination results.

[0048] The binding relationship consistency verification process is performed on the job abstraction data in combination with the version fingerprint data. In the binding relationship consistency verification process, the target binding feature is determined based on the version fingerprint data, the consistency determination boundary is determined according to the version binding consistency threshold parameter, the consistency matching and deviation determination are performed on the binding feature of the job abstraction data and the target binding feature, and the version binding checking result data is formed by aggregating the determination results.

[0049] The operator semantic checking result data, the data contract checking result data and the version binding checking result data are aggregated. In the aggregation process, the aggregation decision boundary is determined according to the semantic rule strength parameter and the version binding consistency threshold parameter, the consistency merging and conflict resolution are performed on the three types of checking results, to form the consistency checking result data. The consistency checking result data is mapped back to the job abstraction data according to the operator identifier and the job structure, and the association is solidified, to form the job semantic data.

[0050] The confidence threshold driven execution step reservation determination formula is analyzed:

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057] : script content text of the script data, : statement segmentation processing operator, used for segmenting the script content text into to generate a script statement sequence , : script statement sequence, : key instruction recognition processing operator, used for recognizing the key instructions in the script statement sequence to generate an instruction fragment set , : instruction fragment set, Hierarchical merge processing operator, used for processing data from... Form a set of script structure nodes This supports the determination of subsequent node order relationships and node inclusion relationships. : A collection of script structure nodes Step boundary determination operator, used to determine the boundary conditions from... Generate a set of candidate execution steps , : Set of candidate execution steps Candidate execution steps, ; Execution steps The characteristics of sentence structure, Execution steps Contextual consistency features : Parse the confidence calculation function, used by... and Calculate the analytical confidence level , Execution steps The resolution confidence level, The confidence threshold parameter is used to control the acceptance boundary of the decision-making process in the step. : The set of retained execution steps corresponding to the execution step description data.

[0058] The abstract mapping formula from execution steps to operator semantic units:

[0059]

[0060]

[0061]

[0062]

[0063] Operator abstract function, used to perform execution steps Perform semantic and functional classification and generate operator semantic units. , : Operator semantic unit, Operator identifier: Used to uniquely identify the semantic unit of an operator. , Operator identifier generation function, Operator attribute description, used to characterize the semantic unit of the operator. Key attributes, Attribute extraction function, used to generate , Operator description data consists of operator identifiers and operator attribute descriptions.

[0064] Directed dependency edge modeling formula driven by dependency decision weight:

[0065]

[0066]

[0067]

[0068]

[0069]

[0070] : set of candidate dependency pairs, : the : extraction result of dependency cues of the : dependency evidence between : component of dependency decision weight parameter, used to weight the : dependency cues of the : number of dependency cue categories, : weighted dependency score of candidate dependency pair : dependency existence decision threshold, : dependency existence indicator function, taking value or , : indicator function, : dependency directionality result, : directionality determination function, : graph structure representation of operator dependency relationship data, : set of directed dependency edges. Data input / output description binding and executable topology formation formula:

[0071]

[0072]

[0073]

[0074] ,

[0075]

[0076]

[0077] : data input description obtained in the execution step analysis process, ​​: data output description obtained in the execution step resolution process, : structure alignment and identity alignment function for keeping consistent representation of input and output descriptions, output aligned and , : aligned data input description set, : aligned data output description set, : binding mapping of input description to operator identity, : binding mapping of output description to operator identity, : topology organization function for forming executable topology structure according to operator dependency relationship , : executable topology structure, : encapsulation function for forming job abstraction data , : job abstraction data.

[0078] Consistency check and output formula driven by semantic rule strength and version binding consistency threshold:

[0079] ,

[0080] ,

[0081] ,

[0082] ,

[0083] ,

[0084] : operator semantic rule check function for generating operator semantic check result data , : data contract consistency check function for generating data contract check result data , : semantic rule strength parameter, : version fingerprint data or its unpacked target binding features used for binding verification, : binding relationship consistency verification function for generating version binding check result data , : version binding consistency threshold parameter, : summary and conflict resolution function for forming consistency check result data from three types of check results , : operator semantic check result data, : data contract check result data, : version binding check result data, : consistency check result data, : merge and solidify function, used to merge and solidify : according to the operator identifier and the job structure, map back to the job abstract data and complete the associated solidification, output job semantic data .

[0085] In this embodiment, after the change impact analysis module receives the job semantic data and the version fingerprint data, it performs feature unpacking and feature index calculation based on the version fingerprint data to obtain the version feature index. In the feature unpacking process, the version fingerprint identifier, fingerprint summary value and fingerprint generation time identifier of the version fingerprint data are read, and the code feature summary data, script feature summary data, dependency feature summary data, configuration feature summary data and environment feature summary data are split according to the aggregation structure. In the feature index calculation process, the split feature summary data of each type is executed for comparability preparation and alignment organization, and a difference metric set is formed. The difference metric set is executed for aggregation calculation and scale normalization to generate the version feature index. The version feature index is compared and processed with the preset interval judgment logic, the interval where the version feature index falls into is determined, based on the interval, the version feature index is matched with the preset parameter mapping rule, the parameter value corresponding to the current version feature is determined, the determined parameter value is collected to form the mapping result of the version fingerprint data, and it is ensured that the version fingerprint data mapping result includes the propagation depth control parameter, the edge weight weighting parameter, the propagation cutoff threshold parameter, the path merging strategy parameter and the change source confidence parameter. The dependency propagation impact domain calculation algorithm is executed based on the parameters, and the change impact evaluation processing is completed;

[0086] Subsequently, the operator node traversal analysis processing is performed on the job semantic data. In the traversal analysis processing, the job abstract data is read from the job semantic data, and the operator description data, operator dependency relationship data, data input description and data output description are organized according to the operator identifier. The in-degree statistics and out-degree statistics of the operator dependency relationship data are executed, and the node connection feature is formed. The binding consistency reading of the data input description and the data output description is executed, and the node data feature is formed. The associated reading of the consistency check result data is executed, and the node check feature is formed. The node connection feature, node data feature and node check feature are summarized and merged according to the operator identifier to form the node analysis result data;

[0087] The data input description and the data output description are obtained by parsing and structuring the input elements referenced in each execution step and the output elements generated in each execution step during the execution step parsing process. During the execution step parsing process, the input reference element extraction process is performed for each execution step, specifically including: parsing the input file path referenced in the execution step; parsing the data table identifier, the theme identifier or the resource identifier referenced in the execution step; parsing the upstream execution step output identifier referenced in the execution step; parsing the input value transmitted in the form of a parameter or a variable in the execution step; and structuring the above input reference elements to form the data input description associated with the corresponding execution step. During the execution step parsing process, the output generation element extraction process is performed for each execution step, specifically including: parsing the output file path declared or implied in the execution step; parsing the generated artifact identifier declared in the execution step; parsing the intermediate product identifier declared in the execution step; parsing the output variable or resource identifier visible to the subsequent step in the execution step; and structuring the above output generation elements to form the data output description associated with the corresponding execution step. Subsequently, the set of change nodes is identified based on the version fingerprint data, and the fingerprint digest value and the fingerprint generation time identifier in the version fingerprint data are compared and read during the identification process to form the change determination basis. According to the change determination basis, the difference measurement results of the code feature digest data, the script feature digest data, the dependency feature digest data, the configuration feature digest data and the environment feature digest data are obtained, the difference measurement results are merged to form the change clue set, the operator identifier corresponding to the change clue is located in the job semantic data based on the change clue set, and the change node set is output, and the starting influence strength of each change node in the change node set is configured according to the change source confidence parameter to form the change node strength configuration result;

[0088] Subsequently, the operator dependency relationship influence propagation analysis process is performed on the node analysis result data, the change node set is taken as the propagation source and the change node strength configuration result is taken as the initial propagation state in the influence propagation analysis process, the propagation front is expanded layer by layer in the direction of the directed edge of the operator dependency relationship data, the propagation level is limited according to the propagation depth control parameter in each layer of propagation, the source operator identifier, the target operator identifier and the edge direction relationship of each propagation edge in the propagation front are extracted and the propagation edge weight is calculated according to the edge weight weighting parameter, the propagation edge weight and the current cumulative influence strength of the source node are combined to obtain the incremental influence strength of the target node, the cumulative influence strength of the target node is updated and the cumulative influence strength is threshold truncated according to the propagation truncation threshold parameter to eliminate the propagation paths that do not meet the propagation condition, the propagation results of the same target node reached by multiple propagation paths are merged according to the path merging strategy parameter to form a single target node propagation state, so that the influence propagation result data is formed after the propagation level is advanced.

[0089] Subsequently, the influence propagation result data is combined with the version fingerprint data to perform a path influence range calculation process. In the influence range calculation process, the cumulative influence strength and propagation level information of each operator identifier are read from the influence propagation result data, and the reachable paths are expanded in combination with the operator dependency data to form an influence path set. The influence path set is aggregated according to the operator identifier sequence to form an influence range candidate set. The influence range candidate set is subjected to level consistency constraint processing according to the propagation depth control parameter, strength consistency constraint processing according to the propagation truncation threshold parameter, and path consistency constraint processing according to the path merging strategy parameter, thereby forming change influence evaluation data.

[0090] Propagation source initialization and starting influence strength configuration formula:

[0091]

[0092]

[0093] : an operator set corresponding to the operator description data, : a set of change nodes, : an operator node corresponding to any operator identifier, ; : the starting cumulative influence strength of the operator node , : a change source confidence parameter, : a propagation truncation threshold parameter, : a set of 0th layer propagation frontiers.

[0094] Edge weight weighting and single-layer propagation incremental influence strength calculation formula:

[0095]

[0096]

[0097] : a set of propagation edges triggered by the propagation frontiers during the propagation of the th layer, : a directed dependency edge in the operator dependency data, indicating the propagation direction from the source operator identifier to the target operator identifier , : the base edge weight of the dependency edge, : an edge weight weighting parameter, : the propagation edge weight, : the target node obtained by the propagation calculation of the th layer The intensity of the incremental impact, After the propagation of the previous layer ends, the source node... The current cumulative impact intensity.

[0098] Cumulative impact intensity update, truncation, and multipath merging formulas:

[0099]

[0100]

[0101]

[0102] : No. The intensity of the untruncation cumulative impact after layer propagation, : No. The cumulative impact intensity after layer propagation is complete. : Path merging processing function : Path merging strategy parameters Indicator functions : Propagation truncation threshold parameter.

[0103] Propagation depth control and influence domain set output formula:

[0104] ,

[0105] : Propagation depth control parameters : No. Layered propagation frontier set, : Dependency propagation influence domain set, representing the set of operator nodes that are reachable and satisfy the threshold condition under the propagation depth constraint and intensity truncation constraint, corresponding to the core coverage domain of the influence propagation result data formed after influence propagation analysis and processing.

[0106] Formulas for forming the set of influence paths and the candidate set of influence ranges:

[0107]

[0108]

[0109]

[0110] : The set of directed edges for operator dependency data The reachable path expansion function is used to expand reachable paths by combining operator dependency data to form a set of affected paths. : Affects the set of paths a path aggregation function for performing path aggregation on the impact path set according to the sequence of operator identifiers to form an impact scope candidate set, an impact scope candidate set, a uniformization constraint function for performing hierarchical uniformization constraint, intensity uniformization constraint and path uniformization constraint on the impact scope candidate set respectively to form an impact scope calculation result and enter the change impact assessment data; an impact scope calculation result

[0111] An impact degree determination process is performed on the change impact assessment data. In the impact degree determination process, the impact scope candidate set, the impact path set and the cumulative impact intensity of each operator identifier in the change impact assessment data are read and regularized and merged to form an impact degree determination basis. The system configures corresponding release control decision types in the preset configuration for different impact degree determination results, solidifies the corresponding relationship between the impact degree determination results and the release control decision types into executable release control rules, compares the impact degree determination basis with the preset release control rules, generates a release control decision based on the release control rules, and solidifies and outputs the release control decision as release control instruction data.

[0112] As Figure 2 In this embodiment, feature unpacking processing is performed on the version fingerprint data. In the feature unpacking processing, first, the fingerprint summary value for representing the fingerprint aggregation result in the version fingerprint data is read, the fingerprint generation time identifier for representing the fingerprint generation time is read, and the version fingerprint identifier for identifying the development input range corresponding to the fingerprint is read.

[0113] Constituent content splitting processing is performed on the version fingerprint data. The version fingerprint data is split into code feature summary data, script feature summary data, dependency feature summary data, configuration feature summary data and environment feature summary data according to the aggregation structure of the version fingerprint data.

[0114] Consistency checking processing is performed on the code feature summary data, script feature summary data, dependency feature summary data, configuration feature summary data and environment feature summary data obtained by splitting. In the consistency checking processing, the summary format, summary length and summary encoding method of each type of feature summary data are determined for consistency, and abnormality marking processing is performed on the feature summary data that does not meet the consistency determination condition to form an unpacking checking mark.

[0115] The structured organization processing is performed on the code feature summary data, the script feature summary data, the dependency feature summary data, the configuration feature summary data and the environment feature summary data, in which a unified classification index relationship is established for the five types of feature summary data according to feature categories, and the five types of feature summary data are bound with the fingerprint generation time identifier according to the version fingerprint identifier, forming a traceable feature unpacking result.

[0116] After the feature unpacking is completed, the feature indicator calculation processing is performed on the feature unpacking result, in which firstly the comparability preparation processing is performed on the code feature summary data, the script feature summary data, the dependency feature summary data, the configuration feature summary data and the environment feature summary data, in which the five types of feature summary data are normalized and organized according to a unified summary representation rule to form a comparable feature sequence; then the difference measurement preparation processing is performed on the comparable feature sequence, in which the measurement benchmark for this calculation is determined according to the version fingerprint identifier and the fingerprint generation time identifier, and an alignment relationship is established between the comparable feature sequence and the measurement benchmark to form an aligned feature sequence.

[0117] The feature difference measurement processing is performed on the aligned feature sequence, in which the difference degree of the code feature summary data relative to the measurement benchmark, the difference degree of the script feature summary data relative to the measurement benchmark, the difference degree of the dependency feature summary data relative to the measurement benchmark, the difference degree of the configuration feature summary data relative to the measurement benchmark and the difference degree of the environment feature summary data relative to the measurement benchmark are calculated respectively, and the five types of difference degrees are organized to form a difference measurement set; then the indicator aggregation calculation processing is performed on the difference measurement set, in which the five types of difference degrees are weighted and aggregated according to the aggregation weight rule in the system preset configuration to generate a single aggregation indicator, and the aggregation indicator is subjected to scale normalization processing to obtain an indicator value falling within a fixed interval; the normalized indicator value is written into the version feature indicator and bound with the version fingerprint identifier and the fingerprint generation time identifier, thereby forming the version feature indicator.

[0118] The interval determination processing is performed on the version feature indicator, in which the version feature indicator is divided into four intervals in the system preset configuration, the four intervals are mutually exclusive intervals, and the four intervals are interval I, interval II, interval III and interval IV in order of low to high according to the version difference degree, and any version feature indicator is only allowed to be determined as one interval at the same time. The system sets a first boundary value, a second boundary value and a third boundary value for the version feature indicator, which are used to divide the version feature indicator into the corresponding intervals, and the interval determination condition is solidified into executable interval determination logic. The interval determination logic only takes the size comparison result between the version feature indicator and the three interval boundary values as the determination basis, and does not depend on other external conditions.

[0119] The determination relationship between the version feature index and the three interval boundary values is: when the version feature index is less than the first boundary value, it is determined to fall into interval I; when the version feature index is greater than or equal to the first boundary value and less than the second boundary value, it is determined to fall into interval II; when the version feature index is greater than or equal to the second boundary value and less than the third boundary value, it is determined to fall into interval III; and when the version feature index is greater than or equal to the third boundary value, it is determined to fall into interval IV. Through the above interval determination logic, the discretization classification of the version difference degree is realized, and a unique basis is provided for the subsequent determination of parameter values.

[0120] When the version feature index falls into interval I, the system adopts a parameter value set for a low version difference scenario, specifically: the analysis confidence threshold parameter value is 0.60; the dependency determination weight parameter value is 1.00; the semantic rule strength parameter value is 0.60; the version binding consistency threshold parameter value is 0.60; the propagation depth control parameter value is 2; the edge weight weighting parameter value is 0.80; the propagation truncation threshold parameter value is 0.20; the path merging strategy parameter value is the maximum value merging; and the change source confidence parameter value is 0.60. This parameter value set is used to reduce the analysis and propagation constraint strength and improve system processing efficiency in the case of small version difference.

[0121] When the version feature index falls into interval II, the system adopts a parameter value set for a medium-low version difference scenario, specifically: the analysis confidence threshold parameter value is 0.70; the dependency determination weight parameter value is 1.20; the semantic rule strength parameter value is 0.70; the version binding consistency threshold parameter value is 0.70; the propagation depth control parameter value is 3; the edge weight weighting parameter value is 1.00; the propagation truncation threshold parameter value is 0.30; the path merging strategy parameter value is weighted merging; and the change source confidence parameter value is 0.70. This parameter value set is used to moderately enhance the dependency relationship determination and influence propagation control ability while ensuring analysis accuracy.

[0122] When the version feature index falls into interval III, the system adopts a parameter value set for a medium-high version difference scenario, specifically: the analysis confidence threshold parameter value is 0.80; the dependency determination weight parameter value is 1.50; the semantic rule strength parameter value is 0.80; the version binding consistency threshold parameter value is 0.80; the propagation depth control parameter value is 4; the edge weight weighting parameter value is 1.20; the propagation truncation threshold parameter value is 0.40; the path merging strategy parameter value is average value merging; and the change source confidence parameter value is 0.80. This parameter value set is used to strengthen semantic constraints, dependency determination and propagation control in the case of significant increase in version difference, and reduce the risk of misjudgment.

[0123] When the version feature index falls into interval IV, the system adopts a parameter value set for the high version difference scenario, specifically: the resolution confidence threshold parameter value is 0.90; the dependency determination weight parameter value is 1.80; the semantic rule strength parameter value is 0.90; the version binding consistency threshold parameter value is 0.90; the propagation depth control parameter value is 5; the edge weight weighting parameter value is 1.50; the propagation truncation threshold parameter value is 0.50; the path merging strategy parameter value is priority merging; and the change source confidence parameter value is 0.90. The parameter value set is used to significantly improve the strictness of resolution and propagation determination in the case of great version difference, to guarantee system stability and release safety, and to determine the version feature index as one of interval I, interval II, interval III or interval IV to obtain the interval determination result.

[0124] Based on the preset parameter mapping rule, the interval determination result is taken as the only input basis, the parameter mapping rule is used to convert the interval determination result into a parameter value set, the interval determination result is read according to the corresponding interval to determine the parameter values of the resolution confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter, the version binding consistency threshold parameter, the propagation depth control parameter, the edge weight weighting parameter, the propagation truncation threshold parameter, the path merging strategy parameter and the change source confidence parameter, and the parameter value set is formed, the parameter value set is configured for each interval respectively, the parameter value set is uniquely determined under the same interval, does not depend on the runtime state information, and does not depend on manual intervention. In the system preset configuration, the corresponding relationship between each interval and the corresponding parameter value set is solidified as an executable mapping rule, which is used to convert the interval determination result corresponding to the version feature index into a parameter value for subsequent processing.

[0125] Based on the mapping rule, the parameter value set is written into the mapping result to obtain the mapping result of the version fingerprint data called by the job semantic modeling module and the change impact analysis module. The mapping result of the job semantic modeling module is one, and the mapping result of the change impact analysis module is two. The job semantic modeling module obtains the analysis confidence threshold parameter, the dependency judgment weight parameter, the semantic rule strength parameter, and the version binding consistency threshold parameter based on the mapping result. The change impact analysis module obtains the propagation depth control parameter, the edge weight weighting parameter, the propagation truncation threshold parameter, the path merging strategy parameter, and the change source confidence parameter based on the mapping result. The analysis confidence threshold parameter is used to control the acceptance boundary of the candidate analysis result in the execution step analysis process. The higher the threshold value, the more stringent the analysis confidence requirement. The dependency judgment weight parameter is used to weight the multiple dependency clues in the operator dependency modeling. The semantic rule strength parameter is used to adjust the strictness of the operator semantic rule verification and data contract consistency verification. The higher the strength, the more stringent the constraint. The version binding consistency threshold parameter is used to determine the binding relationship consistency verification boundary. The higher the threshold value, the more stringent the binding consistency requirement. The propagation depth control parameter is used to limit the maximum level or maximum path length of the impact propagation. The edge weight weighting parameter is used to weight the dependency edge corresponding to the operator dependency relationship data in the job semantic data. The propagation truncation threshold parameter is used to threshold the cumulative impact strength in the impact propagation process. The value is obtained from the system preset configuration or the truncation threshold configuration value obtained from the version fingerprint data mapping. The path merging strategy parameter is used to control the merging rule of the multi-path propagation result. The change source confidence parameter is used to weight the starting impact strength of different change nodes in the change node set identified by the version fingerprint data.

[0126] In the subsequent processing process, the job semantic modeling module obtains the corresponding analysis confidence threshold parameter, dependency judgment weight parameter, semantic rule strength parameter, and version binding consistency threshold parameter based on the mapping result of the version fingerprint data. After processing, the job semantic data is formed. The change impact analysis module obtains the corresponding propagation depth control parameter, edge weight weighting parameter, propagation truncation threshold parameter, path merging strategy parameter, and change source confidence parameter based on the mapping result of the version fingerprint data. After processing, the release control instruction data is obtained.

[0127] In the embodiment, after receiving the job semantic data, the change impact evaluation data and the historical running record data, the big data processing module performs distributed statistical analysis processing on the historical running record data. The historical running record data refers to a set of original running records formed in the previous construction and deployment process of the system, which contains job identification, time information, resource usage record and execution state information, and is used to support distributed statistical analysis of job historical execution behavior. In the distributed statistical analysis processing, the record entries are read by partition according to the record granularity of the historical running record data, and de-duplication reading and missing reading checking are performed. The record entries passing the checking are aggregated and organized in time sequence and job identification dimension to form a statistical serial organization result. Frequency statistics are performed on the serial organization result to extract job historical execution frequency features. Resource dimension statistics are performed on the serial organization result to extract resource usage distribution features. Time length dimension statistics are performed on the serial organization result to extract execution time length statistical features. The job historical execution frequency features, the resource usage distribution features and the execution time length statistical features are merged and solidified to form historical running statistical feature data.

[0128] The job semantic data is subjected to job execution feature aggregation processing. In the job execution feature aggregation processing, job abstract data representing job structure is read from the job semantic data, operator description data is statistically traversed according to operator identification to extract operator quantity features, operator dependency relationship data in the job abstract data is read, dependency relationship is hierarchically expanded and counted according to topology level to extract dependency level features, executable topology structure formed by the operator dependency relationship data is read, and structured statistics are performed on path branching, path convergence and path length in the topology structure to extract execution path structure features. The operator quantity features, the dependency level features and the execution path structure features are merged and solidified to form job execution feature data.

[0129] The change impact evaluation data is subjected to correlation feature analysis processing. In the correlation feature analysis processing, impact range calculation results are read from the change impact evaluation data, range aggregation is performed on the impact range calculation results to extract impact range features, impact propagation and path organization results in the change impact evaluation data are subjected to path merging to extract impact path features, impact degree determination results in the change impact evaluation data are subjected to hierarchical merging to extract impact degree features, and the impact range features, the impact path features and the impact degree features are merged and solidified to form change impact feature data.

[0130] The historical running statistical feature data, the job execution feature data and the change influence feature data are merged; in the merging process, the three types of feature data are aligned and organized according to the job identifier and the version fingerprint identifier, and the repeated mapping is eliminated; the consistent coding and the structured packaging are performed on the aligned feature items, and the unified feature carrier is formed; thus, the job running feature data is formed, and the job running feature data is output.

[0131] In the embodiment, after the adaptive resource concurrency control module receives the job running feature data and the release control instruction data, the resource load index calculation processing is performed on the job running feature data, in the resource load index calculation processing, the historical running statistical feature data, the job execution feature data and the change influence feature data are read from the job running feature data, the three types of feature data are consistently organized, and the index calculation input is formed; the resource related feature extraction is performed on the index calculation input, and the load related statistical quantity representing the resource use distribution feature and the execution time length statistical feature is obtained; the resource load index data is generated by aggregating and calculating the load related statistical quantity according to the preset index calculation rule; at the same time, the range constraint processing is performed on the resource load index data, so that the resource load index data is ensured to be in the effective interval that can be used for control decision, and the result after the range constraint is solidified and output as the resource load index data.

[0132] In the embodiment, after the adaptive resource concurrency control module receives the job running feature data and the release control instruction data, the resource load index calculation processing is performed on the job running feature data, in the resource load index calculation processing, the historical running statistical feature data, the job execution feature data and the change influence feature data are read from the job running feature data, the three types of feature data are consistently organized, and the index calculation input is formed; the resource related feature extraction is performed on the index calculation input, and the load related statistical quantity representing the resource use distribution feature and the execution time length statistical feature is obtained; the resource load index data is generated by aggregating and calculating the load related statistical quantity according to the preset index calculation rule; at the same time, the range constraint processing is performed on the resource load index data, so that the resource load index data is ensured to be in the effective interval that can be used for control decision, and the result after the range constraint is solidified and output as the resource load index data.

[0133] After the resource load indicator data and the concurrent pressure indicator data are generated, the adaptive resource concurrent control module performs adaptive adjustment processing on the computing resource allocation parameter and the job concurrency degree parameter according to the resource load indicator data and the concurrent pressure indicator data under the condition that the release control instruction data allows execution; in the adaptive adjustment processing, first, the release control instruction data is read, and execution permission determination is performed to determine whether the trigger condition for performing adaptive adjustment processing is met; when the execution permission determination result meets the execution condition, the resource load indicator data and the concurrent pressure indicator data are jointly evaluated to form adjustment basis; based on the adjustment basis, the computing resource allocation parameter is subjected to increment, decrement or retention processing to form updated computing resource allocation parameter; at the same time, the job concurrency degree parameter is subjected to increase, decrease or retention processing to form updated job concurrency degree parameter; the updated computing resource allocation parameter and the updated job concurrency degree parameter are consistent encapsulation to form operation control strategy data.

[0134] When the execution permission determination result does not meet the execution condition, the original values of the computing resource allocation parameter and the job concurrency degree parameter are kept, and the original values are solidified as operation control strategy data; the operation control strategy data is output to the software product output module for controlling the execution mode of the build process and the deployment process.

[0135] After the software product output module receives the version fingerprint data, the release control instruction data and the operation control strategy data, the build process for building the standardized development input data set into a software product is controlled according to the release control instruction data under the condition that the release control instruction data allows execution; in the control processing, the release control instruction data is read, and the allowed release state and control decision information in the release control instruction data are parsed to form build trigger determination basis; based on the build trigger determination basis, execution permission determination is performed on the build process to determine whether the build process is in an allowed execution state; when the build process is in the allowed execution state, execution arrangement of the build process is started, build tasks are organized according to the execution order defined in the build process, and a build task sequence is generated; build task scheduling is performed on the build task sequence in sequence to form build task scheduling result; build task running is performed on the build task scheduling result to form build running process record; build result aggregation is performed on the build running process record to obtain build execution result data; when the build process is in the unallowed execution state, the build process is terminated and the termination result is solidified to form build execution result data.

[0136] After obtaining the build execution result data, the software product output module performs deployment processing on the build execution result data according to the running control strategy data; in the deployment processing, the running control strategy data is read, the computing resource allocation parameter and the job concurrency parameter in the running control strategy data are parsed, and deployment execution constraints are formed; based on the deployment execution constraints, pre-deployment checking is performed on the build execution result data, and it is determined that the deployment input is in a deployable state; deployment task organization is performed on the build execution result data in the deployable state, and a deployment task sequence is generated; resource binding processing is performed on the deployment task sequence, so that the deployment task corresponds to the computing resource allocation parameter; concurrency control processing is performed on the deployment task sequence, so that the deployment task corresponds to the job concurrency parameter; deployment task scheduling is performed on the deployment task sequence after resource binding and concurrency control, and deployment scheduling results are formed; deployment task running is performed on the deployment scheduling results, and deployment running process records are formed; deployment result aggregation is performed on the deployment running process records, and deployment completed software product data is formed; at the same time, version fingerprint data is written into software product meta information, and the version fingerprint data is associated and solidified with the deployment completed software product data, to form the deployment completed software product data.

[0137] After forming the deployment completed software product data, the software product output module performs result solidification processing on the deployment completed software product data; in the result solidification processing, the deployment completed software product data is read, product integrity checking is performed, and it is determined that the product content is in a solidifiable state; product packaging processing is performed on the deployment completed software product data in the solidifiable state, and product packaging results are formed; output organization processing is performed on the product packaging results, and outputtable product results are formed; the outputtable product results are solidified and output, and the developed software product is obtained, and the developed software product is output.

[0138] Embodiment one: in this embodiment, the development input collection module receives source code data, build script data, dependency declaration data, configuration file data and development environment description data, performs integrity checking processing on the source code data and generates checked source code data, performs structure parsing processing on the build script data and generates parsed build script data, performs dependency item parsing processing on the dependency declaration data and generates dependency parsing data, performs parameter extraction processing on the configuration file data and generates configuration parameter data, performs environment element parsing processing on the development environment description data and generates environment element data, and performs merging processing on the checked source code data, parsed build script data, dependency parsing data, configuration parameter data and environment element data, and forms a standardized development input data set.

[0139] In this embodiment, the version fingerprint construction module receives the checked source code data, the parsed build script data, the dependency parsing data, the configuration parameter data and the environment element data, performs code feature digest calculation processing on the source code data to form code feature digest data, performs script feature digest calculation processing on the parsed build script data to form script feature digest data, performs dependency feature digest calculation processing on the dependency parsing data to form dependency feature digest data, performs configuration feature digest calculation processing on the configuration parameter data to form configuration feature digest data, performs environment feature digest calculation processing on the environment element data to form environment feature digest data, and performs aggregation calculation processing on the code feature digest data, the script feature digest data, the dependency feature digest data, the configuration feature digest data and the environment feature digest data to form version fingerprint data.

[0140] In this embodiment, the job semantic modeling module receives the version fingerprint data and performs feature unpacking and feature index calculation to form version feature indexes, matches the version feature indexes with preset parameter mapping rules one by one to form mapping results, the system determines that the version feature indexes fall into interval I, and determines that the analysis confidence threshold parameter value is 0.60, the dependency determination weight parameter value is 1.00, the semantic rule strength parameter value is 0.60, and the version binding consistency threshold parameter value is 0.60 from the parameter value set corresponding to interval I; the job semantic modeling module performs step parsing processing on the parsed build script data and completes step retention determination based on the analysis confidence threshold parameter to form execution step description data, performs operator abstraction processing on the execution step description data to form operator description data, performs directed relationship modeling processing on the operator description data and weights the dependency clues according to the dependency determination weight parameter to form operator dependency relationship data, performs unified modeling processing on the data input description, the data output description and the operator dependency relationship data to form job abstraction data, performs operator semantic rule verification processing on the job abstraction data and forms operator semantic verification result data according to the semantic rule strength parameter, performs data contract consistency verification processing on the job abstraction data and forms data contract verification result data according to the semantic rule strength parameter, performs binding relationship consistency verification processing on the job abstraction data in combination with the version fingerprint data and forms version binding verification result data according to the version binding consistency threshold parameter, performs summary processing on the three types of verification results to form consistency verification result data, and merges and solidifies the consistency verification result data and the job abstraction data to form job semantic data.

[0141] In this embodiment, the change impact analysis module receives the job semantic data and the version fingerprint data, performs feature unpacking and feature index calculation, and forms a mapping result of the version fingerprint data. The system determines that the version feature index falls into interval I, and determines from the parameter value set corresponding to interval I that the propagation depth control parameter value is 2, the edge weight weighting parameter value is 0.80, the propagation cut-off threshold parameter value is 0.20, the path merging strategy parameter value is the maximum value merging, and the change source confidence parameter value is 0.60. The change impact analysis module identifies the change node set based on the version fingerprint data, configures the initial impact strength according to the change source confidence parameter, then propagates layer by layer in the direction of the directed edge of the operator dependency data according to the propagation depth control parameter, calculates the propagation edge weight according to the edge weight weighting parameter, and generates the impact propagation result data. The reachable paths are unfolded in combination with the impact propagation result data, and the impact path set is formed. The impact path set is subjected to path aggregation, and the impact range candidate set is formed. The change impact evaluation data is formed according to the propagation cut-off threshold parameter and the path merging strategy parameter. Finally, the impact degree determination processing is performed on the change impact evaluation data, and is compared with the preset release control rule to generate the release control decision and solidify the output as the release control instruction data.

[0142] In this embodiment, the big data processing module receives the job semantic data, the change impact evaluation data and the historical running record data, forms the historical running statistical feature data, the job execution feature data and the change impact feature data, and outputs the job running feature data. The adaptive resource concurrency control module receives the job running feature data and the release control instruction data, and outputs the running control strategy data under the condition that the release control instruction data allows execution. The software artifact output module receives the version fingerprint data, the release control instruction data and the running control strategy data, and completes the build process control, deployment processing and result solidification under the condition that the release is allowed, and outputs the developed software artifact.

[0143] In this embodiment, the version feature index formed by the change impact analysis module is determined to fall into interval II, and the system determines the propagation depth control parameter value to be 3, the edge weight weighting parameter value to be 1.00, the propagation cut-off threshold parameter value to be 0.30, the path merging strategy parameter value to be weighted merging, and the change source confidence parameter value to be 0.70 from the parameter value set corresponding to interval II. The change impact analysis module takes the change node set as the propagation source, configures a higher initial impact strength according to a higher change source confidence parameter, allows deeper level propagation according to a higher propagation depth control parameter, eliminates low-intensity propagation paths according to a higher propagation cut-off threshold parameter, merges the multi-path arrival results of the same target node according to the weighted merging strategy to form a single target node propagation state, and further forms change impact evaluation data and solidifies output release control instruction data.

[0144] In this embodiment, the version feature index formed by the change impact analysis module is determined to fall into interval II, and the system determines the propagation depth control parameter value to be 3, the edge weight weighting parameter value to be 1.00, the propagation cut-off threshold parameter value to be 0.30, the path merging strategy parameter value to be weighted merging, and the change source confidence parameter value to be 0.70 from the parameter value set corresponding to interval II. The change impact analysis module takes the change node set as the propagation source, configures a higher initial impact strength according to a higher change source confidence parameter, allows deeper level propagation according to a higher propagation depth control parameter, eliminates low-intensity propagation paths according to a higher propagation cut-off threshold parameter, merges the multi-path arrival results of the same target node according to the weighted merging strategy to form a single target node propagation state, and further forms change impact evaluation data and solidifies output release control instruction data.

[0145] In this embodiment, the input and output links of the big data processing module, the adaptive resource concurrency control module, and the software product output module are consistent with those in Embodiment One. The system outputs running control strategy data under the condition that the release control instruction data is allowed to be executed, and performs adaptive adjustment on the computing resource allocation parameters and the job concurrency degree parameters in the deployment stage according to the running control strategy data, so as to balance the release safety and resource utilization efficiency under the condition of medium-low difference version change.

[0146] In this embodiment, the version feature index determined by the change impact analysis module falls into interval III, and the system determines the propagation depth control parameter value as 4, the edge weight weighting parameter value as 1.20, the propagation cut-off threshold parameter value as 0.40, the path merging strategy parameter value as average value merging, and the change source confidence parameter value as 0.80 from the parameter value set corresponding to interval III. In the impact propagation analysis process, the system allows a deeper propagation level and performs stronger edge weight weighting on the propagation edge, adopts a higher propagation cut-off threshold for the cumulative impact intensity to eliminate weak propagation paths, and generates a single propagation state by using the average value merging strategy when multiple paths reach the same target node, so that the impact domain and the impact path result are more biased towards robustness, thereby forming more stringent change impact evaluation data and generating release control instruction data.

[0147] In this embodiment, the version feature index determined by the change impact analysis module falls into interval III, and the system determines the propagation depth control parameter value as 4, the edge weight weighting parameter value as 1.20, the propagation cut-off threshold parameter value as 0.40, the path merging strategy parameter value as average value merging, and the change source confidence parameter value as 0.80 from the parameter value set corresponding to interval III. In the impact propagation analysis process, the system allows a deeper propagation level and performs stronger edge weight weighting on the propagation edge, adopts a higher propagation cut-off threshold for the cumulative impact intensity to eliminate weak propagation paths, and generates a single propagation state by using the average value merging strategy when multiple paths reach the same target node, so that the impact domain and the impact path result are more biased towards robustness, thereby forming more stringent change impact evaluation data and generating release control instruction data.

[0148] In this embodiment, after the big data processing module forms the job running feature data, the self-adaptive resource concurrency control module performs joint evaluation on the resource load index data and the concurrency pressure index data under the condition that the release control instruction data allows execution, and outputs running control strategy data. The software artifact output module performs resource binding and concurrency control on the deployment task sequence based on the running control strategy data and completes deployment solidification, thereby improving release stability and reducing resource congestion risk in the deployment phase under the condition of medium-high difference version change.

[0149] In this embodiment, the input and output link of the input collection module and the version fingerprint construction module is developed consistent with embodiment one; the job semantic modeling module forms version feature indicators from version fingerprint data and determines that they fall into interval IV, the system determines from the parameter value set corresponding to interval IV that the resolution confidence threshold parameter value is 0.90, the dependency determination weight parameter value is 1.80, the semantic rule strength parameter value is 0.90, and the version binding consistency threshold parameter value is 0.90; the job semantic modeling module performs step retention determination and dependency relationship modeling with the highest strictness, and adopts the determination boundary of the highest strictness in operator semantic rule checking, data contract consistency checking and binding relationship consistency verification, and forms consistency checking result data after performing consistency merging and conflict resolution on the checking results, and solidifies and outputs job semantic data, thereby maximizing the reduction of job semantic modeling bias under high-difference version conditions.

[0150] In this embodiment, the change impact analysis module determines that the version feature indicators fall into interval IV, and the system determines from the parameter value set corresponding to interval IV that the propagation depth control parameter value is 5, the edge weight weighting parameter value is 1.50, the propagation cutoff threshold parameter value is 0.50, the path merging strategy parameter value is priority merging, and the change source confidence parameter value is 0.90; the change impact analysis module configures a higher initial impact strength based on a higher change source confidence parameter, allows a deeper propagation level and applies a higher cutoff threshold to the propagation path to quickly eliminate low-intensity propagation paths, while adopting a priority merging strategy to merge the propagation results reached by multiple paths, so that the impact domain result is more biased towards high-risk priority coverage, and accordingly forms change impact evaluation data and release control instruction data.

[0151] In this embodiment, after the adaptive resource concurrency control module receives the job running feature data and the release control instruction data, it preferentially performs the permissibility determination of the release control instruction data; when the permissibility determination result does not satisfy the execution condition, the original values of the computing resource allocation parameter and the job concurrency degree parameter are maintained and solidified as running control strategy data; after the software artifact output module receives the release control instruction data, if the construction process is in an unallowable execution state, the construction process is terminated and the termination result is solidified to form construction execution result data, thereby realizing risk isolation and release safety control with release control decision as the priority constraint under high-difference version change conditions.

[0152] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalent features, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A software development cloud computing system with a big data processing module, characterized in that: include: Develop an input acquisition module to receive source code data, build script data, dependency declaration data, configuration file data, and development environment description data. Perform integrity verification, structure parsing, dependency parsing, parameter extraction, and environment element parsing processing respectively to form a standardized development input dataset. The version fingerprint construction module is used to receive standardized development input datasets, calculate and aggregate feature summaries for source code data, build script data, dependency resolution data, configuration parameter data and environmental element data respectively, and form version fingerprint data. The job semantic modeling module receives standardized development input datasets and version fingerprint data. Based on the version fingerprint data, it performs feature unpacking and version feature index calculation, and performs interval judgment processing on the version feature index, determining the version feature index into one of four preset intervals. Based on the interval judgment results, it determines the parsing confidence threshold parameter, dependency judgment weight parameter, semantic rule strength parameter, and version binding consistency threshold parameter. Then, it adopts a static parsing and abstraction algorithm for the construction script data to parse the execution steps, abstract operators, and model dependencies. Combining the data input description and data output description, it forms job abstract data. Finally, it performs operator semantic rule verification, data contract consistency verification, and binding relationship consistency verification on the job abstract data to form job semantic data. The change impact analysis module receives job semantic data and version fingerprint data, performs feature unpacking and version feature index calculation based on the version fingerprint data, performs interval determination processing on the version feature index, determines the version feature index into one of four preset intervals, and determines the propagation depth control parameters, edge weighting parameters, propagation truncation threshold parameters, path merging strategy parameters, and change source confidence parameters based on the interval determination results. Then, it adopts the dependency propagation impact domain calculation algorithm to identify change nodes and perform impact propagation and impact range calculation based on operator dependency relationships to form change impact assessment data, and generates release control instruction data according to preset release control rules. The big data processing module is used to receive job semantic data, change impact assessment data and historical operation record data, perform distributed statistical analysis and feature aggregation processing, and form job operation feature data. The adaptive resource concurrency control module is used to receive job operation characteristic data and issue control command data, calculate resource load indicators and concurrency pressure indicators, and generate operation control strategy data under the condition that the issued control command allows. The software artifact output module receives version fingerprint data, release control command data, and operation control strategy data. Based on the release control command, it controls and deploys the build process used to construct software artifacts from standardized development input datasets, writes version fingerprint data into software artifact metadata, and obtains the completed software artifact.

2. The software development cloud computing system with a big data processing module according to claim 1, characterized in that: The development input acquisition module receives source code data, build script data, dependency declaration data, configuration file data, and development environment description data. It performs integrity verification on the source code data and generates verified source code data. Perform structure parsing processing on the build script data to generate parsed build script data; Dependency resolution processing is performed on the dependency declaration data to generate dependency resolution data; Perform parameter extraction processing on the configuration file data to generate configuration parameter data; Perform environmental element parsing processing on the development environment description data to generate environmental element data; The verified source code data, parsed build script data, dependency resolution data, configuration parameter data, and environment element data are merged to form a standardized development input dataset.

3. A software development cloud computing system with a big data processing module according to claim 1, characterized in that: The version fingerprint building module performs code feature digest calculation on the source code data to obtain code feature digest data; The parsed construction script data is processed to calculate script feature summaries, resulting in script feature summary data. Dependency feature summary data is obtained by performing dependency feature summary calculation on the dependency parsing data; The configuration parameter data is processed by configuration feature summary calculation to obtain configuration feature summary data; Environmental feature summary data is obtained by performing environmental feature summary calculation on environmental element data. The code feature summary data, script feature summary data, dependency feature summary data, configuration feature summary data, and environment feature summary data are aggregated and processed to form version fingerprint data.

4. A software development cloud computing system with a big data processing module according to claim 1, characterized in that: The job semantic modeling module performs feature unpacking and feature index calculation based on version fingerprint data to obtain version feature indexes. It then compares the version feature indexes with the preset interval judgment logic and determines the interval in which the version feature indexes fall based on the size relationship between the version feature indexes and the three interval boundary values. Based on the interval, the version feature indicators are matched with the preset parameter mapping rules to determine the parameter values ​​corresponding to the current version features, forming a mapping result including the parsing confidence threshold parameter, the dependency judgment weight parameter, the semantic rule strength parameter, and the version binding consistency threshold parameter; The parsed build script data is processed to generate a set of candidate execution steps. The parsing confidence score is calculated for each candidate execution step. The parsing confidence score is compared with the parsing confidence threshold parameter. Candidate execution steps that meet the threshold condition are retained and candidate execution steps that do not meet the threshold condition are removed to obtain the execution step description data. The execution step description data is processed by operator abstraction, each execution step is mapped to a corresponding operator semantic unit, and operator identifiers and operator attribute descriptions are generated to obtain operator description data; The execution dependencies between operators in the operator description data are modeled by directed relation modeling. For each pair of candidate dependencies, dependency clues are extracted and weighted according to the dependency judgment weight parameters. Based on the weighted result, the existence and direction of the dependency edges are determined to obtain the operator dependency data. The data input descriptions, data output descriptions, and operator dependency data obtained during the execution step parsing process are uniformly modeled and processed. The data input descriptions and data output descriptions are bound to the corresponding operator identifiers, and the operator dependency data is organized into an executable topology to form job abstract data. The abstract data of the task is processed by operator semantic rule verification. The semantic rule set is selected according to the semantic rule strength parameter and the rule trigger boundary is determined. Semantic consistency judgment and constraint satisfaction judgment are performed on each operator to obtain operator semantic verification result data. Data contract consistency verification is performed on the data input and output descriptions in the job abstract data. The contract verification boundary is determined based on the semantic rule strength parameter. Consistency judgment is performed on the data structure constraints, field constraints and dependency constraints of the input and output data to obtain the data contract verification result data. By combining version fingerprint data, the binding relationship consistency verification process of the job abstract data is performed. The target binding feature is determined based on the version fingerprint data, and the consistency judgment boundary is determined according to the version binding consistency threshold parameter. Consistency matching and deviation judgment are performed on the binding features of the job abstract data and the target binding features to obtain the version binding verification result data. The operator semantic verification result data, data contract verification result data and version binding verification result data are aggregated and processed. The aggregation decision boundary is determined based on the semantic rule strength parameter and the version binding consistency threshold parameter to form consistency verification result data. The abstract data of the job and the consistency verification result data are merged to form the semantic data of the job.

5. A software development cloud computing system with a big data processing module according to claim 1, characterized in that: The change impact analysis module performs feature unpacking and feature index calculation based on version fingerprint data to obtain version feature indexes. It then compares the version feature indexes with the preset interval judgment logic and determines the interval in which the version feature indexes fall based on the relationship between the version feature indexes and the boundary values ​​of the three intervals. The change impact analysis module matches version feature indicators with preset parameter mapping rules based on the interval determination results, determines the parameter values ​​corresponding to the current version features, and aggregates the parameter values ​​to form the mapping results of version fingerprint data. The mapping results include propagation depth control parameters, edge weighting parameters, propagation truncation threshold parameters, path merging strategy parameters, and change source confidence parameters. The semantic data of the job is processed by operator node traversal analysis to form node analysis result data. The set of changed nodes is identified based on version fingerprint data, and the initial influence intensity is configured for each changed node in the set of changed nodes according to the change source confidence parameter. The node analysis results data are processed by operator dependency relationship propagation analysis. The propagation level is limited according to the propagation depth control parameter, the propagation edges are weighted according to the edge weighting parameter, the cumulative impact intensity is truncated according to the propagation truncation threshold parameter, and the multi-path propagation results are merged according to the path merging strategy parameter to form the impact propagation result data. By combining version fingerprint data, the impact propagation result data is processed to calculate the impact range of the execution path. Based on the propagation depth control parameters, propagation truncation threshold parameters, and path merging strategy parameters, the impact range calculation results are subjected to consistency constraint processing to form change impact assessment data. The impact assessment data of the change is processed to determine the degree of impact, and the degree of impact is compared with the preset release control rules. Based on the comparison results, a release control decision is generated, and release control instruction data is obtained.

6. A software development cloud computing system with a big data processing module according to claim 1, characterized in that: The big data processing module performs distributed statistical analysis on historical operation record data, extracts historical execution frequency characteristics, resource usage distribution characteristics, and execution duration statistical characteristics of the job, and forms historical operation statistical characteristic data. The job semantic data is processed by job execution feature aggregation, and the operator quantity feature, dependency hierarchy feature and execution path structure feature are extracted to form job execution feature data. The impact assessment data of the change is processed by correlation feature analysis to extract the impact scope features, impact path features and impact degree features, forming change impact feature data; Historical operational statistical data, job execution data, and change impact data are merged and processed to form job operation characteristic data, which is then output.

7. A software development cloud computing system with a big data processing module according to claim 1, characterized in that: The adaptive resource concurrency control module calculates and processes resource load indicators from job operation characteristic data to generate resource load indicator data. Perform concurrent stress index calculation on the operation characteristic data of the job to generate concurrent stress index data; Under the condition that the control command data allows execution, the computing resource allocation parameters and job concurrency parameters are adaptively adjusted based on the resource load index data and concurrency pressure index data to form the operation control strategy data; The runtime control strategy data is output to the software artifact output module to control the execution of the build and deployment processes.

8. A software development cloud computing system with a big data processing module according to claim 1, characterized in that: Under the condition that the release control command data allows execution, the software artifact output module controls the build process used to construct the standardized development input dataset into a software artifact according to the release control command data, and obtains the build execution result data. Based on the operation control strategy data, the build execution result data is deployed and processed, and the version fingerprint data is written into the software artifact metadata to form the deployed software artifact data; The deployed software artifact data is processed to solidify the results, resulting in the completed software artifact, which is then output.

Images