Software development cloud computing system with big data processing module

Abstract

The invention discloses a software development cloud computing system with a big data processing module, which relates to the technical field of computer systems and data processing, and comprises a development input acquisition module used for receiving source code data, construction script data, dependency declaration data, configuration file data and development environment description data, and respectively executing integrity verification, structure analysis, dependency analysis, parameter extraction and environmental element analysis processing to form a standardized development input data set. According to the method, unified feature abstracting and aggregation are carried out on source codes, construction scripts, dependency declarations, configuration parameters and development environment elements, version fingerprint data capable of stably representing the software version state is formed, the strict degree of analysis, verification and influence analysis is remarkably enhanced in a high-difference version scene, and the software version state can be stably represented. Therefore, the problems of misjudgment, missed judgment or resource waste caused by adoption of a fixed threshold value or manual parameter configuration in the prior art are avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer system and data processing technology, and particularly relates to a software development cloud computing system with a big data processing module. BACKGROUND

[0002] With the rapid development of cloud computing technology and big data technology, software development activities gradually evolve from traditional local development mode to centralized and distributed development mode based on cloud platform. In the existing software development cloud computing environment, the development process usually involves source code management, build script configuration, dependency declaration management, configuration file maintenance and development environment configuration and other types of heterogeneous data. These data jointly determine the build result and running behavior of software products.

[0003] The existing software development cloud computing system generally has the problems of scattered development input data management, lack of unified standardized processing, version identification mainly focusing on code level and being difficult to cover multi-dimensional elements such as build script, dependency relationship, configuration parameter and development environment, build process analysis relying on dynamic execution and lacking static modeling capability for execution steps, dependency relationship and data input / output relationship, difficulty in timely and accurate evaluation of the influence range of changes in the overall build process, and limitation of the use of historical running record data to monitoring and troubleshooting, fixed strategy being used for resource scheduling and concurrent control, and difficulty in adaptive adjustment according to job characteristics and change risk, thereby leading to insufficient version traceability, high release risk, and limited resource utilization efficiency and system stability. SUMMARY

[0004] The present application relates to the field of computer system and data processing technology, and particularly relates to a software development cloud computing system with a big data processing module.

[0005] In order to achieve the above object, the application adopts the following technical scheme: A software development cloud computing system with a big data processing module, comprising: a development input collection module for receiving source code data, build script data, dependency declaration data, configuration file data and development environment description data, performing integrity check, structure analysis, dependency analysis, parameter extraction and environment element analysis processing respectively, and forming a standardized development input data set; a version fingerprint construction module for receiving the standardized development input data set, performing feature abstract calculation and aggregation on the source code data, build script data, dependency analysis data, configuration parameter data and environment element data respectively, and forming version fingerprint data; a job semantic modeling module for receiving the standardized development input data set and the version fingerprint data, performing feature unpacking and version feature index calculation based on the version fingerprint data, performing interval determination processing on the version feature index, determining the version feature index as one of four preset intervals, determining the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter and the version binding consistency threshold parameter based on the interval determination result, and then using a build script static analysis abstraction algorithm to perform step analysis, operator abstraction and dependency relationship modeling on the build script data, combining data input description and data output description to form job abstraction data, and performing operator semantic rule check, data contract consistency check and binding relationship consistency verification on the job abstraction data to form job semantic data; a change impact analysis module for receiving the job semantic data and the version fingerprint data, performing feature unpacking and version feature index calculation based on the version fingerprint data, performing interval determination processing on the version feature index, determining the version feature index as one of four preset intervals, determining the propagation depth control parameter, the edge weight weighting parameter, the propagation cutoff threshold parameter, the path merging strategy parameter and the change source confidence parameter based on the interval determination result, and then using a dependency propagation impact domain calculation algorithm to identify change nodes and perform impact propagation and impact range calculation based on operator dependency relationship, forming change impact evaluation data, and generating release control instruction data according to a preset release control rule; a big data processing module for receiving the job semantic data, the change impact evaluation data and the historical running record data, performing distributed statistical analysis and feature aggregation processing to form job running feature data; an adaptive resource concurrent control module for receiving the job running feature data and the release control instruction data, calculating resource load index and concurrent pressure index, and generating running control strategy data under the condition allowed by the release control instruction; a software product output module for receiving the version fingerprint data, the release control instruction data and the running control strategy data, controlling and deploying the build process for constructing the standardized development input data set into a software product according to the release control instruction, and writing the version fingerprint data into the software product meta information to obtain a developed software product.

[0006] As a further description of the above technical scheme: The development input collection module receives source code data, build script data, dependency declaration data, configuration file data and development environment description data, performs integrity check processing on the source code data to generate checked source code data, performs structure analysis processing on the build script data to generate analyzed build script data, performs dependency item analysis processing on the dependency declaration data to generate dependency analysis data, performs parameter extraction processing on the configuration file data to generate configuration parameter data, performs environment element analysis processing on the development environment description data to generate environment element data, and performs merging processing on the checked source code data, the analyzed build script data, the dependency analysis data, the configuration parameter data and the environment element data to form a standardized development input data set.

[0007] As a further description of the above technical solution: The version fingerprint construction module performs code feature abstract calculation processing on the source code data to obtain code feature abstract data, performs script feature abstract calculation processing on the analyzed build script data to obtain script feature abstract data, performs dependency feature abstract calculation processing on the dependency analysis data to obtain dependency feature abstract data, performs configuration feature abstract calculation processing on the configuration parameter data to obtain configuration feature abstract data, performs environment feature abstract calculation processing on the environment element data to obtain environment feature abstract data, and performs aggregation calculation processing on the code feature abstract data, the script feature abstract data, the dependency feature abstract data, the configuration feature abstract data and the environment feature abstract data to form version fingerprint data.

[0008] As a further description of the above technical solution: The job semantic modeling module performs feature unpacking and feature index calculation based on the version fingerprint data to obtain version feature indexes, compares the version feature indexes with preset interval determination logic, determines the interval in which the version feature indexes fall according to the size relationship between the version feature indexes and three interval boundary values, matches the version feature indexes with a preset parameter mapping rule based on the interval to determine the parameter values corresponding to the current version feature, and forms a mapping result including the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter, and the version binding consistency threshold parameter. The parsed construction script data is subjected to execution step analysis processing to generate a candidate execution step set, and the analysis confidence of each candidate execution step is calculated. The analysis confidence is compared with the analysis confidence threshold parameter, and the candidate execution steps that meet the threshold condition are retained and the candidate execution steps that do not meet the threshold condition are removed to obtain execution step description data. The execution step description data is subjected to operator abstraction processing to map each execution step to a corresponding operator semantic unit, and operator identification and operator attribute description are generated to obtain operator description data. The execution dependency relationship between the operators in the operator description data is subjected to directed relationship modeling processing. For each pair of candidate dependency relationships, a dependency clue is extracted and weighted calculation is performed according to the dependency determination weight parameter. The existence and direction of the dependency edge are determined based on the weighted result to obtain operator dependency relationship data. The data input description and data output description obtained in the execution step analysis process and the operator dependency relationship data are subjected to unified modeling processing. The data input description and the data output description are bound to the corresponding operator identification, and the operator dependency relationship data is organized into an executable topology structure to form job abstraction data. The job abstraction data is subjected to operator semantic rule verification processing. According to the semantic rule strength parameter, a set of semantic rules is selected and a rule triggering boundary is determined. The semantic consistency of each operator is determined and the constraint satisfaction is determined to obtain operator semantic verification result data. The data input description and the data output description in the job abstraction data are subjected to data contract consistency verification processing. According to the semantic rule strength parameter, a contract verification boundary is determined. The data structure constraint, field constraint, and dependency constraint of the input and output data are subjected to consistency determination to obtain data contract verification result data. The job abstraction data is subjected to binding relationship consistency verification processing in combination with the version fingerprint data. The target binding feature is determined based on the version fingerprint data, and the consistency determination boundary is determined according to the version binding consistency threshold parameter. The binding feature of the job abstraction data and the target binding feature are subjected to consistency matching and deviation determination to obtain version binding verification result data. The operator semantic verification result data, the data contract verification result data, and the version binding verification result data are subjected to summary processing. According to the semantic rule strength parameter and the version binding consistency threshold parameter, a summary decision boundary is determined to form consistency verification result data. The job abstraction data and the consistency verification result data are merged to form job semantic data.

[0009] As a further description of the above technical solutions: The change impact analysis module performs feature unpacking and feature index calculation based on the version fingerprint data to obtain version feature indexes, and compares and processes the version feature indexes with preset interval determination logic, determines the interval in which the version feature indexes fall according to the size relationship between the version feature indexes and three interval boundary values; the change impact analysis module matches the version feature indexes with the preset parameter mapping rule based on the interval determination result, determines the parameter values corresponding to the current version features, and collects the parameter values to form a mapping result of the version fingerprint data, the mapping result including a propagation depth control parameter, an edge weight weighting parameter, a propagation cutoff threshold parameter, a path merging strategy parameter, and a change source confidence parameter; the job semantic data is analyzed and processed by operator node traversal to form node analysis result data, the change node set is identified based on the version fingerprint data, and the starting impact strength is configured for each change node in the change node set according to the change source confidence parameter; the node analysis result data is analyzed and processed by operator dependency relationship impact propagation, the propagation level is limited according to the propagation depth control parameter, the propagation edge is weighted according to the edge weight weighting parameter, the cumulative impact strength is threshold cut off according to the propagation cutoff threshold parameter, and the multi-path propagation result is merged according to the path merging strategy parameter to form impact propagation result data; the impact propagation result data is calculated and processed by the execution path impact range according to the version fingerprint data, and the impact range calculation result is uniformly constrained according to the propagation depth control parameter, the propagation cutoff threshold parameter, and the path merging strategy parameter to form change impact evaluation data; the change impact evaluation data is processed by impact degree determination, and the impact degree is compared and processed with the preset release control rule to generate a release control decision to obtain release control instruction data.

[0010] As a further description of the above technical solutions: The historical running record data is analyzed and processed by the big data processing module in a distributed manner to extract job history execution frequency features, resource usage distribution features, and execution time length statistical features to form historical running statistical feature data; the job semantic data is processed by job execution feature aggregation to extract operator quantity features, dependency level features, and execution path structure features to form job execution feature data; the change impact evaluation data is processed by association feature analysis to extract impact range features, impact path features, and impact degree features to form change impact feature data; the historical running statistical feature data, the job execution feature data, and the change impact feature data are merged to form job running feature data, and the job running feature data is output.

[0011] As a further description of the above technical solutions: The adaptive resource concurrency control module performs resource load index calculation and processing on the job operation characteristic data to generate resource load index data, performs concurrency stress index calculation and processing on the job operation characteristic data to generate concurrency stress index data, performs adaptive adjustment processing on the calculation resource allocation parameter and the job concurrency degree parameter according to the resource load index data and the concurrency stress index data under the condition that the execution of the release control instruction data is allowed, forms operation control strategy data, and outputs the operation control strategy data to the software product output module to control the execution mode of the construction process and the deployment process.

[0012] Further description of the above technical solution is as follows: The software product output module controls and processes the construction process for constructing the standardized development input data set into a software product according to the release control instruction data under the condition that the execution of the release control instruction data is allowed, obtains construction execution result data, performs deployment processing on the construction execution result data according to the operation control strategy data, writes version fingerprint data into software product meta information, forms deployed software product data, performs result solidification processing on the deployed software product data, obtains developed software product, and outputs the developed software product.

[0013] The present application has the following beneficial effects: 1、In the application, by unified feature extraction and aggregation of source code, build script, dependency declaration, configuration parameters and development environment elements, version fingerprint data capable of stably representing software version state is formed, and version feature index is further calculated, the version difference degree is discretely divided into multiple intervals, based on the interval determination result, the system automatically determines the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter, the version binding consistency threshold parameter and the propagation control parameter, through the above mechanism, the analysis and verification overhead can be reduced for low difference version scene, and the strictness of analysis, verification and influence analysis can be significantly enhanced for high difference version scene, thereby avoiding the misjudgment, omission or resource waste problem caused by fixed threshold or manual configuration parameter in the prior art, improving the adaptability and stability of the system under different version change intensity, performing static analysis and multi-stage abstract modeling processing on the build script data, mapping the execution steps in the script into operator semantic units with clear semantics, and constructing a directed operator dependency relationship based on the dependency determination weight parameter, further combining data input description and data output description to form an executable topology structure, on this basis, operator semantic rule verification, data contract consistency verification and version binding consistency verification are performed on the job abstract data, so that the build process is upgraded from text-level script to job semantic data with structure semantics, data semantics and version semantics constraints, thereby effectively avoiding the problem that implicit dependencies, data inconsistency or environment binding errors in the build script are not discovered until the subsequent build or deployment stage, and the predictability and verifiability of the build process are significantly improved.

[0014] 2、In the application, the change impact analysis module is based on job semantic data, identifies a set of change nodes corresponding to version changes, and takes it as a propagation source, performs directed influence propagation analysis based on operator dependency relationship, through joint control of propagation depth control parameter, edge weight weighting parameter, propagation truncation threshold parameter and path merging strategy parameter, the system can effectively suppress the diffusion of irrelevant or low confidence paths in the propagation process, and uniformly merge the multi-path propagation results, thereby forming change impact evaluation data that can accurately reflect the actual influence range of version changes on job execution path, operator node and overall job structure, avoiding the influence range expansion or omission problem caused by simple dependency file-level difference or manual experience judgment in the prior art, and providing a reliable basis for subsequent release control and resource scheduling.

[0015] 3、In the present application, based on the change impact evaluation data, the influence degree is determined by rules, and compared with the preset release control rule, the release control instruction data is automatically generated, so as to realize the dynamic control of the construction process and the deployment process. Under the condition that the release control instruction allows execution, the system further combines the historical running record data, the job semantic data and the change impact evaluation data to form the job running feature data, and calculates the resource load index and the concurrent pressure index according to the job running feature data. Through the above processing, the concurrency and resource utilization can be improved when the version change risk is low, and the resource allocation and concurrency control can be automatically tightened when the version change risk is high, or even the release process is blocked, so as to ensure the system stability and release safety, avoid the problem of long-term conservative or excessive allocation of resource configuration, write the version fingerprint data into the software product meta information in the software product output stage, and perform consistency checking and result solidification processing on the construction result and the deployment result, so that the final output software product and its corresponding development input state, construction environment state and version difference characteristics form a stable binding relationship. Therefore, the system can not only realize the automatic construction and deployment of software products, but also provide complete and reliable data basis for subsequent problem tracing, version rollback, impact review and compliance audit, which is significantly superior to the implementation mode in the prior art that the construction result is disconnected with the version state. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The system architecture diagram of the present application is shown in the figure. Figure 2 The version feature index interval determination and parameter mapping flowchart of the present application is shown in the figure. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0018] Reference Figure 1The application provides a software development cloud computing system with a big data processing module, which comprises a development input collection module, a version fingerprint construction module, a job semantic modeling module, a change impact analysis module, a big data processing module, an adaptive resource concurrent control module and a software product output module.

[0019] In the embodiment, the development input collection module receives source code data, build script data, dependency declaration data, configuration file data and development environment description data, first performs integrity check processing on the source code data, reads the file list and directory level information of the source code data and establishes a check object set of the source code data, extracts the file size, file modification time and file content digest from the check object set one by one and forms a check feature set, performs missing file judgment, empty file judgment, duplicate file judgment and unreadable file judgment on the source code data based on the check feature set, further performs content consistency check on the files determined to be readable and not missing and generates check mark information, performs check pass object retention and check failure object elimination processing on the source code data according to the check mark information, and generates the checked source code data; The build script data is subjected to structure parsing processing, the script type identifier and script content text of the build script data are read and script reading result data is formed, the script reading result data is subjected to statement segmentation processing and script statement sequence is generated, the script statement sequence is subjected to key instruction recognition processing and instruction segment set is generated, the instruction segment set is subjected to hierarchical merging processing to form script structure node set, and node order relationship and node inclusion relationship are established for the script structure node set to form script structure relationship set, and the parsed build script data is generated based on the script structure node set and the script structure relationship set; The dependency declaration data is subjected to dependency item parsing processing, the dependency declaration text of the dependency declaration data is read and dependency declaration reading result data is formed, the dependency declaration reading result data is subjected to dependency item entry segmentation processing and dependency item entry set is generated, the dependency item entry set is subjected to field extraction processing and dependency item field set is generated, the dependency item field set is subjected to normalization processing on the dependency item entry set and normalized dependency item set is generated, and the normalized dependency item set is subjected to deduplication merging processing to form dependency parsing data; The configuration file data is subjected to parameter extraction processing, the configuration content of the configuration file data is read and configuration reading result data is formed, the configuration reading result data is subjected to key-value pair positioning processing and candidate parameter entry set is generated, the candidate parameter entry set is subjected to key name extraction and value extraction processing to generate parameter key set and parameter value set, the parameter key set and the parameter value set are paired according to the entry correspondence relationship and parameter entry set is generated, and the parameter entry set is subjected to type judgment and value normalization processing to form configuration parameter data; The development environment description data is subjected to environment element parsing processing, the environment description content of the development environment description data is read and environment reading result data is formed, the environment reading result data is subjected to field positioning and field extraction processing to generate environment element entry set, the environment element entry set is subjected to element deduplication and element normalization processing to form environment element data. The checked source code data, the parsed build script data, the dependency parsing data, the configuration parameter data and the environment element data are merged, a correlation mapping relationship in the same project scope is established according to a unified data identification rule, and correlation index information is generated, the five types of data and the correlation index information are encapsulated and merged, and a standardized development input data set is formed.

[0020] In this embodiment, the version fingerprint construction module receives the checked source code data, the parsed build script data, the dependency parsing data, the configuration parameter data and the environment element data, First, the source code data is subjected to code feature abstract calculation processing, in which the source code content is read file by file according to the code file organization structure of the source code data, and the read source code content is subjected to content normalization processing to eliminate the influence of irrelevant format differences, then the normalized source code content is subjected to content feature extraction processing to obtain stable features capable of representing the code logic and structure, and the stable features are subjected to abstract calculation and aggregation processing to form code feature abstract data having unique representation capability for the source code data; The parsed build script data is subjected to script feature abstract calculation processing, in which the script content is read segment by segment according to the script structure and instruction sequence in the parsed build script data, and the script content is subjected to structure unification processing to unify the sentence organization form, then the key build instructions, key parameter items and instruction interdependence in the script are subjected to feature extraction processing to obtain stable features capable of representing the build behavior, and the stable features are subjected to abstract calculation and aggregation processing to form script feature abstract data; The dependency parsing data is subjected to dependency feature abstract calculation processing, in which the dependency item entries are read in a traversal manner according to the dependency item organization structure in the dependency parsing data, and the dependency item entries are subjected to normalization processing to unify the representation form of the dependency items, then the dependency identification information, version indication information and dependency relationship organization mode in the dependency item entries are subjected to feature extraction processing to obtain stable features capable of representing the state of the dependency set, and the stable features are subjected to abstract calculation and aggregation processing to form dependency feature abstract data; The configuration parameter data is subjected to configuration feature abstract calculation processing, in which the parameter entries are read item by item according to the parameter entry organization structure in the configuration parameter data, and the parameter entries are subjected to key name unification and value unification processing to unify the parameter expression form, then the build options, running options and environment options represented by the parameter entries are subjected to feature extraction processing to obtain stable features capable of representing the configuration state, and the stable features are subjected to abstract calculation and aggregation processing to form configuration feature abstract data; The environmental feature summary calculation processing is performed on the environmental element data, in the environmental feature summary calculation processing, the element entries in the environmental element data are read one by one according to the element entry organization structure, and the element representation unification processing is performed on the element entries to unify the environmental element expression form, then the feature extraction processing is performed on the development environment key elements represented by the element entries to obtain stable features capable of representing the environmental state, and the summary calculation and aggregation processing are performed on the stable features to form the environmental feature summary data; The code feature summary data, the script feature summary data, the dependency feature summary data, the configuration feature summary data and the environmental feature summary data are aggregated and calculated, in the aggregation calculation processing, the five types of feature summaries are uniformly sorted and organized to ensure the stability of the aggregation order, and the joint aggregation calculation is performed on the organized five types of feature summaries to form the version fingerprint data capable of representing the source code state, the build script state, the dependency state, the configuration state and the environmental state.

[0021] In the embodiment, the version feature index is obtained by performing feature unpacking and feature index calculation on the version fingerprint data based on the job semantic modeling module, in the feature unpacking process, the version fingerprint identifier, the fingerprint summary value and the fingerprint generation time identifier of the version fingerprint data are read, the constituent contents of the version fingerprint data are uniformly split to obtain the code feature summary data, the script feature summary data, the dependency feature summary data, the configuration feature summary data and the environmental feature summary data, the comparability preparation and alignment organization are performed on the split feature summary data to form a difference metric set, the aggregation calculation and scale normalization are performed on the difference metric set to generate the version feature index, the version feature index is compared with the preset interval judgment logic, the interval into which the version feature index falls is determined, based on the interval judgment result, the version feature index is matched with the preset parameter mapping rule, the parameter value corresponding to the current version feature is determined, the determined parameter value is collected to form the mapping result of the version fingerprint data, and it is ensured that the mapping result of the version fingerprint data includes the mapping results of the analysis confidence threshold parameter, the dependency judgment weight parameter, the semantic rule strength parameter and the version binding consistency threshold parameter; The execution step parsing processing is performed on the parsed build script data, in the execution step parsing processing, the parsed build script data is read according to the script structure and the instruction order, and the build-related instruction fragments and the control fragments are identified, the step boundary judgment is performed on the identified fragments to generate a candidate execution step set, and the statement structure features and the context consistency features of each candidate execution step in the candidate execution step set are extracted to calculate the parsing confidence, the parsing confidence is compared with the parsing confidence threshold parameter to complete the step retention judgment and the step elimination judgment, so that the candidate execution steps meeting the threshold condition are retained and the candidate execution steps not meeting the threshold condition are eliminated to obtain the execution step description data; The execution step description data is subjected to operator abstraction processing. In the operator abstraction processing, semantic classification and functional classification are performed on each execution step in the execution step description data to map the execution step to a corresponding operator semantic unit, a unique operator identifier is generated for the operator semantic unit, and key attributes of the operator semantic unit are extracted to generate operator attribute description, so as to obtain operator description data; The execution dependency relationship between the operators in the operator description data is subjected to directed relationship modeling processing. In the directed relationship modeling processing, the operator description data is traversed in order of operator identifier to construct a candidate dependency pair set, dependency clues are extracted for each pair of candidate dependency relationships and weighted calculation is performed on the dependency clues according to a dependency judgment weight parameter, the existence and direction of a dependency edge are determined based on the weighted result, and the determined dependency edge is summarized and organized to obtain operator dependency relationship data; The data input description and data output description obtained in the execution step analysis process and the operator dependency relationship data are subjected to unified modeling processing. In the unified modeling processing, the data input description and the data output description are subjected to structure alignment and identifier alignment to maintain consistent expression of the input and output descriptions, the data input description and the data output description are bound to the corresponding operator identifiers, and the bound operator set is topologically organized according to the operator dependency relationship data to form an executable topology structure, so as to form job abstraction data; The job abstraction data is subjected to operator semantic rule verification processing. In the operator semantic rule verification processing, a semantic rule set is selected according to a semantic rule strength parameter and a rule triggering boundary is determined, semantic consistency judgment and constraint satisfaction judgment are performed on each operator in the job abstraction data according to the operator identifier, and the judgment results are summarized to form operator semantic verification result data; The data input description and the data output description in the job abstraction data are subjected to data contract consistency verification processing. In the data contract consistency verification processing, a contract verification boundary is determined according to a semantic rule strength parameter, consistency judgment is performed on the data structure constraint, the field constraint and the dependency constraint in the data input description and the data output description item by item, and the judgment results are summarized to form data contract verification result data; The job abstraction data is subjected to binding relationship consistency verification processing in combination with version fingerprint data. In the binding relationship consistency verification processing, a target binding feature is determined based on the version fingerprint data, a consistency judgment boundary is determined according to a version binding consistency threshold parameter, consistency matching and deviation judgment are performed on the binding feature of the job abstraction data and the target binding feature, and the judgment results are summarized to form version binding verification result data; The operator semantic verification result data, data contract verification result data and version binding verification result data are processed in aggregation, in which the aggregation decision boundary is determined according to the semantic rule strength parameter and the version binding consistency threshold parameter, and the consistency merging and conflict resolution are performed on the three types of verification results to form the consistency verification result data, and the consistency verification result data is mapped back to the job abstract data according to the operator identifier and the job structure and the association is solidified, thereby forming the job semantic data The execution step retention decision formula driven by the resolution confidence threshold is:

[0022]

[0023]

[0024]

[0025]

[0026]

[0027] : script content text of the script data, : statement segmentation processing operator, used for segmenting the script content text into generating a script statement sequence , : script statement sequence, : key instruction recognition processing operator, used for recognizing key instructions from generating an instruction fragment set , : instruction fragment set, : hierarchical merging processing operator, used for forming a script structure node set from the instruction fragment set , and supporting the determination of subsequent node order relationship and node inclusion relationship, : script structure node set, : step boundary determination operator, used for generating a candidate execution step set from the script structure node set , : candidate execution step set, : candidate execution step, ; : execution step of the statement structure feature, : execution step of the context consistency feature, : resolution confidence calculation function, used for calculating the resolution confidence of the execution step from the statement structure feature Calculate the analytical confidence level , Execution steps The resolution confidence level, The confidence threshold parameter is used to control the acceptance boundary of the decision-making process in the step. : The set of retained execution steps corresponding to the execution step description data.

[0028] The abstract mapping formula from execution steps to operator semantic units:

[0029]

[0030]

[0031]

[0032] Operator abstract function, used to perform execution steps Perform semantic and functional classification and generate operator semantic units. , : Operator semantic unit, Operator identifier: Used to uniquely identify the semantic unit of an operator. , Operator identifier generation function, Operator attribute description, used to characterize the semantic unit of the operator. Key attributes, : Attribute extraction function, used to generate , Operator description data consists of operator identifiers and operator attribute descriptions.

[0033] Dependency-determined weight-driven formula for modeling directed dependency edges:

[0034]

[0035]

[0036]

[0037]

[0038] : Set of candidate dependency pairs : No. The extraction results of class dependency clues are used for characterization. and Evidence of dependence between them : component dependent on the weight parameter for determining the dependency score of a candidate pair of dependent components, : weighting of dependency cues by their class, : number of dependency cue classes, : candidate pair of dependent components, : weighted dependency score of a candidate pair of dependent components, : dependency existence decision threshold, : dependency existence indicator function taking values or , : indicator function, : dependency directionality result, : directionality determination function, : graph structure representation of the operator dependency data, : set of directed dependency edges.

[0039] Data input output description binding and executable topology formation formula:

[0040]

[0041] ,

[0042]

[0043]

[0044] : data input description obtained during execution step parsing process, : data output description obtained during execution step parsing process, : structure alignment and identification alignment function for keeping input output descriptions consistent, output aligned and , : set of aligned data input descriptions, : set of aligned data output descriptions, : binding mapping of input descriptions to operator identifications, : binding mapping of output descriptions to operator identifications, : topology organization function for forming executable topology from operator dependencies , : executable topology, : encapsulation function for forming job abstraction data , : job abstraction data.

[0045] The semantic rule strength and version binding consistency threshold driven consistency check and output formula: , , , , , : Operator semantic rule check function, used to generate operator semantic check result data , : Data contract consistency check function, used to generate data contract check result data , : Semantic rule strength parameter, : Version fingerprint data or its unpacked target binding features used for binding verification, : Binding relationship consistency verification function, used to generate version binding check result data , : Version binding consistency threshold parameter, : Summary and conflict resolution function, used to form consistency check result data from three types of check results , : Operator semantic check result data, : Data contract check result data, : Version binding check result data, : Consistency check result data, : Merge and solidification function, used to merge : According to the operator identifier and job structure, map back to the job abstract data and complete the association solidification, output job semantic data .

[0046] In the embodiment, after the change impact analysis module receives the job semantic data and the version fingerprint data, feature unpacking and feature index calculation are performed based on the version fingerprint data to obtain version feature indexes. In the feature unpacking process, the version fingerprint identifier, the fingerprint summary value, and the fingerprint generation time identifier of the version fingerprint data are read, and the code feature summary data, the script feature summary data, the dependency feature summary data, the configuration feature summary data, and the environment feature summary data are split according to the aggregation structure. In the feature index calculation process, the split feature summary data is subjected to comparability preparation and alignment organization, and a difference metric set is formed. Aggregation calculation and scale normalization are performed on the difference metric set to generate version feature indexes. The version feature indexes are compared with the preset interval judgment logic, the interval into which the version feature indexes fall is determined, the version feature indexes are matched with the preset parameter mapping rule based on the interval, the parameter values corresponding to the current version features are determined, the determined parameter values are collected to form the mapping result of the version fingerprint data, and it is ensured that the version fingerprint data mapping result includes the propagation depth control parameter, the edge weight weighting parameter, the propagation cutoff threshold parameter, the path merging strategy parameter, and the change source confidence parameter. The dependency propagation impact domain calculation algorithm is executed based on the parameters, and the change impact evaluation processing is completed. Subsequently, the job semantic data is subjected to operator node traversal analysis processing. In the traversal analysis processing, the job abstract data is read from the job semantic data, and the operator description data, the operator dependency relationship data, the data input description, and the data output description are organized according to the operator identifier. The in-degree statistics and the out-degree statistics of the operator dependency relationship data are performed, and the node connection features are formed. The binding consistency reading of the data input description and the data output description is performed, and the node data features are formed. The associated reading of the consistency check result data is performed, and the node check features are formed. The node connection features, the node data features, and the node check features are summarized and merged according to the operator identifier to form the node analysis result data. The data input description and the data output description are obtained by parsing and structuring the input elements referenced in each execution step and the output elements generated in each execution step during the execution step parsing process. During the execution step parsing process, the input reference element extraction process is performed for each execution step, specifically including: parsing the input file path referenced in the execution step; parsing the data table identifier, theme identifier or resource identifier referenced in the execution step; parsing the upstream execution step output identifier referenced in the execution step; parsing the input value transmitted in the form of a parameter or a variable in the execution step; and structuring the above input reference elements to form the data input description associated with the corresponding execution step. During the execution step parsing process, the output generation element extraction process is performed for each execution step, specifically including: parsing the output file path declared or implied in the execution step; parsing the generated artifact identifier declared in the execution step; parsing the intermediate product identifier declared in the execution step; parsing the output variable or resource identifier visible to the subsequent step in the execution step; and structuring the above output generation elements to form the data output description associated with the corresponding execution step. Subsequently, the set of change nodes is identified based on the version fingerprint data, and during the identification process, the fingerprint digest value and the fingerprint generation time identifier in the version fingerprint data are compared and read to form the change determination basis. According to the difference measurement results of the code feature digest data, the script feature digest data, the dependency feature digest data, the configuration feature digest data and the environment feature digest data, the difference measurement results are merged to form the change clue set. Based on the change clue set, the operator identifier corresponding to the change clue is located in the job semantic data, and the change node set is output. According to the change source confidence parameter, the initial influence strength of each change node in the change node set is configured to form the change node strength configuration result; Subsequently, the operator dependency relationship influence propagation analysis process is performed on the node analysis result data. In the influence propagation analysis process, the set of change nodes is taken as the propagation source, and the change node strength configuration result is taken as the initial propagation state. The propagation front is expanded layer by layer in the direction of the directed edges of the operator dependency relationship data, and the propagation level is limited according to the propagation depth control parameter in each layer of propagation. For each propagation edge in the propagation front, the source operator identifier, the target operator identifier and the edge direction relationship of the edge are extracted, and the propagation edge weight is calculated according to the edge weight parameter. The propagation edge weight and the current cumulative influence strength of the source node are combined to obtain the incremental influence strength of the target node. The cumulative influence strength of the target node is updated by superposition, and the cumulative influence strength is threshold truncated according to the propagation truncation threshold parameter to eliminate the propagation paths that do not meet the propagation condition. For the propagation results of the same target node reached by multiple propagation paths, the merging process is performed according to the path merging strategy parameter to form a single target node propagation state. Thus, the influence propagation result data is formed after the propagation level is advanced. Subsequently, the influence propagation result data is combined with the version fingerprint data to perform path influence range calculation processing. In the influence range calculation processing, the cumulative influence intensity and propagation level information of each operator identifier are read from the influence propagation result data, and the reachable paths are expanded in combination with the operator dependency data to form an influence path set. The influence path set is aggregated according to the operator identifier sequence to form an influence range candidate set. The influence range candidate set is subjected to hierarchical consistency constraint processing according to the propagation depth control parameter, strength consistency constraint processing according to the propagation truncation threshold parameter, and path consistency constraint processing according to the path merging strategy parameter, thereby forming change influence evaluation data. Propagation source initialization and starting influence intensity configuration formula:

[0047]

[0048] : an operator set corresponding to the operator description data, : a change node set, : an operator node corresponding to any operator identifier, ; : the starting cumulative influence intensity of the operator node , : a change source confidence parameter, : a propagation truncation threshold parameter, : a 0th layer propagation front set.

[0049] Edge weight weighting and single-layer propagation incremental influence intensity calculation formula:

[0050]

[0051] : a propagation edge set triggered by the propagation front during the propagation of the th layer, : a directed dependency edge in the operator dependency data, indicating the propagation direction from the source operator identifier to the target operator identifier , : the base edge weight of the dependency edge, : an edge weight weighting parameter, : the propagation edge weight, : the incremental influence intensity of the target node obtained by the propagation calculation of the th layer, : the source node current accumulated impact strength.

[0052] accumulated impact strength update, truncation and multi-path merging formula:

[0053]

[0054]

[0055] : the first untruncated accumulated impact strength after layer propagation, : the first accumulated impact strength after layer propagation, : path merging processing function, : path merging strategy parameter, : indicator function, : propagation truncation threshold parameter.

[0056] propagation depth control and impact domain set output formula: ,

[0057] : propagation depth control parameter, : the first layer propagation front set, : dependent propagation impact domain set, representing the set of operator nodes that can be reached and meet the threshold condition under the propagation depth constraint and strength truncation constraint, corresponding to the core coverage domain of the impact propagation result data formed after the impact propagation analysis processing.

[0058] impact path set and impact range candidate set formation formula:

[0059]

[0060]

[0061] : directed edge set of operator dependency data,: reachable path expansion function, used to expand the reachable path in combination with the operator dependency data to form an impact path set, : impact path set, : path summary function, used to perform path summary on the impact path set according to the operator identifier sequence to form an impact range candidate set, : impact range candidate set, : The uniformization constraint function is used to perform hierarchical uniformization constraint, intensity uniformization constraint and path uniformization constraint processing on the impact range candidate set respectively, so as to form the impact range calculation result and enter the change impact evaluation data; : The impact range calculation result The impact degree determination processing is performed on the change impact evaluation data. In the impact degree determination processing, the impact range candidate set, the impact path set and the cumulative impact intensity of each operator identifier in the change impact evaluation data are read and regularized and merged to form the impact degree determination basis. The system configures corresponding release control decision types for different impact degree determination results in the preset configuration, solidifies the corresponding relationship between the impact degree determination result and the release control decision type into executable release control rules, compares the impact degree determination basis with the preset release control rules, generates a release control decision based on the release control rules, and solidifies and outputs the release control decision as release control instruction data.

[0062] As Figure 2 : In the embodiment, the version fingerprint data is subjected to feature unpacking processing. In the feature unpacking processing, first, the fingerprint summary value for representing the fingerprint aggregation result in the version fingerprint data is read, and the fingerprint generation time identifier for representing the fingerprint generation time and the version fingerprint identifier for identifying the development input range corresponding to the fingerprint are read; The version fingerprint data is subjected to constituent content splitting processing, and the version fingerprint data is split into code feature summary data, script feature summary data, dependency feature summary data, configuration feature summary data and environment feature summary data according to the aggregation structure of the version fingerprint data; The code feature summary data, script feature summary data, dependency feature summary data, configuration feature summary data and environment feature summary data obtained by splitting are subjected to consistency checking processing. In the consistency checking processing, the summary format, summary length and summary encoding mode of each type of feature summary data are subjected to consistency determination, and the feature summary data that does not satisfy the consistency determination condition is subjected to abnormal marking processing to form an unpacking checking mark; The code feature summary data, script feature summary data, dependency feature summary data, configuration feature summary data and environment feature summary data are subjected to structured organization processing. In the structured organization processing, a unified classification index relationship is established for the five types of feature summary data according to the feature category, and the five types of feature summary data are bound with the fingerprint generation time identifier according to the version fingerprint identifier to form a traceable feature unpacking result.

[0063] After the feature unpacking is completed, a feature indicator calculation process is performed on the feature unpacking result. In the feature indicator calculation process, first, the code feature summary data, the script feature summary data, the dependency feature summary data, the configuration feature summary data, and the environment feature summary data are respectively subjected to a comparability preparation process. In the comparability preparation process, the five types of feature summary data are normalized and organized according to a unified summary representation rule to form a comparable feature sequence. Subsequently, a difference measurement preparation process is performed on the comparable feature sequence. In the difference measurement preparation process, a measurement benchmark for this calculation is determined according to the version fingerprint identifier and the fingerprint generation time identifier, and the comparable feature sequence is aligned with the measurement benchmark to form an aligned feature sequence. A feature difference measurement process is performed on the aligned feature sequence. In the feature difference measurement process, the difference degree of the code feature summary data relative to the measurement benchmark, the difference degree of the script feature summary data relative to the measurement benchmark, the difference degree of the dependency feature summary data relative to the measurement benchmark, the difference degree of the configuration feature summary data relative to the measurement benchmark, and the difference degree of the environment feature summary data relative to the measurement benchmark are calculated respectively, and the five types of difference degrees are organized to form a difference measurement set. Subsequently, an indicator aggregation calculation process is performed on the difference measurement set. In the indicator aggregation calculation process, according to the aggregation weight rule in the system preset configuration, the five types of difference degrees are weighted and aggregated to generate a single aggregated indicator, and a scale normalization process is performed on the aggregated indicator to obtain an indicator value falling within a fixed interval. The normalized indicator value is written into the version feature indicator and bound with the version fingerprint identifier and the fingerprint generation time identifier, thereby forming the version feature indicator.

[0064] An interval determination process is performed on the version feature indicator. In the system preset configuration, the version feature indicator is divided into four intervals. The four intervals are mutually exclusive intervals, and the four intervals are interval I, interval II, interval III, and interval IV in order of low to high version difference degree. Any version feature indicator is only allowed to be determined as one interval at the same time. The system sets a first boundary value, a second boundary value, and a third boundary value for the version feature indicator, which are used to divide the version feature indicator into the corresponding intervals, and the interval determination conditions are solidified into executable interval determination logic. The interval determination logic only uses the size comparison result between the version feature indicator and the three interval boundary values as the determination basis, and does not depend on other external conditions.

[0065] The determination relationship between the version feature index and the three interval boundary values is: when the version feature index is less than the first boundary value, it is determined to fall into interval I; when the version feature index is greater than or equal to the first boundary value and less than the second boundary value, it is determined to fall into interval II; when the version feature index is greater than or equal to the second boundary value and less than the third boundary value, it is determined to fall into interval III; and when the version feature index is greater than or equal to the third boundary value, it is determined to fall into interval IV. Through the above interval determination logic, the discretization classification of the version difference degree is realized, and a unique basis is provided for the subsequent determination of parameter values.

[0066] When the version feature index falls into interval I, the system adopts a parameter value set for a low version difference scenario, specifically: the analysis confidence threshold parameter value is 0.60; the dependency determination weight parameter value is 1.00; the semantic rule strength parameter value is 0.60; the version binding consistency threshold parameter value is 0.60; the propagation depth control parameter value is 2; the edge weight weighting parameter value is 0.80; the propagation truncation threshold parameter value is 0.20; the path merging strategy parameter value is the maximum value merging; and the change source confidence parameter value is 0.60. This parameter value set is used to reduce the analysis and propagation constraint strength and improve system processing efficiency in the case of small version difference.

[0067] When the version feature index falls into interval II, the system adopts a parameter value set for a medium-low version difference scenario, specifically: the analysis confidence threshold parameter value is 0.70; the dependency determination weight parameter value is 1.20; the semantic rule strength parameter value is 0.70; the version binding consistency threshold parameter value is 0.70; the propagation depth control parameter value is 3; the edge weight weighting parameter value is 1.00; the propagation truncation threshold parameter value is 0.30; the path merging strategy parameter value is weighted merging; and the change source confidence parameter value is 0.70. This parameter value set is used to moderately enhance the dependency relationship determination and influence propagation control ability while ensuring analysis accuracy.

[0068] When the version feature index falls into interval III, the system adopts a parameter value set for a medium-high version difference scenario, specifically: the analysis confidence threshold parameter value is 0.80; the dependency determination weight parameter value is 1.50; the semantic rule strength parameter value is 0.80; the version binding consistency threshold parameter value is 0.80; the propagation depth control parameter value is 4; the edge weight weighting parameter value is 1.20; the propagation truncation threshold parameter value is 0.40; the path merging strategy parameter value is average value merging; and the change source confidence parameter value is 0.80. This parameter value set is used to strengthen semantic constraints, dependency determination and propagation control in the case of significant increase in version difference, and reduce the risk of misjudgment.

[0069] When the version feature index falls into interval IV, the system adopts a parameter value set for the high version difference scenario, specifically: the resolution confidence threshold parameter value is 0.90; the dependency determination weight parameter value is 1.80; the semantic rule strength parameter value is 0.90; the version binding consistency threshold parameter value is 0.90; the propagation depth control parameter value is 5; the edge weight weighting parameter value is 1.50; the propagation truncation threshold parameter value is 0.50; the path merging strategy parameter value is priority merging; and the change source confidence parameter value is 0.90. The parameter value set is used to significantly improve the strictness of resolution and propagation determination in the case of great version difference, to guarantee system stability and release safety, and to determine the version feature index as one of interval I, interval II, interval III or interval IV to obtain the interval determination result.

[0070] Based on the preset parameter mapping rule, the interval determination result is taken as the only input basis, the parameter mapping rule is used to convert the interval determination result into a parameter value set, the interval determination result is read according to the corresponding interval to determine the parameter values of the resolution confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter, the version binding consistency threshold parameter, the propagation depth control parameter, the edge weight weighting parameter, the propagation truncation threshold parameter, the path merging strategy parameter and the change source confidence parameter, and the parameter value set is formed, the parameter value set is configured for each interval respectively, the parameter value set is uniquely determined under the same interval, does not depend on the runtime state information, and does not depend on manual intervention. In the system preset configuration, the corresponding relationship between each interval and the corresponding parameter value set is solidified as an executable mapping rule, which is used to convert the interval determination result corresponding to the version feature index into a parameter value for subsequent processing.

[0071] Based on the mapping rule, the parameter value set is written into the mapping result to obtain the mapping result of the version fingerprint data called by the job semantic modeling module and the change impact analysis module. The mapping result of the job semantic modeling module is one, and the mapping result of the change impact analysis module is two. The job semantic modeling module obtains the analysis confidence threshold parameter, the dependency judgment weight parameter, the semantic rule strength parameter, and the version binding consistency threshold parameter based on the mapping result. The change impact analysis module obtains the propagation depth control parameter, the edge weight weighting parameter, the propagation truncation threshold parameter, the path merging strategy parameter, and the change source confidence parameter based on the mapping result. The analysis confidence threshold parameter is used to control the acceptance boundary of the candidate analysis result in the execution step analysis process. The higher the threshold value, the more stringent the analysis confidence requirement. The dependency judgment weight parameter is used to weight the multiple dependency clues in the operator dependency modeling. The semantic rule strength parameter is used to adjust the strictness of the operator semantic rule verification and data contract consistency verification. The higher the strength, the more stringent the constraint. The version binding consistency threshold parameter is used to determine the binding relationship consistency verification boundary. The higher the threshold value, the more stringent the binding consistency requirement. The propagation depth control parameter is used to limit the maximum level or maximum path length of the impact propagation. The edge weight weighting parameter is used to weight the dependency edge corresponding to the operator dependency relationship data in the job semantic data. The propagation truncation threshold parameter is used to threshold the cumulative impact strength in the impact propagation process. The value is obtained from the system preset configuration or the truncation threshold configuration value obtained from the version fingerprint data mapping. The path merging strategy parameter is used to control the merging rule of the multi-path propagation result. The change source confidence parameter is used to weight the starting impact strength of different change nodes in the change node set identified by the version fingerprint data.

[0072] In the subsequent processing process, the job semantic modeling module obtains the corresponding analysis confidence threshold parameter, dependency judgment weight parameter, semantic rule strength parameter, and version binding consistency threshold parameter based on the mapping result of the version fingerprint data. After processing, the job semantic data is formed. The change impact analysis module obtains the corresponding propagation depth control parameter, edge weight weighting parameter, propagation truncation threshold parameter, path merging strategy parameter, and change source confidence parameter based on the mapping result of the version fingerprint data. After processing, the release control instruction data is obtained.

[0073] In the embodiment, after receiving the job semantic data, the change impact evaluation data and the historical running record data, the big data processing module performs distributed statistical analysis processing on the historical running record data. The historical running record data refers to a set of original running records formed in the previous construction and deployment process of the system, which contains job identification, time information, resource usage record and execution state information, and is used to support distributed statistical analysis on the historical execution behavior of the job. In the distributed statistical analysis processing, the record entries are read by partition according to the record granularity of the historical running record data, and the read entries are checked for duplication and missing. The checked record entries are aggregated and organized in time sequence and job identification dimension to form a serializable organization result that can be counted. The serializable organization result is counted for frequency to extract the job historical execution frequency feature. The serializable organization result is counted for resource dimension to extract the resource usage distribution feature. The serializable organization result is counted for time length dimension to extract the execution time length statistical feature. The job historical execution frequency feature, the resource usage distribution feature and the execution time length statistical feature are merged and solidified to form the historical running statistical feature data.

[0074] The job semantic data is processed for job execution feature aggregation. In the job execution feature aggregation processing, the operator description data is counted by traversing according to the operator identification from the job semantic data to extract the operator number feature. The operator dependency relationship data in the job abstract data is read to perform hierarchical expansion and hierarchical counting on the dependency relationship according to the topology level to extract the dependency level feature. The executable topology structure formed by the operator dependency relationship data is read to perform structured statistics on the path branching, path convergence and path length in the topology structure to extract the execution path structure feature. The operator number feature, the dependency level feature and the execution path structure feature are merged and solidified to form the job execution feature data.

[0075] The change impact evaluation data is processed for correlation feature analysis. In the correlation feature analysis processing, the influence range calculation result is read from the change impact evaluation data to perform range aggregation on the influence range calculation result to extract the influence range feature. The influence propagation and path organization result in the change impact evaluation data is performed for path merging to extract the influence path feature. The influence degree determination result in the change impact evaluation data is performed for hierarchical merging to extract the influence degree feature. The influence range feature, the influence path feature and the influence degree feature are merged and solidified to form the change impact feature data.

[0076] The historical running statistical feature data, the job execution feature data and the change influence feature data are merged; in the merging process, the three types of feature data are aligned and organized according to the job identifier and the version fingerprint identifier, and the repeated mapping is eliminated; the consistent coding and the structured packaging are performed on the aligned feature items, and the unified feature carrier is formed; thus, the job running feature data is formed, and the job running feature data is output.

[0077] In the embodiment, after the adaptive resource concurrency control module receives the job running feature data and the release control instruction data, the resource load index calculation processing is performed on the job running feature data, in the resource load index calculation processing, the historical running statistical feature data, the job execution feature data and the change influence feature data are read from the job running feature data, the three types of feature data are consistently organized, and the index calculation input is formed; the resource related feature extraction is performed on the index calculation input, and the load related statistical quantity representing the resource use distribution feature and the execution time length statistical feature is obtained; the resource load index data is generated by aggregating and calculating the load related statistical quantity according to the preset index calculation rule; at the same time, the range constraint processing is performed on the resource load index data, so that the resource load index data is ensured to be in the effective interval that can be used for control decision, and the result after the range constraint is solidified and output as the resource load index data.

[0078] In the embodiment, after the adaptive resource concurrency control module receives the job running feature data and the release control instruction data, the resource load index calculation processing is performed on the job running feature data, in the resource load index calculation processing, the historical running statistical feature data, the job execution feature data and the change influence feature data are read from the job running feature data, the three types of feature data are consistently organized, and the index calculation input is formed; the resource related feature extraction is performed on the index calculation input, and the load related statistical quantity representing the resource use distribution feature and the execution time length statistical feature is obtained; the resource load index data is generated by aggregating and calculating the load related statistical quantity according to the preset index calculation rule; at the same time, the range constraint processing is performed on the resource load index data, so that the resource load index data is ensured to be in the effective interval that can be used for control decision, and the result after the range constraint is solidified and output as the resource load index data.

[0079] After the resource load indicator data and the concurrent pressure indicator data are generated, the adaptive resource concurrent control module performs adaptive adjustment processing on the computing resource allocation parameter and the job concurrency degree parameter according to the resource load indicator data and the concurrent pressure indicator data under the condition that the release control instruction data allows execution; in the adaptive adjustment processing, first, the release control instruction data is read, an execution permission determination is performed, and it is determined whether the trigger condition for performing the adaptive adjustment processing is met; when the execution permission determination result meets the execution condition, the resource load indicator data and the concurrent pressure indicator data are jointly evaluated to form adjustment basis; based on the adjustment basis, the computing resource allocation parameter is subjected to increment, decrement or retention processing to form updated computing resource allocation parameter; at the same time, the job concurrency degree parameter is subjected to increase, decrease or retention processing to form updated job concurrency degree parameter; the updated computing resource allocation parameter and the updated job concurrency degree parameter are consistent encapsulation to form running control strategy data.

[0080] When the execution permission determination result does not meet the execution condition, the original values of the computing resource allocation parameter and the job concurrency degree parameter are kept, and the original values are solidified as running control strategy data; the running control strategy data is output to the software product output module for controlling the execution mode of the build process and the deployment process.

[0081] After the software product output module receives the version fingerprint data, the release control instruction data and the running control strategy data, the build process for building the standardized development input data set into a software product is controlled according to the release control instruction data under the condition that the release control instruction data allows execution; in the control processing, the release control instruction data is read, the allowed release state and the control decision information in the release control instruction data are parsed, and build trigger determination basis is formed; based on the build trigger determination basis, the build process is subjected to execution permission determination to determine whether the build process is in an allowed execution state; when the build process is in the allowed execution state, the execution arrangement of the build process is started, the build tasks are organized according to the execution order defined in the build process, and a build task sequence is generated; the build task sequence is subjected to build task scheduling in sequence to form build task scheduling result; the build task scheduling result is subjected to build task running to form build running process record; the build running process record is subjected to build result aggregation to obtain build execution result data; when the build process is in the unallowed execution state, the build process is terminated and the termination result is solidified to form build execution result data.

[0082] After obtaining the build execution result data, the software product output module performs deployment processing on the build execution result data according to the running control strategy data; in the deployment processing, the running control strategy data is read, the computing resource allocation parameter and the job concurrency parameter in the running control strategy data are parsed, and deployment execution constraints are formed; based on the deployment execution constraints, pre-deployment checking is performed on the build execution result data, and it is determined that the deployment input is in a deployable state; deployment task organization is performed on the build execution result data in the deployable state, and a deployment task sequence is generated; resource binding processing is performed on the deployment task sequence, so that the deployment task corresponds to the computing resource allocation parameter; concurrency control processing is performed on the deployment task sequence, so that the deployment task corresponds to the job concurrency parameter; deployment task scheduling is performed on the deployment task sequence after resource binding and concurrency control, and a deployment scheduling result is formed; deployment task running is performed on the deployment scheduling result, and a deployment running process record is formed; deployment result summarization is performed on the deployment running process record, and deployment completed software product data is formed; at the same time, version fingerprint data is written into software product meta information, and the version fingerprint data is associated and solidified with the deployment completed software product data, to form the deployment completed software product data.

[0083] After forming the deployment completed software product data, the software product output module performs result solidification processing on the deployment completed software product data; in the result solidification processing, the deployment completed software product data is read, product integrity checking is performed, and it is determined that the product content is in a solidifiable state; product packaging processing is performed on the deployment completed software product data in the solidifiable state, and a product packaging result is formed; output organization processing is performed on the product packaging result, and an outputtable product result is formed; the outputtable product result is solidified and output, and a development completed software product is obtained, and the development completed software product is output.

[0084] Embodiment one: in this embodiment, the development input collection module receives source code data, build script data, dependency declaration data, configuration file data and development environment description data, performs integrity checking processing on the source code data and generates checked source code data, performs structure parsing processing on the build script data and generates parsed build script data, performs dependency item parsing processing on the dependency declaration data and generates dependency parsing data, performs parameter extraction processing on the configuration file data and generates configuration parameter data, performs environment element parsing processing on the development environment description data and generates environment element data, and performs merging processing on the checked source code data, parsed build script data, dependency parsing data, configuration parameter data and environment element data, and forms a standardized development input data set.

[0085] In this embodiment, the version fingerprint construction module receives the checked source code data, the parsed build script data, the dependency parsing data, the configuration parameter data and the environment element data, performs code feature digest calculation processing on the source code data to form code feature digest data, performs script feature digest calculation processing on the parsed build script data to form script feature digest data, performs dependency feature digest calculation processing on the dependency parsing data to form dependency feature digest data, performs configuration feature digest calculation processing on the configuration parameter data to form configuration feature digest data, performs environment feature digest calculation processing on the environment element data to form environment feature digest data, and performs aggregation calculation processing on the code feature digest data, the script feature digest data, the dependency feature digest data, the configuration feature digest data and the environment feature digest data to form version fingerprint data.

[0086] In this embodiment, the job semantic modeling module receives the version fingerprint data and performs feature unpacking and feature index calculation to form version feature indexes, matches the version feature indexes with preset parameter mapping rules one by one to form mapping results, the system determines that the version feature indexes fall into interval I, and determines that the analysis confidence threshold parameter value is 0.60, the dependency determination weight parameter value is 1.00, the semantic rule strength parameter value is 0.60, and the version binding consistency threshold parameter value is 0.60 from the parameter value set corresponding to interval I; the job semantic modeling module performs step parsing processing on the parsed build script data and completes step retention determination based on the analysis confidence threshold parameter to form execution step description data, performs operator abstraction processing on the execution step description data to form operator description data, performs directed relationship modeling processing on the operator description data and weights the dependency clues according to the dependency determination weight parameter to form operator dependency relationship data, performs unified modeling processing on the data input description, the data output description and the operator dependency relationship data to form job abstraction data, performs operator semantic rule verification processing on the job abstraction data and forms operator semantic verification result data according to the semantic rule strength parameter, performs data contract consistency verification processing on the job abstraction data and forms data contract verification result data according to the semantic rule strength parameter, performs binding relationship consistency verification processing on the job abstraction data in combination with the version fingerprint data and forms version binding verification result data according to the version binding consistency threshold parameter, performs summary processing on the three types of verification results to form consistency verification result data, and merges and solidifies the consistency verification result data and the job abstraction data to form job semantic data.

[0087] In this embodiment, the change impact analysis module receives the job semantic data and the version fingerprint data, performs feature unpacking and feature index calculation, and forms a mapping result of the version fingerprint data. The system determines that the version feature index falls into interval I, and determines from the parameter value set corresponding to interval I that the propagation depth control parameter value is 2, the edge weight weighting parameter value is 0.80, the propagation cut-off threshold parameter value is 0.20, the path merging strategy parameter value is the maximum value merging, and the change source confidence parameter value is 0.60. The change impact analysis module identifies the change node set based on the version fingerprint data, and configures the initial impact strength according to the change source confidence parameter. Then, the change impact analysis module propagates layer by layer in the direction of the directed edge of the operator dependency data according to the propagation depth control parameter, calculates the propagation edge weight according to the edge weight weighting parameter, and generates the impact propagation result data. The change impact analysis module expands the reachable path according to the impact propagation result data, forms the impact path set, performs path aggregation on the impact path set, forms the impact range candidate set, forms the change impact evaluation data according to the propagation cut-off threshold parameter and the path merging strategy parameter, and finally performs the impact degree judgment processing on the change impact evaluation data, and compares the change impact evaluation data with the preset release control rule to generate the release control decision and solidify the output as the release control instruction data.

[0088] In this embodiment, the big data processing module receives the job semantic data, the change impact evaluation data and the historical running record data, forms the historical running statistical feature data, the job execution feature data and the change impact feature data, and outputs the job running feature data. The adaptive resource concurrency control module receives the job running feature data and the release control instruction data, and outputs the running control strategy data under the condition that the release control instruction data allows execution. The software artifact output module receives the version fingerprint data, the release control instruction data and the running control strategy data, completes the build process control, deployment processing and result solidification under the condition that the release is allowed, and outputs the developed software artifact.

[0089] In this embodiment, the version feature index formed by the change impact analysis module is determined to fall into interval II, and the system determines the propagation depth control parameter value to be 3, the edge weight weighting parameter value to be 1.00, the propagation cut-off threshold parameter value to be 0.30, the path merging strategy parameter value to be weighted merging, and the change source confidence parameter value to be 0.70 from the parameter value set corresponding to interval II. The change impact analysis module takes the change node set as the propagation source, configures a higher initial impact strength according to the higher change source confidence parameter, allows deeper level propagation according to the higher propagation depth control parameter, eliminates low-intensity propagation paths according to the higher propagation cut-off threshold parameter, merges the multi-path arrival results of the same target node according to the weighted merging strategy to form a single target node propagation state, and further forms change impact evaluation data and solidifies output release control instruction data.

[0090] In this embodiment, the version feature index formed by the change impact analysis module is determined to fall into interval II, and the system determines the propagation depth control parameter value to be 3, the edge weight weighting parameter value to be 1.00, the propagation cut-off threshold parameter value to be 0.30, the path merging strategy parameter value to be weighted merging, and the change source confidence parameter value to be 0.70 from the parameter value set corresponding to interval II. The change impact analysis module takes the change node set as the propagation source, configures a higher initial impact strength according to the higher change source confidence parameter, allows deeper level propagation according to the higher propagation depth control parameter, eliminates low-intensity propagation paths according to the higher propagation cut-off threshold parameter, merges the multi-path arrival results of the same target node according to the weighted merging strategy to form a single target node propagation state, and further forms change impact evaluation data and solidifies output release control instruction data.

[0091] In this embodiment, the input and output links of the big data processing module, the adaptive resource concurrency control module, and the software product output module are consistent with those in Embodiment One. The system outputs running control strategy data under the condition that the release control instruction data is allowed to be executed, and performs adaptive adjustment on the computing resource allocation parameters and the job concurrency degree parameters in the deployment stage according to the running control strategy data, so as to balance the release safety and resource utilization efficiency under the condition of medium-low difference version change.

[0092] In this embodiment, the version feature index determined by the change impact analysis module falls into interval III, and the system determines the propagation depth control parameter value as 4, the edge weight weighting parameter value as 1.20, the propagation cut-off threshold parameter value as 0.40, the path merging strategy parameter value as average value merging, and the change source confidence parameter value as 0.80 from the parameter value set corresponding to interval III. In the impact propagation analysis process, the system allows a deeper propagation level and performs stronger edge weight weighting on the propagation edge, adopts a higher propagation cut-off threshold for the cumulative impact intensity to eliminate weak propagation paths, and generates a single propagation state by using the average value merging strategy when multiple paths reach the same target node, so that the impact domain and the impact path result are more biased towards robustness, thereby forming more stringent change impact evaluation data and generating release control instruction data.

[0093] In this embodiment, the version feature index determined by the change impact analysis module falls into interval III, and the system determines the propagation depth control parameter value as 4, the edge weight weighting parameter value as 1.20, the propagation cut-off threshold parameter value as 0.40, the path merging strategy parameter value as average value merging, and the change source confidence parameter value as 0.80 from the parameter value set corresponding to interval III. In the impact propagation analysis process, the system allows a deeper propagation level and performs stronger edge weight weighting on the propagation edge, adopts a higher propagation cut-off threshold for the cumulative impact intensity to eliminate weak propagation paths, and generates a single propagation state by using the average value merging strategy when multiple paths reach the same target node, so that the impact domain and the impact path result are more biased towards robustness, thereby forming more stringent change impact evaluation data and generating release control instruction data.

[0094] In this embodiment, after the big data processing module forms the job running feature data, the self-adaptive resource concurrency control module performs joint evaluation on the resource load index data and the concurrency pressure index data under the condition that the release control instruction data allows execution, and outputs running control strategy data. The software artifact output module performs resource binding and concurrency control on the deployment task sequence based on the running control strategy data and completes deployment solidification, thereby improving release stability and reducing resource congestion risk in the deployment phase under the condition of medium-high difference version change.

[0095] In this embodiment, the input and output link of the input collection module and the version fingerprint construction module is developed consistent with embodiment one; the job semantic modeling module forms version feature indicators from version fingerprint data and determines that they fall into interval IV, the system determines from the parameter value set corresponding to interval IV that the resolution confidence threshold parameter value is 0.90, the dependency determination weight parameter value is 1.80, the semantic rule strength parameter value is 0.90, and the version binding consistency threshold parameter value is 0.90; the job semantic modeling module performs step retention determination and dependency relationship modeling with the highest strictness, and adopts the determination boundary of the highest strictness in operator semantic rule checking, data contract consistency checking and binding relationship consistency verification, and forms consistency checking result data after performing consistency merging and conflict resolution on the checking results, and solidifies and outputs job semantic data, thereby maximizing the reduction of job semantic modeling bias under high-difference version conditions.

[0096] In this embodiment, the change impact analysis module determines that the version feature indicators fall into interval IV, and the system determines from the parameter value set corresponding to interval IV that the propagation depth control parameter value is 5, the edge weight weighting parameter value is 1.50, the propagation cutoff threshold parameter value is 0.50, the path merging strategy parameter value is priority merging, and the change source confidence parameter value is 0.90; the change impact analysis module configures a higher initial impact strength based on a higher change source confidence parameter, allows a deeper propagation level and applies a higher cutoff threshold to the propagation path to quickly eliminate low-intensity propagation paths, while adopting a priority merging strategy to merge the propagation results reached by multiple paths, so that the impact domain result is more biased towards high-risk priority coverage, and accordingly forms change impact evaluation data and release control instruction data.

[0097] In this embodiment, after the adaptive resource concurrency control module receives the job running feature data and the release control instruction data, it preferentially performs the permissibility determination of the release control instruction data; when the permissibility determination result does not satisfy the execution condition, the original values of the computing resource allocation parameter and the job concurrency degree parameter are maintained and solidified as running control strategy data; after the software artifact output module receives the release control instruction data, if the construction process is in an unallowable execution state, the construction process is terminated and the termination result is solidified to form construction execution result data, thereby realizing risk isolation and release safety control with release control decision as the priority constraint under high-difference version change conditions.

[0098] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalent features, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A software development cloud computing system with big data processing module, characterized in that: Comprise: Develop input acquisition module, for receiving source code data, build script data, dependency declaration data, configuration file data and development environment description data, respectively perform integrity check, structure analysis, dependency analysis, parameter extraction and environment element analysis processing, form standardized development input data set; Version fingerprint construction module, for receiving standardized development input data set, source code data, build script data, dependency analysis data, configuration parameter data and environment element data are respectively calculated and aggregated, forming version fingerprint data; Job semantic modeling module, for receiving standardized development input data set and version fingerprint data, based on version fingerprint data, feature unpacking and version feature index calculation are performed, and interval judgment processing is performed on version feature index, version feature index is judged as one of four preset intervals, and based on interval judgment result, analysis confidence threshold parameter, dependency judgment weight parameter, semantic rule strength parameter and version binding consistency threshold parameter are determined, and then using build script static analysis abstraction algorithm, step analysis, operator abstraction and dependency relationship modeling are performed on build script data, combined with data input description and data output description, job abstraction data is formed, and operator semantic rule check, data contract consistency check and binding relationship consistency verification are performed on job abstraction data, forming job semantic data; Change impact analysis module, for receiving job semantic data and version fingerprint data, based on version fingerprint data, feature unpacking and version feature index calculation are performed, and interval judgment processing is performed on version feature index, version feature index is judged as one of four preset intervals, and based on interval judgment result, propagation depth control parameter, edge weight weighting parameter, propagation cutoff threshold parameter, path merging strategy parameter and change source confidence parameter are determined, and then using dependency propagation influence domain calculation algorithm, change node is identified and influence propagation and influence range calculation are performed based on operator dependency relationship, forming change impact evaluation data, and generating release control instruction data according to preset release control rule; Big data processing module, for receiving job semantic data, change impact evaluation data and historical running record data, performing distributed statistical analysis and feature aggregation processing, forming job running feature data; Adaptive resource concurrent control module, for receiving job running feature data and release control instruction data, calculating resource load index and concurrent pressure index, and generating running control strategy data under the condition allowed by release control instruction; Software product output module, for receiving version fingerprint data, release control instruction data and running control strategy data, controlling and deploying the build process of standardized development input data set to software product according to release control instruction, writing version fingerprint data into software product meta information, obtaining developed software product. 2.The software development cloud computing system with big data processing module of claim 1, wherein: The development input acquisition module receives source code data, build script data, dependency declaration data, configuration file data and development environment description data, and performs integrity check processing on the source code data to generate checked source code data; Perform structural analysis processing on the build script data to generate parsed build script data; Perform dependency parsing processing on the dependency declaration data to generate dependency parsing data; Perform parameter extraction processing on the configuration file data to generate configuration parameter data; Perform environment element parsing processing on the development environment description data to generate environment element data; Merge the verified source code data, parsed build script data, dependency parsing data, configuration parameter data, and environment element data to form a standardized development input data set. 3.The software development cloud computing system with big data processing module of claim 1, wherein: The version fingerprint construction module performs code feature summary calculation processing on the source code data to obtain code feature summary data; The version fingerprint construction module performs code feature summary calculation processing on the source code data to obtain code feature summary data; The version fingerprint construction module performs code feature summary calculation processing on the source code data to obtain code feature summary data; The version fingerprint construction module performs code feature summary calculation processing on the source code data to obtain code feature summary data; The version fingerprint construction module performs code feature summary calculation processing on the source code data to obtain code feature summary data; The version fingerprint construction module performs code feature summary calculation processing on the source code data to obtain code feature summary data.

4. The software development cloud computing system with big data processing module according to claim 1, characterized in that: The job semantic modeling module performs feature unpacking and feature index calculation based on the version fingerprint data to obtain version feature indexes, and compares the version feature indexes with the preset interval determination logic, determines the interval into which the version feature indexes fall based on the size relationship between the version feature indexes and the three interval boundary values; Based on the interval, the version feature indexes are matched with the preset parameter mapping rule to determine the parameter values corresponding to the current version features, forming a mapping result including the analysis confidence threshold parameter, the dependency determination weight parameter, the semantic rule strength parameter, and the version binding consistency threshold parameter; Perform execution step parsing processing on the parsed build script data to generate a candidate execution step set, and calculate the analysis confidence for each candidate execution step. Compare the analysis confidence with the analysis confidence threshold parameter, retain the candidate execution steps that meet the threshold condition and eliminate the candidate execution steps that do not meet the threshold condition to obtain execution step description data; Perform operator abstraction processing on the execution step description data to map each execution step to the corresponding operator semantic unit and generate operator identification and operator attribute description to obtain operator description data; Perform directed relationship modeling processing on the execution dependency relationship between the operators in the operator description data, extract dependency clues for each pair of candidate dependency relationships and perform weighted calculation according to the dependency determination weight parameter, determine the existence and direction of the dependency edge based on the weighted result to obtain operator dependency relationship data; Perform unified modeling processing on the data input description and data output description obtained in the execution step parsing process and the operator dependency relationship data, bind the data input description and data output description to the corresponding operator identification, and organize the operator dependency relationship data into an executable topology structure to form job abstraction data; The operator semantic rule checking process is performed on the job abstract data, semantic rule sets are selected and rule trigger boundaries are determined according to the semantic rule strength parameters, semantic consistency judgment and constraint satisfaction judgment are performed on each operator, and operator semantic checking result data is obtained; The data contract consistency checking process is performed on the data input description and data output description in the job abstract data, the contract checking boundaries are determined according to the semantic rule strength parameters, consistency judgment is performed on the data structure constraint, field constraint and dependency constraint of the input and output, and data contract checking result data is obtained; The binding relationship consistency verification process is performed on the job abstract data in combination with the version fingerprint data, the target binding feature is determined based on the version fingerprint data, and the consistency judgment boundary is determined according to the version binding consistency threshold parameter, the binding feature of the job abstract data is matched with the target binding feature, and deviation judgment is performed, and version binding checking result data is obtained; The operator semantic checking result data, the data contract checking result data and the version binding checking result data are summarized, the summary decision boundary is determined according to the semantic rule strength parameter and the version binding consistency threshold parameter, and the consistency checking result data is formed; The job semantic data is obtained by merging the job abstract data and the consistency checking result data.

5. The software development cloud computing system with big data processing module according to claim 1, characterized in that: The feature index is calculated based on the version fingerprint data, and the version feature index is compared with the preset interval judgment logic, and the interval into which the version feature index falls is determined according to the size relationship between the version feature index and the three interval boundary values; The version feature index is matched with the preset parameter mapping rule based on the interval judgment result, the parameter value corresponding to the current version feature is determined, and the parameter value is collected to form the mapping result of the version fingerprint data, and the mapping result includes the propagation depth control parameter, the edge weight weighting parameter, the propagation cutoff threshold parameter, the path merging strategy parameter and the change source confidence parameter; The node analysis result data is obtained by performing operator node traversal analysis on the job semantic data, the change node set is identified based on the version fingerprint data, and the starting influence strength is configured for each change node in the change node set according to the change source confidence parameter; The influence propagation result data is obtained by performing operator dependency relationship influence propagation analysis on the node analysis result data, the propagation level is limited according to the propagation depth control parameter, the propagation edge is weighted according to the edge weight weighting parameter, the cumulative influence strength is threshold truncated according to the propagation cutoff threshold parameter, and the multi-path propagation result is merged according to the path merging strategy parameter; The execution path influence range calculation process is performed on the influence propagation result data in combination with the version fingerprint data, and the influence range calculation result is subjected to consistency constraint processing according to the propagation depth control parameter, the propagation cutoff threshold parameter and the path merging strategy parameter, and the change influence evaluation data is formed; The influence degree is determined by performing influence degree judgment processing on the change influence evaluation data, and the influence degree is compared with the preset release control rule, and the release control decision is generated according to the comparison result, and the release control instruction data is obtained.

6. The software development cloud computing system with big data processing module according to claim 1, characterized in that: The big data processing module performs distributed statistical analysis and processing on the historical operation record data, extracts job historical execution frequency characteristics, resource usage distribution characteristics, and execution time length statistical characteristics, and forms historical operation statistical characteristic data; The job semantic data is subjected to job execution characteristic aggregation processing, and the operator quantity characteristics, dependency level characteristics, and execution path structure characteristics are extracted to form job execution characteristic data; The change impact assessment data is subjected to correlation characteristic analysis processing, and the impact range characteristics, impact path characteristics, and impact degree characteristics are extracted to form change impact characteristic data; The historical operation statistical characteristic data, job execution characteristic data, and change impact characteristic data are subjected to merging processing to form job operation characteristic data, and the job operation characteristic data is output.

7. The software development cloud computing system with big data processing module of claim 1, wherein: The adaptive resource concurrency control module performs resource load index calculation processing on the job operation characteristic data to generate resource load index data; The job operation characteristic data is subjected to concurrency pressure index calculation processing to generate concurrency pressure index data; Under the condition that the release control instruction data allows execution, the resource load index data and the concurrency pressure index data are used to perform adaptive adjustment processing on the computing resource allocation parameters and the job concurrency degree parameters to form operation control strategy data; The operation control strategy data is output to the software artifact output module to control the execution mode of the construction process and the deployment process.

8. The software development cloud computing system with big data processing module according to claim 1, wherein: The software artifact output module performs control processing on the construction process for constructing the standardized development input data set into a software artifact according to the release control instruction data under the condition that the release control instruction data allows execution, and obtains construction execution result data; The construction execution result data is subjected to deployment processing according to the operation control strategy data, and the version fingerprint data is written into the software artifact meta information to form software artifact data after deployment is completed; The software artifact data after deployment is completed is subjected to result solidification processing to obtain a developed software artifact, and the developed software artifact is output.

Images