A software development process optimization method based on big data processing

Abstract

This invention discloses a software development process optimization method based on big data processing, comprising the following steps: collecting software development process data, performing correlation and organization to generate a development process dataset; generating a composite event sequence based on the development process dataset; constructing a process write-back neural Hawkes model based on a neural Hawkes process; generating an event state sequence based on the composite event sequence by performing code aggregation and continuous-time state evolution; generating a process propagation result based on the event state sequence; generating process adjustment actions based on the process propagation result, performing state write-back updates to generate a write-back state sequence; and determining the order of requirement processing, code review, build execution, test execution, and release execution based on the write-back state sequence, thereby generating a process optimization scheme. This invention improves the continuity, relevance, and executability of software development process optimization.

Description

Technical Field

[0001] This invention relates to the field of software engineering, and in particular to a method for optimizing software development processes based on big data processing. Background Technology

[0002] Software development process management involves multiple stages, including requirements processing, code review, build execution, test execution, defect handling, and release execution. As the scale of R&D expands, the types of data generated during R&D activities continue to increase. This data is scattered across requirements management, code repositories, continuous integration, test management, defect management, release management, and collaboration platforms. The heterogeneous data sources, inconsistent time granularities, and fragmented relationships make it difficult for existing software development process optimization methods to conduct a unified and continuous overall analysis of the development process.

[0003] In existing technologies, one type of approach focuses on statistical analysis of data from a single stage, including independent calculations of code submissions, test results, defect counts, and build logs; another type focuses on prioritizing and adjusting the development process according to fixed rules. While these approaches can provide auxiliary judgment at certain stages, they typically lack a systematic characterization of the sequential, propagation, and dynamic impact relationships between multi-source development data, making it difficult to accurately reflect the transmission of process bottlenecks between different stages.

[0004] Furthermore, existing technologies for handling software development processes mostly remain at the level of static analysis, single prediction, or local scheduling, lacking a continuous update mechanism for changes in process state. This makes it difficult to adaptively adjust the subsequent sequence based on the process evolution results, and easily leads to the problem of process optimization results being out of sync with the actual development state.

[0005] Therefore, how to provide a software development process optimization method based on big data processing is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a software development process optimization method based on big data processing. This invention generates a development process dataset based on software development process data, further generates a composite event sequence, and constructs a process write-back neural Hawkes model based on neural Hawkes processes. By coupling event encoding units, continuous-time state units, hierarchical propagation intensity units, adjustment action generation units, and write-back update units, the composite event sequence is processed to generate an event state sequence, process propagation results, and a write-back state sequence. This determines the order of requirement processing, code review, build execution, test execution, and release execution, thus generating a process optimization scheme.

[0007] A software development process optimization method based on big data processing according to an embodiment of the present invention includes the following steps: Collect software development process data, perform correlation and organization, and generate a development process dataset; Based on the development process dataset, the interrelated software development process data are aggregated to generate composite development activity units and organize composite event sequences. Based on the neural Hawkes process, a neural Hawkes process write-back model is constructed, including a coupled event encoding unit, a continuous time state unit, a hierarchical propagation intensity unit, a modulating action generation unit, and a write-back update unit. Based on the composite event sequence, in the coupled event coding unit, coding aggregation is performed to generate a composite event representation, and the composite event representation is input into the continuous-time state unit to perform continuous-time state evolution and generate an event state sequence; Input the event state sequence into the hierarchical propagation strength unit to construct the hierarchical propagation strength and generate the process propagation result; In the adjustment action generation unit, the process blocking location, propagation path, and adjustment sequence are determined based on the process propagation results, process adjustment actions are generated, and input into the write-back update unit to perform state write-back update and generate a write-back state sequence. Based on the write-back state sequence, the order of requirement processing, code review, build execution, test execution, and release execution is determined, and a process optimization plan is generated.

[0008] Optionally, the generated development process dataset includes: Collect requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data to generate software development process data; Based on software development process data, perform structured transformation to generate development process records; Based on the development process records, the requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, and release execution data are concatenated to generate requirement flow chain, code change chain, review processing chain, build execution chain, test execution chain, defect handling chain, and release execution chain. Personnel collaboration data are mapped to the corresponding development process records in each chain to generate associated development process records. Based on the associated development process records, the sorting and rearrangement are performed to generate a time-series development process record; Based on the time-series development process records, the process is merged and organized to generate a development process dataset.

[0009] Optionally, generating the composite event sequence includes: Based on the development process dataset, according to the correspondence of development processes, perform correlation filtering to extract data records that belong to the same development process and have a sequential relationship, and generate a development process aggregation window; Based on the development process aggregation window, merge mapping is performed according to fixed positions to generate composite development activity units, including the corresponding positions of requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data. Based on the composite development activity unit, the structured organization is executed according to the preset arrangement order to complete the internal arrangement of the composite development activity unit; Based on the event occurrence time field in the development process aggregation window, composite development activity units are organized and executed in chronological order of occurrence to generate a composite event sequence.

[0010] Optionally, the construction process of writing back the neural Hawkes model includes: Based on the event embedding structure, continuous-time hidden state evolution structure, and conditional strength modeling structure of the neural Hawkes process; Transform the event embedding structure into a coupled event coding unit, setting up a requirement data coding layer, a code submission data coding layer, a code review data coding layer, a build execution data coding layer, a test execution data coding layer, a defect handling data coding layer, a release execution data coding layer, and a personnel collaboration data coding layer, and connect the output of each layer to the aggregation mapping layer. The continuous-time hidden state evolution structure is transformed into a continuous-time state unit, and an event arrival update layer, a time interval evolution layer, and a state output layer are set. The conditional strength modeling structure is transformed into a layered propagation strength unit, with a task advancement layer propagation sub-layer, a quality blocking layer propagation sub-layer, a delivery and release layer propagation sub-layer, and a propagation result convergence layer. Based on the hierarchical propagation intensity unit, an adjustment action generation unit is constructed after the propagation result convergence layer, and a blocking position extraction layer, a propagation path extraction layer, and an adjustment order generation layer are set. Based on the continuous time state unit, a write-back update unit is constructed after adjusting the action generation unit, and an action mapping layer and a state write-back layer are set. Build the connection relationships between the units and generate a neural Hawkes model for process writeback.

[0011] Optionally, the generated event state sequence includes: Based on the composite event sequence, each composite development activity unit is input into the requirement data coding layer, code submission data coding layer, code review data coding layer, build execution data coding layer, test execution data coding layer, defect handling data coding layer, release execution data coding layer, and personnel collaboration data coding layer. The coding process is performed on each layer to generate requirement data coding results, code submission data coding results, code review data coding results, build execution data coding results, test execution data coding results, defect handling data coding results, release execution data coding results, and personnel collaboration data coding results. These results are then combined into sub-item coding results. Based on the itemized coding results, each coding result is input into the aggregation mapping layer. The aggregation mapping is performed according to the fixed position and preset arrangement order in the composite development activity unit to generate a composite event representation composed of the itemized coding results. Based on the time sequence of occurrence in the composite event sequence, the composite event representations are sequentially input into the event arrival update layer in the continuous time state unit. The current composite event representation is updated with the state result corresponding to the previous composite event representation to generate the event arrival state. Based on the event occurrence time intervals corresponding to adjacent composite development activity units in the composite event sequence, the event arrival state is input into the time interval evolution layer, and continuous time state evolution is performed according to the event occurrence time interval to generate time evolution state; Based on the time evolution state, the input state output layer sequentially outputs the state results corresponding to each composite development activity unit, generating an event state sequence.

[0012] Optionally, the propagation results of the generation process include: The event state sequence is input into the hierarchical propagation intensity unit. According to the data source in the composite development activity unit, the event state corresponding to each data is extracted to generate the task advancement layer input state, quality blocking layer input state and delivery release layer input state. The task propagation layer input state is input into the task propagation layer propagation sublayer, and sequential propagation calculation is performed to generate the task propagation layer propagation strength. The mass blocking layer input state and the mission propagation layer propagation strength are input into the mass blocking layer propagation sublayer for sequential propagation calculation, and inter-layer transfer calculation is performed in combination with the mission propagation layer propagation strength to generate the mass blocking layer propagation strength. The delivery release layer input state and the quality blocking layer propagation strength are input into the delivery release layer propagation sublayer, sequential propagation calculation is performed, and inter-layer propagation calculation is performed in combination with the quality blocking layer propagation strength to generate the delivery release layer propagation strength. The propagation strength of the task advancement layer, the propagation strength of the quality blocking layer, and the propagation strength of the delivery release layer are input into the propagation result aggregation layer. Aggregation processing is performed to generate process propagation results. The process propagation results include the propagation strength of the task advancement layer, the propagation strength of the quality blocking layer, the propagation strength of the delivery release layer, the inter-layer transmission direction, and the degree of delivery impact corresponding to the current development process.

[0013] Optionally, generating the write-back state sequence includes: The process propagation results are input into the blocking location extraction layer in the adjustment action generation unit. According to the development process sequence corresponding to the event state sequence, the concentrated location of propagation intensity and the concentrated location of delivery impact are extracted to determine the process blocking location. Input the process propagation results and process blockage locations into the propagation path extraction layer. Based on the inter-layer transmission direction, the occurrence time sequence corresponding to the event state sequence, and the sequential relationship between the process blockage locations, extract the transmission link from the propagation intensity of the task advancement layer to the propagation intensity of the quality blockage layer, and then to the propagation intensity of the delivery and release layer to determine the propagation path. Input the process propagation results, process blockage locations, and propagation paths into the adjustment order generation layer. Based on the order of occurrence of process blockage locations, the order of transmission of propagation paths, and the order of delivery impact, determine the adjustment order. Based on the adjustment order, the execution order of requirement data, code review data, build execution data, test execution data, and release execution data corresponding to the process blockage position is adjusted to generate process adjustment actions; Input the process adjustment actions and event state sequences into the action mapping layer in the write-back update unit, map the execution position of the process adjustment actions, and generate state mapping results; The state mapping results are input into the state write-back layer. The state content corresponding to the task processing adjustment, the code review adjustment, the build execution adjustment, the test execution adjustment, and the release execution adjustment in the event state sequence is corrected and the order is updated to generate the write-back state sequence.

[0014] Optionally, the optimization scheme for the generation process includes: Based on the order of the status content corresponding to task processing adjustment, the status content corresponding to build execution adjustment, and the status content corresponding to test execution adjustment in the write-back status sequence, as well as the order of adjustment of status correction results and corresponding process blocking positions, the order of execution is determined by sorting the status content before and after execution. Based on the order of the state content corresponding to the code review adjustment in the write-back state sequence, the order of state correction results and the order of corresponding propagation path adjustments, the code review order is determined by sorting the execution order. Based on the order of the status content corresponding to the release execution adjustment in the write-back status sequence, the order of the status correction results and the order of the corresponding delivery impact, the execution is sorted to determine the release execution order; Based on the order of requirement processing, code review, build execution, test execution, and release execution, the processes are merged and organized to generate an optimized solution.

[0015] The beneficial effects of this invention are: First, this invention generates a development process dataset based on software development process data, and further generates a composite event sequence. This integrates previously scattered requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data into a unified development process organizational chain. This improves the continuity and overall correlation between software development process data, avoiding the process fragmentation caused by processing data from individual stages separately in existing technologies. Thus, it provides a structurally consistent and temporally continuous data foundation for subsequent process propagation analysis and the generation of process optimization solutions.

[0016] Secondly, this invention constructs a neural Hawkes model for process write-back based on the neural Hawkes process. By coupling event encoding units, continuous-time state units, and hierarchical propagation strength units, it performs continuous-time modeling and hierarchical propagation analysis on composite event sequences. This enables the simultaneous characterization of the transmission relationship between the propagation strength of the task advancement layer, the propagation strength of the quality blocking layer, and the propagation strength of the delivery release layer. This improves the ability to identify the propagation direction, blocking diffusion location, and delivery impact of the development process, avoiding the problem that existing technologies can only provide static analysis results or single prediction results. This improves the pertinence and continuity of software development process optimization.

[0017] Furthermore, by adjusting the action generation unit and the write-back update unit, this invention further transforms the process propagation results into process adjustment actions, and performs state write-back updates on the event state sequence based on the process adjustment actions. Ultimately, it determines the order of requirement processing, code review, build execution, test execution, and release execution, and generates a process optimization scheme. This can improve the consistency between the process optimization results and the actual state of the development process, avoid the problem of the optimization results being disconnected from subsequent process changes in the prior art, and thus improve the executability of software development process adjustments and the overall optimization effect. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1This is an overall flowchart of a software development process optimization method based on big data processing proposed in this invention; Figure 2 This is a schematic diagram of the neural Hawkes model for process writeback in this invention; Figure 3 This is a flowchart illustrating the process of generating process propagation results, generating process adjustment actions, and generating write-back state sequences in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-3 A software development process optimization method based on big data processing includes the following steps: Collect software development process data, which includes requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data. Organize and associate the software development process data according to the corresponding relationships of the development process and the order of occurrence to generate a development process dataset. Based on the development process dataset, interrelated requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data are aggregated to generate composite development activity units. The composite development activity units are then organized according to their chronological order to generate composite event sequences. Based on the neural Hawkes process, a process write-back neural Hawkes model is constructed. The process write-back neural Hawkes model includes a coupled event encoding unit, a continuous time state unit, a hierarchical propagation intensity unit, a regulation action generation unit, and a write-back update unit. The coupled event encoding unit, continuous time state unit, hierarchical propagation intensity unit, regulation action generation unit, and write-back update unit are connected in sequence, and the write-back update unit is connected to the continuous time state unit. Based on the composite event sequence, in the coupled event coding unit, the composite development activity unit is encoded and aggregated to generate a composite event representation. The composite event representation is then input into the continuous-time state unit, and continuous-time state evolution is performed based on the arrival order and time interval in the composite event sequence to generate an event state sequence. Input the event state sequence into the hierarchical propagation strength unit, construct the propagation strength of the task advancement layer, the propagation strength of the quality blocking layer, and the propagation strength of the delivery and release layer based on the event state sequence, and generate the process propagation results; In the adjustment action generation unit, the process blocking location, propagation path, and adjustment sequence are determined based on the process propagation results, and process adjustment actions are generated. The process adjustment actions are then input into the write-back update unit, and the state write-back update is performed in conjunction with the event state sequence to generate the write-back state sequence. Based on the write-back state sequence, the order of requirement processing, code review, build execution, test execution, and release execution is determined, and a process optimization plan is generated.

[0021] In this embodiment, generating the development process dataset includes: Collect requirement data from requirement management logs, code commit data from code repository logs, code review data from code review logs, build execution data from continuous integration pipeline logs, test execution data from test execution logs, defect handling data from defect handling logs, release execution data from release execution logs, and personnel collaboration data from collaboration interaction logs to generate software development process data. Based on software development process data, requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data are structurally transformed according to a unified record structure to generate development process records. The unified record structure includes event type field, event occurrence time field, object identifier field, status field, and participating role field. The development process records are organized according to fixed-length field vectors, discrete marker indexes, and continuous timestamp sequences. Based on the object identifier, status, and participating role fields in the development process records, the development process records corresponding to the requirement data are concatenated according to the flow order of the requirement objects to generate a requirement flow chain. Based on the development process records corresponding to the code submission data, concatenation is performed according to the submission order and version change relationship of the code objects to generate a code change chain. Based on the development process records corresponding to the code review data, concatenation is performed according to the processing order and review status change relationship of the review objects to generate a review processing chain. Based on the development process records corresponding to the build execution data, concatenation is performed according to the triggering order and execution status change relationship of the build tasks to generate a build execution chain. Based on the development process records corresponding to the test execution data... The development process records are linked according to the execution sequence and result status change relationship of test tasks to generate a test execution chain. Based on the development process records corresponding to defect handling data, they are linked according to the processing sequence and status flow relationship of defect objects to generate a defect handling chain. Based on the development process records corresponding to release execution data, they are linked according to the execution sequence and release status change relationship of release tasks to generate a release execution chain. The development process records corresponding to personnel collaboration data are mapped according to the participating role field to the corresponding development process records in the requirement flow chain, code change chain, review processing chain, build execution chain, test execution chain, defect handling chain, and release execution chain to generate associated development process records. Based on the event occurrence time field in the associated development process records, the associated development process records are sorted and rearranged according to a unified timestamp sequence to generate time-series development process records. Based on the time-series development process records, the corresponding development process relationships of the requirement flow chain, code change chain, review and processing chain, build execution chain, test execution chain, defect handling chain, and release execution chain are merged and organized to generate a development process dataset.

[0022] In this embodiment, generating a composite event sequence includes: Based on the development process dataset, according to the correspondence of development processes in the development process dataset, the requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data are correlated and filtered to extract data records that belong to the same development process and have a sequential relationship, and generate a development process aggregation window. Based on the development process aggregation window, requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data are merged and mapped according to fixed positions to generate composite development activity units that include the corresponding positions of requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data. Based on the composite development activity unit, the corresponding positions of requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data are structured and organized according to a preset order to complete the internal arrangement of the composite development activity unit. Based on the event occurrence time field in the development process aggregation window, composite development activity units are organized and executed in chronological order of occurrence to generate a composite event sequence.

[0023] In this embodiment, constructing the neural Hawkes model for process writeback includes: Based on the event embedding structure, continuous-time hidden state evolution structure, and conditional intensity modeling structure of the neural Hawkes process, the event arrival update method, time interval evolution method, and hidden state transmission method in the continuous-time hidden state evolution structure are retained, as are the state input method and intensity output method in the conditional intensity modeling structure. The event embedding structure is transformed into a coupled event coding unit. In the coupled event coding unit, a requirement data coding layer, a code submission data coding layer, a code review data coding layer, a build execution data coding layer, a test execution data coding layer, a defect handling data coding layer, a release execution data coding layer, and a personnel collaboration data coding layer are set. The outputs of the requirement data coding layer, code submission data coding layer, code review data coding layer, build execution data coding layer, test execution data coding layer, defect handling data coding layer, release execution data coding layer, and personnel collaboration data coding layer are connected to the aggregation mapping layer. The continuous-time hidden state evolution structure is transformed into a continuous-time state unit. In the continuous-time state unit, an event arrival update layer, a time interval evolution layer, and a state output layer are set. The output of the event arrival update layer is connected to the time interval evolution layer, and the output of the time interval evolution layer is connected to the state output layer. The conditional strength modeling structure is transformed into a hierarchical propagation strength unit. In the hierarchical propagation strength unit, a task advancement layer propagation sub-layer, a quality blocking layer propagation sub-layer, a delivery release layer propagation sub-layer, and a propagation result aggregation layer are set. The input ends of the task advancement layer propagation sub-layer, the quality blocking layer propagation sub-layer, and the delivery release layer propagation sub-layer are connected to the state output layer of the continuous time state unit, and the output ends of the task advancement layer propagation sub-layer, the quality blocking layer propagation sub-layer, and the delivery release layer propagation sub-layer are connected to the propagation result aggregation layer. Based on the hierarchical propagation intensity unit, an adjustment action generation unit is constructed after the propagation result convergence layer. In the adjustment action generation unit, a blocking position extraction layer, a propagation path extraction layer, and an adjustment order generation layer are set. The outputs of the blocking position extraction layer and the propagation path extraction layer are connected to the adjustment order generation layer. Based on the continuous time state unit, a write-back update unit is constructed after the adjustment action generation unit. An action mapping layer and a state write-back layer are set in the write-back update unit. The output of the adjustment action generation unit is connected to the action mapping layer, the output of the action mapping layer is connected to the state write-back layer, and the output of the state write-back layer is connected to the event arrival update layer in the continuous time state unit. The output of the aggregation mapping layer in the coupled event encoding unit is connected to the event arrival update layer in the continuous time state unit. The output of the state output layer in the continuous time state unit is connected to the task advancement layer propagation sublayer, quality blocking layer propagation sublayer, and delivery release layer propagation sublayer in the hierarchical propagation intensity unit. The output of the propagation result convergence layer in the hierarchical propagation intensity unit is connected to the blocking position extraction layer and propagation path extraction layer in the adjustment action generation unit. The output of the adjustment order generation layer in the adjustment action generation unit is connected to the action mapping layer in the write-back update unit to generate the process write-back neural Hawkes model. The model training utilizes historical software development process data. First, the historical software development process data is correlated and organized to generate a historical development process dataset. Then, historical composite event sequences are generated based on the historical development process dataset and input into the process write-back neural Hawkes model. During training, the coupled event encoding unit, continuous time state unit, hierarchical propagation intensity unit, adjustment action generation unit, and write-back update unit participate in the forward computation in sequence, outputting historical event state sequences, historical process propagation results, historical process adjustment actions, and historical write-back state sequences. Finally, the model output results are compared with the results of the historical real development process, and parameter updates are performed to complete the training of the process write-back neural Hawkes model. The total loss is constructed using a combination of multiple losses, including event arrival loss, process propagation loss, process adjustment loss, and state write-back loss. Event arrival loss is constructed based on the deviation between the historical subsequent event arrival results output by the process write-back neural Hawkes model and the historical subsequent event arrival records during actual development. Process propagation loss is constructed based on the deviation between the historical process propagation results output by the process write-back neural Hawkes model and the propagation records during actual development. Process adjustment loss is constructed based on the deviation between the historical process adjustment actions output by the process write-back neural Hawkes model and the adjustment sequence records during actual development. State write-back loss is constructed based on the deviation between the historical write-back state sequence output by the process write-back neural Hawkes model and the state change records during actual development. The total loss is generated by combining the event arrival loss, process propagation loss, process adjustment loss, and state write-back loss. The process write-back neural Hawkes model is then trained based on this total loss to learn the continuous-time evolution patterns of historical composite event sequences, process propagation patterns, and state write-back patterns. The training parameters were set as follows: batch size of 32, initial learning rate of 0.001, 120 training epochs, and Adam optimization algorithm for parameter updates. During training, composite event sequences were input into the neural Hawkes model in chronological order of event occurrence. The validation set loss was used to determine the training status. When the validation set loss decreased below a preset threshold for several consecutive epochs, training was stopped, and the current model parameters were saved as the final model parameters. When the validation set loss stopped decreasing after the training epochs reached a preset upper limit, the model parameters corresponding to the minimum validation set loss were selected as the final model parameters.

[0024] In this embodiment, generating the event state sequence includes: Based on the composite event sequence, the corresponding locations of requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data in each composite development activity unit are respectively input into the requirement data coding layer, code submission data coding layer, code review data coding layer, build execution data coding layer, test execution data coding layer, defect handling data coding layer, release execution data coding layer, and personnel collaboration data coding layer. Encoding processing is performed on each layer to generate requirement data coding results, code submission data coding results, code review data coding results, build execution data coding results, test execution data coding results, defect handling data coding results, release execution data coding results, and personnel collaboration data coding results, which are then combined into sub-item coding results. Based on the itemized coding results, each coding result is input into the aggregation mapping layer. The aggregation mapping is performed according to the fixed position and preset arrangement order in the composite development activity unit to generate a composite event representation composed of the itemized coding results. Based on the chronological order of occurrence in the composite event sequence, the composite event representations are sequentially input into the event arrival update layer in the continuous time state unit. The current composite event representation and the corresponding state result of the previous composite event representation are then updated to generate the event arrival state. Specifically, the event arrival update involves inputting the current composite event representation into the event arrival update layer, reading the corresponding state result of the previous composite event representation, combining the encoded content in the current composite event representation with the corresponding state content in the state result of the previous composite event representation, adjusting the state result of the previous composite event representation based on the combination result, and generating the event arrival state corresponding to the current composite event representation. Based on the event occurrence time intervals corresponding to adjacent composite development activity units in the composite event sequence, the event arrival state is input into the time interval evolution layer. Continuous time state evolution is performed according to the event occurrence time intervals to generate a time evolution state. Specifically, the continuous time state evolution involves inputting the event arrival state into the time interval evolution layer, reading the event occurrence time intervals corresponding to adjacent composite development activity units, and performing time-progressive decay and retention on each state content in the event arrival state according to the event occurrence time intervals to generate the time evolution state corresponding to the current composite development activity unit. Based on the time evolution state, the time evolution state is input into the state output layer, and the state results corresponding to each composite development activity unit are output according to the occurrence time order in the composite event sequence to generate the event state sequence.

[0025] In this embodiment, the generation process propagation results include: The event state sequence is input into the hierarchical propagation intensity unit. According to the data source in the composite development activity unit, the event states corresponding to the requirement data, code submission data, and code review data in the event state sequence are extracted to generate the task advancement layer input state. The event states corresponding to the build execution data, test execution data, defect handling data, and personnel collaboration data are extracted to generate the quality blocking layer input state. The event states corresponding to the release execution data are extracted to generate the delivery release layer input state. The task advancement layer input state is input into the task advancement layer propagation sublayer. Sequential propagation calculations are performed on the event states corresponding to requirement data, code submission data, and code review data according to the occurrence time order in the event state sequence to generate the task advancement layer propagation strength. Specifically, the sequential propagation calculation is performed by inputting the event states corresponding to requirement data, code submission data, and code review data into the task advancement layer propagation sublayer in the order of occurrence time according to the event state sequence. The current input state and the propagation result of the previous moment are used as the basis for the propagation calculation at the current moment. The propagation result of the task advancement layer is updated every moment to generate the task advancement layer propagation strength. The quality blocking layer input state and the task advancement layer propagation strength are input into the quality blocking layer propagation sublayer. Sequential propagation calculations are performed on the event states corresponding to the construction execution data, test execution data, and defect handling data according to their occurrence time sequence in the event state sequence. This is combined with the task advancement layer propagation strength for inter-layer transfer calculations to generate the quality blocking layer propagation strength. Specifically, the sequential propagation calculation involves inputting the event states corresponding to the construction execution data, test execution data, and defect handling data into the quality blocking layer propagation sublayer in the order of their occurrence time. The current input state, the previous propagation result, and the task advancement layer propagation strength are used as the basis for the current propagation calculation. The quality blocking layer propagation result is updated hourly to generate the quality blocking layer propagation strength. The delivery release layer input state and the quality blocking layer propagation strength are input into the delivery release layer propagation sublayer. The event states corresponding to the release execution data are sequentially propagated according to the occurrence time order in the event state sequence. The inter-layer propagation calculation is combined with the quality blocking layer propagation strength to generate the delivery release layer propagation strength. Specifically, the sequential propagation calculation is performed by sequentially inputting the event states corresponding to the release execution data into the delivery release layer propagation sublayer according to the occurrence time order in the event state sequence. The current input state, the propagation result of the previous moment, and the quality blocking layer propagation strength are used as the basis for the propagation calculation at the current moment. The delivery release layer propagation result is updated every moment to generate the delivery release layer propagation strength. The propagation intensity of the task advancement layer, the propagation intensity of the quality blocking layer, and the propagation intensity of the delivery release layer are input into the propagation result aggregation layer. Aggregation processing is performed according to the time correspondence and inter-layer transmission relationship in the same development process to generate process propagation results. The process propagation results include the propagation intensity of the task advancement layer, the propagation intensity of the quality blocking layer, the propagation intensity of the delivery release layer, the inter-layer transmission direction, and the degree of delivery impact corresponding to the current development process.

[0026] In this embodiment, generating the write-back state sequence includes: The process propagation results are input into the blocking location extraction layer in the adjustment action generation unit. Based on the propagation intensity of the task advancement layer, the propagation intensity of the quality blocking layer, the propagation intensity of the delivery release layer, the inter-layer transmission direction, and the degree of delivery impact in the process propagation results, the concentrated location of propagation intensity and the concentrated location of delivery impact are extracted according to the development process sequence corresponding to the event state sequence, and the process blocking location is determined. The process propagation results and process blockage locations are input into the propagation path extraction layer in the adjustment action generation unit. Based on the inter-layer transmission direction, the occurrence time sequence corresponding to the event state sequence, and the sequential relationship between the process blockage locations, the transmission link from the propagation intensity of the task advancement layer to the propagation intensity of the quality blockage layer, and then to the propagation intensity of the delivery release layer is extracted to determine the propagation path. The process propagation results, process blockage locations, and propagation paths are input into the adjustment order generation layer in the adjustment action generation unit. The adjustment order is determined based on the order of occurrence of process blockage locations, the order of transmission of propagation paths, and the order of delivery impact. Based on the adjustment order, the execution order of requirement data, code review data, build execution data, test execution data, and release execution data corresponding to the process blockage position is adjusted, generating process adjustment actions including task processing adjustment, code review adjustment, build execution adjustment, test execution adjustment, and release execution adjustment; The process adjustment actions and event state sequences are input into the action mapping layer in the write-back update unit. Based on the correspondence between task processing adjustment, code review adjustment, build execution adjustment, test execution adjustment, release execution adjustment and the corresponding state content in the event state sequence, the execution position of the process adjustment actions is mapped, and the state mapping result is generated. The state mapping results are input into the state write-back layer in the write-back update unit. Based on the state mapping results, the state content corresponding to the task processing adjustment, the code review adjustment, the build execution adjustment, the test execution adjustment, and the release execution adjustment in the event state sequence are corrected and updated in order to generate the write-back state sequence.

[0027] In this embodiment, the optimization scheme for the generation process includes: Based on the write-back state sequence, extract the state content corresponding to task processing adjustments, code review adjustments, build execution adjustments, test execution adjustments, and release execution adjustments; Based on the order of the status content corresponding to the task processing adjustment in the write-back status sequence, the order of adjustment of the status correction result and the corresponding process blocking position, the status content corresponding to each task processing adjustment is sorted before and after execution to determine the order of requirement processing. Based on the order of the state content corresponding to the code review adjustment in the write-back state sequence, the order of the state correction results and the order of the corresponding propagation paths, the state content corresponding to each code review adjustment is sorted to determine the code review order. Based on the order of the state content corresponding to the build execution adjustment in the write-back state sequence, the order of the state correction results and the order of the corresponding process blocking positions, the state content corresponding to each build execution adjustment is sorted before and after execution to determine the build execution order; Based on the order of the state content corresponding to the test execution adjustment in the write-back state sequence, the order of the state correction results and the order of the corresponding process blocking positions, the state content corresponding to each test execution adjustment is sorted before and after execution to determine the test execution order. Based on the order of the status content corresponding to the release execution adjustment in the write-back status sequence, the order of the status correction results and the order of the corresponding delivery impact, the status content corresponding to each release execution adjustment is sorted before and after execution to determine the release execution order; Based on the order of requirement processing, code review, build execution, test execution, and release execution, the processes are merged and organized to generate an optimized solution.

[0028] Example 1: To verify the feasibility of this invention in practice, it was applied to a software development management scenario oriented towards parallel multi-project development. This scenario includes multiple stages such as requirement processing, code submission, code review, build execution, test execution, defect handling, release execution, and personnel collaboration. During daily operation, it continuously generates requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data. As parallel development tasks continue to increase, several practical problems gradually emerge in the development process, including a lack of linkage between the requirement processing order and the code review order, a lack of dynamic coordination between the build execution order and the test execution order, the inability to identify the impact of defect handling on release execution in advance, and the difficulty in accurately characterizing the transmission effect of changes in personnel collaboration on process blockages. Existing practices mainly rely on fixed rule scheduling and static statistical analysis, which can observe local queuing phenomena but struggle to identify the entire process of blockages propagating from the task advancement stage to the quality stage and then to the delivery stage. Therefore, problems such as extended build waiting times, increased test reflows, expanded release delays, and an imbalance in the overall development process rhythm are prone to occur.

[0029] In this embodiment, software development process data is first collected, and various types of data are correlated and organized to generate a development process dataset. Each record in the development process dataset maintains a unified organizational format to represent the sequential relationships within the same development process. Then, based on the development process dataset, the interrelated software development process data are aggregated to generate composite development activity units, which are then organized into composite event sequences according to their chronological order. At this point, a composite development activity unit no longer corresponds to a single stage log, but rather to changes in requirement advancement, code submission, code review, build execution, test execution, defect handling, release execution, and personnel collaboration within the same development process. Therefore, it can reflect the true interconnectedness of a single development activity across multiple stages.

[0030] Subsequently, a neural Hawkes model for process write-back is constructed based on the neural Hawkes process. After the composite event sequence is input into the coupled event encoding unit, it enters the requirement data encoding layer, code submission data encoding layer, code review data encoding layer, build execution data encoding layer, test execution data encoding layer, defect handling data encoding layer, release execution data encoding layer, and personnel collaboration data encoding layer, respectively, and then passes through an aggregation mapping layer to generate a composite event representation. After the composite event representation enters the continuous-time state unit, it combines the arrival order and time intervals in the composite event sequence to generate an event state sequence. Compared with static process analysis, this process no longer treats requirements, reviews, builds, tests, and releases as isolated point records, but rather expresses them uniformly within a continuous-time state evolution chain, thus more realistically reflecting the changes in process state over time.

[0031] After the event state sequence is formed, it is input into the hierarchical propagation strength unit to generate propagation strengths for the task advancement layer, quality blocking layer, and delivery release layer, which are then aggregated into a process propagation result. The process propagation result can intuitively reflect whether the current development process blockage is concentrated in the requirements processing and code review stages, or has already spread to the build execution and test execution stages, or has already significantly impacted release execution. Based on the process propagation result, the process blocking location, propagation path, and adjustment order are determined in the adjustment action generation unit, further generating process adjustment actions. After the process adjustment actions are input into the write-back update unit, the state content corresponding to task processing adjustment, code review adjustment, build execution adjustment, test execution adjustment, and release execution adjustment in the event state sequence is corrected and the order is updated, generating a write-back state sequence. Based on the write-back state sequence, the requirements processing order, code review order, build execution order, test execution order, and release execution order are determined, ultimately forming a process optimization scheme. The entire application process does not change the basic functionality of the original R&D platform; improvements are only made at the data organization, propagation identification, and order adjustment levels, thus allowing direct embedding into existing software development management environments.

[0032] To verify the application effect of the method of this invention, a comparative experiment was conducted using samples from the same development process. The comparative methods included traditional rule-based scheduling, software development process analysis based on static statistics, and the method of this invention. Traditional rule-based scheduling mainly arranges the order of review, build, test, and release according to predetermined priorities and human experience; software development process analysis based on static statistics mainly judges the process based on historical average processing time and the number of defects; the method of this invention, however, completes dynamic process optimization based on the development process dataset, composite event sequences, and a neural Hawkes model for process rewrite. The experiment statistically analyzed indicators such as blockage identification accuracy, average code review backlog time, average build waiting time, average number of test reflows, average release delay time, requirement-to-release cycle time, rework rate, and blockage propagation misjudgment rate. The results are as follows.

[0033] Table 1. Comprehensive Comparison of Process Optimization Effects

[0034] As shown in Table 1, the method of this invention outperforms the two comparative methods in all indicators. The blockage identification accuracy is improved from 71.8% of the traditional rule-based scheduling method to 86.9%, and by 7.3 percentage points compared to the software development process analysis method based on static statistics. This indicates that the method of this invention can not only identify whether queuing has occurred at a certain stage, but also identify the transmission trend of blockage between different stages by combining the propagation strength of the task advancement layer, the propagation strength of the quality blockage layer, and the propagation strength of the delivery and release layer. The reason for the performance improvement is that this invention does not make independent judgments on a single log, but puts the information of multiple stages into a unified time-series link through composite development activity units and composite event sequences, and then identifies the real location and propagation path of the blockage through the process propagation results. Therefore, the false judgment rate is reduced to 10.6%.

[0035] As shown in Table 1, the method of this invention has a stable improvement effect on code review backlog time, build waiting time, number of test reflows, and release delay time. The average code review backlog time decreased from 18.4 hours to 12.9 hours, the average build waiting time decreased from 26.3 minutes to 18.6 minutes, the average number of test reflows decreased from 2.7 times / iteration to 1.8 times / iteration, and the average release delay time decreased from 9.6 hours to 6.1 hours. The performance improvement is not simply due to moving all tasks forward, but rather because after the process propagation results are generated, the adjustment action generation unit can combine the process blockage location and propagation path to form an adjustment order, and then the write-back update unit corrects the state sequence and updates the order of events. The resulting write-back state sequence can more accurately reflect the actual process state after adjustment. Therefore, the generated requirement processing order, code review order, build execution order, test execution order, and release execution order are more consistent with the actual development process, avoiding new congestion in subsequent stages caused by local optimization.

[0036] From the perspective of overall R&D cycle metrics, the method of this invention shortens the demand-to-release cycle from 13.8 days to 11.4 days and reduces the rework rate from 11.7% to 8.3%. This demonstrates that the effect of the method of this invention is not limited to a single stage, but has a continuous improvement effect on the entire development process. The performance improvement is due to the fact that this invention completes two types of key processing through a process write-back neural Hawkes model: one is the continuous time state evolution based on a composite event sequence, and the other is the state write-back update based on process adjustment actions. The former ensures that the system can identify continuous changes in the development process, while the latter ensures that optimization actions can be truly fed back into the process state, thereby supporting the subsequent re-determination of the sequence.

[0037] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A software development process optimization method based on big data processing, characterized in that, Includes the following steps: Collect software development process data, perform correlation and organization, and generate a development process dataset; Based on the development process dataset, the interrelated software development process data are aggregated to generate composite development activity units and organize composite event sequences. Based on the neural Hawkes process, a neural Hawkes process write-back model is constructed, including a coupled event encoding unit, a continuous time state unit, a hierarchical propagation intensity unit, a modulating action generation unit, and a write-back update unit. Based on the composite event sequence, in the coupled event coding unit, coding aggregation is performed to generate a composite event representation, and the composite event representation is input into the continuous-time state unit to perform continuous-time state evolution and generate an event state sequence; Input the event state sequence into the hierarchical propagation strength unit to construct the hierarchical propagation strength and generate the process propagation result; In the adjustment action generation unit, the process blocking location, propagation path, and adjustment sequence are determined based on the process propagation results, process adjustment actions are generated, and input into the write-back update unit to perform state write-back update and generate a write-back state sequence. Based on the write-back state sequence, the order of requirement processing, code review, build execution, test execution, and release execution is determined, and a process optimization plan is generated.

2. The software development process optimization method based on big data processing according to claim 1, characterized in that, The generated development process dataset includes: Collect requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data to generate software development process data; Based on software development process data, perform structured transformation to generate development process records; Based on the development process records, the requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, and release execution data are concatenated to generate requirement flow chain, code change chain, review processing chain, build execution chain, test execution chain, defect handling chain, and release execution chain. Personnel collaboration data are mapped to the corresponding development process records in each chain to generate associated development process records. Based on the associated development process records, sorting and rearranging are performed to generate a time-series development process record; Based on the time-series development process records, the process is merged and organized to generate a development process dataset.

3. The software development process optimization method based on big data processing according to claim 1, characterized in that, The generation of the composite event sequence includes: Based on the development process dataset, according to the correspondence of development processes, perform correlation filtering to extract data records that belong to the same development process and have a sequential relationship, and generate a development process aggregation window; Based on the development process aggregation window, merge mapping is performed according to fixed positions to generate composite development activity units, including the corresponding positions of requirement data, code submission data, code review data, build execution data, test execution data, defect handling data, release execution data, and personnel collaboration data. Based on the composite development activity unit, the structured organization is executed according to the preset arrangement order to complete the internal arrangement of the composite development activity unit; Based on the event occurrence time field in the development process aggregation window, composite development activity units are organized and executed in chronological order of occurrence to generate a composite event sequence.

4. The software development process optimization method based on big data processing according to claim 1, characterized in that, The construction process of writing back the neural Hawkes model includes: Based on the event embedding structure, continuous-time hidden state evolution structure, and conditional strength modeling structure of the neural Hawkes process; Transform the event embedding structure into a coupled event coding unit, setting up a requirement data coding layer, a code submission data coding layer, a code review data coding layer, a build execution data coding layer, a test execution data coding layer, a defect handling data coding layer, a release execution data coding layer, and a personnel collaboration data coding layer, and connect the output of each layer to the aggregation mapping layer. The continuous-time hidden state evolution structure is transformed into a continuous-time state unit, and an event arrival update layer, a time interval evolution layer, and a state output layer are set. The conditional strength modeling structure is transformed into a layered propagation strength unit, with a task advancement layer propagation sub-layer, a quality blocking layer propagation sub-layer, a delivery and release layer propagation sub-layer, and a propagation result convergence layer. Based on the hierarchical propagation intensity unit, an adjustment action generation unit is constructed after the propagation result convergence layer, and a blocking position extraction layer, a propagation path extraction layer, and an adjustment order generation layer are set. Based on the continuous time state unit, a write-back update unit is constructed after adjusting the action generation unit, and an action mapping layer and a state write-back layer are set. Build the connection relationships between the units and generate a neural Hawkes model for process writeback.

5. The software development process optimization method based on big data processing according to claim 1, characterized in that, The generated event state sequence includes: Based on the composite event sequence, each composite development activity unit is input into the requirement data coding layer, code submission data coding layer, code review data coding layer, build execution data coding layer, test execution data coding layer, defect handling data coding layer, release execution data coding layer, and personnel collaboration data coding layer. The coding process is performed on each layer to generate requirement data coding results, code submission data coding results, code review data coding results, build execution data coding results, test execution data coding results, defect handling data coding results, release execution data coding results, and personnel collaboration data coding results. These results are then combined into sub-item coding results. Based on the itemized coding results, each coding result is input into the aggregation mapping layer. The aggregation mapping is performed according to the fixed position and preset arrangement order in the composite development activity unit to generate a composite event representation composed of the itemized coding results. Based on the time sequence of occurrence in the composite event sequence, the composite event representations are sequentially input into the event arrival update layer in the continuous time state unit. The current composite event representation is updated with the state result corresponding to the previous composite event representation to generate the event arrival state. Based on the event occurrence time intervals corresponding to adjacent composite development activity units in the composite event sequence, the event arrival state is input into the time interval evolution layer, and continuous time state evolution is performed according to the event occurrence time interval to generate time evolution state; Based on the time evolution state, the input state output layer sequentially outputs the state results corresponding to each composite development activity unit, generating an event state sequence.

6. The software development process optimization method based on big data processing according to claim 1, characterized in that, The propagation results of the generation process include: The event state sequence is input into the hierarchical propagation intensity unit. According to the data source in the composite development activity unit, the event state corresponding to each data is extracted to generate the task advancement layer input state, quality blocking layer input state and delivery release layer input state. The task propagation layer input state is input into the task propagation layer propagation sublayer, and sequential propagation calculation is performed to generate the task propagation layer propagation strength. The mass blocking layer input state and the mission propagation layer propagation strength are input into the mass blocking layer propagation sublayer for sequential propagation calculation, and inter-layer transfer calculation is performed in combination with the mission propagation layer propagation strength to generate the mass blocking layer propagation strength. The delivery release layer input state and the quality blocking layer propagation strength are input into the delivery release layer propagation sublayer, sequential propagation calculation is performed, and inter-layer propagation calculation is performed in combination with the quality blocking layer propagation strength to generate the delivery release layer propagation strength. The propagation strength of the task advancement layer, the propagation strength of the quality blocking layer, and the propagation strength of the delivery release layer are input into the propagation result aggregation layer. Aggregation processing is performed to generate process propagation results. The process propagation results include the propagation strength of the task advancement layer, the propagation strength of the quality blocking layer, the propagation strength of the delivery release layer, the inter-layer transmission direction, and the degree of delivery impact corresponding to the current development process.

7. The software development process optimization method based on big data processing according to claim 1, characterized in that, The generated write-back state sequence includes: The process propagation results are input into the blocking location extraction layer in the adjustment action generation unit. According to the development process sequence corresponding to the event state sequence, the concentrated location of propagation intensity and the concentrated location of delivery impact are extracted to determine the process blocking location. Input the process propagation results and process blockage locations into the propagation path extraction layer. Based on the inter-layer transmission direction, the occurrence time sequence corresponding to the event state sequence, and the sequential relationship between the process blockage locations, extract the transmission link from the propagation intensity of the task advancement layer to the propagation intensity of the quality blockage layer, and then to the propagation intensity of the delivery and release layer to determine the propagation path. Input the process propagation results, process blockage locations, and propagation paths into the adjustment order generation layer. Based on the order of occurrence of process blockage locations, the order of transmission of propagation paths, and the order of delivery impact, determine the adjustment order. Based on the adjustment order, the execution order of requirement data, code review data, build execution data, test execution data, and release execution data corresponding to the process blockage position is adjusted to generate process adjustment actions; Input the process adjustment actions and event state sequences into the action mapping layer in the write-back update unit, map the execution position of the process adjustment actions, and generate state mapping results; The state mapping results are input into the state write-back layer. The state content corresponding to the task processing adjustment, the code review adjustment, the build execution adjustment, the test execution adjustment, and the release execution adjustment in the event state sequence is corrected and the order is updated to generate the write-back state sequence.

8. The software development process optimization method based on big data processing according to claim 1, characterized in that, The optimization scheme for the generation process includes: Based on the order of the status content corresponding to task processing adjustment, the status content corresponding to build execution adjustment, and the status content corresponding to test execution adjustment in the write-back status sequence, as well as the order of adjustment of status correction results and corresponding process blocking positions, the order of execution is determined by sorting the status content before and after execution. Based on the order of the state content corresponding to the code review adjustment in the write-back state sequence, the order of state correction results and the order of corresponding propagation path adjustments, the code review order is determined by sorting the execution order. Based on the order of the status content corresponding to the release execution adjustment in the write-back status sequence, the order of the status correction results and the order of the corresponding delivery impact, the execution is sorted to determine the release execution order; Based on the order of requirement processing, code review, build execution, test execution, and release execution, the processes are merged and organized to generate an optimized solution.

Images