A cloud computing-based software development data hierarchical processing method and system

Abstract

The application discloses a software development data hierarchical processing method and system based on cloud computing, and relates to the technical field of cloud computing.The method comprises the following steps: obtaining data item access sequences of each stage in a historical software development process, and dividing the data item access sequences into a plurality of development sessions; extracting time sequence access relations between data items in the development sessions, and generating a multi-order correlation relation table between data items of each stage; respectively calculating access frequencies of each kind of data item of each stage and multi-order comprehensive trigger probabilities between each two kinds of data items, and then calculating comprehensive hierarchical weights of each kind of data item of each stage; respectively determining data hierarchical storage strategies of each stage based on the comprehensive hierarchical weights; and obtaining latest atomic access event records of a current stage in real time, and determining a data preheating strategy.The application realizes intelligent hierarchical and forward-looking preheating of software development data by comprehensively considering data access heat and internal correlation, and can effectively improve access efficiency and storage resource utilization.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a cloud computing-based method and system for layered processing of software development data. Background Technology

[0002] In cloud-based software development, large amounts of data are generated and frequently accessed, such as source code, dependency libraries, configuration files, and test cases. To balance access performance and storage costs, existing technologies generally adopt a tiered storage architecture, distributing data across storage media of different performance levels according to its access frequency to achieve efficient access and resource optimization. Storage media include high-speed memory, SSDs, high-performance cloud disks, and standard cloud disks. This tiering strategy based on access frequency or recent access time, such as LRU and its variants, is only simple and effective in specific scenarios. However, for cloud computing development platforms, when developers perform core code debugging tasks, existing tiering methods have the following significant drawbacks.

[0003] Existing layering methods primarily rely on the independent access frequency of data as the basis for layering decisions, failing to adequately consider the temporal correlations between data accesses. For example, during development, after accessing a core code file, a developer is highly likely to access its dependent configuration files, specific third-party libraries, or related test cases within a short period. Existing layering methods cannot detect such correlation behavior, resulting in related data not being pre-warmed to the high-speed storage layer, causing subsequent access delays and impacting development smoothness. Furthermore, due to the lack of effective quantification of correlation strength, some high-frequency but non-correlated data, such as frequently read independent log files or general test cases, may occupy valuable cache space for extended periods, while critical related data that has a continuous impact on the development process cannot be loaded in a timely manner, causing cache pollution and resource waste.

[0004] Moreover, existing layering methods are rigid and costly to maintain. Their thresholds, such as frequency and cache size, are mostly statically set based on human experience, making them difficult to adapt to dynamic changes at different stages of software development, such as coding, compiling, debugging, and testing. When development modes switch, static strategies are prone to failure, requiring manual intervention from operations personnel, resulting in high maintenance costs and slow response times. Summary of the Invention

[0005] To improve access efficiency and storage resource utilization, this invention provides a cloud computing-based software development data layering processing method and system. This method comprehensively considers data access frequency and inherent correlation to achieve intelligent layering and proactive pre-warming of software development data. It addresses the problems of existing layering methods, which are prone to access delays and difficulty in adapting to dynamic changes during the software development phase. The technical solution is as follows:

[0006] In a first aspect, the present invention provides a cloud computing-based software development data layering processing method, the steps of which include: acquiring atomic access event records of each stage in the historical software development process and preprocessing them to obtain data item access sequences for each stage; dividing the data item access sequences of each stage into several development sessions, extracting the temporal access relationships between data items within the development sessions, and generating a multi-level association table between data items in each stage; calculating the access frequency of each type of data item and the multi-level comprehensive trigger probability between each pair of data items in each stage based on the multi-level association table of each stage; calculating the comprehensive layering weight of each type of data item in each stage based on the access frequency and the multi-level comprehensive trigger probability; determining the data layering storage strategy for each stage based on the comprehensive layering weight; acquiring the latest atomic access event records of the current stage in real time, and determining the data preheating strategy based on the multi-level comprehensive trigger probability related to the corresponding type of data item;

[0007] Specifically, the access frequency weight of each data item in each stage is obtained based on the access frequency, and the association strength weight of each data item in each stage is obtained based on the multi-order comprehensive trigger probability. The weight coefficients of the two are set, and the sum of the products of the access frequency weight and the association strength weight of the data item with the corresponding weight coefficients is used as the comprehensive hierarchical weight of the corresponding data item. The comprehensive hierarchical weight of each data item in each stage is obtained by traversing all types of data items in each stage.

[0008] Preferably, a data acquisition agent deployed on a cloud computing development platform is used to collect atomic access event records from each stage of the historical software development process. The collected fields include data item ID, timestamp, and access path. The timestamps of all atomic access event records in each stage are standardized. Based on the data item ID and access path in the atomic access event records of each stage, data items under the same access path in the same stage are classified into one category and numbered to obtain the number of data item types accessed in each stage. Then, based on the category number corresponding to the data item, the category numbers corresponding to all data items in each stage are sorted in order from front to back according to the timestamp to obtain the data item access sequence of each stage.

[0009] Preferably, based on the timestamps corresponding to all data items in the data item access sequence of each stage, the time difference between two adjacent data items in the data item access sequence of each stage is calculated. Based on the actual application scenario and the experience of access duration in the software development process, an access interval threshold is set. Consecutive adjacent data items in the data item access sequence of the same stage whose time difference is less than or equal to the access interval threshold are divided into a development session. The data item access sequence of each stage is divided in turn to obtain several development sessions corresponding to each stage. Based on the type number of all data items in the data item access sequence of a certain stage, the numbers of the two types of data items are combined to generate two data item vector groups with opposite access directions. The data item vector group set of that stage is obtained by combining them in turn. Similarly, the data item vector group set of each stage is obtained.

[0010] Preferably, based on the same development session within the same stage's data item access sequence, according to the sequential access relationship from front to back, the difference in the order of the two different types of data items within the data item access sequence is used as the association order of the corresponding data item vector group. Based on the experience of continuously accessing different types of data items during software development, a maximum association order is set. When the association order of a certain data item vector group is less than or equal to the maximum association order, it is determined that there is an association relationship of the corresponding order between the two data items corresponding to that data item vector group. Based on the set of data item vector groups in a certain stage and all development sessions, the number of times each order of association relationship of each data item vector group appears in all development sessions is counted to generate a multi-order association relationship table between data items in that stage. Similarly, a multi-order association relationship table between data items in each stage is obtained.

[0011] Preferably, based on the data item access sequence of each stage, the number of accesses for each type of data item in each stage and the total number of accesses for data items within the data item access sequence of each stage are counted. The ratio between the number of accesses for a certain data item in a certain stage and the total number of accesses in that stage is taken as the access frequency of the corresponding data item in that stage. Similarly, the access frequency of each type of data item in each stage is obtained. Based on the multi-level association table between data items in each stage, the number of occurrences of each level of association relationship of each data item vector group in the data item vector group set of each stage is extracted. The number of occurrences of a certain level of association relationship of a certain data item vector group in a certain stage is taken as the association feature of the data item vector group under that level of association relationship. The number of accesses of the previous type of data item in the data item vector group in that stage is taken as the total association feature of the data item vector group. The ratio between the association feature and the total association feature is taken as the association strength of the data item vector group under that level of association relationship. Similarly, the association strength of the data item vector group under each level of association relationship is obtained. The data item vector group set of each stage is traversed to obtain the association strength of each data item vector group in each stage under each level of association relationship.

[0012] Preferably, an attenuation factor is set based on the actual application scenario. For each order of association, an exponential function is used to map the attenuation factor accordingly to obtain the mapping value of each order of association. Based on the maximum association order, the mapping values ​​of each order of association are accumulated to obtain the total mapping value. The ratio between the mapping value of each order of association and the total mapping value is used as the weight coefficient corresponding to each order of association. The product between the association strength of a certain data item vector group in a certain stage under a certain order of association and the weight coefficient corresponding to that order of association is used as the trigger probability of that data item vector group under that order of association. The accumulated value of the trigger probability of that data item vector group under each order of association is used as the multi-order comprehensive trigger probability of that data item vector group. The multi-order comprehensive trigger probability of each data item vector group in each stage is obtained by traversing the set of data item vector groups in each stage.

[0013] Preferably, based on the data item vector group set of a certain stage, the data item vector group containing a certain data item and following that data item is extracted as the preceding association set of that data item. The multi-order comprehensive trigger probability of each data item vector group in the preceding association set of that data item is obtained. Based on the access frequency of each data item in that stage, the access frequencies of the other types of data items in that stage excluding that data item type are extracted and accumulated to obtain the access association feature of that data item. The multi-order comprehensive trigger probability of each data item vector group in the preceding association set is calculated sequentially and the product between it and the access frequency of another data item in the corresponding data item vector group is accumulated to obtain the association value feature of that data item. The ratio between the association value feature and the access association feature of that data item is used as the association strength weight of that data item. The access frequency of that data item is used as the access frequency weight of that data item. The sum of the weight coefficient of the access frequency weight and the weight coefficient of the association strength weight is 1. Based on the access frequency weight and the association strength weight of that data item and their corresponding weight coefficients, the comprehensive hierarchical weight of that data item is obtained. Then, the comprehensive hierarchical weight of each data item in each stage is calculated sequentially.

[0014] Preferably, based on the comprehensive hierarchical weight of each type of data item in each stage, all types of data items in each stage are sorted in descending order of comprehensive hierarchical weight to obtain the hierarchical priority sequence of data items in each stage; the current stage of the current software development process is obtained, and based on the distribution and remaining amount of device storage space, the various data items corresponding to the current stage are sequentially divided into high-speed storage layer, performance storage layer, and capacity storage layer according to the hierarchical priority sequence of data items in the corresponding stage; similarly, when a change in the stage of the software development process is detected, the data hierarchical storage strategy of the corresponding stage is dynamically adjusted according to the same data item division method.

[0015] Preferably, the latest atomic access event record of the current stage is collected, and the same preprocessing method is used to preprocess the latest atomic access event record of the current stage to obtain the category number corresponding to the latest accessed data item in the latest atomic access event record. The data item vector group with the same data item preceding it is extracted, and the latter data item in each data item vector group is used as the reserve data item for the next access. The corresponding multi-order comprehensive trigger probability is obtained, and each data item vector group is sorted in descending order of multi-order comprehensive trigger probability to obtain the preheating priority order of the reserve data items. Based on the size of the device cache space, the corresponding data items are cached in order of preheating priority. Similarly, when the latest atomic access event record is updated, the data preheating strategy is updated in the same way.

[0016] Secondly, the present invention provides a cloud computing-based software development data layering processing system for implementing the aforementioned cloud computing-based software development data layering processing method, comprising: a processor, a memory, a communication interface, and a cloud computing platform. The processor stores computer program instructions for implementing the aforementioned cloud computing-based software development data layering processing method, and the communication interface is communicatively connected to the cloud computing platform.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] This invention integrates access frequency weights and correlation strength weights to obtain a comprehensive hierarchical weight for each data item at each stage. This comprehensively considers data access popularity and inherent correlation, accurately identifying core data with high access frequency and strong correlation. Quantifying correlation strength can prevent data items with poor correlation from occupying high-speed resources for a long time, allowing valuable cache space to be used to serve the core development process, thus improving overall resource utilization. At the same time, by introducing multi-level correlation analysis, it can capture the complete data access chain from the short to the medium term, achieving more accurate forward-looking data preheating and significantly reducing access latency on the critical path in complex development workflows. Moreover, this invention can automatically adjust the hierarchical strategy according to the stage changes in the software development process, reducing reliance on manual intervention, thereby effectively improving operational efficiency and reducing operational costs. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an implementation of a cloud computing-based software development data layering processing method according to an embodiment of this application.

[0020] Figure 2 This is a structural block diagram of a cloud computing-based software development data layering processing system according to an embodiment of this application. Detailed Implementation

[0021] The technical features of the present invention will be further described in detail below with reference to the accompanying drawings so that those skilled in the art can understand them.

[0022] A cloud computing-based software development data layering processing method, the implementation process of which is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0023] Step S1: Obtain atomic access event records for each stage of the historical software development process and preprocess them to obtain the data item access sequence for each stage.

[0024] Specifically, a data acquisition agent deployed on a cloud computing development platform collects atomic access event records from each stage of the historical software development process. The collected fields include data item ID, timestamp, and access path. The timestamps of all atomic access event records in each stage are standardized. Based on the data item ID and access path in the atomic access event records of each stage, data items under the same access path in the same stage are classified into one category and numbered to obtain the number of data item types accessed in each stage. Then, based on the category number corresponding to the data item, the category numbers corresponding to all data items in each stage are sorted in order from front to back according to the timestamp to obtain the data item access sequence of each stage.

[0025] The specific method for standardizing timestamps is to convert the timestamps of all atomic access event records to standard UTC time (Coordinated-Universal-Time). UTC time is a globally used standard time system, represented in a format from 00:00 to 23:59, which can coordinate clocks and time records around the world. Even for the same type of data items, different category numbers need to be set at different stages to facilitate subsequent differentiation and calculation and prevent system confusion. The order and time of accessing data items in each stage can be obtained based on the data item access sequence of each stage.

[0026] Step S2: Divide the data item access sequence of each stage into several development sessions, extract the temporal access relationship between data items within the development session, and generate a multi-level association table between data items of each stage.

[0027] Specifically, based on the timestamps corresponding to all data items in the data item access sequence of each stage, the time difference between two adjacent data items in the data item access sequence of each stage is calculated. Based on the actual application scenario and the experience of access duration in the software development process, an access interval threshold is set. Consecutive adjacent data items in the data item access sequence of the same stage whose time difference is less than or equal to the access interval threshold are divided into a development session. The data item access sequence of each stage is divided in turn to obtain several development sessions corresponding to each stage. Based on the type number of all data items in the data item access sequence of a certain stage, the numbers of the two types of data items are combined to generate two data item vector groups with opposite access directions. The data item vector group set of that stage is obtained by combining them in turn. Similarly, the data item vector group set of each stage is obtained.

[0028] In this invention, the time difference for the first accessed data item is set to 0, and the access interval threshold is typically set to 10-30 minutes. Preferably, the access interval threshold is set to 15 minutes. When the time difference between two adjacent data items is less than or equal to the access interval threshold, the two data items are considered to be related. The purpose of dividing development sessions is to aggregate related data items together. The process of dividing development sessions is carried out sequentially. For example, for a certain stage of data item access sequence, when the time difference between the first and second data items and the time difference between the second and third data items are both less than or equal to the access interval threshold, the first, second, and third data items are divided into the same development session. When the time difference between the third and fourth data items is greater than the access interval threshold, the previous development session is divided, and the fourth data item is divided into a new development session.

[0029] The essence of a data item vector group is a combination of data item type numbers that represents the access order of two different data items within the same stage. For example, for data item A and data item B within the same stage, a data item vector group... This represents an event where data item A is accessed first and then data item B within the same development session; the data item vector group is used for this event. This indicates an event where data item B is accessed first, followed by data item A, within the same development session.

[0030] Furthermore, based on the same development session within the same stage's data item access sequence, according to the sequential access relationship from front to back, the difference in the order of the two different types of data items within the data item access sequence is used as the association order of the corresponding data item vector group. Based on the experience of continuously accessing different types of data items during software development, a maximum association order is set. When the association order of a certain data item vector group is less than or equal to the maximum association order, it is determined that there is an association relationship of the corresponding order between the two data items corresponding to that data item vector group. Based on the set of data item vector groups in a certain stage and all development sessions, the number of times each order of association relationship of each data item vector group appears in all development sessions is counted to generate a multi-order association relationship table between data items in that stage. Similarly, a multi-order association relationship table between data items in each stage is obtained.

[0031] The association order is calculated by taking the difference between the sequence numbers of each data item in the access sequence. This is used to measure the access interval between two data items within the same development session. For example, if data item A is accessed first, followed by data items B and C, and finally data item D, then the association between data item A and data item B is order 1, the association between data item A and data item C is order 2, and the association between data item A and data item D is order 3. The access interval between data item A and data item D is 3 access steps. The maximum association order is typically set to 3-5. Preferably, in this invention, the maximum association order is set to 3. The purpose of setting the maximum association order is to simplify the calculation process. Since the larger the access interval between two data items within the same development session, the weaker the association between the two data items, setting a maximum association order can eliminate cases with extremely low association, thereby reducing computation and improving system efficiency.

[0032] Step S3: Based on the multi-level association table of each stage, calculate the access frequency of each data item in each stage and the multi-level comprehensive trigger probability between each pair of data items.

[0033] Specifically, based on the data item access sequence of each stage, the number of accesses for each type of data item in each stage and the total number of accesses for data items within the data item access sequence of each stage are counted. The ratio between the number of accesses for a certain data item in a certain stage and the total number of accesses in that stage is taken as the access frequency of the corresponding data item in that stage. Similarly, the access frequency of each type of data item in each stage is obtained. Based on the multi-level association table between data items in each stage, the number of occurrences of each level of association relationship of each data item vector group in the data item vector group set of each stage is extracted. The number of occurrences of a certain level of association relationship of a certain data item vector group in a certain stage is taken as the association feature of the data item vector group under that level of association relationship. The number of accesses of the previous type of data item in the data item vector group in that stage is taken as the total association feature of the data item vector group. The ratio between the association feature and the total association feature is taken as the association strength of the data item vector group under that level of association relationship. Similarly, the association strength of the data item vector group under each level of association relationship is obtained. By traversing the data item vector group set of each stage, the association strength of each data item vector group in each stage under each level of association relationship is obtained.

[0034] Among them, the access frequency of the i-th data item in the same stage is The calculation formula is as follows:

[0035]

[0036] In the formula, This represents the number of times the i-th type of data item is accessed within the same stage, that is, the number of the i-th type of data item contained in the data item access sequence within the same stage. This represents the total number of times all data items are accessed within the same phase, i.e., the total number of data items contained in the data item access sequence within the same phase.

[0037] The data item vector group consisting of the i-th data item and the j-th data item in the same stage The k-th order correlation strength is The calculation formula is as follows:

[0038]

[0039] In the formula, Represents a group of data items in the same stage. The k-order association feature, i.e., the number of times the k-order association occurs. Represents a group of data items in the same stage. The total association characteristic is the number of times the i-th data item is accessed in the same stage, k represents the association order and its value ranges from 1 to K, and K represents the maximum association order.

[0040] Among them, the access frequency of the i-th type of data item This reflects the frequency of access to the i-th data item within a given stage. A higher frequency value indicates greater access popularity for the i-th data item within that stage. (Data item vector group) k-th order correlation strength The conditional probability of triggering access to the j-th data item after accessing the i-th data item with a k-step interval is quantified. This is used to capture and quantify the direct or indirect correlation between the i-th and j-th data items, and the correlation strength. The larger the value, the greater the probability that accessing the i-th data item will be triggered after a k-step interval, and the stronger the k-order association between the i-th and j-th data items in the same stage.

[0041] Furthermore, based on actual application scenarios, attenuation factors are set. For each order of association, an exponential function is used to map the attenuation factors accordingly, obtaining the mapping values ​​of each order of association. Based on the maximum association order, the mapping values ​​of each order of association are accumulated to obtain the total mapping value. The ratio between the mapping value of each order of association and the total mapping value is used as the weight coefficient corresponding to each order of association. The product between the association strength of a certain data item vector group under a certain order of association and the weight coefficient corresponding to that order of association is used as the trigger probability of that data item vector group under that order of association. The accumulated value of the trigger probability of that data item vector group under each order of association is used as the multi-order comprehensive trigger probability of that data item vector group. By traversing the set of data item vector groups in each stage, the multi-order comprehensive trigger probability of each data item vector group in each stage is obtained in the same way.

[0042] Wherein, the weight coefficient of the k-th order association at any stage is: The calculation formula is as follows:

[0043]

[0044] In the formula, Represents the attenuation factor and its value range is Preferably, in this invention, the attenuation factor is set to 0.5, k represents the correlation order and its value ranges from 1 to K, and K represents the maximum correlation order;

[0045] The data item vector group consisting of the i-th data item and the j-th data item in the same stage The multi-level combined trigger probability is The calculation formula is as follows:

[0046]

[0047] In the formula, The weight coefficients representing the k-th order association. Represents a group of data items in the same stage. The k-th order correlation strength, where K represents the maximum correlation order.

[0048] The weighting coefficient reflects the characteristic that the correlation decreases as the correlation order increases. The higher the correlation order, the lower the correlation and correlation value between the two data items within the data item vector group, and the smaller the impact on the access chain formed by the corresponding development session. The weighting coefficients of each correlation order satisfy the following: ,and The multi-order integrated trigger probability combines association behaviors from order 1 to order K, and is used to comprehensively analyze and quantify the association within the access chain formed by all development sessions of the data item vector group at the corresponding stage. Multi-level comprehensive trigger probability The larger the value, the stronger the correlation between the i-th data item and the j-th data item in the same stage, and the greater the probability of accessing the j-th data item after accessing the i-th data item in the corresponding stage.

[0049] Step S4: Based on the access frequency and multi-level comprehensive trigger probability, calculate the comprehensive hierarchical weight of each data item in each stage.

[0050] Specifically, based on a set of data item vector groups for a certain stage, the data item vector groups containing a certain data item and following that data item are extracted as the preceding association set for that data item. The multi-order comprehensive trigger probability of each data item vector group within the preceding association set is obtained. Based on the access frequency of each data item in that stage, the access frequencies of the remaining data items (excluding the data item type) in that stage are extracted and accumulated to obtain the access association feature of that data item. The multi-order comprehensive trigger probability of each data item vector group within the preceding association set is then calculated sequentially, multiplying it with the access frequency of another data item in the corresponding data item vector group. The correlation value feature of the data item is obtained by summing the correlation value feature and the access correlation feature. The ratio between the correlation value feature and the access correlation feature of the data item is used as the correlation strength weight of the data item. The access frequency of the data item is used as the access frequency weight of the data item. The weight coefficient of the access frequency weight and the weight coefficient of the correlation strength weight are set and the sum of the two is 1. The sum of the products of the access frequency weight and the correlation strength weight of the data item with the corresponding weight coefficients is used as the comprehensive hierarchical weight of the data item. Similarly, the comprehensive hierarchical weight of each type of data item in each stage is obtained by traversing all types of data items in each stage.

[0051] Among them, the comprehensive hierarchical weight of the i-th data item in the same stage is: The calculation formula is as follows:

[0052]

[0053] In the formula, This represents the access frequency weight of the i-th type of data item within the same stage, i.e., the access frequency of the i-th type of data item within the same stage. This represents the association strength weight of the i-th data item within the same stage. This represents the associated value characteristic of the i-th data item within the same stage. This represents a data item vector group consisting of the h-th data item and the i-th data item in the same stage. The multi-level comprehensive trigger probability, This represents the access association characteristics of the i-th type of data item within the same stage. This indicates the access frequency of the h-th type of data item within the same stage. This represents the set of all data items of different types in the same stage, excluding the i-th type. In other words, it represents the set of all data item types in the same stage that does not include the i-th type. The weighting coefficient represents the access frequency weight. The weight coefficients representing the association strength weights, and satisfying the following conditions: .

[0054] Among them, the weighting coefficient and weighting coefficients To balance the weight of access frequency and association strength in the final hierarchical decision, preferably, the present invention sets... , The access frequency weight measures the access popularity of a data item, while the association strength weight measures the associated value of a data item. The comprehensive stratification weight integrates the access popularity and associated value of a data item, forming a comprehensive basis for stratification decisions. The quantification process of the association strength weight uses multi-order comprehensive trigger probability and all related data item vector groups for calculation, comprehensively considering the contribution of all association paths, avoiding the excessive influence of a single strong association, thereby improving the stability of the decision. Therefore, the larger the comprehensive stratification weight of a data item, the greater the probability and frequency of access to the corresponding data item, and the more likely the corresponding data item should be prioritized for allocation to the high-speed storage layer.

[0055] Step S5: Based on the comprehensive hierarchical weights, determine the data hierarchical storage strategy for each stage.

[0056] Specifically, based on the comprehensive hierarchical weight of each data item in each stage, all types of data items in each stage are sorted in descending order of comprehensive hierarchical weight to obtain the hierarchical priority sequence of data items in each stage; the current stage of the current software development process is obtained, and based on the distribution and remaining amount of device storage space, the various data items corresponding to the current stage are sequentially divided into high-speed storage layer, performance storage layer, and capacity storage layer according to the hierarchical priority sequence of data items in the corresponding stage; similarly, when a change in the stage of the software development process is detected, the data hierarchical storage strategy of the corresponding stage is dynamically adjusted according to the same data item division method.

[0057] Furthermore, after the current software development process is completed, the atomic access event records collected at each stage can be used as historical data to update the database, thereby updating and optimizing the comprehensive hierarchical weight of each data item at each stage, as well as the data hierarchical storage strategy at each stage, to achieve intelligent hierarchical management of software development data and form a closed-loop optimization structure.

[0058] Specifically, data items in each stage can be stored in layers according to their hierarchical priority sequence and a percentage-based partitioning method. For example, the first 10% of data items in the hierarchical priority sequence can be partitioned into the high-speed storage layer, the next 30% into the performance storage layer, and the last 60% into the capacity storage layer. The percentage threshold for partitioning can be adjusted according to the actual application scenario and the actual storage space of the device.

[0059] Alternatively, a minimum-maximum normalization method can be used to normalize the overall stratification weight of each data item in each stage, and then a stratification threshold can be set for each stage. Based on the stratification threshold, the data items in each stage can be stratified. For example, for a certain stage, data items with a normalized overall stratification weight greater than or equal to 0.7 can be assigned to the high-speed storage layer, data items with a normalized overall stratification weight greater than or equal to 0.3 and less than 0.7 can be assigned to the performance storage layer, and data items with a normalized overall stratification weight less than 0.3 can be assigned to the capacity storage layer. The stratification threshold for each stage can also be adjusted according to the actual application scenario and the actual storage space of the device.

[0060] Step S6: Obtain the latest atomic access event records for the current stage in real time, and determine the data preheating strategy based on the multi-level comprehensive trigger probability related to the corresponding data items.

[0061] Specifically, the latest atomic access event record of the current stage is collected, and the same preprocessing method is used to preprocess the latest atomic access event record of the current stage to obtain the category number corresponding to the latest accessed data item in the latest atomic access event record. The data item vector group with the same data item preceding it is extracted, and the next data item in each data item vector group is used as the reserve data item for the next access. The corresponding multi-level comprehensive trigger probability is obtained, and each data item vector group is sorted in descending order of multi-level comprehensive trigger probability to obtain the preheating priority order of the reserve data items. Based on the size of the device cache space, the corresponding data items are cached in order of preheating priority. Similarly, when the latest atomic access event record is updated, the data preheating strategy is updated in the same way.

[0062] Data preheating refers to loading data into memory in advance before the actual query or access arrives, in order to improve system performance and response speed. Since the multi-level comprehensive trigger probability of a data item vector group can reflect the probability of accessing the next type of data item in the data item vector group after accessing the current data item at the corresponding stage, the type of data item to be accessed next can be predicted based on the magnitude of the multi-level comprehensive trigger probability. This allows us to obtain the preheating priority order of data items, and then load and cache the corresponding high-priority data items to achieve forward-looking preheating of software development data, thereby reducing latency and improving access efficiency.

[0063] This invention also discloses a cloud computing-based software development data layering processing system for implementing the aforementioned cloud computing-based software development data layering processing method. The system structure is as follows: Figure 2 As shown, the system includes a processor, a memory, a communication interface, and a cloud computing platform. The processor stores computer program instructions for implementing the aforementioned cloud-based software development data layering processing method. The communication interface is connected to the cloud computing platform. This system can classify and process large amounts of software development data through the cloud computing platform to identify and statistically analyze the access records of developers to various types of data during the software development process, thereby reducing the system's hardware performance requirements, reducing data storage, and improving system operating efficiency.

[0064] The embodiments included in this invention are merely descriptions of preferred embodiments of the invention and are not limited to the precise structures described above and shown in the accompanying drawings. Various modifications and changes can be made without departing from the scope of protection. Any variations and improvements made by those skilled in the art to the technical solutions of this invention without departing from the design concept of this invention should fall within the scope of protection of this invention.

Claims

1. A cloud computing-based software development data layering processing method, characterized in that: The process involves acquiring and preprocessing atomic access event records from each stage of historical software development to obtain data item access sequences for each stage. These sequences are then divided into several development sessions, and the temporal access relationships between data items within each session are extracted to generate a multi-level association table for each stage's data items. Based on this table, the access frequency of each data item and the multi-level comprehensive trigger probability between any two data items are calculated. Finally, based on the access frequency and multi-level comprehensive trigger probability, the comprehensive hierarchical weight of each data item in each stage is calculated. Based on the comprehensive hierarchical weights, the data hierarchical storage strategy for each stage is determined respectively; The system acquires the latest atomic access event records for the current stage in real time and determines the data preheating strategy based on the multi-level comprehensive trigger probability related to the corresponding data items. The process involves obtaining the access frequency weight of each data item in each stage based on access frequency, obtaining the association strength weight of each data item in each stage based on multi-level comprehensive trigger probability, setting weight coefficients for both, and using the sum of the products of the access frequency weight and association strength weight of the data item with the corresponding weight coefficients as the comprehensive hierarchical weight of the corresponding data item. This process is repeated for all types of data items in each stage to obtain the comprehensive hierarchical weight of each data item in each stage. Based on the comprehensive hierarchical weight of each data item in each stage, all types of data items in each stage are sorted in descending order of comprehensive hierarchical weight to obtain the hierarchical priority sequence of data items in each stage. The current stage of the current software development process is obtained. Based on the distribution and remaining amount of device storage space, the various data items corresponding to the current stage are divided into high-speed storage layer, performance storage layer, and capacity storage layer in sequence according to the hierarchical priority sequence of data items in the corresponding stage. Similarly, when a change in the stage of the software development process is detected, the data hierarchical storage strategy of the corresponding stage is dynamically adjusted according to the same data item division method.

2. The method for layered processing of software development data based on cloud computing according to claim 1, characterized in that, The process of acquiring and preprocessing atomic access event records from each stage of historical software development to obtain data item access sequences for each stage includes: collecting atomic access event records from each stage of historical software development through a data acquisition agent deployed on a cloud computing development platform, with the collected fields including data item ID, timestamp, and access path; standardizing the timestamps of all atomic access event records from each stage; classifying data items under the same access path in the same stage into a category and numbering them based on the data item ID and access path in the atomic access event records of each stage to obtain the number of data item categories accessed in each stage; and then sorting the category numbers corresponding to all data items in each stage according to the timestamp from front to back based on the category number corresponding to the data item to obtain the data item access sequence for each stage.

3. The cloud computing-based software development data layering processing method according to claim 2, characterized in that, The process of dividing the data item access sequence of each stage into several development sessions, extracting the temporal access relationship between data items within the development session, and generating a multi-level association table between data items of each stage includes: calculating the time difference between two adjacent data items in the data item access sequence of each stage based on the timestamps corresponding to all data items in the data item access sequence of each stage; setting an access interval threshold based on actual application scenarios and access duration experience in the software development process; dividing consecutive adjacent data items in the data item access sequence of the same stage whose time difference is less than or equal to the access interval threshold into a development session; dividing the data item access sequence of each stage in turn to obtain several development sessions corresponding to each stage; combining the type numbers of all data items in the data item access sequence of a certain stage with the numbers of two types of data items to generate two data item vector groups with opposite access directions; combining them in turn to obtain the data item vector group set of that stage; and similarly obtaining the data item vector group set of each stage.

4. The method for layered processing of software development data based on cloud computing according to claim 3, characterized in that, The process of generating a multi-level association table between data items at each stage further includes: based on the same development session within the same stage's data item access sequence, according to the sequential access relationship from front to back, using the difference in the order of two different types of data items within the data item access sequence as the association order of the corresponding data item vector group; based on experience of continuously accessing different types of data items during software development, setting a maximum association order; when the association order of a data item vector group is less than or equal to the maximum association order, determining that there is an association relationship of the corresponding order between the two data items corresponding to that data item vector group; based on the set of data item vector groups at a certain stage and all development sessions, counting the number of times each order of association relationship of each data item vector group appears in all development sessions, generating a multi-level association table between data items at that stage, and similarly obtaining a multi-level association table between data items at each stage.

5. The cloud computing-based software development data layering processing method according to claim 4, characterized in that, The multi-level association table based on each stage calculates the access frequency of each data item in each stage and the multi-level comprehensive trigger probability between each pair of data items. This includes: based on the data item access sequence of each stage, counting the access count of each data item in each stage, and the total access count of data items within the data item access sequence of each stage; using the ratio between the access count of a certain data item in a certain stage and the total access count in that stage as the access frequency of the corresponding data item in that stage; similarly, obtaining the access frequency of each data item in each stage; and based on the multi-level association table between data items in each stage, extracting each data item from the data item vector set of each stage. Based on the frequency of occurrence of each level of association relationship in the item vector group, the frequency of occurrence of a certain level of association relationship in a certain stage of a certain data item vector group is taken as the association feature of the data item vector group under that level of association relationship. The frequency of access of the previous type of data item in the data item vector group in that stage is taken as the total association feature of the data item vector group. The ratio between the association feature and the total association feature is taken as the association strength of the data item vector group under that level of association relationship. Similarly, the association strength of the data item vector group under each level of association relationship is obtained. By traversing the set of data item vector groups in each stage, the association strength of each data item vector group under each level of association relationship in each stage is obtained.

6. The method for layered processing of software development data based on cloud computing according to claim 5, characterized in that, The method for calculating the access frequency of each data item and the multi-level comprehensive trigger probability between each pair of data items based on the multi-level association table of each stage also includes: setting a decay factor based on the actual application scenario; mapping the decay factor to each level of association using an exponential function to obtain the mapping value of each level of association; summing the mapping values ​​of each level of association based on the maximum association level to obtain the total mapping value; using the ratio between the mapping value of each level of association and the total mapping value as the weight coefficient corresponding to each level of association; using the product between the association strength of a data item vector group in a certain stage under a certain level of association and the weight coefficient corresponding to that level of association as the trigger probability of that data item vector group under that level of association; using the sum of the trigger probabilities of that data item vector group under each level of association as the multi-level comprehensive trigger probability of that data item vector group; and similarly obtaining the multi-level comprehensive trigger probability of each data item vector group in each stage by traversing the data item vector group set of each stage.

7. A cloud computing-based software development data layering processing method according to any one of claims 3 to 6, characterized in that, The method of calculating the comprehensive hierarchical weight of each data item in each stage based on access frequency and multi-level comprehensive trigger probability includes: extracting the data item vector group containing a certain data item and following it as the preceding association set of the data item based on the data item vector group set of a certain stage; obtaining the multi-level comprehensive trigger probability of each data item vector group in the preceding association set of the data item; extracting the access frequency of the other types of data items in the stage excluding the data item type based on the access frequency of each data item in the stage and accumulating them to obtain the access association feature of the data item; and sequentially calculating the multi-level comprehensive trigger probability of each data item vector group in the preceding association set. The association value feature of a data item is obtained by multiplying the trigger probability by the access frequency of another data item in the corresponding data item vector group and summing the results. The ratio between the association value feature and the access association feature of the data item is used as the association strength weight of the data item, and the access frequency of the data item is used as the access frequency weight of the data item. The sum of the weight coefficient of the access frequency weight and the weight coefficient of the association strength weight is 1. Based on the access frequency weight and the association strength weight of the data item and their corresponding weight coefficients, the comprehensive hierarchical weight of the data item is obtained. Then, the comprehensive hierarchical weight of each data item in each stage is calculated in turn.

8. A cloud computing-based software development data layering processing method according to any one of claims 3 to 6, characterized in that, The real-time acquisition of the latest atomic access event records for the current stage and the determination of the data preheating strategy based on the multi-level comprehensive trigger probability related to the corresponding data items include: collecting the latest atomic access event records for the current stage; preprocessing the latest atomic access event records for the current stage using the same preprocessing method to obtain the category number corresponding to the latest accessed data item in the latest atomic access event records; extracting the data item vector group with the same type of data item preceding it; using the latter type of data item in each data item vector group as the reserve data item for the next access; obtaining the corresponding multi-level comprehensive trigger probability; sorting each data item vector group in descending order of multi-level comprehensive trigger probability to obtain the preheating priority order of the reserve data items; and caching the corresponding data items in order of preheating priority based on the size of the device cache space. Similarly, when the latest atomic access event record is updated, the data preheating strategy is updated in the same way.

9. A cloud computing-based software development data layering processing system, characterized in that: It includes a processor, a memory, a communication interface, and a cloud computing platform. The processor stores computer program instructions for implementing the cloud computing-based software development data layering processing method according to any one of claims 1 to 8. The communication interface is communicatively connected to the cloud computing platform.

Images