A green building material intelligent scheduling method and system based on reinforcement learning

Abstract

The application relates to a green building material intelligent scheduling method and system based on reinforcement learning. The method comprises the following steps: acquiring building material characteristic data; arranging the building materials according to production urgency based on the building material characteristic data and preset inventory data to obtain a basic production scheduling scheme; the basic production scheduling scheme is used for representing the production sequence of each building material, and the production sequence is determined based on the building material characteristic data and the inventory data; adjusting the arrangement sequence of the building materials in the basic production scheduling scheme according to a preset experience database to obtain an optimized production scheduling scheme; the experience database comprises the basic production scheduling scheme of a historical scheduling process; and the optimized production scheduling scheme is a new building material production sequence obtained by scheduling the basic production scheduling scheme according to the basic production scheduling scheme of the historical scheduling process. The method can improve the production scheduling decision quality.

Description

Technical Field

[0001] This invention belongs to the field of intelligent scheduling in building materials manufacturing, and in particular relates to a green building materials intelligent scheduling method and system based on reinforcement learning. Background Technology

[0002] As society places increasingly stringent demands on environmental protection, intelligent scheduling technologies for building materials have emerged. Simultaneously, the complexity of building material manufacturing and engineering projects is constantly increasing, making intelligent scheduling technology for green building materials a crucial factor influencing the success or failure of the entire construction industry's green transformation.

[0003] In existing intelligent scheduling technologies for building materials, static programming algorithms (such as linear programming and genetic algorithms) or human experience are mainly relied upon. The building material scheduling problem is transformed into a Markov decision process. The basic data of building materials and scheduling decisions are reasonably set as state space and action space, a scheduling model is constructed, and the scheduling model is optimized by combining a reward function to achieve green and intelligent scheduling of building materials.

[0004] However, existing intelligent scheduling technologies for building materials primarily rely on historical data and fixed rules for task allocation and production planning. When multiple building materials with varying production cycles need to be processed simultaneously, rationally allocating their processing order becomes a core challenge affecting overall efficiency. Further complicating matters, the diversity of building material characteristics exacerbates the difficulty of dynamically optimizing the scheduling order. Because the urgency of project needs changes as the project progresses, fixed scheduling rules cannot adapt to this dynamism. Delayed scheduling decisions leading to production waste also contradict the requirements of green production, resulting in a decline in overall scheduling efficiency. Summary of the Invention

[0005] Therefore, it is necessary to provide a green building materials intelligent scheduling method and system based on reinforcement learning that can dynamically schedule production sequence and improve the overall scheduling decision efficiency in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a green building materials intelligent scheduling method based on reinforcement learning, including:

[0007] Obtain building material characteristic data; building material characteristic data includes preset building material production characteristics, energy consumption characteristics, and order information;

[0008] Based on building material characteristic data and preset inventory data, building materials are arranged according to their production urgency to obtain a basic production scheduling plan. The basic production scheduling plan is used to characterize the production sequence of each building material, and the production sequence is determined based on the building material characteristic data and inventory data.

[0009] Based on a pre-set experience database, the arrangement order of building materials in the basic production scheduling scheme is adjusted to obtain an optimized production scheduling scheme. The experience database includes the basic production scheduling scheme of the historical scheduling process. The optimized production scheduling scheme is a new production order of building materials obtained by scheduling the basic production scheduling scheme according to the basic production scheduling scheme of the historical scheduling process.

[0010] Furthermore, based on building material characteristic data and preset inventory data, building materials are arranged according to their production urgency to obtain a basic production scheduling plan, including:

[0011] Feature extraction is performed on the building material characteristic data to obtain the building material characteristic matrix;

[0012] Based on the preset urgency score of the engineering projects, the building material characteristic matrix is ​​prioritized to obtain a dynamic priority ranking list;

[0013] Based on the dynamic priority sorting list and the preset inventory data, the preset building materials are arranged to obtain the basic production scheduling plan.

[0014] Furthermore, feature extraction is performed on the building material characteristic data to obtain a building material characteristic matrix, including:

[0015] Independent component analysis (ICA) algorithms are used to extract latent features from building material property data to obtain an independent component matrix.

[0016] The EM algorithm is used to complete the independent component matrix and obtain the complete independent component matrix.

[0017] The building material property matrix is ​​obtained by Z-score normalization of the complete independent component matrix.

[0018] Furthermore, based on the preset urgency score of the engineering projects, the building material characteristic matrix is ​​prioritized to obtain a dynamic priority ranking list, including:

[0019] For each building material characteristic in the building material characteristic matrix, and combining the project urgency score, the entropy weight method is used to calculate the corresponding urgency weight using the following formula:

[0020]

[0021] Among them, w j p is the urgency weight of the j-th building material. ij It is the normalized value of the i-th characteristic of the j-th building material. ∈ is the j-th characteristic of the i-th building material, ∈ is the smoothing factor, m is the amplitude coefficient, S is the urgency score, and L is the score threshold.

[0022] Based on the urgency weight, the building materials corresponding to the urgency weight are sorted in order of numerical value to obtain a dynamic priority sorting list.

[0023] Furthermore, based on the dynamic priority sorting list and the preset inventory data, the preset building materials are arranged to obtain a basic production scheduling plan, including:

[0024] Based on the dynamically prioritized list and preset inventory data, the list fitness corresponding to the dynamically prioritized list is calculated using the following formula:

[0025]

[0026] Where, f(x) i w is the fitness of the list corresponding to the i-th dynamically prioritized list. i Here, r is the urgency weight of the i-th building material, α is the delay penalty coefficient, and r is the urgency weight. i (x) is the position of the i-th building material in the dynamic priority ranking list, λ is the penalty coefficient, and δ i (x) is the position penalty factor, I (s,i) It is standard inventory data, I (n,i) It is inventory quantity data;

[0027] Randomly adjust the order of building materials in the dynamic priority sorting list to obtain a new dynamic priority sorting list;

[0028] Calculate the fitness of each dynamic priority sorting list and determine the dynamic priority sorting list with the highest fitness as the basic production scheduling scheme.

[0029] Furthermore, based on a pre-set experience database, the arrangement order of building materials in the basic production scheduling scheme is adjusted to obtain an optimized production scheduling scheme, including:

[0030] Historical basic production scheduling schemes are extracted from the experience database and a set of historical basic production scheduling schemes is formed.

[0031] A deep learning-based autoencoder is used to extract deep features from the historical basic production scheduling scheme set to obtain a historical scheme deep feature dataset.

[0032] A graph convolutional network is used to cluster the deep feature dataset of historical schemes to obtain a positive sample label set; historical basic production scheduling schemes with labels corresponding to positive samples are selected to form a positive sample historical scheme set.

[0033] The Apriori algorithm is used to perform feature frequency statistics and rule generation on the historical scheme set of positive samples to obtain association rule data.

[0034] For each positive historical solution in the final set of positive historical solutions, the similarity to the basic production scheduling solution is calculated using the following formula; the positive historical solution with the highest similarity value is determined as the reference positive solution:

[0035]

[0036] in, It represents the similarity between the l-th building material and the j-th positive sample historical scheme. This is the building material characteristic matrix of the l-th building material, x j This is the building material characteristic matrix of the l-th building material in the j-th positive sample historical scheme, where n is the total number of building material characteristics, and w i It is the urgency weight of the i-th building material characteristic;

[0037] An Actor-Critic network based on deep reinforcement learning is used to adjust the basic production scheduling scheme according to the reference positive sample scheme and association rule data, so as to obtain an optimized production scheduling scheme.

[0038] Furthermore, the method also includes:

[0039] Obtain actual production efficiency data and inventory change data for building materials;

[0040] Based on the optimized production scheduling plan, actual production efficiency data, and inventory change data, the corresponding efficiency deviation and inventory change deviation are calculated.

[0041] Based on efficiency deviation and inventory change deviation, a Bayesian probability model is used to calculate the effectiveness probability of the optimized production scheduling scheme.

[0042] When the probability of effectiveness reaches a preset threshold, the optimized production scheduling plan will be added to the experience database.

[0043] Secondly, this application also provides a green building materials intelligent scheduling system based on reinforcement learning, comprising:

[0044] The data acquisition module is used to acquire building material characteristic data; the building material characteristic data includes preset production characteristics, energy consumption characteristics and order information of building materials;

[0045] The preliminary scheduling module is used to arrange building materials according to their production urgency based on building material characteristic data and preset inventory data to obtain a basic production scheduling plan. The basic production scheduling plan is used to characterize the production sequence of each building material, and the production sequence is determined based on the building material characteristic data and inventory data.

[0046] The final scheduling module is used to adjust the arrangement order of building materials in the basic production scheduling scheme according to the preset experience database to obtain an optimized production scheduling scheme. The experience database includes the basic production scheduling scheme of the historical scheduling process. The optimized production scheduling scheme is a new production order of building materials obtained by scheduling the basic production scheduling scheme according to the basic production scheduling scheme of the historical scheduling process.

[0047] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the reinforcement learning-based intelligent scheduling methods for green building materials described in the first aspect of this application.

[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, provides any of the reinforcement learning-based intelligent scheduling methods for green building materials described in the first aspect of the application.

[0049] The aforementioned reinforcement learning-based intelligent scheduling method and system for green building materials acquires building material characteristic data, including preset production characteristics, energy consumption characteristics, and order information. Based on this data and preset inventory data, building materials are arranged according to their production urgency to obtain a basic production scheduling scheme. This scheme characterizes the production sequence of each building material, determined by the building material characteristic data and inventory data. An optimized production scheduling scheme is obtained by adjusting the arrangement order of building materials in the basic production scheduling scheme based on a preset experience database, which includes historical basic production scheduling schemes. The optimized scheme is a new production sequence of building materials obtained by adjusting the basic production scheduling scheme based on historical data. This achieves a dynamic adjustment of the production sequence based on the production time, energy consumption, and inventory data of building materials, reducing production waste and improving the efficiency of green production scheduling decisions. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a reinforcement learning-based intelligent scheduling method for green building materials, provided as an embodiment of this application;

[0052] Figure 2This is a structural block diagram of a green building materials intelligent scheduling system based on reinforcement learning, provided as an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] In one embodiment, such as Figure 1 As shown, a reinforcement learning-based intelligent scheduling method for green building materials is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes steps S101-S103, wherein:

[0055] S101, Obtain building material characteristic data; building material characteristic data includes preset building material production characteristics, energy consumption characteristics, and order information.

[0056] Specifically, the terminal acquires production characteristic data, energy consumption characteristic data, and order information data of building materials that require production scheduling. For illustrative purposes, the types and quantities of building materials acquired are determined based on the actual production scheduling needs in operation. Optionally, the production characteristic data includes, but is not limited to, the production time, production cost, and quantity of raw materials for the building materials. For illustrative purposes, energy consumption characteristics include, but are not limited to, energy consumption data and environmental protection ratings.

[0057] S102, Based on the building material characteristic data and the preset inventory data, the building materials are arranged according to the production urgency to obtain a basic production scheduling plan; the basic production scheduling plan is used to characterize the production sequence of each building material, and the production sequence is determined based on the building material characteristic data and inventory data.

[0058] Specifically, the terminal compares the building material characteristic data with the preset inventory data, calculates the urgency of each building material that needs to be scheduled, and sorts the corresponding building materials in descending order of urgency to obtain a basic production scheduling plan.

[0059] S103, Based on the preset experience database, adjust the arrangement order of building materials in the basic production scheduling scheme to obtain an optimized production scheduling scheme; the experience database includes the basic production scheduling scheme of the historical scheduling process; the optimized production scheduling scheme is a new production order of building materials obtained by scheduling the basic production scheduling scheme according to the basic production scheduling scheme of the historical scheduling process.

[0060] Specifically, the terminal obtains a preset experience database, retrieves the basic production scheduling scheme of the historical scheduling process from the experience database, calculates and compares the historical basic production scheduling scheme with the existing basic production scheduling scheme, obtains the closest one, and continues to adjust the basic production scheduling scheme according to the historical basic production scheduling scheme to obtain an optimized production scheduling scheme.

[0061] This embodiment acquires characteristic data of building materials requiring production scheduling, including production cost data, order information, and energy consumption data. Combined with inventory data, the building materials are sorted according to their production urgency to obtain a basic production scheduling plan. The system then searches a historical experience database for the historical scheduling plan that best matches the basic production scheduling plan. Based on the experience and results of this plan, the production order of building materials in the basic production scheduling plan is adjusted to obtain an optimized production scheduling plan. This achieves dynamic sequential arrangement based on the diversity of building materials, reduces production waste, and improves the efficiency of overall scheduling decisions.

[0062] In one embodiment, based on building material characteristic data and preset inventory data, building materials are arranged according to their production urgency to obtain a basic production scheduling plan, including:

[0063] S201, feature extraction is performed on the building material characteristic data to obtain the building material characteristic matrix.

[0064] Specifically, the terminal extracts features from the building material characteristic data, extracts key feature data from the building material characteristic data, and obtains a feature matrix containing linear combinations between key features. If there are missing data in the feature matrix, a numerical missing data completion algorithm is used to fill in the missing values ​​and perform standardization processing to obtain the building material characteristic matrix.

[0065] S202, based on the preset urgency score of the project, prioritize the building material characteristic matrix to obtain a dynamic priority ranking list.

[0066] Specifically, the terminal calculates the urgency weight of each building material characteristic in the building material characteristic matrix based on a preset project urgency score, and sorts all building materials according to their urgency weights to obtain a dynamic priority ranking list. Illustratively, the urgency score can be obtained based on the actual production demand for building materials, or it can be based on building material order data. Optionally, the corresponding building materials can be sorted from the maximum to the minimum dynamic priority to obtain a dynamic priority ranking list.

[0067] S203: Based on the dynamic priority sorting list and combined with the preset inventory data, the preset building materials are arranged to obtain the basic production scheduling plan.

[0068] Specifically, the terminal calculates the fitness of a dynamic priority ranking list by weighting the urgency of building materials with preset inventory data. It then adjusts the production order of building materials in the list, recalculates the fitness, and calculates the fitness for each combination of building material production order combinations. The dynamic priority ranking list with the highest fitness value is then determined as the basic production scheduling scheme. Illustratively, fitness is used to characterize the scheduling benefit under inventory data constraints.

[0069] This embodiment extracts key features from building material characteristic data, captures key feature data and linear combinations between key features, extracts key features and reduces computational redundancy, and obtains a building material characteristic matrix. The building material characteristic matrix is ​​then used to calculate the urgency weight of corresponding building materials based on a preset project urgency score, and sorted according to dynamic priority to obtain a dynamic priority ranking list. Based on this dynamic priority ranking list, a weighted fusion is performed with preset inventory data to obtain a fitness score. The dynamic priority ranking list corresponding to the maximum fitness score is determined as the basic production scheduling scheme. This enables direct response to urgent needs based on production requirements and inventory data, improves production scheduling flexibility, reduces waste caused by overproduction, and meets the needs of green production and green scheduling.

[0070] In one embodiment, feature extraction is performed on the building material characteristic data to obtain a building material characteristic matrix, including:

[0071] S301 uses an independent component analysis algorithm to extract the potential features of building material property data and obtain an independent component matrix.

[0072] Specifically, the terminal employs an independent component analysis (ICA) algorithm to determine the number of independent components to be extracted, sets a separation matrix, and obtains an ICA matrix based on this separation matrix and building material characteristic data. Illustratively, the number of independent components can be calculated through peak analysis of the building material characteristic data. Optionally, this ICA matrix includes linear combinations of several building material characteristics from the building material characteristic data; referring to Table 1, if the number of independent components calculated through peak analysis is 2, where S1 is a linear combination of production time and energy consumption, and S2 is a linear combination of environmental protection level and production cost, and the building material ID can be arranged according to different building material types during actual generation, then the ICA matrix is: {S1, S2}

[0073] Table 1, Example table of independent component matrices:

[0074] Building Materials ID <![CDATA[S1]]> <![CDATA[S2]]> 1 0.8 1.2 2 0.9 1.4 3 1.1 1.5

[0075] S302 uses the EM algorithm to complete the data in the independent component matrix, thus obtaining the complete independent component matrix.

[0076] Specifically, the EM algorithm, short for Expectation Maximization algorithm, estimates the expectation of missing values ​​based on current parameters and predicts missing values ​​through iterative parameter optimization. The terminal calculates the mean and covariance of the building material feature category to which any missing value belongs in the current independent component matrix (ICM), combining this with the numerical values ​​of building material features of similar building materials. Based on this mean and covariance, it calculates the conditional expectation of the missing value and updates the calculated mean and the missing value to maximize the log-likelihood, obtaining the final value of the missing value. This completes the ICM, resulting in a full ICM.

[0077] S303, the complete independent component matrix is ​​Z-score normalized to obtain the building material property matrix.

[0078] Specifically, the terminal performs Z-score standardization on the complete independent component matrix to eliminate dimensional differences and obtain the building material characteristic matrix.

[0079] This embodiment extracts latent features from building material characteristic data using an independent component analysis algorithm, obtaining an independent component matrix containing linear combinations of building materials. The EM algorithm is then used to complete the data in this independent component matrix, resulting in a complete independent component matrix. The complete independent component matrix is ​​then Z-score standardized to eliminate dimensional differences, yielding a building material characteristic matrix. This approach enables the extraction of more interpretable independent features, providing a reliable data source for the generation of basic production scheduling schemes.

[0080] In one embodiment, the building material characteristic matrix is ​​prioritized based on a preset project urgency score to obtain a dynamic priority ranking list, including:

[0081] S401, for each building material characteristic in the building material characteristic matrix, combined with the project urgency score, the entropy weight method is used to calculate the corresponding urgency weight using the following formula:

[0082]

[0083] Among them, w j p is the urgency weight of the j-th building material. ij It is the normalized value of the i-th characteristic of the j-th building material. Let be the j-th characteristic of the i-th building material, ∈ be the smoothing factor, m be the amplitude coefficient, S be the urgency score, and L be the score threshold.

[0084] Specifically, the terminal calculates the dynamic priority of each building material feature in the building material characteristic matrix based on a preset project urgency score. It calculates the normalized weight of each feature for that building material, and then weights all the features together to obtain a unique dynamic priority value for each building material. For example, the urgency weight w of the j-th building material... j This is used to represent the urgency of generating the j-th building material. Optionally, the normalized value p of the i-th building material characteristic of the j-th building material is used. ij This represents the normalized value of the i-th characteristic data of the j-th building material among all the building material characteristics of the j-th building material. Schematic, the j-th characteristic of the i-th building material... This refers to the data of the j-th building material feature of the i-th building material in the building material characteristic matrix obtained in S303. Optionally, a smoothing factor ∈ is used to avoid a denominator of 0. Schematic, n is the total number of feature types contained in the building material characteristic matrix obtained in S303. Optionally, j is the total number of all building material types. Schematic, the amplitude coefficient m controls the growth rate of the basic urgency weight, which can be set according to the requirements of urgency weight in actual work, and the default value is 0.2. Optionally, the urgency score S is the preset urgency score for the i-th building material. Schematic, the scoring threshold L is used to control the threshold at which the urgency score causes the basic urgency weight to increase, which can be set according to the requirements of urgency weight in actual work, and the default value is 7, indicating that when the preset urgency score of the i-th building material is greater than 7, it will cause a linear increase in the urgency weight.

[0085] S402, based on the urgency weight, sort the building materials corresponding to the urgency weight in order of numerical value to obtain a dynamic priority sorting list.

[0086] Specifically, based on the urgency weights of all the building materials obtained above, the corresponding building materials are sorted from largest to smallest according to their urgency weight values ​​to obtain a dynamic priority sorting list.

[0087] This embodiment calculates the urgency weight of each building material in the building material characteristic matrix using the entropy weight method, and incorporates preset urgency scores to influence the urgency weight. Based on the urgency weight, the corresponding building materials are sorted in numerical order to obtain a dynamic priority ranking list. This balances the regularity and personalization of production scheduling, and provides a dynamic priority sequence table containing preset rules and flexible scheduling in actual work for the generation of basic scheduling schemes.

[0088] In one embodiment, based on a dynamic priority sorting list and pre-set inventory data, pre-set building materials are arranged to obtain a basic production scheduling scheme, including:

[0089] S501, based on the dynamic priority sorting list and preset inventory data, calculate the list fitness corresponding to the dynamic priority sorting list using the following formula:

[0090]

[0091] Where, f(x) i w is the fitness of the list corresponding to the i-th dynamically prioritized list. i Here, r is the urgency weight of the i-th building material, α is the delay penalty coefficient, and r is the urgency weight. i (x) is the position of the i-th building material in the dynamic priority ranking list, λ is the penalty coefficient, and δ i (x) is the position penalty factor, I (s,i) It is standard inventory data, I (n,i) This is inventory quantity data.

[0092] Specifically, the terminal calculates the inventory penalty for the building material production scheduling scheme in the dynamic priority sequence list based on the aforementioned obtained dynamic priority sequence list and the preset inventory data, thus obtaining the list fitness corresponding to the dynamic priority sequence list. Illustratively, the list fitness f(x) corresponding to the i-th dynamic priority sorting list... i This represents the scheduling benefit of the i-th dynamic priority ranking list. A higher benefit indicates a more environmentally friendly and efficient production sequence for building materials included in the dynamic priority ranking list. Optionally, a delay penalty coefficient α represents the position attenuation strength; a larger value indicates a higher weight for higher-ranking items. Illustratively, the position r of the i-th building material in the dynamic priority ranking list... i (x) represents the position of the i-th building material in the dynamic priority ranking list, indicating its production order within the entire production plan. Optionally, the penalty coefficient λ represents the intensity of the inventory shortage penalty, which can be set according to the actual standards for inventory shortages in practice. Illustratively, the position penalty factor δ... i (x) and the position r of the i-th building material in the dynamic priority sorting list i (x) Related, the later the position, the greater the penalty. Optionally, inventory standard data I (s,i) This is the minimum inventory quantity for this building material set in the preset inventory data. It can be set according to the actual inventory requirements in practice. (Illustratively, the inventory quantity data I...) (n,i) It is the actual inventory quantity of this building material in the preset inventory data.

[0093] S502, randomly adjust the order of building materials in the dynamic priority sorting list to obtain a new dynamic priority sorting list.

[0094] Specifically, the terminal randomly adjusts the production order of building materials in the dynamic priority sorting list to obtain a new dynamic priority sorting list. Illustratively, it is necessary to adjust all possible combinations of building material production order to obtain a dynamic priority sorting list corresponding to all order combinations.

[0095] S503, calculate the fitness of each dynamic priority sorting list, and determine the dynamic priority sorting list with the highest fitness as the basic production scheduling scheme.

[0096] Specifically, the terminal calculates the fitness of the dynamic priority sorting list corresponding to all sequence combinations, and determines the dynamic priority sorting list with the highest fitness as the basic production scheduling scheme, proving that the dynamic priority sorting list brings the highest scheduling benefits.

[0097] This embodiment constructs a list fitness formula, adjusts the production order of building materials in the dynamic priority ranking list, calculates the list fitness corresponding to all order combinations, and determines the dynamic priority ranking list with the highest list fitness as the basic production scheduling scheme. This realizes the comprehensive generation of building material production order by combining the production urgency and the preset inventory quantity, which is more in line with the diminishing returns law of "prioritizing high-level building materials" in actual scheduling.

[0098] In one embodiment, based on a preset experience database, the arrangement order of building materials in the basic production scheduling scheme is adjusted to obtain an optimized production scheduling scheme, including:

[0099] S601 extracts historical basic production scheduling schemes from the experience database to form a set of historical basic production scheduling schemes.

[0100] Specifically, the terminal extracts historical basic production scheduling schemes from the experience database, forming a historical basic production scheduling scheme set. This historical basic production scheduling scheme includes basic production scheduling schemes generated by S503 during recorded historical scheduling processes.

[0101] S602 employs a deep learning-based autoencoder to extract deep features from a historical basic production scheduling scheme set, resulting in a historical scheme deep feature dataset.

[0102] Specifically, the terminal employs a deep learning-based autoencoder to map historical basic production scheduling scheme data into a low-dimensional latent space, obtaining deep features of the historical basic production scheduling schemes, forming a historical scheme deep feature dataset. Illustratively, the historical scheme deep feature data may include the aforementioned independent component data.

[0103] S603 uses a graph convolutional network to cluster the historical scheme deep feature dataset to obtain a positive sample label set; select the historical basic production scheduling schemes corresponding to the labels of the positive samples to form a positive sample historical scheme set.

[0104] Specifically, the terminal employs a graph convolutional network to transform the aforementioned historical scheme deep feature dataset into a graph structure. Using the historical scheme deep feature data as node data, it calculates the feature similarity between historical schemes and schemes, and uses this feature similarity as edge weights to construct the graph structure. Based on this feature similarity, the historical schemes are clustered into positive sample schemes and negative sample schemes. The historical basic production scheduling schemes labeled as positive samples are selected to form a set of positive sample historical schemes. Illustratively, historical schemes are classified according to clustering probabilities, which represent the success probability of each historical scheme.

[0105] S604 uses the Apriori algorithm to perform feature frequency statistics and rule generation on the historical scheme set of positive samples to obtain association rule data.

[0106] Specifically, the Apriori algorithm is used to capture the frequency of building material features in a historical set of positive samples, and to analyze the linear relationships and implementation effects between these features to obtain association rule data. Illustratively, an association rule can be: {Linear combination score of production time and energy consumption > 0.8 and urgency score > 7 --> Success (confidence 85%)}. The Apriori algorithm is an association rule mining algorithm that scans the dataset multiple times to count the co-occurrence frequency of any feature 1 and another feature 2, and calculates the support and confidence of this combination. When the support and confidence n of this combination are greater than the preset minimum support and minimum confidence, the association rule for that combination is obtained, generally in the form of {Feature 1 -> Feature 2, confidence n}.

[0107] S605, for each positive historical solution in the final set of positive historical solutions, calculate the similarity with the basic production scheduling solution using the following formula; determine the positive historical solution with the highest similarity value as the reference positive solution:

[0108]

[0109] in, It represents the similarity between the l-th building material and the j-th positive sample historical scheme. This is the building material characteristic matrix of the l-th building material, x j This is the building material characteristic matrix of the l-th building material in the j-th positive sample historical scheme, where n is the total number of building material characteristics, and w i It is the urgency weight of the i-th building material characteristic.

[0110] Specifically, for each positive historical solution in the set of positive historical solutions, the similarity to the current basic production scheduling solution is calculated. The positive historical solution with the highest similarity value is determined as the reference positive historical solution. Illustratively, the similarity between any building material in the current basic production scheduling solution and a positive historical solution is calculated using a formula. The similarity of all building materials to this positive historical solution is then averaged to obtain the similarity between the basic production scheduling solution and that positive historical solution. Illustratively, the similarity between the l-th building material and the j-th positive historical solution... This is used to characterize the similarity between the l-th building material and the j-th positive sample historical solution. Higher similarity indicates higher reference value for the historical solution. Optionally, the building material characteristic matrix of the l-th building material... This is the building material characteristic matrix obtained from S303. Schematic, the total number of building material characteristics *n* is the number of building material types included in the current basic production scheduling scheme. Schematic, the building material characteristic matrix *x* of the *l*th building material in the *j*th positive sample historical scheme. j This is the building material characteristic matrix of the l-th building material included in the j-th positive sample historical scheme obtained from S604, used to calculate the scheduling similarity with the scheduling similarity matrix of the l-th building material in the current basic production scheduling scheme. Optionally, the urgency weight w of the i-th building material characteristic is... i It is the urgency weight obtained from S401.

[0111] S606 employs a deep reinforcement learning Actor-Critic network to adjust the basic production scheduling scheme based on the reference positive sample scheme and association rule data, thereby obtaining an optimized production scheduling scheme.

[0112] Specifically, the terminal employs a deep reinforcement learning Actor-Critic network, using reference positive sample schemes and association rule data as input. It sets an evaluation state value function and a reward function to adjust the generation order of building materials in the basic production scheduling scheme, resulting in an optimized production scheduling scheme. The Actor-Critic network is an algorithm combining policy gradient and temporal difference. The Actor is the policy function, typically implemented using a neural network; its input is the current state, and its output is a state adjustment action. The Critic is the value function, used to evaluate the Actor's adjustment to the current state.

[0113] This embodiment acquires historical basic generation scheduling schemes, classifies and summarizes the patterns of these schemes using a deep learning autoencoder and the Apriori algorithm, obtains a set of positive historical schemes and association rule data, calculates the similarity between the set of positive historical schemes and the current basic generation scheduling scheme, determines a reference positive sample scheme, and uses an Actor-Critic network to adjust the building material production sequence of the current basic generation scheduling scheme based on the reference positive sample scheme and association rule data, thus obtaining an optimized production scheduling scheme. This achieves the goal of combining current inventory data and the unique production characteristics of each building material, dynamically integrating historical successful experiences, and improving the accuracy of the scheduling strategy.

[0114] In one embodiment, the method further includes:

[0115] S701, obtain actual production efficiency data and inventory change data for building materials.

[0116] Specifically, the terminal acquires actual production efficiency data and inventory change data during the implementation of the optimized production scheduling scheme. Illustratively, the actual production efficiency data can be the production efficiency of the building materials included in the optimized production scheduling scheme during actual production, or it can be obtained by dividing the actual number of days completed by the quantity of building materials completed. Optionally, the inventory change data is the change in the inventory quantity of the building material types included in the optimized production scheduling scheme before and after its implementation.

[0117] S702 calculates the corresponding efficiency deviation and inventory change deviation based on the optimized production scheduling plan, actual production efficiency data, and inventory change data.

[0118] Specifically, the terminal calculates the theoretical efficiency of each building material production item based on the planned production days and quantity in the optimized production scheduling scheme. It then calculates the deviation between the theoretical and actual production efficiency, as well as the error between the planned production quantity and inventory change data, to obtain the corresponding efficiency deviation and inventory change deviation. Illustratively, the efficiency deviation and inventory change deviation can be calculated using the rate of change formula.

[0119] S703 uses a Bayesian probability model to calculate the effectiveness probability of the optimized production scheduling scheme based on efficiency deviation and inventory change deviation.

[0120] Specifically, based on efficiency deviation and inventory change deviation, a Bayesian probability model is used to iteratively calculate and dynamically update the confidence of the scheduling strategy according to the efficiency deviation and inventory change deviation, thereby calculating the effectiveness probability of the optimized production scheduling scheme.

[0121] S704: When the validity probability reaches a preset threshold, the optimized production scheduling scheme will be added to the experience database.

[0122] Specifically, when the effectiveness probability reaches a preset threshold, the optimized production scheduling plan is added to the experience database and can serve as a historical baseline production scheduling plan for the next plan. Indicatively, the preset threshold can be set according to the actual requirements for plan completion standards in practice.

[0123] This embodiment obtains actual production efficiency data and inventory change data of the optimized production scheduling scheme, calculates the deviation from the planned production efficiency and theoretical inventory quantity, and uses a Bayesian probability model to calculate and predict the effectiveness of the optimized production scheduling scheme. Optimized production scheduling schemes with effectiveness reaching a certain threshold are added to the experience database as a possible reference for the optimization direction of the next basic production scheduling scheme. This provides a reliable historical experience reference for the optimization of the next basic production scheduling scheme, realizing the formation of a closed-loop optimization of production scheduling strategy and the continuous improvement of green scheduling efficiency.

[0124] In the aforementioned reinforcement learning-based intelligent scheduling method for green building materials, building material characteristic data is acquired. This data includes preset production characteristics, energy consumption characteristics, and order information for the building materials. Based on the building material characteristic data and preset inventory data, the building materials are arranged according to their production urgency to obtain a basic production scheduling scheme. This basic production scheduling scheme characterizes the production sequence of each building material, which is determined based on the building material characteristic data and inventory data. Based on a preset experience database, the arrangement order of building materials in the basic production scheduling scheme is adjusted to obtain an optimized production scheduling scheme. The experience database includes basic production scheduling schemes from historical scheduling processes. The optimized production scheduling scheme is a new production sequence of building materials obtained by scheduling the basic production scheduling scheme based on historical scheduling processes. This achieves a scheduling scheme that dynamically adjusts the production sequence based on the production time, energy consumption, and other production characteristics of the building materials, as well as inventory data, reducing production waste and improving the efficiency of green production scheduling decisions.

[0125] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0126] Based on the same inventive concept, this application also provides a reinforcement learning-based intelligent scheduling system for green building materials to implement the aforementioned reinforcement learning-based intelligent scheduling method for green building materials. The solution provided by this device is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of a reinforcement learning-based intelligent scheduling system for green building materials provided below can be found in the limitations of the reinforcement learning-based intelligent scheduling method for green building materials described above, and will not be repeated here.

[0127] In one exemplary embodiment, such as Figure 2 As shown, a green building materials intelligent scheduling system 200 based on reinforcement learning is provided, including:

[0128] The data acquisition module 201 is used to acquire building material characteristic data; the building material characteristic data includes preset production characteristics, energy consumption characteristics and order information of building materials;

[0129] The preliminary scheduling module 202 is used to arrange building materials according to their production urgency based on building material characteristic data and preset inventory data to obtain a basic production scheduling plan. The basic production scheduling plan is used to characterize the production sequence of each building material, and the production sequence is determined based on the building material characteristic data and inventory data.

[0130] The final scheduling module 203 is used to adjust the arrangement order of building materials in the basic production scheduling scheme according to the preset experience database to obtain an optimized production scheduling scheme; the experience database includes the basic production scheduling scheme of the historical scheduling process; the optimized production scheduling scheme is a new production order of building materials obtained by scheduling the basic production scheduling scheme according to the basic production scheduling scheme of the historical scheduling process.

[0131] Furthermore, the preliminary scheduling module also includes:

[0132] The matrix building unit is used to extract features from building material characteristic data to obtain a building material characteristic matrix;

[0133] The list building unit prioritizes the building material characteristic matrix based on the preset urgency score of the engineering projects, resulting in a dynamic priority ranking list.

[0134] The basic scheme generation unit is used to arrange the preset building materials according to the dynamic priority sorting list and the preset inventory data to obtain the basic production scheduling scheme.

[0135] Furthermore, the matrix building unit is also used for:

[0136] Independent component analysis (ICA) algorithms are used to extract latent features from building material property data to obtain an independent component matrix.

[0137] The EM algorithm is used to complete the independent component matrix and obtain the complete independent component matrix.

[0138] The building material property matrix is ​​obtained by Z-score normalization of the complete independent component matrix.

[0139] Furthermore, list building blocks are also used for:

[0140] For each building material characteristic in the building material characteristic matrix, and combining the project urgency score, the entropy weight method is used to calculate the corresponding urgency weight using the following formula:

[0141]

[0142] Among them, w j p is the urgency weight of the j-th building material. ij It is the normalized value of the i-th characteristic of the j-th building material. ∈ is the j-th characteristic of the i-th building material, ∈ is the smoothing factor, m is the amplitude coefficient, S is the urgency score, and L is the score threshold.

[0143] Based on the urgency weight, the building materials corresponding to the urgency weight are sorted in order of numerical value to obtain a dynamic priority sorting list.

[0144] Furthermore, the basic scheme generation unit is also used for:

[0145] Based on the dynamically prioritized list and preset inventory data, the list fitness corresponding to the dynamically prioritized list is calculated using the following formula:

[0146]

[0147] Where, f(x) i w is the fitness of the list corresponding to the i-th dynamically prioritized list. i Here, α is the urgency weight of the i-th building material, α is the delay penalty coefficient, and r is the urgency weight. i (x) is the position of the i-th building material in the dynamic priority ranking list, λ is the penalty coefficient, and δ i (x) is the position penalty factor, I (s,i) It is standard inventory data, I (n,i) It is inventory quantity data;

[0148] Randomly adjust the order of building materials in the dynamic priority sorting list to obtain a new dynamic priority sorting list;

[0149] Calculate the fitness of each dynamic priority sorting list and determine the dynamic priority sorting list with the highest fitness as the basic production scheduling scheme.

[0150] Furthermore, the final scheduling module is also used for:

[0151] Historical basic production scheduling schemes are extracted from the experience database and a set of historical basic production scheduling schemes is formed.

[0152] A deep learning-based autoencoder is used to extract deep features from the historical basic production scheduling scheme set to obtain a historical scheme deep feature dataset.

[0153] A graph convolutional network is used to cluster the deep feature dataset of historical schemes to obtain a positive sample label set; historical basic production scheduling schemes with labels corresponding to positive samples are selected to form a positive sample historical scheme set.

[0154] The Apriori algorithm is used to perform feature frequency statistics and rule generation on the historical scheme set of positive samples to obtain association rule data.

[0155] For each positive historical solution in the final set of positive historical solutions, the similarity to the basic production scheduling solution is calculated using the following formula; the positive historical solution with the highest similarity value is determined as the reference positive solution:

[0156]

[0157] in, It represents the similarity between the l-th building material and the j-th positive sample historical scheme. This is the building material characteristic matrix of the l-th building material, x j This is the building material characteristic matrix of the l-th building material in the j-th positive sample historical scheme, where n is the total number of building material characteristics, and w i It is the urgency weight of the i-th building material characteristic;

[0158] An Actor-Critic network based on deep reinforcement learning is used to adjust the basic production scheduling scheme according to the reference positive sample scheme and association rule data, so as to obtain an optimized production scheduling scheme.

[0159] Furthermore, the system also includes: a database collection module, used for:

[0160] Obtain actual production efficiency data and inventory change data for building materials;

[0161] Based on the optimized production scheduling plan, actual production efficiency data, and inventory change data, the corresponding efficiency deviation and inventory change deviation are calculated.

[0162] Based on efficiency deviation and inventory change deviation, a Bayesian probability model is used to calculate the effectiveness probability of the optimized production scheduling scheme.

[0163] When the probability of effectiveness reaches a preset threshold, the optimized production scheduling plan will be added to the experience database.

[0164] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a power supply safety management method as described above.

[0165] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0166] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0167] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A green building materials intelligent scheduling method based on reinforcement learning, characterized in that, The method includes: Acquire building material characteristic data; the building material characteristic data includes preset building material production characteristics, energy consumption characteristics, and order information; Based on the building material characteristic data and the preset inventory data, the building materials are arranged according to their production urgency to obtain a basic production scheduling scheme; the basic production scheduling scheme is used to characterize the production sequence of each building material, and the production sequence is determined based on the building material characteristic data and the inventory data. Based on a preset experience database, the arrangement order of building materials in the basic production scheduling scheme is adjusted to obtain an optimized production scheduling scheme; the experience database includes the basic production scheduling scheme of historical scheduling processes; the optimized production scheduling scheme is a new production order of building materials obtained by scheduling the basic production scheduling scheme according to the basic production scheduling scheme of historical scheduling processes. The step of adjusting the arrangement order of building materials in the basic production scheduling scheme according to a preset experience database to obtain an optimized production scheduling scheme includes: Historical basic production scheduling schemes are extracted from the experience database to form a set of historical basic production scheduling schemes. A deep learning-based autoencoder is used to extract deep features from the historical basic production scheduling scheme set to obtain a historical scheme deep feature dataset. A graph convolutional network is used to cluster the historical scheme deep feature dataset to obtain a positive sample label set; the historical basic production scheduling schemes corresponding to the labels are selected to form a positive sample historical scheme set; The Apriori algorithm is used to perform feature frequency statistics and rule generation on the historical scheme set of positive samples to obtain association rule data; For each positive historical scheme in the set of positive historical schemes, the similarity to the basic production scheduling scheme is calculated using the following formula; the positive historical scheme corresponding to the highest similarity value is determined as the reference positive scheme: ； in, It is the first Building materials and the first The similarity of historical schemes among positive samples. It is the first Building material characteristic matrix of Xiangjiancai It is the first The first positive sample historical scheme Building material characteristic matrix of Xiangjiancai It is the total number of building material properties. It is the first The urgency weight of the building material characteristics It is the first The first item in the building material characteristic matrix of building materials The value of the building material properties, It is the first The first positive sample historical scheme The first item in the building material characteristic matrix of building materials The value of the building material properties; An Actor-Critic network employing deep reinforcement learning is used to adjust the basic production scheduling scheme based on the reference positive sample scheme and the association rule data, thereby obtaining the optimized production scheduling scheme.

2. The method according to claim 1, characterized in that, The step of arranging the building materials according to their production urgency based on the building material characteristic data and preset inventory data to obtain a basic production scheduling plan includes: Feature extraction is performed on the building material characteristic data to obtain a building material characteristic matrix; Based on the preset urgency score of the engineering project, the building material characteristic matrix is ​​prioritized to obtain a dynamic priority ranking list; Based on the dynamic priority sorting list and the preset inventory data, the preset building materials are arranged to obtain a basic production scheduling scheme.

3. The method according to claim 2, characterized in that, The step of extracting features from the building material characteristic data to obtain a building material characteristic matrix includes: The independent component analysis algorithm is used to extract the latent features of the building material property data to obtain the independent component matrix; The EM algorithm is used to complete the data in the independent component matrix to obtain the complete independent component matrix; The building material property matrix is ​​obtained by Z-score normalization of the complete independent component matrix.

4. The method according to claim 2, characterized in that, The step involves prioritizing the building material characteristic matrix based on a preset project urgency score to obtain a dynamic priority ranking list, including: For each building material characteristic in the building material characteristic matrix, and in conjunction with the project urgency score, the entropy weight method is used to calculate the corresponding urgency weight using the following formula: ； ； in, It is the first The urgency weight of building materials It is the first The first building materials The normalized value of the building material properties, It is the first The first building materials Building material characteristics, It is a smoothing factor. It is the amplitude coefficient. It is an urgency rating. It is the scoring threshold. This represents the total number of building material properties; Based on the urgency weights, the building materials corresponding to the urgency weights are sorted in order of numerical value to obtain the dynamic priority sorting list.

5. The method according to claim 2, characterized in that, The step of arranging the preset building materials according to the dynamic priority sorting list and the preset inventory data to obtain a basic production scheduling scheme includes: Based on the dynamic priority sorting list and the preset inventory data, the list fitness corresponding to the dynamic priority sorting list is calculated using the following formula: ； in, It is the first The list fitness corresponding to a dynamically prioritized list It is the first The urgency weight of building materials It is the delay penalty coefficient. It is the first The position of the building materials in the dynamic priority sorting list. It is the penalty coefficient. It is a positional penalty factor. It is standard inventory data. It is inventory quantity data; The order of the building materials in the dynamic priority sorting list is randomly adjusted to obtain a new dynamic priority sorting list; Calculate the fitness of each of the dynamic priority sorting lists, and determine the dynamic priority sorting list with the highest fitness as the basic production scheduling scheme.

6. The method according to claim 1, characterized in that, The method also includes: Obtain the actual production efficiency data and inventory change data of the building materials; Based on the optimized production scheduling scheme, the actual production efficiency data, and the inventory change data, the corresponding efficiency deviation and inventory change deviation are calculated. Based on the efficiency deviation and the inventory change deviation, the effectiveness probability of the optimized production scheduling scheme is calculated using a Bayesian probability model. When the validity probability reaches a preset threshold, the optimized production scheduling scheme is added to the experience database.

7. A green building materials intelligent scheduling system based on reinforcement learning, characterized in that, The system includes: The data acquisition module is used to acquire building material characteristic data; the building material characteristic data includes preset production characteristics, energy consumption characteristics and order information of building materials; The preliminary scheduling module is used to arrange the building materials according to their production urgency based on the building material characteristic data and preset inventory data to obtain a basic production scheduling plan. The basic production scheduling plan is used to characterize the production sequence of each building material, and the production sequence is determined based on the building material characteristic data and the inventory data. The final scheduling module is used to adjust the arrangement order of building materials in the basic production scheduling scheme according to a preset experience database to obtain an optimized production scheduling scheme; the experience database includes the basic production scheduling scheme of historical scheduling processes; the optimized production scheduling scheme is a new production order of building materials obtained by scheduling the basic production scheduling scheme according to the basic production scheduling scheme of the historical scheduling process. The final scheduling module is also used for: Historical basic production scheduling schemes are extracted from the experience database to form a set of historical basic production scheduling schemes. A deep learning-based autoencoder is used to extract deep features from the historical basic production scheduling scheme set to obtain a historical scheme deep feature dataset. A graph convolutional network is used to cluster the historical scheme deep feature dataset to obtain a positive sample label set; the historical basic production scheduling schemes corresponding to the labels are selected to form a positive sample historical scheme set; The Apriori algorithm is used to perform feature frequency statistics and rule generation on the historical scheme set of positive samples to obtain association rule data; For each positive historical scheme in the set of positive historical schemes, the similarity to the basic production scheduling scheme is calculated using the following formula; the positive historical scheme corresponding to the highest similarity value is determined as the reference positive scheme: ； in, It is the first Building materials and the first The similarity of historical schemes among positive samples. It is the first Building material characteristic matrix of Xiangjiancai It is the first The first positive sample historical scheme Building material characteristic matrix of Xiangjiancai It is the total number of building material properties. It is the first The urgency weight of the building material characteristics It is the first The first item in the building material characteristic matrix of building materials The value of the building material properties, It is the first The first positive sample historical scheme The first item in the building material characteristic matrix of building materials The value of the building material properties; An Actor-Critic network employing deep reinforcement learning is used to adjust the basic production scheduling scheme based on the reference positive sample scheme and the association rule data, thereby obtaining the optimized production scheduling scheme.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images