A cloud manufacturing service combination optimization method considering occupancy

By considering the occupation and waiting mechanism of cloud manufacturing services, screening effective services and building a comprehensive quality evaluation function, and using deep reinforcement learning to optimize the cloud manufacturing service combination, the problem of limited service selection scope in the existing methods is solved and the service quality is improved .

CN118261387BActive Publication Date: 2024-11-08NANJING UNIV OF AERONAUTICS & ASTRONAUTICS +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410434919.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-11
Publication Date
2024-11-08
Estimated Expiration
2044-04-11

AI Technical Summary

Technical Problem

When selecting cloud manufacturing services, the existing cloud manufacturing service combination method only considers the available status of the service at the task scheduling point, resulting in ignoring the services that are about to complete their original manufacturing tasks, narrowing the scope of service selection, and affecting the final service quality.

Method used

A preferred method for cloud manufacturing service combination considering the occupation situation is proposed. By determining the time interval in which the services in the candidate service center have been occupied by other tasks, effective services are selected, and a comprehensive quality evaluation function is constructed, combining manufacturing time, manufacturing cost and Manufacturing reliability, using deep reinforcement learning algorithms to optimize service combinations.

Benefits of technology

This increases the diversity of service choices, improves the possibility of selecting high-quality services, and achieves a comprehensive and optimal cloud manufacturing service combination.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118261387B_ABST
    Figure CN118261387B_ABST
Patent Text Reader

Abstract

The present application discloses a cloud manufacturing service combination optimization method considering occupancy, which relates to the field of cloud manufacturing technology. The method determines the manufacturing interval of any service belonging to the valid service in the candidate service set corresponding to each subtask, and the waiting time of the subtask for the service according to the execution order of all subtasks in the cloud manufacturing task, combined with the time interval of each service being occupied by other tasks, and then constructs a comprehensive quality evaluation function that is negatively correlated with the manufacturing time of each subtask and the manufacturing cost of each subtask, and screens out the cloud manufacturing service combination with the best comprehensive quality. The method not only considers the services that are idle at the scheduling time, but also considers those services that are currently occupied but are about to complete their original manufacturing tasks. By introducing a waiting mechanism, the diversity of service selection is increased so as to select high-quality resources in the cloud service pool, thereby realizing service optimization in a cloud manufacturing environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of cloud manufacturing technology, and in particular to a method for optimizing cloud manufacturing service combinations taking occupancy conditions into consideration. Background Art

[0002] Cloud manufacturing is a new service-oriented intelligent manufacturing model that combines cloud computing with industrial manufacturing processes and encapsulates the resources provided by enterprises into cloud manufacturing services through virtualization to meet the growing demand for personalized customization.

[0003] The general model of cloud manufacturing is that the cloud manufacturing platform accepts product orders submitted by customers, decomposes them into multiple manufacturing subtasks according to their functions, and then selects a batch of cloud manufacturing services that meet the requirements from the cloud manufacturing service library to generate a candidate service set, and selects a group of optimal cloud manufacturing service combinations from them according to customer requirements to complete the manufacturing tasks.

[0004] However, most current studies only consider the available status of services at the task scheduling point when selecting corresponding cloud manufacturing services for manufacturing subtasks, that is, they define the status of cloud manufacturing services as "occupied" and "idle", mark cloud manufacturing services with an "idle" status at the task scheduling point as valid services for selection, and mark cloud manufacturing services with an "occupied" status at the task scheduling point as invalid services and do not participate in the optimization of service composition. However, this approach will miss some cloud manufacturing services that are about to complete their original manufacturing tasks, resulting in a narrowing of the selection range of cloud manufacturing services and affecting the service quality of the final cloud manufacturing service composition. Summary of the invention

[0005] In response to the above problems and technical requirements, this application proposes a cloud manufacturing service combination optimization method considering occupancy conditions. The technical solution of this application is as follows:

[0006] A cloud manufacturing service combination optimization method considering occupancy conditions, the cloud manufacturing service combination optimization method comprising:

[0007] Determine any subtask in the cloud manufacturing task Task i Corresponding candidate service set Set i , and determine the candidate service set Set i Any service RES i,k Time interval that has been occupied by other tasks Occupy i,k , integer parameter 1≤i≤Num, Num is the total number of subtasks contained in the cloud manufacturing task Task, integer parameter 1≤k≤num i , num i Is the candidate service set Set i Used to execute subtasks ithe total number of services provided;

[0008] According to the execution order of all Num subtasks in the cloud manufacturing task Task, combined with the time intervals that each service has been occupied by other tasks, from the subtask st i Corresponding candidate service set Set i Select several services as subtasks i Valid services and determine any service RES that belongs to the valid service i,k For subtask i The manufacturing interval, and subtask st i About Service RES i,k Waiting time Tw i,k ;Service RES i,k For subtask i Manufacturing area and service RES i,k Time interval that has been occupied by other tasks Occupy i,k No conflict;

[0009] Construct a comprehensive quality evaluation function for cloud manufacturing tasks. The comprehensive quality evaluation function is negatively correlated with the manufacturing time and manufacturing cost of each subtask. The manufacturing time of each subtask is the sum of the production time and waiting time of the subtask, and the manufacturing cost of each subtask is the sum of the production cost and transportation and storage cost of the subtask.

[0010] Taking maximizing the comprehensive quality evaluation function as the optimization goal, a service is selected from the effective services corresponding to each subtask of the cloud manufacturing task to obtain the cloud manufacturing service combination with the best comprehensive quality.

[0011] Its further technical solution is to start from the subtask st i Corresponding candidate service set Set i Filter subtasks in i The valid services include the candidate service set Set i Any service in RES i,k :

[0012] When i=1, initialize service RES i,k Plan to start executing subtask st i The moment ts i,k ; When i ≥ 2, according to the execution order of all Num subtasks in the cloud manufacturing task Task, combined with the service RES i,k Time interval that has been occupied by other tasks Occupy i,k , determine the service RES i,k Plan to start executing subtask st i The moment ts i,k ;

[0013] At time ts i,k To time ts i,k +tm i,k RES i,k Time interval that has been occupied by other tasks Occupy i,k When there is no time conflict, determine the service RES i,k Is a valid service and determines the service RES i,k For subtask i The manufacturing interval includes time ts i,k To time ts i,k +tm i,k Time period, time ts i,k Is the service RES i,k For subtask i The execution start time, time te i,k =ts i,k +tm i,k Is the service RES i,k For subtask i The execution end time, tm i,k Is the service RES i,k Execute subtask st i The production time is consuming;

[0014] At time ts i,k To time ts i,k +tm i,k RES i,k Time interval that has been occupied by other tasks Occupy i,k When there is a time conflict, determine the service RES i,k Is an invalid service.

[0015] Its further technical solution is to determine the service RES i,k Plan to start executing subtask st i The moment ts i,k include:

[0016] Determine the subtask st according to the execution order of all Num subtasks in the cloud manufacturing task Task i The previous subtask of i-1 In service RES i-1,j The execution end time te under the execution i-1,j ;

[0017] According to the subtask i-1 Services RES i-1,j Location to ServiceRES i,k Transportation time to the location Determine the service RES i-1,k-1 Complete subtask st i-1 The obtained products are transported to the service RES i,k Arrival time at location

[0018] When the arrival time i,k Not in service RES i,k Time interval that has been occupied by other tasks Occupy i,k In time, the arrival time ta i,k RES as a Service i,k The plan starts based on the completion of subtasks i-1 The obtained product performs the subtask st i The moment ts i,k ;

[0019] When the arrival time i,k In service RES i,k Time interval that has been occupied by other tasks Occupy i,k Internal time, with time ta i,k Time interval Occupy i,k End time near RES as a Service i,k The plan starts based on the completion of subtasks i-1 The obtained product performs the subtask st i The moment ts i,k .

[0020] Its further technical solution is to determine the subtask st i About Service RES i,k Waiting time ts i,k -ta i,k It is the previous subtask st i-1 The obtained products are transported to the service RES i,k Waiting for service after location RES i,k The time taken to complete other tasks.

[0021] Its further technical solution is to construct a comprehensive quality evaluation function for cloud manufacturing tasks:

[0022]

[0023] Among them, Q i It is a subtask i The overall quality and Q i Based on the subtask i Manufacturing time under the corresponding service T i and manufacturing cost Ci OK, manufacturing time is T i The shorter the comprehensive quality Q i The larger the manufacturing cost C i The smaller the comprehensive quality Q i The bigger;

[0024] Subtask i In any service RES i,k The manufacturing time of the following is:

[0025] T i =tm i,k +Tw i,k

[0026] Subtask i In any service RES i,k The manufacturing cost is:

[0027]

[0028] in, Is the service RES i,k Execute subtask st i The production cost of the process, It is a subtask i The previous subtask of i-1 The obtained products are transported to the service RES i,k The transportation cost and transportation time of the location Positive correlation, It is the previous subtask st i-1 The obtained products are transported to the service RES i,k Location to service RES i,k Complete subtask st i The storage cost during the process is related to ts i,k -ta i,k Positive correlation.

[0029] Its further technical solution is that the subtask st i The overall quality Q i Also based on the subtask st i Manufacturing reliability of selected services R i OK, subtask st i Manufacturing reliability of selected services R i The higher the comprehensive quality Q i The bigger.

[0030] Its further technical solution is that the subtask st i The comprehensive quality is:

[0031]

[0032] in, are weights and It is a subtask i The manufacturing time is T i The normalized index of manufacturing time T is calculated by the normalized method of negative index. i The normalization process is obtained; It is a subtask i The manufacturing cost C i The normalized index of the manufacturing cost C is calculated by the normalization method of negative indicators. i The normalization process is obtained; It is a subtask i Manufacturing reliability of selected services R i The normalized index of the selected service is R through the normalization method of the positive index. i Normalized processing is obtained.

[0033] Its further technical solution is to obtain the cloud manufacturing service combination with the best comprehensive quality, including:

[0034] Construct the state vector State of the deep reinforcement learning algorithm 1 ,se 2 ,...,se Num ] and any state se i The corresponding action space Status i The scheduled status indicates that the subtask i The corresponding service has been arranged, or the status is i The unscheduled state indicates that the subtask i Service has not been arranged yet, action a i,k Corresponding subtask st i Candidate service set Set i Services in RES i,k ;

[0035] The comprehensive quality evaluation function of the cloud manufacturing task in each cloud manufacturing service combination is taken as the reward of reinforcement learning. The deep Q-learning algorithm is used to iteratively solve the problem based on the state space and action space to obtain the comprehensive quality evaluation function of each cloud manufacturing service combination.

[0036] Its further technical solution is to use the deep Q learning algorithm to iteratively solve based on the state vector and action space, including:

[0037] Initialize the number of iterations to 1 and initialize the network parameters of the neural network;

[0038] At the beginning of any iteration cycle, reset the state vector State to all states as unscheduled states;

[0039] Initialize parameter i = 1, and select action from state se according to the ε-greedy action selection strategy i Corresponding action space Action i Select an action a i,k ;

[0040] When action a i,k Corresponding service RES i,k Not status i The corresponding subtask i When the effective service is reached, re-execute the ε-greedy action selection strategy from state se i Corresponding action space Action i Select an action a i,k Steps;

[0041] When action a i,k Corresponding service RES i,k is the status i The corresponding subtask i When the service is valid, the status i Update to scheduled status and confirm status i The corresponding subtask i The effective service and its subtasks i manufacturing interval; let i = i + 1 and execute again according to the ε-greedy action selection strategy from state se i Corresponding action space Action i Select an action a i,k Steps;

[0042] When all states in the state vector State are scheduled, the current iteration cycle is determined to be completed. i The corresponding subtask i The overall quality Q i After updating the network parameters of the neural network, the next iteration cycle is entered until the maximum iteration cycle is reached, and the effective services of the subtasks corresponding to all states are output to obtain the cloud manufacturing service combination with the best overall quality.

[0043] Its further technical solution is to calculate the state of each state in the current iteration cycle. i The corresponding subtask i The overall quality Q i Updating the network parameters of the neural network includes:

[0044] When the number of initialization iterations is 1, the network parameters of the initialization neural network include the network parameters θ of the initialization Q network and the network parameters θ of the target network. - same;

[0045] After completing each iteration cycle, according to the cost loss function L(θ)=E[y i -Q(se i , a i,k′ ;θ)] 2 Update the network parameters θ of the Q network according to the gradient descent method, where Q(se i , a i,k′ ; θ) is the state-action pair (se) of the Q network with network parameters θ i , a i,k′ ), the value of a i,k′ is the status i The corresponding subtask i The effective service with network parameters θ - The output of the target network Q i is the status i The corresponding subtask i The overall quality Q i , γ is the discount factor, Q(se i+1 , a i+1,k θ - ) is a network with parameters θ - The state-action pair (se i+1 , a i+1,k )’s value;

[0046] After every predetermined number of iterations, the network parameters θ of the Q network are passed to the target network to update the network parameters θ of the target network. - Until the maximum iteration period is reached.

[0047] The beneficial technical effects of this application are:

[0048] This application discloses a cloud manufacturing service combination optimization method that considers the occupancy situation. The method introduces a waiting mechanism that considers the current occupancy situation of the service. When constructing a cloud manufacturing service combination, not only the services that are idle at the time of scheduling are considered, but also those services that are currently occupied but are about to complete their original manufacturing tasks are considered, increasing the diversity of service selection so as to select high-quality resources in the cloud service pool. On the basis of ensuring the matching of tasks and services and avoiding conflicts with the occupancy time of the original services, the number of optional services is increased, and the possibility of selecting high-quality services is improved, so that the cloud manufacturing service combination with the best comprehensive quality can be obtained, realizing the service optimization in the cloud manufacturing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a method flow chart of a cloud manufacturing service combination optimization method in one embodiment of the present application.

[0050] Figure 2 This is a flow chart of a method for determining the waiting time of a subtask for a selected service in one embodiment of the present application.

[0051] Figure 3 A service in an instance RES i,k The time interval that has been occupied by other tasks, as well as the time interval for executing the subtask st i Timing diagram of the time period.

[0052] Figure 4 is a service in another instance i,k The time interval that has been occupied by other tasks, as well as the time interval for executing the subtask st i Timing diagram of the time period.

[0053] Figure 5 The figure is a comparison chart of the cloud manufacturing service combination optimization method of the present application that introduces a waiting mechanism and the cloud manufacturing service combination obtained by the traditional method in an experimental example. DETAILED DESCRIPTION

[0054] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0055] This application discloses a cloud manufacturing service combination optimization method considering occupancy conditions, please refer to Figure 1 The cloud manufacturing service combination optimization method comprises the following steps:

[0056] Step 1: Determine any subtask in the cloud manufacturing task Task i Corresponding candidate service set Set i , and determine the candidate service set Set i Any service RES i,k Time interval that has been occupied by other tasks Occupy i,k .

[0057] Among them, the integer parameter 1≤i≤Num, Num is the total number of subtasks contained in the cloud manufacturing task Task, and the cloud manufacturing task containing Num subtasks Task={st 1 , st 2 …st i …st Num}.

[0058] For any subtask in the cloud manufacturing task Task i The cloud platform satisfies the subtask sti Cluster the required services and get the subtask st i Corresponding candidate service set Set i The specific method of clustering to obtain the candidate service set for each subtask can refer to the relevant content in the existing cloud manufacturing technology, which will not be repeated in this embodiment.

[0059] Integer parameter 1≤k≤num i , num i Is the candidate service set Set i Used to execute subtasks i The total number of services, including num i The candidate service set of a service is expressed as The total number of services included in the candidate service sets of any two subtasks may be the same or different, and the same service may be included in the candidate service sets of one or more subtasks.

[0060] Any subtask i Candidate service set Set i It contains all services that have the ability to execute the subtask. Each service can be used to sequentially execute multiple subtasks in the same cloud manufacturing task or subtasks in multiple cloud manufacturing tasks. Different subtasks occupy services in the order of allocation. After a service completes a subtask, it can execute other subtasks in its idle time. Therefore, subtasks are i Candidate service set Set i Any service RES i,k It may have been allocated to execute subtasks in other cloud manufacturing tasks, or allocated to execute other subtasks in the same cloud manufacturing task. Therefore, it is also necessary to determine the time interval in which each service has been allocated to execute other subtasks, that is, the time interval of the service that has been occupied by other tasks.

[0061] Assume that the service RES i,k Already Other subtasks are occupied, of which service RES is already occupied i,k Any subtask z of service RES i,k The occupation period is [start z , end z ], also known as service RES i,k In [start z , end z ]Execute the subtask z within the time period, start z Is the service RES i,k The time allocated to start executing subtask z, end z Is the service RES i,kThe time when subtask z is assigned to complete execution. Integer parameter is an integer parameter. The corresponding time period [start z , end z ] do not conflict with each other, that is, the time periods do not overlap, different time periods are continuous or intermittent, then the service RES i,k The time interval that has been occupied by other tasks can be expressed as

[0062] For example, in one instance, a service has been assigned to perform three subtasks, and the time interval that the service has been occupied by other subtasks is expressed as {[12, 52], [150, 172], [260, 300]}, indicating that the service has been assigned to perform a subtask within [12, 52] hours from the system initialization moment, another subtask within [150, 172] hours from the system initialization moment, and another subtask within [260, 300] hours from the system initialization moment.

[0063] Step 2: According to the execution order of all Num subtasks in the cloud manufacturing task Task, combined with the time intervals that each service has been occupied by other tasks, start from the subtask st i Corresponding candidate service set Set i Select several services as subtasks i effective service.

[0064] As above, each subtask st obtained in step 1 i Corresponding candidate service set Set i Contains all tasks that have execution subtasks st i However, not all of these services can be allocated to perform subtasks. i When a service is already occupied by other subtasks and there is no suitable free time, the service cannot be allocated to execute the subtask. i , so we need to further select the candidate service set Set i Filter out the ones with suitable free time to execute subtasks i The service as a subtask i effective service.

[0065] Based on this consideration, in order to screen out effective services, the most basic consideration is the time intervals in which each service has been occupied by other tasks. In addition, the coupling relationship between different subtasks also needs to be considered: This application considers a cloud manufacturing task Task in the form of a sequence, that is, there are strict execution order constraints between different subtasks of the cloud manufacturing task Task. Only after the previous subtask in the execution order is completed can the next subtask in the execution order be executed. This is mainly because in actual use, the next subtask often needs to use the product manufactured by the previous subtask, resulting in a coupling relationship between different subtasks. In this application, the subscript parameter i is used to indicate the execution order between different subtasks. Among all Num subtasks of the cloud manufacturing task Task, starting from i≥2, the subtask st i-1 It is a subtask i According to the previous subtask in the execution order, subtask st i Depends on subtask st i-1 The obtained product.

[0066] Therefore, this step comprehensively considers the execution order of all Num subtasks in the cloud manufacturing task Task, as well as the time intervals during which each service has been occupied by other tasks, and selects the subtask st i After filtering out the valid services, you also need to determine any service RES that belongs to the valid services. i,k For subtask i Determine the service RES obtained i,k For subtask i Manufacturing area and service RES i,k Time interval that has been occupied by other tasks Occupy i,k In addition, unlike conventional practice, this application does not directly consider services that are idle at the time of scheduling as valid services, but also considers those services that are currently occupied but are about to complete the manufacturing task, that is, the accepting subtask can appropriately wait for the service to be idle, so when screening out valid services RES i,k After that, the subtask st i For services that are valid services RES i,k Waiting time Tw i,k .

[0067] In one embodiment, from the candidate service set Set i The method for selecting effective services includes determining the candidate service set Set according to the following method i Any service in RES i,k Is it a valid service or an invalid service?

[0068] (1) Determine the service RES i,k Plan to start executing subtask st i The moment ts i,k .

[0069] When i=1, that is, for the first subtask in the execution order of the cloud manufacturing task Task, the service RES i,k Plan to start executing subtask st 1 The moment is the initialization moment.

[0070] When i ≥ 2, service RES i,k Plan to start executing subtask st 1 The time of the previous subtask needs to be considered. According to the execution order of all Num subtasks in the cloud manufacturing task Task, combined with the service RES i,k Time interval that has been occupied by other tasks Occupy i,k , determine the service RES i,k Plan to start executing subtask st i The moment ts i,k , including the following steps, please refer to Figure 2 The flowchart for the case i ≥ 2 is shown:

[0071] (a) Determine the subtask st according to the execution order of all Num subtasks in the cloud manufacturing task Task i The previous subtask of i-1 In service RES i-1,j The execution end time te under the execution i-1,j , integer parameter 1≤j≤num i-1 , num i-1 It is a subtask i-1 Corresponding candidate service set Set i-1 Used to execute subtasks i-1 The total number of services.

[0072] As above, when screening the effective services for each subtask, the manufacturing time of the service for the subtask will also be determined, so the service RES can be determined i-1,j For subtask i-1 The execution start time and execution end time.

[0073] (b) Theoretically, at the end of execution time te i-1,j After that, you can start executing subtask st i However, different services are usually located in different areas, so the subtasks are i-1 The products are transported to subtask st i There is also time-consuming transportation, so first determine the subtask sti-1 Services RES i-1,j Location to ServiceRES i,k Transportation time to the location The transportation time between different service locations is related to the distance between the two locations and can be determined by means of third-party maps.

[0074] This can determine the service RES i-1,j Complete subtask st i-1 The obtained products are transported to the service RES i,k Arrival time at location

[0075] (c) Testing Service RES i,k At the time of arrival i,k Whether it is in idle state.

[0076] When the arrival time i,k Not in service RES i,k Time interval that has been occupied by other tasks Occupy i,k Internal time, that is, service RES i,k At the time of arrival i,k When no other subtasks are being executed, it indicates that the service RES i,k At the time of arrival i,k In idle state, the arrival time ta i,k RES as a Service i,k The plan starts based on the completion of subtasks i-1 The obtained product performs the subtask st i The moment ts i,k .

[0077] When the arrival time i,k In service RES i,k Time interval that has been occupied by other tasks Occupy i,k When the service is RES i,k At the time of arrival i,k In the occupied state, the plan is to serve RES i,k After completing the current subtask, start executing subtask st i , so that at time ta i,k Time interval Occupy i,k The most recent end time end near RES as a Service i,k The plan starts based on the completion of subtasks i-1 The obtained product performs the subtask st i The moment ts i,k .

[0078] (2) In either case, when determining the service RES i,k The plan starts based on the completion of subtasks i-1 The obtained product performs the subtask st i The moment ts i,k After that, due to the service RES i,k Execute subtask st i Production time tm i,k is fixed and known, so the service RES can be determined i,k Plan execution subtask i The time period is time ts i,k To time ts i,k +tm i,k Then it is necessary to determine whether the time period can be used to execute the subtask st i :

[0079] At time ts i,k To time ts i,k +tm i,k RES i,k Time interval that has been occupied by other tasks Occupy i,k When there is a time conflict, it means that the service RES i,k There is no suitable free time to allow the execution of subtasks i , determine the service RES i,k Is an invalid service.

[0080] At time ts i,k To time ts i,k +tm i,k RES i,k Time interval that has been occupied by other tasks Occupy i,k When there is no time conflict, that is, service RES i,k If the service is completely idle during this period, it means that the service RES i,k There is suitable free time to allow the execution of subtasks i , then determine the service RES i,k It is a subtask i The effective service and can determine the service RES accordingly i,k For subtask i The manufacturing interval is [ts i,k ,te i,k ], time ts i,k Is the service RES i,k For subtask i The execution start time, time te i,k =ts i,k+tm i,k Is the service RES i,k For subtask i The execution end time of .

[0081] When the service RES i,k It is a subtask i When the effective service is as above, theoretically the service RES i,k Should be in the previous subtask st i-1 The execution end time te i-1,j Start executing subtask st i , but actually at time ts i,k Just start executing subtask st i , so the subtask st i About Service RES i,k The waiting time is Tw i,k =ts i,k -te i-1,j The waiting time here is Tw i,k Include the previous subtask st i-1 The obtained products are transported to the service RES i,k Transportation time to the location When serving RRS i,k At the time of arrival i,k When in the occupied state, you still need to wait for the service RES i,k After the current task is completed, the waiting time is Tw i,k Also includes the previous subtask st i-1 The obtained products are transported to the service RES i,k Waiting for service after location RES i,k Time consuming to complete other tasks ts i,k -ta i,k .

[0082] Therefore, the overall can be expressed as subtask st i About Service RES i,k Waiting time When the service RES i,k At the time of arrival i,k When no other subtasks are executed i,k =ta i,k , so there is When the service RES i,k At the time of arrival i,k When other subtasks are being executed i,k ≠ta i,k And ts i,k =end near , so there is

[0083] Through the above steps, the candidate service set Set i By judging each service in the task, we can determine whether each service is a subtask st i The effective service of i All valid services of , and determine each valid service for subtask st i After the effective service of a subtask is screened, the time interval of the effective service of the subtask that has been occupied by other tasks is updated, and the effective service of the next subtask is screened according to the above method.

[0084] For example, in one instance, the service RES i,k The time interval that has been occupied by the other three subtasks Occupy i,k ={[start 1 , end 1 ],[start 2 , end 2 ],[start 3 , end 3 ]} and on the time axis Figure 3 and Figure 4 Indicates. Figure 3 In the example of i-1 The obtained products are transported to the service RES i,k Arrival time at the location end 1 < i,k <start 2 , then the arrival time ta i,k In service RES i,k The idle time of executing the two subtasks is directly transferred to the arrival time ta i,k RES as a Service i,k Plan to start executing subtask st i The moment ts i,k , and then the service RES can be determined i,k Plan execution subtask i The time period is time ts i,k To time ts i,k +tm i,k On the timeline Figure 3 It can be seen that the plan executes subtask st i The time period and [start 2 , end 2 ] time period conflicts, so the service RES i,k There is not enough free time to execute the subtask i , then the service RESi,k Cannot be used as a subtask i effective tasks.

[0085] exist Figure 4 In the example of i-1 The obtained products are transported to the service RES i,k Arrival time at location start 2 < i,k <end 2 , then the arrival time ta i,k In service RES i,k The time period for executing other subtasks [start 2 , end 2 ], then it will reach time ta i,k The time interval [start 2 , end 2 ]'s most recent end time end near =end 2 RES as a Service i,k The plan starts based on the completion of subtasks i-1 The obtained product performs the subtask st i The moment ts i,k , and then the service RES can be determined i,k Plan execution subtask i The time period is time ts i,k To time ts i,k +tm i,k On the timeline Figure 4 Indicates. Figure 4 In the example, start 3 -end 2 >tm i,k , time ts i,k To time ts i,k +tm i,k The time period will not coincide with [start 3 , end 3 ] If there is a conflict, the service RES i,k In the occupied time period [start 2 , end 2 ] and [start 3 , end 3 ] There is enough time to execute the subtask st i , Service RES i,k Can be used as a subtask i effective tasks.

[0086] In another case, when start3 -end 2 <tm i,k , time ts i,k To time ts i,k +tm i,k The time period will be the same as [start 3 , end 3 ] If there is a conflict, the service RES i,k In the occupied time period [start 2 , end 2 ] and [start 3 , end 3 ] There is not enough time to perform the subtask i , Service RES i,k Cannot be used as a subtask i effective tasks.

[0087] Step 3: Construct a comprehensive quality evaluation function for cloud manufacturing tasks.

[0088] The constructed comprehensive quality evaluation function is negatively correlated with the manufacturing time and manufacturing cost of each subtask, that is, the longer the manufacturing time of each subtask, the worse the comprehensive quality reflected by the comprehensive quality evaluation function, and the greater the manufacturing cost of each subtask, the worse the comprehensive quality reflected by the comprehensive quality evaluation function.

[0089] The comprehensive quality evaluation function of the constructed cloud manufacturing task is:

[0090]

[0091] Among them, Q i It is a subtask i The overall quality and Q i Based on the subtask i Manufacturing time under the corresponding service T i and manufacturing cost C i OK. Subtask i The manufacturing time is T i The shorter the comprehensive quality Q i The larger the subtask i The manufacturing cost C i The smaller the comprehensive quality Q i The bigger.

[0092] The manufacturing time of each subtask is the sum of the production time and waiting time of the subtask. i In any service RES i,k The manufacturing time of the following is:

[0093] Ti =tm i,k +Tw i,k

[0094] The manufacturing cost of each subtask is the sum of the production cost and the transportation and storage cost of the subtask. i In any service RES i,k The manufacturing cost is:

[0095]

[0096] in, Is the service RES i,k Execute subtask st i The production cost of the process is a fixed known quantity.

[0097] It is a subtask i The previous subtask of i-1 The obtained products are transported to the service RES i,k Transportation costs at the location, transportation costs Time-consuming transportation Positive correlation can be expressed as block i It is a subtask i The number of modules that need to be manufactured, utc is the subtask st i The transportation cost per unit time of a unit module is a known quantity.

[0098] It is the previous subtask st i-1 The obtained products are transported to the service RES i,k Location to service RES i,k Complete subtask st i The storage cost during the process and the time taken for this process ts i,k -ta i,k +tm i,k Positive correlation can be expressed as sto is the storage cost per unit time and is a known quantity. Since tm i,k is also a fixed known quantity, so the storage cost Previous subtask i-1 The obtained products are transported to the service RES i,k Wait until the service RES i,k The duration ts to complete the subtask being executed i,k -ta i,k Positive correlation.

[0099] In another embodiment, each service also has its own manufacturing reliability, then the subtask st i The overall quality Q i Also based on the subtask st i Manufacturing reliability of selected services R i OK, subtask st i Manufacturing reliability of selected services R i The higher the comprehensive quality Q i The larger the manufacturing reliability R of each service i is a predetermined known quantity.

[0100] Therefore, the subtask i The comprehensive quality is:

[0101]

[0102] in, are weights and The three weights can be customized based on actual needs and experience.

[0103] It is a subtask i The manufacturing time is T i The normalized index of manufacturing time T is calculated by the normalized method of negative index. i Normalized processing is obtained. It is a subtask i The manufacturing cost C i The normalized index of the manufacturing cost C is calculated by the normalization method of negative indicators. i Normalized processing is obtained. It is a subtask i Manufacturing reliability of selected services R i The normalized index of the selected service is R through the normalization method of the positive index. i The formula for normalizing the positive index and the negative index is a general formula, and this embodiment will not be repeated.

[0104] Step 4: Taking maximizing the comprehensive quality evaluation function as the optimization goal, a service for executing each subtask of the cloud manufacturing task Task is selected from the valid services corresponding to the subtask, and the cloud manufacturing service combination with the best comprehensive quality is obtained by integrating the services of all subtasks.

[0105] In order to facilitate the optimization solution, deep reinforcement learning is introduced in one embodiment. Deep reinforcement learning is a machine learning algorithm that implements an optimization method that balances exploration and use of existing knowledge. It takes actions based on the current situation and improves its decisions through repeated trials and errors to maximize the expected benefits. Deep Q learning is one of the classic reinforcement learning methods. It uses a deep learning network to fit the Q table and is suitable for processing high-dimensional data. This embodiment uses deep Q learning to solve the cloud manufacturing service combination with the best comprehensive quality, including the following steps:

[0106] First, construct the state vector State = [se 1 ,se 2 ,...,se Num ]. Among them, the state se i Corresponding to the subtask st in the cloud manufacturing task Task i , status i The scheduled state indicates that the subtask i The corresponding service has been arranged, or the status is i The unscheduled state indicates that the subtask st i The service has not been arranged yet. You can set it in actual application. i =1 means the status is arranged, se i =0 means unscheduled status.

[0107] In addition, construct any state se i The corresponding action space Action a i,k Corresponding subtask st i Candidate service set Set i Services in RES i,k Since the number of services in the candidate service set of each subtask is different, the dimension of the action space corresponding to different states is equal to the number of services contained in the maximum candidate service set.

[0108] Then, the comprehensive quality evaluation function of the cloud manufacturing task under each cloud manufacturing service combination is used as the reward for reinforcement learning, and the deep Q learning algorithm is used to iteratively solve based on the state space and action space to obtain the comprehensive quality evaluation function under each cloud manufacturing service combination. The iterative solution includes the following process:

[0109] (1) Initialize the number of iterations to 1 and initialize the network parameters of the neural network. This includes initializing the network parameters of the Q network θ and the network parameters of the target network θ - same.

[0110] (2) At the beginning of any iteration cycle, the state vector State is reset to all states as unscheduled states, that is, initialization State = [0, 0, ..., 0].

[0111] (3) Initialize parameter i = 1 and select the action from state se according to the ε-greedy action selection strategy i Corresponding action space Action i Select an action a i,k .

[0112] (4) When action a i,k Corresponding service RES i,k Not status i The corresponding subtask i When the service is valid, keep the current state i No change, re-execute the ε-greedy action selection strategy from state se i Corresponding action space Action i Select an action a i,k This step determines the service RES i,k Is it a subtask? i The effective service method is as described in the specific content of step 2 above, which will not be repeated here.

[0113] (5) When action a i,k Corresponding service RES i,k is the status i The corresponding subtask i When the service is valid, the status i Update to scheduled status and confirm status i The corresponding subtask i The effective service and its subtasks i manufacturing area.

[0114] Set i = i + 1 to jump to the next state, and then execute the ε-greedy action selection strategy again for the next state from state se i Corresponding action space Action i Select an action a i,k The steps are repeated in this way.

[0115] (6) When all states in the state vector State are scheduled, the current iteration cycle is determined to be completed. i The corresponding subtask i The overall quality Q iAfter updating the network parameters of the neural network, the next iteration cycle is entered until the maximum iteration cycle is reached, and the effective services of the subtasks corresponding to all states are output to obtain the cloud manufacturing service combination with the best overall quality.

[0116] Among them, after completing each iteration cycle, according to the cost loss function L(θ)=E[y i -Q(se i , a i,k ,;θ)] 2 Update the network parameters θ of the Q network according to the gradient descent method. In the above formula, Q(se i , a i,k′ ; θ) is the state-action pair (se) of the Q network with network parameters θ i , a i,k′ ), the value of a i,k′ is the status i The corresponding subtask i With network parameters θ - The output of the target network Q i is the status i The corresponding subtask i The overall quality Q i , γ is the discount factor, Q(se i+1 , a i+1,k θ - ) is a network with parameters θ - The state-action pair (se i+1 , a i+1,k )’s value.

[0117] Then, after every predetermined number of iterations, the network parameters θ of the Q network are passed to the target network to update the network parameters θ of the target network. - Until the maximum iteration period is reached, until the maximum iteration period is reached.

[0118] In one implementation example, cloud manufacturing task Task = {st 1 , st 2 , st 3 , st 4 , st 5 , st 6 , st 7}, the candidate service set corresponding to each subtask includes 5 services, and any subtask st i The corresponding candidate service set is written as Set i ={RES i,1 , RES i,2 , RES i,3 , RES i,4 , RESi,5}, any subtask st i The corresponding service RES in the candidate service set i,k The attribute information is written as RES i,k ={ID i,k , T i , C i , R i}. Among them, ID i,k Is the service RES i,k Each service identifier is used to uniquely identify a service. i Is the service RES i,k For subtask i The manufacturing time of C i Is the service RES i,k For subtask i The manufacturing cost, R i Is the service RES i,k In this example, the attribute information of each service in the candidate service set corresponding to each subtask is shown in the following table. The units of each attribute in the following table are unified according to actual needs, and the unit of manufacturing time is hours, and the unit of manufacturing cost is yuan:

[0119] <![CDATA[RES i,1 ]]> <![CDATA[RES i,2 ]]> <![CDATA[RES i,3 ]]> <![CDATA[RES i,4 ]]> <![CDATA[RES i,5 ]]> <![CDATA[st 1 ]]> {1,36,242,0.98} {2,42,256,0.93} {3,44,272,0.97} {4,46,260,0.91} {5,40,274,0.93} <![CDATA[st 2 ]]> {6,50,262,0.92} {7,45,268,0.97} {8,52,274,0.98} {9,38,254,0.97} {10,54,280,0.98} <![CDATA[st 3 ]]> {1,58,338,0.94} {2,56,334,0.98} {3,59,336,0.93} {4,50,324,0.96} {5,44,304,0.98} <![CDATA[st 4 ]]> {11,45,312,0.92} {12,47,340,0.95} {13,42,306,0.96} {14,49,320,0.94} {15,54,330,0.95} <![CDATA[st 5 ]]> {6,47,252,0.94} {7,54,276,0.94} {8,43,266,0.97} {9,42,242,0.97} {10,44,268,0.93} <![CDATA[st 6 ]]> {11,88,450,0.94} {12,78,428,0.99} {13,94,458,0.98} {14,72,396,0.94} {15,84,412,0.96} <![CDATA[st 7 ]]> {16,45,248,0.96} {17,54,278,0.95} {18,55,266,0.95} {18,55,266,0.95} {20,54,276,0.96}

[0120] The time intervals and service areas of all 20 different services in the candidate service set corresponding to all the 7 subtasks above that have been occupied by other tasks are shown in the following table. The time interval of each service in the following table that has been occupied by other tasks is expressed in a pre-set expression and the units used are the same:

[0121] Service Logo Service Area Time intervals already occupied by other tasks 1 L2 [0,6]∪[72,96] 2 L3 [56,11]∪[234,278] 3 L4 [0,22]∪[67,98]∪[154,186] 4 L5 [0,32]∪[256,300] 5 L6 [0,30]∪[142,180]

[0122]

[0123] In the above table, the transportation time between different service areas is shown in the following table. The units used for each data in the following table are unified according to actual needs:

[0124] L1 L2 L3 L4 L5 L6 L1 0 6 7 4 5 9 L2 - 0 9 11 11 4 L3 - - 0 4 4 7 L4 - - - 0 4 5 L5 - - - - 0 5 L6 - - - - - 0

[0125] In this example, take the number of modules that need to be manufactured for each subtask, block i The unit time transportation cost of each module in each subtask is 2.6 yuan, and the unit time storage cost is 5 yuan.

[0126] Based on this example, the cloud manufacturing service combination optimization method of this application that introduces a waiting mechanism is compared with the traditional method. The traditional method does not consider those services that are about to be completed but only considers the services that are idle at the task scheduling point. Both methods use the DQN algorithm for optimization. DQNSW represents the cloud manufacturing service combination obtained by the DQN optimization algorithm that introduces a waiting mechanism, while DQNSNW represents the cloud manufacturing service combination obtained by the DQN algorithm that does not adopt the waiting mechanism in the traditional approach. The experimental results are as follows: Figure 5 As shown, Figure 5 The vertical axis represents the services with service IDs 1 to 20, and the horizontal axis represents time. The gray block Origin Time represents the time interval that each service has been occupied by other tasks, and Wait Time is the waiting time of the corresponding subtask for the selected service.

[0127] Comparing the cloud manufacturing service combinations finally obtained by the two methods, it can be seen that in the second subtask, the DQNSW of this application selected the service with a service ID of 9, while the traditional method DQNSNW selected the service with a service ID of 10. Compared with the traditional approach, this application saves time, which also affects the selection of subsequent services. In the end, the manufacturing time of the cloud manufacturing service combination obtained by the DQNSW of this application was 345 hours, the manufacturing cost was 62,545 yuan, and the manufacturing reliability was 0.967. The manufacturing time of the cloud manufacturing service combination obtained by the traditional method DQNSNW was 351 hours, the manufacturing cost was 63,799 yuan, and the manufacturing reliability was 0.967. It can be seen that the method of this application can obtain the cloud manufacturing service combination with the best comprehensive effect of manufacturing time, manufacturing cost, and manufacturing reliability.

[0128] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. A cloud manufacturing service portfolio selection method considering occupancy, It is characterized in that The cloud manufacturing service combination selection method includes: Determine cloud manufacturing tasks Any subtask Corresponding candidate service set , and determine the candidate service set Any service Time intervals already occupied by other tasks , integer parameter , It is a cloud manufacturing task The total number of subtasks included, integer parameter , is the candidate service set Used to execute subtasks the total number of services provided; Manufacturing tasks based on the cloud All The execution order of each subtask is based on the time interval that each service has been occupied by other tasks. Corresponding candidate service set Filter out several services as subtasks Valid services and identify any services that are valid services Subtask Manufacturing intervals and subtasks About Services Waiting time ;Serve Subtask Manufacturing area and services Time intervals already occupied by other tasks No conflict; Building cloud manufacturing tasks The comprehensive quality evaluation function is negatively correlated with the manufacturing time of each subtask and the manufacturing cost of each subtask, and the manufacturing time of each subtask is the sum of the production time and the waiting time of the subtask, and the manufacturing cost of each subtask is the sum of the production cost and the transportation and storage cost of the subtask; Taking maximizing the comprehensive quality evaluation function as the optimization goal, the cloud manufacturing task Select a service from the valid services corresponding to each subtask to obtain the cloud manufacturing service combination with the best comprehensive quality; From subtask Corresponding candidate service set Filter subtasks The valid services include the candidate service set Any service in : when Initialize the service Schedule to start executing subtasks Moment ;when Cloud manufacturing tasks All The execution order of subtasks, combined with the service Time intervals already occupied by other tasks , confirm the service Schedule to start executing subtasks Moment ; When the moment To time Time and service Time intervals already occupied by other tasks When there is no time conflict, confirm the service Is a valid service and confirms the service Subtask The manufacturing range includes time To time time period It is a service Subtask The execution start time, time It is a service Subtask The execution end time of It is a service Execute subtasks The production time is consuming; When the moment To time Time and service Time intervals already occupied by other tasks When there is a time conflict, determine the service It is an invalid service; Determine the service Schedule to start executing subtasks Moment include: According to the cloud manufacturing task All The execution order of subtasks determines the subtasks Previous subtask of In service The execution end time under the execution of ; According to subtask Services Location to Service Transportation time to the location , confirm the service Complete the subtask The obtained products are transported to the service Arrival time at location ; When arrival time Not in service Time intervals already occupied by other tasks Internal time, arrival time As a Service Planning starts based on completing subtasks The obtained product performs the subtask Moment ; When arrival time In service Time intervals already occupied by other tasks Internal time, time Time interval The end moment As a Service Planning starts based on completing subtasks The obtained product performs the subtask Moment ; Determine subtasks About Services Waiting time , It is the previous subtask The obtained products are transported to the service Waiting for service after location The time it takes to complete other tasks; Building cloud manufacturing tasks The comprehensive quality evaluation function is: in, It is a subtask The overall quality and Based on subtask Manufacturing time under the corresponding service and manufacturing costs , and based on subtasks Manufacturing reliability of selected services OK, time-consuming to manufacture The shorter the overall quality The larger the manufacturing cost The smaller the overall quality The larger the subtask Manufacturing reliability of selected services The higher the overall quality The larger the subtask The comprehensive quality is: in, , , are weights and , It is a subtask Manufacturing time The normalized index and the manufacturing time consumption are normalized by the negative index normalization method. The normalization process is obtained; It is a subtask Manufacturing cost The normalized index of the manufacturing cost is normalized by the negative index normalization method. The normalization process is obtained; It is a subtask Manufacturing reliability of selected services The normalized index of the selected service is used to determine the manufacturing reliability of the selected service through the normalization method of the positive index. The normalization process is obtained; Subtasks In any service The manufacturing time of the following is: Subtasks In any service The manufacturing cost is: in, It is a service Execute subtasks The production cost of the process, It is a subtask Previous subtask of The obtained products are transported to the service The transportation cost and transportation time of the location Positive correlation, It is the previous subtask The obtained products are transported to the service Location service Complete the subtask The storage cost during the process is Positive correlation.

2. The cloud manufacturing service combination selection method according to claim 1, It is characterized in that The cloud manufacturing service portfolio with the best comprehensive quality includes: Constructing the state vector for a deep reinforcement learning algorithm and any state The corresponding action space ,state Indicates a subtask with the Scheduled status The corresponding service or status has been arranged Unscheduled to indicate a subtask No service has been arranged yet, action Corresponding subtask The candidate service set Services in ; Cloud manufacturing tasks The comprehensive quality evaluation function under each cloud manufacturing service combination is the reward of reinforcement learning. The deep Q learning algorithm is used to iteratively solve the problem based on the state space and action space to obtain the comprehensive quality evaluation function under each cloud manufacturing service combination.

3. The cloud manufacturing service combination selection method according to claim 2, It is characterized in that Using the deep Q learning algorithm to iteratively solve based on the state vector and action space includes: Initialize the number of iterations to 1 and initialize the network parameters of the neural network; Reset the state vector at the beginning of any iteration cycle Until all statuses are unscheduled; Initialization parameters ,in accordance with The action selection strategy is based on the state The corresponding action space Select an action ; When action Corresponding services Not a status Corresponding subtasks When the effective service is available, re-execute the The action selection strategy is based on the state The corresponding action space Select an action Steps; When action Corresponding services Yes Status Corresponding subtasks When the service is valid, the status Update to scheduled status and confirm status Corresponding subtasks Effective services and their subtasks The manufacturing interval of And execute the above again The action selection strategy is based on the state The corresponding action space Select an action Steps; When the state vector When all states in the current iteration cycle are in the scheduled state, the current iteration cycle is completed. Corresponding subtasks The overall quality After updating the network parameters of the neural network, the next iteration cycle is entered until the maximum iteration cycle is reached, and the effective services of the subtasks corresponding to all states are output to obtain the cloud manufacturing service combination with the best overall quality.

4. The cloud manufacturing service combination selection method according to claim 3, It is characterized in that According to each state in the current iteration cycle Corresponding subtasks The overall quality Updating the network parameters of the neural network includes: When the number of initialization iterations is 1, the network parameters of the initialization neural network include initializing the network parameters of the Q network and the network parameters of the target network same; After completing each iteration cycle, according to the cost loss function Update the network parameters of the Q network according to the gradient descent method ,in, Is a network parameter The state-action pair of the Q network The value of Yes Status Corresponding subtasks A valid service with network parameters The output of the target network , Yes Status Corresponding subtasks The overall quality , is the discount factor, Is a network parameter The state-action pair of the target network the value of After every predetermined number of iterations, the network parameters of the Q network are Passed to the target network to update the network parameters of the target network Until the maximum iteration period is reached.