Assembly process intelligent decision-making method and system based on reinforcement learning optimization

Abstract

The invention discloses an assembly process intelligent decision-making method and system based on reinforcement learning optimization. The method comprises the steps that assembling segmentation operation, assembling parameter data and assembling technology intelligent decision targets of the assembling technology are expressed as knowledge graph objects, and the knowledge graph objects form a directed reasoning network graph; the knowledge graph objects are stored in the knowledge graph knowledge base and are visually expressed as nodes of the knowledge graph network; according to the features of the assembly parts and the intelligent decision-making target of the assembly process, selecting the optimal optional assembly parts according to the optimized sequence, and constructing assembly feature forward reasoning based on the knowledge graph; the assessment requirements and specifications of the assembly process are represented as process inference rules in a formalized mode, the appropriate process inference rules are selected according to the assembly target, the knowledge graph objects and parameter data thereof and evaluation function values matched with the selected process inference rules are calculated, and the knowledge graph object with the maximum evaluation function value and the parameter data thereof are optimally selected; and performing iterative selection optimization until the evaluation function value reaches an expected target threshold value or the number of iterations reaches a preset maximum number of iterations threshold value, and memorizing an optimization learning result (the optimized and selected knowledge graph object and parameter data thereof) into a knowledge base as a stage result of reinforcement learning. And thus, the efficiency of assembly process design is improved, and manual errors and repeated labor of process designers are reduced.

Description

Technical Field

[0001] This application relates to the field of intelligent decision-making for assembly processes, and particularly to an intelligent decision-making method and system for assembly processes optimized based on reinforcement learning. Background Art

[0002] Currently, due to the rise of new assembly processes in intelligent manufacturing, the assembly processes are characterized by complexity, randomness, and uncertainty. As a result, the intelligent decision-making efficiency of assembly processes is low, the influencing factors are complex and numerous, and the uncertain risks are difficult to control. The traditional assembly process management mode is relatively simple, with many manual management links, and is not suitable for the optimization of intelligent decision-making for assembly processes. To solve this problem, it is necessary to improve the intelligent decision-making system for assembly processes. Summary of the Invention

[0003] The purpose of this application is to provide an intelligent decision-making method and system for assembly processes optimized based on reinforcement learning to solve the problems of low intelligent decision-making efficiency and difficult control of uncertain risks in assembly processes.

[0004] To solve the above problems, the first aspect of this application provides an intelligent decision-making method for assembly processes optimized based on reinforcement learning, which is characterized by including: representing the assembly segmented operations, assembly parameter data, and the objectives of intelligent decision-making for assembly processes as knowledge graph objects, and the knowledge graph objects form a directed inference network graph; the knowledge graph objects are stored in the knowledge graph database and visually represented as the nodes of the knowledge graph network; according to the characteristics of the assembly parts and the objectives of intelligent decision-making for assembly processes, select the optimal optional assembly parts in the optimized order to construct a forward inference of assembly features based on the knowledge graph; formalize the assessment requirements and specifications of the assembly process as process inference rules, adapt the appropriate process inference rules according to the assembly objectives, calculate the evaluation function values of the matching of the knowledge graph objects and their parameter data with the selected process inference rules, and optimize and select the knowledge graph objects and their parameter data with the largest evaluation function values; iteratively select and optimize until the evaluation function value reaches the expected target threshold or the number of iterations reaches the preset maximum iteration threshold, and memorize the results of the optimized learning (the knowledge graph objects and their parameter data after optimized selection) into the database as the stage results of reinforcement learning for the priority recommendation reference of the next intelligent decision-making for assembly processes, thereby improving the efficiency of assembly process design and reducing the manual errors and repetitive labor of process designers.

[0005] In a preferred embodiment, the assembly segmented operations of the assembly process include at least one or a combination of: assembly part segmented classification operation, assembly part segmented assembly operation, post-segmented assembly test operation, post-segmented assembly overall assembly operation, and post-overall assembly test operation.

[0006] In a preferred embodiment, the assembly parameter data includes at least one or a combination of: assembly part segmentation category data, assembly part status data, space-time sequence data of segmented assembly of assembly parts, test data after segmented assembly of assembly parts, space-time sequence data of segmented overall assembly after segmented assembly, and test data after overall assembly.

[0007] In a preferred embodiment, the objectives of the intelligent decision-making for the assembly process include at least one or a combination of: the minimum cost of the assembly process, the maximum efficacy of the assembly process, the shortest time consumption of the assembly process operation, and the longest service life of the assembly process product.

[0008] Furthermore, according to the ownership relationship and causal time sequence relationship between the assembly segment operations of the assembly process, the assembly parameter data, and the objectives of the intelligent decision-making for the assembly process, the assembly segment operations of the assembly process, the assembly parameter data, and the objectives of the intelligent decision-making for the assembly process are represented by entity nodes, and the ownership relationship and causal time sequence relationship between the assembly segment operations of the assembly process and the assembly parameter data, and the objectives of the intelligent decision-making for the assembly process are represented by directed arcs between the entity nodes, to construct the knowledge graph object of the assembly process. The knowledge graph object constitutes a network graph capable of directed reasoning. The directed reasoning is to infer a new knowledge graph object according to the knowledge graph object in the current state, the knowledge graph object in the past state, and the process reasoning rule that triggers a conclusion based on conditions, and construct the forward reasoning of assembly features based on the knowledge graph.

[0009] Furthermore, matching the data features of the assembly process knowledge graph object with the conditional part of the assembly process reasoning rule, selecting the most suitable process reasoning rule for matching, and performing the forward reasoning of assembly features based on the knowledge graph, including: the step of collecting assembly parameter data, the step of selecting an empirical segmented assembly combination scheme of assembly parts, the step of forward reasoning and evaluation of assembly features based on the knowledge graph, and the step of preferentially selecting the calculation result of the most suitable process reasoning rule.

[0010] According to another aspect of the present application, there is provided an intelligent decision-making system for assembly process optimized based on reinforcement learning. The system includes: a module for collecting assembly parameter data, which uses sensors of assembly parts and equipment to collect segmented category data of assembly parts, initial state data of assembly parts, and spatio-temporal sequence data of segmented assembly of assembly parts, and is the core hardware of the intelligent decision-making system for assembly process optimized based on reinforcement learning; a module for selecting an empirical segmented assembly combination plan of assembly parts, which is used to randomly select one of the untested best segmented assembly combination plans of assembly parts from the candidate set of common best segmented assembly combination plans of assembly parts according to the segmented category data of assembly parts, the state data of assembly parts, the spatio-temporal sequence data of segmented assembly of assembly parts, and the candidate set of common best segmented assembly combination plans of assembly parts; a module for forward reasoning and evaluation of assembly features based on a knowledge graph, which is used to select the optimal optional assembly parts in an optimized order according to the features of assembly parts and the objectives of intelligent decision-making of assembly process, construct forward reasoning of assembly features based on a knowledge graph, and select the most suitable process reasoning rule with the highest matching degree according to the degree of matching between the data features of the assembly process knowledge graph object and the conditional part of the assembly process reasoning rule, and obtain the conclusion part of the selected process reasoning rule to construct a new assembly process knowledge graph object; a module for preferentially calculating the results of the most matching suitable process reasoning rule, which is used to calculate the matching reasoning conclusion of the data features of the assembly process knowledge graph object and the value of the assembly process knowledge graph evaluation function according to the priority of the assembly process reasoning rule, and optimize and select the most matching suitable process reasoning rule, the matching reasoning conclusion, and the value of the assembly process knowledge graph evaluation function according to the sorting result of the values of these assembly process knowledge graph evaluation functions; a reinforcement learning optimization server for intelligent decision-making of assembly process, which is used to store the program for intelligent decision-making of assembly process and reinforcement learning optimization, and summarize and analyze the data of intelligent decision-making of assembly process.

[0011] In a preferred embodiment, the module for collecting assembly parameter data includes: an assembly part segmented category data collection unit for collecting assembly part segmented category data; an assembly part status data collection unit for collecting assembly part status data; an assembly part segmented assembly spatio-temporal sequence data collection unit for collecting assembly part segmented assembly spatio-temporal sequence data; an assembly part segmented assembly post-test data collection unit for collecting assembly part segmented assembly post-test data; a segmented assembly post-integrated assembly segmented spatio-temporal sequence data collection unit for collecting segmented spatio-temporal sequence data of the integrated assembly after segmented assembly; an integrated assembly post-test data collection unit for collecting integrated assembly post-test data; an assembly part segmented classification operation unit for performing assembly part segmented classification operations; an assembly part segmented assembly operation unit for performing assembly part segmented assembly operations; an assembly part segmented assembly post-test operation unit for performing assembly part segmented assembly post-test operations; a segmented assembly post-integrated assembly operation unit for performing segmented assembly post-integrated assembly operations; an integrated assembly post-test operation unit for performing integrated assembly post-test operations.

[0012] Further, in the module for forward reasoning and evaluation of assembly features based on a knowledge graph, the degree of matching between the data features of the assembly process knowledge graph object and the conditional part of the assembly process reasoning rule is calculated using an assembly process knowledge graph evaluation function, and the value of the assembly process knowledge graph evaluation function is related to the accurate probability of the data features of the assembly process knowledge graph object and the matching threshold of the conditional part of the assembly process reasoning rule.

[0013] Beneficial Effects

[0014] The above technical solutions of the present application have the following beneficial technical effects:

[0015] The method and system for intelligent decision-making of assembly processes optimized based on reinforcement learning proposed in the embodiments of the present application can relatively accurately judge and control the optimization process of intelligent decision-making of assembly processes, and memorize the results of optimization learning (the knowledge graph objects and their parameter data after optimized selection) into the knowledge base as the stage results of reinforcement learning for priority recommendation and reference in the next intelligent decision-making of assembly processes, thereby improving the efficiency of assembly process design and reducing the manual errors and repetitive labor of process designers. Brief Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings:

[0017] Figure 1 is a flowchart of an intelligent decision-making method for assembly processes optimized based on reinforcement learning in the first embodiment of this application;

[0018] Figure 2 is a schematic diagram of an intelligent decision-making system for assembly processes optimized based on reinforcement learning in an alternative embodiment of this application. Detailed implementation manners

[0019] To enable those skilled in the art to better understand the technical solutions proposed in this application, the following will clearly and completely describe the technical solutions in the embodiments of this specification with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on one or more embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application. Obviously, the described embodiments are a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application. In addition, the technical features involved in different implementation manners of this application described below can be combined with each other as long as they do not conflict with each other.

[0020] An intelligent decision-making method for assembly processes optimized based on reinforcement learning in this application includes:

[0021] Represent the assembly segmented operations, assembly parameter data, and the objectives of intelligent decision-making for assembly processes as knowledge graph objects, and the knowledge graph objects form a network graph for directed reasoning;

[0022] The knowledge graph objects are stored in the knowledge graph database and visually represented as the nodes of the knowledge graph network;

[0023] According to the characteristics of the assembled parts and the objectives of intelligent decision-making for assembly processes, select the optimal optional assembled parts in the optimized order to construct a forward reasoning of assembly features based on the knowledge graph;

[0024] Formalize the assessment requirements and specifications of the assembly process as process inference rules, select appropriate process inference rules according to the assembly target, calculate the evaluation function value of the knowledge graph object and its parameter data matching the selected process inference rules, and optimize and select the knowledge graph object and its parameter data with the largest evaluation function value;

[0025] Iteratively select and optimize until the evaluation function value reaches the expected target threshold or the number of iterations reaches the preset maximum iteration threshold, and memorize the result of the optimized learning (the optimized and selected knowledge graph object and its parameter data) into the knowledge base as the stage result of reinforcement learning for the next intelligent decision-making of the assembly process to give priority recommendations for reference, thereby improving the efficiency of the assembly process design and reducing the manual errors and repetitive labor of process designers.

[0026] Next, the intelligent decision-making method and system for the assembly process optimized based on reinforcement learning proposed in this application will be described in detail with reference to the accompanying drawings. In a specific implementation manner, the acquisition of assembly parameter data can be realized through various sensors and underlying controllers related to the assembly process.

[0027] As Figure 1 shown, in the first aspect of the embodiment of this application, an intelligent decision-making method for the assembly process optimized based on reinforcement learning is provided, including:

[0028] S1: Represent the assembly segmentation operation, assembly parameter data, and the target of the intelligent decision-making of the assembly process as knowledge graph objects, and the knowledge graph objects form a directed inference network graph;

[0029] S2: Store the knowledge graph objects in the knowledge graph knowledge base and visually represent them as the nodes of the knowledge graph network;

[0030] S3: According to the characteristics of the assembly parts and the target of the intelligent decision-making of the assembly process, select the optimal optional assembly parts in the optimized order to construct a forward inference of the assembly features based on the knowledge graph;

[0031] S4: Formalize the assessment requirements and specifications of the assembly process as process inference rules, select appropriate process inference rules according to the assembly target, calculate the evaluation function value of the knowledge graph object and its parameter data matching the selected process inference rules, and optimize and select the knowledge graph object and its parameter data with the largest evaluation function value;

[0032] S5: Iteratively select and optimize until the evaluation function value reaches the expected target threshold or the number of iterations reaches the preset maximum iteration threshold, and memorize the result of the optimized learning (the optimized and selected knowledge graph object and its parameter data) into the knowledge base.

[0033] In an implementation manner, let i represent the number of the assembly segmentation operation of the assembly process, oi represents the assembly segmentation operation of the assembly process, j represents the number of the assembly parameter data, d j represents the assembly parameter data, f 1 represents the cost objective function of the assembly process, f 2 represents the efficacy objective function of the assembly process, f 3 represents the time consumption objective function of the assembly process operation, f 4 represents the service life objective function of the assembly process product, and t represents time.

[0034]

[0035]

[0036] Among them, O represents the set of assembly segmentation operations of the assembly process, m represents the total number of assembly segmentation operations of the assembly process, D represents the set of assembly parameter data of the assembly process, and n represents the total number of assembly parameter data of the assembly process.

[0037] In one embodiment, the assembly segmentation operations of the assembly process include: at least one or a combination of assembly part segmentation classification operations, assembly part segmentation assembly operations, post-assembly part segmentation assembly test operations, post-segmentation assembly overall assembly operations, and post-overall assembly test operations. The assembly parameter data includes: at least one or a combination of assembly part segmentation category data, status data of the assembly parts, spatio-temporal sequence data of the assembly part segmentation assembly, test data after the assembly part segmentation assembly, spatio-temporal sequence data of the post-segmentation assembly overall assembly, and test data after the overall assembly. The objectives of the intelligent decision-making of the assembly process include: at least one or a combination of the minimum cost of the assembly process, the maximum efficacy of the assembly process, the shortest time consumption of the assembly process operation, and the longest service life of the assembly process product.

[0038] In one embodiment, the intelligent decision-making steps of the assembly process optimized based on reinforcement learning include:

[0039] Steps for collecting assembly parameter data: Using sensors of the assembly parts and equipment, collect the segmentation category data of the assembly parts, the initial status data of the assembly parts, and the spatio-temporal sequence data of the assembly part segmentation assembly.

[0040] Steps for selecting an empirical assembly part segmentation assembly combination scheme: According to the segmentation category data of the assembly parts, the status data of the assembly parts, the spatio-temporal sequence data of the assembly part segmentation assembly, and the candidate set of the commonly used best assembly part segmentation assembly combination schemes, randomly select one of the best assembly part segmentation assembly combination schemes that have not been selected and tested from the candidate set of the commonly used best assembly part segmentation assembly combination schemes;

[0041] Positive Inference and Evaluation Steps of Assembly Features Based on Knowledge Graph: According to the features of assembly parts and the goals of intelligent decision-making for assembly processes, the optimal optional assembly parts are selected in the optimized order to construct the positive inference of assembly features based on the knowledge graph. According to the degree of matching between the data features of the assembly process knowledge graph object and the conditional part of the assembly process inference rule, the most suitable process inference rule with the highest matching degree is selected, and the conclusion part of the selected process inference rule is obtained to construct a new assembly process knowledge graph object; the degree of matching between the data features of the assembly process knowledge graph object and the conditional part of the assembly process inference rule is calculated using the assembly process knowledge graph evaluation function, and the value of the assembly process knowledge graph evaluation function is related to the accurate probability of the data features of the assembly process knowledge graph object and the matching threshold of the conditional part of the assembly process inference rule;

[0042] Optimal Selection Step of the Calculation Results of the Most Matched Suitable Process Inference Rule: According to the priority of the assembly process inference rule, the matching inference conclusion of the data features of the assembly process knowledge graph object and the value of the assembly process knowledge graph evaluation function are calculated. According to the sorting results of the values of these assembly process knowledge graph evaluation functions, the most matched suitable process inference rule, the matching inference conclusion, and the value of the assembly process knowledge graph evaluation function are optimally selected.

[0043] Next, an assembly process intelligent decision-making system optimized based on reinforcement learning using the above method is described. The assembly process intelligent decision-making system optimized based on reinforcement learning includes:

[0044] A module for collecting assembly parameter data, which uses sensors of assembly parts and equipment to collect the segmented category data of assembly parts, the initial state data of assembly parts, and the spatio-temporal sequence data of segmented assembly of assembly parts, and is the core hardware of the assembly process intelligent decision-making system optimized based on reinforcement learning;

[0045] A module for selecting an empirical segmented assembly combination plan of assembly parts, which is used to randomly select one of the best untested segmented assembly combination plans of assembly parts from the candidate set of common best segmented assembly combination plans of assembly parts according to the segmented category data of assembly parts, the state data of assembly parts, the spatio-temporal sequence data of segmented assembly of assembly parts, and the candidate set of common best segmented assembly combination plans of assembly parts;

[0046] A module for forward reasoning and evaluation of assembly features based on a knowledge graph, which is used to select the optimal optional assembly parts in an optimized order according to the features of the assembly parts and the goals of intelligent decision-making for the assembly process, construct a forward reasoning of assembly features based on the knowledge graph, and select the most suitable process reasoning rule with the highest matching degree according to the degree of matching between the data features of the assembly process knowledge graph object and the conditional part of the assembly process reasoning rule, obtain the conclusion part of the selected process reasoning rule, and construct a new assembly process knowledge graph object;

[0047] A module for calculating the results of the most matching suitable process reasoning rules and making optimal selections, which is used to calculate the matching reasoning conclusions of the data features of the assembly process knowledge graph object and the values of the assembly process knowledge graph evaluation function according to the priorities of the assembly process reasoning rules, and optimize and select the most matching suitable process reasoning rules, matching reasoning conclusions and the values of the assembly process knowledge graph evaluation function according to the sorting results of the values of these assembly process knowledge graph evaluation functions;

[0048] A reinforcement learning optimization server for intelligent decision-making of the assembly process, which is used to store the intelligent decision-making program for the assembly process and the reinforcement learning optimization program, and summarize and analyze the data of the intelligent decision-making of the assembly process.

[0049] In one embodiment, according to the belonging relationship and causal time sequence relationship between the assembly segment operations of the assembly process, the assembly parameter data, and the goals of the intelligent decision-making of the assembly process, entity nodes are used to represent the assembly segment operations of the assembly process, the assembly parameter data, and the goals of the intelligent decision-making of the assembly process, and directed arcs between entity nodes are used to represent the belonging relationship and causal time sequence relationship between the assembly segment operations of the assembly process, the assembly parameter data, and the goals of the intelligent decision-making of the assembly process, construct the knowledge graph object of the assembly process, and the knowledge graph object constitutes a network graph capable of directed reasoning. The directed reasoning is to infer a new knowledge graph object according to the knowledge graph object in the current state, the knowledge graph object in the past state, and the process reasoning rule that triggers the conclusion based on the conditions, and construct a forward reasoning of assembly features based on the knowledge graph.

[0050] In one embodiment, the module for collecting assembly parameter data includes: an assembly part segmented category data collection unit for collecting assembly part segmented category data; an assembly part status data collection unit for collecting assembly part status data; an assembly part segmented assembly spatio-temporal sequence data collection unit for collecting assembly part segmented assembly spatio-temporal sequence data; an assembly part segmented assembly post-test data collection unit for collecting assembly part segmented assembly post-test data; a segmented assembly post-whole assembly segmented spatio-temporal sequence data collection unit for collecting segmented spatio-temporal sequence data after segmented assembly and whole assembly; a whole assembly post-test data collection unit for collecting whole assembly post-test data; an assembly part segmented classification operation unit for performing assembly part segmented classification operations; an assembly part segmented assembly operation unit for performing assembly part segmented assembly operations; an assembly part segmented assembly post-test operation unit for performing assembly part segmented assembly post-test operations; a segmented assembly post-whole assembly operation unit for performing segmented assembly post-whole assembly operations; a whole assembly post-test operation unit for performing whole assembly post-test operations.

[0051] In an alternative embodiment, an intelligent decision-making system for assembly process optimized based on reinforcement learning is provided, and the intelligent decision-making system for assembly process optimized based on reinforcement learning is constructed according to the following steps.

[0052] 1. Construct a reinforcement learning optimization server for intelligent decision-making of assembly process

[0053] The reinforcement learning optimization server for intelligent decision-making of assembly process is the core hardware for intelligent decision-making of assembly process and reinforcement learning optimization, used for storing intelligent decision-making programs of assembly process and reinforcement learning optimization, and aggregating and analyzing data of intelligent decision-making of assembly process, as Figure 2 shown. The reinforcement learning optimization server for intelligent decision-making of assembly process uses a multi-core parallel GPU or high-performance CPU computing unit, and uses a parallel computing algorithm to improve the efficiency of intelligent decision-making of assembly process and reinforcement learning optimization.

[0054] 2. Construct a module for selecting an empirical segmented assembly combination plan of assembly parts

[0055] The module for an empirical segmented assembly combination plan of assembly parts randomly selects one of the best untested segmented assembly combination plans of assembly parts from the candidate set of common best segmented assembly combination plans of assembly parts according to the segmented category data of assembly parts, the status data of assembly parts, the spatio-temporal sequence data of segmented assembly of assembly parts, and the candidate set of common best segmented assembly combination plans of assembly parts.

[0056] 3. Create a module for forward reasoning and evaluation of assembly features based on a knowledge graph

[0057] The module for forward reasoning and evaluation of assembly features based on the knowledge graph selects the optimal optional assembly parts in the optimized order according to the features of the assembly parts and the goals of intelligent decision-making for the assembly process, constructs the forward reasoning of assembly features based on the knowledge graph, and selects the appropriate process reasoning rule with the highest matching degree according to the degree of matching between the data features of the assembly process knowledge graph object and the conditional part of the assembly process reasoning rule, obtains the conclusion part of the selected process reasoning rule, and constructs a new assembly process knowledge graph object.

[0058] The above is only an example. The intelligent decision-making system for assembly process optimization based on reinforcement learning adopted in this example can be extended to other intelligent decision-making optimization systems to achieve intelligent decision-making improvement and intelligent analysis of complex processes.

[0059] This application aims to protect an intelligent decision-making method for assembly process optimization based on reinforcement learning, including: representing the assembly segmentation operation, assembly parameter data, and the goals of intelligent decision-making for the assembly process as knowledge graph objects, and the knowledge graph objects form a directed reasoning network graph; the knowledge graph objects are stored in the knowledge graph database and visually represented as the nodes of the knowledge graph network; according to the features of the assembly parts and the goals of intelligent decision-making for the assembly process, select the optimal optional assembly parts in the optimized order and construct the forward reasoning of assembly features based on the knowledge graph; formalize the assessment requirements and specifications of the assembly process as process reasoning rules, select the appropriate process reasoning rules according to the assembly goals, calculate the evaluation function value of the matching between the knowledge graph object and its parameter data and the selected process reasoning rules, and optimize and select the knowledge graph object and its parameter data with the largest evaluation function value; perform iterative selection and optimization until the evaluation function value reaches the expected target threshold or the number of iterations reaches the preset maximum iteration threshold, and memorize the result of the optimized learning (the optimized and selected knowledge graph object and its parameter data) into the knowledge base as the stage result of reinforcement learning for priority recommendation and reference in the next intelligent decision-making of the assembly process, thereby improving the efficiency of assembly process design and reducing the manual errors and repetitive labor of process designers.

[0060] In addition, the present application also protects an intelligent decision-making system for assembly processes optimized based on reinforcement learning, including: a module for collecting assembly parameter data, which uses sensors of assembly parts and equipment to collect segmented category data of assembly parts, initial state data of assembly parts, and spatio-temporal sequence data of segmented assembly of assembly parts, and is the core hardware of the intelligent decision-making system for assembly processes optimized based on reinforcement learning; a module for selecting an empirical segmented assembly combination plan of assembly parts, which is used to randomly select one of the untested best segmented assembly combination plans of assembly parts from the candidate set of common best segmented assembly combination plans of assembly parts according to the segmented category data of assembly parts, the state data of assembly parts, the spatio-temporal sequence data of segmented assembly of assembly parts, and the candidate set of common best segmented assembly combination plans of assembly parts; a module for forward reasoning and evaluation of assembly features based on a knowledge graph, which is used to select the optimal optional assembly parts in the optimized order according to the features of assembly parts and the objectives of intelligent decision-making for assembly processes, construct forward reasoning of assembly features based on a knowledge graph, and select the most suitable process reasoning rule with the highest matching degree according to the degree of matching between the data features of the assembly process knowledge graph object and the conditional part of the assembly process reasoning rule, and obtain the conclusion part of the selected process reasoning rule to construct a new assembly process knowledge graph object; a module for preferentially calculating the results of the most matching suitable process reasoning rule, which is used to calculate the matching reasoning conclusion of the data features of the assembly process knowledge graph object and the value of the assembly process knowledge graph evaluation function according to the priority of the assembly process reasoning rule, and optimize and select the most matching suitable process reasoning rule, the matching reasoning conclusion, and the value of the assembly process knowledge graph evaluation function according to the sorting result of the values of these assembly process knowledge graph evaluation functions; a reinforcement learning optimization server for intelligent decision-making of assembly processes, which is used to store the programs for intelligent decision-making of assembly processes and reinforcement learning optimization, and summarize and analyze the data of intelligent decision-making of assembly processes.

[0061] It should be noted that the above-described embodiments can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, for this purpose, the program can run on a dedicated integrated circuit programmed for this purpose. It can also be implemented in any type of computing platform operably connected, including but not limited to personal computers, minicomputers, mainframes, workstations, network or distributed computing environments, standalone or integrated computer platforms, or communicating with charged particle tools or other imaging devices, etc. When the medium or device is read by a computer, it can be used to configure and operate the computer to execute the processes described herein. In addition, machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media include instructions or programs that implement the steps described above in combination with a microprocessor or other data processor, the invention described in this embodiment includes these and other different types of non-transitory computer-readable media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0062] The above embodiments are only for illustrating the technical concept and features of the present application, and the purpose is to enable those who are familiar with this technology to understand the content of the present application and implement it accordingly, and it cannot be used to limit the protection scope of the present application. All equivalent transformations or modifications made in the spirit of the present application should be covered within the protection scope of the present application.

Claims

1. An assembly process intelligent decision-making method based on reinforcement learning optimization, characterized in that: include: The assembly segmentation operations, assembly parameter data and assembly process intelligent decision-making goals of the assembly process are represented as knowledge graph objects, and the knowledge graph objects constitute a network graph for directed reasoning; Knowledge graph objects are stored in the knowledge graph knowledge base and are visualized as nodes of the knowledge graph network; According to the characteristics of assembly parts and the goal of intelligent decision-making of assembly process, the optimal optional assembly parts are selected in the order of optimization, and the assembly feature forward reasoning based on knowledge graph is constructed; The assessment requirements and specifications of the assembly process are formalized as process reasoning rules, the appropriate process reasoning rules are selected to adapt to the assembly goals, the evaluation function value of the knowledge graph object and its parameter data matching the selected process reasoning rules is calculated, and the knowledge graph object and its parameter data with the largest evaluation function value are optimized. Iterate the optimization until the evaluation function value reaches the expected target threshold or the number of iterations reaches the preset maximum number of iterations. The results of the optimization learning (the knowledge graph objects and their parameter data after optimization selection) are memorized in the knowledge base as the stage results of reinforcement learning, so as to provide priority reference for the next intelligent decision-making of the assembly process, thereby improving the efficiency of assembly process design and reducing manual errors and repetitive work of process designers.

2. The method according to claim 1, characterized in that The assembly segmentation operation of the assembly process includes: At least one of the following operations or a combination thereof: categorizing the assembled parts in sections, assembling the assembled parts in sections, testing the assembled parts after assembling them in sections, assembling the whole after assembling them in sections, and testing the whole after assembling them. The assembly parameter data includes: At least one of or a combination of segmented category data of assembly parts, status data of assembly parts, spatiotemporal sequence data of segmented assembly of assembly parts, test data after segmented assembly of assembly parts, segmented spatiotemporal sequence data of overall assembly after segmented assembly, and test data after overall assembly, The objectives of the assembly process intelligent decision-making include: At least one of the following, or a combination thereof: minimum cost of the assembly process, maximum efficiency of the assembly process, shortest time consumption of the assembly process operation, and longest service life of the assembly process product.

3. The method according to claim 1, characterized in that Also includes: According to the belonging relationship and causal temporal relationship between the assembly segment operations of the assembly process and the assembly parameter data and the target of the intelligent decision-making of the assembly process, entity nodes are used to represent the assembly segment operations of the assembly process, the assembly parameter data and the target of the intelligent decision-making of the assembly process, and directed arcs between entity nodes are used to represent the belonging relationship and causal temporal relationship between the assembly segment operations of the assembly process and the assembly parameter data and the target of the intelligent decision-making of the assembly process, and a knowledge graph object of the assembly process is constructed. The knowledge graph object constitutes a network graph that can perform directed reasoning. The directed reasoning is to infer a new knowledge graph object based on the knowledge graph object of the current state and the knowledge graph object of the past state and the process reasoning rules of the conditional trigger conclusion, and construct assembly feature forward reasoning based on the knowledge graph.

4. The method according to claim 1, characterized in that: Matching the data features of the assembly process knowledge graph object with the conditional part of the assembly process reasoning rule, selecting the most matching appropriate process reasoning rule, and performing the assembly feature forward reasoning based on the knowledge graph, including: Steps for collecting assembly parameter data: Use sensors of assembly parts and equipment to collect segmented category data of assembly parts, initial state data of assembly parts, and spatiotemporal series data of segmented assembly of assembly parts. The step of selecting an empirical assembly part segmented assembly combination scheme is as follows: based on the segmented category data of the assembly parts, the state data of the assembly parts, the spatiotemporal sequence data of the segmented assembly of the assembly parts, and the candidate set of commonly used optimal assembly part segmented assembly combination schemes, one of the best assembly part segmented assembly combination schemes that has not been tested is randomly selected from the candidate set of commonly used optimal assembly part segmented assembly combination schemes; Assembly feature forward reasoning and evaluation steps based on knowledge graph: according to the characteristics of assembly parts and the goal of assembly process intelligent decision-making, the optimal optional assembly parts are selected in an optimized order, and the assembly feature forward reasoning based on the knowledge graph is constructed. According to the degree of matching between the data characteristics of the assembly process knowledge graph object and the conditional part of the assembly process reasoning rule, the most suitable process reasoning rule with the highest matching degree is selected, the conclusion part of the optional process reasoning rule is obtained, and a new assembly process knowledge graph object is constructed; the degree of matching between the data characteristics of the assembly process knowledge graph object and the conditional part of the assembly process reasoning rule is calculated using the assembly process knowledge graph evaluation function, and the value of the assembly process knowledge graph evaluation function is related to the accuracy probability of the data characteristics of the assembly process knowledge graph object and the matching threshold of the conditional part of the assembly process reasoning rule; The preferred steps for calculating the most matching suitable process reasoning rule result are as follows: according to the priority of the assembly process reasoning rule, the matching reasoning conclusion of the data feature of the assembly process knowledge graph object and the value of the assembly process knowledge graph evaluation function are calculated; according to the sorting results of the values ​​of these assembly process knowledge graph evaluation functions, the most matching suitable process reasoning rule, the matching reasoning conclusion and the value of the assembly process knowledge graph evaluation function are optimally selected.

5. An assembly process intelligent decision-making system based on reinforcement learning optimization, characterized in that: include: The module for collecting assembly parameter data uses sensors of assembly parts and equipment to collect segmented category data of assembly parts, initial state data of assembly parts, and spatiotemporal sequence data of segmented assembly of assembly parts. It is the core hardware of the assembly process intelligent decision-making system based on reinforcement learning optimization. A module for selecting empirical assembly parts segmented assembly combination schemes is used to randomly select one of the best assembly parts segmented assembly combination schemes that have not been tested from the commonly used best assembly parts segmented assembly combination scheme candidate set according to the segmented category data of the assembly parts, the state data of the assembly parts, the spatiotemporal sequence data of the assembly parts segmented assembly and the commonly used best assembly parts segmented assembly combination scheme candidate set; The module of assembly feature forward reasoning and evaluation based on knowledge graph is used to select the best optional assembly parts in the order of optimization according to the characteristics of assembly parts and the goal of assembly process intelligent decision-making, construct assembly feature forward reasoning based on knowledge graph, select the most suitable process reasoning rule with the highest matching degree according to the degree of matching between the data characteristics of the assembly process knowledge graph object and the condition part of the assembly process reasoning rule, obtain the conclusion part of the optional process reasoning rule, and construct a new assembly process knowledge graph object; A module for optimizing the calculation results of the most matching appropriate process reasoning rules, which is used to calculate the matching reasoning conclusions of the data features of the assembly process knowledge graph object and the values ​​of the assembly process knowledge graph evaluation function according to the priority of the assembly process reasoning rules, and optimize the selection of the most matching appropriate process reasoning rules, matching reasoning conclusions and the values ​​of the assembly process knowledge graph evaluation function according to the sorting results of the values ​​of these assembly process knowledge graph evaluation functions; The reinforcement learning optimization server for assembly process intelligent decision-making is used to store assembly process intelligent decision-making and reinforcement learning optimization programs, and to summarize and analyze the data of assembly process intelligent decision-making.

Images