A complex product autonomous construction method and module based on human-machine hybrid enhancement

Abstract

The embodiments of the present invention disclose a method and module for autonomously constructing complex products based on human-machine hybrid enhancement, the method comprising: S10, establishing a parameterized knowledge model mapping from key feature states to actions based on human knowledge, encoding the parameters to be measured in the knowledge model into an output vector of a neural network; S20, encoding the input vector of the neural network according to the key feature states of the environment and the output value of the reward function; S40, intelligent agent configuration; S50, generating training data mapping from key feature states to actions based on human-known empirical knowledge, using the training data to reversely fit the neural network parameters, and driving the knowledge model to deduce in the training environment; S60, continuously optimizing the neural network parameters based on reinforcement learning until the training of the neural network is completed, and finally achieving the self-adaptation of the intelligent agent.

Description

Technical Field

[0001] The present invention relates to the field of complex products, and more specifically, to a complex product autonomous construction method, module, storage medium and computer device based on human-machine hybrid enhancement. Background Art

[0002] Complex products refer to products with complex structure, complicated design and manufacturing process, high technical content, involving the intersection and integration of knowledge from multiple disciplines, and are capital-intensive, technology-intensive, and knowledge-intensive. The design of complex products is a process of parallel design in multiple fields such as mechanics, electronics, and control.

[0003] Traditionally, complex product design, represented by complex aerospace products, mainly relies on human experience and cognition, supplemented by computational analysis and simulation tests, to make indicator system decisions, analyze technical approaches, and design overall plans from top to bottom. As the technical indicators of the new generation of complex products are becoming increasingly higher, the conditions and working conditions are becoming more and more demanding, and the game collaboration requirements are becoming increasingly difficult, the design is difficult to close, difficult to optimize, and difficult to speed up. Traditional methods have become increasingly difficult to adapt to the development of the new generation of complex products.

[0004] In essence, complex product design problems are multi-objective and multi-constraint optimization problems of high-order multi-dimensional nonlinear systems with high technical indicators, complex working conditions, and multi-party game collaboration. Limited by people's lack of abstraction level, association ability, and learning efficiency for complex problems, there are difficulties such as "difficulty in abstracting" the model of high-order multi-dimensional nonlinear systems, "difficulty in associating" feasible solutions under multi-objective and multi-constraint requirements, and "difficulty in learning" the quantitative laws of existing analysis and experimental data. Summary of the invention

[0005] In view of this, machine autonomous learning is better at abstracting complex system models, correlating massive information factors, and learning deep quantitative rules. Through the means of human-machine hybrid enhanced intelligence, it can provide strong support for new methods and tools for complex product design.

[0006] The first embodiment of the present invention provides a complex product autonomous construction method based on human-machine hybrid enhancement, comprising:

[0007] S00, establishing a parameterized knowledge model mapping from key feature states to actions based on human knowledge, and encoding the parameters to be measured in the knowledge model into an output vector of a neural network;

[0008] S20, encoding the input vector of the neural network according to the key feature state of the environment and the output value of the reward function;

[0009] S40, agent configuration;

[0010] S60, generating training data mapping from key feature states to actions based on human experience knowledge, using the training data to reversely fit neural network parameters, and driving the knowledge model to deduce in the training environment;

[0011] S80. Continuously optimize the neural network parameters based on reinforcement learning until the training of the neural network is completed, and finally achieve self-adaptation of the intelligent agent.

[0012] In a specific embodiment, the S20 includes:

[0013] S201, extracting key feature states of the training environment for integration, encoding the integrated key feature states into a type of input vector of the neural network, and normalizing the vector;

[0014] S203: Establish a reward function that maps from the key feature state to the reward value, and encode the output value of the reward function into another type of input vector of the neural network.

[0015] In a specific embodiment, the S40 includes:

[0016] S401. Design and code the intelligent agent decision network and evaluation network based on artificial experience;

[0017] S403, designing and encoding hyperparameters for agent training;

[0018] S405. Encode the relevant design into the machine learning engine and prepare for machine autonomous learning.

[0019] In a specific embodiment, the S60 includes:

[0020] S801, the output vector of the neural network drives the knowledge model to be deduced in a training environment;

[0021] S803, deduce that the key characteristic state value of the training environment is changed, and obtain the reward function value;

[0022] S805. Based on the key feature state value, the reward function value and the output vector of the neural network, obtain a training signal for the deep neural network, and update the neural network parameters through a back propagation algorithm.

[0023] The second embodiment of the present invention provides a complex product autonomous construction module based on human-machine hybrid enhancement, including:

[0024] A parameterized knowledge model and neural network output vector construction unit, used to establish a parameterized knowledge model mapping from key feature states to actions based on human knowledge, and encode the parameters to be measured in the knowledge model into an output vector of the neural network;

[0025] A neural network input vector building unit, used to encode the input vector of the neural network according to the key feature state of the environment and the output value of the reward function;

[0026] An agent configuration unit, used for agent configuration;

[0027] An initial training data generation unit based on human known experience knowledge, used to generate training data mapping from key feature states to actions based on human known experience knowledge, and use the training data to reversely fit neural network parameters to drive the knowledge model to deduce in the training environment;

[0028] The neural network parameter optimization unit is used to continuously optimize the neural network parameters based on reinforcement learning until the training of the neural network is completed, thereby finally achieving the self-adaptation of the intelligent body.

[0029] A third embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method described in the first embodiment of the present invention is implemented.

[0030] A fourth embodiment of the present invention provides a computer device, including a processor and a memory storing a program, wherein the processor implements the method described in the first embodiment of the present invention when executing the program.

[0031] The beneficial effects of the present invention are as follows:

[0032] The method provided by the present invention can make full use of human abstract analysis capabilities, and give full play to the advantages of machine autonomous learning in abstracting complex system models "true", correlating massive information factors "strongly", and learning deep quantitative rules "fast". Based on machine autonomous learning under human guidance, it can efficiently learn the complex correlation between multiple factors from massive data, and realize the intelligent generation of overall plans for complex products, thereby having the ability to close technical pathways to quickly approach optimal performance, bringing about efficiency improvements and changes in design capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A flow chart of a complex product autonomous construction method based on human-machine hybrid enhancement according to an embodiment of the present invention is shown.

[0035] Figure 2A schematic structural diagram of a computer device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] In order to explain the present invention more clearly, the present invention is further described below in conjunction with preferred embodiments and drawings. It should be understood by those skilled in the art that the content described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0037] like Figure 1 As shown, an embodiment of the present invention provides a complex product autonomous construction method based on human-machine hybrid enhancement, including:

[0038] S00, establishing a parameterized knowledge model mapping from key feature states to actions based on human knowledge, and encoding the parameters to be measured in the knowledge model into an output vector of a neural network;

[0039] Introduce the knowledge model used by people in similar product design (which can be a collection of knowledge models), determine the parameters that traditional people need to design, and use neural networks to output the parameters required in the design.

[0040] For the design of complex aerospace products, the classic proportional navigation method of the control system is an existing knowledge model. The parameters to be designed are the navigation ratio and filter bandwidth for different target airspaces. The parameters to be designed are encoded into the output vector of the neural network.

[0041] S20, encoding the input vector of the neural network according to the key characteristic state of the environment and the output value of the reward function, the specific steps are:

[0042] S201, extracting key feature states of the training environment and integrating them, encoding the integrated key feature states into a type of input vector of the neural network, and normalizing them.

[0043] Through human analysis and transformation of the physical training environment or virtual training environment, the key characteristic states of the training environment are extracted and integrated, and the key characteristic states are encoded into a type of input vector of the neural network. Each dimension of the vector is a numerical representation of a certain characteristic of the product state and the environmental state. For the design of complex aerospace products, the product state information involves position, speed, acceleration, inclination, mass, etc., and the environmental state information involves target position, target speed, noise interference, etc.

[0044] S203: Establish a reward function that maps from the key feature state to the reward value, and encode the output value of the reward function into another type of input vector of the neural network.

[0045] The reward function is set based on manual experience, and the reward function is a mapping from key feature states to reward values. Based on manual experience, the reward function can be set for subtasks based on the decomposition of tasks, and finally the output of the reward function is encoded into another type of input vector of the neural network.

[0046] S40, agent configuration, specific steps:

[0047] S401. Design and code the intelligent agent decision network and evaluation network based on artificial experience;

[0048] S403, designing and encoding hyperparameters for agent training;

[0049] S405. Encode the relevant design into the machine learning engine and prepare for machine autonomous learning.

[0050] S60, generating training data mapping from key feature states to actions based on human experience knowledge, using the training data to reversely fit neural network parameters, and driving the knowledge model to deduce in the training environment;

[0051] After relaxing the problem conditions (making a certain value in the key characteristic state known, such as making the unknown information of the target acceleration known in the virtual training environment), the problem is simplified, and training data is generated based on existing empirical knowledge to reversely fit the neural network parameters. The output of the neural network can drive the knowledge model to deduce in the training environment.

[0052] S80. Continuously optimize the neural network parameters based on reinforcement learning until the training of the neural network is completed, and finally achieve self-adaptation of the intelligent agent.

[0053] Each time the output of the neural network drives the knowledge model to perform deduction in the training environment, the deduction changes the key characteristic state values ​​of the training environment and obtains the reward function value. Based on the reward function value and the parameters taken by the knowledge model, the training signal for the deep neural network can be obtained, and through the back propagation algorithm, the training signal is transmitted to all the parameters of the deep neural network for parameter update to complete the training, and finally realize the self-adaptation and self-growth of machine intelligence.

[0054] The method provided by the present invention can make full use of human abstract analysis capabilities, and give full play to the advantages of machine autonomous learning in abstracting complex system models "true", correlating massive information factors "strongly", and learning deep quantitative rules "fast". Based on machine autonomous learning under human guidance, it can efficiently learn the complex correlation between multiple factors from massive data, and realize the intelligent generation of overall plans for complex products, thereby having the ability to close technical pathways to quickly approach optimal performance, bringing about efficiency improvements and changes in design capabilities.

[0055] The second embodiment of the present invention provides a complex product autonomous construction module based on human-machine hybrid enhancement, including:

[0056] A parameterized knowledge model and neural network output vector construction unit, used to establish a parameterized knowledge model mapping from key feature states to actions based on human knowledge, and encode the parameters to be measured in the knowledge model into an output vector of the neural network;

[0057] A neural network input vector building unit, used to encode the input vector of the neural network according to the key feature state of the environment and the output value of the reward function;

[0058] An agent configuration unit, used for agent configuration;

[0059] An initial training data generation unit based on human known experience knowledge, used to generate training data mapping from key feature states to actions based on human known experience knowledge, and use the training data to reversely fit neural network parameters to drive the knowledge model to deduce in the training environment;

[0060] The neural network parameter optimization unit is used to continuously optimize the neural network parameters based on reinforcement learning until the training of the neural network is completed, and finally the self-adaptation of the intelligent agent is achieved. It can be understood by those skilled in the art that the module in this embodiment can also implement other method steps described in Example 1, which will not be repeated here.

[0061] The method provided by the present invention can make full use of human abstract analysis capabilities, and give full play to the advantages of machine autonomous learning in abstracting complex system models "true", correlating massive information factors "strongly", and learning deep quantitative rules "fast". Based on machine autonomous learning under human guidance, it can efficiently learn the complex correlation between multiple factors from massive data, and realize the intelligent generation of overall plans for complex products, thereby having the ability to close technical pathways to quickly approach optimal performance, bringing about efficiency improvements and changes in design capabilities.

[0062] A third embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in the first embodiment of the present invention.

[0063] In practical applications, the computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0064] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0065] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0066] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0067] like Figure 2As shown, a schematic diagram of the structure of a computer device provided by the fourth embodiment of the present invention. Figure 2 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0068] like Figure 2 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0069] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0070] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0071] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 2 not shown, usually called a "hard drive"). Although Figure 2 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0072] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0073] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 20. Figure 2 As shown, the network adapter 20 communicates with other modules of the computer device 12 via the bus 18. It should be understood that although Figure 2 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0074] The processor unit 16 executes the programs stored in the system memory 28 to perform various functional applications and data processing, such as implementing the method provided by the first embodiment of the present invention.

[0075] The method provided by the present invention can make full use of human abstract analysis capabilities, and give full play to the advantages of machine autonomous learning in abstracting complex system models "true", correlating massive information factors "strongly", and learning deep quantitative rules "fast". Based on machine autonomous learning under human guidance, it can efficiently learn the complex correlation between multiple factors from massive data, and realize the intelligent generation of overall plans for complex products, thereby having the ability to close technical pathways to quickly approach optimal performance, bringing about efficiency improvements and changes in design capabilities.

[0076] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. A complex product autonomous construction method based on human-machine hybrid enhancement, characterized in that: include: S00, establishing a parameterized knowledge model mapping from key feature states to actions based on human knowledge, and encoding the parameters to be measured in the knowledge model into an output vector of a neural network; S20, encoding the input vector of the neural network according to the key feature state of the environment and the output value of the reward function; The S20 includes: S201, extracting key feature states of the training environment for integration, encoding the integrated key feature states into a type of input vector of the neural network, and normalizing the vector; S203, establishing a reward function that maps from the key feature state to the reward value, and encoding the output value of the reward function into another type of input vector of the neural network; S40, agent configuration; S60, generating training data mapping from key feature states to actions based on human experience knowledge, and using the training data to reversely fit neural network parameters to drive the knowledge model to deduce in the training environment; The S60 includes: S801, the output vector of the neural network drives the knowledge model to be deduced in a training environment; S803, deduce that the key characteristic state value of the training environment is changed, and obtain the reward function value; S805, obtaining a training signal for a deep neural network based on the key feature state value, the reward function value and the output vector of the neural network, and updating the neural network parameters through a back propagation algorithm; S80. Continuously optimize the neural network parameters based on reinforcement learning until the training of the neural network is completed, and finally achieve self-adaptation of the intelligent agent.

2. The method according to claim 1, characterized in that The S40 includes: S401. Design and code the intelligent agent decision network and evaluation network based on artificial experience; S403, designing and encoding hyperparameters for agent training; S405. Encode the relevant design into the machine learning engine and prepare for machine autonomous learning.

3. A complex product autonomous construction system based on human-machine hybrid enhancement, characterized in that: include: A parameterized knowledge model and neural network output vector construction unit, used to establish a parameterized knowledge model mapping from key feature states to actions based on human knowledge, and encode the parameters to be measured in the knowledge model into an output vector of the neural network; A neural network input vector building unit, used to encode the input vector of the neural network according to the key feature state of the environment and the output value of the reward function; Extracting key feature states of the training environment for integration, encoding the integrated key feature states into a type of input vector of the neural network, and normalizing the vector; Establishing a reward function that maps from the key feature state to the reward value, and encoding the output value of the reward function into another type of input vector of the neural network; An agent configuration unit, used for agent configuration; An initial training data generation unit based on human known experience knowledge, used to generate training data mapping from key feature states to actions based on human known experience knowledge, and use the training data to reversely fit neural network parameters to drive the knowledge model to deduce in the training environment; The output vector of the neural network is used to drive the knowledge model to perform deduction in the training environment; By deducing the key characteristic state values ​​of the training environment, we can obtain the reward function value. Based on the key feature state value, the reward function value and the output vector of the neural network, a training signal for the deep neural network is obtained, and the neural network parameters are updated through a back propagation algorithm; The neural network parameter optimization unit is used to continuously optimize the neural network parameters based on reinforcement learning until the training of the neural network is completed, thereby finally achieving the self-adaptation of the intelligent body.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

5. A computer device comprising a processor and a memory storing a program, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 2 is implemented.

Images