Manned and unmanned collaborative command decision-making system and method based on deep reinforcement learning
Through deep reinforcement learning technology, combined with detection, computing and command decision-making modules, the deep learning neural network model is used to solve the problem of optimal configuration of human and unmanned equipment under the constraints of human power, weapons and costs, and realize the human power and weapons and best defense plan under cost constraints.
Patent Information
- Application Number
- CN202410748612.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-06-12
AI Technical Summary
In the case of manpower, weapons and cost constraints, effective solutions are lacking in the prior art.
Adopt a human-unmanned collaborative command decision-making system based on deep reinforcement learning. Through detection modules, calculation modules and command decision-making modules, multiple sensors and deep learning neural network models are used to identify offensive equipment types and calculate the defense success probability of defense equipment. Assign unmanned defense equipment to obtain the best defense plan.
Under human, weapons and cost constraints, the best defense plan can be obtained, which improves defense efficiency and resource utilization.
Smart Images

Figure CN118534921B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a manned and unmanned collaborative command decision-making system and method based on deep reinforcement learning, and belongs to the technical field of artificial intelligence. Background Art
[0002] In recent years, domestic research institutes have applied various artificial intelligence technologies to various fields of command decision-making and control. In terms of command decision-making, deep learning and reinforcement learning technologies are used to carry out applied research in combat situation recognition, combat mission planning, penetration confrontation, and unmanned swarm system combat.
[0003] However, at present, there are no reports on how to optimally configure manned and unmanned equipment under the constraints of weapons, manpower and costs in the field of manned and unmanned collaborative command, decision-making and control. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, the present invention provides a manned-unmanned collaborative command decision-making system and method based on deep reinforcement learning. The present invention can obtain the best defense plan under the constraints of manpower, weapons and cost through a deep learning neural network.
[0005] To achieve the above-mentioned purpose, the present invention provides a manned and unmanned collaborative command decision-making system based on deep reinforcement learning, which includes: a detection module, a calculation module and a command decision-making module, wherein the detection module is configured to detect the type a of the blue side's offensive equipment and the number of offensive equipment of each type a through multiple sensors. ; The calculation module is configured to calculate the type The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys the resource s , and calculate the success rate of a defensive device of type d defending a resource s against an offensive device of type a The command decision module includes a first deep learning neural network model, which is configured to protect the red side's S resources against all types of offensive equipment from the blue side against the red side's D types of manned defense equipment, and the number of each type of manned and unmanned defense equipment. Allocate to obtain the best defense plan, 1≤a≤A, 1≤s≤S, 1≤d≤D, A and S are positive integers greater than or equal to 1, and D is 2.
[0006] To achieve the above-mentioned purpose, the present invention also provides a manned and unmanned collaborative command decision-making method based on deep reinforcement learning, which comprises the following steps:
[0007] Step 1: Use multiple sensors to detect the blue team's information, and use the recognition module to identify the type a of the offensive equipment and the number of offensive equipment of each type a based on the detection information provided by the multiple sensors. ;
[0008] Step 2: Calculate the type through the calculation module The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys the resource s , and calculate the success rate of a defensive device of type d defending a resource s against an offensive device of type a ;
[0009] Step 3: To protect the Red side’s S resources from all types of offensive equipment from the Blue side, the command decision module including the first deep learning neural network model is used to analyze the Red side’s D types of manned defense equipment and the number of each type of manned and unmanned defense equipment. Allocate to obtain the best defense plan, 1≤a≤A, 1≤s≤S, 1≤d≤D, A and S are positive integers greater than or equal to 1, and D is 2.
[0010] Compared with the prior art, the present invention has the following beneficial effects:
[0011] The present invention can obtain the best defense plan under the constraints of manpower, weapons and costs through deep learning neural networks. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 It is a block diagram of the composition of the manned and unmanned collaborative command decision-making system based on deep reinforcement learning provided by the first embodiment of the present invention.
[0013] Figure 2 It is a workflow diagram of the command decision module provided in the first embodiment of the present invention.
[0014] Figure 3 It is a working flow chart of the detection module provided in the first embodiment of the present invention. DETAILED DESCRIPTION
[0015] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0016] First embodiment
[0017] Figure 1 1 is a block diagram of the manned and unmanned collaborative command decision-making system based on deep reinforcement learning provided by the first embodiment of the present invention. Figure 1As shown, the manned and unmanned collaborative command decision-making system based on deep reinforcement learning in the first embodiment of the present invention includes a detection module, a calculation module and a command decision-making module, wherein the detection module includes multiple sensors and an identification module, wherein the multiple sensors are used to detect the information of the blue side, and the identification module identifies the type a of the offensive equipment and the number R of offensive equipment of each type a according to the detection information provided by the multiple sensors. a The calculation module is configured to calculate the probability g that the attacking equipment of type a breaks through the defense of the defensive equipment and completely destroys the resource s sa , and calculate the success rate of a defensive device of type d defending a resource s against an offensive device of type a The command decision module includes a first deep learning neural network model, which is configured to protect the red side's S resources against all types of offensive equipment from the blue side against the red side's D types of manned defense equipment, and the number of each type of manned and unmanned defense equipment. Allocate to obtain the best defense plan, 1≤a≤A, 1≤s≤S, 1≤d≤D, A and S are positive integers greater than or equal to 1, D is 2, when d is 1, it means manned defense equipment, when d is 2, it means unmanned defense equipment.
[0018] Figure 2 FIG. 1 is a flowchart of the command decision module provided by the first embodiment of the present invention. Figure 2 As shown, the first deep learning neural network model is a five-dimensional continuous Hopfield neural network model, and the five-dimensional continuous Hopfield neural network is configured to protect the S resources of the red side against all types of offensive equipment of the blue side against the D types of manned and unmanned defense equipment of the red side, and the number of manned and unmanned defense equipment of each type Distribution, including:
[0019] S1-01: Use the following formula to calculate the activation equation of the neurons in the continuous Hopfield neural network :
[0020]
[0021] In the formula, For neurons under the energy function constraint and neurons The connection strength of the pass For neurons External inputs constrained by energy functions; For the neuron under the weapon availability constraint and neurons The connection strength, To provide a neural network under the constraint of defense equipment availability External input; For neurons and neurons The strength of the connection under human constraints, For neurons External input under manpower constraints; Neurons under weapon availability constraints and neurons The connection strength, For neurons under cost constraints External input; for A is the total number of offensive equipment types; for The total number of defense equipment of this type; D is the total number of defense equipment types; S is the total number of resources to be protected; , , , is the coefficient, For neurons The learning factor;
[0022] S1-02: Use the first-order Euler method to calculate Input variables of neurons :
[0023] + ;
[0024] S1-03: Calculate the Output variables of neurons :
[0025]
[0026] In the formula, For the The initial input value of the neuron;
[0027] S1-04: Assign a value to t and calculate the energy function according to the following formula:
[0028]
[0029] In the formula, is the value of resource s; For a type The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys resource s; For type Plans for offensive equipment and offensive resources; The probability of successful interception for a defensive device of type d defending a resource s against an offensive device of type a; The total number of Class A offensive equipment; is the total number of type d defense equipment; D is the total number of defense equipment types; S is the total number of resources to be protected;
[0030] S1-05: Determine whether the energy function E1 reaches the minimum. If the energy function E1 does not reach the minimum, return to step S1-01; if the energy function E1 reaches the minimum, the iteration ends and outputs .
[0031] In the first embodiment, after the iteration, the neuron Output value When it is 1, it means that the i-th defensive equipment of type d needs to resist the attack resources s of the j-th offensive equipment of type a; neuron Output value When it is 0, it means that no attack plan is assigned.
[0032] The present invention can obtain the best defense plan under the constraints of manpower, weapons and costs through the above technical solution.
[0033] In the first embodiment, at least one of the multiple sensors is an image sensor, which is used to obtain an image of the blue square.
[0034] In the first embodiment, the recognition module includes a convolutional neural network (CNN) and a template matching module, wherein the convolutional neural network (CNN) obtains an offensive equipment image based on image information; the template matching module includes a circular sampling filter, a radial sampling filter and a template matching filter, wherein the circular sampling filter uses the projections of the offensive equipment image and the template image on a set of rings to divide the pixels of the offensive equipment image into two categories: pixels that have no chance of matching the template image and first-level candidate pixels that have a certain chance of matching the template image, and determines a possible scaling factor for each first-level candidate pixel; the radial sampling filter upgrades some first-level candidate pixels to second-level candidate pixels based on the projections of the offensive equipment image and the template image on a set of radial lines and the possible scaling factors provided by the circular sampling filter, and also assigns a possible rotation angle to each second-level candidate pixel, discarding pixels that have not been upgraded; the template matching filter resizes and rotates the template image to F angles and E scales, and calculates the scale factor at scale S e and angle α f The contrast and brightness perception correlation between the template image at the second-level candidate pixel and the offensive equipment image at the second-level candidate pixel. If the absolute value of the correlation is higher than the threshold, it is considered that the template is found at the second-level candidate pixel, f=0,2…,F-1; e=0,2…,E-1, E and F are positive integers greater than or equal to 2.
[0035] In the first embodiment, the contrast and brightness perception correlation is:
[0036]
[0037] In the formula , yes The average value of is the vector obtained by copying the grayscale of the pixels of the template image; ; yes , q is a vector obtained by copying the grayscale of the pixels of the attack equipment image corresponding to the template image; q=βa+γI+ε, I is the brightness vector; β is the contrast correction factor, and γ is the brightness correction factor, ε is the vector of residual errors.
[0038] In the first embodiment, the circular sampling filter uses the projections of the attacking equipment image and the template image on a set of rings to divide the pixels of the attacking equipment image into two categories: pixels that have no chance of matching the template image and first-level candidate pixels that have a certain chance of matching the template image, and determines a possible scaling factor for each first-level candidate pixel, specifically including:
[0039] S3-01: Resize the template image Q to each scale S e , obtain the resized template Q0,…,Q e , ..., Q E-1 ;
[0040] S3-02: For each resized template Q e Circular sampling is performed with a predefined radius, resulting in a multi-scale rotation-invariant feature C with E rows and L columns Q The 2D matrix:
[0041] C Q e,k = Cis Q e x 0 , y 0 , r k = ∑ f=0 F-1 Q e [ x- x 0 + r k cos α f , y- y 0 + r k sin α f ]
[0042] Where (x0, y0) is the center pixel coordinate of the template image Q; e = 0, 1, ..., E-1; k = 0, 2 ..., L-1;
[0043] S3-03: Given an offensive equipment image A for analysis, construct a 3D image matrix C A (x, y, k):
[0044]
[0045] S3-04: Using Matrix C Q and C A and contrast to detect the circular sampling correlation CisCorr at the best matching scale for each pixel (x, y):
[0046]
[0047] if , t1 is the threshold; the pixel (x, y) is classified as a first-level candidate pixel, otherwise it is a pre-discarded pixel. The best matching scale of the first-level candidate pixel is:
[0048] .
[0049] In the first embodiment, the radial sampling filter uses the projections of the attacking equipment image and the template image on a set of radial lines and possible scaling factors provided by the circular sampling filter to upgrade some primary candidate pixels to secondary candidate pixels, and also assigns possible rotation angles to each secondary candidate pixel including:
[0050] S4-01: Given an image template Q and N angles of inclination (α0, α1, ..., α F-1 ), using λ=r L-1 Radial sampling is performed on the image template Q to generate a vector R with M features Q (f):
[0051] R Q f = Ras Q r L-1 x 0 , y 0 , α f = ∑ k=0 L-1 Q[ x- x 0 + r k cos α f , y- y 0 + r k sin α f ] ,
[0052] S4-02: For each first-level candidate pixel (x, y), attack equipment image A with its possible scales Radial sampling,
[0053] ,
[0054] S4-03: At each first-level candidate pixel (x, y), the radial sampling filter uses the vector R A (x,y,f),R Q (f) and contrast to detect the radial sampling correlation RasCorr at the best matching angle:
[0055]
[0056] In the formula, represents the f positions of the cyclic shifted independent variable vector;
[0057] S4-04:If , If is the threshold, then it is the second-level candidate pixel (x, y), otherwise it is discarded. The optimal rotation angle for the second-level candidate pixel (x, y) is:
[0058] .
[0059] The first embodiment of the present invention can achieve the beneficial effect of quickly identifying the type of attacking equipment through the above technical solution.
[0060] Figure 3 FIG. 1 is a flowchart of a detection module provided in the first embodiment of the present invention. Figure 3 As shown, the detection module includes a second deep learning neural network model, which is a one-dimensional continuous Hopfield neural network model. The one-dimensional continuous Hopfield neural network model is configured to divide the total time of the sensor detecting the offensive equipment into multiple time periods t r , the sensor is a radar, and the one-dimensional continuous Hopfield neural network model is configured to divide the total time of radar detection of offensive equipment into multiple time periods t r Specifically include:
[0061] S2-01: Use the following formula to calculate the activation equation of a neuron in a one-dimensional continuous Hopfield neural network. Use the following formula to calculate the activation equation of a neuron :
[0062]
[0063] In the formula, is the radar scanning time period t r The probability of discovering an attack weapon; v r is the scanning time period t between the radar and the attack weapon r The relative speed of W r is the radar scanning time period t r The scanning width of A r is the radar scanning time period tr The scanning area; R is the total number of time periods;
[0064] S2-02: Calculate the input variable of the rth neuron using the first-order Euler method :
[0065] + ;
[0066] S2-03: Calculate the output variable of the rth neuron according to the following formula :
[0067] ,
[0068] In the formula, is the initial input value of the rth neuron;
[0069] S2-04: Assign a value to t and calculate the energy function according to the following formula:
[0070] ;
[0071] S2-05: Determine the energy function Whether it reaches the minimum, if the energy function If the minimum value is not reached, the process returns to step S2-01; if the energy function E2 reaches the minimum value, the iteration ends and the output is .
[0072] The present invention can obtain the best scanning time period t through the above technical solution r .
[0073] Second embodiment
[0074] The second embodiment of the present invention only scans the same contents as those of the first embodiment, and the same contents are not scanned repeatedly.
[0075] To achieve the above-mentioned purpose, the present invention also provides a manned and unmanned collaborative command decision-making method based on deep reinforcement learning, which comprises the following steps:
[0076] Step 1: Use multiple sensors to detect the blue team's information, and use the recognition module to identify the type a of the offensive equipment and the number of offensive equipment of each type a based on the detection information provided by the multiple sensors. ;
[0077] Step 2: Calculate the type through the calculation module The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys the resource s , and calculate the success rate of a defensive device of type d defending a resource s against an offensive device of type a ;
[0078] Step 3: To protect the Red side’s S resources from all types of offensive equipment from the Blue side, the command decision module including the first deep learning neural network model is used to analyze the Red side’s D types of manned defense equipment and the number of each type of manned and unmanned defense equipment. Allocate to obtain the best defense plan, 1≤a≤A, 1≤s≤S, 1≤d≤D, A and S are positive integers greater than or equal to 1, and D is 2.
[0079] The first deep learning neural network model in the second embodiment is the same as the deep learning neural network model in the first embodiment, both of which are five-dimensional continuous Hopfield neural network models. The second embodiment will not be described again.
[0080] Third embodiment
[0081] A third embodiment of the present invention provides a computer program product, which utilizes a computer language to compile the method described in the second embodiment into a computer program code that can be called by a processor.
[0082] The beneficial effects of the computer program product provided by the third embodiment are the same as those of the first or second embodiment, and will not be described again here.
[0083] Fourth embodiment
[0084] A fourth embodiment of the present invention provides a computer storage device storing a computer program code, wherein the computer program code can be called by a processor to execute the method described in the first and second embodiments. The storage medium includes a memory. The processor includes a CPU.
[0085] In a fourth embodiment, a storage device includes a storage array that can provide persistent data storage for a processor. The storage array can be arranged on a rack, and the storage array can be arranged in another rack. The storage array can include one or more storage array controllers. The storage array controller can be implemented as an automatic computer module including computer hardware, computer software, or a combination of computer hardware and software. The storage array controller can be configured to perform various storage tasks. The storage tasks can include writing data received from the processor to the storage array, erasing data from the storage array, retrieving data from the storage array and providing data to the processor, monitoring and reporting disk utilization and performance, performing redundancy operations, such as data redundancy operations, compressing data, encrypting data, etc.
[0086] The storage array controller may be implemented in various ways, including as a field programmable gate array ("FPGA"), a programmable logic chip ("PLC"), an application specific integrated circuit ("ASIC"), a system on a chip ("SOC"), or any computing device including discrete components such as a processing unit, a central processing unit, a computer memory, or various adapters. The storage array controller may include, for example, a data communications adapter configured to support communications via a SAN or a LAN. The storage array controller may be independently coupled to a LAN. In an implementation, the storage array controller may include an I / O controller, etc., which couples the storage array controller to persistent storage resources via a midplane for data communications. The persistent storage resources may include any number of storage drives and any number of non-volatile random access memory devices.
[0087] The NVRAM device of the persistent storage resource may be configured to receive data stored on the storage drive from the storage array controller. The data may originate from the processor. Writing the data to the NVRAM device may be performed faster than writing the data directly to the storage drive. In an implementation, the storage array controller may be configured to utilize the NVRAM device as a fast accessible buffer for data known to be written to the storage drive. Compared to a system in which the storage array controller writes data directly to the storage drive, the latency of a write request using the NVRAM device as a buffer may be improved.
[0088] In some implementations, the NVRAM device may be implemented in a computer memory in the form of high-bandwidth, low-latency RAM. The NVRAM device is referred to as "non-volatile" because it can accept or include an inherent power source that maintains the RAM state after the main power supply to the NVRAM device is lost. Such a power source can be a battery, one or more capacitors, etc. In response to power loss, the NVRAM device can be configured to write the contents of the RAM to a permanent storage device, such as a storage drive. A storage drive may refer to any device configured to permanently record data, where "permanently" or "permanent" refers to the ability of the device to maintain the recorded data after power loss. In some implementations, a storage drive may represent a non-disk storage medium. For example, a storage drive may be one or more solid-state drives (SSDs), flash-based storage, any type of solid-state non-volatile memory, or any other type of non-mechanical storage device. In other implementations, the storage drive may include a mechanical or rotating hard disk, such as a hard disk drive ("HDD").
[0089] In some implementations, a storage array controller may be configured to remove device management responsibilities from storage drives in a storage array. For example, a storage array controller may manage control information that represents the status of one or more storage blocks of a storage drive. The control information may indicate, for example, that a particular memory block has failed and is no longer being written to, that a particular memory block includes boot code for a storage array controller, the number of program-erase ("P / E") cycles performed on a particular memory block, the age of data stored in a particular memory block, the type of data stored in a particular memory block, and the like.
[0090] In some implementations, the control information can be stored as metadata with the associated memory blocks.
[0091] In other embodiments, the control information of the storage drive may be stored in one or more specific memory blocks in the storage drive selected by the storage array controller. The selected memory blocks may be marked with an identifier indicating that the selected memory blocks include the control information. The storage array controller may utilize the identifier in conjunction with the storage drive to quickly identify the memory blocks that include the control information. For example, the storage controller may issue a command to locate the memory blocks that include the control information.
[0092] It should be noted that because the control information is very large, portions of the control information may be stored in multiple locations, such as for redundancy, or the control information may be otherwise distributed across multiple memory blocks of the storage drive.
[0093] In an implementation, a storage array controller may remove device management responsibilities from a storage driver of a storage array by retrieving control information describing the state of one or more memory blocks of the storage driver from the storage driver. For example, retrieval of control information from a storage driver may be performed by the storage array controller querying the storage driver for the location of control information about a particular storage driver. The storage driver may be configured to execute instructions that enable the storage driver to identify the location of the control information. The instructions may be executed by a controller associated with the storage driver or otherwise located in the storage driver, and may cause the storage driver to scan a portion of each memory block to identify a memory block storing control information for the storage driver. The storage driver may respond by sending a response message including the location of the control information for the storage driver to the storage array controller. In response to receiving the response message, the storage array controller performs control for the storage driver.
[0094] The beneficial effects of the computer storage device provided by the fourth embodiment are the same as those of the first or second embodiment, and will not be described again here.
[0095] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. The meaning of "several" is one or more, unless otherwise clearly and specifically defined.
[0096] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A manned and unmanned collaborative command decision-making system based on deep reinforcement learning, characterized in that: It includes a detection module, a calculation module and a command decision module, wherein the detection module includes multiple sensors and an identification module, wherein the multiple sensors are used to detect the information of the blue side, and the identification module identifies the type a of the attacking equipment and the number of each type a of the attacking equipment according to the detection information provided by the multiple sensors ; The calculation module is configured to calculate the type The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys the resource s , and calculate the success rate of a defensive device of type d defending a resource s against an offensive device of type a The command decision module includes a first deep learning neural network model, which is configured to protect the red side's S resources against all types of offensive equipment from the blue side against the red side's D types of manned defense equipment, and the number of each type of manned and unmanned defense equipment. Allocate to obtain the best defense plan, 1≤a≤A, 1≤s≤S, 1≤d≤D, A and S are positive integers greater than or equal to 1, and D is 2; The first deep learning neural network model is a five-dimensional continuous Hopfield neural network model, wherein the five-dimensional continuous Hopfield neural network is configured to protect the S resources of the Red side against all types of offensive equipment of the Blue side against the D types of manned and unmanned defense equipment of the Red side, and the number of manned and unmanned defense equipment of each type. Distribution, including: S1-01: Use the following formula to calculate the activation equation of the neuron : , where For neurons under the energy function constraint and neurons The connection strength of the pass For neurons External inputs constrained by energy functions; For the neuron under the weapon availability constraint and neurons The connection strength, To provide a neural network under the constraint of defense equipment availability External input; For neurons and neurons Connection strength under human constraints, For neurons External input under manpower constraints; Neurons under weapon availability constraints and neurons The connection strength, For neurons under cost constraints External input, For neurons Medium defensive equipment medium type The cost of n pieces of equipment in; for A is the total number of offensive equipment types; for The total number of defense equipment of this type; D is the total number of defense equipment types; S is the total number of resources to be protected; , , , is the coefficient, For neurons The learning factor; S1-02: Use the first-order Euler method to calculate Input variables of neurons : + ; S1-03: Calculate the Output variables of neurons : , where U 01 For neurons The initial input value of S1-04: Assign a value to t and calculate the energy function according to the following formula: , where is the value of resource s; For a type The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys resource s; For type Plans for offensive equipment and offensive resources; The probability of successful interception for a defensive device of type d defending a resource s against an offensive device of type a; The total number of Class A offensive equipment; is the total number of type d defense equipment; D is the total number of defense equipment types; S is the total number of resources to be protected; S1-05: Determine whether the energy function E1 reaches the minimum. If the energy function E1 does not reach the minimum, return to step S1-01; if the energy function E1 reaches the minimum, the iteration ends and outputs .
2. The manned and unmanned collaborative command decision-making system based on deep reinforcement learning according to claim 1 is characterized in that: Neurons Output value When it is 1, it means that the i-th defensive equipment of type d needs to resist the attack resources s of the j-th offensive equipment of type a; neuron Output value When it is 0, it means that no attack plan is assigned.
3. The manned and unmanned collaborative command decision-making system based on deep reinforcement learning according to claim 1 is characterized in that: The detection module includes a second deep learning neural network model, the second deep learning neural network model includes a one-dimensional continuous Hopfield neural network model, and the one-dimensional continuous Hopfield neural network model is configured to divide the total time of the sensor detecting the offensive equipment into a plurality of time segments t r .
4. The manned and unmanned collaborative command decision-making system based on deep reinforcement learning according to claim 3 is characterized in that: The sensor is a radar, and the one-dimensional continuous Hopfield neural network model is configured to divide the total time of the radar detecting the offensive equipment into a plurality of time segments t r Specifically include: S2-01: Use the following formula to calculate the activation equation of the neuron : , where is the radar scanning time period t r The probability of discovering an attack weapon; v r is the scanning time period t between the radar and the attack weapon r The relative speed of W r is the radar scanning time period t r The scanning width of A r is the radar scanning time period t r The scanning area; R is the total number of time segments S2-02: Calculate the input variable of the rth neuron using the first-order Euler method : + ; S03: Calculate the output variable of the rth neuron according to the following formula : , where U 02 is the initial input value of neuron r; S04: Assign a value to t and calculate the energy function according to the following formula: ; S2-05: Determine the energy function Whether it reaches the minimum, if the energy function If the minimum value is not reached, the process returns to step S2-01; if the energy function E2 reaches the minimum value, the iteration ends and the output is .
5. A manned and unmanned collaborative command decision-making method based on deep reinforcement learning, characterized in that: The steps include: Step 1: Use multiple sensors to detect the blue team's information, and use the recognition module to identify the type a of the offensive equipment and the number of offensive equipment of each type a based on the detection information provided by the multiple sensors. ; Step 2: Calculate the type through the calculation module The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys the resource s , and calculate the success rate of a defensive device of type d defending a resource s against an offensive device of type a ; Step 3: To protect the Red side’s S resources from all types of offensive equipment from the Blue side, the command decision module including the first deep learning neural network model is used to analyze the Red side’s D types of manned defense equipment and the number of each type of manned and unmanned defense equipment. Allocate to obtain the best defense plan, 1≤a≤A, 1≤s≤S, 1≤d≤D, A and S are positive integers greater than or equal to 1, and D is 2; The first deep learning neural network model is a five-dimensional continuous Hopfield neural network model, wherein the five-dimensional continuous Hopfield neural network is configured to protect the S resources of the Red side against all types of offensive equipment of the Blue side against the D types of manned and unmanned defense equipment of the Red side, and the number of manned and unmanned defense equipment of each type. Distribution, including: S1-01: Use the following formula to calculate the activation equation of the neuron : , where For neurons under the energy function constraint and neurons The connection strength of the pass For neurons External inputs constrained by energy functions; For the neuron under the weapon availability constraint and neurons The connection strength, To provide a neural network under the constraint of defense equipment availability External input; For neurons and neurons Connection strength under human constraints, For neurons External input under manpower constraints; Neurons under weapon availability constraints and neurons The connection strength, For neurons under cost constraints External input, For neurons Medium defensive equipment medium type The cost of n pieces of equipment in; for A is the total number of offensive equipment types; for The total number of defense equipment of this type; D is the total number of defense equipment types; S is the total number of resources to be protected; , , , is the coefficient, For neurons The learning factor; S1-02: Use the first-order Euler method to calculate Input variables of neurons : + ; S1-03: Calculate the Output variables of neurons : , where U 01 For neurons The initial input value of S1-04: Assign a value to t and calculate the energy function according to the following formula: , where is the value of resource s; For a type The probability that the offensive equipment breaks through the defense of the defensive equipment and completely destroys resource s; For type Plans for offensive equipment and offensive resources; The probability of successful interception for a defensive device of type d defending a resource s against an offensive device of type a; The total number of Class A offensive equipment; is the total number of type d defense equipment; D is the total number of defense equipment types; S is the total number of resources to be protected; S1-05: Determine whether E1 has reached the minimum value. If E1 has not reached the minimum value, return to step S1-01; if E1 has reached the minimum value, the iteration ends and the output is .
6. The manned and unmanned collaborative command decision-making method based on deep reinforcement learning according to claim 5 is characterized in that: Neurons Output value When it is 1, it means that the i-th defensive equipment of type d needs to resist the attack resources s of the j-th offensive equipment of type a; neuron Output value When it is 0, it means that no attack plan is assigned.
7. The manned and unmanned collaborative command decision-making method based on deep reinforcement learning according to claim 5 is characterized in that: The detection module includes a second deep learning neural network model, which is a one-dimensional continuous Hopfield neural network model. The one-dimensional continuous Hopfield neural network model is configured to divide the total time of the sensor detecting the offensive equipment into a plurality of time segments t r .
8. The manned and unmanned collaborative command decision-making method based on deep reinforcement learning according to claim 7 is characterized in that: The sensor is a radar, and the one-dimensional continuous Hopfield neural network model is configured to divide the total time of the radar detecting the offensive equipment into a plurality of time segments t r Specifically include: S2-01: Use the following formula to calculate the activation equation of the neuron. Use the following formula to calculate the activation equation of the neuron. : , where is the radar scanning time period t r The probability of discovering an attack weapon; v r is the scanning time period t between the radar and the attack weapon r The relative speed of W r is the radar scanning time period t r The scanning width of A r is the radar scanning time period t r The scanning area; R is the total number of time segments; S2-02: Calculate the input variable of the rth neuron using the first-order Euler method : + ; S03: Calculate the output variable of the rth neuron according to the following formula : ; In the formula, is the initial input value of neuron r; S04: Assign a value to t and calculate the energy function according to the following formula: ; S2-05: Determine whether the energy function E2 has reached the minimum. If the energy function E2 has not reached the minimum, return to step S2-01; if the energy function E2 has reached the minimum, the iteration ends and the output is .
Citation Information
Patent Citations
DDoS active defense system and method based on distributed collaborative learning
CN115102767A
Task-oriented and anti-collision dynamic route planning system based on reinforcement learning
CN117666609A
Interpretability-Aware Adversarial Attack and Defense Method for Deep Learnings
US20210216859A1
Feature manipulation-based attack and defense method for continuous learning ability system
WO2023070696A1
