An open source intelligence reasoning skill automatic generation auxiliary decision-making method, device and equipment based on multi-agent self-game

Abstract

The application provides an open source intelligence reasoning skill automatic generation auxiliary decision-making method, device and equipment based on multi-agent self-game, receives multi-source intelligence, extracts key elements containing entities and labels confidence, source type and timeliness, and fuses into situation knowledge abstract; accordingly, a hierarchical skill library containing global basic skills, scene specialized skills and confrontation skills is constructed, and a reasoner agent and a challenger agent are initialized; thereafter, iterative execution of confrontation self-game is performed, the challenger generates a task with verification details according to a difficulty level adjusted according to historical pass rate, the reasoner calls the hierarchical skill library for solution, and the key parameters in the situation knowledge abstract are used to dynamically calculate the false negative cost weight and the false positive cost weight, and the weighted pass rate is calculated to judge the success or failure of the task, the failed task is used to update the scene specialized skill, the successful task is used to update the confrontation skill, and the global basic skill remains unchanged, and finally, the auxiliary decision-making result is output based on the solution conclusion and the judgment result.

Description

Technical Field

[0001] This invention relates to the field of large language model application technology, and in particular to an open-source intelligence reasoning skill automatic generation auxiliary decision-making method, device and equipment based on multi-agent self-play. Background Technology

[0002] Advances in natural language understanding and reasoning by large language models have led to their application in military decision-making support scenarios such as intelligence analysis and command decision-making. Since relying directly on parameterized knowledge is difficult to adapt to complex intelligence documents with high specialization and implicit rules, the industry is further exploring a "multi-agent self-play" approach. This allows the model to continuously generate and accumulate reusable reasoning experience through self-play without updating model parameters. One agent continuously proposes more challenging analytical tasks, while the other agent responds using existing experience. The experience sets of both agents are then iterated back based on the responses, allowing the model's capabilities to accumulate with each round of play without requiring manual annotation.

[0003] The reason why this kind of self-play-based ability accumulation can converge and continuously increase is implicitly dependent on a premise: there is a fixed and objective evaluation metric (such as a clear win or loss in game-like tasks), and the optimization direction of both players is thus stably constrained by this metric.

[0004] However, this premise does not hold true in military intelligence analysis missions. On the one hand, intelligence assessment typically lacks automatically comparable, objectively correct answers; judgments can only be made based on rewriteable rules. On the other hand, the consequences of "missed reports" and "false reports" are not equivalent, and this cost ratio changes with external circumstances such as intelligence timeliness, threat level, and decision-making window—meaning the assessment criteria themselves drift with the situation. When the assessment criteria are approximable and not constant, the optimization directions of both sides are no longer stably constrained but instead continuously drift towards the weaknesses of the current assessment standards: the party proposing the task constantly approaches the assessment blind spot, giving rise to a few extreme cases; the party responding repeatedly rewrites its experience to address these extreme cases, making new experience increasingly biased towards peripheral situations. Experience originally applicable across scenarios is gradually diluted, covered, or redundantly accumulated during iterations. As a result, with the increase in rounds of the game, the overall usability of the accumulated experience set decreases rather than increases.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This invention discloses an open-source intelligence reasoning skill automatic generation auxiliary decision-making method, device and equipment based on multi-agent self-play, which aims to solve the problem that the overall usability of reasoning skills accumulated in military intelligence self-play without objective adjudication benchmarks degrades with each round of play.

[0007] The first embodiment of the present invention provides an open-source intelligence reasoning skill automatic generation auxiliary decision-making method based on multi-agent self-play, comprising: Receive multi-source military intelligence, call a large model to extract key elements, and label each piece of intelligence with confidence level, source type and timeliness, and integrate the key elements and labels into a structured situational knowledge summary; A hierarchical skill base is constructed based on the situation knowledge summary, and a dual agent is initialized. The hierarchical skill base includes global basic skills that remain unchanged across scenarios, scenario-specific skills automatically generated based on entities in the situation knowledge summary, and adversarial skills for generating challenge tasks. The dual agent includes a reasoner agent loaded with global basic skills and scenario-specific skills, and a challenger agent loaded with adversarial skills. Using the hierarchical skill library and situational knowledge summary as input, adversarial self-play is iteratively executed. The challenger agent generates an analysis task with verification details based on the current difficulty level, and the inferrer agent calls the hierarchical skill library to solve it. The difficulty level is adjusted based on the inferrer agent's historical pass rate. The missed detection cost weight and false positive cost weight are dynamically calculated based on key parameters in the situational knowledge summary, and a weighted pass rate is calculated to determine the success or failure of the task. Failed tasks are used to update the scenario-specific skills, and successful tasks are used to update the adversarial skills, while the global basic skills remain unchanged. Based on the solution conclusions and decision results of the reasoner agent, auxiliary decision results are output.

[0008] The second embodiment of the present invention provides an open-source intelligence reasoning skill automatic generation auxiliary decision-making device based on multi-agent self-play, comprising: The intelligence preprocessing module is used to receive multi-source military intelligence, call a large model to extract key elements, and label each piece of intelligence with confidence level, source type and timeliness, and integrate the key elements and labels into a structured situational knowledge summary. An initialization module is used to construct a hierarchical skill library based on the situation knowledge summary and initialize a dual agent; the hierarchical skill library includes global basic skills that remain unchanged across scenarios, scenario-specific skills automatically generated based on entities in the situation knowledge summary, and adversarial skills for generating challenge tasks; the dual agent includes a reasoner agent loaded with global basic skills and scenario-specific skills, and a challenger agent loaded with adversarial skills; The self-play loop module is used to iteratively execute adversarial self-play using the hierarchical skill library and situational knowledge summary as input. The challenger agent generates an analysis task with verification details based on the current difficulty level, and the inferrer agent calls the hierarchical skill library to solve it. The difficulty level is adjusted based on the inferrer agent's historical pass rate. The module dynamically calculates the false negative cost weight and false positive cost weight based on key parameters in the situational knowledge summary, and calculates the weighted pass rate to determine the success or failure of the task. Failed tasks are used to update the scenario-specific skills, and successful tasks are used to update the adversarial skills, while the global basic skills remain unchanged. The decision output module is used to output auxiliary decision results based on the solution conclusions and decision results of the reasoner agent.

[0009] The third embodiment of the present invention provides an automatic generation auxiliary decision-making device based on open-source intelligence reasoning skills of multi-agent self-play, including a memory and a processor. The memory stores a computer program, which can be executed by the processor to realize the automatic generation auxiliary decision-making method based on open-source intelligence reasoning skills of multi-agent self-play as described in any of the above.

[0010] The fourth embodiment of the present invention provides a computer-readable storage medium storing a computer program, which can be executed by the processor of the device where the computer-readable storage medium is located, to realize the open-source intelligence reasoning skill automatic generation auxiliary decision-making method based on multi-agent self-play as described in any of the above.

[0011] Beneficial effects: This invention uses the timeliness, threat level, and decision urgency in the situation knowledge summary as key parameters to dynamically calculate the weight of missed and false alarm costs, and uses the weighted pass rate to determine the success or failure of the task. In this way, an effective decision signal that is adaptive to the situation and reflects the asymmetric costs of missed and false alarms is established in open analysis tasks without standard answers, so that self-play can continuously generate effective adversarial pressure.

[0012] This invention categorizes skills into global basic skills, scenario-specific skills, and adversarial skills. For failed tasks, only scenario-specific skills are updated; for successful tasks, only adversarial skills are updated; and global basic skills remain unchanged. This allows for differentiated and independent updates of general experience and specialized experience, avoiding mutual interference and redundant accumulation caused by mixed updates of general and specialized skills, and ensuring the universality and stability of the accumulated skills.

[0013] This invention performs cross-round skill maintenance on the hierarchical skill base after each iteration, filtering out redundant, inefficient, and unpopular skills. This keeps the skill base streamlined and efficient during continuous adversarial evolution, thereby ensuring that the overall availability of automatically accumulated reusable reasoning skills steadily increases with the number of game rounds without updating model parameters. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating an open-source intelligence reasoning skill automatic generation auxiliary decision-making method based on multi-agent self-play provided in the first embodiment of the present invention. Figure 2 This is a schematic diagram of a module of an open-source intelligence reasoning skill automatic generation auxiliary decision-making device based on multi-agent self-play provided in the second embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0017] This invention discloses an open-source intelligence reasoning skill automatic generation auxiliary decision-making method, device and equipment based on multi-agent self-play, which aims to solve the problem that the overall usability of reasoning skills accumulated in military intelligence self-play without objective adjudication benchmarks degrades with each round of play.

[0018] Please see Figure 1 The first embodiment of the present invention provides an open-source intelligence reasoning skill automatic generation auxiliary decision-making method based on multi-agent self-play, which can be executed by a military intelligence reasoning skill automatic generation device (hereinafter referred to as the generation device or system), specifically, by one or more processors within the generation device, to at least implement the following steps: S101: Receive multi-source military intelligence, call a large model to extract key elements, and label each piece of intelligence with confidence level, source type and timeliness, and integrate the key elements and labels into a structured situational knowledge summary; The system first receives raw intelligence data from multiple sources, including reconnaissance, signals, images, and historical archives. This type of data is typically in natural language text format, highly specialized, and fragmented, making it difficult for subsequent intelligent agents to directly utilize. To obtain directly usable structured knowledge, the system invokes a large language model to extract entities and link relationships from the raw intelligence data. Specifically, by constructing analytical prompts, the large language model is guided to identify and extract key elements such as time, location, entity name, activity events, technical parameters, and information sources from each intelligence report, and outputs them in a unified structured field. These key elements include at least the entities identified from the intelligence. For example, for an intelligence report describing a unit's communication activities, the large language model can extract parameters such as the time of the activity, the units involved, the type of activity, and the frequency band used for communication. For an image-based intelligence report describing the target aggregation situation in a certain area, the corresponding location, entities, and situation description can be extracted. For historical intelligence reports without a specific time or location, the corresponding fields are left blank to ensure the consistency of the intelligence structure.

[0019] After extracting key elements, the system further labels each piece of intelligence with its confidence level, source type, and timeliness. The confidence level is determined based on the source type and the specificity of the content: when the source is a signal and the content is specific and contains verifiable technical details, its confidence level is marked as high; when the source is an image and the description is clear, containing clear location and situational information, its confidence level is marked as medium; when the source is a historical archive and only constitutes indirect evidence rather than direct evidence, its confidence level is marked as low. This categorizes all intelligence pieces into three levels: high, medium, and low. Simultaneously, the system records the source type of each piece of intelligence. Regarding timeliness, the system uses the interval between the intelligence time and the current time as the basis for labeling, and pre-sets a first timeliness threshold and a second timeliness threshold, with the second timeliness threshold being greater than the first timeliness threshold: when the interval does not exceed the first timeliness threshold, the intelligence is labeled as fresh; when the interval is between the first and second timeliness thresholds, it is labeled as normal; when the interval exceeds the second timeliness threshold, or when the intelligence does not have an identifiable intelligence time, it is labeled as outdated.

[0020] Finally, the system uses each intelligence report as a basic unit, fusing the extracted key elements of each report with their corresponding confidence levels, source types, and timeliness labels to form a complete activity record. It also deduplicates and merges entities with the same name from multiple intelligence reports, recording all source types involved in each entity, and ultimately aggregating them into a unified, structured situational knowledge summary. This situational knowledge summary, in structured data format, carries all the above processing results, preserving the elements and labeling information of each intelligence report while merging entities across intelligence reports. This provides a unified input that can be directly invoked for subsequent steps, such as the construction of a hierarchical skill library and reasoning and adversarial interactions between two agents.

[0021] S102, construct a hierarchical skill base based on the situation knowledge summary and initialize dual agents; the hierarchical skill base includes global basic skills that remain unchanged across scenarios, scenario-specific skills automatically generated based on entities in the situation knowledge summary, and adversarial skills used to generate challenge tasks; the dual agents include a reasoner agent loaded with global basic skills and scenario-specific skills, and a challenger agent loaded with adversarial skills; The system constructs a hierarchical skill library based on situational knowledge summaries, and divides the skill set into three layers: global basic skills, scenario-specific skills, and adversarial skills. Skills in each layer are stored in a unified structured file format. Each skill file contains metadata such as skill name, function description, skill type, and the type of intelligence source it depends on. It is followed by specific rule content expressed in the form of "when a certain condition is met, a certain judgment is made". This allows each skill to be directly read and called by the agent, and also facilitates subsequent updates and maintenance.

[0022] Among them, the global basic skills are general reasoning rules that remain unchanged across scenarios and are not dependent on specific situations. They are used to characterize the judgment logic that is generally applicable in intelligence analysis. For example, the rule that the overall confidence level is increased by one level after multiple low-confidence intelligences from different sources are cross-verified, and the rule that conservative judgments are prioritized and missing information is marked when key information is missing or existing intelligence is insufficient to support a clear conclusion. These skills remain unchanged in the subsequent confrontation evolution process, forming the stable foundation of the entire layered skill library. Scenario-specific skills are specialized knowledge closely related to the current situation. The system automatically generates corresponding initial scenario-specific skills based on the entities extracted from the situation knowledge summary. For example, for a certain unit entity appearing in the summary, the system automatically generates a scenario-specific skill to identify the unit based on communication characteristics by combining its characteristic parameters such as communication frequency band and signal strength reflected in signal intelligence; or, by combining the activity precursors reflected in historical intelligence of the entity, the system automatically generates a scenario-specific skill to infer the timing of its subsequent maneuvers based on activities such as resupply. These skills are continuously updated as the confrontation evolves. Adversarial skills are strategic knowledge used to generate challenging tasks for the challenger agent. Examples include strategies for generating new intelligence of unknown origin that contradicts existing high-confidence intelligence to test the reasoner's resistance to interference, and strategies for setting time limits to test the reasoner's ability to make judgments under time pressure. These skills are also constantly updated as the adversarial process evolves. The combination of these global basic skills, scenario-specific skills, and adversarial skills constitutes a complete hierarchical skill library.

[0023] After the hierarchical skill library is constructed, the system further initializes the reasoner agent and the challenger agent. For the reasoner agent, the system sequentially concatenates the file contents of global basic skills and scenario-specific skills, injects them into the system prompts of the large language model, and uses this as the skill set upon which its analysis and reasoning are based. At the same time, the situation knowledge summary is also input as context, enabling the reasoner agent to make judgments on subsequent analysis tasks based on general reasoning rules and current situation-specific knowledge. For the challenger agent, the system loads the file contents of the adversarial skill layer and injects its system prompts, enabling the challenger agent to generate targeted intelligence analysis challenge tasks based on the adversarial skills and the situation knowledge summary. Thus, together with the reasoner agent, they form a sustainable and co-evolving dual-agent structure.

[0024] S103, using the hierarchical skill library and situational knowledge summary as input, iteratively execute adversarial self-play, wherein the challenger agent generates an analysis task with verification details according to the current difficulty level, and the inferrer agent calls the hierarchical skill library to solve the task. The difficulty level is adjusted according to the inferrer agent's historical pass rate. The missed detection cost weight and false positive cost weight are dynamically calculated based on the key parameters in the situational knowledge summary, and the weighted pass rate is calculated accordingly to determine the success or failure of the task. Failed tasks are used to update the scenario-specific skills, and successful tasks are used to update the adversarial skills, while the global basic skills remain unchanged. Based on situational knowledge summaries, hierarchical skill libraries, and dual agents, adversarial self-play is iteratively executed. Each iteration goes through stages such as difficulty scheduling, adversarial task generation, reasoning and solving, asymmetric cost adjudication, and hierarchical skill evolution. After each round, the hierarchical skill library is maintained, and the updated skill library is brought into the next round, thus forming a closed loop of continuous collaborative evolution between the reasoner agent and the challenger agent.

[0025] In the difficulty scheduling phase, the system maintains the pass rate of the reasoning agent in the most recent rounds of tasks and determines the difficulty level of the current round of tasks accordingly: when the pass rate is higher than a first preset threshold, the difficulty level is increased by one level; when the pass rate is lower than a second preset threshold, the difficulty level is decreased by one level; when the pass rate is between the two, the difficulty level remains unchanged, wherein the first preset threshold is greater than the second preset threshold, which in one embodiment can be 70% and 40% respectively. The difficulty level can be set to three levels: easy, medium, and hard, with the upper limit of the difficulty level set as hard to avoid the task difficulty from escalating indefinitely. To prevent the reasoning agent from being exposed to tasks beyond its current capabilities for a long time and causing evolutionary instability, the system also sets a difficulty rollback protection: when the number of consecutive rounds in which the reasoning agent fails at the same difficulty level reaches a preset number of rounds (in one embodiment, this can be three rounds), the system automatically rolls back the difficulty level by one level and records the rollback event. For example, if the average pass rate of the last three rounds is between the two thresholds mentioned above, the difficulty of this round remains unchanged; if the pass rate of a subsequent round increases significantly and exceeds the first preset threshold, the difficulty of the next round will be increased by one level; and if the reasoning agent fails consecutively at the difficult level for a preset number of rounds, the difficulty will automatically revert to medium.

[0026] In the adversarial task generation and reasoning solution stage, the challenger agent, based on the target difficulty level of this round, combined with the situational knowledge summary and its loaded adversarial skill set, calls a large language model to generate one or more intelligence analysis tasks. Each task is accompanied by a corresponding set of verification rules, which are several checks required to determine whether the task has been correctly completed. Subsequently, the reasoner agent loads global basic skills and scenario-specific skills (excluding adversarial skills) from the hierarchical skill library, injects their contents into system prompts, and solves each analysis task in the context of the situational knowledge summary, outputting a solution conclusion that includes the judgment conclusion, the evidence on which it is based, and the reasoning process.

[0027] In the asymmetric cost adjudication phase, the system evaluates the solution conclusions of the reasoning agent, distinguishing between two error types: missed detections and false alarms. Missed detections refer to the failure to identify actual threats, while false alarms refer to issuing alerts for threats that do not exist. Considering that the consequences of these two types of errors are not equitable under different situations, the system uses the timeliness, threat level, and decision urgency in the situation knowledge summary as key parameters, dynamically calculating the cost weights for missed detections and false alarms using the following formula:

[0028]

[0029] In the formula, and These are the underreporting cost weight and the false positive cost weight, respectively; and These are the base weights for false negatives and false positives, respectively. Greater than In one implementation, these can be taken as five and one, respectively; , These are the base weights for underreporting cost and the base weights for false positive cost, respectively. Greater than ; , , These are the timeliness coefficient, threat level coefficient, and decision urgency coefficient, respectively, representing the weighting of the cost of underreporting. , , These are the timeliness coefficient, threat level coefficient, and decision urgency coefficient for the false alarm cost weighting; among them, and The shelf life decreases as the product becomes more stale. and All decrease as the threat level decreases from high to low; however, regarding the urgency of decision-making, Increases with the urgency of the decision. The weight of the cost of underreporting decreases as the urgency of the decision increases, thus the more urgent the decision and the higher the threat, the greater the relative weight of the cost of misreporting compared to the cost of misreporting. For example, when the situation knowledge summary shows that the timeliness is fresh, the threat level is high, and the urgency of the decision is urgent, a larger weight of the cost of underreporting and a smaller weight of the cost of misreporting can be obtained accordingly.

[0030] After obtaining the above weights, the system sets the total number of verification rules for the current task to N, the number of verification rules passed by the inferring agent to M, and calculates the weighted pass rate according to the following formula, distinguishing error types: When it is determined to be a missed report:

[0031] When determined to be a false alarm:

[0032] When there are no errors:

[0033] When the weighted pass rate reaches a preset threshold (which may be 0.8 in one implementation), the task is declared a success; otherwise, it is declared a failure. Thus, under urgent and high-threat circumstances, false negatives will cause the weighted pass rate to drop significantly, making it easier for the task to be judged as a failure, thereby exerting much greater counter-pressure on false negatives than on false positives.

[0034] In the hierarchical skill evolution phase, the system updates the skill base differently based on the adjudication results: For tasks adjudicated as failures, the system routes them to the reasoner's skill update process, calls the large language model to analyze the reasons for the task's failure, summarizes the knowledge or rules missing by the reasoner, generates a new scenario-specific skill, and adds it to the scenario-specific skill set in the form of a structured file; for tasks adjudicated as successes, the system routes them to the challenger's skill update process, calls the large language model to analyze the reasons for the task's success, generates a more challenging adversarial skill, and adds it to the adversarial skill set; throughout the above update process, the global basic skills remain unchanged. This allows general experience and specialized experience, reasoning ability and adversarial ability to evolve independently, avoiding mutual interference and mixed accumulation.

[0035] In the skill maintenance phase, after each iteration, the system sequentially performs structural redundancy detection, utility redundancy assessment, and elimination of unpopular skills in the hierarchical skill library. In structural redundancy detection, the system calculates the semantic similarity between the applicable conditions of each pair of skills. When this similarity exceeds a preset similarity threshold (which can be 0.8 in one implementation) and the conclusions produced by the two skills are consistent, they are identified as a redundant skill pair. The skill with the higher success rate is retained, while the other is deleted. For example, two skills that identify the same unit's communications using similar frequency bands are considered redundant and merged because their applicable conditions are highly similar and their conclusions are consistent. In utility redundancy assessment, the system records the cumulative number of calls and the number of successful calls for each skill. The ratio of successful calls to cumulative calls is used as the success rate of the skill. When the cumulative number of calls for a skill reaches a preset threshold (which can be ten in one implementation) and its success rate is lower than a preset success rate threshold (which can be 30% in one implementation), it is identified as an inefficient skill and removed. In the process of eliminating less popular skills, when a skill remains unused for a certain number of consecutive rounds (in one implementation, this could be ten rounds), it is marked as a less popular skill. If, subsequently, the number of consecutive rounds unused for that skill reaches a second preset number (in one implementation, this could be five rounds), it is removed from the list. To support this maintenance, each skill carries metadata that is automatically updated by the system, including the cumulative number of calls, the number of successful calls, the success rate, and the round number of the last call. After maintenance is completed, the updated and streamlined skill set is carried over to the next iteration, and the entire process of difficulty scheduling to skill maintenance is repeated. This ensures that while continuously maintaining resistance to pressure, the versatility, stability, and overall usability of the accumulated skill set steadily improve with each iteration.

[0036] S104, based on the solution conclusion and decision result of the reasoner agent, output the auxiliary decision result.

[0037] Based on the situation knowledge summary, the solution conclusions generated by the reasoning agent, and the adjudication results, auxiliary decision-making results for commanders are assembled and output. These results include at least the optimal judgment conclusion and its basis, cost estimates, and alternative degrading paths in the case of intelligence source failure. This transforms the reasoning ability accumulated and verified during the self-play process into decision support information that can be directly used by commanders.

[0038] In outputting the optimal judgment conclusion, the system extracts the judgment conclusion and its confidence level obtained by the reasoning agent from the solution conclusion that has been judged as successful. It also parses the skill chain invoked during the reasoning process of the solution conclusion, along with the corresponding supporting information. The skill chain is an ordered set of skills invoked sequentially by the reasoner when solving the task. The supporting information consists of the intelligence elements supporting the judgment conclusion, along with their corresponding confidence levels, source types, and other information. This clearly demonstrates the complete reasoning path from the original intelligence to the final conclusion, enabling commanders to not only know the conclusion itself but also trace the skills and intelligence upon which it is based. For example, for the judgment conclusion of a unit's maneuver to a certain area, the system can simultaneously provide the cross-verification-type global basic skills invoked, the unit's signal feature recognition-type scene-specific skills, and the signal and image intelligence elements supporting the conclusion, demonstrating that the overall confidence level is improved after two intelligences from different sources mutually corroborate each other.

[0039] Regarding output cost estimation, the system calculates and presents the underreporting risk and false alarm risk under this assessment conclusion based on the weighted pass rate and error type determination using the following formula, and gives them as percentages:

[0040]

[0041] In the formula, the weighted pass rate is obtained by a decision, and the underreporting cost weight and the false positive cost weight are the dynamically calculated underreporting cost weights. Weighting of false alarm costs Furthermore, the former is greater than the latter, thus ensuring that the output risk quantification value also reflects the asymmetric costs of underreporting and false alarms, facilitating commanders' assessment of the potential consequences of taking action under the current conclusion. The underreporting risk and false alarm risk are output together with the assessment conclusion, allowing commanders to intuitively understand the risk level of the conclusion in the two types of error directions and their relative magnitude.

[0042] Regarding the output of alternative degradation paths, considering that an intelligence source may become ineffective due to interference or other reasons in a real-world environment, the system further provides alternative solutions for intelligence source failure scenarios. Specifically, when an intelligence source is determined to be unavailable, the system searches for alternative skills with similar functions but not dependent on that intelligence source in the scenario-specific skills. When multiple candidate alternative skills exist, the skill with the highest similarity to the skill chain called by the current judgment conclusion that depends on the unavailable intelligence source is selected as the alternative skill. Subsequently, the system reduces the confidence level of the judgment conclusion to the expected confidence level according to a preset reduction coefficient, and amplifies the missed detection risk to the expected missed detection risk according to a preset amplification coefficient. Then, the alternative skill, the expected confidence level, and the expected missed detection risk are output together as alternative degradation paths. For example, when a certain type of intelligence source upon which the original assessment conclusion relies becomes unavailable due to disruption, the system can recommend using an alternative skill with similar functionality but relying on a different type of intelligence source. It will also provide a reduced expected confidence level and an amplified expected underreporting risk, thus alerting commanders to the decreased reliability of the conclusion and the increased underreporting risk when intelligence sources are unavailable. This provides risk-alert-based decision support even when some intelligence sources are unavailable. Finally, the system assembles the optimal assessment conclusion, its skill chain, supporting documentation, cost estimates, and alternative downgrade paths into a structured decision support output in a unified format for direct use by commanders.

[0043] Please see Figure 2 The second embodiment of the present invention provides an open-source intelligence reasoning skill automatic generation auxiliary decision-making device based on multi-agent self-play, comprising: The intelligence preprocessing module 201 is used to receive multi-source military intelligence, call a large model to extract key elements from it, and label each piece of intelligence with confidence level, source type and timeliness, and integrate the key elements and labels into a structured situational knowledge summary. Initialization module 202 is used to construct a hierarchical skill library based on the situation knowledge summary and initialize dual agents; the hierarchical skill library includes global basic skills that remain unchanged across scenarios, scenario-specific skills automatically generated based on entities in the situation knowledge summary, and adversarial skills for generating challenge tasks; the dual agents include a reasoner agent loaded with global basic skills and scenario-specific skills, and a challenger agent loaded with adversarial skills; The self-play loop module 203 is used to iteratively execute adversarial self-play using the hierarchical skill library and situational knowledge summary as input. Specifically, the challenger agent generates an analysis task with verification details based on the current difficulty level, and the inferrer agent calls the hierarchical skill library to solve the task. The difficulty level is adjusted based on the inferrer agent's historical pass rate. The missed detection cost weight and false positive cost weight are dynamically calculated based on key parameters in the situational knowledge summary, and a weighted pass rate is calculated to determine the success or failure of the task. Failed tasks are used to update the scenario-specific skills, and successful tasks are used to update the adversarial skills, while the global basic skills remain unchanged. The decision output module 204 is used to output auxiliary decision results based on the solution conclusion and decision result of the reasoner agent.

[0044] The third embodiment of the present invention provides an automatic generation auxiliary decision-making device based on open-source intelligence reasoning skills of multi-agent self-play, including a memory and a processor. The memory stores a computer program, which can be executed by the processor to realize the automatic generation auxiliary decision-making method based on open-source intelligence reasoning skills of multi-agent self-play as described in any of the above.

[0045] The fourth embodiment of the present invention provides a computer-readable storage medium storing a computer program, which can be executed by the processor of the device where the computer-readable storage medium is located, to realize the open-source intelligence reasoning skill automatic generation auxiliary decision-making method based on multi-agent self-play as described in any of the above.

[0046] Exemplary examples show that the computer program described in the third and fourth embodiments of the present invention can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in implementing an open-source intelligence reasoning skill automatic generation auxiliary decision-making device based on multi-agent self-play. For example, the apparatus described in the second embodiment of the present invention.

[0047] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the aforementioned open-source intelligence reasoning skill automatic generation auxiliary decision-making method based on multi-agent self-play, utilizing various interfaces and lines to connect the various parts of the method.

[0048] The memory can be used to store the computer program and / or modules. The processor, by running or executing the computer program and / or modules stored in the memory, and by calling the data stored in the memory, implements various functions of an open-source intelligence reasoning skill-based automatic generation auxiliary decision-making method based on multi-agent self-play. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, text conversion function, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, text message data, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0049] If the implemented module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0050] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0051] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for automatically generating auxiliary decision-making based on open-source intelligence reasoning skills using multi-agent self-play, characterized in that, include: Receive multi-source military intelligence, call a large model to extract key elements, and label each piece of intelligence with confidence level, source type and timeliness, and integrate the key elements and labels into a structured situational knowledge summary; A hierarchical skill base is constructed based on the situation knowledge summary, and a dual agent is initialized. The hierarchical skill base includes global basic skills that remain unchanged across scenarios, scenario-specific skills automatically generated based on entities in the situation knowledge summary, and adversarial skills for generating challenge tasks. The dual agent includes a reasoner agent loaded with global basic skills and scenario-specific skills, and a challenger agent loaded with adversarial skills. Using the hierarchical skill library and situational knowledge summary as input, adversarial self-play is iteratively executed. The challenger agent generates an analysis task with verification details based on the current difficulty level, and the inferrer agent calls the hierarchical skill library to solve it. The difficulty level is adjusted based on the inferrer agent's historical pass rate. The missed detection cost weight and false positive cost weight are dynamically calculated based on key parameters in the situational knowledge summary, and a weighted pass rate is calculated to determine the success or failure of the task. Specifically, the missed detection cost weight is calculated using the timeliness, threat level, and decision urgency in the situational knowledge summary as key parameters, according to the following formula. Weighting of false alarm costs : In the formula, , These are the base weights for underreporting cost and the base weights for false positive cost, respectively. Greater than ; , , These are the timeliness coefficient, threat level coefficient, and decision urgency coefficient, respectively, representing the weighting of the cost of underreporting. , , These are the timeliness coefficient, threat level coefficient, and decision urgency coefficient for the false alarm cost weighting; Let N be the total number of verification rules for the current task, and M be the number of verification rules passed by the inferring agent. Distinguishing between two error types—false positives and false negatives—the weighted pass rate is calculated using the following formula: When it is determined to be a missed report: When determined to be a false alarm: When there are no errors: When the weighted pass rate reaches a preset threshold, the task is judged as successful; otherwise, it is judged as a failure. Failed tasks are used to update the scene-specific skills, and successful tasks are used to update the adversarial skills, while the global basic skills remain unchanged. Based on the solution conclusions and decision results of the reasoner agent, auxiliary decision results are output.

2. The method for automatically generating auxiliary decision-making based on open-source intelligence reasoning skills using multi-agent self-play as described in claim 1, characterized in that, The description includes the confidence level, source type, and timeliness of each intelligence report. Based on the source type of the intelligence and the specificity of its content, the confidence level is marked as high, medium, and low; the source type of the intelligence is marked; and the timeliness is marked according to the interval between the intelligence time and the current time. The interval is not more than a first timeliness threshold and is marked as fresh, between the first timeliness threshold and a second timeliness threshold and is marked as normal, and the interval is more than the second timeliness threshold or the intelligence time does not exist and is marked as outdated. The second timeliness threshold is greater than the first timeliness threshold.

3. The method for automatically generating auxiliary decision-making based on open-source intelligence reasoning skills using multi-agent self-play as described in claim 1, characterized in that, The global basic skills, scenario-specific skills, and adversarial skills are all stored in structured files; among them, the global basic skills are general reasoning rules that remain unchanged across scenarios, including rules that increase confidence by cross-verifying multiple low-confidence intelligences, and rules that prioritize conservative judgments when information is missing; The scenario-specific skills are automatically generated based on the entities in the situation knowledge summary, including the signal feature recognition rules and troop movement patterns of specific units. The aforementioned adversarial skills are strategies for generating challenging tasks, including generating new intelligence that contradicts existing intelligence and requiring analysis to be completed within a limited time.

4. The method for automatically generating auxiliary decision-making based on open-source intelligence reasoning skills using multi-agent self-play as described in claim 1, characterized in that, The difficulty level is adjusted based on the historical pass rate of the reasoning agent, specifically as follows: Maintain the pass rate of the reasoning agent in the most recent rounds of the task. When the pass rate is higher than a first preset threshold, increase the difficulty level by one level; when it is lower than a second preset threshold, decrease the difficulty level by one level; otherwise, keep it unchanged. The first preset threshold is greater than the second preset threshold. Set an upper bound for the difficulty level. When the number of consecutive rounds in which the reasoning agent fails at the same difficulty level reaches a preset number of rounds, reduce the difficulty level by one level.

5. The method for automatically generating auxiliary decision-making based on open-source intelligence reasoning skills using multi-agent self-play as described in claim 1, characterized in that, Also includes: After each iteration, skill maintenance is performed on the hierarchical skill library, and the updated hierarchical skill library is brought into the next iteration.

6. The method for automatically generating auxiliary decision-making based on open-source intelligence reasoning skills using multi-agent self-play as described in claim 5, characterized in that, The hierarchical skill base is maintained by performing the following steps: structural redundancy detection, utility redundancy assessment, and elimination of unpopular skills in sequence. The structural redundancy detection is as follows: calculate the semantic similarity between the applicable conditions of each pair of skills; when the semantic similarity is greater than a preset similarity threshold and the conclusions produced by the two skills are consistent, retain the skill with the higher success rate and delete the other skill. The utility redundancy assessment is as follows: for each skill record, the cumulative number of calls and the number of successful calls are recorded, and the ratio of the number of successful calls to the cumulative number of calls is taken as the success rate of the skill. When the cumulative number of calls of a skill reaches a preset threshold and its success rate is lower than a preset success rate threshold, the skill is judged as an inefficient skill and is removed. The elimination of unpopular skills is as follows: when a skill is not used for a certain number of consecutive rounds, the skill is marked as an unpopular skill; when the number of consecutive rounds that the unpopular skill is not used reaches a second preset number of rounds, the unpopular skill is removed.

7. The method for automatically generating auxiliary decision-making based on open-source intelligence reasoning skills using multi-agent self-play as described in claim 1, characterized in that, Also includes: Output alternative degradation paths: When an intelligence source is determined to be unavailable, search for alternative skills with similar functions that do not depend on the intelligence source in the scene-specific skills. When there are multiple alternative skills, select the alternative skill with the highest similarity to the currently invoked skill. The confidence level of the solution is reduced to the expected confidence level by a preset reduction factor, and the underreporting risk is amplified to the expected underreporting risk by a preset amplification factor; the alternative skill, the expected confidence level, and the expected underreporting risk are output as the alternative downgrade path.

8. An open-source intelligence reasoning skill automatic generation auxiliary decision-making device based on multi-agent self-play, characterized in that, include: The intelligence preprocessing module is used to receive multi-source military intelligence, call a large model to extract key elements, and label each piece of intelligence with confidence level, source type and timeliness, and integrate the key elements and labels into a structured situational knowledge summary. An initialization module is used to construct a hierarchical skill library based on the situation knowledge summary and initialize a dual agent; the hierarchical skill library includes global basic skills that remain unchanged across scenarios, scenario-specific skills automatically generated based on entities in the situation knowledge summary, and adversarial skills for generating challenge tasks; the dual agent includes a reasoner agent loaded with global basic skills and scenario-specific skills, and a challenger agent loaded with adversarial skills; The self-play loop module is used to iteratively execute adversarial self-play using the hierarchical skill library and situational knowledge summary as input. The challenger agent generates an analysis task with verification details according to the current difficulty level, and the inferrer agent calls the hierarchical skill library to solve it. The difficulty level is adjusted based on the inferrer agent's historical pass rate. The module dynamically calculates the missed detection cost weight and false positive cost weight based on key parameters in the situational knowledge summary, and calculates a weighted pass rate to determine the success or failure of the task. Specifically, the missed detection cost weight is calculated using the timeliness, threat level, and decision urgency in the situational knowledge summary as key parameters, according to the following formula. Weighting of false alarm costs : In the formula, , These are the base weights for underreporting cost and the base weights for false positive cost, respectively. Greater than ; , , These are the timeliness coefficient, threat level coefficient, and decision urgency coefficient, respectively, representing the weighting of the cost of underreporting. , , These are the timeliness coefficient, threat level coefficient, and decision urgency coefficient for the false alarm cost weighting; Let N be the total number of verification rules for the current task, and M be the number of verification rules passed by the inferring agent. Distinguishing between two error types—false positives and false negatives—the weighted pass rate is calculated using the following formula: When it is determined to be a missed report: When determined to be a false alarm: When there are no errors: When the weighted pass rate reaches a preset threshold, the task is judged as successful; otherwise, it is judged as a failure. Failed tasks are used to update the scene-specific skills, and successful tasks are used to update the adversarial skills, while the global basic skills remain unchanged. The decision output module is used to output auxiliary decision results based on the solution conclusions and decision results of the reasoner agent.

9. A decision-making auxiliary device for automatically generating open-source intelligence reasoning skills based on multi-agent self-play, characterized in that, The system includes a memory and a processor. The memory stores a computer program that can be executed by the processor to implement an open-source intelligence reasoning skill automatic generation auxiliary decision-making method based on multi-agent self-play, as described in any one of claims 1 to 7.

Images