Manual wargame operation data synchronization pushing method, system and device based on large model

Abstract

The application discloses a hand-to-hand combat game operation data synchronization pushing method, system and equipment based on a large model, and belongs to the technical field of war game deduction data processing. Through collection of voice data and chessboard situation image data in a deduction site, voice recognition and military semantic analysis are performed by using a multi-modal large model, and situation incremental data is obtained through change detection and chess piece recognition; the voice semantic result and visual situation data are time-aligned, consistency-verified, confidence-evaluated and rule-constrained-verified, and contradiction is disambiguated or missing information is completed; finally, structured operation instructions are generated and pushed to a computer war game system in real time. The above scheme realizes second-level data synchronization, significantly improves operation semantic capture capability and recognition accuracy, and has the advantages of non-invasion, high robustness and strong universality.

Description

Technical Field

[0001] This invention belongs to the field of wargaming data processing technology, and specifically discloses a method, system and device for synchronously pushing manual wargaming operation data based on a large model. Background Technology

[0002] Wargaming is an important tool for military training and operational research, and it comes in two basic forms: manual wargaming and computer-based wargaming. Manual wargaming uses physical boards, pieces, and dice, and involves face-to-face competition between participants, offering advantages such as high interactivity, flexibility, and immersion. Computer-based wargaming runs on a software platform, boasting strong data processing capabilities, precise situational awareness, and the ability to replay and analyze data. In actual training, both modes often need to be used in conjunction, with computer-based wargaming providing automatic adjudication functions for manual wargaming.

[0003] However, data synchronization between manual and computer-based wargames has always been a key bottleneck restricting their collaborative application. Existing technologies mainly suffer from the following solutions and their shortcomings:

[0004] The first approach is a data synchronization method based on manual input. Simulators perform actions on a manual chessboard, while recorders observe and record these actions. The data is then entered line by line into a computer terminal, and the computer wargaming system receives the data and updates the situation display. This approach suffers from significant time lag; there is typically a delay of several minutes or even tens of minutes between the simulation actions and data entry, making real-time synchronization impossible. Furthermore, it is limited by the recorders' comprehension and recording speed, resulting in the loss of a large amount of tactical semantic information.

[0005] The second approach is a data synchronization method based on traditional image recognition. This method uses a camera to capture images of the chessboard, employs traditional image recognition algorithms to detect changes in the positions of the pieces, and then pushes the recognition results to the system after manual confirmation. However, this approach is affected by factors such as changes in lighting, piece occlusion, and viewing angle deviations, resulting in an accuracy rate of only 70%-85% for identifying piece types and positions. The false recognition rate is relatively high, and due to limitations in algorithm processing speed and recognition accuracy, it typically uses long acquisition intervals, making it difficult to capture rapid operational changes.

[0006] The third approach is based on data synchronization using RFID or sensor tags. This method embeds an RFID chip or other electronic tag into each piece and deploys a reading antenna array beneath the board, automatically acquiring position information by sensing piece movement. However, this approach suffers from signal interference and cross-reading issues when pieces are densely packed, limiting positioning accuracy. Furthermore, it requires specially designed pieces and an antenna array board, resulting in high coupling with existing manual wargame practices and a lack of flexible adaptation mechanisms. Additionally, this approach only detects changes in piece position and completely fails to perceive the tactical intentions behind the actions.

[0007] The fourth approach lacks intelligent information fusion capabilities. Existing technologies typically use only a single information source (purely manual recording, pure images, or pure RFID), lacking the ability to fuse and verify multiple information sources. When a single information source is faulty or missing, the system cannot supplement and correct it using other information sources, leading to unstable data quality.

[0008] In summary, existing technologies have significant shortcomings in terms of real-time data synchronization, preservation of operational semantic information, recognition accuracy, system flexibility, and multi-source information fusion, which seriously restrict the digital utilization of manual wargaming results and the improvement of wargaming efficiency. Summary of the Invention

[0009] To address the aforementioned issues—namely, the low real-time synchronization of manual wargame operation data, missing semantic information, insufficient recognition accuracy, poor versatility, and lack of multimodal information fusion verification in existing technologies—this invention provides a method, system, and device for synchronizing and pushing manual wargame operation data based on a large model. This achieves efficient, accurate, and non-intrusive synchronization of manual wargame operation data to a computer wargame system.

[0010] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for synchronously pushing manual wargame operation data based on a large model, the method comprising: Collect voice data and chessboard situation image data from the manual war game simulation. The speech data is subjected to speech recognition and military semantic analysis using a multimodal large model to obtain the speech semantic analysis results; The chessboard situation image data is subjected to chessboard change detection. When a valid change is detected, chess piece detection, classification and coordinate mapping processing are performed on the changed area to obtain situation incremental data. The speech semantic parsing results and the situational incremental data are time-aligned and checked for consistency. When the checks are inconsistent, confidence assessment and rule constraint verification are performed, and contradiction disambiguation is performed based on the assessment and verification results. When any modal information of the speech semantic parsing results or the situational incremental data is missing, information is completed based on the historical context. Based on the results of conflict resolution or information completion, structured wargame operation instructions are generated and pushed to the computer wargame system via a data interface.

[0011] Optionally, before performing speech recognition and military semantic analysis on the speech data based on a pre-established multimodal large model, the method further includes: Speech activity detection is performed on the speech data to identify valid speech segments and filter out invalid audio segments.

[0012] Optionally, the step of using a multimodal large model to perform speech recognition and military semantic analysis on the speech data to obtain the speech semantic analysis results includes: The speech data is processed using a streaming inference mode to perform speech recognition and output the corresponding recognized text. The identified text is processed by military domain intent classification, entity extraction, and relation extraction. Based on the intent classification results, entity extraction results, and relation extraction results, corresponding speech and semantic parsing results are generated.

[0013] Optionally, the chessboard change detection of the chessboard situation image data includes: Perform a difference analysis between the current chessboard situation image frame and the reference image frame, and use an adaptive background modeling method to distinguish between foreground changes caused by piece movement and background changes caused by illumination fluctuations.

[0014] Optionally, the step of performing piece detection, classification, and coordinate mapping processing on the changed area to obtain incremental situational data includes: Based on the detected changes in the region of change, the position of the chess piece is located and the bounding box of the chess piece is generated. The chess piece images are classified based on their bounding boxes, and the type, faction affiliation, and number of each chess piece are output. Map the pixel coordinates of the categorized chess pieces to chessboard grid coordinates; The current identification results are compared with historical situation snapshots to calculate the situation increment data.

[0015] Optionally, the step of performing time alignment and consistency checks between the speech semantic parsing results and the situational incremental data includes: The timestamps corresponding to the speech semantic parsing results are matched with the timestamps corresponding to the situational incremental data to establish a temporal correspondence between speech events and visual events. The operation subject and target position in the speech semantic parsing results are compared with the chess piece change information in the situational incremental data to perform a consistency check.

[0016] Optionally, the confidence assessment and rule constraint verification include: Calculate the confidence score of the speech semantic parsing result and the confidence score of the situational incremental data, respectively; The wargaming rules knowledge base is invoked to perform compliance verification on the speech semantic parsing results and the situational incremental data.

[0017] Optionally, calculating the confidence score of the speech semantic parsing result and the confidence score of the situational increment data respectively includes: The confidence score of the speech semantic parsing result is calculated based on the acoustic confidence of the speech recognition model and the logical consistency of the semantic parsing model. The confidence score of the situational incremental data is calculated based on the detection score of the chess piece detection model and the grid matching degree of the coordinate mapping. The compliance verification by calling the wargaming rules knowledge base includes: Based on the speech semantic parsing results and the operation type, subject, and target location involved in the situational incremental data, the corresponding movement distance limit, terrain accessibility, and attack range constraint are queried in the wargaming rule knowledge base to determine whether the two meet the preset wargaming rules.

[0018] Secondly, the present invention provides a system for synchronously pushing manual wargame operation data based on a large model, comprising: The data acquisition module is used to collect voice data and chessboard situation image data from the manual war game simulation. The intelligent processing module is used to perform speech recognition and military semantic analysis on the speech data using a multimodal large model to obtain speech semantic analysis results; and to perform chessboard change detection on the chessboard situation image data. When a valid change is detected, chess piece detection, classification and coordinate mapping processing are performed on the changed area to obtain situation incremental data. The multimodal fusion reasoning module is used to perform time alignment and consistency checks on the speech semantic parsing results and the situational incremental data; when the checks are inconsistent, confidence assessment and rule constraint verification are performed, and contradiction disambiguation is performed based on the assessment and verification results; when any modal information is missing, information is completed based on the historical context; and structured wargame operation instructions are generated based on the results of contradiction disambiguation or information completion. The data push module is used to push the structured wargame operation instructions to the computer wargame system via a data interface.

[0019] Thirdly, the present invention provides an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method described in any one of the first aspects.

[0020] Compared with the closest existing technology, the present invention has the following advantages: This application proposes a method, system, and device for synchronizing and pushing manual wargame operation data based on a large model. It utilizes streaming audio processing and incremental image analysis techniques, employing a streaming inference mode for speech recognition to identify the voice commands of the players in real time. A change-detection-based incremental analysis strategy is used for visual recognition, re-identifying only the areas of the chessboard that have changed, avoiding repetitive processing of the entire map. This achieves end-to-end data synchronization within seconds.

[0021] This invention utilizes a multimodal large model to perform military semantic analysis on the natural speech of simulation personnel, extracting tactical intentions, entity relationships, and operational semantics, and transforming unstructured speech commands into structured wargame operation commands, effectively preserving the tactical semantic information in manual simulations.

[0022] This invention utilizes the powerful visual understanding capabilities of a multimodal large model for chess piece detection and classification, exhibiting good robustness to illumination, occlusion handling, and small-sample generalization ability. Furthermore, cross-validation of speech and visual information further enhances the accuracy and reliability of the recognition.

[0023] This invention employs a non-invasive data acquisition method, requiring only the deployment of microphones and cameras at the simulation site to operate, without any physical modification to the chess pieces or board. Furthermore, its large-model-based visual recognition capability exhibits strong generalization, requiring only a small number of samples to adapt when changing chess piece types, and even achieving zero-sample recognition.

[0024] This invention utilizes a multimodal information fusion and disambiguation mechanism to perform time alignment, consistency verification, contradiction disambiguation, and missing information completion on speech semantic information and visual situational information. This achieves intelligent fusion and complementary verification of multi-source information, effectively improving the overall quality and robustness of data synchronization. Attached Figure Description

[0025] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0026] Figure 1 This is a flowchart of a method for synchronizing and pushing manual wargame operation data based on a large model, provided by the present invention. Figure 2 This is a schematic diagram of the structure of the manual wargame operation data synchronization and push system based on a large model provided by the present invention; Figure 3 This is a hardware architecture diagram of the manual wargame operation data synchronization and push system based on a large model provided by the present invention; Figure 4 This is a flowchart of the multimodal data processing provided by the present invention; Figure 5 This is a flowchart of the multimodal information fusion and disambiguation process provided by the present invention; Figure 6 This is a schematic diagram of voice command parsing and mapping provided by the present invention; Figure 7 This is a diagram of the internal structure of the electronic device provided by the present invention. Detailed Implementation

[0027] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore merely examples, and should not be construed as limiting the scope of protection of the present invention.

[0028] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by those skilled in the art to which this invention pertains.

[0029] This invention provides a method, system, and device for synchronously pushing manual wargame operation data based on a large model. The embodiments of this invention are described below with reference to the accompanying drawings.

[0030] Example 1: Example 1 of the present invention provides a method for quantitative evaluation of the dynamic evolution of multiple elements in a wargaming environment, such as... Figure 1 As shown, the method in this embodiment specifically includes the following steps: S101 collects voice data and chessboard situation image data from the manual war game simulation. S102 uses a multimodal large model to perform speech recognition and military semantic analysis on the speech data to obtain speech semantic analysis results; performs chessboard change detection on the chessboard situation image data, and when a valid change is detected, performs chess piece detection, classification and coordinate mapping processing on the changed area to obtain situation incremental data. S103 performs time alignment and consistency checks on the speech semantic parsing results and the situational incremental data; when the checks are inconsistent, confidence assessment and rule constraint verification are performed, and contradiction disambiguation is performed based on the assessment and verification results; when any modal information of the speech semantic parsing results or the situational incremental data is missing, information is completed based on historical context. S104 generates structured wargame operation instructions based on the results of contradiction disambiguation or information completion, and pushes the structured wargame operation instructions to the computer wargame system via the data interface.

[0031] Before performing the above step S102, which involves speech recognition and military semantic parsing of the speech data based on a pre-established multimodal large model, the method further includes: Speech activity detection is performed on the speech data to identify valid speech segments and filter out invalid audio segments.

[0032] In step S102 above, the speech data is subjected to speech recognition and military semantic analysis using a multimodal large model, and the resulting speech semantic analysis results include: The speech data is processed using a streaming inference mode to perform speech recognition and output the corresponding recognized text. The identified text is processed by military domain intent classification, entity extraction, and relation extraction. Based on the intent classification results, entity extraction results, and relation extraction results, corresponding speech and semantic parsing results are generated.

[0033] The multimodal large model can be a pre-trained model using CLIP or Flamingo architectures, fine-tuned on collected hand-crafted wargaming speech-image alignment datasets. The speech recognition part can be based on Wav2Vec2.0 or Whisper, while the military semantic parsing part can use a general-purpose large model (such as ChatGLM3) trained through cue engineering or low-rank adaptation (LoRA). This invention does not limit the specific model structure, as long as it can achieve both speech recognition and military semantic parsing functions.

[0034] In one embodiment, a pre-trained multimodal large model (e.g., a speech-text joint model based on the Transformer architecture, fine-tuned on a military corpus) is used to process effective speech segments: Use streaming inference modes (such as CTC / RNNT-based decoders) to output recognized text in real time, such as "Move Battalion 1 from A3 to B4".

[0035] Military domain intent classification of the identified text: The BERT classification head outputs the probability distribution of operation type, such as "move" probability 0.95, "attack" probability 0.03, and "reconnaissance" probability 0.02.

[0036] Entity extraction: The chess piece unit "1st Battalion" is extracted using the BIO annotation method, with source coordinates "A3" and target coordinates "B4".

[0037] Relation extraction: Dependency parsing determines that "1st Battalion" is the moving subject, and "A3" and "B4" are spatial parameters.

[0038] The generated speech semantic parsing results are as follows: Operation type = movement, subject = 1st battalion, source position = A3, target position = B4, confidence score = 0.92 (obtained by weighting acoustic confidence and semantic logical consistency).

[0039] The above step S102, which involves detecting chessboard changes in the chessboard situation image data, includes: Perform a difference analysis between the current chessboard situation image frame and the reference image frame, and use an adaptive background modeling method to distinguish between foreground changes caused by piece movement and background changes caused by illumination fluctuations.

[0040] In step S102 above, piece detection, classification, and coordinate mapping are performed on the changed area to obtain incremental situational data, including: Based on the detected changes in the region of change, the position of the chess piece is located and the bounding box of the chess piece is generated. The chess piece images are classified based on their bounding boxes, and the type, faction affiliation, and number of each chess piece are output. Map the pixel coordinates of the categorized chess pieces to chessboard grid coordinates; The current identification results are compared with historical situation snapshots to calculate the situation increment data.

[0041] The above step S103, which performs time alignment and consistency checks between the speech semantic parsing results and the situational incremental data, includes: The timestamps corresponding to the speech semantic parsing results are matched with the timestamps corresponding to the situational incremental data to establish a temporal correspondence between speech events and visual events. The operation subject and target position in the speech semantic parsing results are compared with the chess piece change information in the situational incremental data to perform a consistency check.

[0042] In the event of inconsistency, step S104 above involves confidence assessment and rule constraint verification, including: Calculate the confidence score of the speech semantic parsing result and the confidence score of the situational incremental data, respectively; The wargaming rules knowledge base is invoked to perform compliance verification on the speech semantic parsing results and the situational incremental data.

[0043] In the above embodiments, calculating the confidence score of the speech semantic parsing result and the confidence score of the situational incremental data respectively includes: The confidence score of the speech semantic parsing result is calculated based on the acoustic confidence of the speech recognition model and the logical consistency of the semantic parsing model. The confidence score of the situational incremental data is calculated based on the detection score of the chess piece detection model and the grid matching degree of the coordinate mapping. The compliance verification by calling the wargaming rules knowledge base includes: Based on the speech semantic parsing results and the operation type, subject, and target location involved in the situational incremental data, the corresponding movement distance limit, terrain accessibility, and attack range constraint are queried in the wargaming rule knowledge base to determine whether the two meet the preset wargaming rules.

[0044] Step S103 above, which involves conflict resolution based on the evaluation and verification results, includes: when the speech semantic parsing result is inconsistent with the situational incremental data, comparing the confidence scores of the two and selecting the modality with the higher score as the final result; if the confidence scores of the two are similar, then abandoning the one that does not conform to the rules in the rule verification; if both violate the rules, then triggering manual confirmation or rolling back to the historical state.

[0045] Step S104 above, which involves information completion based on historical context, includes: retrieving recent operation sequences (e.g., the last 5 operation records and their corresponding board states) stored in the historical context cache; and performing inference and completion on missing modal information based on the recent operation sequences and the current situation. For example, if speech recognition is missing (e.g., no one is speaking but visual movement of pieces is detected), the possible operation semantics are inferred based on the movement trajectory of the pieces and the rules of wargames; if visual changes are missing (e.g., a voice command is issued but the board does not move), the historical context is used to determine whether it is an implicit state change (e.g., casualties or removal of pieces) or a delayed occurrence.

[0046] Example 2: Based on the same technical concept, Example 2 of this invention also provides a system for synchronously pushing manual wargame operation data based on a large model, such as... Figure 2 As shown, the system, as a product claim of the above method, comprises the following functional modules: a data acquisition module 21, an intelligent processing module 22, a multimodal fusion inference module 23, and a data push module 24, wherein: Data acquisition module 21 is used to collect voice data and chessboard situation image data at the scene of manual war game simulation; The intelligent processing module 22 is used to perform speech recognition and military semantic analysis on the speech data using a multimodal large model to obtain speech semantic analysis results; and to perform chessboard change detection on the chessboard situation image data. When a valid change is detected, chess piece detection, classification and coordinate mapping processing are performed on the changed area to obtain situation incremental data. The multimodal fusion reasoning module 23 is used to perform time alignment and consistency verification between the speech semantic parsing results and the situational incremental data; when the verification is inconsistent, confidence assessment and rule constraint verification are performed, and contradiction disambiguation is performed based on the assessment and verification results; when any modal information is missing, information is completed based on the historical context; and structured wargame operation instructions are generated based on the results of contradiction disambiguation or information completion. The data push module 24 is used to push the structured wargame operation instructions to the computer wargame system via the data interface.

[0047] The functions and implementation methods of each module of the system in this embodiment correspond to the method steps in Embodiment 1. For specific implementation details, please refer to the description in Embodiment 1, which will not be repeated here.

[0048] Example 3: Example 3 of the present invention provides a system for synchronously pushing manual wargame operation data based on a large model. The system is based on a large model system and adopts a hierarchical design to realize the synchronous push of manual wargame operation data.

[0049] like Figure 3 As shown in the figure, this embodiment provides a data synchronization and push system for manual wargame operations based on a large model. The system is divided into four hardware layers: a data acquisition layer, an intelligent processing layer, a fusion decision-making layer, and a data push layer.

[0050] The data acquisition layer includes a microphone array (101) and a high-definition camera (102). The microphone array (101) is arranged in a ring around the manual wargaming table (107) and contains 4-8 directional microphones for collecting voice commands from the players (108). The high-definition camera (102) is mounted directly above the wargaming table to collect complete chessboard situation image data from a top-down angle. The collected data is pre-processed through an edge computing gateway (103) for noise reduction, frame extraction, and other preliminary processing before being transmitted to the intelligent processing layer.

[0051] The intelligent processing layer is deployed on the multimodal large model inference server (104) and is the core processing unit of the system, which includes parallel speech processing channels and vision processing channels.

[0052] The fusion decision layer receives and processes data from the intelligent processing layer, and its core is the multimodal fusion reasoning module.

[0053] The data push layer establishes a connection with the computer war game system (106) through the data synchronization interface module (105) to push the final generated structured operation instructions.

[0054] Example 4: Multimodal Data Processing Flow like Figure 4 As shown, this embodiment describes the system's data processing flow in detail. The method specifically includes the following steps: Step 1: Data Acquisition. Acquire audio data and chessboard situation image data from the manual wargaming simulation. Specifically, acquire audio data through a microphone array in audio stream acquisition step (201), and acquire chessboard situation image data through a high-definition camera in image stream acquisition step (205).

[0055] Step 2: Speech and visual processing.

[0056] Speech Processing Channel: A multimodal large model is used to perform speech recognition and military semantic analysis on the speech data to obtain speech semantic analysis results. First, speech activity detection (VAD) (202) is performed on the speech data to identify valid speech segments. Then, the speech recognition (ASR) module (203) uses a streaming inference mode to convert the speech into text. Finally, the military domain semantic analysis module (204) performs intent classification, entity extraction, and relation extraction on the text to generate structured speech semantic analysis results.

[0057] Visual processing channel: Performs chessboard change detection on the chessboard situation image data. The change detection module (206) performs difference analysis between the current frame and the reference frame to distinguish between real piece movement and illumination changes. When a valid change is detected, the piece detection and classification module (207) performs piece detection, classification and coordinate mapping processing on the changed area to obtain incremental situation data.

[0058] Step 3: Multimodal fusion reasoning. The multimodal fusion reasoning module (208) performs time alignment and consistency checks on the speech semantic parsing results and the situational incremental data. When discrepancies are found, confidence assessment and rule constraint verification are performed, and contradiction disambiguation is carried out based on the assessment and verification results; when any modal information is missing, information is completed based on the historical context.

[0059] Step 4: Generating and Pushing Instructions. The structured operation instruction generation module (209) generates structured wargame operation instructions based on the results of contradiction disambiguation or information completion. Finally, the data push module (210) pushes the structured wargame operation instructions to the computer wargame system via the data interface.

[0060] Example 5: Multimodal Information Fusion and Disambiguation like Figure 5 As shown, this embodiment provides a detailed description of the workflow of the multimodal fusion inference module.

[0061] The multimodal fusion inference module receives the speech semantic parsing results (301) and the visual situation recognition results (302).

[0062] First, time alignment is performed, matching the timestamps of the voice commands with the timestamps of the visual change detections.

[0063] Then, the consistency check module (304) is entered to compare whether the operation subject and target position in the voice are consistent with the chess piece change information recognized by vision.

[0064] If the tests are inconsistent, the disambiguation process is initiated: Confidence assessment module (305): Calculates the confidence of the speech link (based on acoustic confidence and semantic logical consistency) and the confidence of the visual link (based on detection score and grid matching degree).

[0065] Rule constraint verification module (306): Calls the deduction rule knowledge base (303) to verify whether the two results conform to the wargame rules (such as movement distance, terrain accessibility, etc.).

[0066] Disambiguation decision module (307): Combines the confidence score and rule verification results, and selects the most likely correct result as the final output.

[0067] If information about a certain modality is missing, it is inferred and completed using the historical context cache (308). For example, if visual movement is detected but there is no speech, the intention of the move is inferred based on the historical action pattern of the piece.

[0068] Example 6: Voice Command Parsing like Figure 6 As shown, this embodiment illustrates the voice command parsing process. The original voice waveform (401) is processed by ASR to obtain the voice recognition text (402). The military domain semantic parsing model further processes the text, outputting intent classification results (403) (such as movement, attack) and entity extraction results (404) (such as unit number, location). Finally, all this information is integrated into a structured operation command (405) containing fields such as operation type, subject, target, and parameters. This command is converted into a format recognizable by the computer wargaming system via a mapping table (406).

[0069] The workflow of the present invention is illustrated below through a specific embodiment: Example 7: The simulation team issued a voice command, "The Third Armored Battalion advances from A3 to D6," while simultaneously manually moving the piece representing the Third Armored Battalion from square A3 to square D6 on the board.

[0070] Voice channel processing: The microphone array collects voice signals. After the VAD detects the voice segments as valid, the ASR module recognizes them as the text "The Third Armored Battalion advances from A3 to D6". The semantic parsing module extracts the intent as "movement", the subject as "Third Armored Battalion", the starting point as "A3" and the ending point as "D6".

[0071] Visual channel processing: The camera captures images of the chessboard. The change detection module detects the disappearance of a piece in square A3 and the appearance of a new piece in square D6. The piece detection module identifies the piece in square D6 as an armored piece, belonging to the blue side, and numbered 03. By comparing it with the historical snapshot, it is confirmed that the piece moved from A3 to D6.

[0072] Fusion processing: The time alignment module confirms that the time difference between the voice command issuance and the visual change detection time is 2 seconds, which is within the 5-second time window; the consistency check confirms that "Third Armored Battalion" in the voice is consistent with "Armored Type, Number 03" in the visual recognition, and "A3 to D6" is consistent with the movement path detected by the vision, thus determining that the two modal information are consistent.

[0073] Command generation and push: Directly generate structured operation commands {Operation type: move; Subject: 3rd Armored Battalion; Starting point: A3; End point: D6}, and push them to the computer war game system via the WebSocket interface. The system updates the situation display.

[0074] Example 7: The simulation team issued a voice command, "The first infantry company attacks B5," but due to the obstruction of the pieces, the visual module only detected a change in the pieces on B5 but could not identify the specific type.

[0075] Fusion Processing: Time alignment confirms time matching between the two modalities; consistency check reveals missing piece types in the visual information. Due to the missing visual modal information, an information completion mechanism is triggered. The historical context cache shows that the piece was previously positioned on square B4 and was an infantry piece. Combined with the voice command "first infantry unit," it is inferred that the piece changing on square B5 is the first infantry unit, thus completing the missing information.

[0076] Command generation: Generate structured operation commands {Operation type: Attack; Subject: First Infantry Company; Target: B5} and push them.

[0077] Example 8: The simulation team issued a voice command, "Artillery battalion move to E8," but the visual module detected that E8 remained unchanged, while an artillery piece appeared on F8.

[0078] Fusion Processing: Time alignment confirms temporal matching between the two modalities; consistency check reveals inconsistency between the speech target location "E8" and the visual change location "F8", triggering disambiguation. The speech confidence score is calculated to be 0.92 (acoustic clarity, semantic logical consistency), and the visual confidence score is 0.85 (moderate lighting conditions). Verification using the wargame rule knowledge base: cell E8 represents water terrain, inaccessible to the artillery battalion; cell F8 represents plain terrain, accessible. The visual result is deemed more consistent with the rules, but the speech confidence score is significantly higher. After comprehensive evaluation, the speech result is selected and marked as pending confirmation, while the simulation team is prompted to verify it.

[0079] Example 9: In one embodiment, Example 8 of the present invention also provides an electronic device; the electronic device may be a terminal, and its internal structure diagram may be as follows. Figure 7As shown. The electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the method for synchronously pushing manual wargaming operation data based on a large model, as described in any one of steps S101 to S104. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0080] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for synchronously pushing manual wargame operation data based on a large model, characterized in that, The method includes: Collect voice data and chessboard situation image data from the manual war game simulation. The speech data is subjected to speech recognition and military semantic analysis using a multimodal large model to obtain speech semantic analysis results; chessboard change detection is performed on the chessboard situation image data, and when a valid change is detected, chess piece detection, classification and coordinate mapping processing are performed on the changed area to obtain situation incremental data; The speech semantic parsing results and the situational incremental data are time-aligned and checked for consistency. When the checks are inconsistent, confidence assessment and rule constraint verification are performed, and contradiction disambiguation is performed based on the assessment and verification results. When any modal information of the speech semantic parsing results or the situational incremental data is missing, information is completed based on the historical context. Based on the results of conflict resolution or information completion, structured wargame operation instructions are generated and pushed to the computer wargame system via a data interface.

2. The method according to claim 1, characterized in that, Before performing speech recognition and military semantic analysis on the speech data based on a pre-established multimodal large model, the process also includes: Speech activity detection is performed on the speech data to identify valid speech segments and filter out invalid audio segments.

3. The method according to claim 1, characterized in that, The process of using a multimodal large model to perform speech recognition and military semantic analysis on the speech data yields the following speech semantic analysis results: The speech data is processed using a streaming inference mode to perform speech recognition and output the corresponding recognized text. The identified text is processed by military domain intent classification, entity extraction, and relation extraction. Based on the intent classification results, entity extraction results, and relation extraction results, corresponding speech and semantic parsing results are generated.

4. The method according to claim 1, characterized in that, The chessboard change detection of the chessboard situation image data includes: Perform a difference analysis between the current chessboard situation image frame and the reference image frame, and use an adaptive background modeling method to distinguish between foreground changes caused by piece movement and background changes caused by illumination fluctuations.

5. The method according to claim 1, characterized in that, The process of performing piece detection, classification, and coordinate mapping on the changed area to obtain incremental situational data includes: Based on the detected changes in the region of change, the position of the chess piece is located and the bounding box of the chess piece is generated. The chess piece images are classified based on their bounding boxes, and the type, faction affiliation, and number of each chess piece are output. Map the pixel coordinates of the categorized chess pieces to chessboard grid coordinates; The current identification results are compared with historical situation snapshots to calculate the situation increment data.

6. The method according to claim 1, characterized in that, The step of performing time alignment and consistency verification between the speech semantic parsing results and the situational incremental data includes: The timestamps corresponding to the speech semantic parsing results are matched with the timestamps corresponding to the situational incremental data to establish a temporal correspondence between speech events and visual events. The operation subject and target position in the speech semantic parsing results are compared with the chess piece change information in the situational incremental data to perform a consistency check.

7. The method according to claim 1, characterized in that, The confidence assessment and rule constraint verification include: Calculate the confidence score of the speech semantic parsing result and the confidence score of the situational incremental data, respectively; The wargaming rules knowledge base is invoked to perform compliance verification on the speech semantic parsing results and the situational incremental data.

8. The method according to claim 7, characterized in that, The calculation of the confidence scores of the speech semantic parsing results and the situational increment data includes: The confidence score of the speech semantic parsing result is calculated based on the acoustic confidence of the speech recognition model and the logical consistency of the semantic parsing model. The confidence score of the situational incremental data is calculated based on the detection score of the chess piece detection model and the grid matching degree of the coordinate mapping. The compliance verification by calling the wargaming rules knowledge base includes: Based on the speech semantic parsing results and the operation type, subject, and target location involved in the situational incremental data, the corresponding movement distance limit, terrain accessibility, and attack range constraint are queried in the wargaming rule knowledge base to determine whether the two meet the preset wargaming rules.

9. A system for synchronously pushing manual wargame operation data based on a large model, characterized in that, include: The data acquisition module is used to collect voice data and chessboard situation image data from the manual war game simulation. The intelligent processing module is used to perform speech recognition and military semantic analysis on the speech data using a multimodal large model to obtain speech semantic analysis results; and to perform chessboard change detection on the chessboard situation image data. When a valid change is detected, chess piece detection, classification and coordinate mapping processing are performed on the changed area to obtain situation incremental data. The multimodal fusion reasoning module is used to perform time alignment and consistency checks on the speech semantic parsing results and the situational incremental data; when the checks are inconsistent, confidence assessment and rule constraint verification are performed, and contradiction disambiguation is performed based on the assessment and verification results; when any modal information is missing, information is completed based on the historical context; and structured wargame operation instructions are generated based on the results of contradiction disambiguation or information completion. The data push module is used to push the structured wargame operation instructions to the computer wargame system via a data interface.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-8.

Images