A distributed war game deduction system supporting AR device cooperation and a management method thereof

Abstract

The application relates to the technical field of AR devices, and discloses a distributed manual war game deduction system supporting AR device cooperation and a management method. The method comprises the following steps: verifying a registration message sent by an AR device, establishing a long connection channel in a central server, uploading a state reporting message of the AR device to the central server through the long connection channel, updating a device state table of the central server, synchronously pushing global deduction situation data to a browser client, performing operator marking identification on a war game map image, obtaining an operator physical change event, generating situation update data based on the operator physical change event, and broadcasting the situation update data to all online AR devices and browser clients, rendering an electronic map in the browser client according to the situation update data, and generating an augmented reality projection picture in the AR device. The application realizes seamless connection between a physical world manual operation and a digital system, and further realizes operation reliability of a large-scale hybrid war game deduction system.

Description

Technical Field

[0001] This invention relates to the field of AR device technology, and in particular to a distributed manual wargame simulation system and management method that supports AR device collaboration. Background Technology

[0002] Traditional manual wargaming relies on manual recording, manual adjudication, and paper maps, which is inefficient and prone to errors. With technological advancements, purely computer-based wargaming systems have emerged, but they lack the tactile experience and collaborative atmosphere of manual wargaming. In recent years, hybrid wargaming systems combining physical pieces and digital assistance have begun to appear, but they typically suffer from the following problems: 1. Poor terminal collaboration: Data synchronization between AR devices, recording terminals, and servers is not real-time, resulting in inconsistent situations and frequent interruptions in the simulation process. 2. Lack of device management: Multiple AR devices are scattered across large maps, making it impossible to centrally monitor their operational status (online status, camera / projector functionality), leading to difficult system maintenance. 3. Rigid architecture: Existing solutions often use customized clients, making deployment and updates inconvenient and failing to support cross-platform, lightweight access requirements. Summary of the Invention

[0003] The main objective of this invention is to provide a distributed manual wargaming system and management method that supports AR device collaboration. This invention solves the technical problems of poor collaboration, lack of device management, and rigid architecture of traditional manual wargaming terminals, and realizes seamless connection between manual operation in the physical world and digital system, thereby achieving operational reliability of large-scale hybrid wargaming system.

[0004] To achieve the above objectives, this invention provides a distributed manual wargaming method supporting AR device collaboration, comprising the following steps:

[0005] Verify the registration message sent by the AR device and establish a long connection channel on the central server, and upload the status reporting message of the AR device to the central server through the long connection channel;

[0006] The device status table of the central server is updated according to the status reporting message, and the device status table and global simulation situation data are synchronously pushed to the browser client.

[0007] The AR device performs operator tag recognition on the wargame map image it collects, obtains operator physical change events, and uploads them to the central server. Based on the operator physical change events, situation update data is generated and broadcast to all online AR devices and browser clients.

[0008] Based on the situation update data, an electronic map is rendered on the browser client and an augmented reality projection is generated on the AR device.

[0009] Optionally, in a first implementation of the first aspect of the present invention, verifying the registration message sent by the AR device and establishing a long connection channel on the central server, and uploading the status reporting message of the AR device to the central server through the long connection channel, includes:

[0010] The server receives the registration message sent by the AR device, parses and verifies the device identifier, coverage area coordinates and hardware parameters in the registration message, and creates a device session object on the central server.

[0011] A long connection channel is established by sending a registration confirmation message to the AR device based on the device session object.

[0012] The operating parameters of the image acquisition device, the operating parameters of the projection device, and the network connection parameters of the AR device are collected and encapsulated into a status reporting message, which is then uploaded to the central server through the long connection channel.

[0013] Optionally, in a second implementation of the first aspect of the present invention, collecting the image acquisition device operating parameters, projection device operating parameters, and network connection parameters of the AR device and encapsulating them into a status reporting message, and uploading it to the central server through the long connection channel, includes:

[0014] The focal length value and marker recognition status of the image acquisition device in the AR device are collected to obtain the operating parameters of the image acquisition device; the brightness value and temperature value of the projection device in the AR device are collected to obtain the operating parameters of the projection device; the latency value and packet loss rate of the network interface in the AR device are collected to obtain the network connection parameters.

[0015] The operating parameters of the image acquisition device, the operating parameters of the projection device, and the network connection parameters are encapsulated into a status reporting message.

[0016] The status reporting message is sent to the central server through the long connection channel, and the central server updates the corresponding device record in the device status table.

[0017] Optionally, in a third implementation of the first aspect of the present invention, updating the device status table of the central server according to the status reporting message, and synchronously pushing the device status table and global situational simulation data to the browser client, includes:

[0018] Based on the session identifier in the status report message, locate the corresponding device record in the device status table, parse the image acquisition device operating parameters, projection device operating parameters and network connection parameters in the status report message, and update the status field and communication timestamp in the device record;

[0019] The device status table is traversed to extract the device status information of all registered AR devices. The device status information and global inference situation data are encapsulated into a JSON format data packet, and the JSON format data packet is broadcast to all online browser clients via a WebSocket connection.

[0020] Optionally, in a fourth implementation of the first aspect of the present invention, operator marker recognition is performed on the wargame map image collected by the AR device to obtain operator physical change events and upload them to the central server. Based on the operator physical change events, situation update data is generated and broadcast to all online AR devices and browser clients, including:

[0021] The wargame map image acquired by the AR device is subjected to distortion correction and perspective transformation, converted into a top-view coordinate system image, and the operator position coordinates are calculated based on the top-view coordinate system image;

[0022] The Euclidean distance is calculated based on the historical location coordinates and the operator's location coordinates. The physical change event of the operator is determined based on the Euclidean distance and uploaded to the central server.

[0023] The operator generates situation update data based on physical change events and broadcasts it to all online AR devices and browser clients.

[0024] Optionally, in a fifth implementation of the first aspect of the present invention, calculating the Euclidean distance based on the historical location coordinates and the operator location coordinates, determining the operator physical change event based on the Euclidean distance, and uploading it to the central server includes:

[0025] Read historical position coordinates, calculate the Euclidean distance between the historical position coordinates and the operator position coordinates, and if the Euclidean distance exceeds the movement threshold, determine the operator physical change event as a movement event;

[0026] The operator orientation angle and the marked surface state are detected based on the operator position coordinates, and the operator physical change event is determined to be a flipping event based on the operator orientation angle and the marked surface state;

[0027] The operator physical change event is sent to the central server through the long connection channel.

[0028] Optionally, in a sixth implementation of the first aspect of the present invention, generating situation update data based on the operator physical change event and broadcasting it to all online AR devices and browser clients includes:

[0029] The physical change event of the operator is retrieved from the event processing queue and passed to the wargame rule engine to generate the situational change result;

[0030] Based on the situation change results, situation update data is generated, and the situation update data is broadcast to all online browser clients via WebSocket connection, and the situation update data is also broadcast to all online AR devices via the long connection channel.

[0031] Optionally, in the seventh implementation of the first aspect of the present invention, retrieving the operator physical change event from the event processing queue and passing it to the wargaming rule engine to generate a situational change result includes:

[0032] Retrieve the operator physical change event from the event processing queue, and extract unit information based on the operator identifier and coordinate change in the operator physical change event;

[0033] The unit information is passed to the wargaming rule engine for rule matching to generate situational change results.

[0034] Optionally, in an eighth implementation of the first aspect of the present invention, rendering an electronic map on a browser client and generating an augmented reality projection image on an AR device based on the situation update data includes:

[0035] The browser client parses the situation update data received via the WebSocket connection, refreshes the troop marker positions and unit status icons on the electronic map, and parses and updates the device status indicator colors and hardware parameters on the device management panel based on the device status table data.

[0036] The AR device parses the situation update data received by the long connection channel, generates an augmented reality projection image, and projects the augmented reality projection image onto the surface of the physical wargaming map through the projection device.

[0037] This invention also provides a distributed manual wargaming system supporting AR device collaboration, comprising:

[0038] The verification module is used to verify the registration message sent by the AR device and establish a long connection channel with the central server, and upload the status reporting message of the AR device to the central server through the long connection channel;

[0039] The device status update module is used to update the device status table of the central server according to the status reporting message, and push the device status table and global simulation situation data to the browser client in a synchronized manner.

[0040] The operator marker recognition module is used to perform operator marker recognition on the wargame map image collected by the AR device, obtain operator physical change events and upload them to the central server, generate situation update data based on the operator physical change events and broadcast them to all online AR devices and browser clients;

[0041] The image generation module is used to render an electronic map on the browser client and generate an augmented reality projection image on the AR device based on the situation update data.

[0042] In summary, this invention, through automatic registration of AR devices and a long-term connection mechanism combined with periodic status reporting, achieves plug-and-play functionality and centralized monitoring of distributed AR devices. Maintenance personnel can gain a comprehensive understanding of device health without on-site inspection, improving maintainability. Simultaneous push of simulation situation data and device status tables upon browser client connection provides users with a complete system overview the moment they open the application, ensuring information integrity. Real-time image recognition and physical change event reporting by AR devices achieve seamless integration between manual operation in the physical world and the digital system, allowing simulation participants to maintain traditional physical piece operation habits. Centralized rule adjudication and synchronized broadcasting across the network ensure that all terminals receive situation data from a unified authoritative data source, avoiding data inconsistency issues and ensuring the operational reliability of the large-scale hybrid wargaming simulation system. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the steps of a distributed manual wargaming method supporting AR device collaboration in one embodiment of the present invention;

[0044] Figure 2 This is a block diagram of a distributed manual wargaming system supporting AR device collaboration in an embodiment of the present invention.

[0045] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0047] Reference Figure 1 This embodiment provides a distributed manual wargaming method that supports AR device collaboration, including the following steps:

[0048] S1, verify the registration message sent by the AR device and establish a long connection channel on the central server, and upload the status reporting message of the AR device to the central server through the long connection channel;

[0049] S2, update the device status table of the central server according to the status reporting message, and push the device status table and global simulation situation data to the browser client in a synchronized manner;

[0050] S3 performs operator tag recognition on the wargame map images collected by AR devices, obtains operator physical change events and uploads them to the central server, generates situation update data based on operator physical change events and broadcasts it to all online AR devices and browser clients;

[0051] S4 renders an electronic map on the browser client and generates an augmented reality projection on the AR device based on situational update data.

[0052] In one example, the registration message sent by the AR device is verified, and a long-lived connection is established with the central server. The AR device's status reporting message is then uploaded to the central server via this long-lived connection, including:

[0053] Receive the registration message sent by the AR device, and parse and verify the device identifier, coverage area coordinates and hardware parameters in the registration message, and create a device session object on the central server;

[0054] A long-connection channel is established by sending a registration confirmation message to the AR device based on the device session object.

[0055] The system collects the operating parameters of the image acquisition device, the operating parameters of the projection device, and the network connection parameters of the AR device, encapsulates them into a status reporting message, and uploads it to the central server through a long connection channel.

[0056] In this example, when the AR device powers on, its internal client program actively reads the server address and communication port information from the local configuration file, constructs a device registration message, and initiates a TCP connection request to the central server. The registration message uses a custom protocol format, including a fixed-length header and structured data segments. These segments contain a unique device identifier (e.g., a hash value generated based on a combination of the MAC address and the device serial number), the coordinates of the coverage area of ​​the device's physical deployment location (represented using a map coordinate system), and multiple hardware parameters (such as camera model and resolution, projector light source brightness level, built-in CPU and memory specifications, etc.). After the registration message is sent to the server, it is received by the central server's connection access module and passed to the parsing and verification module. The parsing and verification module performs integrity checks on the message structure, ensuring that the message length, encoding method, checksum, and other content conform to the communication protocol specifications. It then extracts the device identifier and compares it with existing device records on the server to confirm the device's legitimacy. Boundary checks and duplicate detection are performed on the coverage area coordinates to ensure that the same map area is not repeatedly covered by multiple devices, avoiding device function conflicts. Simultaneously, the availability of hardware parameters such as the camera and projector is assessed to determine if the device possesses the basic capabilities to participate in inference, recognition, and projection. If all three types of information are verified successfully, the server constructs a device session object for the device and allocates an independent resource area in memory to maintain the device's connection status, parameter information, communication handle, and last heartbeat timestamp. Subsequently, the server proactively sends a registration confirmation message back to the AR device, establishing a long-term connection channel between the device and the server. The registration confirmation message contains parameters such as the session identifier allocated by the server, the maximum allowed reporting frequency, the heartbeat interval, and the connection maintenance strategy. After receiving the registration confirmation message, the AR device caches the session identifier locally and initializes the heartbeat timer and status acquisition module according to the periodic configuration within it. During continuous operation, the AR device's status monitoring module triggers a parameter acquisition process every heartbeat cycle. This includes operating parameters of the image acquisition device (such as current camera focal length, automatic exposure status, image frame rate, and image recognition validity), operating parameters of the projection device (such as brightness percentage, bulb temperature, keystone correction coefficient, and whether overheat protection is triggered), and network connection-related parameters (such as RTT latency, packet loss rate, and signal quality). After a complete acquisition, these parameters are uniformly encoded into a structured data format and encapsulated into a status reporting message, which contains the current device session identifier, acquisition timestamp, and binary representations of all monitoring indicators. The status reporting message is sent to the central server in real time via an established long-term connection channel. Upon receiving the status reporting message, the server locates the device record based on the session identifier and writes the status data into the device status table.

[0057] In one example, the operating parameters of the AR device's image acquisition unit, projection unit, and network connection parameters are collected and encapsulated into a status reporting message, which is then uploaded to the central server via a long-lived connection channel. This includes:

[0058] The focal length value and tag recognition status of the image acquisition device in the AR device are collected to obtain the operating parameters of the image acquisition device; the brightness value and temperature value of the projection device in the AR device are collected to obtain the operating parameters of the projection device; the latency value and packet loss rate of the network interface in the AR device are collected to obtain the network connection parameters.

[0059] The operating parameters of the image acquisition device, the operating parameters of the projection device, and the network connection parameters are encapsulated into a status reporting message;

[0060] The status reporting message is sent to the central server through a long connection channel, and the central server updates the corresponding device record in the device status table.

[0061] In this example, a status acquisition module is built on the AR device side. This module consists of an image acquisition device monitoring subsystem, a projection device monitoring subsystem, and a network interface status evaluation subsystem. Within each preset acquisition cycle, the status acquisition module calls the image acquisition device's status reading function through the underlying driver interface to obtain the current lens focal length value in real time. The focal length value reflects the camera's focus position on the physical map. Simultaneously, combined with the marker recognition status of the most recent frame output by the image recognition module, it determines whether the device has successfully detected operator markers on the map within this cycle. These two fields are combined into the image acquisition device's operating parameter structure. The module calls the projection device driver interface to read the current brightness percentage of the projection light source and the temperature value output by the projection's internal temperature control module, constructing projection device operating parameters reflecting projection intensity and thermal stability. Through the network stack interface provided by the operating system, it obtains the current round-trip time (RTT) with the server and the packet loss rate statistics over a recent period, characterizing the device's network connection quality and forming a network connection parameter set. The status acquisition module structurally integrates the operating parameters of the image acquisition device, the projection device, and the network connection parameters. It then constructs status reporting messages according to the communication protocol specifications. The format of these messages includes a fixed header field (identifying the device session ID and timestamp) and tightly packed status data fields (encapsulating various acquisition results in byte sequences), forming a status data packet with complete semantics and structural constraints. After construction, the status acquisition module sends the status reporting message to the server through the long-lived connection established with the central server upon device startup. Upon receiving the status reporting message, the central server calls the status receiving module to perform protocol parsing. Based on the session identifier, it locates the corresponding device record in the device status table maintained in the server's memory. The server then updates the corresponding fields in the device status table with the parsed focal length, recognition status, brightness, temperature, latency, and packet loss rate, while simultaneously refreshing the device's last communication timestamp, ensuring that the device status table always reflects the latest operating status of the current AR device group.

[0062] In one example, the device status table on the central server is updated based on the status report message, and the device status table and the global simulated situation data are synchronized and pushed to the browser client, including:

[0063] Based on the session identifier in the status report message, locate the corresponding device record in the device status table, parse the image acquisition device working parameters, projection device working parameters and network connection parameters in the status report message, and update the status field and communication timestamp in the device record.

[0064] The device status table is traversed to extract the device status information of all registered AR devices. The device status information and global situational data are encapsulated into a JSON data packet, and the JSON data packet is broadcast to all online browser clients via a WebSocket connection.

[0065] In this example, the server's status receiving module performs protocol-level decoding on the status report messages and extracts a unique device session identifier from the message header fields. The device session identifier is allocated and recorded in memory by the server during the device registration phase, allowing the server to locate the corresponding device's record in the device status table. The device status table is a data structure indexed by session identifiers; each record contains the device identifier, hardware parameters, status fields, last communication timestamp, and coordinates of the device's coverage area. Once the target record is located, the server sequentially extracts the image acquisition device's operating parameters (e.g., camera focal length, recognition status Boolean value), the projection device's operating parameters (e.g., light source brightness percentage, device internal temperature value), and network connection parameters (e.g., current round-trip time (RTT) and packet loss rate) from the status report message body. After parsing, these status values ​​are updated in the corresponding fields of the device status table, and the current system timestamp is written to the "last communication time" field to reflect the device's latest active status update time. After completing the partial update of the device status table, the server periodically or upon triggering a status update event performs a global device status broadcast operation. The broadcast scheduling module in the server iterates through all valid records in the device status table and, for each registered and currently online AR device, extracts its device identifier, online / offline status, image acquisition and projection device parameters, network metrics, and coordinates of the covered map area, organizing these into a structured set of device status data entries. Simultaneously, the server reads core simulation information from its maintained global simulation situation database, including the current round number, the location distribution of each unit on the map, battle result statistics, and resource usage, packaging this information into a single data structure to form a system snapshot containing both a "device status subset" and a "situation information subset." The server then calls the data serialization module to convert the structured system snapshot into a JSON format data packet. JSON format offers both readability and cross-platform compatibility, making it suitable for direct parsing in web clients. After conversion, the server iterates through all currently established WebSocket sessions with browser clients, broadcasting the JSON format data packet as message frame content to all online clients. After each client-side front-end application receives the message, it automatically triggers the rendering logic to synchronously render the simulated situation onto the electronic map on the main interface, and updates the real-time status of all AR devices to the device management panel in the form of cards.

[0066] It also includes equipment anomaly detection and fault-tolerant handling steps: threshold judgment is performed on the image acquisition device operating parameters in the equipment status table; when the marker recognition status shows image loss or focal length value exceeding the normal range, an equipment anomaly mark is generated, the equipment anomaly mark is added to the corresponding device record, and pushed to the device management panel of all online browser clients via WebSocket connection for yellow warning display; based on the equipment anomaly mark, the coordinates of the coverage area of ​​the faulty device are determined, and the coverage area is marked as a temporary monitoring blind zone in the event receiving module of the central server, and the adjudication processing continues for operator physical change events from other normally operating AR devices; the status field update of the faulty device is monitored in the equipment status table, and when a status report message of recovery from the device is received, the equipment anomaly mark is cleared and the temporary monitoring blind zone mark is removed, and event receiving and situation update push of the coverage area are restored.

[0067] The device status information and global simulation situation data are encapsulated into a JSON format data packet, which is then broadcast to all online browser clients via a WebSocket connection. This includes: after the browser client completes the WebSocket protocol upgrade handshake with the central server, a WebSocket session object is created for the browser client, and the central server immediately executes a dual data synchronization push process; complete simulation situation data, including the current simulation round number, the deployment positions of all parties' forces, battle damage statistics, and map control area, is read from the global situation database, serialized into a first JSON data packet, and sent to the browser client via the WebSocket session object to trigger the rendering of the main simulation interface; the device status table is traversed to extract the device identifier, online / offline status, coverage area coordinates, and latest hardware status parameters of all registered AR devices, serialized into a second JSON data packet, and sent to the browser client via the WebSocket session object to trigger the rendering of the device management panel, ensuring that users obtain a complete system panoramic view of the simulation situation and device status as soon as the connection is established.

[0068] In one example, operator marker recognition is performed on the wargame map image captured by the AR device to obtain operator physical change events, which are then uploaded to the central server. Based on these operator physical change events, situational update data is generated and broadcast to all online AR devices and browser clients, including:

[0069] Distortion correction and perspective transformation are performed on the wargame map images captured by AR devices to convert them into top-view coordinate system images, and the operator position coordinates are calculated based on the top-view coordinate system images;

[0070] The Euclidean distance is calculated based on the historical location coordinates and the operator location coordinates. The physical change events of the operator are determined based on the Euclidean distance and uploaded to the central server.

[0071] The situation update data is generated based on the physical change events of the operator and broadcast to all online AR devices and browser clients.

[0072] In this example, the original image contains certain optical distortions and viewpoint shifts. Image preprocessing is performed on the wargaming map image captured by the AR device. The image preprocessing module calls a distortion correction algorithm to eliminate radial and tangential distortions caused by lens nonlinearity in the original image. Camera calibration parameters are used to remap the coordinates of each pixel, eliminating edge stretching or compression effects. , ,in, For uncorrected normalized image coordinates, The radial distortion coefficient is... The tangential distortion coefficient is... To correct the coordinates, a perspective transformation function is called to convert the processed image from an oblique view to an isometric planar view in a top-down coordinate system. The reference matrix for the perspective transformation is automatically generated during device initialization using four map corner markers, ensuring that the projection relationship of each frame after transformation is perfectly aligned with the map coordinate system. A perspective transformation is then performed on the image to construct the top-down view. The perspective transformation formula is:

[0073]

[0074] in, The homography matrix is ​​calculated by back-calculation from four map points. This represents the operator's position in the top-view coordinate system. Based on the top-view coordinate system image, the image recognition module uses a feature point matching algorithm to identify the marker patterns (such as color codes, QR codes, or chess piece shape outlines) of each operator in the image and extracts their pixel position coordinates in the image. Then, combining the known map scale and pixel mapping relationship, the pixel positions are converted back to their absolute positions in the map coordinate system, thus obtaining the operator's current position coordinates. Historical frame positions are obtained in consecutive frames. and current frame position Spatial displacement between two frames is calculated using Euclidean distance. When the Euclidean distance exceeds a preset minimum movement threshold (e.g., half the length of a unit square), it is determined that the operator has undergone a valid physical displacement between the current frame and a historical frame, generating an "operator movement" event. If the recognized pattern undergoes flipping, state switching, or addition / removal, a corresponding type of operator physical change event is generated. All operator physical change events confirmed by the image recognition module are uniformly packaged into structured event messages, which include an event type identifier (movement, flipping, appearance, disappearance), a unique operator ID, coordinates before and after the change, a recognition timestamp, and an image confidence score. The event messages are uploaded to the central server via a long connection channel. After receiving the event messages, the server passes them to the wargaming rule engine to execute the situation adjudication logic. The rule engine determines whether the operation is legal based on the attributes, positional relationships, and terrain information of each unit in the current global simulation situation database, and generates a situation update data packet based on the adjudication rules. The data packet structure includes the new coordinates of the affected units, attribute state changes, terrain control changes, etc., and is serialized in standard JSON format. The central server uses a dual-channel broadcast mechanism to send situation update data packets to all online browser clients via WebSocket connections and to all online AR devices via persistent connections. Upon receiving the update data, the browser clients automatically refresh the electronic map interface and recording form, while the AR devices parse relevant data items based on their own map coverage area and update the unit positions and control markers in the projected image in real time.

[0075] The system performs distortion correction and perspective transformation on the wargame map images captured by the AR device, converting them into top-view coordinate system images. Based on these top-view coordinate system images, it calculates the operator position coordinates. This includes: extracting the current frame image from the video stream output by the AR device's image acquisition device at the frame rate; performing distortion correction and perspective transformation preprocessing on the current frame image to convert the image from the camera's viewpoint coordinate system to the standard top-view map coordinate system; running a feature point matching algorithm to search for predefined operator marker patterns in the top-view coordinate system image; decoding operator identifiers and calculating the position coordinates of all operators detected in the current frame in the map coordinate system, storing this as the current frame operator position set; reading the historical operator position set recorded during the processing of the previous frame image from the AR device's local state cache; comparing the position coordinates of each operator in the current frame operator position set with the historical operator position set; calculating the Euclidean distance between the position vectors of the same operator identifier; generating a movement event identifier when the Euclidean distance exceeds a preset movement threshold; simultaneously detecting changes in the orientation angle of the operator marker surface to determine flipping events; recording the detected change events and updating the local state cache as the current frame operator position set for processing in the next frame.

[0076] In one example, the Euclidean distance is calculated based on historical location coordinates and operator location coordinates. The physical change events of the operator are then determined based on the Euclidean distance and uploaded to the central server, including:

[0077] Read the historical position coordinates, calculate the Euclidean distance based on the historical position coordinates and the operator position coordinates, and if the Euclidean distance exceeds the movement threshold, then determine the operator physical change event as a movement event;

[0078] The operator orientation angle and the marked surface state are detected based on the operator position coordinates, and the operator physical change event is determined to be a flipping event based on the operator orientation angle and the marked surface state;

[0079] The operator physical change event is sent to the central server through a long connection channel.

[0080] In this example, the AR device continuously maintains historical position records for each identified operator. The latest position coordinates of a given operator are extracted from the current frame image using an image recognition algorithm, denoted as... Simultaneously, the historical position coordinates of the operator are read from the previous frame buffer. The spatial displacement between the current position and historical positions is calculated using the two-dimensional Euclidean distance formula. The formula is ,in, These are the operator coordinates identified in the current image frame, in pixels or map coordinates under projection mapping; These are the coordinates of the same operator recorded in the previous frame. The calculated distance... With the preset minimum movement threshold Compare, if satisfied If a significant physical movement of the operator is detected, it is determined that the operator has moved, confirming it as a "movement event," and the event reporting process begins. Simultaneously, the image recognition module analyzes the operator's orientation angle and the state of the marker surface in the current image. The orientation angle can be determined by detecting preset directional patterns (such as arrows, direction bars, or QR codes) on the operator and calculating its rotation angle relative to the image coordinate system. The state of the marked surface is determined by the identification symbol or color pattern on the top surface of the operator, for example, by using a color histogram, texture features, or a CNN classification model to identify whether the current orientation is upward or downward. The orientation angle of the current frame is then... Angle from historical frames Comparison: If the angle difference between two frames exceeds the set flip angle threshold (e.g., ...), ), A "flipping event" is defined as a change in the marked face state from "front" to "back". For identified movement and flipping events, an event message is constructed, containing the following fields: Event Type: "Move" or "Flip", Operator ID: Mark status Event messages are sent to the server in real time via a long-lived connection between the AR device and the central server.

[0081] In one example, situation update data is generated based on operator physical change events and broadcast to all online AR devices and browser clients, including:

[0082] Retrieve operator physical change events from the event processing queue and pass them to the wargame rule engine to generate situational change results;

[0083] Based on the situational changes, situational update data is generated and broadcast to all online browser clients via WebSocket connection, and also to all online AR devices via long connection channel.

[0084] In this example, an event processing queue maintained by a central server handles event reception and sequential scheduling. The event processing queue manages all structured operator physical change event messages uploaded by AR devices in a first-in, first-out (FIFO) manner. Each message contains fields such as event type (e.g., move, flip), unique operator identifier, position coordinates before and after the change, timestamp, and image recognition confidence level. When the event processing thread starts, it retrieves an event to be processed from the head of the queue, unpacks its content, and passes it to the wargaming rule engine module integrated within the server. Upon receiving the event, the rule engine searches for the corresponding unit entity record in the global simulation situation database based on the operator identifier contained in the event, loads the context information of the current simulation round, such as terrain layers, enemy and friendly unit positions, and combat status, and then adjudicates the event according to the built-in rule base. For example, for a move event, the rule engine verifies whether the movement path is legal, whether it crosses obstacles, whether it enters enemy fire coverage, and whether it triggers interception rules. For a flip event, it updates the unit attribute status, such as changing from "concealed" to "exposed," or indicating a change in unit type. After the ruling is completed, the rules engine generates a structured situational change result. This structure records all state changes triggered by the current event, including new unit positions, status codes, changes in control area ownership, and updates to simulation statistics. This situational change result is then passed to the data encapsulation module to generate a unified-format situational update data packet. The situational update data packet is organized using a JSON structure, with fields including round number, unit status list, unit position mapping table, combat event description, timestamp, etc., and encoded according to the terminal parsing format. After encapsulation, the server synchronously pushes the packet to all online terminals through two types of broadcast channels: The server iterates through the list of all currently maintained browser client WebSocket connection sessions and pushes a JSON situational update data packet to each online connection. Upon receiving the data packet, the browser client triggers rendering logic, updating the map unit icon positions, status colors, animation effects, and record table contents in the simulation interface to ensure that the client view remains synchronized with the server's ruling situational change. The server iterates through all active AR device session connection channels and, based on the map area covered by the device, sends the corresponding situational update data packet to each AR device via a proprietary protocol. After receiving the data, the AR device parses the data items related to its physical area and performs operations such as unit animation, status annotation, and control boundary drawing in the augmented reality image rendering module to update the virtual and real superimposed image in real time.

[0085] The situation update data is broadcast to all online AR devices via a long-connection channel. This includes: extracting the coverage area coordinates of each online AR device from the device status table; determining whether each unit is within the coverage area of ​​the corresponding AR device based on the unit location coordinates in the situation update data; generating a regional situation subset for each AR device; filtering each regional situation subset to retain only the location coordinates, status changes, movement trajectories, and fire range data of units within the coverage area, while removing information about units outside the coverage area, and encapsulating it into a dedicated situation update message for that AR device; traversing the device session object list and sending the corresponding dedicated situation update message through the long-connection channel of each AR device, so that each AR device only receives and renders augmented reality projection content within its own coverage area.

[0086] The situation update data is broadcast to all online browser clients via WebSocket connections and to all online AR devices via long-lived connection channels. This includes: performing dual-protocol serialization processing on the situation update data to generate JSON data packets conforming to the WebSocket message frame format and binary data packets conforming to the AR device's proprietary protocol. Both types of data packets contain the same situation change content but use different encoding methods; initiating a dual-threaded parallel broadcast process, the first thread iterates through the WebSocket session object list and sends JSON data packets to each active WebSocket connection, and the second thread iterates through the device session object list and sends binary data packets to each active long-lived connection channel. The two threads execute synchronously to ensure that all terminals receive the situation update within the same time window; recording the situation update reception confirmation timestamp of each terminal on the central server, marking terminals that have not returned confirmation for more than a preset time threshold as having a network delay status, updating the network connection parameters of the corresponding terminal in the device status table, and pushing the notification to the browser client for monitoring and alarm purposes.

[0087] The situation update data is broadcast to all online browser clients via WebSocket connection. This includes: generating a first broadcast data packet containing a description of the adjudication result, the latest location and status of the unit, changes in the map control area, and inference statistics based on the situation change results, and serializing it into JSON format; at the same time as generating the first broadcast data packet, reading the current complete status of the device status table, extracting the device identifier, online status, working parameters, and communication timestamp of all AR devices, generating a second broadcast data packet, and serializing it into JSON format; traversing the WebSocket session object list of all online browser clients, and simultaneously sending the first and second broadcast data packets in a single broadcast operation to ensure that the main inference interface and device management panel of the browser client complete the data update within the same time window, maintaining the synchronized refresh rhythm of the inference situation display and device status monitoring.

[0088] In one example, operator physical change events are retrieved from the event processing queue and passed to the wargaming rules engine to generate situational change results, including:

[0089] Retrieve operator physical change events from the event processing queue, and extract unit information based on the operator identifier and coordinate changes in the operator physical change events;

[0090] The unit information is passed to the wargaming rules engine for rule matching, generating the situational change results.

[0091] In this example, the central server has a structured event processing queue. This queue caches all operator physics change event messages uploaded in real-time by AR devices in chronological order. Each event message contains core fields such as event type, operator unique identifier, coordinates before and after the change, identification timestamp, and identification confidence level. When the event processing module is activated, it retrieves an event from the queue according to a first-in-first-out (FIFO) rule and immediately performs field-level parsing on the event content. The two most critical fields are the operator identifier and coordinate change information. The server uses the operator identifier to query the corresponding unit information record in the global simulation situation database. This record includes elements such as unit number, unit type, faction affiliation, current health, ammunition, available tactical actions, and terrain adaptability. The coordinate change information is used to determine the unit's current geographical location and its potential impact on the map environment. By combining the unit's historical coordinates with the current coordinate change, it can preliminarily determine whether the unit has entered a terrain boundary area, crossed a boundary, or is approaching enemy units, and extract related spatial adjacency units, such as the enemy's observable range, fire coverage area, or terrain obstacle distribution. The server extracts complete unit information and current position parameters as input data and passes them to the wargame rule engine module. The rule engine performs pattern matching and condition triggering based on its built-in rule base. The rule base defines the allowed actions, movement path legality, engagement criteria, flipping conditions, and damage calculation methods for various unit types under different environments and states. The rule matching process compares the unit's type, current state, and coordinate changes with the preconditions set by each rule in the rule set, using Boolean judgments and logical expressions to determine if the triggering criteria for a specific tactical rule are met. If a match is successful, the rule engine generates a situational change result based on the execution path corresponding to the rule. This result is represented in a structured format and includes information such as the unit's new position, changes in combat state (e.g., entering suppression, attrition, retreat), descriptions of triggered combat events, changes in local terrain control, and incremental updates to global damage statistics.

[0092] In one example, based on situational update data, an electronic map is rendered on the browser client and an augmented reality projection is generated on the AR device, including:

[0093] The browser client parses the situation update data received from the WebSocket connection, refreshes the troop position markers and unit status icons on the electronic map, and parses and updates the device status indicator colors and hardware parameters on the device management panel by parsing the device status table data.

[0094] The AR device parses the situation update data received through the long connection channel, generates an augmented reality projection image, and projects the augmented reality projection image onto the surface of the physical wargaming map through a projection device.

[0095] In this example, the browser client and the AR device respectively perform the dual functions of graphics rendering and augmented reality presentation based on received data. When the central server broadcasts a JSON-formatted situation update data packet to each online browser client via WebSocket, the WebSocket listener on the client's front end immediately calls the parsing function to deconstruct the data content. The parsing result contains two main substructures: "Situational Situation Segment," including unit identifiers, map coordinates, status codes, and changes in control; and "Device Status Table Segment," including device numbers, connection statuses, image acquisition and projection device operating parameters, network quality indicators, and timestamps for all online AR devices. The front-end system passes the Situational Situation Segment to the main simulation interface module. The main simulation interface module updates the troop marker positions on the electronic map based on each unit's unique identifier and coordinate data. By submitting new style attributes and position information to the front-end rendering engine, it dynamically moves the troop marker icons to the new coordinate positions and adjusts the unit icon colors or adds tactical status icons based on the status codes, such as marking "suppression," "assault," and "destruction" status effects. Simultaneously, the device status table segment is transmitted to the device management panel module. This module refreshes the status indicator color corresponding to each device card based on the device's online status field; for example, green indicates online, red indicates offline, and yellow indicates abnormal status. It also reads fields such as image recognition status, projection brightness, temperature, and network latency of each device, updating device operating parameters in real time in a table or chart format to assist administrators in remote monitoring. On the AR device side, after receiving the situation update data packet broadcast by the server through its long-term connection channel with the central server, the local communication processing module parses out the unit coordinate changes and status update information related to the device's coverage area and transmits this data to the augmented reality image generation module. The augmented reality image generation module constructs a projection content layer based on the received unit coordinate information, drawing updated unit position graphics, movement trajectory arrows, fire range coverage circles, and situational effects such as combat explosion animations and control area boundary lines. It also overlays unit status identifiers, such as unit type labels and life status indicators, onto the layer. The AR device transmits the generated augmented reality layer to the projection driver module, which drives the projection device to project the synthesized image onto the surface of the physical wargaming map in a registered manner, achieving a fusion display of digital information and the physical map. The projected image is dynamically refreshed according to changes in the situation.

[0096] Reference Figure 2 This embodiment provides a distributed manual wargaming system that supports AR device collaboration, including:

[0097] Verification module 1 is used to verify the registration message sent by the AR device and establish a long connection channel with the central server, and upload the status reporting message of the AR device to the central server through the long connection channel;

[0098] Device status update module 2 is used to update the device status table of the central server according to the status report message, and push the device status table and global simulation situation data to the browser client in a synchronized manner;

[0099] The operator marker recognition module 3 is used to perform operator marker recognition on the wargame map image collected by the AR device, obtain operator physical change events and upload them to the central server, generate situation update data based on the operator physical change events and broadcast them to all online AR devices and browser clients;

[0100] The image generation module 4 is used to render an electronic map on the browser client and generate an augmented reality projection image on the AR device based on the situation update data.

[0101] In this embodiment, the specific implementation of each unit in the above system embodiment is described in the above method embodiment, and will not be repeated here.

[0102] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, system, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, system, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, system, article, or method that includes that element.

[0103] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A distributed manual wargaming method supporting AR device collaboration, characterized in that, include: The process involves verifying the registration message sent by the AR device and establishing a long-term connection channel on the central server. The AR device's status reporting message is then uploaded to the central server through this long-term connection channel. Specifically, this includes: receiving the registration message sent by the AR device and parsing and verifying the device identifier, coverage area coordinates, and hardware parameters in the registration message; creating a device session object on the central server; sending a registration confirmation message to the AR device based on the device session object to establish a long-term connection channel; and collecting the AR device's image acquisition device operating parameters, projection device operating parameters, and network connection parameters, encapsulating them into a status reporting message, and uploading it to the central server through the long-term connection channel. The device status table of the central server is updated according to the status reporting message, and the device status table and global simulation situation data are synchronously pushed to the browser client. The process involves: identifying operator markers in the wargame map image captured by the AR device to obtain operator physical change events, which are then uploaded to the central server. Based on these operator physical change events, situational update data is generated and broadcast to all online AR devices and browser clients. Specifically, this includes: performing distortion correction and perspective transformation on the wargame map image captured by the AR device, converting it to a top-down coordinate system image, and calculating operator position coordinates based on the top-down coordinate system image; reading historical position coordinates, calculating the Euclidean distance between the historical position coordinates and the operator position coordinates, and determining the operator physical change event as a movement event if the Euclidean distance exceeds a movement threshold; detecting the operator orientation angle and marker surface state based on the operator position coordinates, and determining the operator physical change event as a flipping event based on the operator orientation angle and marker surface state; sending the operator physical change event to the central server via the long connection channel; and generating situational update data based on the operator physical change event and broadcasting it to all online AR devices and browser clients. Based on the situation update data, an electronic map is rendered on the browser client and an augmented reality projection is generated on the AR device.

2. The distributed manual wargaming method supporting AR device collaboration according to claim 1, characterized in that, The system collects the operating parameters of the image acquisition device, the operating parameters of the projection device, and the network connection parameters of the AR device, encapsulates them into a status reporting message, and uploads it to the central server through the long connection channel, including: The focal length value and marker recognition status of the image acquisition device in the AR device are collected to obtain the operating parameters of the image acquisition device; the brightness value and temperature value of the projection device in the AR device are collected to obtain the operating parameters of the projection device; the latency value and packet loss rate of the network interface in the AR device are collected to obtain the network connection parameters. The operating parameters of the image acquisition device, the operating parameters of the projection device, and the network connection parameters are encapsulated into a status reporting message. The status reporting message is sent to the central server through the long connection channel, and the central server updates the corresponding device record in the device status table.

3. The distributed manual wargaming method supporting AR device collaboration according to claim 2, characterized in that, Update the device status table of the central server according to the status reporting message, and push the device status table and global simulation situation data to the browser client synchronously, including: Based on the session identifier in the status report message, locate the corresponding device record in the device status table, parse the image acquisition device operating parameters, projection device operating parameters and network connection parameters in the status report message, and update the status field and communication timestamp in the device record; The device status table is traversed to extract the device status information of all registered AR devices. The device status information and global inference situation data are encapsulated into a JSON format data packet, and the JSON format data packet is broadcast to all online browser clients via a WebSocket connection.

4. The distributed manual wargaming method supporting AR device collaboration according to claim 1, characterized in that, Based on the physical change events of the operator, situation update data is generated and broadcast to all online AR devices and browser clients, including: The physical change event of the operator is retrieved from the event processing queue and passed to the wargame rule engine to generate the situational change result; Based on the situation change results, situation update data is generated, and the situation update data is broadcast to all online browser clients via WebSocket connection, and the situation update data is also broadcast to all online AR devices via the long connection channel.

5. The distributed manual wargaming method supporting AR device collaboration according to claim 4, characterized in that, The operator physical change event is retrieved from the event processing queue and passed to the wargaming rules engine to generate situational change results, including: Retrieve the operator physical change event from the event processing queue, and extract unit information based on the operator identifier and coordinate change in the operator physical change event; The unit information is passed to the wargaming rule engine for rule matching to generate situational change results.

6. The distributed manual wargaming method supporting AR device collaboration according to claim 5, characterized in that, Based on the situation update data, the browser client renders an electronic map and the AR device generates an augmented reality projection, including: The browser client parses the situation update data received via the WebSocket connection, refreshes the troop marker positions and unit status icons on the electronic map, and parses and updates the device status indicator colors and hardware parameters on the device management panel based on the device status table data. The AR device parses the situation update data received by the long connection channel, generates an augmented reality projection image, and projects the augmented reality projection image onto the surface of the physical wargaming map through the projection device.

7. A distributed manual wargaming system supporting AR device collaboration, characterized in that, The steps for implementing the distributed manual wargaming method supporting AR device collaboration as described in any one of claims 1 to 6 include: The verification module is used to verify the registration message sent by the AR device and establish a long connection channel with the central server, and upload the status reporting message of the AR device to the central server through the long connection channel; The device status update module is used to update the device status table of the central server according to the status reporting message, and push the device status table and global simulation situation data to the browser client in a synchronized manner. The operator marker recognition module is used to perform operator marker recognition on the wargame map image collected by the AR device, obtain operator physical change events and upload them to the central server, generate situation update data based on the operator physical change events and broadcast them to all online AR devices and browser clients; The image generation module is used to render an electronic map on the browser client and generate an augmented reality projection image on the AR device based on the situation update data.

Images