A touch method and system for wargaming based on action capture and projection fusion

Abstract

This invention discloses a wargaming touch control method and system based on motion capture and projection fusion, belonging to the field of wargaming technology. Through projection fusion technology, the wargaming materials are projected onto the target wargaming area, and the various wargaming distribution maps are fused and projected, greatly expanding the projection interface of the wargaming projection and supporting the needs of large-scale wargaming. Furthermore, the system captures the target operator's action video stream in real time, establishing an operator action model, achieving real-time and accurate motion capture, and obtaining complete and accurate operator action information. Moreover, a pre-trained operator action recognition model analyzes the target operator's action information in real time, identifying the corresponding action recognition results. Based on the action recognition results, the target wargaming object is selected, and the corresponding wargaming update parameters are calculated, completing precise wargaming touch control execution.

Description

Technical Field

[0001] This invention belongs to the field of wargaming technology, specifically relating to a wargaming touch control method and system based on motion capture and projection fusion. Background Technology

[0002] Wargaming, as an important tool for military command training and operational research, is mainly divided into two categories: manual wargaming and electronic wargaming. Traditional manual wargaming relies on physical chessboards, paper counting tools, rulebooks, and dice for simulations, with adjudicators manually consulting adjudication tables to determine the outcome. While this method is intuitive, it suffers from problems such as long production cycles for counting tools and maps, inconvenience in carrying them, and complex adjudication rules, resulting in low simulation efficiency and difficulty in recording and reviewing the process. Electronic wargaming, on the other hand, uses computers for simulations and adjudication, solving some of the cumbersome problems of manual operation. However, it is limited by screen size, and large-scale simulations require a large space. Furthermore, the complexity of electronic wargaming operation makes it difficult for commanders, especially senior commanders, to directly participate, greatly limiting the flexibility and realism of the simulations.

[0003] In terms of existing technologies, while touchscreens and electronic sand tables can achieve digital interaction, general-purpose all-in-one laser projectors, although possessing projection capabilities, have touch ranges limited by infrared scanning technology, supporting only finger touch operations within a 27-inch projection area. This cannot meet the needs of large-scale wargaming simulations exceeding 50 inches, and they cannot recognize manually crafted wargaming pieces and maps. Motion capture and image recognition technologies lack specialized training for wargaming operation habits, making it difficult to accurately recognize specific gestures such as picking up, dragging, and releasing pieces, thus hindering the widespread application of wargaming simulations in realistic combat training scenarios.

[0004] As mentioned above, how to provide a wargaming touch control method and system based on motion capture and projection fusion that can support the needs of large-scale wargaming simulations, achieve real-time motion capture, and complete accurate motion recognition and touch execution has become an urgent problem to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a touch-based method and system for wargaming simulation based on motion capture and projection fusion, in order to solve the above-mentioned problems existing in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a touch-based wargaming simulation method based on motion capture and projection fusion, comprising: Acquire preset wargaming materials and, based on projection fusion technology, project the wargaming materials onto the target wargaming area to form a wargaming projection interface. The wargaming materials include multiple wargaming operators and multiple wargaming distribution maps. Based on the war game projection interface, motion capture is performed on the target operator to obtain the target operator's motion video stream. An operator motion model is established based on the target operator's motion video stream, and motion information is extracted from the target operator's motion video stream according to the operator motion model to obtain the operator's motion information. A pre-trained operator action recognition model is obtained, and the operator action information of the target operator is input into the operator action recognition model to perform real-time action recognition to obtain the action recognition result of the target operator. The action recognition result includes the action object, action position, and action pattern. Based on the action recognition results of the target operator, the target wargaming object is selected, and the corresponding wargaming update parameters are calculated. Based on the corresponding wargaming update parameters, the target wargaming object is projected and updated in the wargaming projection interface to complete the wargaming touch control.

[0007] In one possible design, preset wargaming materials are acquired, and based on projection fusion technology, the wargaming materials are projected onto a target wargaming area to form a wargaming projection interface, including: Multiple preset wargaming operators and multiple wargaming distribution maps are obtained as wargaming materials, and the initial position information of each wargaming operator and the map code of each wargaming distribution map are obtained. Using short-throw projection technology, according to the map codes of each war game simulation distribution map, the war game simulation distribution maps are sequentially projected and stitched together in the target simulation area to form an initial war game simulation projection interface. The initial wargaming projection interface is calibrated in a plane, and using projection fusion technology, each wargaming operator is projected onto the initial wargaming projection interface that has completed the plane calibration according to the initial position information of each wargaming operator, thus obtaining the wargaming projection interface.

[0008] In one possible design, based on the wargaming projection interface, motion capture is performed on the target operator to obtain a motion video stream of the target operator. An operator motion model is then established based on the target operator's motion video stream. Finally, motion information is extracted from the target operator's motion video stream according to the operator motion model to obtain operator motion information, including: The interface size parameters of the wargaming projection interface are obtained, the target operator's field of view is calculated based on the interface size parameters of the wargaming projection interface, and the target operator's motion is captured based on the target operator's field of view to obtain the target operator's original motion video stream. The original motion video stream of the target operator is subjected to distortion removal processing and effective extrapolation interval cropping to obtain the motion video stream of the target operator; A preset motion feature localization model is obtained, and the motion video stream of the target operator is used as input and input frame by frame into the motion feature localization model so that the motion feature localization model can output the motion feature position corresponding to each frame of the motion video stream. Based on the action feature position corresponding to each frame of the action video stream, the operator's action region is cropped for each frame of the target operator's action video stream to obtain the action positioning region map corresponding to each frame of the image. The key action nodes of the action localization region map corresponding to each frame image are extracted to obtain the key action node features of the target operator corresponding to each frame image. Coordinate values ​​are extracted from the key node features of the target operator's actions corresponding to each frame of the image, so as to extract the coordinate values ​​of each key node of the operator's actions in each frame of the image. Obtain the preset connection order of key action nodes, and according to the connection order of key action nodes, connect the coordinate values ​​of each key action node of the operator in each frame of the image in sequence to obtain the operator action model corresponding to each frame of the image. Based on the operator action model corresponding to each frame image, action information is extracted from each frame image to obtain the action coordinate information and action posture information of the target operator in each frame image. These are then integrated into the action coordinate information and action posture information of the target operator in each frame image to form operator action information.

[0009] In one possible design, the pre-training method for the operator action recognition model includes: Collect multiple effective simulation actions from the war game simulation data records, and integrate the effective simulation actions of each operator into a war game simulation action library; Original operation images and wargame simulation operation intentions are collected from multiple operators. The collected original operation images are enhanced to obtain operation image samples. The operation image samples are matched with the collected wargame simulation operation intentions to form model training samples. A preset image recognition base model is obtained, and the model parameters of the image recognition base model are trained in multiple rounds using the model training samples to obtain an operator action recognition model.

[0010] In one possible design, the operator's action information is input into the operator action recognition model for real-time action recognition to obtain the target operator's action recognition result, including: The operator's action information is used as input to the operator action recognition model, so that the corresponding operator action recognition result is output through the operator action recognition model, and the operator action recognition result is used as the operator's wargame simulation operation intention. Based on the operator's wargaming operation intention, the real-time wargaming operation object of the target operator is identified as the action object, wherein the action object is the corresponding wargaming operator or the corresponding wargaming distribution map. Based on the operator's wargaming operation intention, the real-time wargaming operation position of the target operator is identified as the action position, wherein the action position includes the starting position of the action object and the operation target position of the action object; Based on the operator's wargaming operation intention, the real-time wargaming operation mode of the target operator is identified as an action mode, wherein the action mode includes at least object selection, object picking, object dragging and object releasing. The action object, the action location, and the action pattern are integrated as the action recognition result.

[0011] In one possible design, based on the action recognition result of the target operator, a target wargaming object is selected, and the wargaming update parameters corresponding to the target wargaming object are calculated. Based on the wargaming update parameters corresponding to the target wargaming object, the target wargaming object is projected and updated on the wargaming projection interface to complete the wargaming touch control, including: Based on the action object in the action recognition result of the target operator, the corresponding war game simulation operator or the corresponding war game simulation distribution map is selected from the war game simulation projection interface as the target war game simulation object. Based on the action pattern in the action recognition result of the target operator, the inference update parameters to be generated are determined, wherein the inference update parameters to be generated include object movement distance parameters, object movement direction parameters and / or object scaling ratio parameters; Based on the action position in the action recognition result of the target operator, the simulation update parameters to be generated are calculated to obtain the simulation update parameters corresponding to the target wargame simulation object; Based on the simulation update parameters corresponding to the target wargaming object, the target wargaming object is projected and updated in the wargaming projection interface to complete the wargaming touch control.

[0012] In one possible design, after completing the touch control for wargaming simulation, it also includes: The action recognition results of the wargaming touch control, the target wargaming object, and the wargaming update parameters are integrated to form a corresponding wargaming touch control record; The wargame simulation data records are updated by integrating the touch records of each wargame simulation, resulting in updated wargame simulation data records.

[0013] Secondly, this invention provides a wargaming simulation touch control system based on motion capture and projection fusion, comprising: The wargaming projection interface generation unit is used to acquire preset wargaming materials and, based on projection fusion technology, project the wargaming materials onto the target wargaming area to form a wargaming projection interface. The wargaming materials include multiple wargaming operators and multiple wargaming distribution maps. The operator motion information extraction unit is used to capture the motion of the target operator based on the war game projection interface to obtain the motion video stream of the target operator, establish an operator motion model based on the motion video stream of the target operator, and extract motion information from the motion video stream of the target operator according to the operator motion model to obtain operator motion information. The action recognition result calculation unit is used to obtain a pre-trained operator action recognition model, input the operator action information of the target operator into the operator action recognition model, perform real-time action recognition, and obtain the action recognition result of the target operator, wherein the action recognition result includes the action object, action position and action pattern; The wargaming touch execution unit is used to select a target wargaming object based on the action recognition result of the target operator, calculate the wargaming update parameters corresponding to the target wargaming object, and perform projection update on the target wargaming object in the wargaming projection interface based on the wargaming update parameters corresponding to the target wargaming object, thereby completing the wargaming touch control.

[0014] Thirdly, the present invention provides an electronic device comprising a memory, a processor, and a transceiver connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the wargaming touch control method based on motion capture and projection fusion as described in the first aspect or any possible design of the first aspect.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, perform the wargaming touch control method based on motion capture and projection fusion as described in the first aspect or any possible design of the first aspect.

[0016] Fifthly, the present invention provides a computer program product containing instructions that, when the instructions are executed on a computer, cause the computer to perform the wargaming touch control method based on motion capture and projection fusion as described in the first aspect or any possible design of the first aspect.

[0017] Beneficial Effects: This invention provides a touch-screen method and system for wargaming simulation based on motion capture and projection fusion, comprising: First, acquiring preset wargaming materials and projecting the wargaming materials onto a target simulation area based on projection fusion technology to form a wargaming projection interface, wherein the wargaming materials include multiple wargaming operators and multiple wargaming distribution maps; Second, based on the wargaming projection interface, performing motion capture on the target operator to obtain a motion video stream of the target operator, establishing an operator motion model based on the target operator motion video stream, and extracting motion information from the target operator's motion video stream according to the operator motion model. The process involves obtaining operator action information, acquiring a pre-trained operator action recognition model, inputting the target operator's action information into the model, and performing real-time action recognition to obtain the target operator's action recognition result. This result includes the action object, action position, and action pattern. Finally, based on the target operator's action recognition result, a target wargame simulation object is selected, and the simulation update parameters corresponding to that object are calculated. Based on these parameters, the target wargame simulation object is then projected and updated on the wargame simulation projection interface, completing the wargame simulation touch control. By using projection fusion technology, the wargaming materials are projected onto the target wargaming area, and the various wargaming distribution maps are fused and projected, greatly expanding the projection interface of the wargaming projection and supporting the needs of large-scale wargaming. Furthermore, the system captures the target operator's action video stream in real time, establishes an operator action model, and achieves real-time and accurate motion capture, obtaining complete and accurate operator action information. In addition, a pre-trained operator action recognition model analyzes the target operator's action information in real time, identifies the corresponding action recognition results, selects the target wargaming object based on the action recognition results, calculates the corresponding wargaming update parameters, and completes precise wargaming touch execution. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the touch control method for wargaming simulation based on motion capture and projection fusion provided in an embodiment of the present invention; Figure 2 A functional structure diagram of a wargaming touch control system based on motion capture and projection fusion provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0020] It should be understood that although the terms first, second, etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, without departing from the scope of the exemplary embodiments of the invention.

[0021] It should be understood that the term "and / or" that may appear in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" that may appear in this document describes another relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the related objects before and after it are in an "or" relationship.

[0022] Example: like Figure 1 As shown, the first aspect of this embodiment provides a touch-based wargaming simulation method based on motion capture and projection fusion, which may include, but is not limited to, the following steps: S1. Obtain preset wargaming materials, and based on projection fusion technology, project the wargaming materials onto the target wargaming area to form a wargaming projection interface, wherein the wargaming materials include multiple wargaming operators and multiple wargaming distribution maps. In one possible implementation, step S1 involves acquiring preset wargaming materials and projecting these materials onto a target wargaming area using projection fusion technology to form a wargaming projection interface. This step can be broken down into, but is not limited to, the following steps S11-S13, specifically including: S11. Obtain multiple preset wargaming operators and multiple wargaming distribution maps as wargaming materials, and obtain the initial position information of each wargaming operator and the map code of each wargaming distribution map. S12. Using short-throw projection technology, according to the map codes of each of the war game simulation distribution maps, the war game simulation distribution maps are sequentially projected and stitched together in the target simulation area to form an initial war game simulation projection interface. S13. Perform planar calibration on the initial wargaming projection interface, and use projection fusion technology to project each wargaming operator onto the initial wargaming projection interface that has completed planar calibration according to the initial position information of each wargaming operator, so as to obtain the wargaming projection interface.

[0023] In specific application scenarios, short-throw (or ultra-short-throw) projection lenses can be used to project the generated digital images onto a physical plane in a large size (covering an area of ​​50 to 100 inches) and high definition. Due to the use of short-throw technology, the device can achieve large-screen coverage at a position close to the edge of the simulation area, solving the problem of traditional projection being limited by insufficient distance.

[0024] It should be noted that the wargaming touch control method provided in this embodiment realizes physical electronic manual wargaming AR touch control simulation. This embodiment uses projection fusion technology to project wargaming operators and wargaming distribution maps onto any plane. In possible implementations, dual-camera visual recognition and key point detection technology can be combined, allowing the target operator to directly click, pick up, and drag virtual wargaming operators on the wargaming projection interface with their fingers, just like operating physical wargaming pieces. This retains the advantages of intuitive and simple manual wargaming operation, while eliminating the need to carry physical wargaming pieces and maps, greatly improving the portability and flexibility of the simulation. The dual-camera visual recognition technology can help overcome the size limitations of existing projection touch control technology, achieving precise touch operation under large-scale projections of 50 inches to 100 inches. One camera is used to capture the operator's simulation actions, while the other camera is used to locate the mapping relationship between the target operator's specific action parts and the wargaming projection interface for adaptive field of view adjustment, ensuring high-precision touch control can still be achieved in large-size projection areas, meeting the needs of simulations at different scales such as strategic, operational, and tactical levels.

[0025] Furthermore, the projection fusion technology described in this embodiment solves the problem of limited coverage of a single projector by separately projecting and codedly arranging and stitching multiple wargaming distribution maps. In scenarios involving large-scale wargaming (such as large-scale battle scenarios), this embodiment supports matrix stitching and fusion of at least four (even number) or more projectors. The projectors exchange coordinate information through a local area network or Bluetooth to achieve seamless map stitching and continuous movement of wargaming operators across projection areas. This can construct ultra-wide wargaming scenarios to support the needs of large-scale command and on-site tactical discussions.

[0026] S2. Based on the war game projection interface, the target operator's motion is captured to obtain the target operator's motion video stream. An operator motion model is established based on the target operator's motion video stream. The operator motion information is extracted from the target operator's motion video stream according to the operator motion model to obtain the operator motion information. In one possible implementation, in step S2, motion capture is performed on the target operator based on the wargaming projection interface to obtain a motion video stream of the target operator. An operator motion model is then established based on the target operator's motion video stream, and motion information is extracted from the target operator's motion video stream according to the operator motion model to obtain operator motion information. This can be, but is not limited to, decomposed into the following steps S21-S28, specifically including: S21. Obtain the interface size parameters of the war game projection interface, calculate the target operator's shooting field of view based on the interface size parameters of the war game projection interface, and perform motion capture on the target operator based on the target operator's shooting field of view to obtain the target operator's original motion video stream. S22. Perform distortion removal processing and effective extrapolation interval cropping on the original motion video stream of the target operator to obtain the motion video stream of the target operator; S23. Obtain a preset motion feature localization model, and input the motion video stream of the target operator as the input, frame by frame into the motion feature localization model, so as to output the motion feature position corresponding to each frame of the motion video stream through the motion feature localization model. S24. Based on the action feature position corresponding to each frame image in the action video stream, crop the operator action region of each frame image in the action video stream of the target operator to obtain the action positioning region map corresponding to each frame image. S25. Extract key action nodes from the action localization region map corresponding to each frame image to obtain the key action node features of the target operator corresponding to each frame image; S26. Extract coordinate values ​​from the key node features of the target operator's actions corresponding to each frame image, so as to extract the coordinate values ​​of each key node of the operator's actions in each frame image. S27. Obtain the preset connection order of key action nodes, and according to the connection order of key action nodes, connect the coordinate values ​​of each key action node of the operator in each frame of the image in sequence to obtain the operator action model corresponding to each frame of the image. S28. Based on the operator action model corresponding to each frame image, extract action information from each frame image to obtain the target operator's action coordinate information and action posture information in each frame image, and integrate them into the target operator's action coordinate information and action posture information in each frame image to form operator action information.

[0027] The action feature localization model uses a lightweight convolutional neural network (such as MobileNetV2 or a small CNN) as its backbone network. It performs convolution layer by layer on each input frame image to generate feature maps of different resolutions. These feature maps retain information from low-level details to high-level semantics. This backbone network outputs multiple feature maps through layer-by-layer convolution (the resolution of each feature map decreases layer by layer, while the semantic information is enhanced layer by layer). Starting from the highest-level feature map, it upsamples (such as bilinear interpolation) to match the resolution of the next-level feature map. Then, the next-level feature map is fused with the upsampled high-level feature map through lateral connection (1×1 convolution, adjusting the number of channels and then adding them together) to generate a new feature map. Repeating this step yields a set of multi-scale feature maps, where each layer takes into account both high resolution and strong semantic information. This design allows small-scale actions (the target operator is located at the far end of the projection) to be detected on high-resolution feature maps, while large-scale actions (the target operator is located at the near end of the projection) can be detected on low-resolution feature maps, thereby improving the multi-scale detection capability of the target operator's actions. Furthermore, the action feature localization model pre-defines a set of default boxes with different sizes and aspect ratios at each pixel location in each feature map (e.g., sizes of {0.1, 0.2, 0.4} times the image size, and aspect ratios of {1:1, 1:2, 2:1}). These default boxes serve as candidate regions, covering possible action locations and sizes. Then, for each feature map layer, two sets of 3×3 convolutional kernels are used in parallel to predict the probability that each default box contains the effective action of the target operator (binary classification: target operator and background). A sigmoid or softmax algorithm is then used to obtain the confidence score for the action feature localization corresponding to each default box. After obtaining the confidence score for the action feature localization of each default box, the default boxes are filtered according to a pre-define confidence score threshold (e.g., 0.5) to retain high-confidence default boxes. All retained candidate boxes are then sorted in descending order of confidence. Select the highest-scoring default box as the baseline, calculate the Intersection over Union (IoU) of the remaining boxes with this box, and remove redundant default boxes with an IoU greater than a set threshold (such as 0.3) (because these default boxes overlap too much with the baseline, indicating that they are detecting the same target). Repeat the above process for the remaining default boxes until all default boxes have been traversed, and finally obtain several (usually one, but multiple if there are multiple action parts) high-confidence and non-overlapping action feature localization bounding boxes, i.e. action feature locations.

[0028] When extracting key motion nodes from the motion localization region map corresponding to each frame of the image, the backbone network can be used to extract the depth features of the target operator's motion parts through multi-layer convolution and pooling operations. A heatmap regression strategy is then employed, setting the last layer of the backbone network as a convolutional layer to output heatmaps with multiple channels (the number of channels is preset to the number of key motion nodes, such as 13 or 21). Each channel corresponds to a confidence distribution map of a key motion node (typically 64×64 or 128×128 in size). The peak position in the heatmap represents the most probable location of that key point. Then, for each heatmap channel, the pixel coordinates of the maximum value are found to map the pixel coordinates on the heatmap back to the coordinates in the original image, obtaining the absolute pixel coordinates of each key motion node in the image coordinate system of each frame. To capture features of small areas, the backbone network also employs a feature pyramid structure, fusing shallow high-resolution detail features with deep semantic features to generate feature maps with rich spatial information.

[0029] S3. Obtain a pre-trained operator action recognition model, input the operator action information of the target operator into the operator action recognition model, perform real-time action recognition, and obtain the action recognition result of the target operator, wherein the action recognition result includes the action object, action position and action pattern; In one possible implementation, the pre-training method for the operator action recognition model in step S3 may include, but is not limited to, the following steps S301-S303, specifically: S301. Collect multiple effective simulation actions from the war game simulation data records, and integrate the effective simulation actions of each operator into a war game simulation action library; S302. Collect original operation images and wargame simulation operation intentions from multiple operators, perform image enhancement processing on the collected original operation images to obtain operation image samples, and match the operation image samples with the collected wargame simulation operation intentions to form model training samples. S303. Obtain a preset image recognition base model, and use the model training samples to train the model parameters of the image recognition base model in multiple rounds to obtain an operator action recognition model.

[0030] Examples of image enhancement processing may include, but are not limited to, random cropping, rotation, brightness and contrast adjustment, Gaussian noise addition, and temporal frame interpolation and temporal frame extraction. This increases the sample size by 3-5 times to prevent overfitting.

[0031] It should be noted that, in the exemplified but not limited implementation, in wargame simulations of general complexity (few types and numbers of wargame operators, few wargame distribution maps, and relatively simple content), YOLOv8 or Mediapipe is used as the front-end processing layer of the image recognition base model, and a temporal network (such as LSTM) is used as the back-end classification layer of the image recognition base model to form an efficient image recognition base model for classification and recognition. However, in wargame simulations of high complexity (many types and numbers of wargame operators, many wargame distribution maps, and relatively complex content), the front-end processing layer can be replaced with 3D convolutional networks such as SlowFast, C3D, or VideoSwinTransformer to perform more accurate analysis and recognition of the wargame simulation operation intentions.

[0032] In one possible implementation, step S3, where the operator's action information is input into the operator action recognition model for real-time action recognition to obtain the target operator's action recognition result, can be broken down into, but is not limited to, the following steps S31-S35, specifically including: S31. The operator's action information of the target operator is used as input to the operator action recognition model, so as to output the corresponding operator action recognition result through the operator action recognition model, and the operator action recognition result is used as the operator's war game simulation operation intention. S32. Based on the operator's wargaming operation intention, identify the target operator's real-time wargaming operation object as the action object, wherein the action object is the corresponding wargaming operator or the corresponding wargaming distribution map. S33. Based on the operator's wargaming operation intention, identify the target operator's real-time wargaming operation position as the action position, wherein the action position includes the starting position of the action object and the operation target position of the action object; S34. Based on the operator's wargaming operation intention, identify the target operator's real-time wargaming operation mode as an action mode, wherein the action mode includes at least object selection, object picking, object dragging and object release; S35. Integrate the action object, the action location, and the action pattern as the action recognition result.

[0033] It should be noted that in the wargaming touch control method provided in this embodiment, the real-time wargaming operation position of the target operator identified in step S33 is the position of the target operator's hand in the image space. Using it as the action position requires coordinate transformation to map it to the starting position of the action object and the operation target position of the action object on the projection interface.

[0034] S4. Based on the action recognition results of the target operator, select the target wargame simulation object, calculate the simulation update parameters corresponding to the target wargame simulation object, and based on the simulation update parameters corresponding to the target wargame simulation object, perform projection update on the target wargame simulation object in the wargame simulation projection interface to complete the wargame simulation touch control.

[0035] In one possible implementation, in step S4, based on the action recognition result of the target operator, a target wargaming object is selected, and the wargaming update parameters corresponding to the target wargaming object are calculated. Based on the wargaming update parameters corresponding to the target wargaming object, the target wargaming object is projected and updated in the wargaming projection interface to complete the wargaming touch control. This can be, but is not limited to, decomposed into the following steps S41-S44, specifically including: S41. Based on the action object in the action recognition result of the target operator, select the corresponding war game simulation operator or the corresponding war game simulation distribution map from the war game simulation projection interface as the target war game simulation object. S42. Based on the action pattern in the action recognition result of the target operator, determine the simulation update parameters to be generated, wherein the simulation update parameters to be generated include object movement distance parameters, object movement direction parameters, object combat decision parameters, object placement position parameters, object pickup position parameters and / or object scaling ratio parameters; S43. Calculate the simulation update parameters to be generated based on the action position in the action recognition result of the target operator to obtain the simulation update parameters corresponding to the target wargame simulation object; S44. Based on the simulation update parameters corresponding to the target wargaming object, the target wargaming object is updated by projection in the wargaming projection interface to complete the wargaming touch control.

[0036] In one possible implementation, step S4, after completing the wargaming simulation touch control, may also include, but is not limited to, the following steps S45-S46, specifically: S45. Integrate the action recognition results of the wargaming touch control, the target wargaming object, and the wargaming update parameters to form a corresponding wargaming touch control record; S46. Integrate the touch records of each wargaming simulation to update the wargaming data record, forming an updated wargaming data record.

[0037] It should be noted that the wargaming touch control method provided in this embodiment realizes intelligent adjudication assistance and electronic storage and review of wargaming. Through the action recognition results, the target wargaming object and the wargaming update parameters are analyzed to execute the instructions issued by the target operator in real time, complete the target operator's wargaming operation intention, and automatically complete operations such as moving, scaling, placing, picking up and engaging in combat of the target wargaming object. The wargaming update parameters calculated through the action recognition results can include, but are not limited to, distance calculation, combat result adjudication and other steps that traditionally require manual querying of rule tables. Each operation record is processed into structured data that is easy to analyze and store, so as to support the complete playback and review analysis of the wargaming process after the completion of the complete wargaming operation, and provide data support for tactical research.

[0038] like Figure 2 As shown, the second aspect of this embodiment provides a hardware system for implementing the wargaming touch control method based on motion capture and projection fusion described in the first aspect of the embodiment, including: The wargaming projection interface generation unit is used to acquire preset wargaming materials and, based on projection fusion technology, project the wargaming materials onto the target wargaming area to form a wargaming projection interface. The wargaming materials include multiple wargaming operators and multiple wargaming distribution maps. The operator motion information extraction unit is used to capture the motion of the target operator based on the war game projection interface to obtain the motion video stream of the target operator, establish an operator motion model based on the motion video stream of the target operator, and extract motion information from the motion video stream of the target operator according to the operator motion model to obtain operator motion information. The action recognition result calculation unit is used to obtain a pre-trained operator action recognition model, input the operator action information of the target operator into the operator action recognition model, perform real-time action recognition, and obtain the action recognition result of the target operator, wherein the action recognition result includes the action object, action position and action pattern; The wargaming touch execution unit is used to select a target wargaming object based on the action recognition result of the target operator, calculate the wargaming update parameters corresponding to the target wargaming object, and perform projection update on the target wargaming object in the wargaming projection interface based on the wargaming update parameters corresponding to the target wargaming object, thereby completing the wargaming touch control.

[0039] The working process, working details and technical effects of the system provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0040] like Figure 3As shown, the third aspect of this embodiment provides an electronic device, including: a memory, a processor, and a transceiver that are sequentially and communicatively connected, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the wargaming simulation touch control method based on motion capture and projection fusion as described in the first aspect of the embodiment.

[0041] For specific examples, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out (FIFO) memory, and / or first-in-last-out (FILO) memory, etc.; specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor, also known as the CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0042] In some embodiments, the processor may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. For example, the processor may not be limited to microprocessors of the STM32F105 series, reduced instruction set computer (RISC) microprocessors, x86 architecture processors, or processors with integrated neural network processing units (NPUs). The transceiver may be, but is not limited to, a Wi-Fi transceiver, a Bluetooth transceiver, a General Packet Radio Service (GPRS) transceiver, a ZigBee transceiver (a low-power LAN protocol based on the IEEE 802.15.4 standard), a 3G transceiver, a 4G transceiver, and / or a 5G transceiver. Furthermore, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0043] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0044] The fourth aspect of this embodiment provides a storage medium that stores instructions containing the wargaming touch control method based on motion capture and projection fusion as described in the first aspect of the embodiment. That is, the storage medium stores instructions that, when executed on a computer, perform the wargaming touch control method based on motion capture and projection fusion as described in the first aspect of the embodiment.

[0045] The storage medium refers to a carrier for storing data, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0046] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment, and will not be repeated here.

[0047] The fifth aspect of this embodiment provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the wargaming touch control method based on motion capture and projection fusion as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0048] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A touch-based method for wargaming simulation based on motion capture and projection fusion, characterized in that, include: Acquire preset wargaming materials and, based on projection fusion technology, project the wargaming materials onto the target wargaming area to form a wargaming projection interface. The wargaming materials include multiple wargaming operators and multiple wargaming distribution maps. Based on the war game projection interface, motion capture is performed on the target operator to obtain the target operator's motion video stream. An operator motion model is established based on the target operator's motion video stream, and motion information is extracted from the target operator's motion video stream according to the operator motion model to obtain the operator's motion information. A pre-trained operator action recognition model is obtained, and the operator action information of the target operator is input into the operator action recognition model to perform real-time action recognition to obtain the action recognition result of the target operator. The action recognition result includes the action object, action position, and action pattern. Based on the action recognition results of the target operator, the target wargaming object is selected, and the corresponding wargaming update parameters are calculated. Based on the corresponding wargaming update parameters, the target wargaming object is projected and updated in the wargaming projection interface to complete the wargaming touch control.

2. The touch control method for wargaming simulation based on motion capture and projection fusion according to claim 1, characterized in that, Acquire preset wargaming materials and, based on projection fusion technology, project the wargaming materials onto the target wargaming area to form a wargaming projection interface, including: Multiple preset wargaming operators and multiple wargaming distribution maps are obtained as wargaming materials, and the initial position information of each wargaming operator and the map code of each wargaming distribution map are obtained. Using short-throw projection technology, according to the map codes of each war game simulation distribution map, the war game simulation distribution maps are sequentially projected and stitched together in the target simulation area to form an initial war game simulation projection interface. The initial wargaming projection interface is calibrated in a plane, and using projection fusion technology, each wargaming operator is projected onto the initial wargaming projection interface that has completed the plane calibration according to the initial position information of each wargaming operator, thus obtaining the wargaming projection interface.

3. The touch-based wargaming simulation method based on motion capture and projection fusion according to claim 1, characterized in that, Based on the wargaming projection interface, motion capture is performed on the target operator to obtain a motion video stream of the target operator. An operator motion model is established based on the target operator's motion video stream, and motion information is extracted from the target operator's motion video stream according to the operator motion model to obtain operator motion information, including: The interface size parameters of the wargaming projection interface are obtained, the target operator's field of view is calculated based on the interface size parameters of the wargaming projection interface, and the target operator's motion is captured based on the target operator's field of view to obtain the target operator's original motion video stream. The original motion video stream of the target operator is subjected to distortion removal processing and effective extrapolation interval cropping to obtain the motion video stream of the target operator; A preset motion feature localization model is obtained, and the motion video stream of the target operator is used as input and input frame by frame into the motion feature localization model so that the motion feature localization model can output the motion feature position corresponding to each frame of the motion video stream. Based on the action feature position corresponding to each frame of the action video stream, the operator's action region is cropped for each frame of the target operator's action video stream to obtain the action positioning region map corresponding to each frame of the image. The key action nodes of the action localization region map corresponding to each frame image are extracted to obtain the key action node features of the target operator corresponding to each frame image. Coordinate values ​​are extracted from the key node features of the target operator's actions corresponding to each frame of the image, so as to extract the coordinate values ​​of each key node of the operator's actions in each frame of the image. Obtain the preset connection order of key action nodes, and according to the connection order of key action nodes, connect the coordinate values ​​of each key action node of the operator in each frame of the image in sequence to obtain the operator action model corresponding to each frame of the image. Based on the operator action model corresponding to each frame image, action information is extracted from each frame image to obtain the action coordinate information and action posture information of the target operator in each frame image. These are then integrated into the action coordinate information and action posture information of the target operator in each frame image to form operator action information.

4. The touch-based wargaming simulation method based on motion capture and projection fusion according to claim 1, characterized in that, Pre-training methods for operator action recognition models include: Collect multiple effective simulation actions from the war game simulation data records, and integrate the effective simulation actions of each operator into a war game simulation action library; Original operation images and wargame simulation operation intentions are collected from multiple operators. The collected original operation images are enhanced to obtain operation image samples. The operation image samples are matched with the collected wargame simulation operation intentions to form model training samples. A preset image recognition base model is obtained, and the model parameters of the image recognition base model are trained in multiple rounds using the model training samples to obtain an operator action recognition model.

5. The touch control method for wargaming simulation based on motion capture and projection fusion according to claim 1, characterized in that, The operator's action information is input into the operator action recognition model for real-time action recognition to obtain the target operator's action recognition result, including: The operator's action information is used as input to the operator action recognition model, so that the corresponding operator action recognition result is output through the operator action recognition model, and the operator action recognition result is used as the operator's wargame simulation operation intention. Based on the operator's wargaming operation intention, the real-time wargaming operation object of the target operator is identified as the action object, wherein the action object is the corresponding wargaming operator or the corresponding wargaming distribution map. Based on the operator's wargaming operation intention, the real-time wargaming operation position of the target operator is identified as the action position, wherein the action position includes the starting position of the action object and the operation target position of the action object; Based on the operator's wargaming operation intention, the real-time wargaming operation mode of the target operator is identified as an action mode, wherein the action mode includes at least object selection, object picking, object dragging and object releasing. The action object, the action location, and the action pattern are integrated as the action recognition result.

6. The touch control method for wargaming simulation based on motion capture and projection fusion according to claim 1, characterized in that, Based on the action recognition results of the target operator, a target wargaming object is selected, and the corresponding wargaming update parameters are calculated. Based on the corresponding wargaming update parameters, the target wargaming object is projected and updated on the wargaming projection interface to complete the wargaming touch control, including: Based on the action object in the action recognition result of the target operator, the corresponding war game simulation operator or the corresponding war game simulation distribution map is selected from the war game simulation projection interface as the target war game simulation object. Based on the action pattern in the action recognition result of the target operator, the inference update parameters to be generated are determined, wherein the inference update parameters to be generated include object movement distance parameters, object movement direction parameters and / or object scaling ratio parameters; Based on the action position in the action recognition result of the target operator, the simulation update parameters to be generated are calculated to obtain the simulation update parameters corresponding to the target wargame simulation object; Based on the simulation update parameters corresponding to the target wargaming object, the target wargaming object is projected and updated in the wargaming projection interface to complete the wargaming touch control.

7. The touch control method for wargaming simulation based on motion capture and projection fusion according to claim 1, characterized in that, After completing the wargaming simulation touch control, it also includes: The action recognition results of the wargaming touch control, the target wargaming object, and the wargaming update parameters are integrated to form a corresponding wargaming touch control record; The wargame simulation data records are updated by integrating the touch records of each wargame simulation, resulting in updated wargame simulation data records.

8. A wargaming touchscreen system based on motion capture and projection fusion, characterized in that, The touch-based wargaming simulation method based on motion capture and projection fusion as described in any one of claims 1 to 7 includes: The wargaming projection interface generation unit is used to acquire preset wargaming materials and, based on projection fusion technology, project the wargaming materials onto the target wargaming area to form a wargaming projection interface. The wargaming materials include multiple wargaming operators and multiple wargaming distribution maps. The operator motion information extraction unit is used to capture the motion of the target operator based on the war game projection interface to obtain the motion video stream of the target operator, establish an operator motion model based on the motion video stream of the target operator, and extract motion information from the motion video stream of the target operator according to the operator motion model to obtain operator motion information. The action recognition result calculation unit is used to obtain a pre-trained operator action recognition model, input the operator action information of the target operator into the operator action recognition model, perform real-time action recognition, and obtain the action recognition result of the target operator, wherein the action recognition result includes the action object, action position and action pattern; The wargaming touch execution unit is used to select a target wargaming object based on the action recognition result of the target operator, calculate the wargaming update parameters corresponding to the target wargaming object, and perform projection update on the target wargaming object in the wargaming projection interface based on the wargaming update parameters corresponding to the target wargaming object, thereby completing the wargaming touch control.

9. An electronic device, characterized in that, The device includes a memory, a processor, and a transceiver that are sequentially and communicatively connected. The memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the wargaming simulation touch control method based on motion capture and projection fusion as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instructions are executed by the computer, they implement the wargaming simulation touch control method based on motion capture and projection fusion as described in any one of claims 1 to 7.

Images