An image shooting training system and method based on posture analysis

Abstract

The application discloses an image shooting training system and method based on posture analysis, and the method comprises a hardware subsystem and a software subsystem; the hardware subsystem comprises an image acquisition module, a laser simulator, a bullet point acquisition module, a projector device and a guide computer, and is used for acquiring image information set and laser signals of a user shooting a target by using the laser simulator; the software subsystem comprises a shooting training management module, a training scene editing module, a shooting subject management module and a shooting evaluation analysis module, and is used for comprehensively evaluating and processing the image information set and the laser signals to obtain user shooting evaluation result information; the user shooting evaluation result information is used for representing the standardization of the shooting posture of the user; and the software subsystem is installed on the guide computer. The application is favorable for the real-time analysis and feedback of the shooting posture and shooting action of a trainer, thereby comprehensively guiding the shooting posture and action accuracy of the trainer, and meanwhile, the editing and adjustment of a system training scene can be realized quickly and conveniently.

Description

Technical Field

[0001] This invention relates to the field of virtual simulation technology, and in particular to an image-based shooting training system and method based on posture analysis. Background Technology

[0002] Currently, traditional video-based shooting training systems primarily utilize laser-simulated guns and projection systems for simulated shooting training. However, during shooting practice, standardized shooting movements significantly impact accuracy. Existing video-based shooting training systems lack real-time analysis and feedback on trainees' shooting posture and movements, making it difficult to comprehensively guide trainees in ensuring accurate shooting posture and movements. Furthermore, editing and adjusting training scenarios in existing systems typically requires substantial time and manpower. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an image shooting training system and method based on posture analysis, which is conducive to real-time analysis and feedback of the trainee's shooting posture and shooting action, thereby comprehensively guiding the trainee's shooting posture and action accuracy, while also enabling quick and convenient editing and adjustment of the system training scene.

[0004] To address the aforementioned technical problems, the first aspect of this invention discloses an image shooting training system based on attitude analysis, comprising: a hardware subsystem and a software subsystem; The hardware subsystem includes an image acquisition module, a laser simulator, a bullet point acquisition module, a projector device, and a guidance computer, used to acquire image information sets and laser signals from the user shooting at the target using the laser simulator; the image information set includes image information recording the entire process from the user aiming at the target to shooting at the target using the laser simulator; the laser signal is emitted by the user after shooting at the target using the laser simulator. The software subsystem includes a shooting training management module, a training scenario editing module, a shooting subject management module, and a shooting evaluation and analysis module, which are used to comprehensively evaluate and process the image information set and laser signals to obtain user shooting evaluation result information; the user shooting evaluation result information is used to characterize the standardization of the user's shooting action posture; the software subsystem is installed on the directing computer.

[0005] The image acquisition module is used to acquire a set of image information of the user shooting at the target using a laser simulator, and send the set of image information to the command and control computer; The laser simulator is used to receive the user's firing command and, according to the firing command, emit a laser signal to the bullet point acquisition module; The bullet point acquisition module is used to acquire and record the intersection point of the laser signal and the screen, including the screen and the high-speed industrial camera; The high-speed industrial camera is used to acquire hit image information of laser signals emitted onto the screen at a set frame rate, and to extract the target position coordinates from the hit image information. The high-speed industrial camera includes a narrowband filter; the narrowband filter is used to perform spectral filtering on signals outside the laser signal frequency band to obtain signals within the laser signal frequency band. The projector device is used to receive the information to be displayed sent by the directing computer and to project and display the information to be displayed. The guidance computer is used to provide a hardware operating environment for the software subsystem; send the received image information set and target position coordinates to the shooting evaluation and analysis module; and send the received information to be displayed to the projector device.

[0006] The shooting training management module is used to implement user management and performance management; the user management is to manage all users of the posture analysis-based image shooting training system, including functions such as adding new users, batch importing users, deleting users, and modifying user information. The performance management refers to the management of users' training performance, including functions for querying performance, deleting performance, and sorting performance based on criteria. The training scene editing module includes a scene editing and generation sub-module, a scene model library, and a simulation effects library; The scene model library includes model information for various types of scenes or devices; The simulation effects library includes sound effects information, behavioral action effects information, and lighting effects information; The scene editing and generation submodule is used to select corresponding scene model information from the scene model library and corresponding special effects information from the simulation special effects library according to the user scene instruction information, and send the model information and special effects information as information to be displayed to the directing computer. The shooting subject management module is used to manage shooting subjects, select the corresponding shooting subject as the information to be displayed according to the user's shooting subject instruction information, and send the information to be displayed to the directing computer; The shooting evaluation and analysis module is used to evaluate and analyze the received image information set and the target position coordinates to obtain user shooting evaluation result information.

[0007] The process of evaluating and analyzing the received image information set and the target position coordinates to obtain user shooting evaluation result information includes: For each image in the image information set, pose analysis is performed to extract the human body key point location information for each image; A motion description matrix is ​​constructed using the human body key point location information of all images; the row vectors of the motion description matrix are the human body key point location information of an image. The action description matrix is ​​compared with the standard action matrix to obtain a set of action difference evaluation values. The standard action matrix and target position coordinates are stored in the shooting evaluation and analysis module; the row vector of the standard action matrix is ​​the position information of key human body points in the image information acquired by the image acquisition module for the standard shooting posture. The difference between the target position coordinates and the target point position coordinates is calculated to obtain the miss distance value. The set of motion difference evaluation values ​​and the miss value are sent to the control computer as information to be displayed. The set of action difference evaluation values ​​and the miss distance value are confirmed as user shooting evaluation result information.

[0008] The difference discrimination process includes: The motion description matrix and the standard motion matrix are compared to perform motion difference evaluation to obtain user correction information; The action description matrix and the standard action matrix are comprehensively evaluated to obtain the user action evaluation value; The user correction information and user action evaluation values ​​are merged to obtain a set of action difference evaluation values.

[0009] The step of performing action difference evaluation processing on the action description matrix and the standard action matrix to obtain user correction information includes: The difference matrix is ​​obtained by subtracting the action description matrix from the standard action matrix item by item. The element in the i-th row and j-th column of the difference matrix is ​​obtained by subtracting the element in the i-th row and j-th column of the action description matrix from the element in the i-th row and j-th column of the standard action matrix. Factor normalization is performed on the difference matrix to obtain the standardized matrix. ; The calculation expression for the factor normalization process is as follows: , in, Represents the normalized matrix The element in the i-th row and j-th column, Let m represent the element in the i-th row and j-th column of the difference matrix, where m and n are the number of rows and columns of the difference matrix, respectively. Determine the normalization matrix The row vector of the user correction information represents the user's action deviation value in the corresponding image information acquired by the image acquisition module.

[0010] The process of comprehensively evaluating the action description matrix and the standard action matrix to obtain the user action evaluation value includes: A reference vector is obtained by performing a reference calculation on the action description matrix and the standard action matrix; The expression for the reference calculation is: , in, Action description matrix The element in the i-th row and j-th column, Standard Action Matrix The element in the i-th row and j-th column, This refers to the j-th element of the reference vector; For the standardized matrix and reference vector Perform relevant calculations to obtain the correlation matrix. The calculation expression for the related calculation process is: , in, This represents the element in the i-th row and j-th column of the correlation matrix. For calculation factors; Perform column optimization processing on the correlation matrix to obtain the optimal vector; The optimal processing of the columns involves extracting the largest number from each column to form the optimal vector. The expression for the optimal vector is: , in, Represents the j-th term of the optimal vector; Perform column worst-case processing on the correlation matrix to obtain the worst-case vector; The worst-case processing of the columns involves extracting the smallest number from each column to form the worst-case vector. The expression for the worst-case vector is: , in, This represents the j-th term of the worst-case vector; The worst and best vectors are scored to obtain the user action evaluation value s; the expression for the scoring calculation is: , In the formula, s is the user action evaluation value.

[0011] In a second aspect of this embodiment, a posture analysis-based image shooting training method is disclosed, implemented using the posture analysis-based image shooting training device, comprising: S1, acquire user scenario command information and user shooting subject command information; S2, based on user scenario instruction information, select corresponding model information from the scenario model library, select corresponding special effects information from the simulation special effects library, and display the model information and feature information using the projector device; S3, based on the user's shooting subject instruction information, select the corresponding shooting subject and display the shooting subject using the projector device; S4, using the hardware subsystem, acquire the image information set and laser signal of the user shooting at the target using the laser simulator; S5, perform comprehensive evaluation and processing on the image information set and laser signal to obtain user shooting evaluation result information.

[0012] The process of comprehensively evaluating and processing the image information set and laser signal to obtain user shooting evaluation result information includes: For each image in the image information set, pose analysis is performed to extract the human body key point location information for each image; A motion description matrix is ​​constructed using the human body key point location information of all images; the row vectors of the motion description matrix are the human body key point location information of an image. The action description matrix is ​​compared with the standard action matrix to obtain a set of action difference evaluation values. The standard action matrix and target position coordinates are stored in the shooting evaluation and analysis module; the row vectors of the standard action matrix are the human body key point position information in the image information acquired by the image acquisition module for the standard shooting posture; the row vectors of the standard action matrix are sorted in the standard action matrix according to the chronological order of the entire process from the user aiming at the target to shooting the target with the laser simulator. The difference between the target position coordinates and the target point position coordinates is calculated to obtain the miss distance value. The set of motion difference evaluation values ​​and the miss value are sent to the control computer as information to be displayed. The set of action difference evaluation values ​​and the miss distance value are confirmed as user shooting evaluation result information.

[0013] The difference discrimination process includes: The motion description matrix and the standard motion matrix are compared to perform motion difference evaluation to obtain user correction information; The action description matrix and the standard action matrix are comprehensively evaluated to obtain the user action evaluation value; The user correction information and user action evaluation values ​​are merged to obtain a set of action difference evaluation values.

[0014] The beneficial effects of this invention are as follows: This invention provides an image-based shooting training system and method based on posture analysis, which is beneficial for real-time analysis and feedback of the trainee's shooting posture and shooting actions, thereby comprehensively guiding the trainee's shooting posture and action accuracy, while also enabling quick and convenient editing and adjustment of the system's training scenarios.

[0015] This invention evaluates user images to obtain user correction information, enabling users to correct their actions in real time during training based on the correction information, thereby improving training efficiency.

[0016] This invention discloses an image shooting training system and method based on posture analysis, which can analyze the trainee's shooting posture and standard shooting movements in real time and provide personalized guidance to help trainees master the correct shooting posture and continuous shooting movements, improve the standard and accuracy of shooting, and also realize the instant editing of training scenes, which greatly saves the time and manpower costs of scene editing.

[0017] This invention utilizes image recognition technology and a camera to capture real-time image data of the shooting posture and sequence of shooting actions during each shooting session. This image data is then transmitted to a command computer for analysis. Through posture analysis technology, the computer extracts image features from the captured image data using posture analysis algorithms, including the trainee's posture, key skeletal point positions, and key point movement trajectories. The system compares and analyzes these image features with standard shooting postures and shooting action sequences, recording the data and providing evaluation and guidance after training. Furthermore, the system features real-time scene editing capabilities, allowing trainees to edit and adjust the training scene in real-time via a graphical user interface. This includes adding, deleting, and moving targets, and adjusting obstacle positions. The system employs efficient graphics processing algorithms and an interactive design, enabling rapid response and scene updates for trainees, thus achieving real-time scene editing. Therefore, this invention helps solve the problems of existing video shooting training systems, such as the lack of real-time shooting posture analysis and feedback, and the time-consuming and laborious nature of training scene editing. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention; Figure 2 This is a schematic diagram illustrating the composition principle of the device of the present invention. Detailed Implementation

[0019] To better understand the content of this invention, an embodiment is provided here.

[0020] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention; Figure 2 This is a schematic diagram illustrating the composition principle of the device of the present invention.

[0021] To address the problems of existing video shooting training systems, such as the lack of real-time analysis and feedback of shooting posture and the time-consuming and laborious editing of training scenes, this invention aims to provide a video shooting training system and method based on posture analysis. This system can analyze the trainee's shooting posture and standard shooting movements in real time and provide personalized guidance, helping trainees master the correct shooting posture and continuous shooting movements, improving the standardization and accuracy of shooting. It can also enable instant editing of training scenes, greatly saving time and manpower costs in scene editing.

[0022] The first aspect of this invention discloses an image shooting training system based on attitude analysis, comprising: a hardware subsystem and a software subsystem; The hardware subsystem includes an image acquisition module, a laser simulator, a bullet point acquisition module, a projector device, and a guidance computer, used to acquire image information sets and laser signals of the user shooting at the target using the laser simulator; the image information set includes image information recording the entire process from the user aiming at the target to shooting at the target using the laser simulator; The laser signal is emitted by the user after shooting at the target using a laser simulator. The software subsystem includes a shooting training management module, a training scenario editing module, a shooting subject management module, and a shooting evaluation and analysis module, which are used to comprehensively evaluate and process the image information set and laser signals to obtain user shooting evaluation result information; the user shooting evaluation result information is used to characterize the standardization of the user's shooting action posture; the software subsystem is installed on the directing computer.

[0023] The image acquisition module is used to acquire a set of image information of the user shooting at the target using a laser simulator, and send the set of image information to the command and control computer; The laser simulator is used to receive the user's firing command and, according to the firing command, emit a laser signal to the bullet point acquisition module; The bullet point acquisition module is used to acquire and record the intersection point of the laser signal and the screen, including the screen and a high-speed industrial camera; the high-speed industrial camera is used to acquire the hit image information of the laser signal emitted onto the screen at a set frame rate, and extract the target position coordinates from the hit image information. The high-speed industrial camera includes a narrowband filter; the narrowband filter is used to perform spectral filtering on signals outside the laser signal frequency band to obtain signals within the laser signal frequency band; so that the signals within the laser signal frequency band can be used for subsequent intersection point extraction operations.

[0024] The acquisition of the impact image information of the laser signal emitted onto the screen, the extraction of the target position coordinates from the impact image information, and the implementation using an edge detection algorithm specifically include: The image region with a gray value higher than a set gray value threshold in the hit image information is acquired; the image region is determined as the target region; the coordinates of the center pixel in the target region in the image two-dimensional coordinate system are determined as the target position coordinates; The projector device is used to receive the information to be displayed sent by the directing computer and to project and display the information to be displayed. The command and control computer is used to provide a hardware operating environment for the software subsystem; send the received image information set and target position coordinates to the shooting evaluation and analysis module; and send the received information to be displayed to the projector device. The shooting training management module is used to implement user management and performance management; the user management is to manage all users of the posture analysis-based image shooting training system, including functions such as adding new users, batch importing users, deleting users, and modifying user information. The performance management refers to the management of users' training performance, including functions such as querying performance, deleting performance, and sorting performance based on criteria.

[0025] The sorting of scores includes sorting from highest to lowest. A user's training score is determined by their shooting evaluation results.

[0026] The training scene editing module includes a scene editing and generation sub-module, a scene model library, and a simulation effects library; The scene model library includes grassland scene models, jungle scene models, mountain scene models, desert scene models, city street scene models, island scene models, village scene models, station scene models, airport scene models, hotel scene models, subway scene models, office building scene models, shopping mall scene models, restaurant scene models, residential building scene models, hospital scene models, warehouse scene models, factory scene models, ruin scene models, outdoor shooting range scene models, as well as city road models, transportation vehicle models, auxiliary equipment models, equipment and facilities models, and character models of iconic urban areas with a high sense of realism.

[0027] The simulation effects library includes sound effects information, behavioral action effects information, lighting effects information, etc. The sound effects information is used to simulate a realistic shooting environment, including sound and scene effects, and the sound effects produced by shooting hits; the lighting effects information includes weather, time of day, lighting, season, and lighting effects in the scene; the behavioral action effects information includes behavioral action effects of various entities. Entities include users, characters in the scene, and moving vehicles.

[0028] The scene editing and generation submodule is used to select corresponding model information from the scene model library and corresponding special effects information from the simulation special effects library according to the user scene instruction information, and send the model information and special effects information as information to be displayed to the directing computer. The shooting subject management module is used to manage shooting subjects. Based on the user's shooting subject instruction information, it selects the corresponding shooting subject as the information to be displayed and sends it to the directing computer.

[0029] The shooting evaluation and analysis module is used to evaluate and analyze the received image information set and the target position coordinates to obtain user shooting evaluation result information.

[0030] The process of evaluating and analyzing the received image information set and the target position coordinates to obtain user shooting evaluation result information includes: For each image in the image information set, pose analysis is performed to extract the human body key point location information for each image; A motion description matrix is ​​constructed using the human body key point location information of all images; the row vectors of the motion description matrix are the human body key point location information of an image. The action description matrix is ​​compared with the standard action matrix to obtain a set of action difference evaluation values. The standard action matrix and target position coordinates are stored in the shooting evaluation and analysis module; the row vectors of the standard action matrix are the human body key point position information in the image information acquired by the image acquisition module; the row vectors of the standard action matrix are sorted in the standard action matrix according to the chronological order of the standard shooting posture from the user aiming at the target to shooting the target with the laser simulator. The difference between the target position coordinates and the target point position coordinates is calculated to obtain the miss distance value. The set of motion difference evaluation values ​​and the miss value are sent to the control computer as information to be displayed. The set of action difference evaluation values ​​and the miss distance value are confirmed as user shooting evaluation result information.

[0031] The difference calculation can be to obtain the absolute value of the difference between the target position coordinates and the target point position coordinates.

[0032] The difference discrimination process includes: The motion description matrix and the standard motion matrix are compared to perform motion difference evaluation to obtain user correction information; The action description matrix and the standard action matrix are comprehensively evaluated to obtain the user action evaluation value; The user correction information and user action evaluation values ​​are merged to obtain a set of action difference evaluation values; The step of performing action difference evaluation processing on the action description matrix and the standard action matrix to obtain user correction information includes: The difference matrix is ​​obtained by subtracting the action description matrix from the standard action matrix item by item. The element in the i-th row and j-th column of the difference matrix is ​​obtained by subtracting the element in the i-th row and j-th column of the action description matrix from the element in the i-th row and j-th column of the standard action matrix. Factor normalization is performed on the difference matrix to obtain the standardized matrix. ; The calculation expression for the factor normalization process is as follows: , in, Represents the normalized matrix The element in the i-th row and j-th column, Let m represent the element in the i-th row and j-th column of the difference matrix, where m and n are the number of rows and columns of the difference matrix, respectively. Determine the normalization matrix The row vector of the user correction information represents the user's action deviation value in the corresponding image information acquired by the image acquisition module, and is used to correct the user's designed actions. The process of comprehensively evaluating the action description matrix and the standard action matrix to obtain the user action evaluation value includes: A reference vector is obtained by performing a reference calculation on the action description matrix and the standard action matrix; The expression for the reference calculation is: , in, Action description matrix The element in the i-th row and j-th column, Standard Action Matrix The element in the i-th row and j-th column, It is the j-th element of the reference vector.

[0033] For the standardized matrix and reference vector Perform relevant calculations to obtain the correlation matrix. The calculation expression for the related calculation process is: , in, This represents the element in the i-th row and j-th column of the correlation matrix. This is a calculation factor, and its value can be 0.5.

[0034] Perform column optimization processing on the correlation matrix to obtain the optimal vector; The optimal processing of the columns involves extracting the largest number from each column to form the optimal vector. The expression for the optimal vector is: , in, Represents the j-th term of the optimal vector; Perform column worst-case processing on the correlation matrix to obtain the worst-case vector; The worst-case processing of the columns involves extracting the smallest number from each column to form the worst-case vector. The expression for the worst-case vector is: , in, This represents the j-th term of the worst-case vector; The worst and best vectors are scored to obtain the user action evaluation value s; the expression for the scoring calculation is: , In the formula, s is the user action evaluation value.

[0035] The step of calculating the difference between the target position coordinates and the target point position coordinates to obtain the miss distance value includes: The target position coordinates of several shots fired by the user are collected; the difference between the target position coordinates and the target point coordinates of each shot is calculated; and an error vector is constructed using all the difference values. The error vector is processed by normalizing the entropy value to obtain the miss distance value; The normalized entropy value calculation process is expressed as follows: , in, for The normalized value, Let M1 be the i-th term of the error vector, and M1 be the number of shots fired by the user. This is the off-target value.

[0036] The pose analysis process extracts the human key point location information for each image, including: After the directing computer 1015 receives the image data of the shooting posture and action sequence of the person transmitted from the image acquisition module 1011, it inputs the image to be processed into the posture analysis algorithm through the AddImage() function; it uses the detectPeople() human body detection function to detect the position of the human body in the image; it uses the estimateKeyPoints() key point detection function to perform key point detection on the detected human body; and it uses the estimatePoses() posture inference function to infer the posture of the human body and connect them into a skeleton to obtain the shooting posture and the action sequence during the shooting process.

[0037] For difference discrimination, the PoseContrast() posture comparison function can be used to compare and analyze the shooting posture and shooting action sequence with the standard shooting posture. The functions mentioned above are all general functions in the OpenCV framework.

[0038] The posture analysis and processing is implemented through human posture analysis and processing algorithms, specifically through PoseC3D. PoseC3D is a skeletal behavior recognition framework based on 3D-CNN.

[0039] A second aspect of this invention discloses an image-based shooting training method based on attitude analysis, implemented using the attitude-based image-based shooting training device, comprising: Obtain user scenario command information and user shooting subject command information; Based on user scenario command information, select corresponding model information from the scenario model library and corresponding special effects information from the simulation special effects library, and display the model information and feature information using the projector device; Based on the user's shooting subject instruction information, the corresponding shooting subject is selected, and the shooting subject is displayed using the projector device; Using the aforementioned hardware subsystem, image information sets and laser signals of the user shooting at a target using a laser simulator are acquired; The image information set and laser signal are comprehensively evaluated and processed to obtain the user's shooting evaluation result information.

[0040] The step of performing action difference evaluation processing on the action description matrix and the standard action matrix to obtain user correction information includes: The difference matrix is ​​obtained by subtracting the action description matrix from the standard action matrix item by item. The element in the i-th row and j-th column of the difference matrix is ​​obtained by subtracting the element in the i-th row and j-th column of the action description matrix from the element in the i-th row and j-th column of the standard action matrix. Factor normalization is performed on the difference matrix to obtain the standardized matrix. ; The calculation expression for the factor normalization process is as follows: , in, Represents the normalized matrix The element in the i-th row and j-th column, Let m represent the element in the i-th row and j-th column of the difference matrix, where m and n are the number of rows and columns of the difference matrix, respectively. Determine the normalization matrix The row vector of the user correction information represents the user's action deviation value in the corresponding image information acquired by the image acquisition module, and is used to correct the user's designed actions. The process of comprehensively evaluating the action description matrix and the standard action matrix to obtain the user action evaluation value includes: A reference vector is obtained by performing a reference calculation on the action description matrix and the standard action matrix; The expression for the reference calculation is: , in, Action description matrix The element in the i-th row and j-th column, Standard Action Matrix The element in the i-th row and j-th column, It is the j-th element of the reference vector.

[0041] For the standardized matrix and reference vector Perform relevant calculations to obtain the correlation matrix. The calculation expression for the related calculation process is: , in, This represents the element in the i-th row and j-th column of the correlation matrix. This is a calculation factor, and its value can be 0.5.

[0042] Perform column optimization processing on the correlation matrix to obtain the optimal vector; The optimal processing of the columns involves extracting the largest number from each column to form the optimal vector. The expression for the optimal vector is: , in, Represents the j-th term of the optimal vector; Perform column worst-case processing on the correlation matrix to obtain the worst-case vector; The worst-case processing of the columns involves extracting the smallest number from each column to form the worst-case vector. The expression for the worst-case vector is: , in, This represents the j-th term of the worst-case vector; The worst and best vectors are scored to obtain the user action evaluation value s; the expression for the scoring calculation is: , In the formula, s is the user action evaluation value.

[0043] The step of calculating the difference between the target position coordinates and the target point position coordinates to obtain the miss distance value includes: The target position coordinates of several shots fired by the user are collected; the difference between the target position coordinates and the target point coordinates of each shot is calculated; and an error vector is constructed using all the difference values. The error vector is processed by normalizing the entropy value to obtain the miss distance value; The normalized entropy value calculation process is expressed as follows: , in, for The normalized value, Let M1 be the i-th term of the error vector, and M1 be the number of shots fired by the user. This is the off-target value.

[0044] Example 2: This embodiment discloses a posture analysis-based image shooting training system, comprising a hardware module 101 and a software module 102. The hardware module 101 includes an image acquisition module 1011, a laser simulator 1012, a bullet point acquisition module 1013, a projector device 1014, and a directing computer 1015. The software module 102 includes shooting training management 1021, training scene editing 1022, shooting subject management 1023, and shooting evaluation and analysis 1024. Trainees use the laser simulator 1012 to shoot at a projected figure. The system uses the image acquisition module 1011 to collect shooting posture image data of the trainee during each shot and transmits it to the directing computer 1015. The directing computer 1015 uses a posture analysis algorithm 10241 to identify key points of the human body and compare them with standard shooting postures. After training, it provides evaluation and guidance. The bullet point acquisition module 1013 filters and acquires laser points hitting the screen and transmits them to the directing computer for coordinate distance calculation. The hit rate is calculated by comparing the distance between the bullet point and the target. The software module 102 is deployed on the directing computer 1015 and projects the system screen through the projector device 1014. The software module 102 provides training management, scene management, subject management and training evaluation support for the entire system.

[0045] The hardware module 101 includes an image acquisition module 1011, a laser simulator 1012, a bullet point acquisition module 1013, a projector device 1014, and a directing computer 1015. The image acquisition module 1011 acquires shooting posture image data of the trainee during each shot and transmits it to the directing computer 1015. The laser simulator 1012 provides a safe and practical shooting training method for trainees, eliminating the need for actual bullets. Training can be conducted in a controlled environment, reducing the risk of accidental injury. The bullet point acquisition module 1013, composed of a high-speed industrial camera and a narrowband filter, is used to record and analyze the position information of the bullet hitting the target. The projector device 1014 projects the video and images from the directing computer 1015. The data is projected onto the screen. This invention uses three projectors to project the system image onto the left, right, and front screens respectively. By adjusting parameters such as the projector's position, tilt angle, focal length, color, and brightness, the projected image can be accurately projected onto the designated screen. The directing computer 1015 deploys the software module 102 of the image shooting training system and receives image data transmitted from the image acquisition module 1011 and the bullet point acquisition module 1013. It uses a posture analysis algorithm to extract key nodes from the acquired shooting postures of the trainees and compares them with standard shooting postures. Additionally, the image data to be displayed can be segmented according to the screen layout, mapping each segment of image data onto the corresponding projector 1014.

[0046] The software module 102 includes shooting training management 1021, training scenario editing 1022, shooting subject management 1023, and shooting evaluation and analysis 1024. The shooting training management 1021 is used to efficiently and effectively organize shooting simulation training. The system includes user management and performance management; the training scene editing 1022, including a scene model library and simulation effects, allows for secondary editing of scenes in the scene model library, enabling users to select training scenes, set target positions, weather conditions, and camera trajectories according to training needs; the shooting subject management 1023 provides management functions for all shooting training subjects, including basic shooting, applied shooting, competitive shooting, combat shooting, tactical shooting, and person-to-person combat; the shooting evaluation and analysis 1024, on the one hand, compares and analyzes the shooting postures and movements of trainees collected by the image acquisition module 1011 with standard shooting postures and movement sequences using the posture analysis algorithm 10241, and records the results, providing evaluation and guidance after training; on the other hand, it can calculate shooting results in real time during training, and comprehensively evaluate and analyze the trainees' shooting results after training, presenting the evaluation conclusions intuitively through charts, and automatically analyzing problems in the trainees' shooting.

[0047] The shooting evaluation analysis 1024 includes a posture analysis algorithm 10241. The posture analysis algorithm 10241 is an algorithm used to analyze the shooting posture and movement of trainees. After the directing computer 1015 receives the image data of the shooting posture and action sequence transmitted from the image acquisition module 1011, the system inputs the image to be processed into the posture analysis algorithm using the AddImage() function; uses the detectPeople() human body detection function to detect the human body position in the image; uses the estimateKeyPoints() keypoint detection function to perform keypoint detection on the detected human body; uses the estimatePoses() posture inference function to infer the human body posture and connect it into a skeleton to obtain the shooting posture and the action sequence during the shooting process; uses the PoseContrast() posture comparison function to compare and analyze with the standard shooting posture and shooting action sequence, including: using the Euclidean distance method to calculate the distance between the trainee's posture features and the standard posture features. Assuming that the elbow coordinates of person 1 are (x1, y1) and the hand coordinates are (x2, y2), then the Euclidean distance from person 1's elbow to their hand is: We can convert Euclidean distance into a similarity score using the following formula: [\text{Similarity}=\frac{1}{1+\text{Euclidean Distance}}], to obtain the similarity score for person 1. Feedback is provided to trainees based on similarity scores to guide them in improving their shooting posture. Specific suggestions and guidance are offered to help trainees adjust their posture to better meet standards; the Export() function is used to output the results of the comparative analysis.

[0048] The shooting training management system 1021 includes user management 10211 and performance management 10212. User management 10211 is used for managing all users in the system, including functions such as adding new users, batch importing users, deleting users, and modifying user information. Performance management 10212 is used for managing the subject performance of trainees, including functions such as querying, deleting, and sorting based on conditions.

[0049] The training scene editor 1022 includes a scene model library 10221 and simulation effects 10222. The scene model library 10221 provides scene and model resources for the training scene editor 1022, offering multiple sets of 3D scenes, including grasslands, jungles, mountains, deserts, city streets, islands, villages, train stations, airports, hotels, subways, office buildings, shopping malls, restaurants, residential buildings, hospitals, warehouses, factories, ruins, and outdoor shooting ranges. It also provides highly immersive models of iconic urban areas, including road models, vehicle models, auxiliary equipment, facilities, character models, and simulation effects models. The simulation effects 10222 are used to simulate realistic shooting environments and sounds, including weather, time of day, lighting, seasons, the behavior of entities in the scene, scene sound effects, and the sound effects produced by gunfire.

[0050] The shooting subject management 1023 includes basic shooting 10231, applied shooting 10232, competitive shooting 10233, combat shooting 10234, tactical shooting 10235, and person-to-person combat 10236. Basic shooting 10231 provides a basic target shooting training mode, supporting basic training subjects such as fixed targets, falling targets, moving targets, hidden / revealed targets, rising and falling targets, side-facing targets, and half-body ring targets. During shooting training, various training scenarios with different terrains, climates, and lighting conditions can be selected, and parameters such as target distance, target velocity, and ammunition quantity can be set. Applied shooting 10232 includes different training content such as applied shooting of rifles and pistols, combat shooting, shooting at downward angles, and shooting at upward angles. Competitive shooting 10233 can simulate the format and rules of IPSC competitions, allowing trainees to conduct combat simulations and training in different scenarios, improving their shooting skills and ability to cope with complex situations. Combat shooting 10234... It provides a practical human-machine combat training mode, where trainees engage in combat against AI enemies in a virtual environment, honing their shooting skills, tactical thinking, and teamwork abilities; the tactical shooting mode 10235 offers three shooting modes: vehicle-based shooting, boat-based shooting, and aircraft-based shooting, increasing the complexity and challenge of the shooting environment by having trainees shoot from different modes of transportation; the human-versus-human combat mode 10236 is a real-person combat training mode where both red and blue teams are real trainees, and during training, trainees achieve human-versus-human combat training by shooting at images of the opposing character displayed on a screen.

[0051] This embodiment discloses an image-based shooting training method based on attitude analysis, including: S101: Start the image acquisition module 1011, laser simulator 1012, bullet point acquisition module 1013, projector device 1014 and directing computer device 1015; S201: Log in to the directing computer 1015, start the software module 102 and log in to the system with the user account and password. The user management 10211 in the shooting training management 1021 will verify whether the account and password entered by the directing personnel are correct. S301: After logging into the system, the instructor can choose to enter the training scenario editing 1022 to create a new scenario subject, or enter the design subject management 1023 to select any subject from basic shooting 10231, applied shooting 10232, competitive shooting 10233, combat shooting 10234, tactical shooting 10235 and person-to-person confrontation 10236; S401: After entering the scene subject, the shooting subject content can be seen on the projection screen. The trainees use the laser simulator 1012 to shoot at the shooting targets in the scene. The image acquisition module 1011 will collect the shooting posture of the trainees in real time and transmit it to the directing computer 1015. The shooting evaluation and analysis 1024 will use the posture analysis algorithm 10241 to compare and analyze the collected shooting posture with the standard shooting posture, and record the posture evaluation results for each shot.

[0052] S501: The bullet point acquisition module 1013 will collect and filter the laser points shot by the laser simulator 1012 onto the screen, and the directing computer 1015 will calculate the shooting results and provide real-time feedback.

[0053] S601: After training, the shooting evaluation analysis 1024 will evaluate and analyze the shooting results of the trainees and display the shooting scores, posture evaluation results and guidance.

[0054] S701: Shooting evaluation analysis 1024 stores training results, and instructors can view historical training results in results management 10212.

[0055] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A video-based shooting training system based on attitude analysis, characterized in that, include: Hardware subsystems and software subsystems; The hardware subsystem includes an image acquisition module, a laser simulator, a bullet point acquisition module, a projector device, and a guidance computer, used to acquire image information sets and laser signals from the user shooting at the target using the laser simulator; the image information set includes image information recording the entire process from the user aiming at the target to shooting at the target using the laser simulator; the laser signal is emitted by the user after shooting at the target using the laser simulator. The software subsystem includes a shooting training management module, a training scenario editing module, a shooting subject management module, and a shooting evaluation and analysis module, which are used to comprehensively evaluate and process the image information set and laser signals to obtain user shooting evaluation result information; the user shooting evaluation result information is used to characterize the standardization of the user's shooting action posture; The software subsystem is installed on the debugging computer; The shooting training management module is used to implement user management and performance management; the user management is to manage all users of the posture analysis-based image shooting training system, including functions such as adding new users, batch importing users, deleting users, and modifying user information. The performance management refers to the management of users' training performance, including functions for querying performance, deleting performance, and sorting performance based on criteria. The training scene editing module includes a scene editing and generation sub-module, a scene model library, and a simulation effects library; The scene model library includes model information for various types of scenes or devices; The simulation effects library includes sound effects information, behavioral action effects information, and lighting effects information; The scene editing and generation submodule is used to select corresponding scene model information from the scene model library and corresponding special effects information from the simulation special effects library according to the user scene instruction information, and send the model information and special effects information as information to be displayed to the directing computer. The shooting subject management module is used to manage shooting subjects, select the corresponding shooting subject as the information to be displayed according to the user's shooting subject instruction information, and send the information to be displayed to the directing computer; The shooting evaluation and analysis module is used to evaluate and analyze the received image information set and target position coordinates to obtain user shooting evaluation result information, including: For each image in the image information set, pose analysis is performed to extract the human body key point location information for each image; A motion description matrix is ​​constructed using the human body key point location information of all images; the row vectors of the motion description matrix are the human body key point location information of an image. The action description matrix is ​​compared with the standard action matrix to obtain a set of action difference evaluation values. The standard action matrix and target position coordinates are stored in the shooting evaluation and analysis module; the row vector of the standard action matrix is ​​the position information of key human body points in the image information acquired by the image acquisition module for the standard shooting posture. The difference between the target position coordinates and the target point position coordinates is calculated to obtain the miss distance value. The set of motion difference evaluation values ​​and the miss value are sent to the control computer as information to be displayed. Confirm the set of action difference evaluation values ​​and the miss distance value to obtain the user's shooting evaluation result information; The difference discrimination process includes: The motion description matrix and the standard motion matrix are compared to perform motion difference evaluation to obtain user correction information; The action description matrix and the standard action matrix are comprehensively evaluated to obtain the user action evaluation value; The user correction information and user action evaluation values ​​are merged to obtain a set of action difference evaluation values; The step of performing action difference evaluation processing on the action description matrix and the standard action matrix to obtain user correction information includes: The difference matrix is ​​obtained by subtracting the action description matrix from the standard action matrix item by item. The element in the i-th row and j-th column of the difference matrix is ​​obtained by subtracting the element in the i-th row and j-th column of the action description matrix from the element in the i-th row and j-th column of the standard action matrix. The difference matrix is ​​factored and normalized to obtain the standardized matrix. ; The calculation expression for the factor normalization process is as follows: ， in, Represents the normalized matrix The element in the i-th row and j-th column, Let m represent the element in the i-th row and j-th column of the difference matrix, where m and n are the number of rows and columns of the difference matrix, respectively. Determine the normalization matrix The row vector of the user correction information represents the user's action deviation value in the corresponding image information acquired by the image acquisition module.

2. The image-based shooting training system based on attitude analysis as described in claim 1, characterized in that, The image acquisition module is used to acquire a set of image information of the user shooting at the target using a laser simulator, and send the set of image information to the command and control computer; The laser simulator is used to receive the user's firing command and, according to the firing command, emit a laser signal to the bullet point acquisition module; The bullet point acquisition module is used to acquire and record the intersection point of the laser signal and the screen, including the screen and the high-speed industrial camera; The high-speed industrial camera is used to acquire hit image information of laser signals emitted onto the screen at a set frame rate, and to extract the target position coordinates from the hit image information. The high-speed industrial camera includes a narrowband filter; the narrowband filter is used to perform spectral filtering on signals outside the laser signal frequency band to obtain signals within the laser signal frequency band. The projector device is used to receive the information to be displayed sent by the directing computer and to project and display the information to be displayed. The guidance computer is used to provide a hardware operating environment for the software subsystem; send the received image information set and target position coordinates to the shooting evaluation and analysis module; and send the received information to be displayed to the projector device.

3. The image-based shooting training system based on attitude analysis as described in claim 1, characterized in that, The process of comprehensively evaluating the action description matrix and the standard action matrix to obtain the user action evaluation value includes: A reference vector is obtained by performing a reference calculation on the action description matrix and the standard action matrix; The expression for the reference calculation is: ， in, Action description matrix The element in the i-th row and j-th column, Standard Action Matrix The element in the i-th row and j-th column, This refers to the j-th element of the reference vector; For the standardized matrix and reference vector Perform relevant calculations to obtain the correlation matrix. The calculation expression for the related calculation process is: ， in, This represents the element in the i-th row and j-th column of the correlation matrix. For calculation factors; Perform column optimization processing on the correlation matrix to obtain the optimal vector; The optimal processing of the columns involves extracting the largest number from each column to form the optimal vector. The expression for the optimal vector is: ， in, Represents the j-th term of the optimal vector; Perform column worst-case processing on the correlation matrix to obtain the worst-case vector; The worst-case processing of the columns involves extracting the smallest number from each column to form the worst-case vector. The expression for the worst-case vector is: ， in, This represents the j-th term of the worst-case vector; The worst and best vectors are scored to obtain the user action evaluation value s; the expression for the scoring calculation is: ， In the formula, s is the user action evaluation value.

4. A method for image-based shooting training based on attitude analysis, characterized in that, Implemented using the image-based shooting training system based on attitude analysis as described in any one of claims 1 to 3, comprising: S1, acquire user scenario command information and user shooting subject command information; S2, based on user scenario instruction information, select corresponding model information from the scenario model library, select corresponding special effects information from the simulation special effects library, and display the model information and feature information using the projector device; S3, based on the user's shooting subject instruction information, select the corresponding shooting subject and display the shooting subject using the projector device; S4, using the hardware subsystem, acquire the image information set and laser signal of the user shooting at the target using the laser simulator; S5, perform comprehensive evaluation and processing on the image information set and laser signal to obtain user shooting evaluation result information.

5. The image-based shooting training method based on attitude analysis as described in claim 4, characterized in that, The process of comprehensively evaluating and processing the image information set and laser signal to obtain user shooting evaluation result information includes: For each image in the image information set, pose analysis is performed to extract the human body key point location information for each image; A motion description matrix is ​​constructed using the human body key point location information of all images; the row vectors of the motion description matrix are the human body key point location information of an image. The action description matrix is ​​compared with the standard action matrix to obtain a set of action difference evaluation values. The standard action matrix and target position coordinates are stored in the shooting evaluation and analysis module; the row vectors of the standard action matrix are the human body key point position information in the image information acquired by the image acquisition module for the standard shooting posture; the row vectors of the standard action matrix are sorted in the standard action matrix according to the chronological order of the entire process from the user aiming at the target to shooting the target with the laser simulator. The difference between the target position coordinates and the target point position coordinates is calculated to obtain the miss distance value. The set of motion difference evaluation values ​​and the miss value are sent to the control computer as information to be displayed. The set of action difference evaluation values ​​and the miss distance value are confirmed as user shooting evaluation result information.

6. The image-based shooting training method based on attitude analysis as described in claim 5, characterized in that, The difference discrimination process includes: The motion description matrix and the standard motion matrix are compared to perform motion difference evaluation to obtain user correction information; The action description matrix and the standard action matrix are comprehensively evaluated to obtain the user action evaluation value; The user correction information and user action evaluation values ​​are merged to obtain a set of action difference evaluation values.

Images