An image recognition-based simulation training method and system

Abstract

This application discloses a simulation training method and system based on image recognition. The method includes: placing multiple targets in a training area; acquiring target images in the aiming direction of a simulated gun; analyzing the target images to determine whether the simulated gun's firing action hit the target; if the simulated gun hits the target, the hit target is marked by AR glasses worn by the trainee, and the mark is displayed on the AR glasses; and statistically analyzing the data of simulated gun hits to obtain a training result report. Using this method, only traditional targets need to be placed in the training area, and the performance evaluation during the training process is completed by devices worn by the trainee. It has no requirements for the training area, can adapt to various training contents, and allows training in outdoor areas to provide trainees with a near-realistic combat environment, which is beneficial to improving training effectiveness.

Description

Technical Field

[0001] This application relates to the field of military training technology, and in particular to a simulation training method and system based on image recognition. Background Technology

[0002] Among various military training subjects, shooting training is a fundamental skill, and possessing excellent shooting ability is a prerequisite for being a qualified military personnel. Therefore, in order to maintain and even improve personnel's shooting ability, shooting training is frequently conducted in daily military training.

[0003] Traditional shooting training primarily involves live-fire exercises, which not only consumes a large amount of ammunition but also poses significant safety hazards during training. With the widespread adoption of digital technology in military training, simulation training technology has entered routine shooting drills. For example, CN1560556A proposes an ultrasonic imaging shooting training system. This patent uses ultrasonic positioning technology to determine the position of projectiles, offering advantages over traditional live-fire training, including higher accuracy and wider applicability.

[0004] However, the aforementioned patents require the use of numerous electronic devices, which are large in size and can only be installed in fixed indoor locations. Therefore, shooting training can only be conducted indoors, which greatly limits the training content. Moreover, the indoor environment is very different from the real outdoor combat scenario, and trainees cannot experience a near-real combat atmosphere indoors, which is not conducive to improving training effectiveness. Summary of the Invention

[0005] This application provides an image recognition-based simulation training method and system to solve the problem that simulation training in the prior art can only be carried out indoors.

[0006] On one hand, embodiments of this application provide a simulation training method based on image recognition, including:

[0007] Multiple targets were placed in the training area;

[0008] Acquire the target image in the aiming direction of the simulated gun;

[0009] Analyze the target image to determine whether the simulated gun firing action hit the target;

[0010] If the simulated gunshot hits the target, the AR glasses worn by the trainee mark the hit target and display the mark on the AR glasses;

[0011] Statistical analysis of data from simulated gunfire hitting targets yields training result reports.

[0012] In one possible implementation, the target image in the aiming direction of the simulated gun is acquired after the simulated gun is detected to be fired.

[0013] In one possible implementation, the target image is analyzed to determine whether the simulated gun's firing action hits the target, including: identifying whether a target exists in the target image; if a target exists in the target image, extracting the real target region in the target image; determining the center coordinates of the target image; determining whether the center coordinates are located in the real target region, and if so, determining that the simulated gun hit the target.

[0014] In one possible implementation, when the center coordinates are located within the actual target area, the position of the center coordinates within the actual target area is determined, and the shooting score is determined based on the position.

[0015] In one possible implementation, the AR glasses also acquire visual images and analyze them to obtain analysis results on whether the simulated gun hits the target. After obtaining the analysis results based on the visual images, the analysis results based on the target images are corrected to obtain the final analysis results.

[0016] In one possible implementation, an identification object is set on the simulated gun, the AR glasses determine the position of the identification object in the visual image, the center coordinates of the visual image are extracted based on the position of the identification object in the visual image, the AR glasses also extract the visual target area in the visual image, and obtain the analysis results based on the visual image based on whether the center coordinates of the visual image are located in the visual target area.

[0017] In one possible implementation, the target is equipped with identification information. When the target is hit, the AR glasses obtain the identification information of the hit target and generate a marker corresponding to the identification information. When the hit target reappears in the image captured by the AR glasses, the AR glasses display the marker at the position corresponding to the hit target.

[0018] On the other hand, embodiments of this application also provide an image recognition-based simulation training system, including:

[0019] Targets were set up in multiple locations on the training field;

[0020] An image acquisition unit, mounted on the simulated gun, is used to acquire target images in the aiming direction of the simulated gun;

[0021] An image processing device is used to analyze target images and determine whether the simulated gun's firing action hits the target.

[0022] AR glasses, worn by trainees, mark the hit target if the image processing device determines that the simulated gun has hit the target, and display the mark on the AR glasses.

[0023] The background recording device is used to collect statistics on the data of simulated shooting at targets and obtain training result reports.

[0024] In one possible implementation, the image processing device is a wearable device worn by the trainee. The image processing device is connected to the image acquisition unit and AR glasses via cables, and supplies power to the image acquisition unit and AR glasses while communicating with them.

[0025] The image recognition-based simulation training method and system disclosed in this application have the following advantages:

[0026] It only requires placing traditional targets in the training area, and the performance evaluation during the training process is completed by the equipment worn by the trainees. There are no requirements for the training area, it can adapt to various training contents, and training in the outdoor area can also allow trainees to experience an environment close to actual combat, which is conducive to improving training results. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a simulation training method based on image recognition provided in this application embodiment;

[0029] Figure 2 This is a functional block diagram of an image recognition-based simulation training system provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] Figure 1 A flowchart illustrating an image recognition-based simulation training method provided in this application embodiment. This application embodiment provides an image recognition-based simulation training method, including:

[0032] The S100 places multiple targets in the training area.

[0033] For example, the training ground can be located outdoors to make the training environment more similar to a real combat scenario. The targets can be traditional mechanical targets, such as chest ring targets or target boards printed with images of bandits and hostages.

[0034] S110, acquire the target image in the aiming direction of the simulated gun.

[0035] For example, the simulated gun uses a device manufactured to mimic the size and shape of a real firearm, on which an image acquisition unit for acquiring target images can be installed. Since the image acquisition unit is located on the simulated gun, the target image acquired by the image acquisition unit is the image from the perspective of the simulated gun, and the information in the target image can be considered as information acquired from the perspective of the simulated gun.

[0036] In the embodiments of this application, the simulated gun also includes a trigger detection unit and a trigger simulation unit. The trigger detection unit is located at the trigger. When the trigger is activated by a trainee to generate a triggering action on the simulated gun, the trigger detection unit detects the state of the trigger and generates a trigger signal. Only after this trigger signal is generated will the image acquisition unit acquire the target image. Furthermore, after the simulated gun is triggered, the trigger simulation unit can also simulate the sound and tactile feedback of the gun being triggered, such as playing a pre-recorded trigger sound through a speaker or generating the recoil of the gun being triggered through a vibrator.

[0037] S120 analyzes the target image to determine whether the simulated gun's firing action hit the target.

[0038] For example, S120 specifically includes: identifying whether a target exists in the target image; if a target exists in the target image, extracting the real target region in the target image; determining the center coordinates of the target image; determining whether the center coordinates are located in the real target region, and if so, determining that the simulated gunshot hits the target.

[0039] In the embodiments of this application, a target recognition model can be used to identify whether a target exists in the target image and extract the real target region. This target recognition model can be based on a neural network, such as a convolutional neural network. Since the target image is considered to be an image obtained from the perspective of a simulated gun, the center coordinates of the target image are the coordinates of the simulated gun's aiming direction in the target image. If the center coordinates are located in the real target region, it indicates that the trainee's aiming direction is also located in the real target region when the simulated gun is triggered. Since the firing direction and aiming direction of the firearm are consistent, it can be considered that the simulated gun hit the target when the center coordinates of the target image are located in the real target region.

[0040] Furthermore, if the target used is a chest ring target, then after determining the target hit in the simulated shooting, it is also necessary to determine the position of the center coordinates in the actual target area, and determine the shooting score based on the position.

[0041] S130: If the simulated gunshot hits the target, the AR glasses worn by the trainee mark the hit target and display the mark on the AR glasses.

[0042] For example, AR (Augmented Reality) technology adds necessary virtual information to a real-world scene to simultaneously display both real and virtual information. AR glasses are devices worn by trainees, primarily consisting of glasses, an image acquisition unit, and a data processing unit. The lenses of the glasses have a transparent electronic display screen. When the screen is not displaying information, it is transparent and does not obstruct the trainee's view of surrounding information. When displaying information, the information is overlaid on the real-world information. The image acquisition unit can be mounted on the frame of the glasses, and its orientation is the same as that of the glasses, ensuring that the information acquired by the image acquisition unit is consistent with the information obtained by the trainee wearing the AR glasses. The data processing unit processes the information acquired by the image acquisition unit and controls the content displayed on the screen in the glasses based on the processing results.

[0043] In the embodiments of this application, due to the size limitations of AR glasses, their power consumption and data processing capabilities cannot meet the requirements of long-term, high-intensity military training. Therefore, this application uses an image processing device worn by trainees to replace the data processing part in traditional AR glasses. This image processing device can take various wearable forms such as wrist-worn, head-worn, and backpack-style. The image processing device can be connected to the image acquisition unit and AR glasses via cables, and can process not only the information acquired by the AR glasses but also the target images acquired by the image acquisition unit. Moreover, this application also includes a battery along with the image processing device. This battery powers both the image processing device and the image acquisition unit and AR glasses. The simultaneous operation of the image processing device and the battery can greatly improve data processing capabilities and power consumption.

[0044] Furthermore, in complex external environments, processing target images using image processing devices may lead to misidentification, directly affecting the accuracy of the analysis results. To reduce the impact of misidentification, this application also uses AR glasses to acquire visual images and analyzes them to obtain analysis results on whether the simulated gun hit the target. After obtaining the analysis results based on the visual images, the analysis results based on the target images are corrected to obtain the final analysis results.

[0045] Specifically, the simulated gun is equipped with an identification object. The AR glasses determine the position of the identification object in the visual image. Based on the position of the identification object in the visual image, the relative center coordinates of the visual image are corrected. The AR glasses also extract the visual target area in the visual image and obtain the analysis result based on the visual image by determining whether the corrected relative center coordinates are within the visual target area. The aforementioned identification object can be a scope or other structure fixed on the simulated gun. The position of the identification object on the simulated gun is fixed. During shooting, the trainee will look directly at the scope with their eyes, i.e., the AR glasses in this application. At this time, it can be considered that the visual direction of the AR glasses is consistent with the aiming direction of the simulated gun. After the AR glasses acquire the visual image, the image processing device first uses the identification object recognition model to extract the identification object and determine the position of the identification object in the visual image. If the position does not match the preset position data, it means that the trainee is not in the aiming state at this time. In this case, the visual image acquired this time will not participate in the correction of the analysis result based on the target image. If the position of the target in the visual image matches the preset position data, it indicates that the trainee is in aiming mode. At this point, the image processing device extracts the center coordinates of the visual image and the visual target area within the visual image. If the center coordinates of the visual image are within the visual target area, the analysis result obtained by the image processing device based on the visual image is that the simulated gun hit the target. During correction, if both the analysis result based on the target image and the analysis result based on the visual image indicate that the simulated gun hit the target, the corrected analysis result is also that the simulated gun hit the target. Conversely, if either analysis result indicates that the simulated gun missed the target, the corrected analysis result is also that the simulated gun missed the target.

[0046] In the embodiments of this application, the targets are equipped with identification information, which can be digital codes or graphic codes printed on the targets. Each target has a unique identification information, allowing the number of targets to be added or removed at any time in the training area without further adjustments. When a target is hit, the AR glasses acquire the identification information of the hit target and generate a marker corresponding to that information. When the hit target reappears in the image captured by the AR glasses, the AR glasses display the marker at the position corresponding to the hit target. For example, when a target is determined to have been hit, the AR glasses acquire a real-time visual image of the external environment, which is then processed in real-time by an image processing device to determine if a target exists in the visual image. If a target exists, the identification information on the target is extracted and matched with the stored markers. If the match is successful, it indicates that the target has been hit. At this time, the AR glasses will display a marker such as "Hit" at the position corresponding to the target. Trainees can then see which targets in the current environment have been hit through the AR glasses. These hit targets can be ignored by the trainees without needing to shoot again.

[0047] S140: Statistically analyze the data of simulated gunfire hitting targets to obtain a training result report.

[0048] For example, the training results report is compiled according to the trainees. The training results of multiple trainees can be compiled into one report file, and the training results of each trainee need to show the number of targets hit and specific scores.

[0049] This application also provides an image recognition-based simulation training system, such as... Figure 2 As shown, the system includes:

[0050] Targets were set up in multiple locations on the training field;

[0051] An image acquisition unit, mounted on the simulated gun, is used to acquire target images in the aiming direction of the simulated gun;

[0052] An image processing device is used to analyze target images and determine whether the simulated gun's firing action hits the target.

[0053] AR glasses, worn by trainees, mark the hit target if the image processing device determines that the simulated gun has hit the target, and display the mark on the AR glasses.

[0054] The background recording device is used to collect statistics on the data of simulated shooting at targets and obtain training result reports.

[0055] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0056] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A simulation training method based on image recognition, characterized in that, include: Multiple targets were placed in the training area; Acquire the target image in the aiming direction of the simulated gun; Analyzing the target image to determine whether the shooting action of the simulated gun hits the target includes: identifying whether a target exists in the target image; if a target exists in the target image, extracting the real target region in the target image; determining the center coordinates of the target image; determining whether the center coordinates are located in the real target region, and if so, determining that the simulated gun hit the target; If the simulated gunshot hits the target, the AR glasses worn by the trainee mark the hit target and display the mark on the AR glasses; Statistical analysis of the data from the simulated shooting at the target was performed to obtain a training result report; The AR glasses also acquire visual images and analyze them to obtain an analysis result on whether the simulated gun hits the target. After obtaining the analysis result based on the visual images, the analysis result based on the target image is corrected to obtain the final analysis result. The simulated gun is equipped with an identification object. The AR glasses determine the position of the identification object in the visual image. Based on the position of the identification object in the visual image, the center coordinates of the visual image are extracted. The AR glasses also extract the visual target area in the visual image and obtain the analysis result based on the visual image based on whether the center coordinates of the visual image are within the visual target area.

2. The simulation training method based on image recognition according to claim 1, characterized in that, After the simulated gun is detected to be fired, the target image in the aiming direction of the simulated gun is then acquired.

3. The simulation training method based on image recognition according to claim 1, characterized in that, When the center coordinates are located within the actual target area, the position of the center coordinates within the actual target area is determined, and the shooting score is determined based on the position.

4. The simulation training method based on image recognition according to claim 1, characterized in that, The target is equipped with identification information. When the target is hit, the AR glasses acquire the identification information of the hit target and generate a mark corresponding to the identification information. When the hit target reappears in the image captured by the AR glasses, the AR glasses display the mark at the position corresponding to the hit target.

5. A simulation training system based on image recognition, characterized in that, The system employs the image recognition-based simulation training method as described in claim 1, and the system comprises: Targets were set up in multiple locations on the training field; An image acquisition unit, mounted on the simulated gun, is used to acquire target images in the aiming direction of the simulated gun; An image processing device is used to analyze the target image and determine whether the firing action of the simulated gun hits the target; AR glasses, worn by trainees, mark the hit target if the image processing device determines that the simulated gunshot hits the target, and display the mark on the AR glasses; A background recording device is used to statistically analyze the data of the simulated shooting at the target and obtain a training result report.

6. The image recognition-based simulation training system according to claim 5, characterized in that, The image processing device is a wearable device worn by the trainee. The image processing device is connected to the image acquisition unit and AR glasses via cables, and supplies power to the image acquisition unit and AR glasses while communicating with them.

Images