Posture control system and posture recognition method for 3D image target
By performing skeletal digital abstraction of the movements of trained personnel and interacting with 3D video programs in real time, the problem of lack of interactivity and fun in the prior art is solved, and a higher sense of substitution and fun is achieved.
Patent Information
- Application Number
- CN202411160749.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-15
AI Technical Summary
The existing image target training system lacks interactivity with the training scenes, and the training personnel cannot change the direction of the program, resulting in insufficient integration and fun.
Using a posture control system and posture recognition method for 3D image targets, the movements of trainers are skeletized digital abstracted and converted into corresponding action instructions, and interact with the 3D image programs on the large screen in real time. , using the MediPipe algorithm to collect and identify three-dimensional coordinates to realize action recognition and scene control.
It enhances the sense of substitution for shooters and the fun of shooting, providing a more interactive and interesting training experience.
Smart Images

Figure CN118949403A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of shooting training equipment in a shooting range, and in particular to a posture control system and a posture recognition method for a 3D image target. Background Art
[0002] The image target training system has been widely used in student military training, recruit shooting, police training, national defense education, children's military schools, shooting experience venues, etc. The mainstream image targets on the market are generally shot in advance with various video materials, which are imported into the system after editing. The shooter uses a laser gun or a real gun to shoot at various simulated scenes in the image, which is mainly used for various anti-terrorism combat tactical shooting training. This model lacks interactivity with the training scene, and the trainers cannot change the direction of the program, and the sense of integration is not strong. Summary of the invention
[0003] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a posture control system and posture recognition method for 3D image targets, which skeletonize the trainee's movements and convert them into corresponding action instructions, interact with the 3D image program on the large screen in real time, and enhance the shooter's sense of immersion and the fun of shooting.
[0004] To achieve the above-mentioned purpose, the present invention adopts the following technical solution: a posture control system and posture recognition method for a 3D image target, the method is specifically as follows: S1: Sample data collection before leaving the factory; The trainees stand in the designated shooting area and perform corresponding actions in sequence according to the pre-set action instructions. Each time they perform a complete action, they collect the current picture. This is repeated several times. The collected pictures are classified and saved according to the action instruction name. The MediPipe algorithm is used to detect the pictures to obtain the preceding three-dimensional coordinates. The preceding three-dimensional coordinates are linearly transformed according to the length and width of the image. Finally, a table with a corresponding name is established according to the action instruction name. The data in the picture is saved in the table named after the action name to form the factory sample data. S2: Identification and analysis of the scene images; Power on and connect to the backstage central control, open the 3D image program and project the picture onto the large screen. The trainee stands in the designated shooting area and the posture controller automatically starts the recognition program facing the trainee. After receiving the recognition command, the camera is turned on and the factory sample data in step S1 is read into the memory; a current picture is collected by the camera for detection to obtain the subsequent three-dimensional coordinates, and at the same time, the subsequent three-dimensional coordinates are linearly transformed according to the length and width of the image; the data after the linear transformation of the subsequent three-dimensional coordinates and the factory sample data in the memory are calculated and filtered, and finally the action instruction with the highest current frequency is obtained, which is the current recognition result.
[0005] Preferably, in step S1, the sample data collection steps before shipment are as follows: S1-1: The trainee stands in the designated shooting area, places the attitude controller on the tray of the tripod, at a distance from the trainee and facing the trainee, turns on the power, and connects the keyboard, mouse and monitor through the panel interface of the attitude controller; S1-2: Open the real-time camera image, adjust the focus and clarity of the camera lens, and ensure that the image quality is clear and the trainee occupies about 60% of the entire image; S1-3: The trainee performs corresponding actions in sequence according to the pre-set action instructions. Each time a complete action is performed, a frame of the current image is captured, and the capture is repeated until all action instructions are completed; S1-4: The trainee sets a rectangular area of a*b with the current position as the center, randomly reaches a certain position, and repeats step S1-3 for a total of n times; S1-5: Classify the collected images according to the action instruction names and save them in folders named after the action names; S1-6: The program reads the pictures in each folder in turn, and uses the Pose module in the MediPipe algorithm to detect 33 bone points of the human body in the picture, and obtains the three-dimensional coordinates of the front sequence , linearly transform the coordinates according to the length and width of the image; S1-7: Create a table with a corresponding name according to the action instruction name, and save the 33 skeleton point data in the picture into the table named by the action name. Each row in the table stores the skeleton point data of one picture.
[0006] Preferably, in step S2, the steps of identifying and analyzing the scene image are specifically as follows: S2-1: Power on the backstage central control, open the 3D video program and project the image onto the large screen; S2-2: The trainee stands in the designated shooting area, places the attitude controller on the tray of the tripod, a certain distance away from the trainee and facing the trainee, powers it on, and the recognition program starts automatically; the central control program waits for the attitude controller to come online and then sends a start recognition command to the attitude controller through the wireless network. After receiving the start recognition command, the attitude controller automatically turns on the local camera; S2-3: The attitude controller reads all local sample data tables into memory; S2-4: The posture controller collects a current picture and uses the human posture analysis Pose module in the MediPipe algorithm to detect 33 bone points of the human body in the picture to obtain the subsequent three-dimensional coordinates. , perform linear transformation on the subsequent three-dimensional coordinates according to the length and width of the image. If no person is detected in the picture, repeat step S2-4 at the specified frame rate; S2-5: filter according to the maximum distance, calculate the point-to-point distance between the 33 coordinate point data after linear transformation and each row of the skeleton point coordinates in the memory data table, and record the maximum distance of each 33 sets of distances and the row number of the current table; S2-6: Filter according to the average distance, sort the results recorded in step S2-5 from small to large, intercept the first 30 rows of records, and then call the coordinates of the bone points and the data after linear transformation according to the table row number in the recorded data to perform point-to-point distance calculation, and record the average distance of each 33 groups of distances and the row number of the current table; S2-7: Sort the results recorded in step S2-6 from small to large, intercept the first 10 rows of records, obtain the name of the action instruction from the memory table according to the row number in the record, and the action instruction with the highest frequency is the current recognition result; S2-8: Send the command identified in step S2-7 to the background central control system, and the central control system controls the direction of the scene according to the action command.
[0007] Furthermore, a device for a posture control system and a posture recognition method for a 3D image target is provided, the device also includes a background central control, a large-screen projection system and a posture controller; a tray is also provided at the lower end of the posture controller; the tray is arranged on an adjustable tripod.
[0008] Furthermore, the posture controller includes an upper cover plate and a lower box body; a power display module is installed in the left area of the upper cover plate; a first heat dissipation window is provided as a whole in the right area; a cooling fan is installed at the bottom of the first heat dissipation window; a second heat dissipation window is provided around the lower box body as a whole; height adjustment studs are installed at the four corner points of the bottom of the lower box body; a collection window is provided on the front end surface of the lower box body; the collection window is circular transparent glass, and is installed on the front end surface of the lower box body through a lens flange.
[0009] Furthermore, a panel interface is provided on the rear end face of the lower box body, including an HDMI interface socket, a USB interface socket, a network interface socket, a power-on switch and a charging socket; the interior of the lower box body contains an industrial computer, a camera module, a power converter and a battery; the camera module communicates with the industrial computer via a USB data cable.
[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a posture control system and a posture recognition method for 3D image targets. The posture controller can move arbitrarily in the venue according to the position of the trainees, and the skeletonized digital abstract processing of the trainees' movements is recognized and converted into corresponding action instructions, which interacts with the 3D image program on the large screen in real time, enhances the shooter's sense of immersion and the fun of shooting, and is simple and reliable to operate. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 This is a flow chart of sample data collection before the present invention leaves the factory; Figure 2 This is a flow chart of the recognition and analysis of the scene images of the present invention; Figure 3 This is a diagram of the posture control system architecture for a 3D imaging target according to the present invention; Figure 4 A three-dimensional diagram of the layout and installation of the attitude controller of the present invention; Figure 5 It is a stereoscopic diagram of the posture controller of the present invention; Figure 6 This is an internal diagram of the attitude controller of the present invention; In the figure: 1-backstage central control; 2-large-screen projection system; 3-gesture controller; 4-trainees; 5-first heat dissipation window; 6-tray; 7-tripod; 8-second heat dissipation window; 9-upper cover; 10-power display module; 11-lower box; 12-adjusting studs; 13-lens flange; 14-capture window; 15-battery; 16-camera module; 17-power converter; 18-panel interface; 19-HDMI socket; 20-USB socket; 21-network port socket; 22-industrial computer; 23-power on switch; 24-charging socket. DETAILED DESCRIPTION
[0012] In order to provide a further understanding of the purpose, structure, features, and functions of the present invention, the following detailed description is given in conjunction with the embodiments.
[0013] Please refer to Figures 1 to 6 The present invention provides a posture control system and posture recognition method for 3D image targets, which are mainly divided into two parts, including sample data collection before delivery and recognition and analysis of on-site images.
[0014] The steps for collecting sample data before leaving the factory are as follows: S1-1: The trainee stands in the designated shooting area, places the attitude controller on the tray of the tripod, at a distance from the trainee and facing the trainee, turns on the power, and connects the keyboard, mouse and monitor through the panel interface of the attitude controller; S1-2: Open the real-time camera image, adjust the focus and clarity of the camera lens, and ensure that the image quality is clear and the trainee occupies about 60% of the entire image; S1-3: The trainee performs corresponding actions in sequence according to the pre-set action instructions. Each time a complete action is performed, a frame of the current image is captured, and the capture is repeated until all action instructions are completed; S1-4: The trainee sets a rectangular area of a*b with the current position as the center, randomly reaches a certain position, and repeats step S1-3 for a total of n times; S1-5: Classify the collected images according to the action instruction names and save them in folders named after the action names; S1-6: The program reads the pictures in each folder in turn, and uses the Pose module in the MediPipe algorithm to detect 33 bone points of the human body in the picture, and obtains the three-dimensional coordinates of the front sequence , linearly transform the coordinates according to the length and width of the image; S1-7: Create a table with a corresponding name according to the action instruction name, and save the 33 skeleton point data in the picture into the table named by the action name. Each row in the table stores the skeleton point data of one picture.
[0015] The specific steps for identifying and analyzing the on-site images are as follows: S2-1: Power on the backstage central control, open the 3D video program and project the image onto the large screen; S2-2: The trainee stands in the designated shooting area, places the attitude controller on the tray of the tripod, a certain distance away from the trainee and facing the trainee, powers it on, and the recognition program starts automatically; the central control program waits for the attitude controller to come online and then sends a start recognition command to the attitude controller through the wireless network. After receiving the start recognition command, the attitude controller automatically turns on the local camera; S2-3: The attitude controller reads all local sample data tables into memory; S2-4: The posture controller collects a current picture and uses the human posture analysis Pose module in the MediPipe algorithm to detect 33 bone points of the human body in the picture to obtain the subsequent three-dimensional coordinates. , perform linear transformation on the subsequent three-dimensional coordinates according to the length and width of the image. If no person is detected in the picture, repeat step S2-4 at the specified frame rate; S2-5: filter according to the maximum distance, calculate the point-to-point distance between the 33 coordinate point data after linear transformation and each row of the skeleton point coordinates in the memory data table, and record the maximum distance of each 33 sets of distances and the row number of the current table; S2-6: Filter according to the average distance, sort the results recorded in step S2-5 from small to large, intercept the first 30 rows of records, and then call the coordinates of the bone points and the data after linear transformation according to the table row number in the recorded data to perform point-to-point distance calculation, and record the average distance of each 33 groups of distances and the row number of the current table; S2-7: Sort the results recorded in step S2-6 from small to large, intercept the first 10 rows of records, obtain the name of the action instruction from the memory table according to the row number in the record, and the action instruction with the highest frequency is the current recognition result; S2-8: Send the command identified in step S2-7 to the background central control system, and the central control system controls the direction of the scene according to the action command.
[0016] A device for a posture control system and a posture recognition method for a 3D image target mainly comprises three parts: a background central control 1, a large-screen projection system 2 and a posture controller 3.
[0017] The backstage control 2 is generally arranged in the command and control room of the shooting range, running the 3D image target main control program, receiving the human motion command code sent back in real time by the posture controller 3 through the wireless network, and the corresponding motion command triggers the flow of the program in the 3D image subject, and transmits the picture to the large-screen projection system 2 in real time via the optical fiber line.
[0018] The large-screen projection system 2 is installed in front of the shooting area, facing the shooter 4, and is composed of a screen with good bullet-resistant performance and multiple projectors. The 3D image picture of the background central control 2 is transmitted to the projector via optical fiber. Multiple projectors fuse the pictures and project them on the large screen to form a complete 3D image picture.
[0019] The posture controller 3 is placed on a tray 5 , which is located on a height-adjustable tripod 6 , and is located between the large screen and the trainer 4 .
[0020] The attitude controller 3 includes an upper cover plate 9 and a lower box body 11, wherein the upper cover plate 9 is integrally embedded in the lower box body 11, a power display module 10 is installed in the left area, a strip-shaped heat dissipation window 5 is integrally opened in the right area, and a heat dissipation fan is installed at the bottom of the heat dissipation window 5; the lower box body 11 is integrally opened with a strip-shaped heat dissipation window 8, and four height adjustment studs 12 are installed at the four corners of the bottom to adjust the tilt angle of the controller; the front face acquisition window 14 is a round transparent glass installed through a flange 13, and the window is used to capture the camera image; the rear face panel interface 18 includes an HDMI interface seat 19, a USB interface seat 20, and a network interface seat 2 1. Power-on switch 23 and charging socket 24; the box body contains industrial computer 22, camera module 16, power converter 17 and battery 15; the camera module 16 contains acquisition camera and bracket, and communicates with industrial computer 22 through USB data line; the power converter 17 is used to convert the battery output into a suitable voltage value to power industrial computer 22; the industrial computer 22 has its own wireless module, which abstracts the collected human body images into skeletons and converts them into corresponding action instructions, and sends the analyzed action instructions to the background central control system 1 via the wireless network, interacts with the 3D image program in real time, and controls the direction of the program.
[0021] The present invention has been described by the above-mentioned relevant embodiments, however, the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of patent protection of the present invention.
Claims
1. A posture control system and posture recognition method for a 3D image target, characterized in that: The specific methods are as follows: S1: Sample data collection before leaving the factory; The trainees stand in the designated shooting area and perform corresponding actions in sequence according to the pre-set action instructions. Each time they perform a complete action, they collect the current picture. This is repeated several times. The collected pictures are classified and saved according to the action instruction name. The MediPipe algorithm is used to detect the pictures to obtain the preceding three-dimensional coordinates. The preceding three-dimensional coordinates are linearly transformed according to the length and width of the image. Finally, a table with a corresponding name is established according to the action instruction name. The data in the picture is saved in the table named after the action name to form the factory sample data. S2: Identification and analysis of the scene images; Power on and connect to the backstage central control, open the 3D image program and project the picture onto the large screen. The trainee stands in the designated shooting area and the posture controller automatically starts the recognition program facing the trainee. After receiving the recognition command, the camera is turned on and the factory sample data in step S1 is read into the memory; a current picture is collected by the camera for detection to obtain the subsequent three-dimensional coordinates, and at the same time, the subsequent three-dimensional coordinates are linearly transformed according to the length and width of the image; the data after the linear transformation of the subsequent three-dimensional coordinates and the factory sample data in the memory are calculated and filtered, and finally the action instruction with the highest current frequency is obtained, which is the current recognition result.
2. A posture control system and posture recognition method for a 3D image target as claimed in claim 1, characterized in that: In step S1, the sample data collection steps before delivery are as follows: S1-1: The trainee stands in the designated shooting area, places the attitude controller on the tray of the tripod, at a distance from the trainee and facing the trainee, turns on the power, and connects the keyboard, mouse and monitor through the panel interface of the attitude controller; S1-2: Open the real-time camera image, adjust the focus and clarity of the camera lens, and ensure that the image quality is clear and the trainee occupies about 60% of the entire image; S1-3: The trainee performs corresponding actions in sequence according to the pre-set action instructions. Each time a complete action is performed, a frame of the current image is captured, and the capture is repeated until all action instructions are completed; S1-4: The trainee sets a rectangular area of a*b with the current position as the center, randomly reaches a certain position, and repeats step S1-3 for a total of n times; S1-5: Classify the collected images according to the action instruction names and save them in folders named after the action names; S1-6: The program reads the pictures in each folder in turn, and uses the Pose module in the MediPipe algorithm to detect 33 bone points of the human body in the picture, and obtains the three-dimensional coordinates of the front sequence , linearly transform the coordinates according to the length and width of the image; S1-7: Create a table with a corresponding name according to the action instruction name, and save the 33 skeleton point data in the picture into the table named by the action name. Each row in the table stores the skeleton point data of one picture.
3. A posture control system and posture recognition method for a 3D image target as claimed in claim 1, characterized in that: In step S2, the steps of identifying and analyzing the scene image are specifically as follows: S2-1: Power on the backstage central control, open the 3D video program and project the image onto the large screen; S2-2: The trainee stands in the designated shooting area, places the attitude controller on the tray of the tripod, a certain distance away from the trainee and facing the trainee, powers it on, and the recognition program starts automatically; the central control program waits for the attitude controller to come online and then sends a start recognition command to the attitude controller through the wireless network. After receiving the start recognition command, the attitude controller automatically turns on the local camera; S2-3: The attitude controller reads all local sample data tables into memory; S2-4: The posture controller collects a current picture and uses the human posture analysis Pose module in the MediPipe algorithm to detect 33 bone points of the human body in the picture to obtain the subsequent three-dimensional coordinates. , perform linear transformation on the subsequent three-dimensional coordinates according to the length and width of the image. If no person is detected in the picture, repeat step S2-4 at the specified frame rate; S2-5: filter according to the maximum distance, calculate the point-to-point distance between the 33 coordinate point data after linear transformation and each row of the skeleton point coordinates in the memory data table, and record the maximum distance of each 33 sets of distances and the row number of the current table; S2-6: Filter according to the average distance, sort the results recorded in step S2-5 from small to large, intercept the first 30 rows of records, and then call the coordinates of the bone points and the data after linear transformation according to the table row number in the recorded data to perform point-to-point distance calculation, and record the average distance of each 33 groups of distances and the row number of the current table; S2-7: Sort the results recorded in step S2-6 from small to large, intercept the first 10 rows of records, obtain the name of the action instruction from the memory table according to the row number in the record, and the action instruction with the highest frequency is the current recognition result; S2-8: Send the command identified in step S2-7 to the background central control system, and the central control system controls the direction of the scene according to the action command.
4. A device for a posture control system and a posture recognition method for a 3D image target as claimed in claim 1, characterized in that: The device further comprises a backstage central control (1), a large screen projection system (2) and a posture controller (3); a tray (6) is also provided at the lower end of the posture controller (3); and the tray (6) is arranged on an adjustable tripod (7).
5. The device for a posture control system and posture recognition method for a 3D image target according to claim 4, characterized in that: The attitude controller (3) comprises an upper cover plate (9) and a lower box body (11); a power display module (10) is installed in the left area of the upper cover plate (9); a first heat dissipation window (5) is provided as a whole in the right area; a heat dissipation fan is installed at the bottom of the first heat dissipation window (5); a second heat dissipation window (8) is provided around the lower box body (11); four height adjustment studs (12) are installed at four corner points of the bottom of the lower box body (11); a collection window (14) is provided on the front surface of the lower box body (11); the collection window (14) is a circular transparent glass and is installed on the front surface of the lower box body (11) through a lens flange (13).
6. The device for a posture control system and posture recognition method for a 3D image target according to claim 5, characterized in that: A panel interface (18) is provided on the rear end surface of the lower box body (11), comprising an HDMI interface seat (19), a USB interface seat (20), a network interface seat (21), a power-on switch (23) and a charging socket (24); the lower box body (11) contains an industrial computer (22), a camera module (16), a power converter (17) and a battery (15); the camera module (16) and the industrial computer (22) communicate via a USB data cable.
