Target identification and tracking method based on multi-sensor fusion technology

Abstract

The invention discloses a target identification and tracking method based on a multi-sensor fusion technology, and relates to the field of image processing, and the method comprises the steps: S1, carrying out the spatial modeling of a target shooting region, obtaining a shooting region spatial model, carrying out the target detector image collection and analysis of each training student, and obtaining a target detector image; s2, creating a model space rectangular coordinate system in the shooting area space model, performing posture angle deviation analysis on the process tracking student through the model space rectangular coordinate system, and obtaining shooting image static analysis data according to the analysis result; s3, performing shooting stability analysis on the process tracking student in the shooting monitoring period to obtain the peak oscillation amplitude of the shooting period, and S4, performing tracking result feedback on the process tracking student according to the shooting image static analysis data and the peak oscillation amplitude of the shooting period, so that the shooting training effect of the student can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and relates to multi-sensor fusion technology, in particular to a target recognition and tracking method based on multi-sensor fusion technology. Background Art

[0002] The existing target recognition and tracking methods based on multi-sensor fusion technology have the following specific defects when providing feedback on shooting training results:

[0003] 1. Existing target recognition and tracking methods are unable to categorize trainees based on the shooting results displayed by the target detector, and are unable to create a shooting area spatial model to perform static and dynamic shooting image analysis on trainees who fail shooting training, resulting in a lack of accuracy in image monitoring results.

[0004] 2. Existing target recognition and tracking methods are unable to create a model space rectangular coordinate system in the shooting area space model to analyze the posture angle deviation of trainees, and are unable to provide feedback on the key correction points of the shooting posture based on the analysis results, resulting in poor shooting training results.

[0005] To this end, we propose a target recognition and tracking method based on multi-sensor fusion technology. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a target recognition and tracking method based on multi-sensor fusion technology. The present invention aims to improve the comprehensiveness and pertinence of image tracking during the shooting process, thereby ensuring the effect of shooting training.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: a target recognition and tracking method based on multi-sensor fusion technology, comprising the following specific steps:

[0008] Step S1: spatially modeling the target shooting area to obtain a shooting area spatial model, selecting a number of trainees who are shooting in the shooting area spatial model, collecting and analyzing target detector images for each trainee, and classifying the trainees into qualified shooting trainees and process tracking trainees based on the analysis results to obtain trainee shooting collection data;

[0009] Step S2: creating a model space rectangular coordinate system in the shooting area space model based on the student's shooting data, performing posture angle deviation analysis on the process tracking student through the model space rectangular coordinate system, and obtaining shooting image static analysis data based on the analysis results;

[0010] Step S3: performing shooting stability analysis on the process tracking students in the shooting monitoring period based on the students' shooting collection data and shooting image static analysis data to obtain the peak oscillation amplitude during the shooting period;

[0011] Step S4: providing tracking result feedback to the process tracking trainees based on the static analysis data of the shooting image and the peak oscillation amplitude during the shooting period.

[0012] Furthermore, the step S1 further includes the following specific steps:

[0013] Step S11: Marking the shooting area of the trainees in the shooting training range to obtain the target shooting area, and using 3D structured light technology to create a three-dimensional space of the target shooting area to obtain a three-dimensional model of the shooting area;

[0014] Step S12: dividing the three-dimensional model of the shooting area into a plurality of model monitoring sub-areas, and randomly selecting a sample model monitoring sub-area from the plurality of obtained model monitoring sub-areas;

[0015] Step S13: performing camera perspective analysis on the sample model monitoring sub-area, and obtaining the optimal spatial perspective camera corresponding to the sample monitoring model sub-area according to the analysis result;

[0016] Step S14: obtaining the optimal spatial viewing angle camera corresponding to each model monitoring sub-area;

[0017] Step S15: using the real-time image captured by the optimal spatial perspective camera to fill the three-dimensional model of the shooting area, thereby obtaining a spatial model of the shooting area;

[0018] Step S16: acquiring trainees who are shooting in the shooting area space model, and arbitrarily selecting a target trainee from the acquired multiple trainees;

[0019] Step S17: Analyzing the shooting level of the target training students and classifying the target training students into different types according to the analysis results;

[0020] Step S18: performing a shooting level analysis on each trainee, and dividing the trainees into process tracking trainees and shooting qualified trainees according to the analysis results, thereby obtaining trainee type classification data;

[0021] Step S19: Define the shooting area space model and the student type classification data as student shooting acquisition data.

[0022] Furthermore, the step S13 further includes the following specific steps:

[0023] Step S131: Acquire the real-time images corresponding to each environmental camera in the shooting training range, mark the environmental cameras with the sample model monitoring sub-area in the real-time images as viewpoint cameras to be analyzed, and select a sample camera to be analyzed from the multiple viewpoint cameras to be analyzed;

[0024] Step S132: performing camera angle analysis on the sample camera to be analyzed, and obtaining the environmental sampling distance of the sample camera to be analyzed to the sample monitoring model sub-area according to the analysis result;

[0025] Step S133: Obtain the environmental sampling distance of each camera to be analyzed for the sample monitoring model sub-area respectively, compare the numerical values of the multiple environmental sampling distances obtained, and mark the camera to be analyzed with the minimum numerical value as the optimal spatial perspective camera corresponding to the sample monitoring model sub-area.

[0026] Furthermore, the step S132 further includes the following specific steps:

[0027] Step S1321: Select multiple environmental monitoring points in the sample monitoring model sub-area, obtain the spatial distance between each environmental monitoring point and the sample camera to be analyzed, obtain multiple camera spatial distances, and average the obtained camera spatial distances to obtain the camera shooting distance;

[0028] Step S1322: Acquire the focal length corresponding to the sample camera to be analyzed to obtain the camera focal length, and acquire the image sensor size corresponding to the sample camera to be analyzed to obtain the sensor size value;

[0029] Step S1323: Obtain the horizontal pixel number value corresponding to the sample camera image to be analyzed to obtain the camera pixel number value;

[0030] Step S1324: The camera shooting distance, the camera focal length, the sensor size value and the camera pixel number value are calculated to obtain the environmental sampling distance of the sample to-be-analyzed camera to the sample monitoring model sub-area;

[0031] The environmental sampling distance of the sample to be analyzed camera to the sample monitoring model sub-area is calculated. The specific formula is as follows:

[0032]

[0033] Among them, Gsd is the environmental sampling distance of the sample to be analyzed camera to the sample monitoring model sub-area, Cci is the sensor size value, Psj is the camera shooting distance, Jdj is the camera focal length, and Xss is the number of camera pixels.

[0034] Furthermore, the step S17 further includes the following specific steps:

[0035] Step S171: acquiring an image of the training target after the target training student shoots to obtain a training target image, counting the number of shooting bullet holes in the training target image to obtain the number of shooting bullet holes, acquiring the number of shootings corresponding to the target training student to obtain the student's shooting number;

[0036] Step S172: If the number of shooting bullet holes is not equal to the value corresponding to the number of shots taken by the trainee, the target training trainee is classified as a process tracking trainee;

[0037] Step S173: If the number of bullet holes is equal to the number of shots taken by the trainee, the distance between each bullet hole and the bull's eye in the training target image is averaged to obtain the shooting distance deviation.

[0038] Step S174: Obtain the preset deviation interval of the shooting distance. If the shooting distance deviation is within the preset deviation interval of the shooting distance, the target training student is classified as a qualified shooting student. If the shooting distance deviation is not within the preset deviation interval of the shooting distance, the target training student is classified as a process tracking student.

[0039] Furthermore, the step S2 further includes the following specific steps:

[0040] Step S21: Acquire student shooting data, acquire a shooting area spatial model and student type classification data based on the student shooting data, and acquire process tracking students based on the student type classification data;

[0041] Step S22: setting a plurality of different types of shooting posture monitoring angles in the historical shooting image, and naming the set shooting posture monitoring angles as J1 posture monitoring angle to Ja posture monitoring angle respectively;

[0042] Step S23: creating a spatial rectangular coordinate system in the shooting area spatial model to obtain a model spatial rectangular coordinate system;

[0043] Step S24: when the J1 posture monitoring angle is the shoulder joint abduction angle, the J1 posture monitoring angle is acquired to obtain a value of the J1 posture monitoring angle;

[0044] Step S25: acquiring angle values from the J2 posture monitoring angle to the Ja posture monitoring angle, respectively, to obtain values from the J2 posture monitoring angle to the Ja posture monitoring angle;

[0045] Step S26: respectively obtaining the posture monitoring reference angle intervals corresponding to the J1 posture monitoring angle value to the Ja posture monitoring angle value, and obtaining the J1 posture monitoring angle interval to the Ja posture monitoring angle interval;

[0046] Step S27: Obtain the interval difference between the J1 posture monitoring angle value and the Ja posture monitoring angle interval, obtain the interval angle range corresponding to the J1 posture monitoring angle interval, calculate the ratio of the interval difference to the interval angle range, and obtain the J1 posture angle deviation ratio;

[0047] Step S28: respectively obtaining posture angle deviation ratios corresponding to the J2 posture monitoring angle to the Ja posture monitoring angle, and obtaining the J2 posture angle deviation ratio to the Ja posture angle deviation ratio;

[0048] Step S29: defining the model space rectangular coordinate system and the J1 posture angle deviation ratio to the Ja posture angle deviation ratio as shooting image static analysis data.

[0049] Furthermore, the step S24 further includes the following specific steps:

[0050] In the rectangular coordinate system of the model space, mark the highest point of the acromion process of the process tracking student as posture coordinate point A, mark the center point of the greater tuberosity of the humerus of the process tracking student as posture coordinate point B, and mark the center point of the sternoclavicular joint of the process tracking student as posture coordinate point C;

[0051] The vector is calculated by the pose coordinate point A (x1, y1, z1) and the pose coordinate point B (x2, y2, z2)

[0052] vector

[0053] The vector is calculated by the posture coordinate point A (x1, y1, z1) and the posture coordinate point C (x3, y3, z3)

[0054] vector

[0055] Pair Vector Calculate the modulus length Pair Vector Calculate the modulus length

[0056] Calculating vectors With vector The dot product of

[0057] Calculated by the cosine formula The angle value of θ is calculated by the inverse cosine function arccos(cos(θ)) to obtain the J1 posture monitoring angle value.

[0058] Furthermore, the step S3 further includes the following specific steps:

[0059] Step S31: Acquire student shooting data, acquire a shooting area spatial model and student type classification data based on the student shooting data, and acquire process tracking students based on the student type classification data;

[0060] Step S32: acquiring static analysis data of the shooting image, and acquiring a rectangular coordinate system of the model space according to the static analysis data of the shooting image;

[0061] Step S33: During the process of shooting monitoring of the process tracking student, the time point when the process tracking student pulls the trigger is marked as the cycle start time point, the time point when the projectile leaves the shooting equipment is marked as the cycle end time point, and the period between the cycle start time point and the cycle end time point is marked as the shooting monitoring cycle;

[0062] Step S34: selecting a plurality of position feature points in the shooting port area of the shooting device, and selecting a sample position feature point from the plurality of position feature points obtained;

[0063] Step S35: performing periodic motion amplitude analysis on the sample position feature points, and obtaining the initial oscillation amplitude corresponding to the sample position feature points according to the analysis results;

[0064] Step S36: respectively obtain the initial oscillation amplitude corresponding to each position feature point, and average the obtained multiple initial oscillation amplitudes to obtain the peak oscillation amplitude during the shooting period.

[0065] Furthermore, the step S35 further includes the following specific steps:

[0066] The video stream image of the sample position feature points in the shooting monitoring period is acquired through the shooting area space model to obtain the sample position video stream image;

[0067] The sample position video stream image is intercepted frame by frame to obtain a plurality of frame sample position images, and the time points at which the obtained plurality of frame sample position images are intercepted are marked in chronological order as Q1 interception time point to Qb interception time point;

[0068] In the rectangular coordinate system of the model space, the coordinate position of the sample position feature point at the Q1 interception time point is marked as the initial coordinate position, and the coordinate positions corresponding to the sample position feature point from the Q1 interception time point to the Qa interception time point are obtained respectively, and the coordinate positions from Q1 to Qb are obtained;

[0069] In the rectangular coordinate system of the model space, the coordinate distance values from the Q1 coordinate position to the Qb coordinate position and the initial coordinate position are obtained respectively, and the oscillation distance from the Q1 feature point to the Qb feature point is obtained;

[0070] In the existing plane rectangular coordinate system, the interception time point is marked as the horizontal coordinate, and the characteristic point oscillation distance is marked as the vertical coordinate to obtain the oscillation plane rectangular coordinate system;

[0071] In the rectangular coordinate system of the oscillation plane, the Q1 interception time point is set as the horizontal coordinate, and the Q1 characteristic point oscillation distance is set as the vertical coordinate to obtain the Q1 oscillation coordinate point. Similarly, the Qb interception time point is set as the horizontal coordinate, and the Qb characteristic point oscillation distance is set as the vertical coordinate to obtain the Qb oscillation coordinate point. The Q1 oscillation coordinate point and the Qb oscillation coordinate point are connected respectively to obtain the sample oscillation curve;

[0072] The oscillation coordinate point corresponding to the extreme point of the ordinate in the sample oscillation curve is obtained to obtain multiple extreme oscillation coordinate points, and the horizontal coordinate values of the multiple extreme oscillation coordinate points are compared respectively. The extreme oscillation coordinate point with the smallest horizontal coordinate value is marked as the initial positive oscillation coordinate point, and the extreme oscillation coordinate point with a horizontal coordinate value just smaller than the initial positive oscillation coordinate point is marked as the initial negative oscillation coordinate point. The sum of the ordinates of the initial positive oscillation coordinate point and the initial negative oscillation coordinate point is calculated to obtain the initial oscillation amplitude corresponding to the characteristic point of the sample position.

[0073] Furthermore, the step S4 further includes the following specific steps:

[0074] Obtain the static analysis data of the shooting image and the peak oscillation amplitude during the shooting period respectively;

[0075] Obtain an oscillation amplitude reference interval; if the peak oscillation amplitude during the shooting period is not within the oscillation amplitude reference interval, determine that there is an operational abnormality in the student's firing process during the process tracking, and output a firing operation abnormality warning; if the peak oscillation amplitude during the shooting period is within the oscillation amplitude reference interval, determine that there is no operational abnormality in the student's firing process during the process tracking;

[0076] Analyze the shooting posture tracking results of process tracking students based on the static analysis data of shooting images, and provide feedback on the key correction points of shooting posture to process tracking students based on the analysis;

[0077] The details are as follows:

[0078] Obtain static analysis data of the shooting image, and obtain the J1 posture angle deviation ratio to the Ja posture angle deviation ratio based on the static analysis data of the shooting image. If the J1 posture angle deviation ratio to the Ja posture angle deviation ratio are both 0, there is no need to provide feedback on the key correction points of the shooting posture to the process tracking student;

[0079] If any value of the J1 posture angle deviation ratio to the Ja posture angle deviation ratio is not 0, a numerical comparison is performed on the J1 posture angle deviation ratio to the Ja posture angle deviation ratio, and the operation part corresponding to the maximum posture angle deviation ratio is marked as the key correction part of the shooting posture, and the key correction part of the shooting posture is output.

[0080] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0081] 1. The present invention divides trainees into categories based on the shooting results displayed by the target detector, creates a shooting area spatial model, performs static shooting image analysis and dynamic shooting image analysis on trainees who fail shooting training, and provides monitoring result feedback based on the analysis results, which can improve the accuracy of image monitoring results;

[0082] 2. The present invention creates a model space rectangular coordinate system in the shooting area space model to analyze the posture angle deviation of the trainees, and feeds back the key correction parts of the shooting posture based on the analysis results, which can improve the shooting training effect of the trainees. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0084] Figure 1 It is a diagram of the implementation steps of the present invention;

[0085] Figure 2 is the training target image of the present invention;

[0086] Figure 3 Schematic diagram of the sample oscillation curve of the present invention. DETAILED DESCRIPTION

[0087] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0088] Example 1

[0089] See also Figure 1 The present invention provides a technical solution: a target recognition and tracking method based on multi-sensor fusion technology, comprising the following specific steps:

[0090] Step S1: spatially modeling the target shooting area to obtain a shooting area spatial model, selecting a number of trainees who are shooting in the shooting area spatial model, collecting and analyzing target detector images for each trainee, and classifying the trainees into qualified shooting trainees and process tracking trainees based on the analysis results to obtain trainee shooting collection data;

[0091] The step S1 further includes the following specific steps:

[0092] In the shooting training range, the shooting area of the students is marked to obtain the target shooting area, and the 3D structured light technology is used to create a three-dimensional space of the target shooting area to obtain a three-dimensional model of the shooting area;

[0093] The three-dimensional model of the shooting area is divided into a plurality of model monitoring sub-areas, and a sample model monitoring sub-area is randomly selected from the plurality of obtained model monitoring sub-areas;

[0094] Perform camera perspective analysis on the sample model monitoring sub-area, and obtain the optimal spatial perspective camera corresponding to the sample monitoring model sub-area based on the analysis results;

[0095] The details are as follows:

[0096] Acquire the real-time images corresponding to each environmental camera in the shooting training range, mark the environmental cameras with sample model monitoring sub-areas in the real-time images as viewpoint cameras to be analyzed, and select a sample camera to be analyzed from the multiple viewpoint cameras to be analyzed;

[0097] Perform camera perspective analysis on the sample camera to be analyzed, and obtain the environmental sampling distance of the sample camera to be analyzed to the sample monitoring model sub-area based on the analysis results;

[0098] The details are as follows:

[0099] Select multiple environmental monitoring points in the sample monitoring model sub-area, obtain the spatial distance between each environmental monitoring point and the sample camera to be analyzed, obtain multiple camera spatial distances, and calculate the average of the obtained camera spatial distances to obtain the camera shooting distance;

[0100] Obtain the focal length corresponding to the sample camera to be analyzed to obtain the camera focal length, and obtain the image sensor size corresponding to the sample camera to be analyzed to obtain the sensor size value;

[0101] Obtain the horizontal pixel number value corresponding to the sample camera image to be analyzed, and obtain the camera pixel number value;

[0102] The camera shooting distance, camera focal length, sensor size value and camera pixel number value are calculated to obtain the environmental sampling distance of the sample to be analyzed camera to the sample monitoring model sub-area;

[0103] The environmental sampling distance of the sample to be analyzed camera to the sample monitoring model sub-area is calculated. The specific formula is as follows:

[0104]

[0105] Among them, Gsd is the environmental sampling distance of the sample to be analyzed camera to the sample monitoring model sub-area, Cci is the sensor size value, Psj is the camera shooting distance, Jdj is the camera focal length, and Xss is the number of camera pixels;

[0106] Repeat the process of obtaining the environmental sampling distance of the sample camera to be analyzed to the sample monitoring model sub-area, obtain the environmental sampling distance of each camera to be analyzed from the sample monitoring model sub-area respectively, and compare the numerical values of the obtained multiple environmental sampling distances, and mark the camera to be analyzed with the smallest numerical value as the best spatial viewing angle camera corresponding to the sample monitoring model sub-area;

[0107] Repeat the process of acquiring the best spatial perspective camera corresponding to the sample monitoring model sub-area, and acquire the best spatial perspective camera corresponding to each model monitoring sub-area respectively;

[0108] The real-time image captured by the best spatial perspective camera is used to fill the three-dimensional model of the shooting area to obtain a spatial model of the shooting area;

[0109] Acquire trainees who are shooting in the shooting area spatial model, and arbitrarily select a target trainee from the acquired multiple trainees;

[0110] Conduct shooting level analysis on target training students and classify them into different types based on the analysis results;

[0111] The details are as follows:

[0112] An image of the training target after the target training student shoots is acquired to obtain a training target image, the number of shooting bullet holes in the training target image is counted to obtain the number of shooting bullet holes, and the number of shootings corresponding to the target training student is acquired to obtain the number of shootings by the student;

[0113] It should be noted here that:

[0114] In the present application, the shooting training involved here requires multiple shooting, that is, firing multiple bullets.

[0115] It should be noted here that:

[0116] In this application, there is no situation where a bullet hole has multiple shooting impact points.

[0117] If the number of shooting bullet holes and the number of shots taken by the trainee are not equal, the target training trainee will be classified as a process tracking trainee;

[0118] See also Figure 2 If the number of bullet holes is equal to the number of shots taken by the trainee, the distance between each bullet hole and the center of the target image is averaged to obtain the shooting distance deviation.

[0119] Obtaining a preset shooting distance deviation interval; if the shooting distance deviation is within the preset shooting distance deviation interval, classifying the target training student as a qualified shooting student; if the shooting distance deviation is not within the preset shooting distance deviation interval, classifying the target training student as a process tracking student;

[0120] It should be noted here that:

[0121] In this application, the qualified shooting students mentioned here include those whose shooting distance deviation is at the boundary of the preset shooting distance deviation range;

[0122] Get the preset deviation range of the shooting distance as follows:

[0123] The lower limit of the preset deviation range of the shooting distance involved here is specifically 0, that is, there is no shooting distance deviation;

[0124] Obtain historical shooting data, obtain the shooting distance deviations corresponding to several qualified shooting students based on the historical shooting values, and mark the shooting distance deviation with the largest value as the upper limit of the preset shooting distance deviation range;

[0125] The numerical range between the lower limit of the shooting distance preset deviation interval and the upper limit of the shooting distance preset deviation interval is marked as the shooting distance preset deviation interval.

[0126] Repeat the process of analyzing the shooting level of the target trainees, analyze the shooting level of each trainee respectively, and divide the trainees into process tracking trainees and shooting qualified trainees according to the analysis results, and obtain trainee type classification data;

[0127] The shooting area spatial model and student type classification data are defined as student shooting acquisition data.

[0128] Step S2: creating a model space rectangular coordinate system in the shooting area space model based on the student's shooting data, performing posture angle deviation analysis on the process tracking student through the model space rectangular coordinate system, and obtaining shooting image static analysis data based on the analysis results;

[0129] The step S2 further includes the following specific steps:

[0130] Obtain the students' shooting data, obtain the shooting area space model and student type classification data based on the students' shooting data, and obtain the process tracking students based on the student type classification data;

[0131] In the historical shooting image, multiple different types of shooting posture monitoring angles are set, and the set shooting posture monitoring angles are named J1 posture monitoring angle to Ja posture monitoring angle respectively;

[0132] It should be noted here that:

[0133] In this application, J mentioned here is the sign symbol corresponding to the shooting posture monitoring angle, a mentioned here is the numerical value corresponding to the shooting posture monitoring angle, and a is an integer greater than 0;

[0134] In the present application, the J1 posture monitoring angle involved here can be the shoulder joint abduction angle, the J2 posture monitoring angle can be the elbow joint flexion angle, the J3 posture monitoring angle can be the wrist joint neutral position, and the J4 posture monitoring angle can be the ipsilateral hip joint abduction angle.

[0135] Creating a spatial rectangular coordinate system in the shooting area spatial model to obtain a model spatial rectangular coordinate system;

[0136] The details are as follows:

[0137] In the shooting area spatial model, the plane where the process tracking student stands is marked as the first model feature plane. An endpoint of the first model feature plane is randomly selected as the first model feature point. A plane perpendicular to the first model feature plane is drawn through the first model feature point to obtain the second model feature plane.

[0138] In the first model characteristic plane, a straight line is drawn through the first model characteristic point to obtain the first model characteristic line. In the first model characteristic plane, a straight line is drawn through the first model characteristic point and perpendicular to the first model characteristic line to obtain the second model characteristic line.

[0139] In the second model characteristic plane, a straight line perpendicular to the first model characteristic line is drawn through the first model characteristic point to obtain a third model characteristic line;

[0140] Mark the first model feature point as the coordinate origin, the first model feature line as the coordinate x-axis, the second model feature line as the coordinate y-axis, and the third model feature line as the coordinate z-axis to obtain the model space rectangular coordinate system;

[0141] When the J1 posture monitoring angle is the shoulder joint abduction angle, the J1 posture monitoring angle is acquired to obtain a value of the J1 posture monitoring angle;

[0142] The details are as follows:

[0143] In the rectangular coordinate system of the model space, mark the highest point of the acromion process of the process tracking student as posture coordinate point A, mark the center point of the greater tuberosity of the humerus of the process tracking student as posture coordinate point B, and mark the center point of the sternoclavicular joint of the process tracking student as posture coordinate point C;

[0144] It should be noted here that:

[0145] In this application, the greater tuberosity of the humerus referred to herein is the greater tuberosity of the humerus closest to the acromion process, and the sternoclavicular joint referred to herein is the sternoclavicular joint closest to the acromion process;

[0146] The vector is calculated by the pose coordinate point A (x1, y1, z1) and the pose coordinate point B (x2, y2, z2)

[0147] vector

[0148] The vector is calculated by the posture coordinate point A (x1, y1, z1) and the posture coordinate point C (x3, y3, z3)

[0149] vector

[0150] Pair Vector Calculate the modulus length Pair Vector Calculate the modulus length

[0151] Calculating vectors With vector The dot product of

[0152] Calculated by the cosine formula The angle value of θ is calculated by the inverse cosine function arccos(cos(θ)) to obtain the J1 posture monitoring angle value;

[0153] Repeat the process of acquiring the J1 posture monitoring angle value, and acquire the angle values of the J2 posture monitoring angle to the Ja posture monitoring angle respectively, to obtain the J2 posture monitoring angle value to the Ja posture monitoring angle value;

[0154] Respectively obtain the posture monitoring reference angle intervals corresponding to the J1 posture monitoring angle value to the Ja posture monitoring angle value, and obtain the J1 posture monitoring angle interval to the Ja posture monitoring angle interval;

[0155] It should be noted here that:

[0156] In the present application, several historical training students with standard shooting postures are selected, and the J1 posture monitoring angle value corresponding to each historical training student is obtained respectively. The obtained multiple J1 posture monitoring angle values are averaged to obtain a first posture angle index value, the obtained multiple J1 posture monitoring angle values are standard deviation calculated to obtain a second posture angle index value, the sum of the first posture angle index value and the second posture angle index value is calculated to obtain an upper limit of the J1 posture monitoring angle interval, the difference between the first posture angle index value and the second posture angle index value is calculated to obtain a lower limit of the J1 posture monitoring angle interval, and the numerical range between the lower limit of the J1 posture monitoring angle interval and the upper limit of the J1 posture monitoring angle interval is marked as the J1 posture monitoring angle interval;

[0157] Obtain the interval difference between the J1 posture monitoring angle value and the Ja posture monitoring angle interval, obtain the interval angle range corresponding to the J1 posture monitoring angle interval, calculate the ratio of the interval difference to the interval angle range, and obtain the J1 posture angle deviation ratio;

[0158] It should be noted here that:

[0159] In this application, if the J1 posture monitoring angle value is within the J1 posture monitoring angle interval, the J1 posture monitoring angle value is 0; if the J1 posture monitoring angle value is greater than the J1 posture monitoring angle interval, the J1 posture monitoring angle value is the difference between the J1 posture monitoring angle value and the upper limit of the J1 posture monitoring angle interval; if the J1 posture monitoring angle value is less than the J1 posture monitoring angle interval, the J1 posture monitoring angle value is the difference between the J1 posture monitoring angle value and the lower limit of the J1 posture monitoring angle interval;

[0160] Repeat the process of obtaining the J1 posture angle deviation ratio to obtain the posture angle deviation ratios corresponding to the J2 posture monitoring angle to the Ja posture monitoring angle, and obtain the J2 posture angle deviation ratio to the Ja posture angle deviation ratio;

[0161] The model space rectangular coordinate system and the J1 posture angle deviation ratio to the Ja posture angle deviation ratio are defined as shooting image static analysis data;

[0162] Step S3: performing shooting stability analysis on the process tracking students in the shooting monitoring period based on the students' shooting collection data and shooting image static analysis data to obtain the peak oscillation amplitude during the shooting period;

[0163] The step S3 further includes the following specific steps:

[0164] Obtain the students' shooting data, obtain the shooting area space model and student type classification data based on the students' shooting data, and obtain the process tracking students based on the student type classification data;

[0165] Obtaining static analysis data of the shooting image, and obtaining a rectangular coordinate system of the model space according to the static analysis data of the shooting image;

[0166] During the process of shooting monitoring of the process tracking student, the time point when the process tracking student pulls the trigger is marked as the cycle start time point, the time point when the projectile leaves the shooting device is marked as the cycle end time point, and the period between the cycle start time point and the cycle end time point is marked as the shooting monitoring cycle;

[0167] Selecting a plurality of position feature points in the shooting port area of the shooting device, and selecting a sample position feature point from the plurality of position feature points obtained;

[0168] Perform periodic motion amplitude analysis on the characteristic points of the sample position, and obtain the initial oscillation amplitude corresponding to the characteristic points of the sample position according to the analysis results;

[0169] The details are as follows:

[0170] The video stream image of the sample position feature points in the shooting monitoring period is acquired through the shooting area space model to obtain the sample position video stream image;

[0171] The sample position video stream image is intercepted frame by frame to obtain a plurality of frame sample position images, and the time points at which the obtained plurality of frame sample position images are intercepted are marked in chronological order as Q1 interception time point to Qb interception time point;

[0172] It should be noted here that:

[0173] In the present application, Q referred to here is the sign symbol corresponding to the interception time point, b is the quantity value corresponding to the interception time point, and b is an integer greater than 0.

[0174] In the rectangular coordinate system of the model space, the coordinate position of the sample position feature point at the Q1 interception time point is marked as the initial coordinate position, and the coordinate positions corresponding to the sample position feature point from the Q1 interception time point to the Qa interception time point are obtained respectively, and the coordinate positions from Q1 to Qb are obtained;

[0175] It should be noted here that:

[0176] In this application, the Q1 coordinate position involved here coincides with the initial coordinate position.

[0177] In the rectangular coordinate system of the model space, the coordinate distance values from the Q1 coordinate position to the Qb coordinate position and the initial coordinate position are obtained respectively, and the oscillation distance from the Q1 feature point to the Qb feature point is obtained;

[0178] It should be noted here that:

[0179] In this application, since the oscillation distance of the Q1 feature point coincides with the initial coordinate phase, the oscillation distance of the Q1 feature point is 0;

[0180] In the existing plane rectangular coordinate system, the interception time point is marked as the horizontal coordinate, and the characteristic point oscillation distance is marked as the vertical coordinate to obtain the oscillation plane rectangular coordinate system;

[0181] In the rectangular coordinate system of the oscillation plane, the Q1 interception time point is set as the horizontal coordinate, and the Q1 characteristic point oscillation distance is set as the vertical coordinate to obtain the Q1 oscillation coordinate point. Similarly, the Qb interception time point is set as the horizontal coordinate, and the Qb characteristic point oscillation distance is set as the vertical coordinate to obtain the Qb oscillation coordinate point. The Q1 oscillation coordinate point and the Qb oscillation coordinate point are connected respectively to obtain the sample oscillation curve;

[0182] See also Figure 3 , obtaining the oscillation coordinate point corresponding to the extreme value point of the ordinate in the sample oscillation curve to obtain multiple extreme value oscillation coordinate points, comparing the abscissa values of the multiple extreme value oscillation coordinate points respectively, marking the extreme value oscillation coordinate point with the smallest abscissa value as the initial positive oscillation coordinate point, marking the extreme value oscillation coordinate point with an abscissa value just smaller than the initial positive oscillation coordinate point as the initial negative oscillation coordinate point, calculating the sum of the ordinates of the initial positive oscillation coordinate point and the initial negative oscillation coordinate point, and obtaining the initial oscillation amplitude corresponding to the sample position characteristic point;

[0183] Repeat the process of obtaining the initial oscillation amplitude corresponding to the sample position feature point, obtain the initial oscillation amplitude corresponding to each position feature point respectively, and calculate the average of the obtained multiple initial oscillation amplitudes to obtain the peak oscillation amplitude during the shooting period;

[0184] Step S3: providing tracking result feedback to the process tracking trainees based on the static analysis data of the shooting image and the peak oscillation amplitude during the shooting period;

[0185] The details are as follows:

[0186] Obtain the static analysis data of the shooting image and the peak oscillation amplitude during the shooting period respectively;

[0187] Obtain an oscillation amplitude reference interval; if the peak oscillation amplitude during the shooting period is not within the oscillation amplitude reference interval, determine that there is an operational abnormality in the student's firing process during the process tracking, and output a firing operation abnormality warning; if the peak oscillation amplitude during the shooting period is within the oscillation amplitude reference interval, determine that there is no operational abnormality in the student's firing process during the process tracking;

[0188] It should be noted here that:

[0189] In this application, the transmission process involved herein does not have any operational anomalies including the oscillation amplitude reference interval boundary;

[0190] The oscillation amplitude reference interval is obtained as follows:

[0191] Selecting several historical trainees whose firing processes had no operational anomalies, obtaining the peak oscillation amplitude of the shooting period corresponding to each historical trainee, averaging the obtained peak oscillation amplitudes of the multiple shooting periods to obtain a first oscillation amplitude index value, calculating the standard deviation of the obtained peak oscillation amplitudes of the multiple shooting periods to obtain a second oscillation amplitude index value, calculating the sum of the first oscillation amplitude index value and the second oscillation amplitude index value to obtain an upper limit of the oscillation amplitude reference interval, calculating the difference between the first oscillation amplitude index value and the second oscillation amplitude index value to obtain a lower limit of the oscillation amplitude reference interval, and marking the numerical range between the lower limit of the oscillation amplitude reference interval and the upper limit of the oscillation amplitude reference interval as the oscillation amplitude reference interval;

[0192] Step S4: analyzing the shooting posture tracking results of the process tracking student based on the static analysis data of the shooting image, and providing feedback on the key correction points of the shooting posture to the process tracking student based on the analysis;

[0193] The step S4 further includes the following specific steps:

[0194] Obtain static analysis data of the shooting image, and obtain the J1 posture angle deviation ratio to the Ja posture angle deviation ratio based on the static analysis data of the shooting image. If the J1 posture angle deviation ratio to the Ja posture angle deviation ratio are both 0, there is no need to provide feedback on the key correction points of the shooting posture to the process tracking student;

[0195] If any value of the J1 posture angle deviation ratio to the Ja posture angle deviation ratio is not 0, then the J1 posture angle deviation ratio to the Ja posture angle deviation ratio is numerically compared, and the operating part corresponding to the maximum posture angle deviation ratio is marked as the key correction part of the shooting posture, and the key correction part of the shooting posture is output;

[0196] It should be noted here that:

[0197] In this application, the shooting posture focus correction parts involved here can exist in multiple places at the same time;

[0198] In this application, the operating part corresponding to the maximum posture angle deviation ratio is specifically the part that constitutes the posture angle. For example, if the maximum posture angle deviation ratio is the shoulder joint abduction angle, the key correction parts of the output shooting posture are the arms and shoulders.

[0199] In this application, if a corresponding calculation formula appears, the above calculation formula is dimensionless and its numerical calculation is performed. The weight coefficient, proportional coefficient and other coefficients in the formula are set to a result value obtained by quantifying each parameter. Regarding the size of the weight coefficient and the proportional coefficient, as long as it does not affect the proportional relationship between the parameter and the result value, it is acceptable.

[0200] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A target recognition and tracking method based on multi-sensor fusion technology, characterized in that: include: Step S1: spatially modeling the target shooting area to obtain a shooting area spatial model, collecting and analyzing target detector images for each trainee, and classifying several trainees into qualified shooting trainees and process tracking trainees based on the analysis results to obtain trainee shooting collection data; Step S2: creating a model space rectangular coordinate system in the shooting area space model based on the student's shooting data, performing posture angle deviation analysis on the process tracking student through the model space rectangular coordinate system, and obtaining shooting image static analysis data based on the analysis results; Step S3: performing shooting stability analysis on the process tracking students in the shooting monitoring period based on the students' shooting collection data and shooting image static analysis data to obtain the peak oscillation amplitude during the shooting period; Step S4: providing tracking result feedback to the process tracking trainees based on the static analysis data of the shooting image and the peak oscillation amplitude during the shooting period.

2. The target recognition and tracking method based on multi-sensor fusion technology according to claim 1, characterized in that: The step S1 further includes the following steps: Step S11: Mark the student's shooting area to obtain a target shooting area, create a three-dimensional space for the target shooting area, and obtain a three-dimensional model of the shooting area; Step S12: Divide the three-dimensional model of the shooting area into a number of model monitoring sub-areas, and arbitrarily select a sample model monitoring sub-area; Step S13: performing camera perspective analysis on the sample model monitoring sub-area, and obtaining the optimal spatial perspective camera corresponding to the sample monitoring model sub-area according to the analysis result; Step S14: obtaining the optimal spatial viewing angle camera corresponding to each model monitoring sub-area; Step S15: using the real-time image captured by the best spatial viewing angle camera to perform image filling on the three-dimensional model of the shooting area to obtain a spatial model of the shooting area; Step S16: acquiring trainees who are shooting in the shooting area space model, and arbitrarily selecting a target trainee from the acquired multiple trainees; Step S17: Analyzing the shooting level of the target training students and classifying the target training students into different types according to the analysis results; Step S18: performing a shooting level analysis on each trainee, and dividing the trainees into process tracking trainees and shooting qualified trainees according to the analysis results, thereby obtaining trainee type classification data; Step S19: Define the shooting area space model and the student type classification data as student shooting acquisition data.

3. The target recognition and tracking method based on multi-sensor fusion technology according to claim 2, characterized in that: The step S13 further includes the following specific steps: Step S131: Acquire the real-time images corresponding to each environmental camera in the shooting training range, mark the environmental cameras with the sample model monitoring sub-area in the real-time images as viewpoint cameras to be analyzed, and select a sample camera to be analyzed from the multiple viewpoint cameras to be analyzed; Step S132: performing camera angle analysis on the sample camera to be analyzed, and obtaining the environmental sampling distance of the sample camera to be analyzed to the sample monitoring model sub-area according to the analysis result; Step S133: Obtain the environmental sampling distance of each camera to be analyzed for the sample monitoring model sub-area respectively, compare the numerical values of the multiple environmental sampling distances obtained, and mark the camera to be analyzed with the minimum numerical value as the optimal spatial perspective camera corresponding to the sample monitoring model sub-area.

4. The target recognition and tracking method based on multi-sensor fusion technology according to claim 3 is characterized in that: The step S132 further includes the following steps: Select multiple environmental monitoring points in the sample monitoring model sub-area, obtain the average spatial distance between the multiple environmental monitoring points and the sample to be analyzed camera, and obtain the camera shooting distance; Obtain the focal length corresponding to the sample camera to be analyzed to obtain the camera focal length, and obtain the image sensor size corresponding to the sample camera to be analyzed to obtain the sensor size value; Obtain the horizontal pixel number value corresponding to the sample camera image to be analyzed, and obtain the camera pixel number value; The camera shooting distance Psj, camera focal length Jdj, sensor size value Cci and camera pixel number value Xss are calculated to obtain the environmental sampling distance Gsd of the sample to be analyzed camera to the sample monitoring model sub-area. The specific formula is as follows:

5. The target recognition and tracking method based on multi-sensor fusion technology according to claim 2, characterized in that: The step S17 further includes the following specific steps: An image of the training target after the target training student shoots is acquired to obtain a training target image, the number of shooting bullet holes in the training target image is counted to obtain the number of shooting bullet holes, and the number of shootings corresponding to the target training student is acquired to obtain the number of shootings by the student; If the number of shooting bullet holes and the number of shots taken by the trainee are not equal, the target training trainee will be classified as a process tracking trainee; If the number of bullet holes is equal to the number of shots taken by the trainee, the distance between each bullet hole and the bull's eye in the training target image is averaged to obtain the shooting distance deviation. Obtain the preset deviation interval of the shooting distance. If the shooting distance deviation is within the preset deviation interval, the target training student is classified as a qualified shooting student. If the shooting distance deviation is not within the preset deviation interval, the target training student is classified as a process tracking student.

6. The target recognition and tracking method based on multi-sensor fusion technology according to claim 1, characterized in that: The step S2 further includes the following steps: Step S21: Acquire student shooting data, acquire a shooting area spatial model and student type classification data based on the student shooting data, and acquire process tracking students based on the student type classification data; Step S22: In the historical shooting image, set the J1 posture monitoring angle to the Ja posture monitoring angle; Step S23: creating a spatial rectangular coordinate system in the shooting area spatial model to obtain a model spatial rectangular coordinate system; Step S24: when the J1 posture monitoring angle is the shoulder joint abduction angle, the J1 posture monitoring angle is acquired to obtain a value of the J1 posture monitoring angle; Step S25: acquiring angle values from the J2 posture monitoring angle to the Ja posture monitoring angle, respectively, to obtain values from the J2 posture monitoring angle to the Ja posture monitoring angle; Step S26: respectively obtaining the posture monitoring reference angle intervals corresponding to the J1 posture monitoring angle value to the Ja posture monitoring angle value, and obtaining the J1 posture monitoring angle interval to the Ja posture monitoring angle interval; Step S27: Obtain the interval difference between the J1 posture monitoring angle value and the Ja posture monitoring angle interval, obtain the interval angle range corresponding to the J1 posture monitoring angle interval, calculate the ratio of the interval difference to the interval angle range, and obtain the J1 posture angle deviation ratio; Step S28: respectively obtaining posture angle deviation ratios corresponding to the J2 posture monitoring angle to the Ja posture monitoring angle, and obtaining the J2 posture angle deviation ratio to the Ja posture angle deviation ratio; Step S29: defining the model space rectangular coordinate system and the J1 posture angle deviation ratio to the Ja posture angle deviation ratio as shooting image static analysis data.

7. The target recognition and tracking method based on multi-sensor fusion technology according to claim 6, characterized in that: The step S24 further includes the following specific steps: In the rectangular coordinate system of the model space, mark the highest point of the acromion process of the process tracking student as posture coordinate point A, mark the center point of the greater tuberosity of the humerus of the process tracking student as posture coordinate point B, and mark the center point of the sternoclavicular joint of the process tracking student as posture coordinate point C; The vector is calculated by the pose coordinate point A (x1, y1, z1) and the pose coordinate point B (x2, y2, z2) vector The vector is calculated by the posture coordinate point A (x1, y1, z1) and the posture coordinate point C (x3, y3, z3) vector Pair Vector Calculate the modulus length Pair Vector Calculate the modulus length Calculating vectors With vector The dot product of Calculated by the cosine formula The angle value of θ is calculated by the inverse cosine function arccos(cos(θ)) to obtain the J1 posture monitoring angle value.

8. The target recognition and tracking method based on multi-sensor fusion technology according to claim 1, characterized in that: The step S3 further includes the following steps: Step S31: Acquire student shooting data, acquire a shooting area spatial model and student type classification data based on the student shooting data, and acquire process tracking students based on the student type classification data; Step S32: acquiring static analysis data of the shooting image, and acquiring a rectangular coordinate system of the model space according to the static analysis data of the shooting image; Step S33: marking a shooting monitoring cycle during the process of performing shooting monitoring on the process tracking student; Step S34: selecting a plurality of position feature points in the shooting port area of the shooting device, and selecting a sample position feature point from the plurality of position feature points obtained; Step S35: performing periodic motion amplitude analysis on the sample position feature points, and obtaining the initial oscillation amplitude corresponding to the sample position feature points according to the analysis results; Step S36: respectively obtain the initial oscillation amplitude corresponding to each position feature point, and average the obtained multiple initial oscillation amplitudes to obtain the peak oscillation amplitude during the shooting period.

9. The target recognition and tracking method based on multi-sensor fusion technology according to claim 8, characterized in that: The step S35 further includes the following steps: The video stream image of the sample position feature points in the shooting monitoring period is acquired through the shooting area space model to obtain the sample position video stream image; The video stream image of the sample position is intercepted frame by frame, and the interception time point is obtained to obtain the interception time point Q1 to the interception time point Qb; In the rectangular coordinate system of the model space, the coordinate position of the sample position feature point at the Q1 interception time point is marked as the initial coordinate position, and the coordinate positions corresponding to the sample position feature point from the Q1 interception time point to the Qa interception time point are obtained respectively, and the coordinate positions from Q1 to Qb are obtained; In the rectangular coordinate system of the model space, the coordinate distance values from the Q1 coordinate position to the Qb coordinate position and the initial coordinate position are obtained respectively, and the oscillation distance from the Q1 feature point to the Qb feature point is obtained; In the existing plane rectangular coordinate system, the interception time point is marked as the horizontal coordinate, and the characteristic point oscillation distance is marked as the vertical coordinate to obtain the oscillation plane rectangular coordinate system; In the rectangular coordinate system of the oscillation plane, mark the Q1 oscillation coordinate point to the Qb oscillation coordinate point, and connect the Q1 oscillation coordinate point to the Qb oscillation coordinate point to obtain the sample oscillation curve; The oscillation coordinate points corresponding to the extreme value points of the ordinate in the sample oscillation curve are obtained to obtain multiple extreme value oscillation coordinate points. The extreme value oscillation coordinate point with the smallest abscissa value is marked as the initial positive oscillation coordinate point, and the extreme value oscillation coordinate point with abscissa value just smaller than the initial positive oscillation coordinate point is marked as the initial negative oscillation coordinate point. The sum of the ordinates of the initial positive oscillation coordinate point and the initial negative oscillation coordinate point is calculated to obtain the initial oscillation amplitude corresponding to the characteristic point of the sample position.

10. The target recognition and tracking method based on multi-sensor fusion technology according to claim 1, characterized in that: The step S4 further includes the following specific steps: Obtain the static analysis data of the shooting image and the peak oscillation amplitude during the shooting period respectively; Obtain an oscillation amplitude reference interval; if the peak oscillation amplitude during the shooting period is not within the oscillation amplitude reference interval, determine that there is an operational abnormality in the student's firing process during the process tracking, and output a firing operation abnormality warning; if the peak oscillation amplitude during the shooting period is within the oscillation amplitude reference interval, determine that there is no operational abnormality in the student's firing process during the process tracking; Analyze the shooting posture tracking results of process tracking students based on the static analysis data of shooting images, and provide feedback on the key correction points of shooting posture to process tracking students based on the analysis; The details are as follows: Obtain static analysis data of the shooting image, and obtain the J1 posture angle deviation ratio to the Ja posture angle deviation ratio based on the static analysis data of the shooting image. If the J1 posture angle deviation ratio to the Ja posture angle deviation ratio are both 0, there is no need to provide feedback on the key correction points of the shooting posture to the process tracking student; If any value of the J1 posture angle deviation ratio to the Ja posture angle deviation ratio is not 0, a numerical comparison is performed on the J1 posture angle deviation ratio to the Ja posture angle deviation ratio, and the operation part corresponding to the maximum posture angle deviation ratio is marked as the key correction part of the shooting posture, and the key correction part of the shooting posture is output.

Images