A fast calibration method and system for an infrared light gun

Abstract

A fast aiming calibration method and system of an infrared light gun, comprising: acquiring a screen image containing a reference image collected by a first camera; detecting a reference mark in the reference image, calculating a first homography matrix according to the detected position and a pre-stored reference position, and projecting a corner point of the reference image to a first camera image coordinate system through the matrix as a screen corner point position; acquiring coordinate mapping parameters between the first camera and a second camera, mapping the screen corner point position to a second camera image coordinate system to obtain a screen region boundary; acquiring an infrared image collected by the second camera, detecting a position of an infrared aiming light spot within the screen region boundary, and performing filtering and forward prediction compensation on the position to output a current aiming point position, mapping the current aiming point position to a standard output coordinate system to generate a final aiming coordinate output. The present application can realize fast automation of aiming positioning and calibration, while ensuring high precision and low delay of the aiming point output.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and interactive shooting simulation, and in particular to a rapid calibration method and system for an infrared light gun. Background Technology

[0002] Light gun positioning technology has evolved from the CRT monitor era to the present day, resulting in various technical approaches. Early scan-line detection methods utilized the scanning timing characteristics of CRT monitors for calibration. When the player pulled the trigger, a photoelectric sensor inside the light gun captured the light signal emitted from the screen, and the aiming position was determined by calculating the signal timing. However, with the widespread adoption of flat panel display technologies such as LCD, LED, and OLED, the scan-line detection method has become completely ineffective due to its strong dependence on the scanning timing of CRT monitors.

[0003] To adapt to flat panel displays, the industry has proposed several alternative solutions. One approach involves placing an infrared LED strip above or below the display, with a built-in infrared camera in the light gun detecting the strip's position to calculate the aiming direction. However, this solution provides relative position information, and positioning errors increase as the user moves, requiring frequent calibration. Another approach involves installing infrared LED markers at the four corners of the screen. The light gun calculates its own attitude and aiming position by detecting the marker coordinates, offering higher accuracy than the sensor strip solution. However, this requires additional infrared LED hardware to be attached or installed on the display, increasing system complexity and deployment costs. Therefore, existing light gun positioning solutions generally rely on additional hardware and lack a stable and reliable screen positioning mechanism.

[0004] In light gun positioning schemes using a binocular system comprised of a visible light camera and an infrared camera, the screen area located by the visible light camera needs to be mapped to the coordinate system of the infrared camera to limit the spot detection range. Traditional binocular positioning methods typically use a checkerboard calibration board, solving for the geometric relationship between the cameras by simultaneously capturing multiple sets of images of the calibration board from different angles using two cameras. However, when the two cameras operate within different spectral ranges, the infrared camera cannot observe the checkerboard pattern on the visible light calibration board, and the visible light camera also cannot observe the calibration features that are only sensitive to infrared light.

[0005] To address this, existing technologies have proposed solutions such as setting feature points on a calibration board that can be observed by both cameras simultaneously, or synthesizing composite maps by calculating the homography mappings of each spectral band separately. However, these solutions generally require dedicated calibration boards or specially designed cooperative targets, the calibration process is cumbersome and requires manual intervention, lacks an automated calibration quality assessment mechanism, and is difficult to quickly reproduce in real-world deployment scenarios.

[0006] In addition, existing light gun systems typically use the detected light spot coordinates as the output for aiming point tracking. On the one hand, image noise and algorithm uncertainty cause spatial jitter in the aiming point, affecting aiming accuracy. On the other hand, there is a system delay from image acquisition to coordinate output, and when the player moves the light gun quickly, the output position lags significantly behind the actual aiming position. Summary of the Invention

[0007] The main objective of this invention is to overcome the shortcomings of existing technologies, such as reliance on additional hardware for light gun calibration, cumbersome calibration processes, and jitter and delay in aiming point tracking. This invention provides a rapid calibration method and system for infrared light guns, which can achieve automatic screen positioning and automated calibration processes without the need for additional hardware deployment, and ensure high accuracy and low latency in aiming point output.

[0008] The present invention adopts the following technical solution:

[0009] A rapid aiming and calibration method for an infrared light gun includes the following steps:

[0010] S1 acquires a screen image captured by a first camera. The screen image includes a displayed reference image, which contains multiple reference markers. S1 detects each reference marker in the screen image, calculates a first homography matrix based on the position of each reference marker in the screen image and its pre-stored reference position, and projects the corner points of the reference image onto the first camera image coordinate system based on the first homography matrix, which are then used as the corner point positions of the screen.

[0011] S2 obtains the pre-calibrated coordinate mapping parameters between the first camera and the second camera, and uses the coordinate mapping parameters to map the corner position of the screen from the first camera image coordinate system to the second camera image coordinate system to obtain the screen area boundary in the second camera image coordinate system.

[0012] S3 acquires the infrared image of the screen captured by the second camera in real time, detects the infrared aiming spot within the boundary of the screen area, calculates the position of the infrared aiming spot in the image coordinate system of the second camera, performs filtering and forward prediction compensation processing on the detected position of the infrared aiming spot, and outputs the current aiming point position.

[0013] S4 maps the current aiming point position to the standard output coordinate system based on the perspective correspondence between the screen area boundary in the second camera image coordinate system and the preset standard output coordinate system, and generates the final aiming coordinate output.

[0014] In step S1, before detecting each reference marker in the screen image, the method further includes:

[0015] Frame buffer noise reduction processing is performed on the screen image captured by the first camera: a frame buffer with a preset number of frames is maintained, the current screen image is stored in the frame buffer, and the median value of multiple frames in the frame buffer is taken as the current frame output.

[0016] In step S1, the reference position pre-stored by the reference mark includes at least the coordinates of the four vertices of the reference mark in the reference image;

[0017] Based on the coordinates of the four vertices of each reference mark in the reference image and the corresponding vertex coordinates detected in the screen image, the first homography matrix corresponding to each reference mark is calculated respectively, and the corner point of the reference image is projected to the first camera image coordinate system through the corresponding first homography matrix to obtain the candidate projection position of the corner point of the reference image corresponding to each reference mark.

[0018] For each corner point of the reference image, calculate the Euclidean distance from the center of each reference marker to that corner point; assign a weight w_ij based on the Euclidean distance, with a larger weight for closer Euclidean distances.

[0019] w_ij = D / (d_ij + ε);

[0020] Where D is the diagonal length of the reference image, d_ij is the Euclidean distance from the center of the reference marker to the j-th corner point, and ε is a small constant to prevent division by zero;

[0021] The candidate projection position with the highest weight is selected as the final projection position of the corner point.

[0022] In step S1, the final projected position of each corner point is input into a Kalman filter for filtering. The state vector of the Kalman filter contains the position and velocity of the corner point. The Kalman filter predicts the position of the corner point based on the current state vector, extracts the current predicted position of the corner point using a measurement matrix, compares the current predicted position with the input position of the corner point, and corrects the predicted value based on the comparison result.

[0023] It also includes calculating the instability index `instability_i` for each corner point based on the state vector of the Kalman filter. This instability index is directly proportional to the speed and inversely proportional to the distance from the corner point to the center of the screen.

[0024] instability_i = ||v_i|| × D / 2 / (dist_i + ε) ;

[0025] Where ||v_i|| is the magnitude of the velocity of the i-th corner point, dist_i is the distance from the corner point to the center of the screen, and ε is a small constant to prevent division by zero;

[0026] Calculate the stability score (stability_score) by taking the largest instability index from each corner point:

[0027] stability_score = max(0, 100 - max_instability);

[0028] Where max_instability is the maximum value of the instability index, and max(.) is the maximum value;

[0029] When the stability score exceeds a preset threshold and remains so for a preset time, the corner positions of the current filtered output are determined as the corner positions of the screen.

[0030] The coordinate mapping parameter is the second homography matrix. In step S2, the positions of the screen corners in the first camera image coordinate system are first converted into homogeneous coordinates; then, the homogeneous coordinates are multiplied by the second homography matrix and normalized to obtain the positions of the screen corners in the second camera image coordinate system.

[0031] [x\', y\', w\']^T = H_color_to_ir × [x_c, y_c, 1]^T;

[0032] x_ir = x\' / w\', y_ir = y\' / w\';

[0033] Where [x', y', w']^T is the result of multiplying the homogeneous coordinates with the second homography matrix, [x_c, y_c, 1]^T is the homogeneous coordinates, H_color_to_ir is the second homography matrix, and (x_ir, y_ir) is the corresponding position of the corner point in the second camera coordinate system.

[0034] The screen area boundary is determined by the position of the corner point of the screen in the second camera image coordinate system; in step S3, detecting the infrared aiming spot within the screen area boundary specifically includes:

[0035] The infrared image is converted into a grayscale image. A region of interest (ROI) is constructed using the screen area boundary, and the ROI is used as a mask to limit the detection range. The grayscale image after limiting the detection range is Gaussian blurred. The blurred grayscale image is then segmented by a brightness threshold to generate a binarized image. Opening and closing operations are sequentially performed on the binarized image. Candidate spots are extracted through contour detection, and contours are filtered by area. The centroid of each filtered contour is calculated to obtain the position of the candidate spot in the second camera image coordinate system. When there are multiple candidate spots, the candidate spot with the largest area or the highest average brightness is selected as the infrared aiming spot.

[0036] In step S3, the detected position of the infrared aiming spot is filtered and forward prediction compensation is performed to output the current aiming point position, specifically including:

[0037] The detected position of the infrared aiming spot is input into a Kalman filter for filtering. The state vector of the Kalman filter contains the position and velocity of the infrared aiming spot. The state transition matrix of the Kalman filter is dynamically adjusted according to the time interval between adjacent frames.

[0038] When the infrared aiming spot is not detected, the predicted position is extrapolated based on the speed in the current filtering state and the preset prediction time, and the predicted position is output as the current aiming point position.

[0039] In step S4, the coordinates of the four corner points of the screen area boundary are calculated to determine the correspondence between each corner point and the four vertices of the rectangle of the standard output coordinate system; then, the size of the rectangle of the standard output coordinate system is determined according to the correspondence, and the four vertices of the standard output coordinate system are calculated according to a preset aspect ratio; a perspective transformation matrix is ​​calculated based on the four corner points and the four vertices of the standard output coordinate system; and the current aiming point position is mapped to the standard output coordinate system through the perspective transformation matrix.

[0040] Obtain the reference image containing multiple reference markers; extract the corner positions, image size, and vertex coordinates of each reference marker in the reference image, and store them in the calibration data file; the identification information and vertex coordinates of the reference markers are used as the reference positions of the reference markers.

[0041] A rapid aiming and calibration system for an infrared light gun, comprising:

[0042] A screen corner calibration module is used to acquire a screen image captured by a first camera. The screen image includes a displayed reference image, which contains multiple reference markers. The module detects each reference marker in the screen image, calculates a first homography matrix based on the position of each reference marker in the screen image and its pre-stored reference position, and projects the corner points of the reference image onto the first camera image coordinate system based on the first homography matrix, which are then used as the corner points of the screen.

[0043] The coordinate cross-domain mapping module is used to obtain the pre-calibrated coordinate mapping parameters between the first camera and the second camera, and use the coordinate mapping parameters to map the corner position of the screen from the first camera image coordinate system to the second camera image coordinate system to obtain the screen area boundary in the second camera image coordinate system.

[0044] The aiming point detection and tracking module is used to acquire infrared images of the screen captured by the second camera in real time, detect infrared aiming spots within the boundary of the screen area, calculate the position of the infrared aiming spots in the image coordinate system of the second camera, and perform filtering and forward prediction compensation processing on the detected position of the infrared aiming spots to output the current aiming point position.

[0045] The coordinate mapping output module is used to map the current aiming point position to the standard output coordinate system according to the perspective correspondence between the screen area boundary under the second camera image coordinate system and the preset standard output coordinate system, and generate the final aiming coordinate output.

[0046] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. In this invention, screen corner points are calibrated using a reference image, mapped to the coordinate system of a second camera, infrared light spots are detected and filtered within a defined range, the aiming point position is output, the aiming point is mapped to a standard coordinate system, and the final coordinates are output. Calibration can be completed quickly without manual intervention, ensuring high accuracy and low latency in the aiming point output.

[0048] 2. This invention utilizes reference marks displayed on the screen to achieve screen positioning, replacing the traditional approach of pasting or installing infrared LED markers on the screen border. This reduces system hardware costs and deployment complexity, and improves the system's versatility and ease of use.

[0049] 3. This invention adopts a distance-based weighted fusion strategy, which comprehensively utilizes the projection results of multiple reference markers. For each screen corner point, the projection value corresponding to the nearest marker is selected, which effectively improves the corner point positioning accuracy. At the same time, when some markers are occluded or detection fails, the system can still rely on the remaining markers to complete the positioning, which has good fault tolerance.

[0050] 4. This invention calculates the stability score based on the velocity state estimated by the Kalman filter, and requires the stability score to exceed a preset threshold and remain so for a preset time before the calibration is considered complete. This eliminates the subjectivity of manual judgment of calibration quality and ensures the consistency and reliability of the calibration results.

[0051] 5. This invention employs a low-latency Kalman filter, which adapts to frame rate fluctuations through an adaptive time step mechanism and extrapolates the predicted position when spot detection is lost through a forward prediction mechanism. This effectively suppresses spatial jitter of the aiming point and compensates for system latency, making the tracking performance in fast-moving aiming scenarios superior to traditional direct detection schemes.

[0052] 6. In this invention, the coordinate mapping parameters between the first camera and the second camera can be pre-calibrated and stored offline. When deployed online, only screen calibration and coordinate mapping need to be performed to quickly complete the entire calibration process, thus shortening the deployment time.

[0053] 7. In infrared spot detection, this invention utilizes screen corner points to construct a region of interest mask, limiting the detection range to the screen area, effectively eliminating interference from infrared light sources outside the screen, and improving the accuracy and stability of spot detection.

[0054] 8. This invention can be widely applied to scenarios requiring high-precision, low-latency light gun aiming and positioning, such as arcade shooting games, virtual reality (VR) shooting training, military and police simulated shooting training, and interactive multimedia displays. Attached Figure Description

[0055] Figure 1 This is the main flowchart of the method of the present invention;

[0056] Figure 2 This is a schematic diagram of the reference image of the present invention;

[0057] Figure 3 A schematic diagram of the homography projection of the reference image;

[0058] Figure 4 This is a schematic diagram of perspective transformation and screen coordinate mapping;

[0059] Figure 5 This is a diagram showing the main modules of the system of the present invention.

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0061] The present invention will be further described below through specific embodiments.

[0062] This embodiment proposes a rapid calibration method for an infrared light gun, based on a binocular camera and an infrared light gun. The binocular camera includes a first camera for acquiring visible light images and a second camera for acquiring infrared images. Both cameras have a field of view covering the entire screen area. The second camera is equipped with an infrared bandpass filter, allowing light in the near-infrared band (e.g., around 850nm) to pass through. The infrared light gun has a built-in infrared LED or infrared laser emitter. When the player pulls the trigger or aims, it emits an infrared beam towards the screen surface, forming a bright infrared spot. This infrared spot can be captured by the second camera to determine the position of the aiming point on the screen.

[0063] See Figure 1 The specific rapid calibration method includes the following steps:

[0064] S1 acquires a screen image captured by the first camera. The screen image contains a displayed reference image, which contains multiple reference markers. S1 detects each reference marker in the screen image, calculates a first homography matrix based on the position of each reference marker in the screen image and its pre-stored reference position, and projects the corner points of the reference image onto the image coordinate system of the first camera based on the first homography matrix to determine the projection position of the corner points of the reference image. This projection position is used as the corner point position of the screen.

[0065] In this step, the reference image can be marked with ArUco tags. ArUco tags are square reference tags based on binary encoding, consisting of a black border and an internal binary matrix. Each tag has a unique ID, facilitating rapid identification and differentiation within the image. See also... Figure 2 This is a schematic diagram of the reference image, where m1, m2, m3, and m4 represent four reference markers, and the vertices of the rectangle are the four corner points of the reference image.

[0066] The reference positions of each pre-stored benchmark are obtained in the following way: a benchmark image containing multiple benchmarks is acquired; the corner positions, image size, and vertex coordinates of each benchmark in the benchmark image are extracted and stored in the calibration data file; the identification information and vertex coordinates of the benchmarks are used as the reference positions of the benchmarks.

[0067] During the initialization phase, a baseline image (Aruco.jpg) containing multiple ArUco markers is read and displayed full-screen. An ArUco detector (e.g., using a DICT_6X6_250 dictionary) is used to detect all baseline markers in the image, extracting the four vertex coordinates of each marker and recording the original size of the baseline image. All detected baseline marker IDs and their corresponding vertex coordinates are saved to a calibration data file (DualCam.json) as reference benchmarks for subsequent screen calibration. Through this offline pre-calibration method, the reference positions of the markers in the baseline image only need to be extracted once during system deployment and can be reused. They can be directly called during subsequent online calibration, eliminating the need to re-detect the marker positions in the baseline image during each calibration, thus simplifying the on-site calibration process.

[0068] In the specific implementation of detection using the ArUco detector, the reference image is first converted to grayscale, and then the reference markers are detected using the detectMarkers method. The detection results include a vertex array of reference markers (corners), an ID array (ids), and a rejected candidate region array (rejected). For each detected reference marker, its four vertices are arranged in the order of top left, top right, bottom right, and bottom left. After detection, the original image size (width and height) of the reference image, along with the IDs of each reference marker and their corresponding vertex positions, are saved to the calibration data file. The original size of the reference image is used to subsequently calculate the four corner positions of the reference image: top left (0,0), top right (W,0), bottom right (W,H), and bottom left (0,H), where W and H are the width and height of the reference image, respectively. These four corner coordinates will serve as the source points for homography projection during the screen calibration stage.

[0069] Additionally, screen refresh may cause abnormal brightness in some areas of the baseline marker, and screen reflections may produce false edges. To improve detection robustness, the ArUco detector parameters can be specifically optimized: the adaptive threshold constant is set to 3 (default value), the adaptive threshold window size range is 5-31 (step size 10), the minimum Otsu standard deviation is set to 5.0 (higher than the default value to avoid false detections in low-contrast areas), the marker perimeter ratio range is 0.02-5.0 (a lenient range to accommodate markers at different distances), and the polygon approximation accuracy is set to 0.03. These parameters have been verified through extensive experiments and can achieve stable and reliable marker detection in screen shooting scenarios.

[0070] During screen calibration, the first camera capturing the screen image is affected by various noise sources: scan line effect caused by screen refresh, brightness fluctuations caused by LED backlight PWM dimming, and readout noise from the camera sensor itself. To effectively suppress these noises, this embodiment employs a frame buffer median filtering strategy:

[0071] The screen images are captured in real time by a first camera, and frame buffer noise reduction is performed on the screen images: a frame buffer with a preset number of frames (e.g., a length of 5) is maintained. The currently captured screen image is stored in the frame buffer, and the median of multiple frames in the frame buffer is taken as the current frame output. Through frame buffer median filtering, flicker noise caused by screen refresh and random noise from the camera sensor can be effectively suppressed. Median filtering can better preserve image edge information while removing impulse noise. Compared with mean filtering, it is more suitable for high-frequency noise interference commonly found in screen shooting scenarios, thus providing a higher quality input image for subsequent benchmark marker detection.

[0072] Each reference marker in the screen image is detected. A first homography matrix is ​​calculated based on the position of each reference marker in the screen image and its pre-stored reference position. The corner points of the reference image are then projected onto the first camera image coordinate system based on the first homography matrix to determine the projected positions of the corner points of the reference image. These projected positions are used as the corner positions of the screen. Specifically, the following steps are taken:

[0073] Detect each reference marker in the screen image and obtain the coordinates of the four vertices of each reference marker in the screen image;

[0074] For each reference marker (marker ID m_i), the corresponding reference position is read from the pre-stored calibration data. Based on the correspondence between the coordinates of the four vertices of the reference position and the coordinates of the four detected vertices, the first homography matrix H_i corresponding to the reference marker is calculated:

[0075] H_i = findHomography(P_orig_i, P_detected_i);

[0076] P_orig_i represents the vertex coordinates of the reference position, P_detected_i represents the vertex coordinates of the detected position, and findHomography(.) is the function for calculating the homography matrix.

[0077] Then, the four corner points of the reference image are projected onto the coordinate system of the first camera image using the first homography matrix to obtain the candidate projection positions of the reference image corner points corresponding to each reference marker:

[0078] C_projected_i = perspectiveTransform(P_orig_corners, H_i) .

[0079] P_orig_corners are the corner points of the reference image, and perspectiveTransform(.) is a function that performs perspective transformation on a set of corner points using a homography matrix.

[0080] Since the projection error increases with distance from the reference marker, a distance-based weighting mechanism is introduced: for each corner point in the reference image, the Euclidean distance from the center of each reference marker to that corner point is calculated. (See [link to relevant documentation]). Figure 2 In the diagram, the Euclidean distances from the baseline marker m1 to the four corner points are d11, d12, d13, and d14, respectively. Weights w_ij are assigned based on these Euclidean distances, with closer Euclidean distances resulting in larger weights.

[0081] w_ij = D / (d_ij + ε);

[0082] Where D is the diagonal length of the reference image, d_ij is the Euclidean distance from the center of the reference marker mi to the j-th corner point, j=1,2,3,4, and ε is a small constant to prevent division by zero;

[0083] When multiple reference markers are detected simultaneously, for each corner point j, the candidate projection position with the highest weight is selected as the final projection position of that corner point. This ensures that the reference marker closest to the corner point contributes the most, thereby improving the corner point localization accuracy. See also Figure 3 The bold blue line represents the Euclidean distance corresponding to the maximum weight, while the dashed lines represent other Euclidean distances.

[0084] This step, which smooths out corner position jitter and achieves automatic stability assessment, also includes inputting the final projected position of each corner point into a Kalman filter for filtering. The state vector of the Kalman filter contains the position and velocity of the corner point. The Kalman filter predicts the position of the corner point based on the current state vector, extracts the current predicted position of the corner point using the measurement matrix, compares the current predicted position with the input position of the corner point, corrects the predicted value based on the comparison result, and outputs the corrected corner point position.

[0085] Specifically, a Kalman filter is configured for each corner point. Each Kalman filter uses a 4-dimensional state vector [x, y, vx, vy]^T, where (x, y) represents the corner point's position, (vx, vy) represents its velocity, and the measurement matrix H is a 2×4 matrix. The introduction of the Kalman filter effectively smooths the corner point position fluctuations caused by marker detection jitter and image noise, providing a smooth corner point trajectory for subsequent stability evaluation.

[0086] The process noise covariance of the Kalman filter was set to 0.001×I_4 (I_4 is a 4th order identity matrix), the measurement noise covariance was set to 2.0×I_2, and the initial value of the error covariance was set to 0.1×I_4. The measurement noise was set to a relatively large value (2.0) because there is a certain degree of uncertainty in the marker detection in the screen calibration scenario. The larger measurement noise makes the filter rely more on its own state prediction rather than a single measurement value, thus obtaining a smoother output.

[0087] In this embodiment, the method further includes calculating the instability index instability_i for each corner point based on the state vector of the Kalman filter. The instability index is directly proportional to the speed and inversely proportional to the distance from the corner point to the center of the screen.

[0088] instability_i = ||v_i|| × D / 2 / (dist_i + ε) ;

[0089] Where ||v_i|| is the magnitude of the velocity of the i-th corner point, dist_i is the distance from the corner point to the center of the screen, and ε is a small constant to prevent division by zero;

[0090] Calculate the stability score (stability_score) by taking the largest instability index from each corner point:

[0091] stability_score = max(0, 100 - max_instability);

[0092] Where max_instability is the maximum value of the instability index, and max(.) is the maximum value;

[0093] When the stability score exceeds a preset threshold (e.g., 99 points) and remains above it for a preset time (e.g., 3 seconds), the calibration is automatically completed. The current Kalman filter output values ​​for the four corner points are saved, and the current corner positions are used as the screen's corner positions. If the stability score fails to reach the preset threshold within the preset time window, data acquisition and filtering updates continue until the stability condition is met, at which point the calibration is automatically completed. This automatic mechanism requires no manual intervention, ensuring the consistency of the calibration results.

[0094] S2 obtains the pre-calibrated coordinate mapping parameters between the first and second cameras, and uses these parameters to map the corner positions of the screen from the first camera image coordinate system to the second camera image coordinate system, thus obtaining the screen area boundary in the second camera image coordinate system. The screen area boundary is determined by the position of the screen corners in the second camera image coordinate system.

[0095] In this embodiment, the coordinate mapping parameter is a pre-calibrated second homography matrix, and coordinate transformation is performed using this second homography matrix. This second homography matrix can be obtained by using a calibration board with both visible and infrared responses. After the first and second cameras acquire images of the calibration board at double the size, a 3×3 homography matrix between the image planes of the two cameras is calculated through feature point matching and stored in a configuration file. This offline pre-calibration strategy avoids the complexity of online calibration, which requires simultaneous acquisition of images from two cameras. A 3×3 second homography matrix, H_color_to_ir, can be used, which describes the perspective transformation relationship between the first and second camera image planes. This matrix is ​​acquired through a dedicated stereo calibration process and stored in the DualCam.json configuration file before system deployment.

[0096] The specific coordinate transformation process is as follows:

[0097] Read the screen corner calibration results (screen_corners) and the pre-calibrated second homography matrix (homography_matrix) from the DualCam.json file.

[0098] First, the positions of the screen corners in the first camera image coordinate system are converted to homogeneous coordinates to avoid data format compatibility issues with OpenCV's perspectiveTransform function in certain situations. Then, the homogeneous coordinates are multiplied by the second homography matrix and normalized to obtain the positions of the screen corners in the second camera image coordinate system.

[0099] [x\', y\', w\']^T = H_color_to_ir × [x_c, y_c, 1]^T;

[0100] x_ir = x\' / w\', y_ir = y\' / w\';

[0101] Where [x', y', w']^T is the result of multiplying the homogeneous coordinates by the second homography matrix, [x_c, y_c, 1]^T is the homogeneous coordinates, H_color_to_ir is the second homography matrix, and (x_ir, y_ir) is the corresponding position of the corner point in the second camera coordinate system.

[0102] For each of the four corner points of the screen, perform the above transformation to obtain the screen corner points (infrared_corners) in the second camera coordinate system. Save the transformed four corner points (infrared_corners) to the configuration file for use by the aiming point detection module.

[0103] This embodiment also includes an infrared bright spot pre-detection function: after the corner point conversion of the screen is completed, a second camera can be temporarily activated to capture images (for example, for 2 seconds) to detect abnormal bright spots within the boundary of the screen area defined by the corner point. If a bright spot is detected, the user is prompted to adjust the position of the second camera or ignore the bright spot and continue, so as to avoid interference from ambient infrared light sources on subsequent aiming point detection.

[0104] S3 acquires the infrared image including the screen from the second camera in real time, detects the infrared aiming spot within the screen area boundary, calculates the position of the infrared aiming spot in the second camera image coordinate system, and performs filtering and forward prediction compensation processing on the detected infrared aiming spot position to output the current aiming point position. This step continues to run after the system enters aiming mode (e.g., processing each frame of the image acquired by the second camera at a frame rate of 60fps). This step specifically includes the following:

[0105] Converting a BGR format infrared image to a grayscale image results in the highest grayscale value for the infrared light spots, which appear as white or light-colored bright areas in the infrared image.

[0106] A polygonal Region of Interest (ROI) is constructed based on the screen area boundaries, and the ROI is used as a mask to limit the detection range, effectively eliminating interference from ambient light sources outside the screen. In practical applications, if screen corner point parameters are provided, a zero-value mask image of the same size as the input grayscale image is generated. Then, the fillPoly function is used to fill the polygonal area formed by the screen corner points with a preset grayscale value (e.g., 255) to mark that area as a valid detection region. Then, the grayscale image and the mask image are multiplied by bitwise_and, setting all pixels outside the screen area to zero. This step limits the detection range to the screen area, effectively eliminating interference from ambient infrared light sources (such as remote controls, warm light sources, etc.) outside the screen.

[0107] Gaussian blur (5×5 kernel) is applied to the grayscale image after limiting the detection range to suppress high-frequency noise. For example, a 5×5 Gaussian kernel is used to blur the grayscale image, and the σ value is automatically calculated. The purpose of Gaussian blur is to smooth high-frequency noise in the image while preserving the overall contour features of the light spot, avoiding noise points from generating false light spots in subsequent thresholding.

[0108] The blurred grayscale image is segmented using a brightness threshold (default threshold 160-200) to generate a binarized image. Pixel grayscale values ​​greater than the threshold are set to 255 (white), and those less than the threshold are set to 0 (black). The threshold selection needs to balance detection sensitivity and anti-interference ability: too low a threshold will introduce environmental noise, while too high a threshold may miss weak infrared spots.

[0109] Opening and closing operations are performed sequentially on the binarized image, with a 5×5 elliptical kernel as the structuring element. The opening operation, through erosion followed by dilation, eliminates isolated noise points discretely distributed in the binarized image; the closing operation, through dilation followed by erosion, fills small holes inside the light spot, making the light spot outline more complete.

[0110] Finally, candidate light spots are extracted through contour detection, filtered by area (default 5-1000 pixels), and the centroid of each selected contour is calculated to obtain the position of the candidate light spots in the second camera image coordinate system.

[0111] For example, the `findContours` function (in RETR_EXTERNAL mode, only detecting the outermost contour) is used to extract all contours in the binarized image. The bounding area is calculated for each contour, and only contours with areas within a preset range (default 5-1000 pixels) are retained. Contours with excessively small areas are usually noise residue, while those with excessively large areas may be infrared radiation regions of the screen itself. For each valid contour, the centroid coordinates are calculated using image moments. When m00 is not zero, the centroid coordinates are cx = m10 / m00, cy = m01 / m00; when m00 is zero (in the degradation case), the bounding box center is used as an approximation, where m00 is the zeroth moment, m10 is the first moment in the x-direction, and m01 is the first moment in the y-direction.

[0112] When there are multiple candidate spots, the candidate spot with the largest area or the highest average brightness is selected as the infrared aiming spot to deal with multi-source interference scenarios. The spot emitted by the infrared light gun is usually larger or brighter than the area of ​​ambient infrared interference, so the spot with the largest area or the highest brightness is most likely the true aiming point.

[0113] Image detection results can exhibit spatial jitter due to noise and algorithmic uncertainties, affecting aiming accuracy and user experience. Furthermore, there is a system delay from image acquisition to coordinate output; when the light gun moves rapidly, the output position significantly lags behind the actual aiming position. Therefore, this embodiment performs filtering and forward prediction compensation on the detected infrared aiming spot position to suppress jitter and compensate for delay, outputting the current aiming point position.

[0114] In this embodiment, the detected position of the infrared aiming spot is input into a Kalman filter for filtering. The state vector of the Kalman filter contains the position and velocity of the infrared aiming spot. For example, a 4-dimensional state vector [x, y, vx, vy]^T is used, where (x, y) are the pixel coordinates of the aiming point in the second camera coordinate system, and (vx, vy] are the velocities (pixels / second) of the aiming point in the x and y directions. The measurement vector is a 2-dimensional vector z = [x, y]^T, for example, a constant 2×4 matrix: H = [[1,0,0,0], [0,1,0,0]]. Low-latency tracking is achieved through the following three mechanisms:

[0115] First, adaptive time step. The state transition matrix F of the Kalman filter is dynamically adjusted according to the actual time interval dt between two adjacent frames, ensuring that the motion state can still be correctly modeled when the frame rate fluctuates.

[0116] F(dt) = [[1, 0, dt, 0], [0, 1, 0, dt], [0, 0, 1, 0], [0, 0, 0, 1]].

[0117] In real-time systems, the frame interval is not a fixed value and may fluctuate due to factors such as system load and USB bandwidth contention. If the state transition matrix uses a fixed time step, it will lead to deviations in speed estimation, thus affecting prediction accuracy. This embodiment records the timestamp of each frame, calculates the actual time interval dt, and dynamically constructs the state transition matrix F(dt) accordingly to ensure the accuracy of speed estimation. When the timestamp is unavailable or the frame interval is zero, the default value dt = 0.033 seconds (approximately 30 fps) is used as a backup.

[0118] Second, forward prediction. When no infrared aiming spot is detected (e.g., the light gun moves out of the screen range), the Kalman filter extrapolates the predicted position of the aiming point based on the current velocity state and a preset prediction time (prediction_time, default 50ms), effectively compensating for the position lag caused by system latency.

[0119] [x_pred, y_pred]^T = [x, y]^T + prediction_time × [vx, vy]^T;

[0120] Where (x_pred, y_pred) are the coordinates of the predicted position of the aiming point.

[0121] Choosing the prediction time requires a trade-off between latency compensation and prediction accuracy: too short a prediction time has limited compensation effect, while too long a prediction time may cause the predicted position to deviate from the actual trajectory. The default value of 50ms is based on end-to-end latency estimation of a typical interactive system (including 10-15ms for image acquisition, 5-10ms for algorithm processing, and 10-20ms for display output), which can effectively compensate for most system latency. Users can adjust this parameter through the configuration file.

[0122] Third, adjustable noise parameters. The process noise covariance Q and measurement noise covariance R of the Kalman filter are both diagonal matrices, with the default values ​​being Q = 0.001 × I_4 and R = 0.001 × I_2. The initial error covariance P = 0.1 × I_4, where I_2 is a 2 × 2 identity matrix and I_4 is a 4 × 4 identity matrix. Lower process noise makes the filter respond more smoothly to state changes, while lower measurement noise makes the filter more confident in the detected values. These two parameters can be adjusted according to the actual application scenario to achieve the optimal balance between jitter suppression and response speed.

[0123] S4 maps the current aiming point position to the standard output coordinate system based on the perspective correspondence between the screen area boundary in the second camera image coordinate system and the preset standard output coordinate system, generating the final aiming coordinate output.

[0124] Because there is usually tilt and perspective distortion between the second camera and the screen, the screen in the infrared image captured by the second camera is not a standard rectangle, but rather appears as an arbitrary quadrilateral. To eliminate the effects of perspective distortion, this embodiment uses perspective transformation to map the screen area in the second camera image to a standard 16:9 aspect ratio rectangular display area. Specifically:

[0125] First, the coordinates of the four corner points of the screen area boundary are calculated to determine the one-to-one correspondence between the four corner points and the four vertices of the standard output coordinate system. Utilizing the geometric properties of quadrilaterals, the sum of the coordinates (x+y) of the four corner points is calculated and sorted. The point with the smallest sum is the top-left corner, and the point with the largest sum is the bottom-right corner. The coordinate difference (xy) is then calculated, with the smallest difference assigned to the top-right corner and the largest difference to the bottom-left corner. This statistically based sorting method is computationally simple, robust, and applicable to screen quadrilaterals at any tilt angle.

[0126] Then, based on the four corner points that determine the correspondence, the rectangular dimensions of the standard output coordinate system are determined. Specifically, the distance from the top left corner to the top right corner is used as the width W of the target rectangle, and the height H is calculated according to a preset aspect ratio (e.g., 16:9), thereby determining the four vertices of the standard output coordinate system:

[0127] W = ||P_tl - P_tr||;

[0128] H = W × 9 / 16;

[0129] Where P_tl and P_tr are the coordinates of the top left and top right corners, respectively, and the four corners of the target rectangle are (0,0), (W-1,0), (W-1,H-1), and (0,H-1), forming a standard coordinate system starting from the origin. See [link to documentation]. Figure 4 .

[0130] Finally, the perspective transformation matrix is ​​calculated based on the sorted coordinates of the four corner points and the four vertices of the standard output coordinate system. This matrix is ​​then used to map the current aiming point position to the standard output coordinate system, generating the final aiming coordinate output. Specifically, the OpenCV `findHomography` function can be used to calculate the perspective transformation matrix (e.g., the third homography matrix), and then the `perspectiveTransform` function can be used to map the aiming point coordinates to the normalized screen coordinate system. The final output screen coordinates can be directly used in game logic to determine hit status or to drive cursor display.

[0131] In aiming mode, the complete coordinate mapping process is as follows: the pixel coordinates (cx, cy) of the aiming point in the second camera image are output. The perspective transformation module maps these coordinates to the normalized screen coordinate system (proj_x, proj_y) through H_display, and draws the aiming mark at the corresponding position in a 640×480 black background window. The final output (proj_x, proj_y) can then be used as the position coordinates of the light gun aiming point on the screen for use by the game logic or interaction system.

[0132] The system displays two windows simultaneously: the raw image window from the second camera (Light Gun Aim) and the projected display window (Display). In the raw image window, screen corners are marked with yellow polygons, detected aiming spots are marked with red dots, and filtered aiming points are marked with green dots. When the aiming spot is lost, the predicted position of the Kalman filter is marked with a blue dot.

[0133] The projection display window presents the standardized screen coordinate space against a black background. The system uses the `getPerspectiveTransform` function to calculate the perspective transformation matrix from the corner point of the infrared camera screen to the corner point of a 640×480 standard rectangle. Then, the detected aiming point coordinates are mapped to the standard coordinate system using this matrix, and a red dot is drawn at the corresponding position in the projection window to represent the aiming position. When the light spot is lost, the predicted position is displayed as a blue dot in the projection window. This projection display scheme allows users to intuitively see the mapped position of the light gun aiming point on the screen, facilitating the verification of calibration and aiming accuracy.

[0134] Based on this, this embodiment also proposes a rapid aiming and calibration system for an infrared light gun, used to implement the aforementioned rapid aiming and calibration method for an infrared light gun. The system includes a screen corner calibration module, a coordinate cross-domain mapping module, an aiming point detection and tracking module, and a coordinate mapping output module. The functions of each module are as follows:

[0135] The screen corner calibration module is used to acquire the screen image captured by the first camera. The screen image includes a displayed reference image, which contains multiple reference marks. The module detects each reference mark in the screen image, calculates the first homography matrix based on the position of each reference mark in the screen image and its pre-stored reference position, and projects the corner points of the reference image onto the first camera image coordinate system based on the first homography matrix, which are then used as the corner points of the screen.

[0136] The coordinate cross-domain mapping module is used to obtain the pre-calibrated coordinate mapping parameters between the first camera and the second camera, and uses these parameters to map the corner positions of the screen from the first camera image coordinate system to the second camera image coordinate system, thus obtaining the screen area boundary in the second camera image coordinate system. This module is used to implement step S2 described above.

[0137] The aiming point detection and tracking module is used to acquire infrared images including the screen from the second camera in real time, detect the infrared aiming spot within the screen area boundary, calculate the position of the infrared aiming spot in the second camera image coordinate system, and perform filtering and forward prediction compensation processing on the detected infrared aiming spot position to output the current aiming point position. This module is used to implement step S3 above.

[0138] The coordinate mapping output module is used to map the current aiming point position to the standard output coordinate system according to the perspective correspondence between the screen area boundary under the second camera image coordinate system and the preset standard output coordinate system, and generate the final aiming coordinate output. This module is used to implement the above step S4.

[0139] In this embodiment, see Figure 5The system also includes a first camera, a second camera, a screen, an infrared light gun, and a host computer. The first camera is a color camera (used for detecting ArUco markers during screen calibration), the second camera is an infrared camera (used for detecting infrared light spots during aiming), the screen displays the ArUco baseline map and game graphics, etc. The infrared light gun has a built-in infrared LED or infrared laser emitter. The first and second cameras are connected to the host computer via a USB interface. The camera index range is 700-720, enumerated and identified using the cv2_enumerate_cameras library. Both cameras are configured with 640×480 resolution, 60fps frame rate, and YUY2 color format to ensure real-time image acquisition and color reproduction accuracy.

[0140] The host system is configured with the four functional modules mentioned above. All calibration data is stored uniformly in the DualCam.json configuration file in JSON format, including key parameters such as ArUco marker positions, screen corner coordinates, and homography matrix. The system adopts a modular design, with data passed between modules via configuration files. This low coupling between modules facilitates independent upgrades and maintenance. Examples of system parameter configurations are shown in Table 1 below.

[0141]

[0142] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0143] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0144] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0145] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A rapid aiming and calibration method for an infrared light gun, characterized in that, Includes the following steps: S1 acquires a screen image captured by a first camera. The screen image includes a displayed reference image, which contains multiple reference markers. S1 detects each reference marker in the screen image, calculates a first homography matrix based on the position of each reference marker in the screen image and its pre-stored reference position, and projects the corner points of the reference image onto the first camera image coordinate system based on the first homography matrix, which are then used as the corner point positions of the screen. S2 obtains the pre-calibrated coordinate mapping parameters between the first camera and the second camera, and uses the coordinate mapping parameters to map the corner position of the screen from the first camera image coordinate system to the second camera image coordinate system to obtain the screen area boundary in the second camera image coordinate system. S3 acquires the infrared image of the screen captured by the second camera in real time, detects the infrared aiming spot within the boundary of the screen area, calculates the position of the infrared aiming spot in the image coordinate system of the second camera, performs filtering and forward prediction compensation processing on the detected position of the infrared aiming spot, and outputs the current aiming point position. S4 maps the current aiming point position to the standard output coordinate system based on the perspective correspondence between the screen area boundary in the second camera image coordinate system and the preset standard output coordinate system, and generates the final aiming coordinate output.

2. The rapid aiming and calibration method for an infrared light gun as described in claim 1, characterized in that, In step S1, before detecting each reference marker in the screen image, the method further includes: Frame buffer noise reduction processing is performed on the screen image captured by the first camera: a frame buffer with a preset number of frames is maintained, the current screen image is stored in the frame buffer, and the median value of multiple frames in the frame buffer is taken as the current frame output.

3. The rapid aiming and calibration method for an infrared light gun as described in claim 1, characterized in that, In step S1, the reference position pre-stored by the reference mark includes at least the coordinates of the four vertices of the reference mark in the reference image; Based on the coordinates of the four vertices of each reference mark in the reference image and the corresponding vertex coordinates detected in the screen image, the first homography matrix corresponding to each reference mark is calculated respectively, and the corner point of the reference image is projected to the first camera image coordinate system through the corresponding first homography matrix to obtain the candidate projection position of the corner point of the reference image corresponding to each reference mark. For each corner point of the reference image, calculate the Euclidean distance from the center of each reference marker to that corner point; assign a weight w_ij based on the Euclidean distance, with a larger weight for closer Euclidean distances. w_ij = D / (d_ij + ε); Where D is the diagonal length of the reference image, d_ij is the Euclidean distance from the center of the reference marker to the j-th corner point, and ε is a small constant to prevent division by zero; The candidate projection position with the highest weight is selected as the final projection position of the corner point.

4. The rapid aiming and calibration method for an infrared light gun as described in claim 3, characterized in that, Step S1 further includes inputting the final projected position of each corner point into a Kalman filter for filtering. The state vector of the Kalman filter contains the position and velocity of the corner point. The Kalman filter predicts the position of the corner point based on the current state vector, extracts the current predicted position of the corner point using a measurement matrix, compares the current predicted position with the input position of the corner point, and corrects the predicted value based on the comparison result. It also includes calculating the instability index `instability_i` for each corner point based on the state vector of the Kalman filter. This instability index is directly proportional to the speed and inversely proportional to the distance from the corner point to the center of the screen. instability_i = ||v_i|| × D / 2 / (dist_i + ε) ; Where ||v_i|| is the magnitude of the velocity of the i-th corner point, dist_i is the distance from the corner point to the center of the screen, and ε is a small constant to prevent division by zero; Calculate the stability score (stability_score) by taking the largest instability index from each corner point: stability_score = max(0, 100 - max_instability); Where max_instability is the maximum value of the instability index, and max(.) is the maximum value; When the stability score exceeds a preset threshold and remains so for a preset time, the corner positions of the current filtered output are determined as the corner positions of the screen.

5. The rapid aiming and calibration method for an infrared light gun as described in claim 1, characterized in that, The coordinate mapping parameter is the second homography matrix. In step S2, the positions of the screen corners in the first camera image coordinate system are first converted into homogeneous coordinates; then, the homogeneous coordinates are multiplied by the second homography matrix and normalized to obtain the positions of the screen corners in the second camera image coordinate system. [x\', y\', w\']^T = H_color_to_ir × [x_c, y_c, 1]^T; x_ir = x\' / w\', y_ir = y\' / w\'; Where [x', y', w']^T is the result of multiplying the homogeneous coordinates with the second homography matrix, [x_c, y_c, 1]^T is the homogeneous coordinates, H_color_to_ir is the second homography matrix, and (x_ir, y_ir) is the corresponding position of the corner point in the second camera coordinate system.

6. The rapid aiming and calibration method for an infrared light gun as described in claim 1, characterized in that, The screen area boundary is determined by the position of the corner point of the screen in the second camera image coordinate system; in step S3, detecting the infrared aiming spot within the screen area boundary specifically includes: The infrared image is converted into a grayscale image. A region of interest (ROI) is constructed using the screen area boundary, and the ROI is used as a mask to limit the detection range. The grayscale image after limiting the detection range is Gaussian blurred. The blurred grayscale image is then segmented by a brightness threshold to generate a binarized image. Opening and closing operations are sequentially performed on the binarized image. Candidate spots are extracted through contour detection, and contours are filtered by area. The centroid of each filtered contour is calculated to obtain the position of the candidate spot in the second camera image coordinate system. When there are multiple candidate spots, the candidate spot with the largest area or the highest average brightness is selected as the infrared aiming spot.

7. The rapid aiming and calibration method for an infrared light gun as described in claim 1, characterized in that, In step S3, the detected position of the infrared aiming spot is filtered and forward prediction compensation is performed to output the current aiming point position, specifically including: The detected position of the infrared aiming spot is input into a Kalman filter for filtering. The state vector of the Kalman filter contains the position and velocity of the infrared aiming spot. The state transition matrix of the Kalman filter is dynamically adjusted according to the time interval between adjacent frames. When the infrared aiming spot is not detected, the predicted position is extrapolated based on the speed in the current filtering state and the preset prediction time, and the predicted position is output as the current aiming point position.

8. The rapid aiming and calibration method for an infrared light gun as described in claim 1, characterized in that, In step S4, the coordinates of the four corner points of the screen area boundary are calculated to determine the correspondence between each corner point and the four vertices of the rectangle of the standard output coordinate system; then the size of the rectangle of the standard output coordinate system is determined according to the correspondence, and the four vertices of the standard output coordinate system are calculated according to the preset aspect ratio; the perspective transformation matrix is ​​calculated based on the four corner points and the four vertices of the standard output coordinate system. The current aiming point position is mapped to the standard output coordinate system using the perspective transformation matrix.

9. The rapid aiming and calibration method for an infrared light gun as described in claim 1, characterized in that, Obtain the reference image containing multiple reference markers; extract the corner positions, image size, and vertex coordinates of each reference marker in the reference image, and store them in a calibration data file; The identification information and vertex coordinates of the reference mark serve as the reference position of the reference mark.

10. A rapid aiming and calibration system for an infrared light gun, characterized in that, include: A screen corner calibration module is used to acquire a screen image captured by a first camera. The screen image includes a displayed reference image, and the reference image includes multiple reference marks. Each reference mark in the screen image is detected, and a first homography matrix is ​​calculated based on the position of each reference mark in the screen image and its pre-stored reference position. The corner points of the reference image are projected onto the first camera image coordinate system based on the first homography matrix and used as the corner point positions of the screen. The coordinate cross-domain mapping module is used to obtain the pre-calibrated coordinate mapping parameters between the first camera and the second camera, and use the coordinate mapping parameters to map the corner position of the screen from the first camera image coordinate system to the second camera image coordinate system to obtain the screen area boundary in the second camera image coordinate system. The aiming point detection and tracking module is used to acquire infrared images of the screen captured by the second camera in real time, detect infrared aiming spots within the boundary of the screen area, calculate the position of the infrared aiming spots in the image coordinate system of the second camera, and perform filtering and forward prediction compensation processing on the detected position of the infrared aiming spots to output the current aiming point position. The coordinate mapping output module is used to map the current aiming point position to the standard output coordinate system according to the perspective correspondence between the screen area boundary under the second camera image coordinate system and the preset standard output coordinate system, and generate the final aiming coordinate output.

Images