Camera aiming point fusion calculation method and device, equipment and storage medium

Abstract

This invention relates to the field of computer vision and image processing technology, and discloses a method, apparatus, device, and storage medium for calculating camera aiming points. The method improves the calculation accuracy and robustness of the camera's main optical axis aiming points by introducing a distance-based inverse weighted fusion mechanism, and can adaptively suppress errors caused by lens distortion. Specifically, the method dynamically adjusts the weight of each marker aiming point in the fusion result by calculating the distance from the geometric center coordinates of the marker to the center coordinates of the image, so that the marker aiming points closer to the main optical axis contribute more to the final result. Through weight adjustment, even in extreme cases where the camera is shooting at close range with a skewed viewing angle or where some markers obscure the image and cause severe distortion, it can still output smooth and near-true aiming point coordinates, effectively avoiding severe aiming point jitter caused by individual marker recognition jumps, and eliminating the need for complex lens intrinsic parameter calibration and distortion correction calculations, thus simplifying the system deployment process.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, and in particular to a method, apparatus, device and storage medium for calculating camera aiming point fusion. Background Technology

[0002] In scenarios requiring real-time acquisition of the intersection of the camera's main optical axis and the plane, such as multimedia interaction, projection calibration, augmented reality, and shooting simulation, existing technologies typically achieve positioning by setting up manual markers on the plane. Among these, ARTag and AprilTag are two widely used types of manual markers. Both encode unique ID information in black and white squares, which can be quickly detected and recognized by computer vision detection algorithms. They also have high-precision corner features and are suitable for mapping calculations between screen or planar coordinate systems and image coordinate systems.

[0003] The existing technical solution is generally as follows: display multiple ARTag or AprilTag markers with known locations on a plane, capture an image including the markers with a camera, calculate the homography matrix using the correspondence between the physical coordinates of the marker corner points on the plane and the image coordinates, and then map the coordinates of the image center point to the plane coordinate system to obtain the aiming point.

[0004] However, when a camera captures multiple markers simultaneously, the aiming point coordinates calculated independently for each marker often have deviations due to lens distortion, image sensor resolution limitations, and corner detection errors. Existing technologies lack effective multi-result fusion mechanisms, making it difficult to eliminate errors caused by factors such as large edge distortion and corner jitter, resulting in insufficient aiming point accuracy and poor stability. The deviation problem is even more prominent when the camera's viewing angle is skewed or some markers are severely distorted.

[0005] It is evident that existing technologies still need improvement and enhancement. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention aims to provide a camera aiming point fusion calculation method. By introducing a distance-based inverse weighted fusion mechanism, the calculation accuracy and robustness of the camera's main optical axis aiming point are improved, and errors caused by lens distortion can be adaptively suppressed.

[0007] The first aspect of this invention provides a method for calculating camera aiming points through fusion, comprising: acquiring a real-time camera image and detecting marker points; when multiple artificial markers are detected, determining the image center coordinates of the real-time camera image; for each artificial marker, acquiring the planar physical coordinates of its corner points and determining the sub-pixel coordinates of its corner points in the real-time camera image, and calculating a homography matrix based on the planar physical coordinates and the sub-pixel coordinates; substituting the image center coordinates into the homography matrix of each artificial marker to obtain multiple marker aiming point coordinates, wherein the marker aiming point coordinates are located in the planar physical coordinates corresponding to the image center coordinates; calculating the geometric center coordinates of each artificial marker in the real-time camera image, and determining the distance allocation weight of each marker aiming point coordinate based on the geometric center coordinates and the image center coordinates; and performing a weighted average calculation on the marker aiming point coordinates based on the distance allocation weights to obtain the final aiming point coordinates.

[0008] Optionally, in a first implementation of the first aspect of the present invention, the step of acquiring a real-time camera image and performing marker detection includes: acquiring a real-time camera image, identifying artificial markers with preset coding features and preset geometric shapes in the real-time camera image; when multiple artificial markers are detected, parsing the unique identifier carried by each artificial marker to confirm the target plane included in the real-time camera image, and distinguishing the marker regions corresponding to each artificial marker.

[0009] Optionally, in a second implementation of the first aspect of the present invention, the step of obtaining the planar physical coordinates of the corner points and determining the sub-pixel coordinates of the corner points in the real-time camera image for each artificial marker, and calculating the homography matrix based on the planar physical coordinates and the sub-pixel coordinates, includes: obtaining the planar physical coordinates of each artificial marker in a preset planar coordinate system, wherein the preset planar coordinate system is a two-dimensional coordinate system corresponding to the target plane including the artificial marker; performing image preprocessing on the marking area of ​​each artificial marker to obtain multiple preprocessed images; using a sub-pixel level corner detection algorithm to locate the sub-pixel level coordinates of the four corner points of each artificial marker based on each preprocessed image; and calculating the homography matrix for each artificial marker using a direct linear transformation algorithm based on the sub-pixel coordinates and planar physical coordinates of its four corner points.

[0010] Optionally, in a third implementation of the first aspect of the present invention, the step of calculating the geometric center coordinates of each of the artificial markers in the real-time camera image, and determining the distance allocation weight of the aiming point coordinates of each marker based on the geometric center coordinates and the image center coordinates, includes: calculating the arithmetic mean of the horizontal and vertical coordinates of the four corner points of each artificial marker to obtain the geometric center coordinates corresponding to each artificial marker; calculating the Euclidean distance between each of the geometric center coordinates and the image center coordinates; and determining the distance allocation weight of the aiming point coordinates of each marker based on the calculated Euclidean distance, wherein the distance allocation weight is inversely proportional to the Euclidean distance.

[0011] Optionally, in the fourth implementation of the first aspect of the present invention, the step of calculating the weighted average of the coordinates of each of the marker aiming points based on the distance allocation weights to obtain the final aiming point coordinates includes: normalizing each of the distance allocation weights so that the sum of all distance allocation weights is 1, obtaining the normalized weights corresponding to the coordinates of each of the marker aiming points; multiplying each of the marker aiming point coordinates by its corresponding normalized weight to obtain multiple product results; summing the multiple product results, and using the summed result as the final aiming point coordinates.

[0012] A second aspect of the present invention provides a camera aiming point fusion calculation device, comprising: a detection module, configured to acquire a real-time camera image and perform marker point detection, and when multiple artificial markers are detected, determine the image center coordinates of the real-time camera image; a matrix calculation module, configured to, for each artificial marker, acquire the planar physical coordinates of its corner points and determine the sub-pixel coordinates of its corner points in the real-time camera image, and calculate a homography matrix based on the planar physical coordinates and the sub-pixel coordinates; an aiming determination module, configured to substitute the image center coordinates into the homography matrix of each artificial marker to obtain multiple marker aiming point coordinates, wherein the marker aiming point coordinates are located in the planar physical coordinates corresponding to the image center coordinates; a weight determination module, configured to calculate the geometric center coordinates of each artificial marker in the real-time camera image, and determine the distance allocation weight of each marker aiming point coordinate based on the geometric center coordinates and the image center coordinates; and a coordinate calculation module, configured to perform a weighted average calculation on the marker aiming point coordinates based on the distance allocation weights to obtain the final aiming point coordinates.

[0013] A third aspect of the present invention provides a camera aiming point fusion computing device, the camera aiming point fusion computing device comprising: a memory and at least one processor, the memory storing instructions; at least one processor calling the instructions in the memory to cause the camera aiming point fusion computing device to execute each step of the camera aiming point fusion computing method described in any of the preceding claims.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the camera aiming point fusion calculation method described in any of the preceding claims.

[0015] The camera aiming point fusion calculation method disclosed in this invention improves the calculation accuracy and robustness of the camera's main optical axis aiming point by introducing a distance-based inverse weighted fusion mechanism, and can adaptively suppress errors caused by lens distortion. Specifically, the method dynamically adjusts the weight of each marker aiming point in the fusion result by calculating the distance from the geometric center coordinates of the marker to the center coordinates of the image, so that the marker aiming points closer to the main optical axis contribute more to the final result. Through weight adjustment, even in extreme cases where the camera is shooting at close range with a skewed viewing angle or where some markers obstruct the view and cause severe distortion, it can still output smooth and near-true aiming point coordinates, effectively avoiding severe aiming point jitter caused by individual marker recognition jumps, and eliminating the need for complex lens intrinsic parameter calibration and distortion correction calculations, thus simplifying the system deployment process. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the camera aiming point fusion calculation method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the camera aiming point fusion computing device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the camera aiming point fusion computing device provided in an embodiment of the present invention. Detailed Implementation

[0017] This invention provides a method, apparatus, device, and storage medium for calculating camera aiming point fusion. In this invention, the terms "first," "second," "third," "fourth," etc. (if present)," in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] This application discloses a method for fusion calculation of camera aiming points. For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1One embodiment of the camera aiming point fusion calculation method in this invention includes: 101. Acquire real-time camera images and perform marker point detection. When multiple manual markers are detected, determine the coordinates of the image center of the real-time camera image. In this embodiment, the camera driver is invoked to acquire continuous image frames, i.e., real-time camera images. When it is confirmed that there are two or more artificial markers that meet preset characteristics in the real-time camera image, the geometric center coordinates of the camera image plane, i.e., the image center coordinates, are calculated according to the resolution parameters of the current camera image. The image center coordinates are usually defined as half of the image width and height. By monitoring the number of artificial markers in the image in real time, it is ensured that the fusion calculation is started only when there is multi-source reference data, thus avoiding the risk of single-point failure. By determining the image center coordinates as the optical axis reference of the camera, the consistency and accuracy of subsequent mapping calculations are ensured.

[0019] 102. For each artificial marker, obtain the planar physical coordinates of its corner points and determine the sub-pixel coordinates of its corner points in the real-time camera image, and calculate the homography matrix based on the planar physical coordinates and sub-pixel coordinates; In this embodiment, for each identified artificial marker, its physical position in the real-time camera image is located, i.e., sub-pixel coordinates. Compared with traditional pixel-level detection, sub-pixel-level positioning significantly reduces the quantization error caused by discretization sampling, making the spatial position relationship calculated based on the artificial marker more accurate, improving the calculation accuracy of the homography matrix, and thus improving the calculation accuracy of the aiming point. At the same time, the position data of each pre-calibrated artificial marker in the coordinate system of the target plane itself (such as the screen pixel coordinate system or two-dimensional plane physical coordinates) is retrieved, i.e., the plane physical coordinates. Using the sub-pixel coordinates and plane physical coordinates corresponding to the four corner points of each artificial marker, the homography matrix corresponding to the artificial marker is calculated. Through the homography matrix, the projection plane (target plane) coordinates corresponding to the pixel coordinates of the image center point of the real-time camera image can be calculated.

[0020] 103. Substitute the image center coordinates into the homography matrix of each of the artificial markers to obtain multiple marker aiming point coordinates, wherein the marker aiming point coordinates are located in the plane physical coordinates corresponding to the image center coordinates; In this embodiment, during the camera's manufacturing or calibration process, its principal optical axis is designed to pass through the geometric center coordinates of the image sensor. Therefore, the image center coordinates correspond to the projection point of the camera's principal optical axis on the image plane, meaning the image center coordinates can represent the direction of the optical axis. The homography matrix is ​​a 3x3 homogeneous transformation matrix that describes the projection geometry between the image plane and the physical plane. During calculation, the image center coordinates are substituted into the homography matrix of each artificial marker for perspective transformation mapping to obtain the projection point of the image center point on the projection plane, i.e., the marker aiming point. The homography matrix corresponding to each artificial marker can independently calculate a marker aiming point corresponding to the image center coordinates. Multiple marker aiming points serve as multi-source observation data for the same physical point (image center projection point). Through the homography matrix, the pixel coordinates of the image center point of the real-time camera image can be mapped to the screen pixel coordinates of the target plane, thereby achieving precise positioning of the image center point on the projection plane. Using homography matrices from different artificial markers, multiple physical plane coordinate estimates are generated for the same image center point. Due to lens distortion, the homography matrices calculated from artificial markers located at the image edge differ from those calculated from those located at the image center, leading to deviations in the mapping results. By generating multiple estimates, multi-source observation data is provided for subsequent multi-view weighted fusion, thereby achieving adaptive suppression of distortion errors. 104. Calculate the geometric center coordinates of each of the artificial markers in the real-time camera image, and determine the distance allocation weight of the aiming point coordinates of each marker based on the geometric center coordinates and the image center coordinates; In this embodiment, geometric distance is transformed into mathematical weights for fusion calculation. Specifically, based on the optical characteristic that the greater the distance from the geometric center coordinates of the manually marked data to the center coordinates of the image, the greater the distortion and the higher the error, an inverse proportional function is defined so that the shorter the distance, the higher the weight, and the longer the distance, the lower the weight. This quantifies the credibility of each manually marked data. Without additional error calibration data, the influence of high-error data can be automatically identified and eliminated, realizing intelligent filtering of multi-source data.

[0021] 105. Based on the distance allocation weight, perform a weighted average calculation on the coordinates of each of the marked aiming points to obtain the final aiming point coordinates; In this embodiment, the coordinates of each target aiming point obtained in step 103 are weighted and summed according to the corresponding distance weights calculated in step 104. The final output coordinates are the final aiming point coordinates that integrate multiple target information and have been corrected for errors. The weighted average algorithm effectively smooths outlier fluctuations caused by individual target distortions. Compared with taking the arithmetic mean, the final aiming point coordinates calculated in this embodiment are more robust and can continuously output low-jitter, high-stability aiming point trajectories, significantly improving the system's anti-interference capability and practical value.

[0022] The camera aiming point fusion calculation method disclosed in this application improves the calculation accuracy and robustness of the camera's main optical axis aiming point by introducing a distance-based inverse weighted fusion mechanism, and can adaptively suppress errors caused by lens distortion. Specifically, the method dynamically adjusts the weight of each manual marker in the fusion result by calculating the distance from the geometric center coordinates of the marker to the center coordinates of the image, so that the aiming point of the marker closer to the main optical axis contributes more to the final result. Through weight adjustment, even in extreme cases where the camera is shooting at close range with a skewed viewing angle or where some markers obstruct the view and cause severe distortion, it can still output smooth and near-true final aiming point coordinates, effectively avoiding severe aiming point jitter caused by individual marker recognition jumps, and eliminating the need for complex lens intrinsic parameter calibration and distortion correction calculations, thus simplifying the system deployment process.

[0023] Furthermore, in this embodiment of the invention, acquiring real-time camera images and performing marker point detection includes: 201. Acquire real-time camera images and identify artificial markers with preset coding features and preset geometric shapes in the real-time camera images; In this embodiment, the real-time camera image originates from a camera sensor connected to the target device. The camera sensor acquires light signals within its field of view at a fixed frame rate and converts them into digital image signals. The preset geometric shape is a black and white square border, which is beneficial for edge detection and corner extraction. The preset encoding feature refers to the binary encoding pattern inside the square border, such as ARTag or AprilTag. By scanning the quadrilateral outline in the real-time camera image and reading its internal encoding information, it is confirmed that the pattern is a legitimate artificial marker rather than background noise. By combining geometric shape and encoding feature dual verification, the anti-interference capability of marker recognition is improved, effectively preventing similar rectangular objects in the environment from being misjudged as artificial markers, and ensuring the purity and security of subsequent calculation data.

[0024] 202. When multiple artificial markers are detected, the unique identifier carried by each artificial marker is parsed to confirm the target plane included in the real-time camera image and to distinguish the marker area corresponding to each artificial marker; In this embodiment, each artificial marker is assigned a unique numerical code during its creation. Scene understanding and data association are achieved through the built-in ID information of the artificial marker. Specifically, after reading the numerical code, the system queries a preset configuration file to determine whether the read numerical code belongs to the currently used target plane. The preset configuration file is a mapping table stored in the system memory, recording the correspondence between marker ID information and target planes, as well as the physical coordinates of each marker. By comparing the ID information with the configuration file, the system confirms which physical plane the current field of view includes, for example, confirming that it is a marker on a screen rather than a marker on a wall, thus realizing marker area differentiation and scene adaptation in a multi-marker environment. It can accurately locate the target working area in environments with multiple planes or complex backgrounds, avoiding coordinate mapping errors and enhancing the system's environmental adaptability. In addition, by comparing ID information with configuration files, it can also determine whether multiple markers are on the same target plane. For example, in indoor scenes, it may be necessary to project images onto multiple walls. In this case, the ID information of the manual markers can determine whether multiple manual markers in the real-time camera image are on the same projection plane. Only manual markers that are on the same target plane can be used for subsequent fusion calculations, avoiding coordinate mapping errors caused by cross-plane data mixing, and further enhancing the system's environmental adaptability and calculation accuracy.

[0025] Further, in this embodiment of the invention, for each artificial marker, obtaining the planar physical coordinates of its corner points and determining the sub-pixel coordinates of its corner points in the real-time camera image, and calculating the homography matrix based on the planar physical coordinates and the sub-pixel coordinates, includes: 301. Obtain the planar physical coordinates of each of the artificial markers in a preset planar coordinate system, wherein the preset planar coordinate system is a two-dimensional coordinate system corresponding to the target plane including the artificial markers; In this embodiment, during the system initialization phase, the operator measures and records the actual position of each artificial marker in the real world, so that the four corner points of each artificial marker have known planar physical coordinates in a preset planar coordinate system. Only when the actual physical size and position of the artificial marker are known can the perspective distortion in the image be restored to the real planar coordinates to achieve physical aiming and positioning. The preset planar coordinate system is established with a certain physical vertex of the target plane as the origin and the two adjacent sides of the plane as the coordinate axes, with the unit being millimeters or meters.

[0026] 302. Perform image preprocessing on the marked regions of each of the artificially marked areas to obtain multiple preprocessed images; In this embodiment, for each detected artificial marker, the circumscribed polygon of the region is first marked based on the image coordinates of its four corner points, i.e., the marked region. Then, in order to reserve sufficient pixel buffer for subsequent sub-pixel level corner point extraction, the cropping range is increased by several pixels before cropping and extraction, resulting in a sub-image containing only a single artificial marker with edge redundancy. Next, a Gaussian filtering algorithm is applied to remove high-frequency noise from each sub-image, and histogram equalization is performed to enhance image contrast and mitigate the effects of uneven illumination, thereby obtaining a higher-quality local image of the marker, i.e., the preprocessed image. Through preprocessing, some imaging noise and illumination interference are eliminated, highlighting the corner features of the artificial marker, providing a high-quality input image for subsequent sub-pixel level localization, and further ensuring the accuracy of corner point extraction.

[0027] 303. Based on each of the preprocessed images, a subpixel-level corner detection algorithm is used to locate the subpixel-level coordinates of the four manually marked corners; In this embodiment, based on the preprocessed image, the initial pixel coordinates of the corner points are first determined. Then, a neighborhood window of a certain size is selected with the initial pixel coordinates as the center. The size of the neighborhood window is determined according to the pixel size of the manually marked area in the image. When the side length of the manually marked area is greater than 50 pixels, a 7×7 window is selected, and when it is less than or equal to 50 pixels, a 5×5 window is selected. By analyzing the distribution gradient of gray values ​​within the neighborhood window, the corner point position is fine-tuned at the sub-pixel level using the least squares method or the centroid method to obtain floating-point coordinate values ​​accurate to decimal places. Through sub-pixel level fine-tuning, the corner point positioning is no longer limited to the integer constraints of the pixel grid, breaking through the limitations of hardware resolution and significantly improving the positioning accuracy.

[0028] 304. For each artificial marker, the homography matrix is ​​calculated using a direct linear transformation algorithm based on the sub-pixel coordinates and planar physical coordinates of its four corner points. In this embodiment, the direct linear transformation algorithm solves for the optimal solution of the projection mapping between two planes by constructing a system of linear equations. The homography matrix is ​​a 3x3 homogeneous matrix that serves as a mathematical tool for mapping the coordinates of any point in the image back to the physical plane. It includes various transformation information such as rotation, translation, and scaling, and can be used to describe the imaging relationship of planar objects under different viewpoints.

[0029] Further, in this embodiment of the invention, the step of calculating the geometric center coordinates of each of the artificial markers in the real-time camera image, and determining the distance allocation weight of each marker aiming point coordinate based on the geometric center coordinates and the image center coordinates, includes: 401. Calculate the arithmetic mean of the horizontal and vertical coordinates of the four corner points of each artificial mark to obtain the geometric center coordinates corresponding to each artificial mark; In this embodiment, the X coordinates of the four manually marked corner points are added together and divided by four to obtain the X coordinate of the geometric center point, and the Y coordinates of the four manually marked corner points are added together and divided by four to obtain the Y coordinate of the geometric center point. The geometric center coordinates represent the average position of the manually marked points in the real-time camera image.

[0030] 402. Calculate the Euclidean distance between each of the geometric center coordinates and the image center coordinates; In this embodiment, the Pythagorean theorem is used to calculate the straight-line distance between the geometric center coordinates and the image center coordinates. This straight-line distance reflects the distance between the manually marked imaging position and the principal optical axis of the camera, that is, it reflects the degree of radial distortion of the lens. The larger the distance, the more severe the distortion usually is. By calculating the Euclidean distance, an objective numerical basis is provided for weight allocation.

[0031] 403. Determine the distance allocation weight of each of the target aiming points based on the calculated Euclidean distance, wherein the distance allocation weight is inversely proportional to the Euclidean distance; In this embodiment, the distance allocation weight is set to the reciprocal of the Euclidean distance, i.e., distance allocation weight = 1 / Euclidean distance. For example, when the Euclidean distance is 100 pixels, the distance allocation weight corresponding to the manual marker is 0.01; when the Euclidean distance is 200 pixels, the distance allocation weight corresponding to the manual marker is 0.005. In other embodiments, the negative square relationship or exponential function relationship of the distance can also be used, such as distance allocation weight = negative square of the Euclidean distance, or distance allocation weight = negative Euclidean distance power of the natural constant e. By setting the distance allocation weight to be inversely proportional to the Euclidean distance, it is ensured that the marker closer to the center has a larger weight, which offsets the systematic error caused by edge distortion and achieves adaptive error suppression.

[0032] Further, in this embodiment of the invention, the step of calculating a weighted average of the coordinates of each of the marked aiming points based on the distance allocation weight to obtain the final aiming point coordinates includes: 501. Normalize each of the distance allocation weights so that the sum of all distance allocation weights is 1, and obtain the normalized weights corresponding to each of the marker aiming points; In this embodiment, the distance allocation weights are normalized based on the following formula: W_i_normalized = W_i / ΣW_i; Wherein, W_i_normalized represents the normalization weight of the i-th marker aiming point coordinate, W_i represents the distance allocation weight of the i-th marker aiming point coordinate, and ΣW_i represents the sum of the distance allocation weights of all marker aiming point coordinates, ensuring that the sum of all normalized weights is 1; through normalization, the influence of the absolute value of the weights is eliminated, ensuring that the scale of the subsequent weighted average result is consistent with the physical coordinate scale, and preventing errors in the order of magnitude of the calculation results.

[0033] 502. Multiply the coordinates of each of the marked aiming points by their corresponding normalized weights to obtain multiple product results; In this embodiment, the X coordinate of the first marker aiming point is multiplied by its normalized weight, and its Y coordinate is also multiplied by the same weight to obtain the weighted coordinate components, i.e., the product result. The same multiplication calculation is performed on the coordinates of each marker aiming point to obtain multiple product results. Through multiplication calculation, the contribution of reliable data is amplified and the contribution of unreliable data is reduced, thus realizing the selection of the best data.

[0034] 503. Summate the multiple product results and use the sum as the final aiming point coordinates; In this embodiment, all weighted coordinate components are added together on the X and Y axes respectively, and the resulting composite coordinates are the final aiming point coordinates. By summing the results, the scattered multi-source information is integrated into an optimal estimate, making full use of available information, suppressing noise while preserving the true signal characteristics, and outputting the most reasonable aiming point position.

[0035] The camera aiming point fusion calculation method disclosed in this invention will be further described in detail below with reference to specific implementation scenarios: In a specific application scenario of a projection shooting game system, the game screen is projected onto a screen via a projector; four ARTag markers are fixedly displayed in the four corners of the screen; the player holds a simulated gun with a camera, the main optical axis of which coincides with the firing direction of the simulated bullet; the system needs to calculate in real time the position of the screen that the gun's camera is aiming at in order to determine whether the target in the game has been hit.

[0036] During the initialization phase, the operator measures and records the physical coordinates of the four corner points of the four ARTag tags (IDs are Tag1-Tag4) on the screen in the screen coordinate system. These physical coordinates are stored in the system configuration file with the upper left corner of the screen as the origin, establishing a mapping relationship between tag IDs and physical coordinates. During real-time operation, the system executes the following steps at a preset fixed frequency: First, the camera captures the real-time image of the current frame and determines the coordinates of the image center. The system detects the ARTag tags in the real-time image. Assuming that the current frame successfully identifies two tags numbered Tag2 and Tag4, it queries the preset configuration file to confirm that both tags belong to the current screen plane. Subsequently, the subpixel coordinates of the four corner points of Tag2 and Tag4 are extracted. For Tag2, using the subpixel coordinates of one corner point and the corresponding physical coordinates of the screen, the homography matrix H2 is calculated using a direct linear transformation algorithm. The image center coordinates are then substituted into H2 for perspective transformation to obtain the estimated projection value of the image center on the screen based on the Tag2 homography matrix, i.e., the tag aiming coordinates P2(x2, y2). Similarly, the homography matrix H4 is calculated for Tag4, and the estimated projection value of the image center on the screen P4(x4, y4) is calculated based on the Tag4 homography matrix. Due to lens distortion, there is a deviation between P2 and P4. Next, the geometric center coordinates of the two tags are calculated; the geometric center coordinates M2 are obtained by averaging the image coordinates of the four corner points of Tag2, and the Euclidean distance from it to the image center coordinates is calculated to obtain D2; similarly, the geometric center coordinates M4 are obtained by averaging the image coordinates of the four corner points of Tag4, and the Euclidean distance from it to the image center coordinates is calculated to obtain D4; then, the weights are assigned according to the inverse proportional relationship, with the weight W2 of Tag2 being 1 / D2 and the weight W4 of Tag4 being 1 / D4. Finally, normalization is performed, with a total weight of W2 + W4. The normalized weight W_2_norm for P2 is W2 / (W2 + W4), and the normalized weight W_4_norm for P4 is W4 / (W2 + W4). Then, the final aiming point coordinates are calculated using the weighted fusion formula: P_final = W_2_norm × P2 + W_4_norm × P4. Since Tag4 is closer to the image center coordinates, its estimated value dominates the final result, effectively suppressing the edge distortion error that may exist in Tag2, and achieving high-precision, low-jitter real-time aiming point tracking.

[0037] The above describes the camera aiming point fusion calculation method in the embodiments of the present invention. The following describes the camera aiming point fusion calculation device in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the camera aiming point fusion computing device in this invention includes: The detection module 601 is used to acquire real-time camera images and perform marker point detection. When multiple manual markers are detected, the coordinates of the image center of the real-time camera image are determined. The matrix calculation module 602 is used to obtain the planar physical coordinates of the corner points of each artificial marker and determine the sub-pixel coordinates of the corner points in the real-time camera image, and calculate the homography matrix based on the planar physical coordinates and the sub-pixel coordinates. The aiming determination module 603 is used to substitute the image center coordinates into the homography matrix of each of the artificial marks to obtain multiple mark aiming point coordinates, wherein the mark aiming point coordinates are located in the plane physical coordinates corresponding to the image center coordinates; The weight determination module 604 is used to calculate the geometric center coordinates of each of the artificial markers in the real-time camera image, and to determine the distance allocation weight of the aiming point coordinates of each marker based on the geometric center coordinates and the image center coordinates. The coordinate calculation module 605 is used to perform a weighted average calculation of the coordinates of each of the marked aiming points based on the distance allocation weights to obtain the final aiming point coordinates.

[0038] Based on the same ideas as the methods in the above embodiments, the apparatus provided in this application can implement the methods in the above embodiments.

[0039] above Figure 2 The camera aiming point fusion computing device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The camera aiming point fusion computing device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0040] Figure 3 This is a schematic diagram of the structure of a camera aiming point fusion computing device 700 provided in an embodiment of the present invention. The camera aiming point fusion computing device 700 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 710 and memory 720, and one or more storage media 730 (e.g., one or more mass storage devices) for storing application programs 733 or data 732. The memory 720 and storage media 730 can be temporary or persistent storage. The program stored in the storage media 730 may include one or more modules (not shown in the diagram), each module including a series of instruction operations performed on the camera aiming point fusion computing device 700. Furthermore, the processor 710 may be configured to communicate with the storage media 730 and execute a series of instruction operations in the storage media 730 on the camera aiming point fusion computing device 700 to implement the steps of the camera aiming point fusion computing method provided in the above-described method embodiments.

[0041] The camera aiming point fusion computing device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input / output interfaces 760, and / or one or more operating systems 731, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated camera aiming point fusion computing device structure does not constitute a limitation on the camera aiming point fusion computing device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0042] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the camera aiming point fusion calculation method.

[0043] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0044] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0045] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for fusion calculation of camera aiming points, characterized in that, include: Acquire real-time camera images and perform marker point detection. When multiple manual markers are detected, determine the image center coordinates of the real-time camera image. For each artificial marker, obtain the planar physical coordinates of its corner points and determine the sub-pixel coordinates of its corner points in the real-time camera image. Calculate the homography matrix based on the planar physical coordinates and the sub-pixel coordinates. Substituting the image center coordinates into the homography matrix of each of the artificial markers, multiple marker aiming point coordinates are obtained, and the marker aiming point coordinates are located in the plane physical coordinates corresponding to the image center coordinates; The geometric center coordinates of each of the artificial markers in the real-time camera image are calculated, and the distance allocation weight of each marker aiming point coordinate is determined based on the geometric center coordinates and the image center coordinates. Specifically, the arithmetic mean of the horizontal and vertical coordinates of the four corner points of each artificial marker is calculated to obtain the geometric center coordinates corresponding to each artificial marker; the Euclidean distance between each geometric center coordinate and the image center coordinate is calculated; and the distance allocation weight of each marker aiming point coordinate is determined according to the calculated Euclidean distance, wherein the distance allocation weight is inversely proportional to the Euclidean distance. Based on the distance allocation weights, a weighted average is calculated for the coordinates of each of the marked aiming points to obtain the final aiming point coordinates.

2. The camera aiming point fusion calculation method according to claim 1, characterized in that, The process of acquiring real-time camera images and performing marker point detection includes: Acquire real-time camera images and identify artificial markers with preset coded features and preset geometric shapes in the real-time camera images; When multiple artificial markers are detected, the unique identifier carried by each artificial marker is parsed to confirm the target plane included in the real-time camera image and to distinguish the marker area corresponding to each artificial marker.

3. The camera aiming point fusion calculation method according to claim 2, characterized in that, For each artificial marker, the planar physical coordinates of its corner points are obtained and the sub-pixel coordinates of its corner points in the real-time camera image are determined. Based on the planar physical coordinates and sub-pixel coordinates, a homography matrix is ​​calculated, including: Obtain the planar physical coordinates of each of the artificial markers in a preset planar coordinate system, wherein the preset planar coordinate system is a two-dimensional coordinate system corresponding to the target plane of the artificial markers; Each of the manually marked regions is preprocessed to obtain multiple preprocessed images; Based on each of the preprocessed images, a subpixel-level corner detection algorithm is used to locate the subpixel-level coordinates of the four manually marked corners. For each artificial marker, the homography matrix is ​​calculated using a direct linear transformation algorithm based on the sub-pixel coordinates and planar physical coordinates of its four corner points.

4. The camera aiming point fusion calculation method according to claim 1, characterized in that, The step of calculating the weighted average of the coordinates of each of the marked aiming points based on the distance allocation weights to obtain the final aiming point coordinates includes: The distance allocation weights are normalized so that the sum of all distance allocation weights is 1, thus obtaining the normalized weights corresponding to the coordinates of each of the target aiming points. The coordinates of each of the marked aiming points are multiplied by their corresponding normalized weights to obtain multiple product results; The multiple product results are summed, and the sum is used as the coordinates of the final aiming point.

5. A camera aiming point fusion computing device, characterized in that, include: The detection module is used to acquire real-time camera images and detect marker points. When multiple manual markers are detected, the coordinates of the image center of the real-time camera image are determined. The matrix calculation module is used to obtain the planar physical coordinates of the corner points of each artificial marker and determine the sub-pixel coordinates of the corner points in the real-time camera image, and calculate the homography matrix based on the planar physical coordinates and sub-pixel coordinates. The aiming determination module is used to substitute the image center coordinates into the homography matrix of each of the artificial markers to obtain multiple marker aiming point coordinates, wherein the marker aiming point coordinates are located in the plane physical coordinates corresponding to the image center coordinates; The weight determination module is used to calculate the geometric center coordinates of each of the artificial markers in the real-time camera image, and to determine the distance allocation weight of each marker aiming point coordinate based on the geometric center coordinates and the image center coordinates. Specifically, it calculates the arithmetic mean of the horizontal and vertical coordinates of the four corner points of each artificial marker to obtain the geometric center coordinates corresponding to each artificial marker; it calculates the Euclidean distance between each geometric center coordinate and the image center coordinate; and it determines the distance allocation weight of each marker aiming point coordinate based on the calculated Euclidean distance, wherein the distance allocation weight is inversely proportional to the Euclidean distance. The coordinate calculation module is used to perform a weighted average calculation of the coordinates of each of the target aiming points based on the distance allocation weights, so as to obtain the final aiming point coordinates.

6. A camera aiming point fusion computing device, characterized in that, The camera aiming point fusion computing device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the camera aiming point fusion computing device to perform the steps of the camera aiming point fusion computing method as described in any one of claims 1-4.

7. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the camera aiming point fusion calculation method as described in any one of claims 1-4.

Images