A micro-sensing trapezoidal method, device, equipment and medium based on Aruco recognition

Abstract

The present invention discloses a micro-sensing trapezoid method, device, equipment and medium based on Aruco recognition, which relates to the field of computer vision. The method comprises: S1: projecting the Aruco grid and collecting images, extracting the marked corner points to construct a position set; S2: calculating the trapezoid tension factor based on the trapezoid area and the distance between opposite sides, and quantifying the local deformation; S3: fusing the spatial parallax and tension information of multiple markers to construct a continuous tension balance domain, and generating the final mapping position by gradient-correcting the basic inverse projection mapping and combining boundary perturbation compensation; S4: outputting the inverse perspective matrix based on the fusion result, and estimating the microscopic posture offset parameters for projector fine-tuning. By modeling the tension factor and spatial parallax, adaptive correction of the distorted area is achieved, the image edge consistency and overall symmetry are improved, and the stability and accuracy of the projected image are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a micro-sensing trapezoidal method, device, equipment and medium based on Aruco recognition. Background Art

[0002] Projected image distortion and deformation are common problems in projector applications, significantly affecting visual presentation and the accuracy of information conveyed. To address this issue, the market has developed a variety of keystone correction technologies.

[0003] Among them, camera-based automatic keystone correction technology represents the current mainstream direction. This technology captures the projected image through a built-in or external camera, uses image processing algorithms to identify the geometric distortion features of the image edges, and then calculates the key parameters that characterize the keystone distortion. The system then applies digital image geometric transformation or drives the optical lens to perform physical displacement to dynamically compensate for the detected deformation. Compared with traditional manual adjustment, this solution, which combines computer vision and intelligent algorithms, demonstrates higher correction accuracy and efficiency when dealing with complex scenarios such as side projection angles and non-standard projection surfaces. It can also adapt to the need for recalibration after the projector position changes. Therefore, it has become a standard configuration for mid-to-high-end commercial and home projection equipment.

[0004] However, the existing automatic keystone correction technology system still has significant limitations, which restricts further improvement of user experience:

[0005] Reliance on dedicated calibration patterns: Most systems must interrupt normal projected content during the automatic calibration process to project a specially designed, structured reference pattern. These patterns typically take the form of a grid, dot matrix, or stripes. This forced pattern projection process interrupts user experience and is visually intrusive. Calibration cannot be completed without the user's awareness or while content continues to play.

[0006] Requires a screen reset: To ensure the algorithm can clearly and accurately extract distortion information from the captured image, the system typically requires the projected image to be temporarily adjusted to a preset, standardized initial state before performing the critical analysis step. The most common approach is to center the reference pattern and display it full screen. This "reset" step not only complicates and time-consuming the calibration process, but more importantly, requires the projector to temporarily stop displaying the content the user is actually viewing, disrupting the user's experience and immersion.

[0007] Limited real-time performance and adaptability: These two deficiencies combined make it difficult for existing technologies to achieve truly "unnoticeable" real-time correction. Relying on specialized patterns and reset operations means the system cannot silently and continuously monitor and compensate for minor posture deviations that may occur while the user is viewing dynamic content. Correction actions are often discrete, user-perceived, and require active triggering or specific conditions to execute.

[0008] These shortcomings indicate that, despite significant progress in camera-based automatic keystone correction technology, there is still room for improvement in terms of a smooth and seamless user experience and the need for continuous fine-tuning. Summary of the Invention

[0009] In view of the shortcomings of the prior art described above, the purpose of the present invention is to provide a micro-sensing trapezoidal method, device, equipment and medium based on Aruco recognition to solve the above technical problems.

[0010] To achieve the above object, the present invention provides the following technical solution: a micro-sensing trapezoidal method based on Aruco recognition, comprising:

[0011] S1: Pre-project the Aruco grid structure through the projector, use the monocular camera to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct the Aruco marker position set;

[0012] S2: Calculation of the trapezoidal tension factor for each Aruco marker based on the area of ​​the Aruco-marked trapezoid;

[0013] S3: Based on the spatial angle deviation and normal offset of multiple Aruco markers, a parallax interference factor is constructed, the trapezoidal deformation strength of each marker is fused, and a continuous tension equalization domain is generated through Gaussian spatial attenuation weights. The basic back projection mapping is corrected based on the tension gradient, and the boundary normal distance disturbance compensation is combined to obtain the final mapping position;

[0014] S4: Based on the fusion results of multiple Aruco markers, the inverse perspective transformation matrix for projected image correction is output, and the micro-pose offset parameters of the projected image are further estimated. The projector is fine-tuned according to the micro-pose offset parameters.

[0015] The present invention is further configured such that S1 includes:

[0016] The projector projects a standard square Aruco marker grid as a marker layer combined with the projection image onto the target area;

[0017] The monocular camera captures the projection area and captures the deformed Aruco markers caused by the shape of the projection surface or the camera angle;

[0018] Detect the coordinates of the four corner points of each Aruco marker and construct the Aruco marker position set;

[0019] Edge detection is performed on the projection area image, and Hough transform is applied to extract valid edge segments. All the extracted straight line segments are combined into an image boundary segment set.

[0020] The present invention is further configured such that S2 includes:

[0021] Based on the coordinates of the four corner points marked by Aruco, the lengths of the upper and lower bases of the trapezoid and the height of the vertical projection of the center line are determined, and the area of ​​the trapezoid is obtained by multiplying the height by the average of the upper and lower bases.

[0022] Measure the Euclidean distances of two sets of opposite sides respectively and calculate their product. Calculate the ratio of the product of the opposite sides to twice the area to obtain the trapezoidal tension factor.

[0023] The present invention is further configured such that S3 includes:

[0024] Based on three sets of consecutive Aruco-marked image corner points, the spatial angle difference of the plane formed by the three points is calculated, and the normalized parallax interference factor is constructed by combining the angle between the projection direction and the normal vector of the marked plane.

[0025] Taking the center of each Aruco marker as the reference source point, a Gaussian spatial attenuation function is used as the pixel-level weight function, and combined with the inverse weighted form of the parallax interference factor for weighted integration to construct a continuous tension balance domain.

[0026] The present invention is further configured to calculate a homography matrix between the known plane positions of the Aruco markers in the world coordinate system and their pixel coordinates in the image, and then invert the homography matrix to obtain the basic back-projection position;

[0027] Based on the back-projection position, the gradient response of the local tension equalization domain is introduced as the offset driving factor for each pixel position. The mapping correction strength is controlled by setting the tension response factor parameter to obtain the corrected projection position.

[0028] The present invention is further configured to calculate the normal projection distance of each pixel point to all boundary lines based on the image boundary line segment set, and take the maximum value as the boundary disturbance index;

[0029] Combined with the tension equilibrium domain, the boundary disturbance index is normalized and weighted as a boundary disturbance compensation term.

[0030] On the basis of correcting the projection position, the boundary disturbance compensation term is superimposed to realize the projection position correction and obtain the final mapping position.

[0031] The present invention is further configured to include Aruco markers that simultaneously satisfy the parallax interference factor being less than the parallax threshold and the trapezoidal tension factor being less than the tension threshold among all current Aruco markers into the trusted marker set;

[0032] For each Aruco marker in the trusted marker set, the difference between its observed center coordinates in the image and the theoretical undistorted center position is calculated based on the corresponding camera rotation matrix to obtain the projection offset of the marker;

[0033] The 3D offsets of all reliable Aruco markers are averaged to obtain the overall microscopic pose offset parameters.

[0034] The present invention also provides a micro-sensing trapezoidal device based on Aruco recognition, the system comprising:

[0035] Image acquisition module: Pre-project the Aruco grid structure through the projector, use the monocular camera to capture the image of the projection area, detect the corner points of each Aruco marker in the image, and construct the Aruco marker position set;

[0036] Distortion perception module: Calculates the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of ​​the Aruco marker;

[0037] Keystone Correction Module: This module constructs a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuses the trapezoidal deformation strength of each marker, generates a continuous tension equalization domain through Gaussian spatial attenuation weights, corrects the basic inverse projection mapping based on the tension gradient, and combines boundary normal distance disturbance compensation to obtain the final mapping position.

[0038] Pose output module: Based on the fusion results of multiple Aruco markers, it outputs the inverse perspective transformation matrix for projected image correction, further estimates the micro-pose offset parameters of the projected image, and fine-tunes the projector according to the micro-pose offset parameters.

[0039] The present invention further provides an electronic device, comprising:

[0040] one or more processors;

[0041] A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement a micro-sensing trapezoidal method based on Aruco recognition as described in any one of the above.

[0042] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is caused to execute a micro-sensing trapezoidal method based on Aruco recognition as described in any one of the above.

[0043] The present invention provides a micro-sensing trapezoid method, device, equipment and medium based on Aruco recognition. The method comprises the following steps: S1: pre-projecting an Aruco grid structure through a projector, using a monocular camera to collect an image of the projection area, detecting the corner points of each Aruco marker in the image, and constructing an Aruco marker position set; S2: calculating the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of ​​the Aruco marker; S3: constructing a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fusing the trapezoidal deformation strength of each marker, generating a continuous tension equalization domain through Gaussian spatial attenuation weights, correcting the basic inverse projection mapping based on the tension gradient, and combining boundary normal distance disturbance compensation to obtain the final mapping position; S4: outputting an inverse perspective transformation matrix for projected image correction based on the fusion results of multiple Aruco markers, further estimating the microscopic posture offset parameters of the projected picture, and fine-tuning the projector according to the microscopic posture offset parameters. The beneficial effects produced include:

[0044] A trapezoidal tension factor is introduced to accurately quantify the degree of local deformation: Using the trapezoidal geometric structure formed by Aruco-marked corner points, the area is calculated based on its upper and lower bases and vertical height, and the distance relationship between opposite sides is combined to construct a "trapezoidal tension factor" to quantify the intensity of local deformation, solving the problem that existing solutions cannot accurately depict the degree of local distortion of the projected image.

[0045] The multi-dimensional spatial parallax and tension are integrated to construct a continuous tension equalization domain to enhance the dynamic adaptability of local areas: the parallax interference factor is constructed through spatial angle difference and normal projection offset, and the Gaussian weight decay function and inverse interference weighting mechanism are introduced to achieve continuous fusion of tension fields between multiple Aruco markers, effectively avoiding the global correction error caused by a single local reference, thereby forming a "continuous tension equalization domain" with spatial adaptability.

[0046] Boundary disturbance terms are considered for compensation to improve edge imaging consistency and overall image symmetry: Hough transform is used to extract edge segments, and a boundary disturbance index is constructed based on the boundary normal distance. This index is then normalized and superimposed on the projection correction position to effectively compensate for blur or drift caused by uneven image edge projection, thereby improving edge image quality and structural symmetry.

[0047] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. In the drawings:

[0049] Figure 1 A flowchart of a micro-sensing trapezoidal method based on Aruco recognition is shown as an exemplary embodiment of the present invention;

[0050] Figure 2 The figure is a schematic structural diagram of a micro-sensing trapezoidal device based on Aruco recognition, which is an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following describes the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art will readily appreciate the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the various details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are intended only to illustrate the present invention and are not intended to limit the scope of protection of the present invention.

[0052] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0053] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0054] Example 1

[0055] A micro-sensing trapezoidal method based on Aruco recognition, such as Figure 1 As shown, including:

[0056] S1: Pre-project the Aruco grid structure through the projector, use the monocular camera to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct the Aruco marker position set;

[0057] S2: Calculation of the trapezoidal tension factor for each Aruco marker based on the area of ​​the Aruco-marked trapezoid;

[0058] S3: Based on the spatial angle deviation and normal offset of multiple Aruco markers, a parallax interference factor is constructed, the trapezoidal deformation strength of each marker is fused, and a continuous tension equalization domain is generated through Gaussian spatial attenuation weights. The basic back projection mapping is corrected based on the tension gradient, and the boundary normal distance disturbance compensation is combined to obtain the final mapping position;

[0059] S4: Based on the fusion results of multiple Aruco markers, the inverse perspective transformation matrix for projected image correction is output, and the micro-pose offset parameters of the projected image are further estimated. The projector is fine-tuned according to the micro-pose offset parameters.

[0060] The present invention is further configured such that S1 includes:

[0061] The projector projects a standard square Aruco marker grid as a marker layer combined with the projection image onto the target area;

[0062] The monocular camera captures the projection area and captures the deformed Aruco markers caused by the shape of the projection surface or the camera angle;

[0063] Detect the coordinates of the four corner points of each Aruco marker and construct the Aruco marker position set;

[0064] Perform edge detection on the projection area image, apply Hough transform to extract valid edge segments, and combine all extracted straight line segments into a set of image boundary segments. Specifically, the marker layer does not need to reset the screen when performing trapezoidal correction, nor does it need to fully block the projection screen. It only needs to add a separate marker layer to the projection screen during correction, and display the Aruco code in the four corner areas of the separate marker layer for recognition. It has higher anti-interference ability than the traditional checkerboard, and the user can continue to watch the picture, ensuring that the user experience is greatly improved on the basis of the trapezoidal algorithm. The monocular camera is deployed in front of the projector, at the same level as the projection lens; the Android API is used to capture the Aruco code image through the monocular camera; the method provided by opencv is used to find the pixel coordinates of the corner points of the Aruco code, and construct the Aruco marker position set. , ,in, For the The two-dimensional coordinates of the four corner points marked by Aruco in the image space are collected in clockwise order, and are respectively 、 、 、 ; Integrate all collected Aruco marker positions to obtain the Aruco marker position set . Image boundary segment set The acquisition is achieved through the following steps: based on the distribution of Aruco markers pre-projected by the projector, the peripheral corner points of all markers are detected, the convex hull or the minimum circumscribed polygon is calculated, and the polygon edges are extracted as the initial boundary segment set; the distorted image captured by the camera is edge detected using the Canny operator, and then the significant straight line segments are extracted through the Hough transform, and the line segments that match the geometric constraints of the projection area are screened. , update the boundary set to obtain the final image boundary segment set This method is a prior art and will not be described in detail here.

[0065] The present invention is further configured such that S2 includes:

[0066] Based on the coordinates of the four corner points marked by Aruco, the lengths of the upper and lower bases of the trapezoid and the height of the vertical projection of the center line are determined, and the area of ​​the trapezoid is obtained by multiplying the height by the average of the upper and lower bases.

[0067] The Euclidean distances of the two sets of opposite sides are measured separately and their product is calculated. The ratio of the product of the opposite sides to twice the area is calculated to obtain the trapezoidal tension factor. Specifically, based on the trapezoidal area marked by Aruco, the lines connecting the adjacent corner points constitute the two bases and sides of the trapezoid; the projection length of the line connecting the centers of the marks in the vertical direction is used as the height of the trapezoid, and the actual Euclidean distance of the upper and lower bases is extracted as the basic geometric parameter; the area of ​​the trapezoidal area is calculated by taking half the product of the height of the trapezoid and the sum of the lengths of the upper and lower bases; the ratio of the product of the two sets of opposite side lengths to twice the area of ​​the trapezoid is defined as the trapezoidal tension factor ; Trapezoidal tension factor Used to measure whether the mark has trapezoidal deformation, ideally the trapezoidal tension factor It should be close to 1. The smaller the value, the more significant the mark is deformed by perspective compression or stretching, and the more serious the distortion. The specific calculation formula for the trapezoid area is: ,in, is the area of ​​the trapezoid, The projection height is obtained by projecting the line connecting the midpoints of both sides to the vertical direction. is the length of the upper side, calculated by get, is the length of the lower side, calculated by The calculation formula of the trapezoidal tension factor is: ,in, is the trapezoidal tension factor, is the Euclidean distance.

[0068] The present invention is further configured such that S3 includes:

[0069] Based on three sets of consecutive Aruco-marked image corner points, the spatial angle difference of the plane formed by the three points is calculated, and the normalized parallax interference factor is constructed by combining the angle between the projection direction and the normal vector of the marked plane.

[0070] Using the center of each Aruco marker as the reference source, a Gaussian spatial attenuation function is used as the pixel-level weight function, and weighted integration is performed in combination with the inverse weighted form of the parallax interference factor to construct a continuous tension balance domain. Specifically, the parallax interference factor is the final credibility that combines the angular structural consistency and the perspective interference effect. The larger the parallax interference factor, the higher the perspective distortion and structural instability of the marker. The specific calculation formula is: ,in, is the parallax interference factor; The spatial angle difference is used to detect whether three adjacent Aruco markers can form a right angle. The closer the triangle structure is to a right angle, the closer it is to the ideal state. The greater the deviation, the more serious the distortion. The specific calculation formula is: ,in, 、 、 They are the center coordinates of three adjacent Aruco markers, which are obtained by averaging the sub-pixel points at the four corners. This is the existing technology and will not be described in detail here. is the angle formula, Used to constrain the angle range to , when the angle is at 90°, the spatial angle difference , when the angle is 180° , when the angle is 0° ; It is the angle between the projection direction and the normal vector of the marker plane. The larger the value, the more serious the angular distortion. The specific calculation formula is: , where is the arc cosine function, which is used to limit the output to , is the overall projection direction, which is determined by the acquired main camera direction. The normal vector of the current marker is obtained by substituting the marker coordinates into the plane equation and using SVD decomposition to obtain the least squares solution. This is a prior art and will not be described in detail here. It is the system's tension evaluation value for each pixel, indicating whether the overall tension at this pixel is balanced based on the degree of deformation and credibility, and is used to guide the correction position. The specific calculation formula is: ,in, is the tension equilibrium domain; is the total number of pixels in the image; is the Gaussian weight, indicating the current pixel Aruco-marked The attenuation weight of the influence, the value range is The specific calculation formula is: ,in, is the attenuation coefficient, which is used to control the speed at which the Gaussian weight decays with distance. Its value range is between 0.01 and 0.1, and its value increases with the density of the markers. is the center coordinate of the Aruco marker, Calculate the Euclidean distance from the current pixel to the center coordinate of the Aruco marker.

[0071] The present invention is further configured to calculate a homography matrix between the known plane positions of the Aruco markers in the world coordinate system and their pixel coordinates in the image, and then invert the homography matrix to obtain the basic back-projection position;

[0072] Based on the back-projection position, the gradient response of the local tension equalization domain is introduced as the offset driving factor for each pixel position. The mapping correction strength is controlled by setting the tension response factor parameter to obtain the corrected projection position. Specifically, the basic back-projection position solution method is an existing mature technology and is widely used in plane calibration and image geometry correction. I will not go into details here. Corrected projection position Indicates the new projection position of each pixel after fine-tuning according to the tension; the calculation logic is: ,in, To correct the projection position; is the basic back-projection position; is the tension response factor, which is used to control the dynamic coefficient of the gradient correction intensity. get, is the global scaling factor, ranging from 0.1 to 1.0; is the gradient of the tension equalization domain, which represents the gradient vector of the tension field Ω(u,v) in the tension equalization domain, pointing to the fastest direction of deformation reduction. It is obtained by discrete calculation of the Sobel operator, which is an existing technology and will not be described in detail here.

[0073] The present invention is further configured to calculate the normal projection distance of each pixel point to all boundary lines based on the image boundary line segment set, and take the maximum value as the boundary disturbance index;

[0074] Combined with the tension equilibrium domain, the boundary disturbance index is normalized and weighted as a boundary disturbance compensation term.

[0075] On the basis of correcting the projection position, the boundary disturbance compensation term is superimposed to achieve the projection position correction and obtain the final mapping position. Specifically, the edge area is most susceptible to distortion and must be processed separately. Indicates the deviation distance of the pixel point in the normal direction of the boundary line. The larger the value, the more it deviates from the boundary position, and the more likely it is edge distortion. The boundary disturbance index calculation logic is: ,in, is the boundary disturbance index; For the boundary line The unit normal vector of the boundary line The coordinates are substituted into the plane equation and the least squares solution is obtained by SVD decomposition. It is a prior art and will not be described in detail here. is the original pixel coordinate; For the boundary line Any point on the line segment is usually the midpoint of the line segment, which is obtained by taking the average of the coordinates of the endpoints of the line segment; is the modulus of the normal vector, which is used for normalization to make the projection distance unit consistent; Used to calculate the absolute distance from the current pixel to the normal projection of the directed boundary segment, indicating the pixel in the image To boundary segment The normal distance, that is, the vertical distance; calculate the current pixel by traversing each boundary line The normal distance to the boundary line, the maximum value of all distances is taken as the maximum boundary disturbance intensity of the current pixel as the boundary disturbance index. The final mapping position is at the corrected projection position. The final mapping position is obtained after correction based on the boundary disturbance; the calculation logic is: ,in, is the final mapping position; The disturbance weight coefficient is used to control the compensation amplitude. The value range is between 0.05 and 1.0. Too large will lead to erroneous correction, while too small will lose the correction meaning. is the boundary disturbance compensation term, which is used to compensate for the image boundary area. Large, and the trapezoidal tension control is weak, that is The larger the compensation, the smaller the time, while the trapezoidal tension dominant area inside the image is If it is large, the disturbance is suppressed to prevent geometric distortion caused by overcorrection; It acts as a dampening factor in the denominator, preventing overcorrection of the trapezoidal tension-dominated region.

[0076] The present invention is further configured to include Aruco markers that simultaneously satisfy the parallax interference factor being less than the parallax threshold and the trapezoidal tension factor being less than the tension threshold among all current Aruco markers into the trusted marker set;

[0077] For each Aruco marker in the trusted marker set, the difference between its observed center coordinates in the image and the theoretical undistorted center position is calculated based on the corresponding camera rotation matrix to obtain the projection offset of the marker;

[0078] The three-dimensional offsets of all credible Aruco markers are averaged and solved to obtain the overall microscopic pose offset parameters. Specifically, the S4 aims to infer the micro-projection offset of the current system from a set of highly credible Aruco markers based on the offset relationship between their image observation center and the ideal distortion-free space center, combined with the spatial pose information of the marker itself (i.e., the rotation matrix). The micro-projection offset can be regarded as the micro-pose of the system, including directional micro-offset and center displacement, which is used for feedback guidance of high-precision projection alignment and active correction, and fine-tuning the projection angle to achieve high-precision correction of the projection image and obtain a clear projection image. Credible marker set The filter conditions are: , is the parallax threshold, with a default value of 0.3. A parallax interference factor less than 0.3 indicates that the angle deviation is small and fine-tuning correction can be achieved; is the tension threshold, the default value is 1.2, and the trapezoidal tension factor is close to the ideal value, indicating that the trapezoidal structure is relatively stable without large distortion; the Aruco markers that meet the conditions are included in the trusted marker set. The corresponding camera rotation matrix calculation method: using the triangulation method provided by opencv, the 3D world coordinates are calculated through the coordinates of the Aruco marker corners of the camera and the projection lens, and then the 3D world coordinates are used to solve the normal vector of the plane, and the three rotation angles are calculated with the z-axis of the projection lens to construct the camera's rotation matrix. This method is an existing technology and will not be elaborated on here. Micro pose offset parameter calculation formula: ,in, is the microscopic posture offset parameter, which represents the deviation adjustment of the overall system relative to the theoretical alignment state; is the total number of credible tags, is the set of credible tags The number of tags that meet the trust judgment; For the The transpose of the rotation matrix of each marker is used to transform the image observation coordinates back to the local space coordinate system of the marker; For the The observation center coordinates of a marker in the image are obtained by averaging the sub-pixel points at the four corners; For the The ideal center position of each marker under undistorted conditions is obtained by averaging the sub-pixel points of the final mapped position.

[0079] Example 2

[0080] See also Figure 2 , the exemplary micro-sensing trapezoidal device based on Aruco recognition includes:

[0081] Image acquisition module: Pre-project the Aruco grid structure through the projector, use the monocular camera to capture the image of the projection area, detect the corner points of each Aruco marker in the image, and construct the Aruco marker position set;

[0082] Distortion perception module: Calculates the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of ​​the Aruco marker;

[0083] Keystone Correction Module: This module constructs a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuses the trapezoidal deformation strength of each marker, generates a continuous tension equalization domain through Gaussian spatial attenuation weights, corrects the basic inverse projection mapping based on the tension gradient, and combines boundary normal distance disturbance compensation to obtain the final mapping position.

[0084] Pose output module: Based on the fusion results of multiple Aruco markers, it outputs the inverse perspective transformation matrix for projected image correction, further estimates the micro-pose offset parameters of the projected image, and fine-tunes the projector according to the micro-pose offset parameters.

[0085] It should be noted that the micro-sensing trapezoidal device based on Aruco recognition provided in the above embodiment and the micro-sensing trapezoidal method based on Aruco recognition provided in the above embodiment are based on the same concept. The specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the micro-sensing trapezoidal device based on Aruco recognition provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the system into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0086] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements a micro-sensing trapezoidal method based on Aruco recognition provided in the above-mentioned embodiments.

[0087] The embodiment of the present application further provides a computer system for an electronic device. It should be noted that the computer system for an electronic device is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present application.

[0088] Specifically, a computer system includes a central processing unit (CPU), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage into random access memory (RAM), such as the methods described in the above embodiments. RAM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0089] The following components are connected to the I / O interface: an input section including a keyboard and mouse; an output section including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; a storage section including a hard disk; and a communication section including network interface cards such as LAN (Local Area Network) cards and modems. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the I / O interface as needed. Removable media such as magnetic disks, optical disks, magneto-optical disks, and semiconductor memories are installed in the drive as needed, allowing computer programs read from these media to be installed in the storage section as needed.

[0090] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component and / or installed from removable media. When executed by a central processing unit (CPU), the computer program performs the various functions defined in the system of the present application.

[0091] It should be noted that the computer-readable medium described in the embodiments of this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may, for example, be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. This propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0092] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0093] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the aforementioned micro-sensing trapezoidal method based on Aruco recognition. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently without being incorporated into the electronic device.

[0094] Another aspect of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a micro-sensing trapezoidal method based on Aruco recognition provided in each of the above embodiments.

[0095] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, any equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A micro-sensing trapezoidal method based on Aruco recognition, characterized in that: include: S1: Pre-project the Aruco grid structure through the projector, use the monocular camera to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct the Aruco marker position set; S2: Calculation of the trapezoidal tension factor for each Aruco marker based on the area of ​​the Aruco-marked trapezoid; S3: Based on the spatial angle deviation and normal offset of multiple Aruco markers, the parallax interference factor is constructed, the trapezoidal deformation strength of each marker is integrated, and a continuous tension balance domain is generated through the Gaussian spatial attenuation weight. The basic inverse projection mapping is corrected based on the tension gradient, and the boundary normal distance disturbance compensation is combined to obtain the final mapping position, including: according to the image corner points of three groups of continuous Aruco markers, the spatial angle difference of the plane composed of three points is calculated, and the angle between the projection direction and the normal vector of the marker plane is combined to construct a normalized parallax interference factor; the center of each Aruco marker is used as the reference source point, and the Gaussian spatial attenuation function is used as the image A pixel-level weight function is used, and the inverse weighted form of the parallax interference factor is combined for weighted integration to construct a continuous tension equalization domain; the homography matrix is ​​calculated between the known plane position of the Aruco marker in the world coordinate system and its pixel coordinates in the image, and then the homography matrix is ​​inverted to obtain the basic back-projection position; based on the back-projection position, the gradient response of the local tension equalization domain is introduced as the offset driving factor for each pixel position, and the mapping correction strength is controlled by setting the tension response factor parameter to obtain the corrected projection position; based on the image boundary segment set, the normal projection distance of each pixel point to all boundary lines is calculated, and the maximum value is taken as the boundary disturbance index; Combined with the tension balance domain, the boundary disturbance index is normalized and weighted as a boundary disturbance compensation term. On the basis of correcting the projection position, the boundary disturbance compensation term is superimposed to achieve projection position correction and obtain the final mapping position. S4: Based on the fusion results of multiple Aruco markers, the inverse perspective transformation matrix for projected image correction is output, and the micro-pose offset parameters of the projected image are further estimated. The projector is fine-tuned according to the micro-pose offset parameters.

2. A micro-sensing trapezoidal method based on Aruco recognition according to claim 1, characterized in that: Said S1 comprises: The projector projects a standard square Aruco marker grid as a marker layer combined with the projection image onto the target area; The monocular camera captures the projection area and captures the deformed Aruco markers caused by the shape of the projection surface or the camera angle; Detect the coordinates of the four corner points of each Aruco marker and construct the Aruco marker position set; Edge detection is performed on the projection area image, and Hough transform is applied to extract valid edge segments. All the extracted straight line segments are combined into an image boundary segment set.

3. The micro-sensing trapezoidal method based on Aruco recognition according to claim 1 is characterized in that: The S2 includes: Based on the coordinates of the four corner points marked by Aruco, the lengths of the upper and lower bases of the trapezoid and the height of the vertical projection of the center line are determined, and the area of ​​the trapezoid is obtained by multiplying the height by the average of the upper and lower bases. Measure the Euclidean distances of two sets of opposite sides respectively and calculate their product. Calculate the ratio of the product of the opposite sides to twice the area to obtain the trapezoidal tension factor.

4. The micro-sensing trapezoidal method based on Aruco recognition according to claim 1 is characterized in that: Among all the current Aruco markers, the Aruco markers that simultaneously satisfy the parallax interference factor less than the parallax threshold and the trapezoidal tension factor less than the tension threshold are included in the credible marker set; For each Aruco marker in the trusted marker set, the difference between its observed center coordinates in the image and the theoretical undistorted center position is calculated based on the corresponding camera rotation matrix to obtain the projection offset of the marker; The 3D offsets of all reliable Aruco markers are averaged to obtain the overall microscopic pose offset parameters.

5. A micro-sensing trapezoidal device based on Aruco recognition, used to implement a micro-sensing trapezoidal method based on Aruco recognition according to any one of claims 1 to 4, characterized in that: include: Image acquisition module: Pre-project the Aruco grid structure through the projector, use the monocular camera to capture the image of the projection area, detect the corner points of each Aruco marker in the image, and construct the Aruco marker position set; Distortion perception module: Calculates the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of ​​the Aruco marker; Keystone Correction Module: This module constructs a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuses the trapezoidal deformation strength of each marker, generates a continuous tension equalization domain through Gaussian spatial attenuation weights, corrects the basic inverse projection mapping based on the tension gradient, and combines boundary normal distance disturbance compensation to obtain the final mapping position. Pose output module: Based on the fusion results of multiple Aruco markers, it outputs the inverse perspective transformation matrix for projected image correction, further estimates the micro-pose offset parameters of the projected image, and fine-tunes the projector according to the micro-pose offset parameters.

6. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement a micro-sensing trapezoidal method based on Aruco recognition as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute a micro-sensing trapezoidal method based on Aruco recognition according to any one of claims 1 to 4.

Images