Micro-sensing trapezoidal method, device and equipment based on Auco identification and medium

Abstract

The invention discloses a micro-sensing trapezoidal method, device and equipment based on Aruco recognition and a medium, and relates to the field of computer vision. The method comprises the following steps: S1, projecting an Aruco grid and collecting an image, and extracting marked angular points to construct a position set; s2, calculating a trapezoid tension factor based on the trapezoid area and the opposite side distance, and quantifying local deformation; s3, fusing multi-labeled spatial parallax and tension information, constructing a continuous tension equilibrium domain, and generating a final mapping position through gradient correction basic inverse projection mapping and boundary disturbance compensation; and S4, outputting an inverse perspective matrix based on a fusion result, and estimating a microscopic pose offset parameter for fine adjustment of the projector. And through tension factor and spatial parallax modeling, adaptive correction of a distortion region is realized, the edge consistency and overall symmetry of the image are improved, and the stability and precision of the projected image are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a micro-sensing trapezoid method, device, equipment and medium based on Aruco recognition. Background Art

[0002] Distortion and deformation of the projection screen are common problems in projector applications, significantly affecting the visual presentation effect and the accuracy of information transmission. To solve this problem, a variety of trapezoid correction technologies have been developed in the market.

[0003] Among them, the automatic trapezoid correction technology based on a camera represents the current mainstream direction. This technology captures the projection screen through an internal or external camera, uses image processing algorithms to identify the geometric distortion features of the screen edge, and then calculates the key parameters representing trapezoid distortion. Subsequently, the system applies digital image geometric transformation or drives the optical lens for physical displacement to perform dynamic compensation on the detected deformation. This solution that combines computer vision and intelligent algorithms shows higher correction accuracy and efficiency when dealing with complex scenarios such as side projection angles and non-standard projection surfaces compared to traditional manual adjustment, and can adapt to the re-calibration requirements after the position of the projector changes. Therefore, it has become a standard configuration for mid- to high-end commercial and household projection devices.

[0004] However, the existing automatic trapezoid correction technology system still has significant limitations, restricting the further improvement of the user experience: Relying on a dedicated calibration pattern: Most systems must interrupt the normal projection content and instead project a specially designed, structured reference pattern when performing the automatic correction process. These patterns usually appear in the form of grids, dot matrices or stripes. This mandatory pattern projection process interrupts the normal use of the user, is visually invasive, and cannot complete the correction without the user's perception or during continuous content playback.

[0005] Requiring a screen reset operation: To ensure that the algorithm can clearly and accurately extract the distortion information from the captured image, the system usually requires the projection screen to be temporarily adjusted to a preset, standardized initial state before performing the key analysis step. The most common is to center and fill the screen with the reference pattern. This "reset" step not only increases the complexity and time consumption of the correction process, but more importantly, it requires the projector to temporarily stop displaying the content that the user is actually watching, destroying the continuity and immersion of the use.

[0006] Limited real-time performance and adaptability: The above two deficiencies jointly lead to the difficulty of the prior art in achieving true "seamless" real-time correction. Relying on dedicated patterns and reset operations means that the system cannot continuously and imperceptibly monitor and compensate for possible minor pose offsets during the user's normal viewing of dynamic content. Its correction behavior is often discrete, perceptible to the user, and requires active triggering or execution under specific conditions.

[0007] These deficiencies indicate that although significant progress has been made in camera-based automatic trapezoidal correction technology, there is still room for improvement in terms of the smoothness, imperceptibility of the user experience, and the ability to handle continuous fine-tuning requirements. Summary of the Invention

[0008] Based on the above-mentioned disadvantages of the prior art, the object of the present invention is to provide a micro-sensing trapezoidal method, device, equipment, and medium based on Aruco recognition to solve the above technical problems.

[0009] To achieve the above object, the present invention provides the following technical solution: A micro-sensing trapezoidal method based on Aruco recognition, including: S1: Project an Aruco grid structure in advance through a projector, use a monocular camera to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct an Aruco marker position set; S2: Calculate the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of the Aruco marker; S3: Construct a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuse the trapezoidal deformation intensity of each marker, generate a continuous tension equilibrium domain through Gaussian spatial attenuation weights, correct the basic inverse projection mapping based on the tension gradient, and combine the boundary normal distance perturbation compensation to obtain the final mapping position; S4: Based on the fusion result of multiple Aruco markers, output an inverse perspective transformation matrix for correcting the projection image, and further estimate the microscopic pose offset parameters of the projection screen, and fine-tune the projector according to the microscopic pose offset parameters.

[0010] The present invention is further configured such that the S1 includes: The projector projects a standard square Aruco marker grid as a marker layer onto the target area in combination with the projection screen; The monocular camera captures the projection area and captures the deformed Aruco markers caused by the shape of the projection surface or the camera's perspective; Detect the four corner point coordinates of each Aruco marker and construct an Aruco marker position set; Perform edge detection on the projected area image, apply the Hough transform to extract valid edge segments, and form a set of image boundary line segments from all the extracted straight line segments.

[0011] The present invention is further configured such that the S2 includes: Based on the four corner coordinates of the Aruco marker, determine the lengths of the upper and lower bases of the trapezoid and the vertical projection height of the center connection line, and obtain the trapezoid area by multiplying the height by the average value of the upper and lower bases; Measure the Euclidean distances of two pairs of opposite sides respectively and calculate their product, and calculate the ratio of the product of the opposite sides to twice the area to obtain the trapezoid tension factor.

[0012] The present invention is further configured such that the S3 includes: Based on the image corner points of three groups of consecutive Aruco markers, calculate the spatial angle difference of the plane formed by the three points, and combine the angle between the projection direction and the normal vector of the marker plane to construct a normalized parallax interference factor; Taking the center of each Aruco marker as the reference source point, using the Gaussian form of the spatial attenuation function as the pixel-level weight function, and combining the inverse weighted form of the parallax interference factor for weighted integration to construct a continuous tension equilibrium domain.

[0013] The present invention is further configured such that the homography matrix is calculated between the plane position of the known Aruco marker in the world coordinate system and its pixel coordinates in the image, and then the inverse of the homography matrix is obtained to get the basic back-projection position; Based on the back-projection position, introduce the gradient response of the local tension equilibrium domain as the offset driving factor for each pixel position, and control the mapping correction intensity by setting the tension response factor parameter to obtain the corrected projection position.

[0014] The present invention is further configured such that the normal projection distance from each pixel point to all the boundary lines is calculated based on the set of image boundary line segments, and the maximum value is taken as the boundary perturbation index; Perform normalized weight assignment to the boundary perturbation index in combination with the tension equilibrium domain as the boundary perturbation compensation term; On the basis of the corrected projection position, superimpose the boundary perturbation compensation term to achieve the correction of the projection position and obtain the final mapping position.

[0015] The present invention is further configured such that the Aruco markers that simultaneously satisfy the condition that the parallax interference factor is less than the parallax threshold and the trapezoid tension factor is less than the tension threshold among all the current Aruco markers are included in the credible marker set; For each Aruco marker in the credible marker set, based on the rotation matrix of the corresponding camera, calculate the difference between its observed center coordinate in the image and the theoretical undistorted center position to obtain the projection offset of the marker; Average the three-dimensional offsets of all trusted Aruco markers to obtain the overall microscopic pose offset parameters.

[0016] The present invention also provides a micro-sensing trapezoidal device based on Aruco recognition, and the system includes: Image acquisition module: Project an Aruco grid structure in advance through a projector, use a monocular camera to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct a set of Aruco marker positions; Distortion perception module: Calculate the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of the Aruco marker; Trapezoidal correction module: Construct a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuse the trapezoidal deformation intensity of each marker, generate a continuous tension equilibrium domain through Gaussian spatial attenuation weight, correct the basic inverse projection mapping based on the tension gradient, and combine the boundary normal distance perturbation compensation to obtain the final mapping position; Pose output module: Based on the fusion result of multiple Aruco markers, output an inverse perspective transformation matrix for correcting the projection image, further estimate the microscopic pose offset parameters of the projection screen, and fine-tune the projector according to the microscopic pose offset parameters.

[0017] The present invention also provides an electronic device, and the electronic device includes: One or more processors; A storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements a micro-sensing trapezoidal method based on Aruco recognition as described in any one of the above.

[0018] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor of a computer, the computer executes a micro-sensing trapezoidal method based on Aruco recognition as described in any one of the above.

[0019] The present invention provides a micro-sensing trapezoid method, device, equipment and medium based on Aruco recognition. The method includes: S1: The projector pre-projects an Aruco grid structure, and a monocular camera is used to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct a set of Aruco marker positions; S2: Calculate the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of the Aruco marker; S3: Construct a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuse the trapezoidal deformation intensity of each marker, generate a continuous tension equilibrium domain through Gaussian spatial attenuation weight, correct the basic inverse projection mapping based on the tension gradient, and combine the boundary normal distance perturbation compensation to obtain the final mapping position; S4: Based on the fusion result of multiple Aruco markers, output an inverse perspective transformation matrix for correcting the projection image, and further estimate the microscopic pose offset parameters of the projection screen, and fine-tune the projector according to the microscopic pose offset parameters. The beneficial effects generated include: Introduce the trapezoidal tension factor to accurately quantify the degree of local deformation: Through the trapezoidal geometric structure formed by the corner points of the Aruco marker, calculate the area based on its upper and lower bottom edges and vertical height, and construct a "trapezoidal tension factor" in combination with the distance relationship between opposite sides to quantify the local deformation intensity, and solve the problem that the local distortion degree of the projection image cannot be finely characterized in the existing solutions.

[0020] Fuse multi-dimensional spatial parallax and tension to construct a continuous tension equilibrium domain and enhance the dynamic self-adaptability of the local area: Construct a parallax interference factor through the spatial angle difference and normal projection offset, and introduce a Gaussian weight attenuation function and an inverse interference weighting mechanism to realize the continuous fusion of the tension fields between multiple Aruco markers, effectively avoiding the global correction error caused by a single local reference, thereby forming a set of "continuous tension equilibrium domains" with spatial self-adaptability.

[0021] Consider the boundary perturbation term for compensation to improve the edge imaging consistency and the overall symmetry of the image: Use the Hough transform to extract the edge line segments, construct a boundary perturbation index in combination with the boundary normal distance, and add it to the projection correction position through normalization to effectively compensate for the blurring or drifting problems caused by uneven projection of the image edge, and improve the edge image quality and structural symmetry.

[0022] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. Description of the Drawings

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings. In the accompanying drawings: Figure 1 FIG. is a flowchart of a micro-sensing trapezoid method based on Aruco recognition shown in an exemplary embodiment of the present invention; Figure 2 FIG. is a schematic structural diagram of a micro-sensing trapezoid device based on Aruco recognition shown in an exemplary embodiment of the present invention. Detailed implementation manners

[0024] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for explaining the present invention and not for limiting the protection scope of the present invention.

[0025] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present invention. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0026] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0027] Embodiment 1 A micro-sensing trapezoid method based on Aruco recognition, as Figure 1 shown, includes: S1: Project an Aruco grid structure in advance through a projector, collect an image of the projection area using a monocular camera, detect the corner points of each Aruco marker in the image, and construct a set of Aruco marker positions; S2: Calculate the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of the Aruco marker; S3: Based on the spatial angle deviation and normal offset of multiple Aruco markers, construct a parallax interference factor, fuse the trapezoidal deformation intensity of each marker, generate a continuous tension equilibrium domain through Gaussian spatial attenuation weights, correct the basic inverse projection mapping based on the tension gradient, and combine the boundary normal distance perturbation compensation to obtain the final mapping position; S4: Based on the fusion result of multiple Aruco markers, output the inverse perspective transformation matrix for projective image correction, further estimate the micro pose offset parameters of the projection screen, and fine-tune the projector according to the micro pose offset parameters.

[0028] The present invention is further configured such that the S1 includes: The projector projects a standard square Aruco marker grid as a marker layer onto the target area in combination with the projection screen; The monocular camera captures the projection area and captures the deformed Aruco markers caused by the shape of the projection surface or the camera's perspective; Detect the four corner coordinates of each Aruco marker and construct an Aruco marker position set; Perform edge detection on the projection area image, apply the Hough transform to extract effective edge line segments, and form an image boundary line segment set with all the extracted line segments. Specifically, the marker layer does not need to reset the screen during trapezoidal correction, nor does it need to fully block the projection screen. Only a separate marker layer needs to be added to the projection screen during correction, and Aruco codes are displayed in the four corner areas of the separate marker layer for recognition. It has higher anti-interference ability than the traditional checkerboard grid. Users can continue to watch the screen, greatly improving the user experience on the basis of ensuring the trapezoidal algorithm. The monocular camera is deployed directly in front of the projector, at the same level as the projection lens position; use the Android API to capture the Aruco code image through the monocular camera; use the method provided by opencv to find the corner pixel coordinates of the Aruco code and construct the Aruco marker position set , , where is the two-dimensional coordinate of the four corners of the th Aruco marker in the image space. The pixel coordinates of the four corners of the Aruco marker are collected in a clockwise manner and are respectively , , , ; Integrate all the collected Aruco marker positions to obtain the Aruco marker position set . The image boundary line segment set The acquisition is achieved through the following steps: Based on the distribution of Aruco markers pre-projected by the projector, detect the peripheral corner points of all markers, calculate their convex hull or minimum circumscribed polygon, and extract the polygon edges as the initial set of boundary line segments; perform edge detection on the distorted image captured by the camera using the Canny operator, and then extract significant line segments through the Hough transform, and filter the line segments that match the geometric constraints of the projection area , update the boundary set to obtain the final set of image boundary line segments , this method is an existing technology and will not be elaborated here

[0029] The present invention is further configured such that the S2 includes: Based on the four corner coordinates of the Aruco marker, determine the lengths of the upper and lower bases of the trapezoid and the vertical projection height of the center connection line, and obtain the trapezoid area by multiplying the height by the average value of the upper and lower bases; Measure the Euclidean distances of two groups of opposite sides respectively and calculate their product, and calculate the ratio of the product of the opposite sides to twice the area to obtain the trapezoid tension factor. Specifically, based on the trapezoid area of the Aruco marker, where the connection lines of adjacent corner points form the two bases and the sides of the trapezoid; based on the projection length of the connection line of the marker centers in the vertical direction as the trapezoid height, and at the same time extract the actual Euclidean distances of the upper and lower bases as the basic geometric parameters; calculate the area of the trapezoid area by taking half of the product of the trapezoid height and the sum of the upper and lower base lengths; define the ratio of the product of the lengths of two groups of opposite sides to twice the trapezoid area as the trapezoid tension factor ; trapezoid tension factor is used to measure whether the marker has trapezoidal deformation. Ideally, the trapezoid tension factor should approach 1. The smaller the value, the more significant the perspective compression or stretching deformation of the marker, and the more severe the distortion. The specific calculation formula for the trapezoid area is: , where is the trapezoid area, is the chasing projection height obtained by projecting the connection line of the midpoints of the two sides to the vertical direction, is the length of the upper side, obtained by calculating , is the length of the lower side, obtained by calculating ; the calculation formula for the trapezoid tension factor is: , where is the trapezoid tension factor, is the Euclidean distance

[0030] The present invention is further configured such that the S3 includes: Based on the image corner points of three groups of consecutive Aruco markers, calculate the spatial angle difference of the plane formed by three points, and combine the angle between the projection direction and the normal vector of the marker plane to construct a normalized parallax interference factor; Taking the center of each Aruco marker as the reference source point, a spatial attenuation function in the form of a Gaussian is used as the pixel-level weight function, and combined with the inverse weighted form of the parallax interference factor for weighted integration to construct a continuous tension equilibrium domain. Specifically, the parallax interference factor is the final credibility that fuses the consistency of the angle structure and the influence of the viewing angle interference. The larger it is, the higher the distortion of the marker viewing angle and the structural instability; the specific calculation formula is: , where is the parallax interference factor; is the spatial angle difference, used to detect whether three adjacent Aruco markers can form a right angle. The closer the triangular structure is to a right angle, the closer it is to the ideal state. The more it deviates, the more serious the distortion. The specific calculation formula is: , where , , are the center coordinates of three adjacent Aruco markers respectively, obtained by averaging the sub-pixel points at the four corners. This is prior art and will not be elaborated here is the angle formula, is used to constrain the angle range to . When the angle is 90°, the spatial angle difference . When the angle is 180°, . When the angle is 0°, ; is the angle between the projection direction and the normal vector of the marker plane. The larger the value, the more serious the angle distortion. The specific calculation formula is: , where is the arccosine function, used to limit the output to , is the overall projection direction, determined by the direction of the main camera collected, is the normal vector of the current marker, calculated by substituting the marker coordinates into the plane equation and obtaining the least squares solution using svd decomposition. This is prior art and will not be elaborated here. The tension equilibrium domain is the tension evaluation value of the system for each pixel, indicating whether the overall tension at this pixel point is balanced by comprehensively considering the deformation degree and credibility, and is used to guide the correction position; the specific calculation formula is: , where is the tension equilibrium domain; is the total number of pixel points in the image; is the Gaussian weight, indicating that the current pixel point is affected by the Aruco marker attenuation weight, with the value range between , and the specific calculation formula is: , where is the attenuation coefficient, which is used to control the attenuation speed of the Gaussian weight with distance. The value range is between 0.01 and 0.1, and the value increases with the increase of the marker density. is the center coordinate of the Aruco marker. is to calculate the Euclidean distance from the current pixel point to the center coordinate of the Aruco marker.

[0031] The present invention is further configured to calculate the homography matrix between the known planar position of the Aruco marker in the world coordinate system and its pixel coordinates in the image, and then invert the homography matrix to obtain the basic back-projection position. Based on the back-projection position, introduce the gradient response of the local tension equilibrium domain as the offset driving factor for each pixel position, and control the mapping correction intensity by setting the tension response factor parameter to obtain the corrected projection position. Specifically, the method for solving the basic back-projection position is a mature existing technology, which is widely used in planar calibration and image geometric correction, and will not be elaborated here. The corrected projection position represents the new projection position of each pixel after fine-tuning according to the tension; the calculation logic is: , where is the corrected projection position; is the basic back-projection position; is the tension response factor, a dynamic coefficient used to control the gradient correction intensity, obtained by calculating and is the global scaling factor, and the value range is 0.1 to 1.0; is the gradient of the tension equilibrium domain, which represents the gradient vector of the tension equilibrium domain of the tension field Ω(u, v), pointing to the direction where the deformation decreases fastest, and is obtained by discrete calculation using the Sobel operator. It is an existing technology and will not be elaborated here.

[0032] The present invention is further configured to calculate the normal projection distance from each pixel point to all boundary lines based on the set of image boundary line segments, and take the maximum value as the boundary perturbation index. Combine the tension equilibrium domain to perform normalized weight assignment on the boundary perturbation index as the boundary perturbation compensation term. On the basis of the corrected projection position, superimpose the boundary perturbation compensation term to realize the correction of the projection position and obtain the final mapping position. Specifically, the edge area is most vulnerable to distortion and must be processed separately. The boundary perturbation index represents the deviation distance of the pixel point in the normal direction of the boundary line. The larger the value, the more deviated from the proper position of the boundary, and the more likely it is an edge distortion. The calculation logic of the boundary perturbation index is: , where is the boundary perturbation index; is the boundary line 's unit normal vector, by The least squares solution is obtained by substituting coordinates into the plane equation and using SVD decomposition, which is an existing technology and will not be elaborated here; is the original pixel coordinate; is the boundary line Any point on, usually the midpoint of the line segment, is obtained by taking the average of the endpoint coordinates of the line segment; is the modulus of the normal vector, which is used for normalization to make the unit of the projection distance consistent; is used to calculate the absolute distance of the normal projection of the current pixel point to the directed boundary line segment, representing the pixel point in the image to the boundary line segment The normal distance, that is, the vertical distance; by traversing each boundary line to calculate the normal distance of the current pixel to the boundary line, taking the maximum value among all distances as the maximum boundary perturbation influence intensity received by the current pixel as the boundary perturbation index. The final mapping position is obtained by correcting the projection position after considering the boundary perturbation; the calculation logic is: , where, is the final mapping position; is the perturbation weight coefficient, which is used to control the compensation amplitude, and its value range is between 0.05 and 1.0. If it is too large, it will cause incorrect correction, and if it is too small, it will lose the meaning of correction; is the boundary perturbation compensation term, which is used in the image boundary region, that is is large, and the trapezoidal tension control is weak, that is is small, and a large compensation is given, while in the trapezoidal tension-dominated region inside the image, that is is large, the perturbation is suppressed to prevent geometric distortion caused by excessive correction; plays an inhibitory factor role in the denominator to prevent excessive correction of the trapezoidal tension-dominated region.

[0033] The present invention is further set to include the Aruco markers that simultaneously satisfy the parallax interference factor being less than the parallax threshold and the trapezoidal tension factor being less than the tension threshold among all current Aruco markers into the credible marker set; For each Aruco marker in the credible marker set, based on the rotation matrix of the corresponding camera, the difference between its observed center coordinate in the image and the theoretical distortion-free center position is calculated to obtain the projection offset of the marker; Average the three-dimensional offsets of all reliable Aruco markers to obtain the overall microscopic pose offset parameters. Specifically, S4 aims to deduce the small projection offset of the current system from a set of highly reliable Aruco markers based on the offset relationship between their image observation centers and the centers of the ideal distortion-free space, combined with the spatial pose information (i.e., rotation matrix) of the markers themselves. The small projection offset can be regarded as the micro-pose of the system, including the direction micro-offset and the center displacement, which are used for the feedback guidance of high-precision projection alignment and active correction, and fine-tune the projection angle to achieve high-precision correction of the projection screen and obtain a clear projection image. The set of reliable markers The screening conditions for are as follows: , is the parallax threshold, with a default value of 0.3. A parallax interference factor less than 0.3 indicates that the angle deviation is small and fine-tuning correction can be achieved; is the tension threshold, with a default value of 1.2. The trapezoidal tension factor close to the ideal value indicates that the trapezoidal structure is relatively stable without large distortion; Incorporate the Aruco markers that meet the conditions into the set of reliable markers. The calculation method of the rotation matrix of the corresponding camera: Use the triangulation method provided by opencv to calculate the 3D world coordinates through the coordinates of the Aruco marker corners of the camera and the projection lens, and then use the 3D world coordinates to solve the normal vector of the plane, and calculate 3 rotation angles with the z-axis of the projection lens to construct the rotation matrix of the camera. This method is an existing technology and will not be elaborated here. Microscopic pose offset parameter calculation formula: , where is the microscopic pose offset parameter, indicating the deviation adjustment amount of the overall system relative to the theoretical alignment state; is the total number of reliable markers, which is the number of markers in the set of reliable markers that meet the reliable judgment; is the transpose of the rotation matrix of the th marker, which is used to convert the image observation coordinates back to the local space coordinate system where the marker is located; is the observation center coordinate of the th marker in the image, which is obtained by averaging the sub-pixel points of the four corners; is the ideal center position of the th marker under the condition of no distortion, which is obtained by averaging the sub-pixel points of the final mapping position.

[0034] Embodiment 2 Please refer to Figure 2 , the exemplary micro-sensing trapezoidal device based on Aruco recognition includes: Image acquisition module: Project an Aruco grid structure through a projector, use a monocular camera to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct a set of Aruco marker positions; Distortion perception module: Calculate the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of the Aruco marker; Trapezoidal correction module: Construct a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuse the trapezoidal deformation intensity of each marker, generate a continuous tension equilibrium domain through Gaussian spatial attenuation weights, correct the basic inverse projection mapping based on the tension gradient, and combine the boundary normal distance perturbation compensation to obtain the final mapping position; Pose output module: Based on the fusion result of multiple Aruco markers, output an inverse perspective transformation matrix for correcting the projected image, and further estimate the microscopic pose offset parameters of the projected image, and fine-tune the projector according to the microscopic pose offset parameters.

[0035] It should be noted that a micro-sensing trapezoidal device based on Aruco recognition provided by the above embodiment and a micro-sensing trapezoidal method based on Aruco recognition provided by the above embodiment belong to the same concept. The specific ways in which each module and unit perform operations have been described in detail in the method embodiment, and will not be repeated here. A micro-sensing trapezoidal device based on Aruco recognition provided by the above embodiment can, in practical applications, allocate the above functions to different functional modules as needed, that is, divide the internal structure of the system into different functional modules to complete all or part of the functions described above. This is not limited here either.

[0036] An embodiment of the present application also provides an electronic device, including: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements a micro-sensing trapezoidal method based on Aruco recognition provided in each of the above embodiments.

[0037] An embodiment of the present application also provides a computer system of an electronic device. It should be noted that the computer system of the electronic device is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present application.

[0038] Specifically, the computer system includes a Central Processing Unit (CPU), which can perform various appropriate actions and processes according to the program stored in the Read-Only Memory (ROM) or the program loaded from the storage section into the Random Access Memory (RAM), such as executing the methods described in the above embodiments. In the RAM, various programs and data required for system operation are also stored. The CPU, ROM, and RAM are connected to each other via a bus. An Input / Output (I / O) interface is also connected to the bus.

[0039] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc. and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable medium, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive as needed so that the computer program read from it can be installed into the storage section as needed.

[0040] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section, and / or installed from a removable medium. When the computer program is executed by a Central Processing Unit (CPU), various functions defined in the system of the present application are executed.

[0041] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0042] The units involved in the embodiments described in the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0043] Another aspect of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a micro-sensing trapezoidal method based on Aruco recognition as described above. The computer-readable storage medium can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device.

[0044] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes a micro-sensing trapezoidal method based on Aruco recognition provided in each of the above embodiments.

[0045] The above embodiments are only used to exemplarily illustrate the principles and effects of the present invention, rather than to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A micro-sensing trapezoid method based on Aruco recognition, characterized in that, Including: S1: Project an Aruco grid structure through a projector, use a monocular camera to collect images of the projection area, detect the corner points of each Aruco marker in the image, and construct a set of Aruco marker positions; S2: Calculate the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of the Aruco marker; S3: Construct a parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuse the trapezoidal deformation intensities of each marker, generate a continuous tension equilibrium domain through Gaussian spatial attenuation weights, correct the basic inverse projection mapping based on the tension gradient, and combine the boundary normal distance perturbation compensation to obtain the final mapping position; S4: Based on the fusion results of multiple Aruco markers, output an inverse perspective transformation matrix for correcting the projection image, and further estimate the micro pose offset parameters of the projection screen, and fine-tune the projector according to the micro pose offset parameters.

2. The micro-sensing trapezoidal method based on Aruco recognition according to claim 1, wherein, The S1 includes: The projector projects a grid of standard square Aruco markers as a marker layer onto the target area in combination with the projection screen; The monocular camera captures the projection area and captures the deformed Aruco markers caused by the shape of the projection surface or the camera's perspective; Detect the four corner point coordinates of each Aruco marker and construct a set of Aruco marker positions; Perform edge detection on the projection area image, apply the Hough transform to extract effective edge line segments, and form a set of image boundary line segments from all the extracted line segments.

3. The micro-sensing trapezoidal method based on Aruco recognition according to claim 1, characterized in that, The S2 includes: Determine the lengths of the upper and lower bases of the trapezoid and the vertical projection height of the center connection line based on the four corner point coordinates of the Aruco marker, and obtain the trapezoidal area by multiplying the height by the average of the upper and lower bases; Measure the Euclidean distances of two pairs of opposite sides respectively and calculate their product, and calculate the ratio of the product of the opposite sides to twice the area to obtain the trapezoidal tension factor.

4. A micro-sensing trapezoidal method based on Aruco recognition according to claim 1, characterized in that, The S3 includes: Based on the image corner points of three groups of consecutive Aruco markers, calculate the spatial angle difference of the plane formed by three points, and combine the angle between the projection direction and the normal vector of the marker plane to construct a normalized parallax interference factor; Using the center of each Aruco marker as the reference source point, adopting a Gaussian form of spatial attenuation function as the pixel-level weight function, and combining the inverse weighted form of the parallax interference factor for weighted integration to construct a continuous tension equilibrium domain.

5. A micro-sensing trapezoid method based on Aruco recognition according to claim 4, characterized in that Calculate the homography matrix between the known planar position of the Aruco marker in the world coordinate system and its pixel coordinates in the image, and then invert the homography matrix to obtain the basic back-projection position; Based on the back-projection position, introduce the gradient response of the local tension equilibrium domain as the offset driving factor for each pixel position, and control the mapping correction intensity by setting the tension response factor parameter to obtain the corrected projection position.

6. A micro-sensing trapezoid method based on Aruco recognition according to claim 5, characterized in that Calculate the normal projection distance from each pixel point to all boundary lines based on the set of image boundary line segments, and take the maximum value as the boundary perturbation index; Combine the tension equilibrium domain to perform normalized weight assignment on the boundary perturbation index as the boundary perturbation compensation term; On the basis of correcting the projection position, a boundary perturbation compensation term is superimposed to achieve the correction of the projection position and obtain the final mapping position.

7. A micro-sensing trapezoidal method based on Aruco recognition according to claim 1, characterized in that Aruco markers that simultaneously satisfy the condition that the parallax interference factor is less than the parallax threshold and the trapezoidal tension factor is less than the tension threshold among all current Aruco markers are included in the credible marker set; For each Aruco marker in the credible marker set, based on the rotation matrix of the corresponding camera, the difference between its observed center coordinates in the image and the theoretical undistorted center position is calculated to obtain the projection offset of the marker; The three-dimensional offsets of all credible Aruco markers are averaged to obtain the overall microscopic pose offset parameter.

8. A micro-sensing trapezoidal device based on Aruco recognition, which is used to implement a micro-sensing trapezoidal method based on Aruco recognition according to any one of claims 1-7, characterized in that, Including: Image acquisition module: The Aruco grid structure is pre-projected by the projector, and the monocular camera is used to acquire the image of the projection area, detect the corner points of each Aruco marker in the image, and construct the Aruco marker position set; Distortion perception module: Calculate the trapezoidal tension factor of each Aruco marker based on the trapezoidal area of the Aruco marker; Trapezoidal correction module: Construct the parallax interference factor based on the spatial angle deviation and normal offset of multiple Aruco markers, fuse the trapezoidal deformation intensity of each marker, generate a continuous tension equilibrium domain through Gaussian spatial attenuation weight, correct the basic inverse projection mapping based on the tension gradient, and combine the boundary normal distance perturbation compensation to obtain the final mapping position; Pose output module: Based on the fusion result of multiple Aruco markers, output the inverse perspective transformation matrix for correcting the projection image, further estimate the microscopic pose offset parameter of the projection screen, and fine-tune the projector according to the microscopic pose offset parameter.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enabling the electronic device to implement a micro-sensing trapezoidal method based on Aruco recognition according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, when the computer program is executed by the processor of the computer, enabling the computer to execute a micro-sensing trapezoidal method based on Aruco recognition according to any one of claims 1 to 7.

Images