Mirror image display image overturning processing method and system for AR (Augmented Reality) equipment

Abstract

The invention discloses a mirror image display image overturning processing method and system for AR equipment. The method comprises the steps that layered mirror image overturning is carried out; performing unified horizontal flipping processing on the input image of the physical camera and the virtual rendering layer through a GPU shader; independently rendering the UI channel; in a UI channel independent of the main image channel, drawing a UI layer according to the original space coordinates; calibrating interactive coordinates; performing reverse mapping on the position coordinate of the user operation event through a coordinate reverse mapping algorithm, so that the position coordinate is matched with the logic position of the UI control which is not overturned; optimizing dynamic performance; and monitoring a system load index in real time, and dynamically adjusting an image processing parameter according to a performance threshold so as to maintain a stable frame rate. The invention provides an efficient integrated solution, and the consistency of vision and interaction is realized through a cooperative processing mechanism of Shader image flipping and multi-channel rendering decoupling design. And the image direction logic consistency, the operation interface readability and the user experience real-time performance are effectively considered.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method and system for image flipping processing in mirrored display of AR devices. Background Technology

[0002] With the increasing prevalence of augmented reality (AR) technology in social entertainment, education and training, industrial inspection, and other scenarios, users' demand for immersive first-person perspective selfie experiences is growing. However, existing AR devices (such as AR glasses and mobile AR applications) have the following functional deficiencies in implementing selfie functions:

[0003] 1. Lack of mirrored visual feedback (spatial perception misalignment): Traditional AR systems directly display the camera's raw image without horizontal mirroring, causing user actions and screen feedback to appear reversed (e.g., when a user raises their right hand, the screen displays the action as a left hand). This perceptual misalignment leads to cognitive load for the user, affecting operational accuracy.

[0004] 2. Interactive interface failure (UI element flipping damage): If the entire screen is simply mirrored, the UI controls (buttons, text, icons) will be flipped synchronously, resulting in loss of readability and confusion in the interaction logic.

[0005] 3. Performance and real-time challenges (increased rendering load): Image processing requires additional computing resources, which can easily lead to a drop in frame rate and affect smoothness; in addition, the real-time requirements are high, and excessive latency will destroy the immersion and reduce the user experience. Summary of the Invention

[0006] To address the shortcomings of existing technologies where processing modules are fragmented and lack a unified framework for specific applications, this invention provides a method and system for image flipping in AR devices. It proposes an integrated and efficient solution that achieves visual and interactive consistency through a collaborative processing mechanism that decouples shader image flipping and multi-channel rendering. Furthermore, it effectively balances the logical consistency of screen orientation, the readability of the user interface, and the real-time performance of the user experience.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for image flipping in a mirrored display for AR devices includes the following steps:

[0009] Step S1: Layered mirror flip; Perform a uniform horizontal flip on the physical camera input image and the virtual rendering layer using the GPU shader;

[0010] Step S2: Independent rendering of the UI channel; In the UI channel, which is independent of the main channel, draw the UI layer according to the original spatial coordinates;

[0011] Step S3: Interactive coordinate calibration; The position coordinates of the user operation event are reverse-mapped using a coordinate reverse mapping algorithm to match the logical position of the unflipped UI control;

[0012] Step S4: Dynamic performance optimization; real-time monitoring of system load indicators, and dynamic adjustment of image processing parameters based on performance thresholds to maintain stable frame rate.

[0013] Preferably, step S1 specifically includes:

[0014] Step S11: Use a fragment shader to perform a mapping transformation on the input texture coordinates: map the original horizontal coordinate u to u´ = 1.0 - u;

[0015] Step S12: Enable depth testing and retain depth writing (ZWrite On) in the main image channel to ensure that the camera image and the virtual layer are correctly occluded and rendered according to the actual depth relationship; avoid mirror flipping to destroy the original occlusion order.

[0016] Step S13: The camera image and the virtual layer share the same shader, and the layer type is distinguished by passing in different materials and textures in the shader.

[0017] Preferably, step S2 specifically includes:

[0018] Step S21: A multi-channel rendering pipeline architecture is adopted. The main image channel is responsible for rendering the flipped camera image and virtual layer. The UI layer is rendered using Unity's Screen Space - Overlay mode, which does not participate in depth testing and writing by default, ensuring that the rendering order is always above the image channel.

[0019] Step S22: Overlay the UI layer onto the flipped main image channel output screen using Alpha blending technology.

[0020] Preferably, the coordinate reverse mapping algorithm in step S3 specifically includes:

[0021] Step S31: Convert the screen click coordinates (x, y) to coordinates (Wx, y) in the mirror coordinate system, where W is the screen width;

[0022] Step S32: Use the transformed coordinates to trigger the original interaction logic of the UI control.

[0023] Preferably, the dynamic performance optimization in step S4 specifically includes:

[0024] Step S41: Performance monitoring metrics; real-time collection of at least one of frame rate, GPU single-frame rendering time, and frame drop rate;

[0025] Step S42: Dynamic resolution adjustment; if the frame rate is lower than the set threshold for more than 5 consecutive frames or the GPU single frame rendering time is greater than 16.6ms, the input image resolution will be downgraded and bilinear interpolation will be enabled to compensate for image quality loss.

[0026] Step S43: Shader optimization; replace branch instructions with branchless functions to reduce GPU computational overhead; use mediump precision for variables in non-critical computation parts to reduce floating-point computation costs; merge camera images and virtual layers and store them in the same Texture2DArray, and sample them in the Shader using array indexing.

[0027] Preferably, in step S42, the resolution downgrading is simultaneously applied to both the image acquisition end and the shader processing end, and the downgrading switch satisfies the following conditions:

[0028] After each gear shift, maintain stable operation at the current gear position for ≥1.5 seconds;

[0029] RenderTexture scaling, combined with a Mipmap strategy, is used to avoid screen jumps.

[0030] Preferably, the method further includes step S5, delay control, which specifically includes:

[0031] Step S51: Mark a unified timestamp for the camera image frame and the virtual rendering frame;

[0032] Step S52: When the virtual layer rendering is delayed, virtual keyframes are generated by predicting using Kalman filtering based on the gyroscope and acceleration data from the IMU sensor.

[0033] Step S53: When the camera image is delayed, the rendered virtual frame is temporarily stored in the frame buffer object and awaits synchronous compositing.

[0034] Preferably, the method further includes a misalignment correction mechanism in step S6, specifically comprising:

[0035] Step S61: Reverse map the flipped camera image back to the original orientation;

[0036] Step S62: Extract local feature points from the camera image and the virtual layer, and perform preliminary matching using Hamming distance or Euclidean distance;

[0037] Step S63: Filter out erroneous matching points using the random sampling consensus algorithm;

[0038] Step S64: Calculate the affine transformation matrix based on the matching point pairs, and apply the transformation matrix to the virtual layer rendering coordinate system to achieve sub-pixel level alignment.

[0039] Preferably, the misalignment correction mechanism in step S6 is triggered only when the feature point matching confidence is >85%, and the affine transformation matrix is ​​limited to a local region.

[0040] The present invention also discloses a mirror display image flipping processing system for implementing the above-described mirror display image flipping processing method for AR devices, comprising:

[0041] Layered mirroring module: configured to perform synchronous horizontal flipping and depth management of virtual and real layers;

[0042] UI Channel Isolation Module: Configured to render the positive UI layer independently and composite it using Alpha;

[0043] Dynamic optimization engine: Configured to monitor GPU load in real time and trigger resolution / shader optimization strategies;

[0044] Delay control unit: configured to maintain an operation response delay of <20ms through timestamp synchronization, keyframe prediction and misalignment correction.

[0045] This invention provides a method and system for image flipping in AR devices, offering the following advantages: By replacing the traditional Z-Test with a depth ratio threshold mixing algorithm, the virtual-to-real alignment error is reduced to <0.5px, resolving the rendering anomaly issue of semi-transparent virtual objects. Furthermore, through a collaborative processing mechanism that decouples shader image flipping and multi-channel rendering, visual and interactive consistency is achieved. Simultaneously, it balances the consistency of image orientation logic, the readability of the user interface, and the real-time performance of the user experience, demonstrating good practicality and portability, and is suitable for various AR application scenarios such as social check-ins, educational interactions, and industrial inspections. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of this invention or the prior art will be briefly introduced below.

[0047] Figure 1 Schematic diagram of the steps of this invention;

[0048] Figure 2 Flowchart of camera image and virtual layer mirroring processing in this invention;

[0049] Figure 3 A diagram illustrating the mechanism for achieving consistency between user operation and visual feedback in this invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0051] Examples, such as Figure 2 As shown, this invention discloses a method and system for image flipping processing in mirrored display for AR devices, comprising the following steps:

[0052] Step S1: Layered Mirror Flip; The GPU shader performs a unified horizontal flip on the camera image input from the physical camera and the virtual rendering layer, ensuring a "mirror-like" visual feedback experience for the user. The camera image and the virtual rendering layer share a unified shader during image post-processing. Different material and texture values ​​are passed into the shader to distinguish layer types, facilitating unified control of the coordinate transformation matrix and improving synchronization.

[0053] Specifically, including:

[0054] Step S11: Use a fragment shader to map the input texture coordinates (u, v): map the original horizontal coordinate u to u´=1.0-u, and the original vertical coordinate v to v´=v;

[0055] Step S12: Preserve the original value of the depth buffer (Z-Buffer), and maintain the occlusion relationship between the real and virtual layers by combining layer depth sorting control (through Z-Test and Z-Write strategies); ensure that the occlusion relationship is not destroyed during the flipping process, and that the real and virtual layers blend naturally;

[0056] Step S13: The camera image and the virtual layer share the same shader, and the layer type is distinguished by passing in different materials and textures in the shader.

[0057] Furthermore, in this invention, real-time rendering efficiency can be improved by optimizing and simplifying the shader instruction set, thereby reducing the number of computational instructions, optimizing data flow, and eliminating performance bottlenecks. Specifically, this includes:

[0058] Matrix multiplication optimization: Reduce frequent calls to mat3 * vec3 and mat4 * vec4, retaining only one necessary transformation in vertex transformations. Simultaneously, pre-compute transformation results are sent to a Uniform, prohibiting matrix multiplication in fragment shaders.

[0059] Conditional branch elimination: Try to avoid branching instructions such as if / else or discard, and use branchless functions such as step() / mix() / clamp() instead.

[0060] Texture sampling optimization: Avoid multiple sampling and multi-channel texture switching, and uniformly bind the camera image and virtual layer to the same Texture2DArray index position; at the same time, reduce the number of texture() calls in the fragment shader to avoid wasting bandwidth.

[0061] High-precision computation downgrade: In non-essential computations, highp precision is reduced to mediump or lower quality, which can significantly reduce register pressure and launch latency, especially on mobile devices (such as Adreno GPUs).

[0062] The streamlined shader can control single-frame image processing to not exceed 25–30% of the total GPU frame budget (approximately 3–5ms), leaving room for subsequent UI compositing, effects rendering, and other operations. In a tile-based architecture (such as Mali GPU), single-tile processing time is less than 2ms under multi-tile concurrency. Furthermore, the low-cycle design ensures that the system maintains a stable frame rate and temperature even under continuous operation, avoiding GPU thread blocking and thermal management triggers. It does not cause abnormal frame latency peaks.

[0063] Step S2: Independent rendering of the UI channel; In the UI channel, which is independent of the main channel, draw the UI layer according to the original spatial coordinates; Ensure that UI elements are not affected by mirror flipping, and maintain the interface's intuitive readability; Specifically, this includes:

[0064] Step S21: To address the impact of mirror flipping on UI readability and interaction logic, a multi-channel rendering pipeline architecture is adopted; the main image channel (physical screen + virtual layer) and the UI channel are rendered separately, thereby decoupling the visual and interaction logic.

[0065] The main image channel (including camera images and virtual layers) undergoes uniform horizontal mirroring by the shader to ensure users receive a natural "mirror view" feedback. The flipping process is completed through the fragment shader, applying a uniform texture coordinate mapping function (e.g., u´=1.0-u).

[0066] The UI rendering path operates independently: UI layers do not participate in mirroring. To achieve independent rendering and correct overlay of UI layers, the system uses an auxiliary UI rendering channel (Overlay Camera). This channel is configured above the main image channel, maintaining the original orientation of the output content without mirroring. When the system uses the Unity UI framework (UGUI), all UI elements are managed by the Canvas component, and their rendering order is controlled by Sort Order and RenderQueue. By setting Render Mode to Screen Space - Overlay or Camera mode and fixing the rendering order, the system ensures that the UI layer is always placed above the main image layer, thus avoiding the effects of mirroring.

[0067] Z-Buffer Management Strategy: At this time, the UI layer disables depth writing (ZWrite Off) and is forced to be set at the maximum Z value position; the main image layer enables Z-Test to ensure natural occlusion between virtual and real objects; and UI controls are prevented from being occluded by virtual objects, improving visibility and operability.

[0068] Step S22: UI and main image compositing synchronization logic; Frame synchronization mechanism: UI frame updates and image frames use the same timestamp (TimeStamp), and are merged according to frame number during compositing to ensure visual frame consistency; Alpha channel fusion: The UI layer is overlaid in the image post-processing module through AlphaBlend technology (blending mode enabled: GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) to ensure smooth edge blending and natural occlusion transition.

[0069] Step S3: Interaction Coordinate Calibration; The UI layer is not flipped, so its click response area is not affected by the shader; Keep the UI component's screen space coordinates unchanged, ensuring that input events are consistent with pixel positions; Reverse map the position coordinates of user operation events using a coordinate reverse mapping algorithm to match the logical positions of the unflipped UI controls; Specifically, the coordinate reverse mapping algorithm includes:

[0070] Step S31: Convert the screen click coordinates (x, y) to coordinates (Wx, y) in the mirror coordinate system, where W is the screen width;

[0071] Step S32: Use the transformed coordinates to trigger the original interaction logic of the UI control.

[0072] Step S4: Dynamic Performance Optimization; This invention employs a multi-dimensional dynamic performance monitoring mechanism to determine in real time whether the current device load exceeds the stable operating threshold. Specific monitoring indicators include:

[0073] Average frame rate (FPS) fluctuation: Sample the actual rendering frame rate over the past second (using Time.deltaTime to roll the average). If the frame rate is lower than the set threshold (e.g., 60 FPS) for more than 5 consecutive frames, it is considered to be an overload trend.

[0074] GPU Frame Time: The time it takes for the GPU to process each frame of an image is collected through Unity's FrameTimingManager or the platform's underlying interface; if it exceeds the set limit (e.g., 16.6ms, approximately 60 FPS) continuously, it is marked as increased GPU stress.

[0075] Frame Drop Ratio: The percentage of frames that are actively dropped or skipped per unit of time. A percentage exceeding 3% is considered to negatively impact user experience.

[0076] Power consumption and temperature profiles (optional): On some high-end devices, additional monitoring of CPU / GPU temperature profiles can help predict performance degradation risks in advance. Tile-Based Rendering optimization: Enable tile-based batch rendering on mobile GPU architectures such as Mali and Adreno to reduce bandwidth bottlenecks.

[0077] If any of the above indicators exceeds the set threshold continuously (for a duration of 1–2 seconds), the dynamic resolution adjustment strategy will be triggered.

[0078] The specific resolution dynamic adjustment strategy includes:

[0079] The system supports a multi-resolution switching mechanism, dynamically adjusting the input image resolution according to different devices and performance conditions (simultaneous downsampling at the acquisition end and the shader processing end):

[0080]

[0081] Dynamic resolution adjustment; if the frame rate is lower than the set threshold for more than 5 consecutive frames or the GPU single-frame rendering time is greater than 16.6ms, the input image resolution will be downgraded and bilinear interpolation will be enabled to compensate for image quality loss.

[0082] Shader optimization: replace branch instructions with branchless functions to reduce GPU computational overhead; use mediump precision for variables in non-critical computations to reduce floating-point computation costs; merge camera images and virtual layers into the same Texture2DArray and sample them in the Shader using array indexing to avoid frequent texture switching and multi-channel sampling overhead.

[0083] The resolution switching must meet the following conditions:

[0084] After each gear shift, maintain stable operation in the current gear for ≥1.5 seconds before upshifting / downshifting is allowed.

[0085] Resolution adjustment is achieved by scaling the RenderTexture and using a Mipmap / bilinear interpolation strategy to avoid noticeable image jumps.

[0086] The impact of resolution reduction on image processing and corresponding countermeasures are as follows:

[0087] Regarding the mirrored image section:

[0088] Impact: Loss of detail or blurred edges, affecting visual quality. No impact on UI layers, as they retain the original high-resolution rendering path.

[0089] Countermeasures: Employ bilinear interpolation reconstruction algorithm (bilinear sampling) and enable edge enhancement at the edges.

[0090] Regarding the feature points:

[0091] Impact: Feature point (such as SIFT, ORB) matching has a lower recognition rate at low resolution, especially for detail key points.

[0092] Countermeasures: When reducing resolution, the system automatically switches to multi-scale feature extraction of image pyramids to ensure that the features of key structural regions are not compressed. In areas with large structural changes or occlusion, IMU + depth map prediction is enabled to assist in correction.

[0093] Preferably, in step S42, the resolution downgrading is simultaneously applied to both the image acquisition end and the shader processing end, and the downgrading switch satisfies the following conditions:

[0094] After each gear shift, maintain stable operation at the current gear position for ≥1.5 seconds;

[0095] RenderTexture scaling, combined with a Mipmap strategy, is used to avoid screen jumps.

[0096] Furthermore, to ensure consistency between user actions and virtual feedback during selfies, step S5 delay control is also included, specifically:

[0097] Step S51: Timestamp Synchronization Method: Mark each frame of camera image and virtual rendering frame with a unified timestamp; ensure that GPU rendering submission and final compositing are synchronized at the millisecond level;

[0098] Step S52: Insert keyframes forward: When camera image delay or GPU rendering fluctuations cause the virtual layer to be ready but waiting for image frames, predictive virtual keyframes are actively inserted to fill the empty window frames; the inserted frame is predicted and calculated from the current virtual object state and IMU (gyroscope + accelerometer) data; the position and orientation of the virtual object at the next moment are estimated using short-time linear interpolation or Kalman filtering to generate a composite frame for GPU submission;

[0099] Step S53: Delay the output frame: When the virtual layer or UI layer fails to render in time, the system will briefly retain the main image frame in the frame buffer (using FrameBuffer Object) and delay the output of the current composite frame.

[0100] Furthermore, it also includes a misalignment correction mechanism in step S6, which activates the following misalignment correction method when there is a displacement deviation between the camera image and the virtual object:

[0101] Step S61: Reverse map the flipped camera image of the current frame to the original unflipped direction; ensure that the feature point extraction direction is consistent with the virtual layer.

[0102] Step S62: Extract local key points of the restored camera image and virtual layer, and select an appropriate matching metric (e.g., Hamming distance for binary descriptors and Euclidean distance for floating-point descriptors) according to the type of descriptor used to perform preliminary feature matching;

[0103] Step S63: Filter out erroneous matching points using the Random Sample Consensus Algorithm (RANSAC);

[0104] Step S64: Calculate the local affine transformation matrix based on the matching point pairs, and apply the matrix to the rendering coordinate system of the virtual layer to achieve fine alignment and position compensation of the camera image. This matrix only works on relative transformations to avoid global jumps.

[0105] Performance testing was conducted on the above system:

[0106] Test Environment: Hardware Platform: Qualcomm Snapdragon 8 Gen2 mobile device. Software Environment: Unity ARFoundation + OpenGL ES post-processing framework. Camera Frame Rate: 60FPS. Test Scenario: Real user performs 5 minutes of AR selfie interaction, including: rapid gesture movement; face rotation, forward and backward translation; click interaction and virtual feedback detection. Statistical Period and Rules: Sampling is performed continuously for 5 minutes (N = total frames), approximately 18,000 frames; erroneous frames are automatically detected through log recording + frame synchronization log comparison + image frame feature analysis; the system marks the trigger of "erroneous frames". Frame Error Rate Calculation Formula: Number of erroneous frames / Total frames * 100%.

[0107] Actual test results:

[0108] Single-frame flip processing latency < 8ms;

[0109] UI composition latency <12ms;

[0110] Total action response latency < 20ms;

[0111] Click interaction error < 2 pixels;

[0112] The frame error rate is less than 0.5%, ensuring high synchronization stability.

[0113] Among them, the frame error rate refers to any of the following types of frame-level anomalies occurring during the AR mirror selfie image synthesis process, causing the current frame to fail to meet the requirements of visual consistency and interactive usability in the user's display, specifically including:

[0114] Layer compositing error frames: Virtual layers and camera images are not aligned at the same timestamp; the screen displays only physical images or only virtual content, UI layers are missing, etc.; the frame content shows discontinuities or some layers are missing rendering.

[0115] Synchronization failure frame: The timestamp error of each layer exceeds the allowable range (Δt > ±20ms); causing the virtual object to be misaligned with the user's actions (e.g., virtual gestures are later than real actions); the degree of misalignment can be automatically detected by comparing user action landmarks (e.g., face and hand positions).

[0116] Teared frames: Image content in the same frame comes from multiple timestamps.

[0117] In Example 2, after the AR device is activated, the system sequentially starts the camera image acquisition module and the virtual scene rendering engine. Combined with the shader image post-processing module and the rendering channel management mechanism, it completes the real-time mirror selfie image generation process. Details are as follows:

[0118] Step S1: Image Acquisition and Virtual / Real Initialization:

[0119] Once the system is powered on, the camera captures the user's first-person perspective of the physical scene in real time, while simultaneously loading virtual content (such as virtual characters, objects, and tags) and constructing the corresponding 3D rendered scene. All content is then synchronously sent to the image post-processing module.

[0120] Step S2: Image Mirroring Processing

[0121] The mirroring module is accelerated by GPU shaders, performing a unified horizontal flipping process on both the physical image and the virtual layer:

[0122] The image is flipped using texture coordinate mapping (u' = 1.0 - u);

[0123] In actual system testing, the frame rate dropped from 80FPS to 75FPS after the flip, but the average power consumption of the GPU only increased by 4.7%, maintaining smooth system operation.

[0124] Shader processing latency is controlled to <8ms, ensuring a smooth visual experience.

[0125] Step S3: Independent rendering of the UI channel:

[0126] The UI layer is drawn in its original orientation in a separate rendering pass and does not participate in the flipping operation. The system uses alpha blending and Z-buffer priority configuration to overlay the UI layer onto the flipped image in a positive orientation.

[0127] Ensure that the orientation of UI buttons, text, icons, and other interactive elements remains unchanged;

[0128] Ensure that the user's click area is completely consistent with the logical coordinates of the control;

[0129] Actual click deviation is less than 2px, which meets the requirements for precise operation.

[0130] Step S4: Synchronization and Interactive Response Processing:

[0131] During the selfie process, the system automatically synchronizes virtual and real coordinates and aligns with visual perception, including:

[0132] Camera images and virtual layers share timestamps to ensure frame-level synchronization;

[0133] Introducing IMU (gyroscope + accelerometer) data to predict and compensate for user actions, reducing the perceived screen lag;

[0134] Combined with dynamic buffer frame management, the measured user operation and feedback latency error is less than 0.1 seconds, ensuring a WYSIWYG interactive experience.

[0135] Step S5: Content Enhancement and Personalized Creation

[0136] Users can enable enhancement features, including filters, beauty effects, dynamic stickers, and other content customization modules. They can also access auxiliary functions such as "check-in templates," "pose prompts," and "action guidance" to improve shooting efficiency and artistic expression.

[0137] Step S6: Content Generation and Storage

[0138] After a user clicks to check in, the system generates a video / image segment or frame that matches the mirrored perspective, and supports the following storage and output methods:

[0139] It supports local storage and also supports synchronization to the cloud;

[0140] All data is protected by AES encryption.

[0141] The output content can be shared to social media platforms (such as WeChat, Xiaohongshu, Douyin, Instagram, etc.) with one click.

[0142] Step S7: Evaluation of Practical Application Effect

[0143] This invention has been deployed and used in multiple scenarios, and significant performance improvements and user experience optimizations have been verified:

[0144] Educational Scenario: In the AR remote dance teaching project, after enabling the mirror function, students' accuracy in imitating movements increased by 28%, the number of misalignment corrections decreased by 42%, and the overall learning efficiency increased by more than 30%.

[0145] Social dissemination: According to usage statistics, after enabling the mirror selfie function, the forwarding rate and likes of short video content increased by 46% and 38% respectively, significantly enhancing users' willingness to share and the dissemination of content;

[0146] Consistency feedback test: In 100 user interaction tests conducted in a laboratory environment, the system achieved a consistency of over 97% between action recognition and virtual screen matching.

[0147] Example 3: The present invention also discloses a mirror display image flipping processing system for implementing the above-described mirror display image flipping processing method for AR devices, comprising:

[0148] Layered mirroring module: configured to synchronously flip physical / virtual layers using Texture2DArray, employing a depth-to-threshold blending algorithm;

[0149] UI Channel Isolation Module: Configured to render the positive UI layer independently and composite it using Alpha;

[0150] Dynamic optimization engine: Configured to monitor GPU load in real time and trigger resolution / shader optimization strategies;

[0151] Delay control unit: configured to maintain an operation response delay of <20ms through timestamp synchronization, keyframe prediction and misalignment correction.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for image flipping processing in mirrored display for AR devices, characterized in that, Includes the following steps: Step S1: Layered mirror flip; The GPU shader performs a uniform horizontal flipping process on the physical camera input image and the virtual rendering layer; Step S2: Independent rendering of the UI channel; In the UI channel, which is independent of the main channel, draw the UI layer according to the original spatial coordinates; Step S3: Interactive coordinate calibration; The position coordinates of the user operation event are reverse-mapped using a coordinate reverse mapping algorithm to match the logical position of the unflipped UI control; Step S4: Dynamic performance optimization; real-time monitoring of system load indicators, and dynamic adjustment of image processing parameters based on performance thresholds to maintain stable frame rate.

2. The image flipping processing method for mirrored display of AR devices according to claim 1, characterized in that, Step S1 specifically includes: Step S11: Use a fragment shader to perform a mapping transformation on the input texture coordinates: map the original horizontal coordinate u to u´=1.0-u; Step S12: Enable depth testing and retain depth writing (ZWrite On) in the main image channel to ensure that the camera image and the virtual layer are correctly occluded and rendered according to the actual depth relationship; avoid mirror flipping to destroy the original occlusion order; Step S13: The camera image and the virtual layer share the same shader, and the layer type is distinguished by passing in different materials and textures in the shader.

3. The image flipping processing method for mirrored display of AR devices according to claim 1, characterized in that, Step S2 specifically includes: Step S21: A multi-channel rendering pipeline architecture is adopted. The main image channel is responsible for rendering the flipped camera image and virtual layer; the UI layer is rendered using Unity's Screen Space - Overlay mode, which does not participate in depth testing and writing by default, ensuring that the rendering order is always above the image channel. Step S22: Overlay the UI layer onto the flipped main image channel output screen using Alpha blending technology.

4. The image flipping processing method for mirrored display of AR devices according to claim 1, characterized in that, The coordinate reverse mapping algorithm in step S3 specifically includes: Step S31: Convert the screen click coordinates (x, y) to coordinates (Wx, y) in the mirror coordinate system, where W is the screen width; Step S32: Use the transformed coordinates to trigger the original interaction logic of the UI control.

5. The image flipping processing method for mirrored display of an AR device according to claim 1, characterized in that, The dynamic performance optimization in step S4 specifically includes: Step S41: Performance monitoring metrics; real-time collection of at least one of frame rate, GPU single-frame rendering time, and frame drop rate; Step S42: Dynamic resolution adjustment; if the frame rate is lower than the set threshold for more than 5 consecutive frames or the GPU single frame rendering time is greater than 16.6ms, the input image resolution will be downgraded and bilinear interpolation will be enabled to compensate for image quality loss. Step S43: Shader optimization; replace branch instructions with branchless functions to reduce GPU computation overhead; use mediump precision for variables in non-critical computation parts to reduce floating-point computation costs; merge camera images and virtual layers and store them in the same Texture2DArray, and sample them in the Shader through array index to avoid frequent texture switching and multi-channel sampling overhead.

6. The image flipping processing method for mirrored display of an AR device according to claim 5, characterized in that, In step S42, the resolution downgrading is simultaneously applied to both the image acquisition end and the shader processing end, and the downgrading switch meets the following conditions: After each gear shift, maintain stable operation at the current gear position for ≥1.5 seconds; RenderTexture scaling, combined with a Mipmap strategy, is used to avoid screen jumps.

7. The image flipping processing method for mirrored display of an AR device according to claim 1, characterized in that, It also includes step S5, which involves delay control, specifically including: Step S51: Mark a unified timestamp for the camera image frame and the virtual rendering frame; Step S52: When the virtual layer rendering is delayed, virtual keyframes are generated by predicting using Kalman filtering based on the gyroscope and acceleration data from the IMU sensor. Step S53: When the camera image is delayed, the rendered virtual frame is temporarily stored in the frame buffer object and awaits synchronous compositing.

8. The image flipping processing method for mirrored display of an AR device according to claim 1, characterized in that, It also includes the misalignment correction mechanism in step S6, which specifically includes: Step S61: Reverse map the flipped camera image back to the original orientation; Step S62: Extract local key points from the camera image and the virtual layer, and select an appropriate matching metric for preliminary feature matching based on the descriptor type used; Step S63: Filter out erroneous matching points using the random sampling consensus algorithm; Step S64: Calculate the local affine transformation matrix based on the matching point pairs, and apply the matrix to the rendering coordinate system of the virtual layer to achieve fine alignment and position compensation of the camera image.

9. The image flipping processing method for mirrored display of an AR device according to claim 8, characterized in that, The misalignment correction mechanism in step S6 is triggered only when the feature point matching confidence is greater than 85%, and the scope of the affine transformation matrix is ​​limited to a local region.

10. A mirror display image flipping processing system for implementing the mirror display image flipping processing method for AR devices according to any one of claims 1-9, characterized in that, include: Layered mirroring module: configured to perform synchronous horizontal flipping and depth management of virtual and real layers; UI Channel Isolation Module: Configured to render the positive UI layer independently and composite it using Alpha; Dynamic optimization engine: Configured to monitor GPU load in real time and trigger resolution / shader optimization strategies; Delay control unit: configured to maintain an operation response delay of <20ms through timestamp synchronization, keyframe prediction and misalignment correction.

Images