Holographic wall display method based on ai target perception and head-mounted device positioning

Abstract

The application discloses a holographic wall display method based on AI target perception and head-mounted device positioning, belongs to the field of immersive visualization technology, and solves the problems of fixed view angle, low intelligent viewfinder, picture jitter and inconsistent view angle space with head-mounted devices of the existing public display system. The method acquires pose information of the head-mounted device relative to the world coordinate system in real time, models and constructs a virtual camera in combination with human eye position parameters, and renders a main view angle image; target discovery and saliency detection are synchronously completed, an optimal picture is solved based on detection information through a dynamic viewfinder optimization process, and the picture is output to the holographic wall for real-time display after geometric mapping of the holographic wall coordinate system, and the steps are continuously and circularly executed. The application realizes matching of the view angle space of the head-mounted device and the holographic wall, improves the intelligence and stability of public display, and is suitable for virtual-real and augmented reality human-computer interaction scenes.

Description

Technical Field

[0001] This invention relates to the fields of virtual reality, augmented reality, and immersive visualization technology, and specifically to a holographic wall display method based on AI target perception and head-mounted device positioning. Background Technology

[0002] With the development of virtual reality, augmented reality, and immersive visualization technologies, human-computer interaction and multi-terminal collaborative display based on head-mounted display devices have gradually become important research directions. In existing systems, head-mounted devices typically acquire their own pose information through external tracking systems or built-in sensors, and use this information to render the corresponding viewpoint. However, these viewpoint results are mostly used only for local display on the headset, lacking a spatially consistent mapping mechanism with large-size public display devices (such as holographic walls and immersive screens). When it is necessary to synchronously display the user's viewpoint on a large screen, existing solutions often use fixed viewpoints, manually specified framing areas, or simple proportional cropping methods, which are difficult to accurately reflect the user's actual observation direction and cannot adapt to the dynamic changes caused by the user's head movements.

[0003] Meanwhile, existing public display systems typically lack target-level and pixel-level intelligent analysis during image generation, and lack the ability to automatically determine the focus of the image. This results in the display content potentially including a large amount of irrelevant background, and key operational areas or interactive objects not being highlighted. Furthermore, in dynamic scenes, if the framing area is adjusted only based on the instantaneous target position, frequent jumps can easily occur, causing image jitter and visual discomfort. Although some studies have attempted to introduce object detection or saliency analysis methods, most are only used for offline processing or single visual tasks, and have not yet formed a complete technical chain that integrates with real-time helmet pose, virtual camera models, and geometric mapping at the display end.

[0004] Therefore, there is an urgent need for a technical solution that can model the user's perspective as a virtual camera based on the real-time pose of the helmet, and then combine artificial intelligence target discovery and saliency analysis to automatically generate a stable and reasonable first-person perspective view for subsequent consistent display on a large screen, so as to improve the intelligence and spatial consistency of multi-terminal collaborative display. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a holographic wall display method based on AI target perception and head-mounted device positioning. This method solves the problems of fixed viewing angles, low framing intelligence, image jitter, and inconsistencies between the viewing angle of the head-mounted device and the viewing angle of the holographic wall in public display systems. It achieves dynamic matching and stable intelligent presentation between the viewing angle of the head-mounted device and the holographic wall display.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0007] A holographic wall display method based on AI target perception and head-mounted device positioning includes the following steps:

[0008] S1. Real-time acquisition of the pose information of the head-mounted device relative to the world coordinate system, wherein the pose information includes the spatial position and orientation of the head-mounted device;

[0009] S2. Based on the pose information and pre-calibrated human eye position parameters, perform main view modeling, determine the user's main viewpoint and gaze direction, construct an equivalent virtual camera based on the main viewpoint and gaze direction, and render and generate the main view image.

[0010] S3. Simultaneously perform target discovery and saliency detection processing on the main view image to obtain target-level detection information and pixel-level saliency information respectively;

[0011] S4. Based on the target-level detection information and pixel-level saliency information, a dynamic framing optimization process is constructed to solve for the optimal image corresponding to the optimal framing area;

[0012] S5. The optimal image is geometrically mapped through the holographic wall coordinate system and then output to the holographic wall for real-time display. Steps S1 to S5 are executed continuously and cyclically over time.

[0013] Furthermore, in step S1, the precise pose information of each head-mounted device relative to the world coordinate system at the corresponding moment is obtained through rigid body transformation. The rigid body transformation method is used to establish the spatial correspondence between the head-mounted device's own coordinate system and the world coordinate system.

[0014] Furthermore, in step S2, the human eye position parameters are the fixed positions of the left and right eyes pre-calibrated in the coordinate system of the head-mounted device itself. The fixed positions of the left and right eyes are mapped to the world coordinate system through homogeneous transformation, and the midpoint of the two eyes is taken as the user's main viewpoint.

[0015] Furthermore, in step S2, the forward unit vector of the head-mounted device's own coordinate system is transformed to the world coordinate system through spatial mapping to obtain the user's line of sight. Combined with the user's main viewpoint, spatial modeling of the user's observation perspective is completed. The parameters of the virtual camera include an extrinsic parameter matrix for associating spatial positions and an intrinsic parameter matrix for defining imaging parameters.

[0016] Furthermore, in step S3, the target discovery processing uses the YOLOv8-L target detection model. The main view image is uniformly scaled and filled to a resolution of 640×640 before being input into the target detection model. The target confidence threshold is set to 0.5. The target-level detection information includes the target bounding box, target category label, and target confidence score.

[0017] Furthermore, in step S3, the saliency detection processing uses the U²-Net model, takes a 640×640 resolution first-view image as input, outputs pixel-level saliency information, and divides the image region into obvious background, secondary structure or transition region, and highly salient target region based on the pixel-level saliency information.

[0018] Further, in step S4, the construction of the dynamic framing optimization process specifically includes: defining an adjustable rectangular framing window on the main view image, characterized by center coordinates, width parameters, and height parameters; constructing a framing quality function, which comprehensively considers the coverage of the target bounding box in the framing window, the offset between the center of the target bounding box and the center of the framing window, the saliency distribution of pixels within the framing window, and the temporal continuity constraint of the framing window.

[0019] Furthermore, in step S4, obtaining the optimal framing area means selecting the framing window that maximizes the framing quality function and using it as the optimal framing area at the current moment.

[0020] Furthermore, in step S5, the holographic wall coordinate system includes the holographic wall planar coordinate system and the projection unit coordinate system. Through the collaborative geometric mapping of the two coordinate systems, the optimal image is accurately displayed in the holographic wall multi-projection splicing module.

[0021] Furthermore, when there are multiple head-mounted devices in the system, the system determines the set of head-mounted devices and assigns the main viewing angle order of the holographic wall display to the head-mounted devices in the set; when there is only one head-mounted device in the system, the viewing angle of that head-mounted device is directly used as the main viewing angle of the holographic wall display.

[0022] The beneficial effects of this invention are as follows:

[0023] 1. This invention achieves accurate modeling of the user's observation perspective by acquiring the pose information of the head-mounted device in real time and constructing a virtual camera. It establishes a spatial correspondence between the head-mounted device and the holographic wall, effectively avoiding the problems of perspective shift and spatial inconsistency, and improving the consistency between the public display screen and the user's actual observation.

[0024] 2. This invention integrates YOLOv8-L object detection and U²-Net saliency detection, achieving joint intelligent analysis at the object level and pixel level. It can automatically identify key interactive areas and visual focal points, reduce irrelevant background display, and improve information transmission efficiency.

[0025] 3. The dynamic framing optimization process constructed in this invention comprehensively considers target coverage, salience distribution and time continuity constraints, effectively suppresses drastic changes in the framing window, avoids visual discomfort caused by image shaking, and achieves stable display in dynamic scenes.

[0026] 4. This invention integrates head-mounted device pose acquisition, virtual camera modeling, AI visual analysis, dynamic framing optimization, and holographic wall geometric mapping into a complete technical chain, realizing technical synergy among modules and significantly improving the intelligence level and multi-terminal collaborative efficiency of the public display system. Attached Figure Description

[0027] Figure 1 This is a flowchart of the holographic wall display method based on AI target perception and head-mounted device positioning according to the present invention.

[0028] Figure 2 This is an interactive diagram of the holographic wall display system based on AI target perception and head-mounted device positioning according to the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0031] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0032] Furthermore, for clarity and brevity, descriptions of well-known structures, functions, and configurations may have been omitted. Those skilled in the art will recognize that various changes and modifications can be made to the examples described herein without departing from the spirit and scope of this disclosure.

[0033] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0034] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0035] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0036] like Figure 1-2 As shown, the holographic wall display method based on AI target perception and head-mounted device positioning includes the following steps:

[0037] S1. Real-time acquisition of the pose information of the head-mounted device relative to the world coordinate system, wherein the pose information includes the spatial position and orientation of the head-mounted device;

[0038] S2. Based on the pose information and pre-calibrated human eye position parameters, perform main view modeling, determine the user's main viewpoint and gaze direction, construct an equivalent virtual camera based on the main viewpoint and gaze direction, and render and generate the main view image.

[0039] S3. Simultaneously perform target discovery and saliency detection processing on the main view image to obtain target-level detection information and pixel-level saliency information respectively;

[0040] S4. Based on the target-level detection information and pixel-level saliency information, a dynamic framing optimization process is constructed to solve for the optimal image corresponding to the optimal framing area;

[0041] S5. The optimal image is geometrically mapped through the holographic wall coordinate system and then output to the holographic wall for real-time display. Steps S1 to S5 are executed continuously and cyclically over time.

[0042] This invention discloses a holographic wall display method based on AI target perception and head-mounted device localization, belonging to the field of immersive visualization technology. It addresses the technical problems of fixed viewpoints, low framing intelligence, image jitter, and inconsistencies between the viewpoint space and that of the head-mounted device in existing public display systems. The method first acquires the spatial position and orientation information of the head-mounted device relative to the world coordinate system in real time. Then, it combines this with pre-calibrated human eye position parameters to complete the main viewpoint modeling, determine the user's main viewpoint and gaze direction, and construct an equivalent virtual camera to render and generate the main viewpoint image. Simultaneously, target discovery and saliency detection are performed on this image, acquiring target-level detection information and pixel-level saliency information. Based on these two types of information, a dynamic framing optimization process is constructed, and the optimal image corresponding to the optimal framing area is solved. Finally, the optimal image is geometrically mapped to the holographic wall coordinate system and output to the holographic wall for real-time display. The above steps are continuously and cyclically executed over time. This invention achieves precise matching of the viewpoint space between the head-mounted device and the holographic wall, effectively improving the intelligence and dynamic stability of public displays, and is applicable to human-computer interaction scenarios in virtual reality and augmented reality.

[0043] Furthermore, in step S1, the precise pose information of each head-mounted device relative to the world coordinate system at the corresponding moment is obtained through rigid body transformation. The rigid body transformation method is used to establish the spatial correspondence between the head-mounted device's own coordinate system and the world coordinate system.

[0044] Furthermore, in step S2, the human eye position parameters are the fixed positions of the left and right eyes pre-calibrated in the coordinate system of the head-mounted device itself. The fixed positions of the left and right eyes are mapped to the world coordinate system through homogeneous transformation, and the midpoint of the two eyes is taken as the user's main viewpoint.

[0045] Furthermore, in step S2, the forward unit vector of the head-mounted device's own coordinate system is transformed to the world coordinate system through spatial mapping to obtain the user's line of sight. Combined with the user's main viewpoint, spatial modeling of the user's observation perspective is completed. The parameters of the virtual camera include an extrinsic parameter matrix for associating spatial positions and an intrinsic parameter matrix for defining imaging parameters.

[0046] Furthermore, in step S3, the target discovery processing uses the YOLOv8-L target detection model. The main view image is uniformly scaled and filled to a resolution of 640×640 before being input into the target detection model. The target confidence threshold is set to 0.5. The target-level detection information includes the target bounding box, target category label, and target confidence score.

[0047] Furthermore, in step S3, the saliency detection processing uses the U²-Net model, takes a 640×640 resolution first-view image as input, outputs pixel-level saliency information, and divides the image region into obvious background, secondary structure or transition region, and highly salient target region based on the pixel-level saliency information.

[0048] Further, in step S4, the construction of the dynamic framing optimization process specifically includes: defining an adjustable rectangular framing window on the main view image, characterized by center coordinates, width parameters, and height parameters; constructing a framing quality function, which comprehensively considers the coverage of the target bounding box in the framing window, the offset between the center of the target bounding box and the center of the framing window, the saliency distribution of pixels within the framing window, and the temporal continuity constraint of the framing window.

[0049] Furthermore, in step S4, obtaining the optimal framing area means selecting the framing window that maximizes the framing quality function and using it as the optimal framing area at the current moment.

[0050] Furthermore, in step S5, the holographic wall coordinate system includes the holographic wall planar coordinate system and the projection unit coordinate system. Through the collaborative geometric mapping of the two coordinate systems, the optimal image is accurately displayed in the holographic wall multi-projection splicing module.

[0051] Furthermore, when there are multiple head-mounted devices in the system, the system determines the set of head-mounted devices and assigns the main viewing angle order of the holographic wall display to the head-mounted devices in the set; when there is only one head-mounted device in the system, the viewing angle of that head-mounted device is directly used as the main viewing angle of the holographic wall display.

[0052] The present invention will now be described in further detail with reference to specific embodiments.

[0053] The holographic wall display method based on AI target perception and head-mounted device positioning of the present invention is applicable to human-computer interaction and multi-terminal collaborative display scenarios in virtual reality and augmented reality. Its core is to achieve dynamic matching and intelligent and stable presentation of the real-time view of the head-mounted device with the holographic wall display. The specific implementation steps are as follows:

[0054] Step S1: Real-time pose acquisition of the head-mounted device

[0055] After system startup, the pose information of each head-mounted device relative to the world coordinate system at time t is acquired in real time through an external tracking system or built-in sensors. The pose information includes the spatial position and orientation of the head-mounted device. A rigid body transformation method is used to establish the spatial correspondence between the head-mounted device's own coordinate system and the world coordinate system. The rigid body transformation matrix is:

[0056]

[0057] Where R is a 3×3 orthogonal rotation matrix describing the orientation relationship of the coordinate system, p is a 3×1 translation vector representing the translation of the origin, and 0 T = [0,0,0] represents homogeneous coordinates with zero-filled terms.

[0058] The pose matrix of the i-th head-mounted device at time t is:

[0059]

[0060] Where R i (t) represents a 3×3 orthogonal rotation matrix formed by the orientation of the head-mounted device, p i (t) is a 3×1 head-mounted device position vector, which serves as the basic input for subsequent view modeling and image generation.

[0061] Step S2: First-person perspective modeling, virtual camera construction, and image rendering

[0062] The main viewpoint is modeled based on pose information and pre-calibrated human eye position parameters, wherein the human eye position parameters are the pre-calibrated fixed positions of the left and right eyes in the coordinate system of the head-mounted device itself. By using a homogeneous transformation, the fixed positions of the left and right eyes are mapped to the world coordinate system, thus obtaining the position of the left eye in the world coordinate system. and the right eye position :

[0063]

[0064] The midpoint between the two eyes is taken as the user's primary viewpoint, and the calculation formula is as follows:

[0065]

[0066] Define the forward unit vector of the head-mounted device's own coordinate system as z0 = [0, 0, 1]. T Through rotation matrix R i (t) is mapped to the world coordinate system to obtain the user's gaze direction vector: Spatial modeling of the user's observation perspective is completed by combining the main viewpoint.

[0067] In this invention, the viewpoint of the head-mounted device is equivalent to a virtual camera; an equivalent virtual camera is constructed based on the main viewpoint and the direction of the gaze, and the extrinsic and intrinsic parameter matrices of the virtual camera are configured, wherein the extrinsic parameter matrix is:

[0068]

[0069] The intrinsic parameter matrix is:

[0070]

[0071] f x f y c is the equivalent focal length in the x / y directions. x c y Principal point coordinates refer to the position of the intersection of the camera's optical axis and the imaging plane in the image coordinate system;

[0072] The main viewpoint image is rendered in real-time using the GPU based on virtual camera parameters. The rendering relationship is as follows:

[0073]

[0074] Generation and User Observation Figure 1 A first-person perspective image.

[0075] Step S3: Simultaneous processing of target discovery and saliency detection

[0076] The purpose of this step is to automatically identify the most important visual target area in the main viewpoint image determined by the head-mounted device, and generate the optimal viewfinder window while ensuring image stability, for subsequent holographic wall display.

[0077] Let the original image corresponding to the main viewpoint at time t be:

[0078]

[0079] Simultaneously perform target discovery and saliency detection processing on the rendered main-view image I(t). First, uniformly scale and fill the main-view image to a resolution of 640×640 to obtain I(t)∈R. 640×640×3 Then, two types of tests are performed separately:

[0080] 1. Target Detection Processing: The standardized first-view image is input into the YOLOv8-L target detection model. The target confidence threshold is set to 0.5. The model automatically detects targets such as people, interactive objects, and key equipment in the scene, and outputs target-level detection information, including the bounding box B of the m-th target. m =(x m ,y m ,w m ,h m Category label C m (e.g., "people", "interactive objects", "key equipment") and target confidence s m ∈[0,1], where x m y m w represents the coordinates of the target in the image. m h m Let the dimensions of the target be its length and width in the image; the output set is:

[0081]

[0082] 2. Significance detection processing:

[0083] Using a pre-trained U²-Net model:

[0084] The input is an image I(t)∈R 640×640×3 The output is a significance plot: s sal (p,t)∈[0,1]

[0085] Where p is any pixel position:

[0086]

[0087] Its output is defined as follows:

[0088]

[0089] Step S4: Dynamic framing optimization and optimal image solution

[0090] Define a rectangular viewfinder at time t on the main viewpoint image I(t):

[0091]

[0092] in:

[0093] Viewfinder center coordinates; W R H R This refers to the width and height of the viewfinder.

[0094] To comprehensively consider target integrity, visual salience, and temporal continuity, this invention constructs the following framing quality function:

[0095]

[0096] in: : The bounding box of the target output by the object detection network (YOLOv8-L);

[0097] Each target includes: target bounding box B m The center point of the target bounding box Confidence level s m ∈[0,1];

[0098] Intersection and Union Terms

[0099] This item measures whether the target is fully covered by the viewfinder:

[0100]

[0101] Center offset penalty term:

[0102]

[0103] This describes the degree to which the subject is centered in the viewfinder:

[0104] : The center point of the target bounding box; : Center coordinates of the viewfinder; the smaller the distance, the closer the target is to the center of the image; This is a custom weight for this item.

[0105] Pixel saliency:

[0106]

[0107] s sal(p,t)∈[0,1]: The pixel-level saliency value output by U²-Net, representing the visual importance of pixel p in the current image; It is a custom weight that determines the salience of a pixel.

[0108] Time continuity constraint:

[0109]

[0110] The center of the viewfinder at the previous moment; : A custom weight for time continuity.

[0111] This feature is used to suppress drastic changes in the viewfinder over time, prevent frequent image jitter, and ensure visual continuity and viewing comfort.

[0112] Optimal framing:

[0113] When the function Q(R) reaches its maximum value, the screen is the optimal screen.

[0114]

[0115] Step S5: Holographic Wall Geometric Mapping and Real-time Display

[0116] The optimal image obtained from the solution is geometrically mapped to the holographic wall coordinate system. The holographic wall coordinate system includes the holographic wall plane coordinate system and the projection unit coordinate system. Through the collaborative geometric mapping of the two coordinate systems, the optimal image based on the world coordinate system / virtual camera coordinate system is accurately converted into the physical display coordinates of the holographic wall, so as to realize the accurate display of the optimal image in the holographic wall multi-projection splicing module. At the same time, steps S1 to S5 are continuously and cyclically executed over time to adapt to changes in user head movement and dynamic interaction scenarios in real time.

[0117] When multiple active head-mounted devices are connected to the system, the set of active head-mounted devices is H(t)={H1,H2...}. The system will maintain a main view index, and the order of the main view displayed on the holographic wall will be specified by the staff or by preset rules. This will clarify the set of currently active head-mounted devices and ensure that the holographic wall always displays the specified user view. If there is only one head-mounted device, then the view of that device will be directly used as the main view displayed on the holographic wall.

[0118] Compared to the existing image generation process of public display systems, this invention makes changes to both the overall technical approach and key processing steps. In existing technologies, public display images are typically generated from a fixed camera perspective, a preset virtual perspective, or a manually specified framing area. There is no direct correlation between the displayed content and the user's real-time observation position and line of sight. Image updates largely rely on static rules or simple triggering conditions, making it difficult to reflect the user's true spatial observation intentions. Under this process, important targets and irrelevant backgrounds often appear simultaneously, lacking automatic identification and highlighting of key operational areas. Furthermore, in dynamic scenes, framing adjustments are usually based on instantaneous target positions or single detection results, easily causing frequent image jumps, affecting display stability and viewing experience.

[0119] In contrast, this invention uses the real-time pose information of the helmet as the starting point for image generation, and models the viewpoint of the head-mounted device as an equivalent virtual camera, establishing a geometric correspondence between the user's spatial observation behavior and the displayed image from the source. Building upon this, the invention further introduces object detection and saliency analysis to jointly perceive key objects and high-importance areas in the main viewpoint image, enabling automatic judgment of key content in the image, thus avoiding reliance solely on manual settings or simple rules for display framing. Simultaneously, this invention constructs a dynamic framing optimization model that integrates target coverage, saliency intensity, and temporal continuity constraints. While ensuring the complete presentation of key targets, it smoothly controls the temporal changes of the framing window, effectively suppressing image jitter and abrupt changes.

[0120] Therefore, this invention not only improves the consistency between the displayed image and the user's observation intention, but also significantly enhances the image's ability to focus on key interactive information and the display stability in dynamic scenes, realizing the transformation of public display systems from passive display to intelligent perception and adaptive display.

[0121] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A holographic wall display method based on AI target perception and head-mounted device positioning, characterized in that, Includes the following steps: S1. Real-time acquisition of the pose information of the head-mounted device relative to the world coordinate system, wherein the pose information includes the spatial position and orientation of the head-mounted device; S2. Based on the pose information and pre-calibrated human eye position parameters, perform main view modeling, determine the user's main viewpoint and gaze direction, construct an equivalent virtual camera based on the main viewpoint and gaze direction, and render and generate the main view image. S3. Simultaneously perform target discovery and saliency detection processing on the main view image to obtain target-level detection information and pixel-level saliency information respectively; S4. Based on the target-level detection information and pixel-level saliency information, a dynamic framing optimization process is constructed to solve for the optimal image corresponding to the optimal framing area; The dynamic framing optimization process specifically includes: defining an adjustable rectangular framing window on the main view image, characterized by center coordinates, width parameters, and height parameters; constructing a framing quality function, which comprehensively considers the coverage of the target bounding box within the framing window, the offset between the center of the target bounding box and the center of the framing window, the saliency distribution of pixels within the framing window, and the temporal continuity constraint of the framing window; the framing quality function is constructed as follows: Where: s m ∈[0,1] represents the confidence level; B m represents the target bounding box; M represents the number of target bounding boxes; Let I be a rectangular viewfinder defined at time t on the main viewpoint image I(t), where I(t) = ... W represents the center coordinates of the viewfinder window. R h R The width and height of the viewfinder; Custom weights; The center point of the target bounding box; The weights for pixel saliency; s sal (p,t)∈[0,1] represents the pixel-level saliency value; Center of the viewfinder window at the previous moment; Weights for custom time continuity; S5. The optimal image is geometrically mapped through the holographic wall coordinate system and then output to the holographic wall for real-time display. Steps S1 to S5 are executed continuously and cyclically over time.

2. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 1, characterized in that, In step S1, the precise pose information of each head-mounted device relative to the world coordinate system at the corresponding moment is obtained through rigid body transformation. The rigid body transformation method is used to establish the spatial correspondence between the head-mounted device's own coordinate system and the world coordinate system.

3. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 1, characterized in that, In step S2, the human eye position parameters are the fixed positions of the left and right eyes pre-calibrated in the coordinate system of the head-mounted device itself. The fixed positions of the left and right eyes are mapped to the world coordinate system through homogeneous transformation, and the midpoint of the two eyes is taken as the user's main viewpoint.

4. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 3, characterized in that, In step S2, the forward unit vector of the head-mounted device's own coordinate system is transformed to the world coordinate system through spatial mapping to obtain the user's line of sight. Combined with the user's main viewpoint, spatial modeling of the user's observation perspective is completed. The parameters of the virtual camera include an extrinsic parameter matrix for associating spatial positions and an intrinsic parameter matrix for defining imaging parameters.

5. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 1, characterized in that, In step S3, the target discovery process uses the YOLOv8-L target detection model. The main view image is uniformly scaled and filled to a resolution of 640×640 before being input into the target detection model. The target confidence threshold is set to 0.

5. The target-level detection information includes the target bounding box, target category label, and target confidence score.

6. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 1, characterized in that, In step S3, the saliency detection processing uses the U²-Net model, taking a 640×640 resolution first-view image as input, outputting pixel-level saliency information, and dividing the image region into obvious background, secondary structure or transition region, and highly salient target region based on the pixel-level saliency information.

7. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 1, characterized in that, In step S4, obtaining the optimal framing area means selecting the framing window that maximizes the framing quality function and using it as the optimal framing area at the current moment.

8. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 1, characterized in that, In step S5, the holographic wall coordinate system includes the holographic wall planar coordinate system and the projection unit coordinate system. Through the collaborative geometric mapping of the two coordinate systems, the optimal image is accurately displayed in the holographic wall multi-projection splicing module.

9. The holographic wall display method based on AI target perception and head-mounted device positioning according to claim 1, characterized in that, When there are multiple head-mounted devices in the system, the system determines the set of head-mounted devices and assigns the main viewing angle order of the holographic wall display to the head-mounted devices in the set; when there is only one head-mounted device in the system, the viewing angle of the head-mounted device is directly used as the main viewing angle of the holographic wall display.

Images