An AR-based intelligent art work display method and system

Abstract

The application is suitable for the field of AR interaction technology, and particularly relates to an art work intelligent display method and system based on AR, which comprises the following steps: positioning the device based on the AR device, and obtaining the visual angle direction of the device; obtaining an exhibition area map, determining whether there is an exhibit in the visual line direction, and interacting with the user when there is an exhibit, and displaying the work based on the user request; calling the video data corresponding to the exhibit, obtaining real-time images through an image acquisition device, and generating predicted images based on the real-time images; performing local compensation on the display picture in the video data based on the predicted images, adjusting the color value of each pixel in the display picture, and outputting the compensated picture. The application collects real-time pictures, compares the real-time pictures with AR pictures, determines the difference between the two, completes dynamic compensation of the AR pictures, increases the difference between the AR pictures and the real-time pictures, significantly improves the definition of the pictures, and improves the user experience.

Description

Technical Field

[0001] This invention belongs to the field of AR interactive technology, and in particular relates to an intelligent display method and system for AR-based artworks. Background Technology

[0002] AR product display is an interactive display method that uses augmented reality technology to overlay virtual 3D product models onto the user's real environment. Users can simply point their smartphones, tablets, or AR glasses at a specific area or scan product logos to see realistic virtual products "placed" in their home, on their desk, or in any real-world scene. It also supports interactive operations such as 360° rotation, zooming, and changing colors or materials.

[0003] Current AR glasses display images directly overlaid on the real-world scene. The real-world scene as a background will interfere with the AR display, easily resulting in unclear images. Increasing the overall brightness can not only cause visual fatigue but also affect the device's battery life. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent display method for AR-based artworks, aiming to solve the problem that the current AR glasses display images directly superimposed on the real-world image, and the real-world image as a background will interfere with the AR display, easily resulting in unclear image display.

[0005] This invention is implemented as follows: an AR-based intelligent display method for artworks, the method comprising:

[0006] The AR device is used for device positioning and to obtain the device's viewing direction. The AR device is equipped with a positioning module and two sets of image acquisition devices, which correspond to the left eye and the right eye respectively. The image acquisition devices are located between the eyes and the display screen.

[0007] Obtain a map of the exhibition area, determine whether there are exhibits in the direction of the line of sight based on the AR device's positioning data and viewing direction, and interact with the user when exhibits are present, displaying artworks based on user requests;

[0008] The video data corresponding to the exhibit is retrieved, real-time images are acquired through an image acquisition device, and a predicted image is generated based on the real-time images.

[0009] The video data is locally compensated based on the predicted image, the color values ​​of each pixel in the displayed image are adjusted, and the compensated image is output for display.

[0010] Preferably, the steps of acquiring the exhibition area map, determining whether exhibits exist in the line of sight based on AR device positioning data and viewing direction, and interacting with the user to display artworks based on user requests when exhibits exist, specifically include:

[0011] The device's location data, collected in real time by the AR device's positioning module, is combined with data from the device's built-in orientation sensor to calculate the user's current viewing direction.

[0012] The device location data and viewing direction vector are compared with the pre-stored exhibition area map in real time to determine whether there is an exhibit model and its corresponding ID in the user's line of sight.

[0013] When an exhibit is detected in the direction of the user's gaze, an interactive prompt icon is generated and overlaid at the corresponding position on the AR display screen. In response to the user's request for the icon, sample data associated with the exhibit ID is retrieved and displayed.

[0014] Preferably, the step of retrieving the video data corresponding to the exhibit, acquiring real-time images through an image acquisition device, and generating a predicted image based on the real-time images includes:

[0015] Based on the identified exhibit ID, retrieve the corresponding panoramic display video from the server as the video data stream to be played;

[0016] Real-time images of the user's current field of vision are simultaneously acquired through image acquisition devices located in front of the left and right eyes;

[0017] Based on real-time image sequences and the historical motion trajectory of AR devices, the system predicts the user's head pose at the next display frame and generates corresponding left-eye and right-eye prediction images accordingly.

[0018] Preferably, the step of performing local compensation on the display image in the video data based on the predicted image, adjusting the color values ​​of each pixel in the display image, outputting the compensated image, and displaying it specifically includes:

[0019] The predicted image is fused with the video data frames of the exhibit to be overlaid, and the analysis results are obtained by analyzing each region in the fused image. The analysis results include the brightness, contrast and color difference at the boundary between the edge of the video screen and the real background.

[0020] Based on the analysis results, pixel-level color values ​​are adjusted for the video areas that need to be highlighted in the merged image, and a compensated image is output.

[0021] The compensated image, after partial compensation, is sent to the left and right eye displays of the AR device for simultaneous display.

[0022] Preferably, during the display process, the display screen and the real-time screen are synchronously acquired by the image acquisition device, a verification screen is output, and the parameters of local compensation are corrected based on the verification screen.

[0023] Another objective of this invention is to provide an AR-based intelligent art display system, the system comprising:

[0024] The device positioning module is used to locate the device based on the AR device and obtain the device's viewing direction. The AR device is equipped with a positioning module and two sets of image acquisition devices, which correspond to the left eye and the right eye respectively. The image acquisition devices are located between the eyes and the display screen.

[0025] The exhibit detection module is used to acquire a map of the exhibition area, determine whether there are exhibits in the line of sight based on the AR device's positioning data and viewing direction, and interact with the user when exhibits are present, displaying works based on user requests.

[0026] The image prediction module is used to retrieve the video data corresponding to the exhibit, acquire real-time images through the image acquisition device, and generate predicted images based on the real-time images.

[0027] The image compensation module is used to perform local compensation on the display image in the video data based on the predicted image, adjust the color values ​​of each pixel in the display image, and output the compensated image for display.

[0028] Preferably, the exhibit detection module includes:

[0029] The viewing angle determination unit is used to obtain the device position data collected in real time by the AR device's positioning module, and combine it with the orientation sensor data built into the device to calculate the user's current viewing angle direction.

[0030] The exhibit recognition unit is used to perform real-time collision detection between the device position data and the viewing direction vector and the pre-stored exhibition area map to determine whether there is an exhibit model and its corresponding ID in the user's line of sight.

[0031] The interactive response unit is used to generate and overlay an interactive prompt icon at the corresponding position on the AR display screen when an exhibit is detected in the direction of the user's gaze. In response to the user's request for the icon, the unit retrieves sample data associated with the exhibit ID for display.

[0032] Preferably, the image prediction module includes:

[0033] The exhibit video retrieval unit is used to retrieve the corresponding panoramic display video from the server as the video data stream to be played based on the identified exhibit ID;

[0034] The real-time image acquisition unit is used to simultaneously acquire real-time images of the user's current field of vision through image acquisition devices located in front of the left and right eyes;

[0035] The dynamic prediction unit is used to predict the user's head pose at the next display frame based on real-time image sequences and the historical motion trajectory of the AR device, and generate corresponding left-eye and right-eye prediction images accordingly.

[0036] Preferably, the image compensation module includes:

[0037] The fusion analysis unit is used to fuse the predicted image with the exhibit video data frames to be superimposed, and analyze each region in the fused image to obtain analysis results, including brightness, contrast and color differences at the boundary between the video frame edge and the real background.

[0038] The pixel compensation unit is used to adjust the pixel-level color values ​​of the video areas that need to be highlighted in the fused image based on the analysis results, and output the compensated image.

[0039] The fusion display unit is used to send the compensated image, after local compensation, to the left and right eye displays of the AR device for synchronous display.

[0040] Preferably, during the display process, the display screen and the real-time screen are synchronously acquired by the image acquisition device, a verification screen is output, and the parameters of local compensation are corrected based on the verification screen.

[0041] This invention provides an intelligent display method for AR-based artworks. By capturing real-time images and comparing them with AR images, the differences between the two are determined, and dynamic compensation is performed on the AR images to increase the difference between the AR and real-time images, significantly improving image clarity and enhancing user experience. Attached Figure Description

[0042] Figure 1 A flowchart illustrating an AR-based intelligent display method for artworks provided in this embodiment of the invention;

[0043] Figure 2 The flowchart illustrates the steps of obtaining an exhibition area map, determining whether exhibits exist in the line of sight based on AR device positioning data and viewing direction, interacting with the user when exhibits exist, and displaying artworks based on user requests, as provided in this embodiment of the invention.

[0044] Figure 3 A flowchart of the steps provided in this embodiment of the invention for retrieving video data corresponding to the exhibit, acquiring real-time images through an image acquisition device, and generating predicted images based on the real-time images;

[0045] Figure 4 This is a flowchart illustrating the steps of performing local compensation on the display screen in video data based on predicted images, adjusting the color values ​​of each pixel in the display screen, outputting the compensated screen, and displaying the image, as provided in an embodiment of the present invention.

[0046] Figure 5 An architecture diagram of an AR-based intelligent art display system provided in this embodiment of the invention;

[0047] Figure 6 An architectural diagram of an exhibit detection module provided in an embodiment of the present invention;

[0048] Figure 7 An architecture diagram of an image prediction module provided in an embodiment of the present invention;

[0049] Figure 8 This is an architectural diagram of a screen compensation module provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0051] like Figure 1 The diagram shows a flowchart of an AR-based intelligent display method for artworks provided by an embodiment of the present invention. The method includes:

[0052] S100, based on the AR device, the device is located and the viewing direction of the device is obtained. The AR device is equipped with a positioning module and two sets of image acquisition devices, which correspond to the left eye and the right eye respectively. The image acquisition devices are located between the eyes and the display screen.

[0053] In this step, device positioning is performed based on the AR device. The AR device has a built-in positioning module, and in indoor scenes, RTK positioning can be used to determine the AR device's specific location indoors. The AR device's internal sensors acquire the AR device's viewing direction. By combining the device's position and viewing direction with a preset map, it can be determined whether there are exhibits in the line of sight. An image acquisition device is set up to acquire real-time images. When displaying AR images, AR images and real-time images are acquired synchronously. When positioning the AR device, a positioning base station is set up indoors. The positioning base station can detect the positioning module in the AR device. By solving the carrier phase difference from different positioning base stations and combining it with the known base station coordinates, the three-dimensional coordinates of the AR device in the indoor global coordinate system are calculated, thus obtaining the AR device's positioning data. When determining the device's viewing direction, the IMU sensor inside the AR device acquires data in real time. Based on the acquired angular velocities of the AR device around the three axes and the linear acceleration of the AR device on the three axes, the IMU sensor obtains the angle change by integrating the angular velocity and the velocity and displacement changes by integrating the acceleration, thereby determining the AR device's viewing direction at various times.

[0054] S200 acquires a map of the exhibition area, determines whether exhibits exist in the direction of the user's gaze based on the AR device's positioning data and viewing direction, and interacts with the user when exhibits exist, displaying artworks based on the user's request.

[0055] In this step, an exhibition area map is obtained, which records the overall layout of the exhibition area, including the placement and size of the exhibits. Based on the AR device's positioning data and viewing direction, the field of view of the AR device in the exhibition area map can be determined. The presence of exhibits within the field of view is determined by the overlap between the field of view and the exhibits. If exhibits are present, a preset AR model is retrieved and displayed on the AR screen, and the AR device interacts with the user. If the user makes a request, the artwork display begins.

[0056] S300 retrieves the video data corresponding to the exhibit, acquires real-time images through the image acquisition device, and generates a predicted image based on the real-time images.

[0057] In this step, the video data corresponding to the exhibit is retrieved. The video data is pre-stored in the AR device or cloud server. The video data is a pre-made video used to display the exhibit, that is, the exhibit is pre-recorded, such as a video of a guide explaining the exhibit. It is called up when needed. During this process, real-time images are acquired through an image acquisition device. The real-time images are the actual images without AR screens. Using a prediction algorithm, the position and viewing direction of the next moment are predicted based on the movement trajectory of the AR device. A predicted image is generated based on the already acquired real-time images. The predicted image is the image that the image acquisition device may capture at the next moment by analyzing the current real-time images.

[0058] The S400 performs local compensation on the displayed image in the video data based on the predicted image, adjusts the color values ​​of each pixel in the displayed image, outputs the compensated image, and displays it.

[0059] In this step, local compensation is performed on the display screen in the video data based on the predicted image to retrieve the AR screen to be displayed. The corresponding predicted image is retrieved according to the playback time of the AR screen, and the color difference between overlapping pixels is calculated to complete the pixel color difference analysis. Based on the analysis results, the corresponding pixels in the AR screen are enhanced, including modifying the pixel color information and increasing the pixel brightness to output a compensated screen. During the display process, the displayed screen and the real-time screen are synchronously acquired through an image acquisition device, and a verification screen is output. Based on the verification screen, the local compensation parameters are corrected. Specifically, the verification screen is extracted, and pixels at the same position in the verification screen are compared with those in the real-time screen. If the pixel difference is less than a set range (e.g., the brightness difference between pixels is less than a preset value), the compensation parameter value is further increased in subsequent compensation processes, such as further increasing brightness or increasing the amount of color information modification. Conversely, if the pixel difference is detected to be too large, the compensation parameter value is decreased.

[0060] like Figure 2 As shown, in a preferred embodiment of the present invention, the steps of obtaining an exhibition area map, determining whether exhibits exist in the line of sight based on AR device positioning data and viewing direction, and interacting with the user to display artworks based on user requests when exhibits exist, specifically include:

[0061] S201: Obtain the device position data collected in real time by the AR device's positioning module, and combine it with the data from the device's built-in orientation sensor to calculate the user's current viewing direction.

[0062] In this step, the device location data collected in real time by the AR device's positioning module is obtained. Based on the device location data, the relative positional relationship between the AR device and the exhibition area can be determined. The AR device is equipped with a sensor that can determine the current orientation of the AR device, which is the viewing direction.

[0063] S202 performs real-time collision detection between the device location data and the viewing direction vector and the pre-stored exhibition area map to determine whether there is an exhibit model and its corresponding ID in the user's line of sight.

[0064] In this step, the device location data and view direction vector are compared with the pre-stored exhibition area map in real time for collision detection. Collision detection determines whether there are exhibits within the view range. If the positions of the two overlap, it is determined that there are exhibits. Based on the positional relationship between the exhibit and the exhibition area, the relevant exhibit information, including the exhibit model and the exhibit ID, can be retrieved.

[0065] S203: When an exhibit is detected in the direction of the line of sight, an interactive prompt icon is generated and superimposed at the corresponding position on the AR display screen. In response to the user's request for the icon, sample data associated with the exhibit ID is retrieved and displayed.

[0066] In this step, when an exhibit is detected in the direction of the user's gaze, an interactive prompt icon is generated and overlaid at the corresponding position on the AR display screen. That is, an icon is displayed in the AR screen seen by the user. At this time, the user can complete the interaction process with the AR device through physical buttons or virtual buttons on the AR device to submit a user request. Based on the user request, the exhibit will be displayed, including a comprehensive display and explanation of the exhibit.

[0067] like Figure 3 As shown, in a preferred embodiment of the present invention, the step of retrieving video data corresponding to the exhibit, acquiring real-time images through an image acquisition device, and generating a predicted image based on the real-time images includes:

[0068] S301 retrieves the corresponding panoramic display video from the server as the video data stream to be played, based on the identified exhibit ID.

[0069] In this step, based on the identified exhibit ID, exhibit data is retrieved, including a panoramic display video of the exhibit. The panoramic display video includes a display of the exhibit from various angles and is equipped with corresponding annotations and explanatory audio to help customers understand the exhibit. The above video, audio and text are integrated into the video data stream, which is a continuous video frame.

[0070] S302, through image acquisition devices located in front of the left and right eyes, synchronously acquires real-time images of the user's current field of vision.

[0071] In this step, the two image acquisition devices located in front of the left and right eyes are controlled to start synchronously in global shutter mode to capture RGB environmental images and obtain real-time images. Timestamps are added to each frame of the image synchronously. The inertial measurement unit built into the AR device continuously samples the data from the three-axis gyroscope and accelerometer and performs time alignment processing. The visual inertial odometry pipeline is started: distortion correction and stereo correction are performed on the binocular images. The inter-frame visual displacement is calculated through feature point extraction and optical flow tracing algorithms. These visual displacement observations are fused with the angular velocity and acceleration measured by the IMU to solve for the optimal pose estimate of the device relative to the initial coordinate system at the current moment, including three-dimensional position and three-dimensional orientation.

[0072] S303, based on real-time image sequences and the historical motion trajectory of AR devices, predicts the user's head pose at the next display frame and generates corresponding left-eye and right-eye prediction images accordingly.

[0073] In this step, based on the optimal pose, and combined with the real-time angular velocity and acceleration provided by the IMU, the pose that the head may reach at the prediction time is calculated based on the preset prediction algorithm. The predicted pose is used as the view projection matrix of the virtual camera. According to the same predicted pose, the real-time acquired real environment image is subjected to inverse geometric transformation to generate a background layer that matches the predicted viewpoint, thus obtaining the left eye prediction image and the right eye prediction image.

[0074] like Figure 4 As shown, in a preferred embodiment of the present invention, the step of performing local compensation on the display image in the video data based on the predicted image, adjusting the color values ​​of each pixel in the display image, outputting the compensated image, and displaying it specifically includes:

[0075] S401, the predicted image is fused with the exhibit video data frame to be overlaid, and the analysis results are obtained by analyzing each region in the fused image. The analysis results include the brightness, contrast and color difference at the boundary between the edge of the video screen and the real background.

[0076] In this step, the predicted image is fused with the exhibit video data frames to be overlaid to generate a preliminary image. The entire image is scanned, and the perceptual difference between the virtual foreground and the real background in each local area is calculated. This is achieved by calculating brightness contrast (such as the ratio of foreground brightness A to background brightness a), color conflict (the Euclidean distance between foreground coordinates (B,C) and background coordinates (b,c)), and edge sharpness. The analysis results are then recorded in the analysis results, and the pixels that need to be adjusted are recorded.

[0077] S402, based on the analysis results, performs pixel-level color value adjustments on the video areas that need to be highlighted in the merged image, and outputs a compensated image.

[0078] In this step, each pixel requiring adjustment is processed individually, and compensation is performed in the CIELAB color space. The process involves separating the target foreground color (A, B, C) and the sampled background color (a, b, c) from the blended image. The foreground brightness A is dynamically adjusted based on the background brightness a to improve readability. If the background is too bright (a > 50), the foreground brightness is increased according to the formula A' = A + k * (a - 50), where k is the dynamically adjusted gain coefficient. The vector difference between the foreground and background on the chromaticity plane is calculated, and the foreground brightness (B, C) is shifted in the opposite direction to the background color (b, c) to enhance differentiation. After brightness adjustment, the foreground brightness (B, C) is adjusted. All adjustments are completed in the LAB space. Subsequently, the compensated (A', B', C') values ​​are converted back to the RGB color space of the display screen to generate the final compensated image.

[0079] S403 will send the compensated image, which has completed local compensation, to the left and right eye displays of the AR device for synchronous display.

[0080] In this step, the compensation image buffer is submitted to the display controller, low-latency correction is achieved through asynchronous time warping, the latest head rotation data is obtained, and the entire prepared compensation image is quickly reprojected to offset the last prediction error, and the image is output to the AR device.

[0081] like Figure 5 As shown in the figure, an AR-based intelligent art display system is provided in an embodiment of the present invention. The system includes:

[0082] The device positioning module 100 is used to locate the device based on the AR device and obtain the viewing direction of the device. The AR device is equipped with a positioning module and two sets of image acquisition devices, which correspond to the left eye and the right eye respectively. The image acquisition devices are located between the eyes and the display screen.

[0083] In this system, the device positioning module 100 performs device positioning based on the AR device. The AR device has a built-in positioning module. In indoor scenes, it can use RTK to locate the AR device and determine its specific location indoors. Based on the sensors inside the AR device, the viewing direction of the AR device is obtained. By combining the device position and viewing direction with a preset map, it can be determined whether there are exhibits in the line of sight. Real-time images are acquired by setting up an image acquisition device. When displaying AR images, AR images and real-time images are acquired synchronously.

[0084] The exhibit detection module 200 is used to acquire a map of the exhibition area, determine whether exhibits exist in the direction of the line of sight based on the AR device's positioning data and viewing direction, and interact with the user when exhibits exist, displaying works based on user requests.

[0085] In this system, the exhibit detection module 200 acquires an exhibition area map, which records the overall layout of the exhibition area, including the placement and size of the exhibits. Based on the AR device's positioning data and viewing direction, the AR device's field of vision in the exhibition area map can be determined. The presence of exhibits within the field of vision is determined by the overlap between the field of vision and the exhibits. If exhibits are present, a preset AR model is retrieved and displayed on the AR screen, and the AR device interacts with the user. If the user initiates a request, the artwork display begins.

[0086] The image prediction module 300 is used to retrieve the video data corresponding to the exhibit, acquire real-time images through the image acquisition device, and generate a predicted image based on the real-time images.

[0087] In this system, the image prediction module 300 retrieves the video data corresponding to the exhibit. The video data is pre-stored in the AR device or cloud server and is called when needed. During this process, real-time images are acquired through the image acquisition device. The real-time images are the actual images without AR screens. Using a prediction algorithm, the position and viewing direction of the next moment are predicted based on the motion trajectory of the AR device. A predicted image is generated based on the acquired real-time images. The predicted image is the image that the image acquisition device may capture at the next moment by analyzing the current real-time images.

[0088] The image compensation module 400 is used to perform local compensation on the display image in the video data based on the predicted image, adjust the color values ​​of each pixel in the display image, and output the compensated image for display.

[0089] In this system, the image compensation module 400 performs local compensation on the display screen in the video data based on the predicted image, retrieves the AR screen to be displayed, retrieves the corresponding predicted image according to the playback time of the AR screen, calculates the color difference between the overlapping pixels of the two, completes the color difference analysis of the pixels, and enhances the corresponding pixels in the AR screen based on the analysis results. The adjustment method includes modifying the color information of the pixels and increasing the color brightness of the pixels to output the compensated screen and display it. During the display process, the image acquisition device synchronously acquires the display screen and the real-time screen, outputs a verification screen, and corrects the local compensation parameters based on the verification screen. Specifically, it extracts the verification screen and compares the pixels at the same position in the verification screen with the pixels in the real-time screen. If the pixel difference between the two is less than a set range, such as the brightness difference between the pixels being less than a preset value, then in the subsequent compensation process, the value of the compensation parameter is further increased, such as further increasing the brightness or increasing the amount of compensation for modifying the color information. Conversely, if the pixel difference is detected to be too large, the value of the compensation parameter is decreased.

[0090] like Figure 6 As shown, in a preferred embodiment of the present invention, the exhibit detection module 200 includes:

[0091] The viewing angle determination unit 201 is used to obtain the device position data collected in real time by the positioning module of the AR device, and calculate the user's current viewing angle direction by combining the data from the device's built-in orientation sensor.

[0092] In this module, the viewing angle determination unit 201 acquires the device position data collected in real time by the positioning module of the AR device. Based on the device position data, the relative positional relationship between the AR device and the exhibition area can be determined. The AR device is equipped with a sensor that can determine the current orientation of the AR device, which is the viewing angle direction.

[0093] The exhibit identification unit 202 is used to perform real-time collision detection between the device location data and the viewing direction vector and the pre-stored exhibition area map to determine whether there is an exhibit model and its corresponding ID in the user's line of sight.

[0094] In this module, the exhibit identification unit 202 performs real-time collision detection between the device location data and the viewing direction vector and the pre-stored exhibition area map. Collision detection determines whether there is an exhibit within the viewing range. If the positions of the two overlap, it is determined that there is an exhibit. Based on the positional relationship between the exhibit and the exhibition area, the relevant exhibit information, including the exhibit model and the exhibit ID, can be retrieved.

[0095] The interactive response unit 203 is used to generate and overlay an interactive prompt icon at the corresponding position on the AR display screen when an exhibit is detected in the direction of the line of sight, respond to the user's request for the icon, and retrieve sample data associated with the exhibit ID for display.

[0096] In this module, when the interactive response unit 203 detects an exhibit in the direction of the user's gaze, it generates and overlays an interactive prompt icon at the corresponding position on the AR display screen. That is, an icon is displayed in the AR screen seen by the user. At this time, the user can complete the interaction process with the AR device through physical buttons or virtual buttons on the AR device to submit a user request. Based on the user request, the exhibit will be displayed, including a comprehensive display and explanation of the exhibit.

[0097] like Figure 7 As shown, in a preferred embodiment of the present invention, the image prediction module 300 includes:

[0098] The exhibit video retrieval unit 301 is used to retrieve the corresponding panoramic display video from the server as the video data stream to be played, based on the identified exhibit ID.

[0099] In this module, the exhibit video retrieval unit 301 retrieves exhibit data based on the identified exhibit ID, including a panoramic display video of the exhibit. The panoramic display video includes a display of the exhibit from various angles and is equipped with corresponding annotations and explanatory audio to help customers understand the exhibit. The above video, audio and text are integrated into the video data stream, which is a continuous video frame.

[0100] The real-time image acquisition unit 302 is used to synchronously acquire real-time images of the user's current field of vision through image acquisition devices located in front of the left and right eyes.

[0101] In this module, the real-time image acquisition unit 302 controls two image acquisition devices located in front of the left and right eyes to start synchronously in global shutter mode, capture RGB environmental images, obtain real-time images, and synchronously timestamp each frame of the image. The inertial measurement unit built into the AR device continuously samples the data from the three-axis gyroscope and accelerometer and performs time alignment processing. The visual inertial odometry pipeline is started: distortion correction and stereo correction are performed on the binocular images, and the inter-frame visual displacement is calculated through feature point extraction and optical flow tracing algorithms. These visual displacement observations are fused with the angular velocity and acceleration measured by the IMU to solve for the optimal pose estimate of the device relative to the initial coordinate system at the current moment, including three-dimensional position and three-dimensional orientation.

[0102] The dynamic prediction unit 303 is used to predict the user's head pose at the next display frame based on the real-time image sequence and the historical motion trajectory of the AR device, and generate corresponding left-eye and right-eye prediction images accordingly.

[0103] In this module, the dynamic prediction unit 303 uses the optimal pose as a reference, combines the real-time angular velocity and acceleration provided by the IMU, and calculates the possible pose of the head at the prediction time based on a preset prediction algorithm. The predicted pose is used as the view projection matrix of the virtual camera. Based on the same predicted pose, the real-time acquired real environment image is subjected to inverse geometric transformation to generate a background layer that matches the prediction viewpoint, thus obtaining the left eye prediction image and the right eye prediction image.

[0104] like Figure 8 As shown, in a preferred embodiment of the present invention, the image compensation module 400 includes:

[0105] The fusion analysis unit 401 is used to fuse the predicted image with the exhibit video data frame to be superimposed, and analyze each region in the fused image to obtain the analysis results, including the brightness, contrast and color difference at the boundary between the edge of the video screen and the real background.

[0106] In this module, the fusion analysis unit 401 fuses the predicted image with the exhibit video data frames to be overlaid to generate a preliminary image. It scans the entire image and calculates the perceptual difference between the virtual foreground and the real background in each local area. This is achieved by calculating brightness contrast (such as the ratio of foreground brightness A to background brightness a), color conflict (the Euclidean distance between foreground coordinates (B,C) and background coordinates (b,c)), and edge sharpness. The analysis results are then recorded in the analysis results, showing the pixels that need to be adjusted.

[0107] The pixel compensation unit 402 is used to adjust the pixel-level color values ​​of the video areas that need to be highlighted in the fused image based on the analysis results, and output the compensated image.

[0108] In this module, the pixel compensation unit 402 processes each pixel that needs adjustment individually, performing compensation in the CIELAB color space. The process involves separating the target foreground color (A, B, C) and the sampled background color (a, b, c) from the blended image. The foreground brightness A is dynamically adjusted based on the background brightness a to improve readability. If the background is too bright (a > 50), the foreground brightness is increased according to the formula A' = A + k * (a - 50), where k is the dynamically adjusted gain coefficient. The vector difference between the foreground and background on the chromaticity plane is calculated, and the foreground brightness (B, C) is shifted in the opposite direction to the background color (b, c) to enhance differentiation. After brightness adjustment, the foreground brightness (B, C) is adjusted. All adjustments are completed in the LAB space. Subsequently, the compensated (A', B', C') values ​​are converted back to the RGB color space of the display screen to generate the final compensated image.

[0109] The fusion display unit 403 is used to send the compensated image after local compensation to the left and right eye displays of the AR device for synchronous display.

[0110] In this module, the fusion display unit 403 submits the compensation image buffer to the display controller, performs low-latency correction by asynchronous time warping, obtains the latest head rotation data, performs a fast image reprojection on the prepared compensation image to offset the last prediction error, and outputs the image to the AR device.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for AR-based intelligent display of art works, characterized in that, The method includes: The AR device is used for device positioning and to obtain the device's viewing direction. The AR device is equipped with a positioning module and two sets of image acquisition devices, which correspond to the left eye and the right eye respectively. The image acquisition devices are located between the eyes and the display screen. Obtain an exhibition area map, determine whether there are exhibits in the line of sight based on AR device positioning data and viewing direction, and interact with users when exhibits are present. Display artworks based on user requests. AR device positioning data refers to the three-dimensional coordinate data of AR devices within the exhibition hall. The video data corresponding to the exhibit is retrieved, real-time images are acquired through an image acquisition device, and a predicted image is generated based on the real-time images. The video data is a pre-made video used to display the exhibit. Based on the predicted image, local compensation is performed on the display of the video data, the color values ​​of each pixel in the display are adjusted, and the compensated image is output for display. The steps of performing local compensation on the display image in the video data based on the predicted image, adjusting the color values ​​of each pixel in the display image, outputting the compensated image, and displaying it specifically include: The predicted image is fused with the video data frames of the exhibit to be overlaid, and the analysis results are obtained by analyzing each region in the fused image. The analysis results include the brightness, contrast and color difference at the boundary between the edge of the video screen and the real background. Based on the analysis results, pixel-level color values ​​are adjusted for the video areas that need to be highlighted in the merged image, and a compensated image is output. The compensated image, after partial compensation, is sent to the left and right eye displays of the AR device for simultaneous display.

2. The AR-based intelligent display method for artworks according to claim 1, characterized in that, The steps of acquiring an exhibition area map, determining whether exhibits exist in the line of sight based on AR device positioning data and viewing direction, and interacting with the user to display artworks based on user requests, specifically include: The device's location data, collected in real time by the AR device's positioning module, is combined with data from the device's built-in orientation sensor to calculate the user's current viewing direction. The device location data and viewing direction vector are compared with the pre-stored exhibition area map in real time to determine whether there is an exhibit model and its corresponding ID in the user's line of sight. When an exhibit is detected in the direction of the user's gaze, an interactive prompt icon is generated and overlaid at the corresponding position on the AR display screen. In response to the user's request for the icon, sample data associated with the exhibit ID is retrieved and displayed.

3. The AR-based intelligent display method for artworks according to claim 1, characterized in that, The steps of retrieving video data corresponding to the exhibit, acquiring real-time images through an image acquisition device, and generating predicted images based on the real-time images include: Based on the identified exhibit ID, retrieve the corresponding panoramic display video from the server as the video data stream to be played; Real-time images of the user's current field of vision are simultaneously acquired through image acquisition devices located in front of the left and right eyes; Based on real-time image sequences and the historical motion trajectory of AR devices, the system predicts the user's head pose at the next display frame and generates corresponding left-eye and right-eye prediction images accordingly.

4. The AR-based intelligent display method for artworks according to claim 1, characterized in that, During the display process, the display screen and real-time screen are synchronously acquired through an image acquisition device, and a verification screen is output. The parameters of local compensation are then corrected based on the verification screen.

5. An AR-based intelligent art display system, characterized in that, The system includes: The device positioning module is used to locate the device based on the AR device and obtain the device's viewing direction. The AR device is equipped with a positioning module and two sets of image acquisition devices, which correspond to the left eye and the right eye respectively. The image acquisition devices are located between the eyes and the display screen. The exhibit detection module is used to acquire a map of the exhibition area, determine whether there are exhibits in the direction of the line of sight based on the AR device's positioning data and viewing direction, and interact with the user when exhibits are present, displaying works based on user requests. The AR device positioning data is the three-dimensional coordinate data of the AR device in the exhibition hall. The image prediction module is used to retrieve the video data corresponding to the exhibit, acquire real-time images through the image acquisition device, and generate predicted images based on the real-time images. The video data is a pre-made video used to display the exhibit. The image compensation module is used to perform local compensation on the display image in the video data based on the predicted image, adjust the color values ​​of each pixel in the display image, and output the compensated image for display. The image compensation module includes: The fusion analysis unit is used to fuse the predicted image with the exhibit video data frames to be superimposed, and analyze each region in the fused image to obtain analysis results, including brightness, contrast and color differences at the boundary between the video frame edge and the real background. The pixel compensation unit is used to adjust the pixel-level color values ​​of the video areas that need to be highlighted in the fused image based on the analysis results, and output the compensated image. The fusion display unit is used to send the compensated image, after local compensation, to the left and right eye displays of the AR device for synchronous display.

6. The AR-based intelligent art display system according to claim 5, characterized in that, The exhibit detection module includes: The viewing angle determination unit is used to obtain the device position data collected in real time by the AR device's positioning module, and combine it with the orientation sensor data built into the device to calculate the user's current viewing angle direction. The exhibit recognition unit is used to perform real-time collision detection between the device position data and the viewing direction vector and the pre-stored exhibition area map to determine whether there is an exhibit model and its corresponding ID in the user's line of sight. The interactive response unit is used to generate and overlay an interactive prompt icon at the corresponding position on the AR display screen when an exhibit is detected in the direction of the user's gaze. In response to the user's request for the icon, the unit retrieves sample data associated with the exhibit ID for display.

7. The AR-based intelligent art display system according to claim 5, characterized in that, The image prediction module includes: The exhibit video retrieval unit is used to retrieve the corresponding panoramic display video from the server as the video data stream to be played based on the identified exhibit ID; The real-time image acquisition unit is used to simultaneously acquire real-time images of the user's current field of vision through image acquisition devices located in front of the left and right eyes; The dynamic prediction unit is used to predict the user's head pose at the next display frame based on real-time image sequences and the historical motion trajectory of the AR device, and generate corresponding left-eye and right-eye prediction images accordingly.

8. The AR-based intelligent art display system according to claim 5, characterized in that, During the display process, the display screen and real-time screen are synchronously acquired through an image acquisition device, and a verification screen is output. The parameters of local compensation are then corrected based on the verification screen.

Images