Museum eye movement data acquisition and analysis method and system based on augmented reality equipment

Abstract

The invention discloses a museum eye movement data acquisition and analysis method based on augmented reality equipment, and the method comprises the steps: collecting a movement track of a user in a museum according to eye wearable equipment, and recording a first timestamp when the user stops in an exhibit region; staring events of all exhibits in the museum are constructed based on the moving tracks and the corresponding first timestamps; collecting eye movement data of the user through an eye wearable device, wherein the eye movement data comprises a gazing point coordinate of a focus point of human eyes in the exhibit area and a corresponding second timestamp; integrating all the eye movement data in the range of each exhibit to generate an ornamental path of the corresponding exhibit; and according to the staring event and the viewing path of each user, performing statistics to generate a user behavior report. The invention also provides a museum eye movement data acquisition and analysis system. According to the method provided by the invention, the exhibition experience of the user can be quantified through real data, so that a more perfect reference is provided for subsequent exhibition optimization.

Description

Technical Field

[0001] The present invention belongs to the field of human-computer interaction and data visualization analysis, and particularly relates to a method and system for collecting and analyzing museum eye movement data based on an augmented reality device. Background Art

[0002] With the rapid development of augmented reality (AR) technology, more and more museums adopt AR technology to provide visitors with an immersive display experience.

[0003] However, understanding the attention distribution, interest points, and behavior patterns of visitors has become an important means to improve the exhibition effect and optimize the layout design. Currently, eye tracking technology is widely used in fields such as psychology, advertising effect research, and user experience optimization, and can study attention and cognitive processes by analyzing user eye movement data.

[0004] However, in the prior art, eye tracking devices are usually bulky and cannot be seamlessly integrated with visitors' mobile devices, which limits their application in open environments such as museums. Some studies combine virtual reality (VR) or augmented reality technology for eye movement data collection, but these solutions are usually complex and costly, and are not suitable for large-scale application in the behavior analysis of museum visitors. In addition, how to dynamically associate the visual focus of visitors with exhibits is also one of the challenges faced by current technologies.

[0005] Patent document CN116909408A discloses a content interaction method based on an MR smart glasses, including judging the state and needs of a user, and acquiring a local image of a visual fixation area; performing artificial intelligence processing on the acquired image content, and according to the processing result, sorting the priority of notification icons of related applications, services, and information; retrieving a database to obtain an answer or open a related application according to the needs of the user and the priority sorting, and presenting the answer or application on an optical display. Provide a more accurate content interaction experience and implement functions such as object detection and semantic segmentation.

[0006] Patent document CN116090879A discloses a method for evaluating flight training quality based on eye movement data. First, collect the eye movement data of different pilots completing the same flight task under the same environmental conditions, and use an existing automatic evaluation system for pilots' flight operation quality to evaluate their task completion situation and record their scores. Then, process the data to construct a pilot flight training quality evaluation model using a principal component analysis-backpropagation neural network evaluation model to evaluate the flight training quality of flight trainees to be evaluated to obtain an evaluation result. Summary of the Invention

[0007] The object of the present invention is to provide a method and system for collecting and analyzing eye movement data in a museum based on an augmented reality device. This method can quantify the exhibition experience of users through real data, thereby providing a more perfect reference for subsequent exhibition optimization.

[0008] To achieve the first object of the present invention, the following solution is provided: A method for collecting and analyzing eye movement data in a museum based on an augmented reality device, including: Collect the movement trajectory of the user in the museum according to the eye-wearable device, and record the first time stamp when staying in the exhibit area. Based on the movement trajectory and the corresponding first time stamp, construct the gazing events of all exhibits in the museum. Collect the eye movement data of the user through the eye-wearable device, and the eye movement data includes the gazing point coordinates of the human eye's attention point in the exhibit area and the corresponding second time stamp. Integrate all the eye movement data within the range of each exhibit to generate the viewing path corresponding to the exhibit. According to the gazing events and viewing paths of each user, statistically generate a user behavior report on the exhibits exhibited in the museum this time.

[0009] The present invention collects the eye movement data of the audience in the museum exhibition scene, analyzes their appreciation behavior patterns, and uses the displayed analysis results as a reference for optimizing subsequent exhibition plans.

[0010] Specifically, the movement trajectory takes the entrance of the museum as the origin, regards the museum floor as a plane, and takes the eye-wearable device as a moving point to collect the movement trajectory in the plane.

[0011] Specifically, the gazing point coordinates are obtained by constructing a two-dimensional plane coordinate system with the two ends of the exhibit as reference points, taking the center point of the exhibit as the center origin of the two-dimensional plane coordinate system, and converting the three-dimensional coordinate information of the human eye's attention point collected by the eye-wearable device into the two-dimensional plane coordinate system.

[0012] Specifically, the conversion is to omit the height information in the three-dimensional coordinate information and then perform the change through the following expression: ; where represents the three-dimensional coordinate information, where is the depth, represents the coordinate information of the upper left corner of the exhibit in the three-dimensional plane coordinate system, represents the coordinate information of the lower right corner of the exhibit in the three-dimensional plane coordinate system, represents the width of the exhibit in the two-dimensional plane coordinate system, represents the height of the exhibit in the two-dimensional plane coordinate system.

[0013] Specifically, the second timestamp is obtained by dividing the exhibit into multiple grids and recording the time that the human eye focus point collected by the eyewear device stays in each grid.

[0014] Specifically, the exhibits are divided in the horizontal and vertical directions to construct corresponding grid areas, and the expression is as follows: ; ;in, Indicates the number of horizontal grids, Indicates the number of vertical grids, Indicates the size of a single grid, Indicates the width of the exhibit in the two-dimensional plane coordinate system, Indicates the height of the exhibit in the two-dimensional plane coordinate system.

[0015] Specifically, the user behavior report includes the user's gaze time in front of each exhibit, the user's sight transfer path, and the user's stay time and movement trajectory in the museum.

[0016] In order to achieve the second object of the present invention, the following technical scheme is provided: a museum eye movement data collection and analysis system, which is implemented by the above-mentioned museum eye movement data collection and analysis method based on augmented reality device, including a data collection module, a data analysis module and a visualization module; the data collection module collects the movement trajectory and eye movement data of the user in the museum through the eye wear device; The data analysis module generates corresponding gaze events and viewing paths according to the collected movement trajectory and eye movement data; The visualization module generates corresponding user behavior reports according to gaze events and viewing paths, and performs visualization output.

[0017] Specifically, the output of the visualization module includes a heat map for displaying the distribution of gaze events and a gaze trajectory map for displaying the user's line of sight movement path.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The use of augmented reality devices enables real-time collection and analysis of visitors’ eye movement data, breaking through the application limitations of traditional eye tracking devices in mobile scenarios, allowing museums to more accurately capture visitors’ points of interest and attention distribution; Through data-driven exhibition optimization and analysis of the eye movement data of a large number of visitors, museums can obtain quantitative feedback on exhibition layout, exhibit attractiveness, etc., to help them optimize exhibition content and improve exhibition effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1Flowchart of the museum eye movement data acquisition and analysis method based on the augmented reality device provided in this embodiment; Figure 2 Fixation trajectory map output by the visualization module provided in this embodiment; Figure 3 Heat map output by the visualization module provided in this embodiment. Detailed implementation manners

[0020] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only some, but not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and illustrated herein generally may be arranged and designed in a variety of different configurations. Therefore, the detailed description of the embodiments of the present invention provided herein is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] As Figure 1 shown, a museum eye movement data acquisition and analysis method based on the augmented reality device provided in this embodiment collects the eye movement data of the audience in the museum exhibition scene, analyzes the appreciation behavior patterns of the audience based on the eye movement data, and visually outputs the corresponding analysis results, so as to more intuitively understand the interest points and attention paths of the audience.

[0022] To achieve the above objectives, the following specific processes are provided in this embodiment: Collect the movement trajectories of the user in the museum according to the eye-wearable device, and record the first time stamp when staying in the exhibit area; Based on the movement trajectories and the corresponding first time stamps, construct the fixation events of all exhibits in the museum; Collect the eye movement data of the user through the eye-wearable device, where the eye movement data includes the fixation point coordinates of the focus of the human eye in the exhibit area and the corresponding second time stamp; Integrate all the eye movement data within the range of each exhibit to generate the viewing path of the corresponding exhibit; According to the fixation events and viewing paths of each user, statistically generate the user behavior report of the exhibits exhibited in the museum this time.

[0023] More specifically, in this embodiment, augmented reality technology and high-precision eye tracking devices are used to record the interaction between the audience and the exhibits in real time, and perform data collection and processing.

[0024] In this embodiment, the high-precision eye tracking device mentioned is the Pico 4 Enterprise device.

[0025] The process of this step is as follows: Import the museum exhibit paintings as image tracking objects in the Unity environment. By calling the eye tracking API of the Pico 4 Enterprise device (2160*2160 resolution, 90Hz refresh rate), the gaze point coordinates of the audience's focus are captured in real time, and the gaze time and gaze position on the paintings are recorded. In this embodiment, the gaze point coordinates are obtained by constructing a two-dimensional plane coordinate system with the two ends of the exhibit as reference points, and taking the center point of the exhibit as the center origin of the two-dimensional plane coordinate system, and converting the three-dimensional coordinate information of the human eye focus collected by the eye-worn device into the two-dimensional plane coordinate system.

[0026] After the audience wears the augmented reality glasses, the system is initialized at the entrance of the exhibition hall, records the initial coordinates of the audience, and real-time records the spatial position information of the audience in the exhibition hall. The system analyzes the interaction behavior between the audience and the exhibits by tracking the movement trajectory of the audience in the exhibition hall, providing basic data for subsequent appreciation behavior analysis. The movement trajectory takes the entrance of the museum as the origin, regards the museum floor as a plane, and takes the eye-worn device as a moving point to collect the movement trajectory in the plane.

[0027] According to the collected eye movement data, the curator is provided with visual analysis results of the audience's behavior, including that during the data collection process, the eye tracking system will record the moment of each gaze of the audience and its corresponding spatial coordinates .

[0028] Whenever the user gazes at a certain exhibit, the system will record the start time and end time of the gaze. For the gaze data of the same painting or exhibit, since it is a plane, all the collected gaze point data are the same on the Z axis. Therefore, the gaze time is mainly processed based on the XY coordinates. During data processing, for each painting, all the gaze points on the painting and the corresponding timestamps are recorded, and all gaze events are processed in chronological order: ; In addition, for the gaze data of the painting exhibits, since the exhibits are planes, the data collected in the Z coordinate direction is the same, and only the data in the XY direction needs to be converted.

[0029] That is, assuming that the center point of the painting in the Unity coordinate system is when the width and height are W and H respectively, and the scaling ratio is .

[0030] Then the Unity coordinates of the upper left corner of the painting are: ; The Unity coordinates at the lower right corner of the painting are: Convert the gaze point coordinates in the Unity coordinate system into points in the two-dimensional pixel coordinate system , and its conversion formula is as follows: ; Among them, represents three-dimensional coordinate information, where is the depth, represents the coordinate information of the upper left corner of the exhibit in the three-dimensional plane coordinate system, represents the coordinate information of the lower right corner of the exhibit in the three-dimensional plane coordinate system, represents the width of the exhibit in the two-dimensional plane coordinate system, represents the height of the exhibit in the two-dimensional plane coordinate system.

[0031] Gaze time calculation: For the gaze data of the same painting or exhibit, since it is a plane, all the collected gaze point data are the same on the Z-axis. Therefore, the gaze time is mainly processed based on the XY coordinates. When processing the data, for each painting, record all the gaze points on the painting and the corresponding timestamps, and process all the gaze events in chronological order: ; For each painting, first divide the painting area into multiple grid sub-regions as needed, that is, divide the exhibit along the horizontal and vertical directions to construct the corresponding grid area, and its expression is as follows: ; ; Among them, represents the number of horizontal grids, represents the number of vertical grids, represents the size of a single grid, represents the width of the exhibit in the two-dimensional plane coordinate system, represents the height of the exhibit in the two-dimensional plane coordinate system.

[0032] Calculate the gaze time of each sub-region: Each eye movement tracking record has a timestamp, and the duration of each gaze can be calculated using the following formula: ; Among them, is the timestamp of the next record, is the timestamp of the current record, calculated in the above way then represents the duration of the viewer's gaze in a certain area.

[0033] For the gaze time on each region j, considering the grid division of the painting area, it can be expressed as: ; Among them, represents the set of all gaze point indices falling on region j, and Represents the timestamps of two consecutive fixation points.

[0034] For the movement trajectory of the audience inside the exhibition hall, the system takes the entrance of the exhibition hall as the coordinate origin, records the XZ coordinates of the audience, and ignores the Y coordinate. The mapping formula for the audience's position is: ; where f() represents the mapping function, which is used to convert the collected spatial position data into the corresponding points on the exhibition hall map.

[0035] This embodiment also provides a museum eye movement data acquisition and analysis system, which is implemented by the above-mentioned museum eye movement data acquisition and analysis method based on augmented reality devices, and includes: A data acquisition module that acquires the movement trajectory and eye movement data of the user inside the museum through the eye-worn device.

[0036] A data analysis module that generates corresponding fixation events and viewing paths according to the acquired movement trajectory and eye movement data.

[0037] A visualization module that generates a corresponding user behavior report according to the fixation events and viewing paths and performs visual output, where the visual output includes a heat map for displaying the distribution of fixation events and a fixation trajectory map for displaying the movement path of the user's line of sight.

[0038] As Figure 2 shown, it is the fixation trajectory map provided by this embodiment, where the blue is the trajectory route, and the red frame in the figure is the area where the audience's line of sight stays the longest in this fixation trajectory map.

[0039] As Figure 3 shown, it is the heat map provided by this embodiment, which is displayed in the form of a heat map on the surface of the painting, so that the content of the area that the audience pays the most attention to can be obtained very intuitively.

[0040] In addition, the terms "upper", "lower", "inner", "outer", "front", and "rear" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions, and values of the components and steps described in these embodiments do not limit the scope of the present invention.

[0041] Of course, the above are only specific embodiments of the present invention and do not limit the scope of implementation of the present invention. Any equivalent changes or modifications made according to the structure, features, and principles described in the scope of the patent application of the present invention should be included in the scope of the patent application of the present invention.

[0042] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for collecting and analyzing eye movement data in a museum based on an augmented reality device, characterized in that, Including: Collect the movement trajectory of the user in the museum by the eye-worn device and record the first timestamp of staying in the exhibit area; Based on the movement trajectory and the corresponding first timestamp, construct the gazing events of all exhibits in the museum; Collect the eye movement data of the user by the eye-worn device, where the eye movement data includes the gazing point coordinates of the human eye's focus in the exhibit area and the corresponding second timestamp; Integrate all the eye movement data within the range of each exhibit to generate the viewing path of the corresponding exhibit; According to the gazing events and viewing paths of each user, statistically generate the user behavior report of the exhibits displayed in the museum this time.

2. The method for collecting and analyzing museum eye movement data based on an augmented reality device according to claim 1, wherein The movement trajectory takes the entrance of the museum as the origin, regards the museum floor as a plane, and collects the movement trajectory of the eye-worn device as a moving point within the plane.

3. The method for collecting and analyzing museum eye movement data based on an augmented reality device according to claim 1, wherein, The gazing point coordinates are obtained by constructing a two-dimensional plane coordinate system with the two ends of the exhibit as the reference points, taking the center point of the exhibit as the center origin of the two-dimensional plane coordinate system, and converting the three-dimensional coordinate information of the human eye's focus collected by the eye-worn device into the two-dimensional plane coordinate system.

4. The method for collecting and analyzing museum eye movement data based on an augmented reality device according to claim 3, wherein The transformation is to omit the height information in the three-dimensional coordinate information and then perform the transformation through the following expression: ; where represents the three-dimensional coordinate information, where is the depth, represents the coordinate information of the upper left corner of the exhibit in the three-dimensional plane coordinate system, represents the coordinate information of the lower right corner of the exhibit in the three-dimensional plane coordinate system, represents the width of the exhibit in the two-dimensional plane coordinate system, represents the height of the exhibit in the two-dimensional plane coordinate system.

5. The method for collecting and analyzing museum eye movement data based on an augmented reality device according to claim 1, wherein The second timestamp is obtained by dividing the exhibit into multiple grids and recording the time that the human eye's focus collected by the eye-worn device stays in each grid.

6. The method for collecting and analyzing museum eye movement data based on an augmented reality device according to claim 5, wherein Divide the exhibit in both the horizontal and vertical directions to construct the corresponding grid regions, and the expression is as follows: ; ; where represents the number of horizontal grids, represents the number of vertical grids, represents the size of a single grid, represents the width of the exhibit in the two-dimensional plane coordinate system, represents the height of the exhibit in the two-dimensional plane coordinate system.

7. The method for collecting and analyzing museum eye movement data based on an augmented reality device according to claim 1, wherein The user behavior report includes the gazing time of the user in front of each exhibit, the line-of-sight transfer path of the user, the staying time of the user in the museum, and the movement trajectory.

8. A museum eye movement data acquisition and analysis system, characterized in that, It is implemented by the museum eye movement data acquisition and analysis method based on the augmented reality device as described in any one of claims 1 to 7, including a data acquisition module, a data analysis module, and a visualization module; The data acquisition module collects the movement trajectory and eye movement data of the user in the museum by the eye-worn device; The data analysis module generates the corresponding gazing events and viewing paths according to the collected movement trajectory and eye movement data; The visualization module generates the corresponding user behavior report according to the gazing events and viewing paths and performs visual output.

9. The museum eye movement data acquisition and analysis system according to claim 8, characterized in that The output of the visualization module includes a heat map for displaying the distribution of gazing events and a gaze trajectory map for displaying the user's line-of-sight movement path.

Images