Immersive digital tourism experience configuration method and system based on eye tracking technology

Abstract

The present invention belongs to the field of virtual tourism experience, and in particular relates to an immersive digital tourism experience configuration method and system based on eye tracking technology. The method first initializes a virtual device, configures a multi-category virtual tourism scene library, and calibrates viewpoints and visual parameters. Secondly, based on user selection or voice commands, a personalized tourism route is intelligently generated and embedded. At the same time, through real-time eye trajectory analysis, user points of interest are identified. Combined with a global-local scene change model, scene synchronization optimization and accurate information push are achieved. Finally, related scenes are dynamically pushed based on a list of points of interest, and the travel planning route is adjusted in real time. The present invention realizes real-time identification of user points of interest and dynamic adjustment of scenes through eye tracking technology, enhancing the immersion and interactivity of virtual tourism and improving the quality of user experience.

Description

Technical Field

[0001] The present invention belongs to the field of virtual tourism experience, and in particular relates to an immersive digital tourism experience configuration method and system based on eye tracking technology. Background Art

[0002] With the rapid development of VR and AR technologies, immersive digital tourism is gradually becoming a new trend in the tourism industry. However, most current immersive tourism experiences are still limited to traditional keyboard, mouse and touch screen operations, which are difficult to provide sufficient natural immersion and intuitive interaction. The introduction of eye tracking technology provides an innovative solution to this problem. However, although there have been attempts at existing immersive tourism systems based on eye tracking, such as the patent with publication number CN113534835A, which discloses a virtual remote experience system and method for tourism, a remote flight experience combining drones with eye tracking, or a tourist service platform integrated with VR equipment, there are still many deficiencies in actual application. Among them, the accuracy and real-time performance of eye tracking need to be improved to more accurately capture user intentions, avoid misoperations, and improve the authenticity and smoothness of the experience.

[0003] The above existing technologies have the following problems: although the existing virtual tourism experience has made some progress in terms of vision and hearing, it still has some shortcomings in terms of changes from global to local landscapes. To this end, the present invention provides an immersive digital tourism experience configuration method and system based on eye tracking technology. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention proposes an immersive digital tourism experience configuration method based on eye tracking technology. The method first initializes the virtual device, configures a multi-category virtual tourism scene library, and calibrates the viewpoint and visual parameters. Secondly, based on user selection or voice commands, it intelligently generates and embeds personalized tourism routes. At the same time, through real-time eye trajectory analysis, it identifies user points of interest. Combined with the global-local scene change model, it realizes scene synchronization optimization and accurate information push. Finally, based on the point of interest list, it dynamically pushes related scenes and adjusts the tourism planning path in real time. This method realizes real-time identification of user points of interest and dynamic adjustment of scenes through eye tracking technology, enhances the immersion and interactivity of virtual tourism, and improves the quality of user experience.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The immersive digital tourism experience configuration method based on eye tracking technology includes:

[0007] S1. Configure a multi-classification virtual tourism scene library and correct the initial viewpoint, scene brightness and diopter through multi-point positioning;

[0008] S2. Selecting an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching questionnaire or voice, generating an initial virtual tourism planning path through a path generation algorithm based on the obtained matching list information or voice information, and embedding the path into the selected initial virtual tourism global scene;

[0009] S3. Using the configured eye trajectory recognition model, the user's eye trajectory in the initial virtual tourism global scene and the gaze time corresponding to each viewpoint position on the trajectory are obtained in real time. Based on the gaze time corresponding to each viewpoint position on the trajectory, an evaluation algorithm and graph rendering technology are used to obtain a hierarchical distribution map of the user's areas of interest and a list of points of interest;

[0010] S4. Based on the user's interest area level distribution map and gaze time, the global-local scene transformation is performed using the configured global-local scene change model. At the same time, a textual explanation of the transformed local scene is pushed through a voice or text algorithm.

[0011] S5. According to the list of points of interest, the corresponding local scene is pushed through the matching algorithm, and the initial virtual tourism planning path and the initial virtual tourism global scene are updated according to the pushed local scene information.

[0012] Specifically, the steps of obtaining the initial virtual tourism global scene include:

[0013] S201, according to the configured scenario matching questionnaire or voice recognition, obtain user demand scenario text information and pre-process the obtained text information;

[0014] S202: Build a matching push model and integrate it into the configured multi-classification virtual tourism scene library, input the pre-processed text information into the matching push model, and obtain a scene matching score;

[0015] S203: Setting a score threshold. When only one scene type corresponds to a scene with a scene matching score greater than the score threshold, using the scene with a score greater than the score threshold as the initial virtual tourism global scene, and generating an initial virtual tourism planning path using a path generation algorithm based on the text information and the initial virtual tourism global scene.

[0016] S204: When there is more than one scene type with a scene matching score greater than the score threshold, the scene types greater than the score threshold are sorted according to the scene matching score, and the corresponding scenes are spliced according to the score sorting result using a splicing algorithm to obtain an initial comprehensive virtual tourism global scene;

[0017] S205, generating an initial virtual tourism planning path through a path generation algorithm based on the initial comprehensive virtual tourism global scene, scene matching score and text information;

[0018] S206. When the scene matching scores corresponding to all scenes are less than the score threshold, a global scene corresponding to the type and geographical location is generated based on the text information and the configured scene generation model, and the generated scene is added to the multi-classification virtual tourism scene library, and the S203 process is repeated to generate the corresponding initial virtual tourism planning path.

[0019] Specifically, the steps of constructing the eye trajectory recognition model in S3 include:

[0020] S301: Acquire an initially corrected eye image, construct a two-dimensional eye coordinate system centered on the pupil position, and fit an initial elliptical iris curve using the corresponding iris region in the eye image based on the two-dimensional eye coordinate system. Simultaneously, construct an initial line of sight based on a line connecting the initial gaze point position and the pupil position corresponding to the initial corrected eye image, and add the initial line of sight to the two-dimensional eye coordinate system to obtain a three-dimensional eye coordinate system.

[0021] S302: Obtain eye images corresponding to viewpoints at different positions and the time of change between corresponding adjacent viewpoints, fit an elliptical iris curve to the preprocessed iris region corresponding to the eye image, and synchronize the corresponding line of sight to the three-dimensional eye coordinate system, thereby obtaining the deviation angle between the corresponding line of sight and the initial line of sight in the three-dimensional eye coordinate system and the ratio between the intersection area of the corresponding elliptical iris curve and the initial elliptical iris curve and the non-intersection area;

[0022] S303: Based on the deviation angles between the sight lines corresponding to the adjacent time points and the initial sight line, a trigonometric function is used to obtain the spatial angle between the adjacent sight lines, and based on the spatial angle between the adjacent sight lines and the time between the adjacent viewpoint changes, the angular velocity of the eye viewpoint change and the corresponding trajectory curve are obtained;

[0023] S304. Input the obtained deviation angle between the corresponding line of sight and the initial line of sight, the ratio of the intersection area and the non-intersection area between the corresponding elliptical iris curve and the initial elliptical iris curve, the viewpoint discrimination threshold, the angular velocity of the eye viewpoint change, and the corresponding trajectory curve into the constructed eye trajectory recognition model for training, and output the viewpoint position predicted by the model and the trajectory between adjacent viewpoints.

[0024] Specifically, the steps of constructing the eye trajectory recognition model in S3 also include:

[0025] S305, obtaining a viewpoint error and a trajectory error based on the viewpoint position predicted by the eye trajectory recognition model, the trajectory between adjacent viewpoints, and the corresponding true values;

[0026] S306, setting an angular velocity viewpoint output threshold. When the angular velocity of the eye viewpoint change of the corresponding viewpoint is greater than the angular velocity viewpoint output threshold, the current viewpoint is determined to be a saccade point, and the corresponding saccade point position coordinates are not output. Otherwise, the current viewpoint is determined to be a fixation point, and the corresponding fixation point position coordinates are output.

[0027] S307, embedding the discrimination process set in S306 into the eye trajectory recognition model for training, obtaining the ratio of the scan point to the fixation point in the output viewpoint, and constructing the discrimination error using the obtained ratio of the scan point to the fixation point;

[0028] S308: Set a comprehensive loss error threshold, and use the viewpoint error, trajectory error and discrimination error to construct a comprehensive loss function, and use the constructed comprehensive loss function to train the eye trajectory recognition model. When the comprehensive loss function value is less than the comprehensive loss error threshold, stop training to obtain a trained eye trajectory recognition model.

[0029] Specifically, the steps of obtaining the user's interest area level distribution map and the interest point list include:

[0030] S311, obtaining each gaze point position, the corresponding gaze time, and the local scene information of the area corresponding to the gaze point position based on the eye trajectory recognition model, and obtaining the interest level evaluation score of the local scene corresponding to each gaze point position area through an evaluation algorithm;

[0031] S312. Based on the interest evaluation score of the local scene corresponding to each gaze point location area, a user interest area level distribution map is obtained through a region map rendering algorithm. A score-color depth mapping function is established based on the interest evaluation score of each area and the color depth corresponding to each area. The color depth of the area corresponding to different gaze points in the current global scene is updated in real time through the established score-color depth mapping function.

[0032] S313: construct a list of points of interest based on the corresponding local scene information and interest level evaluation scores in each area, and establish a mapping relationship between the list of points of interest and the initial virtual tourism planning path through a linear function;

[0033] S314. According to the list of points of interest, the local scene information in the multi-classification virtual tourism scene library is matched through a matching algorithm, and the matching result is fed back to the corresponding user. The user confirms whether to add the matching local scene by blinking. When the addition of the local scene is confirmed, the matched local scene is added to the area corresponding to the current global scene in the list of points of interest through a mapping relationship, and the initial virtual tourism global scene is updated.

[0034] Specifically, the steps of constructing the global-local scene change model include:

[0035] S401, determining the viewing angle range of the global scene at the current moment according to the attributes of the virtual device, and constructing a global scene information space based on the global scene information within the viewing angle range;

[0036] S402: Obtain a local scene information subspace corresponding to the region in the global scene information space based on the user interest region level distribution map;

[0037] S403: Using the interest level evaluation score corresponding to each local area scene in the global scene as the upper limit of the magnification of the current local scene; when the upper limit of the magnification of the current local scene is greater than the maximum zoom range specified by the device properties, using the maximum zoom upper limit of the device as the upper limit of the magnification of the current local scene;

[0038] S404, obtaining a local scene magnification rate according to the magnification factor of the local scene in the area and the length of the gaze time;

[0039] S405. When the current viewpoint is determined to be the fixation point through the eye trajectory recognition model, the local scene corresponding to the fixation point position is magnified within a determined viewing angle range by obtaining the upper limit of the local scene magnification factor, the local scene magnification rate, and the local scene information subspace.

[0040] Specifically, the steps of constructing the global-local scene change model also include:

[0041] S406: When the gaze point deviates from the corresponding local scene range in the user interest area level distribution map, the corresponding magnified local scene is reduced according to the magnification rate until it returns to the zoom factor corresponding to the global scene;

[0042] S407: When the current viewpoint is a scanning viewpoint, the corresponding local scene is not scaled;

[0043] S408. When it is determined in S405 that the gaze point area is to be enlarged and changed, a virtual question-answering robot configured with a voice or text algorithm is called to provide information explanation and user questions and answers for the enlarged local area.

[0044] An immersive digital tourism experience configuration system based on eye tracking technology, including: an initialization module, a path generation module, a recognition module, and a scene transformation module;

[0045] The initialization module includes an initialization unit and a correction unit; the initialization unit is used to initialize the virtual device and configure the multi-classification virtual tourism scene library; the correction unit is used to correct the initial viewpoint, background brightness and diopter through multi-point positioning;

[0046] The path generation module includes a scene selection unit and an initial path unit; the scene selection unit is used to select an initial virtual tourism global scene of corresponding type and geographical location from a multi-category virtual tourism scene library through a configured scene matching list or voice input; the initial path unit is used to generate an initial virtual tourism planning path through a path generation algorithm based on the matching list information or voice information, and embed the path into the selected initial virtual tourism global scene.

[0047] Specifically, the recognition module includes a trajectory recognition unit and a cluster analysis unit;

[0048] The trajectory recognition unit is used to obtain the user's real-time eye movement trajectory and the corresponding gaze time of each viewpoint position on the trajectory in real time through the configured eye trajectory recognition model; the cluster analysis unit is used to obtain the user's interest area level distribution map and interest point list based on the gaze time corresponding to each viewpoint position on the trajectory through evaluation algorithms and graph rendering technology;

[0049] The scene change module includes a scene change unit; the scene change unit is used to perform synchronous scene change through a global-local scene change model according to the user's interest area level distribution map and gaze time.

[0050] A computer-readable storage medium stores computer instructions, which, when executed, execute an immersive digital tourism experience configuration method based on eye tracking technology.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] In response to the shortcomings of the existing technology, the present invention significantly improves the shortcomings of the existing virtual tourism experience, especially the defects in the change from global to local landscape, by introducing eye tracking technology and a dynamic global-local scene transformation model. By obtaining the user's eye movement trajectory and gaze time in real time, and combining evaluation algorithms and graph rendering technology, the system can accurately identify the user's area of interest and dynamically adjust the scene details accordingly to achieve a seamless transition from global to local. In addition, by constructing a user interest area level distribution map and a list of points of interest, the system can intelligently push local scenes of interest to the user and dynamically update the virtual tourism planning path based on user feedback, making the experience more personalized and rich. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a method for configuring an immersive digital tourism experience based on eye tracking technology according to Example 1 of the present invention;

[0054] Figure 2 This is a flowchart of obtaining the initial virtual tourism planning route according to embodiment 1 of the present invention;

[0055] Figure 3 This is a module diagram of the immersive digital tourism experience configuration system based on eye tracking technology in Example 2 of the present invention. DETAILED DESCRIPTION

[0056] Example 1

[0057] See also Figure 1 The present invention provides an embodiment of an immersive digital tourism experience configuration method based on eye tracking technology, comprising the following steps:

[0058] S1. Initialize the virtual device, configure a multi-category virtual tourism scene library, and calibrate the initial viewpoint, scene brightness, and diopter through multi-point positioning. Furthermore, the specific steps of configuring the multi-category virtual tourism scene library in this embodiment include:

[0059] Import pre-prepared multi-classified virtual tourism scene data into the system data and create a quick index for each scene; the quick index includes: type, geographical location, theme, keyword, etc.; further, in this embodiment, the tourism scenes include natural scenery scenic spots, historical and cultural scenic spots, urban sightseeing scenic spots, nature reserves and religious and cultural scenic spots, etc.;

[0060] Furthermore, the diopter correction obtains the user's diopter data through a built-in diopter measurement tool, and adjusts the focus of the virtual display according to the user's diopter data to ensure that the user does not experience visual blur or discomfort when viewing the virtual scene.

[0061] S2. Selecting an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching questionnaire or voice, generating an initial virtual tourism planning path through a path generation algorithm based on the obtained matching list information or voice information, and embedding the path into the selected initial virtual tourism global scene;

[0062] Further, see Figure 2 The steps of obtaining the initial virtual tourism global scene in this embodiment include:

[0063] S201, according to the configured scenario matching questionnaire or voice recognition, obtain user demand scenario text information and pre-process the obtained text information;

[0064] Furthermore, the scenario matching questionnaire configured in this embodiment sets different questionnaire options according to the quick index in the multi-classification virtual tourism scenario library to obtain the user's corresponding initial tourism intention.

[0065] S202: Build a matching push model and integrate it into the configured multi-classification virtual tourism scene library, input the pre-processed text information into the matching push model, and obtain a scene matching score;

[0066] S203: Setting a score threshold. When only one scene type corresponds to a scene with a scene matching score greater than the score threshold, using the scene with a score greater than the score threshold as the initial virtual tourism global scene, and generating an initial virtual tourism planning path using a path generation algorithm based on the text information and the initial virtual tourism global scene.

[0067] S204: When there is more than one type of scene with a scene matching score greater than the score threshold, sort the type of scenes greater than the score threshold according to the scene matching score, and based on the sorting result, stitch the corresponding scenes according to the score sorting result using a stitching algorithm to obtain an initial comprehensive virtual tourism global scene. Furthermore, in this embodiment, the results are sorted in descending order according to the score, and the scenes that meet the conditions are stitched together, with the scene with the larger score being ranked higher.

[0068] S205, generating an initial virtual tourism planning path through a path generation algorithm based on the initial comprehensive virtual tourism global scene, scene matching score and text information;

[0069] S206. When the scene matching scores corresponding to all scenes are less than the score threshold, a global scene corresponding to the type and geographical location is generated based on the text information and the configured scene generation model, and the generated scene is added to the multi-classification virtual tourism scene library, and the S203 process is repeated to generate the corresponding initial virtual tourism planning path.

[0070] This process uses a configured scenario matching questionnaire or voice recognition technology, combined with a matching push model and a path generation algorithm, to accurately obtain users' virtual tourism needs and generate personalized initial virtual tourism global scenarios and planned routes. When there is more than one scenario with a matching score greater than the threshold, a comprehensive virtual tourism global scenario is generated through scenario matching score sorting and splicing algorithms, ensuring the continuity and diversity of the tourism experience. When no scenario matching score exceeds the threshold, the system can also generate new scenarios based on user needs and add them to the scenario library, further enriching the content of the scenario library.

[0071] S3. Using the configured eye trajectory recognition model, the user's eye trajectory in the initial virtual tourism global scene and the gaze time corresponding to each viewpoint position on the trajectory are obtained in real time. Based on the gaze time corresponding to each viewpoint position on the trajectory, an evaluation algorithm and graph rendering technology are used to obtain a hierarchical distribution map of the user's areas of interest and a list of points of interest;

[0072] Furthermore, the steps of constructing the eye trajectory recognition model in this embodiment include:

[0073] S301: Acquire an initially corrected eye image, construct a two-dimensional eye coordinate system centered on the pupil position, and fit an initial elliptical iris curve using the corresponding iris region in the eye image based on the two-dimensional eye coordinate system. Simultaneously, construct an initial line of sight based on a line connecting the initial gaze point position and the pupil position corresponding to the initial corrected eye image, and add the initial line of sight to the two-dimensional eye coordinate system to obtain a three-dimensional eye coordinate system.

[0074] S302: Obtain eye images corresponding to viewpoints at different positions and the time of change between corresponding adjacent viewpoints, fit an elliptical iris curve to the preprocessed iris region corresponding to the eye image, and synchronize the corresponding line of sight to the three-dimensional eye coordinate system, thereby obtaining the deviation angle between the corresponding line of sight and the initial line of sight in the three-dimensional eye coordinate system and the ratio between the intersection area of the corresponding elliptical iris curve and the initial elliptical iris curve and the non-intersection area;

[0075] S303: Based on the deviation angles between the sight lines corresponding to the adjacent time points and the initial sight line, a trigonometric function is used to obtain the spatial angle between the adjacent sight lines, and based on the spatial angle between the adjacent sight lines and the time between the adjacent viewpoint changes, the angular velocity of the eye viewpoint change and the corresponding trajectory curve are obtained;

[0076] S304: Input the obtained deviation angle between the corresponding line of sight and the initial line of sight, the ratio of the intersection area and the non-intersection area between the corresponding elliptical iris curve and the initial elliptical iris curve, the viewpoint discrimination threshold, the angular velocity of the eye viewpoint change, and the corresponding trajectory curve into the constructed eye trajectory recognition model for training, and output the viewpoint position and trajectory between adjacent viewpoints predicted by the model. Furthermore, the eye trajectory recognition model in this embodiment is constructed using a pupil-cornea tracking algorithm.

[0077] S305, obtaining a viewpoint error and a trajectory error based on the viewpoint position predicted by the eye trajectory recognition model, the trajectory between adjacent viewpoints, and the corresponding true values;

[0078] S306, setting an angular velocity viewpoint output threshold. When the angular velocity of the eye viewpoint change of the corresponding viewpoint is greater than the angular velocity viewpoint output threshold, the current viewpoint is determined to be a saccade point, and the corresponding saccade point position coordinates are not output. Otherwise, the current viewpoint is determined to be a fixation point, and the corresponding fixation point position coordinates are output.

[0079] S307, embedding the discrimination process set in S306 into the eye trajectory recognition model for training, obtaining the ratio of the scan point to the fixation point in the output viewpoint, and constructing the discrimination error using the obtained ratio of the scan point to the fixation point;

[0080] S308: Set a comprehensive loss error threshold, and use the viewpoint error, trajectory error and discrimination error to construct a comprehensive loss function, and use the constructed comprehensive loss function to train the eye trajectory recognition model. When the comprehensive loss function value is less than the comprehensive loss error threshold, stop training to obtain a trained eye trajectory recognition model.

[0081] Furthermore, the step of obtaining the user interest area level distribution map and the interest point list in this embodiment includes:

[0082] S311, obtaining each gaze point position, the corresponding gaze time, and the local scene information of the area corresponding to the gaze point position based on the eye trajectory recognition model, and obtaining the interest level evaluation score of the local scene corresponding to each gaze point position area through an evaluation algorithm;

[0083] Furthermore, in this embodiment, the size of the local scene corresponding to each gaze point position area is a circular area with a radius of half the base of a triangle with the gaze point as the center and the maximum viewing angle as the vertex angle under the current viewing angle state;

[0084] S312. Based on the interest evaluation score of the local scene corresponding to each gaze point location area, a user interest area level distribution map is obtained through a region map rendering algorithm. A score-color depth mapping function is established based on the interest evaluation score of each area and the color depth corresponding to each area. The color depth of the area corresponding to different gaze points in the current global scene is updated in real time through the established score-color depth mapping function.

[0085] S313: construct a list of points of interest based on the corresponding local scene information and interest level evaluation scores in each area, and establish a mapping relationship between the list of points of interest and the initial virtual tourism planning path through a linear function;

[0086] S314. According to the list of points of interest, the local scene information in the multi-classification virtual tourism scene library is matched through a matching algorithm, and the matching result is fed back to the corresponding user. The user confirms whether to add the matching local scene by blinking. When the addition of the local scene is confirmed, the matched local scene is added to the area corresponding to the current global scene in the list of points of interest through a mapping relationship, and the initial virtual tourism global scene is updated.

[0087] This process achieves accurate identification of the user's area of interest by constructing an eye trajectory recognition model and obtaining the user's eye trajectory and the gaze time corresponding to each viewpoint position in the initial virtual tourism global scene in real time. Among them, the eye trajectory recognition model constructs an eye coordinate system and fits the iris curve to accurately calculate the line of sight deviation angle and the angular velocity of the eye viewpoint change, thereby distinguishing between scanning points and gaze points, improving the accuracy and reliability of eye trajectory recognition, and ensuring the effective capture of user points of interest.

[0088] This process also generates a user interest area level distribution map and a list of points of interest through evaluation algorithms and graph rendering technology. It can not only update the color depth of the areas corresponding to different gaze points in the current global scene in real time, but also dynamically adjust the virtual tourism planning path according to the list of points of interest, thereby improving the personalization and interactivity of the user experience. Specifically, by updating the color depth of the user's interest area in real time, the user can intuitively see his or her focus, and through matching algorithms with local scene information in a multi-classification virtual tourism scene library, the user can confirm whether to add a local scene with a simple blink, making the virtual tourism experience more natural and rich, and allowing users to see more different geographical locations and the same type of scenery in the same scene.

[0089] S4. Based on the user's interest area level distribution map and gaze duration, the global-local scene transformation is performed using the configured global-local scene change model. At the same time, a textual explanation of the transformed local scene is pushed through a voice or text algorithm.

[0090] Furthermore, the steps of constructing the global-local scene change model include:

[0091] S401, determining the viewing angle range of the global scene at the current moment according to the attributes of the virtual device, and constructing a global scene information space based on the global scene information within the viewing angle range;

[0092] S402: Obtain a local scene information subspace corresponding to the region in the global scene information space based on the user interest region level distribution map;

[0093] S403: Using the interest level evaluation score corresponding to each local area scene in the global scene as the upper limit of the magnification of the current local scene; when the upper limit of the magnification of the current local scene is greater than the maximum zoom range specified by the device properties, using the maximum zoom upper limit of the device as the upper limit of the magnification of the current local scene;

[0094] S404, obtaining a local scene magnification rate according to the magnification factor of the local scene in the area and the length of the gaze time;

[0095] S405: When the eye trajectory recognition model determines that the current viewpoint is the fixation point, the local scene corresponding to the fixation point is magnified within a determined viewing angle range using the obtained upper limit of the local scene magnification factor, the local scene magnification rate, and the local scene information subspace;

[0096] S406: When the gaze point deviates from the corresponding local scene range in the user interest area level distribution map, the corresponding magnified local scene is reduced according to the magnification rate until it returns to the zoom factor corresponding to the global scene;

[0097] S407: When the current viewpoint is a scanning viewpoint, the corresponding local scene is not scaled;

[0098] S408. When it is determined in S405 that the gaze point area is to be enlarged and changed, a virtual question-answering robot configured with a voice or text algorithm is called to provide information explanation and user questions and answers for the enlarged local area.

[0099] The process first determines the viewing angle range of the global scene at the current moment based on the properties of the virtual device and constructs a global scene information space. Secondly, based on the user's interest area level distribution map, the local scene information subspace of the corresponding area is obtained, and the upper limit of the local scene magnification factor is set according to the evaluation score. By calculating the local scene magnification rate, the local scene at the user's gaze point is magnified. When the gaze point leaves the local scene range, the local scene will automatically shrink to the global scene magnification factor. In addition, for the scanning point, no zoom change is performed and the global perspective is maintained. When the gaze point area is magnified, the system will call a virtual question-and-answer robot to explain the information and answer user questions. This process not only enhances the personalization and interactivity of the user experience, but also improves the user's immersion by dynamically adjusting scene details.

[0100] S5. According to the list of points of interest, the corresponding local scene is pushed through the matching algorithm, and the initial virtual tourism planning path and the initial virtual tourism global scene are updated according to the pushed local scene information.

[0101] Example 2

[0102] See also Figure 3 , another embodiment provided by the present invention: an immersive digital tourism experience configuration system based on eye tracking technology, comprising: an initialization module, a path generation module, a recognition module, a scene change module and a push update module;

[0103] Initialization module, used for initialization of equipment and scene and correction of viewpoint; the initialization module includes an initialization unit and a correction unit;

[0104] The initialization unit is used to initialize the virtual device and configure the multi-classification virtual tourism scene library; the correction unit is used to correct the initial viewpoint, background brightness and diopter through multi-point positioning to ensure a clear and comfortable user field of view;

[0105] A path generation module is used to select an initial virtual tourism global scene and generate an initial virtual tourism planning path based on the selected scene; the path generation module includes a scene selection unit and an initial path unit;

[0106] The scene selection unit is used to select an initial virtual tourism global scene of corresponding type and geographical location from a multi-category virtual tourism scene library through a configured scene matching list or voice input; the initial path unit is used to generate an initial virtual tourism planning path through a path generation algorithm based on the matching list information or voice information, and embed the path into the selected initial virtual tourism global scene;

[0107] Recognition module, used for eye trajectory recognition and generation of gaze time distribution map and interest point list; the recognition module includes trajectory recognition unit and cluster analysis unit;

[0108] The trajectory recognition unit is used to obtain the user's real-time eye movement trajectory and the corresponding gaze time of each viewpoint position on the trajectory in real time through the configured eye trajectory recognition model; the cluster analysis unit is used to obtain the user's interest area level distribution map and interest point list based on the gaze time corresponding to each viewpoint position on the trajectory through evaluation algorithms and graph rendering technology;

[0109] The scene change module is used to switch between global and local scenes and push the voice or text information corresponding to the local scene; the scene change module includes a scene change unit and an information push unit;

[0110] A scene transformation unit, configured to synchronously transform the scene using a global-local scene change model based on the user's interest area level distribution map and gaze duration;

[0111] The information push unit is used to push relevant information of the current local scene to the user through voice or text algorithms to enhance user interactivity and immersion;

[0112] The push update module is used to push the corresponding scenes according to the user's point of interest list through a matching push algorithm, and update the initial virtual tourism planning path and the initial virtual tourism global scene according to the information of the pushed scene, so that the user's travel route is in the scene of interest to the user in real time, improving the user's travel experience.

[0113] Example 3

[0114] A computer-readable storage medium stores computer instructions, which, when executed, execute an immersive digital tourism experience configuration method based on eye tracking technology.

[0115] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements an immersive digital tourism experience configuration method based on eye tracking technology when executing the computer program.

[0116] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are protected by the present invention.

[0117] If the technical solution disclosed herein involves personal information, the product using the technical solution disclosed herein has clearly informed the individual of the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution disclosed herein involves sensitive personal information, the product using the technical solution disclosed herein has obtained the individual's separate consent before processing the sensitive personal information and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the individual has entered the personal information collection scope and that personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information. The personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. An immersive digital tourism experience configuration method based on eye tracking technology, characterized in that: include: S1. Configure a multi-classification virtual tourism scene library and correct the initial viewpoint, scene brightness and diopter through multi-point positioning; S2. Selecting an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching questionnaire or voice, generating an initial virtual tourism planning path through a path generation algorithm based on the obtained matching list information or voice information, and embedding the path into the selected initial virtual tourism global scene; S3. Using the configured eye trajectory recognition model, the user's eye trajectory in the initial virtual tourism global scene and the gaze time corresponding to each viewpoint position on the trajectory are obtained in real time. Based on the gaze time corresponding to each viewpoint position on the trajectory, an evaluation algorithm and graph rendering technology are used to obtain a hierarchical distribution map of the user's areas of interest and a list of points of interest; S4. Based on the user's interest area level distribution map and gaze time, the global-local scene transformation is performed using the configured global-local scene change model. At the same time, a textual explanation of the transformed local scene is pushed through a voice or text algorithm. S5. According to the list of points of interest, the corresponding local scene is pushed through the matching algorithm, and the initial virtual tourism planning path and the initial virtual tourism global scene are updated according to the pushed local scene information; The steps of constructing the eye trajectory recognition model in S3 include: S301: Acquire an initially corrected eye image, construct a two-dimensional eye coordinate system centered on the pupil position, and fit an initial elliptical iris curve using the corresponding iris region in the eye image based on the two-dimensional eye coordinate system. Simultaneously, construct an initial line of sight based on a line connecting the initial gaze point position and the pupil position corresponding to the initial corrected eye image, and add the initial line of sight to the two-dimensional eye coordinate system to obtain a three-dimensional eye coordinate system. S302: Obtain eye images corresponding to viewpoints at different positions and the time of change between corresponding adjacent viewpoints, fit an elliptical iris curve to the preprocessed iris region corresponding to the eye image, and synchronize the corresponding line of sight to the three-dimensional eye coordinate system, thereby obtaining the deviation angle between the corresponding line of sight and the initial line of sight in the three-dimensional eye coordinate system and the ratio between the intersection area of the corresponding elliptical iris curve and the initial elliptical iris curve and the non-intersection area; S303: Based on the deviation angles between the sight lines corresponding to the adjacent time points and the initial sight line, a trigonometric function is used to obtain the spatial angle between the adjacent sight lines, and based on the spatial angle between the adjacent sight lines and the time between the adjacent viewpoint changes, the angular velocity of the eye viewpoint change and the corresponding trajectory curve are obtained; S304: Input the obtained deviation angle between the corresponding sight line and the initial sight line, the ratio of the intersection area and the non-intersection area between the corresponding elliptical iris curve and the initial elliptical iris curve, the viewpoint discrimination threshold, the angular velocity of the eye viewpoint change, and the corresponding trajectory curve into the constructed eye trajectory recognition model for training, and output the viewpoint position and trajectory between adjacent viewpoints predicted by the model; S305, obtaining a viewpoint error and a trajectory error based on the viewpoint position predicted by the eye trajectory recognition model, the trajectory between adjacent viewpoints, and the corresponding true values; S306, setting an angular velocity viewpoint output threshold. When the angular velocity of the eye viewpoint change of the corresponding viewpoint is greater than the angular velocity viewpoint output threshold, the current viewpoint is determined to be a saccade point, and the corresponding saccade point position coordinates are not output. Otherwise, the current viewpoint is determined to be a fixation point, and the corresponding fixation point position coordinates are output. S307, embedding the discrimination process set in S306 into the eye trajectory recognition model for training, obtaining the ratio of the scan point to the fixation point in the output viewpoint, and constructing the discrimination error using the obtained ratio of the scan point to the fixation point; S308: Set a comprehensive loss error threshold, and use the viewpoint error, trajectory error and discrimination error to construct a comprehensive loss function, and use the constructed comprehensive loss function to train the eye trajectory recognition model. When the comprehensive loss function value is less than the comprehensive loss error threshold, stop training to obtain a trained eye trajectory recognition model.

2. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 1, characterized in that: The step of obtaining the initial virtual tourism global scene includes: S201, according to the configured scenario matching questionnaire or voice recognition, obtain user demand scenario text information and pre-process the obtained text information; S202: Build a matching push model and integrate it into the configured multi-classification virtual tourism scene library, input the pre-processed text information into the matching push model, and obtain a scene matching score; S203: Setting a score threshold. When only one scene type corresponds to a scene with a scene matching score greater than the score threshold, using the scene with a score greater than the score threshold as the initial virtual tourism global scene, and generating an initial virtual tourism planning path using a path generation algorithm based on the text information and the initial virtual tourism global scene. S204: When there is more than one scene type with a scene matching score greater than the score threshold, the scene types greater than the score threshold are sorted according to the scene matching score, and the corresponding scenes are spliced according to the score sorting result using a splicing algorithm to obtain an initial comprehensive virtual tourism global scene; S205, generating an initial virtual tourism planning path through a path generation algorithm based on the initial comprehensive virtual tourism global scene, scene matching score and text information; S206. When the scene matching scores corresponding to all scenes are less than the score threshold, a global scene corresponding to the type and geographical location is generated based on the text information and the configured scene generation model, and the generated scene is added to the multi-classification virtual tourism scene library, and the S203 process is repeated to generate the corresponding initial virtual tourism planning path.

3. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 2, characterized in that: The step of obtaining the user's interest area level distribution map and the interest point list includes: S311, obtaining each gaze point position, the corresponding gaze time, and the local scene information of the area corresponding to the gaze point position based on the eye trajectory recognition model, and obtaining the interest level evaluation score of the local scene corresponding to each gaze point position area through an evaluation algorithm; S312. Based on the interest evaluation score of the local scene corresponding to each gaze point location area, a user interest area level distribution map is obtained through a region map rendering algorithm. A score-color depth mapping function is established based on the interest evaluation score of each area and the color depth corresponding to each area. The color depth of the area corresponding to different gaze points in the current global scene is updated in real time through the established score-color depth mapping function. S313: construct a list of points of interest based on the corresponding local scene information and interest level evaluation scores in each area, and establish a mapping relationship between the list of points of interest and the initial virtual tourism planning path through a linear function; S314. According to the list of points of interest, the local scene information in the multi-classification virtual tourism scene library is matched through a matching algorithm, and the matching result is fed back to the corresponding user. The user confirms whether to add the matching local scene by blinking. When the addition of the local scene is confirmed, the matched local scene is added to the area corresponding to the current global scene in the list of points of interest through a mapping relationship, and the initial virtual tourism global scene is updated.

4. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 3, characterized in that: The steps of constructing the global-local scene change model include: S401, determining the viewing angle range of the global scene at the current moment according to the attributes of the virtual device, and constructing a global scene information space based on the global scene information within the viewing angle range; S402: Obtain a local scene information subspace corresponding to the region in the global scene information space based on the user interest region level distribution map; S403: Using the interest level evaluation score corresponding to each local area scene in the global scene as the upper limit of the magnification of the current local scene; when the upper limit of the magnification of the current local scene is greater than the maximum zoom range specified by the device properties, using the maximum zoom upper limit of the device as the upper limit of the magnification of the current local scene; S404, obtaining a local scene magnification rate according to the magnification factor of the local scene in the area and the gaze time length; S405. When the current viewpoint is determined to be the fixation point through the eye trajectory recognition model, the local scene corresponding to the fixation point position is magnified within a determined viewing angle range by obtaining the upper limit of the local scene magnification factor, the local scene magnification rate, and the local scene information subspace.

5. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 4, characterized in that: The step of constructing the global-local scene change model also includes: S406: When the gaze point deviates from the corresponding local scene range in the user interest area level distribution map, the corresponding magnified local scene is reduced according to the magnification rate until it returns to the zoom factor corresponding to the global scene; S407: When the current viewpoint is a scanning viewpoint, the corresponding local scene is not scaled; S408. When it is determined in S405 that the gaze point area is to be enlarged and changed, a virtual question-answering robot configured with a voice or text algorithm is called to provide information explanation and user questions and answers for the enlarged local area.

6. An immersive digital tourism experience configuration system based on eye tracking technology, which is used to implement the immersive digital tourism experience configuration method based on eye tracking technology according to any one of claims 1 to 5, characterized in that: include: Initialization module, path generation module, recognition module and scene transformation module; The initialization module includes an initialization unit and a correction unit; the initialization unit is used to initialize the virtual device and configure the multi-classification virtual tourism scene library; the correction unit is used to correct the initial viewpoint, background brightness and diopter through multi-point positioning; The path generation module includes a scene selection unit and an initial path unit; the scene selection unit is used to select an initial virtual tourism global scene of corresponding type and geographical location from a multi-category virtual tourism scene library through a configured scene matching list or voice input; the initial path unit is used to generate an initial virtual tourism planning path through a path generation algorithm based on the matching list information or voice information, and embed the path into the selected initial virtual tourism global scene.

7. The immersive digital tourism experience configuration system based on eye tracking technology according to claim 6, characterized in that: The recognition module includes a trajectory recognition unit and a cluster analysis unit; The trajectory recognition unit is used to obtain the user's real-time eye movement trajectory and the gaze time corresponding to each viewpoint position on the trajectory in real time through the configured eye trajectory recognition model; the cluster analysis unit is used to obtain the user's interest area level distribution map and interest point list based on the gaze time corresponding to each viewpoint position on the trajectory through an evaluation algorithm and graph rendering technology; The scene change module includes a scene change unit; the scene change unit is used to perform synchronous scene change through a global-local scene change model according to the user's interest area level distribution map and gaze time.

8. A computer-readable storage medium, characterized in that Computer instructions are stored thereon, and when the computer instructions are executed, the immersive digital tourism experience configuration method based on eye tracking technology described in any one of claims 1 to 5 is executed.

Images