Immersive digital tourism experience configuration method and system based on eyeball tracking technology

Abstract

The invention belongs to the field of virtual tourism experience, and particularly relates to an immersive digital tourism experience configuration method and system based on an eyeball tracking technology, and the method comprises the steps: firstly initializing a virtual device, configuring a multi-classification virtual tourism scene library, and correcting viewpoints and visual parameters; secondly, according to user selection or a voice instruction, a personalized travel path is intelligently generated and embedded, and meanwhile, through real-time eyeball trajectory analysis, user interest points are recognized, and a global-local scene change model is combined, so that scene synchronous optimization and accurate information pushing are realized; and finally, dynamically pushing a related scene based on the interest point list, and adjusting a travel planning path in real time. According to the method, real-time identification of user interest points and dynamic adjustment of scenes are realized through an eyeball tracking technology, the immersion and interactivity of virtual travel are enhanced, and the quality of user experience is improved.

Description

Technical Field

[0001] The present invention belongs to the field of virtual tourism experience, and in particular relates to an immersive digital tourism experience configuration method and system based on eye tracking technology. Background Art

[0002] With the rapid development of VR and AR technologies, immersive digital tourism is gradually becoming a new trend in the tourism industry. However, most of the current immersive tourism experiences are still limited by traditional keyboard, mouse and touch screen operations, which are difficult to provide sufficient natural immersion and intuitive interaction. The introduction of eye tracking technology provides an innovative solution to this problem. However, although there are attempts at existing immersive tourism systems based on eye tracking, such as the patent with publication number CN113534835A, which discloses a tourism virtual remote experience system and method, a remote flight experience combining drones with line of sight tracking, or a tourist service platform integrating VR equipment, there are still many deficiencies in practical applications, among which the accuracy and real-time performance of eye tracking need to be improved to more accurately capture user intentions, avoid misoperation, and improve the authenticity and fluency of the experience.

[0003] The above existing technologies have the following problems: although the existing virtual tourism experience has made some progress in vision and hearing, it still has some shortcomings in the change from global to local landscape. For this reason, the present invention provides an immersive digital tourism experience configuration method and system based on eye tracking technology. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes an immersive digital tourism experience configuration method based on eye tracking technology. The method first initializes the virtual device, configures a multi-classification virtual tourism scene library, and calibrates the viewpoint and visual parameters; secondly, based on user selection or voice instructions, intelligently generates and embeds personalized tourism routes, and simultaneously identifies user points of interest through real-time eye trajectory analysis, and combines the global-local scene change model to achieve scene synchronization optimization and accurate information push; finally, dynamically push related scenes based on the point of interest list, and adjust the tourism planning path in real time; the method realizes real-time identification of user points of interest and dynamic adjustment of scenes through eye tracking technology, enhances the immersion and interactivity of virtual tourism, and improves the quality of user experience.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The immersive digital tourism experience configuration method based on eye tracking technology includes:

[0007] S1. Configure a multi-classification virtual tourism scene library and correct the initial viewpoint, scene brightness and diopter through multi-point positioning;

[0008] S2, selecting an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching questionnaire or voice, and generating an initial virtual tourism planning path through a path generation algorithm according to the obtained matching list information or voice information, and embedding the path into the selected initial virtual tourism global scene;

[0009] S3, obtaining the user's eye movement trajectory in the initial virtual tourism global scene and the gaze time corresponding to each viewpoint position on the trajectory in real time through the configured eye trajectory recognition model, and obtaining the user's interest area level distribution map and interest point list through the evaluation algorithm and graph rendering technology according to the gaze time corresponding to each viewpoint position on the trajectory;

[0010] S4, according to the user's interest area level distribution map and gaze time, the global-local scene change model is configured to perform a global-local transformation of the scene, and at the same time, a textual information push of the transformed local scene is performed through a voice or text algorithm;

[0011] S5. According to the list of points of interest, the corresponding local scene is pushed through a matching algorithm, and the initial virtual tourism planning path and the initial virtual tourism global scene are updated according to the pushed local scene information.

[0012] Specifically, the steps of obtaining the initial virtual tourism global scene include:

[0013] S201, according to the configured scenario matching questionnaire or speech recognition, obtain the user demand scenario text information and pre-process the obtained text information;

[0014] S202, constructing a matching push model and integrating it into the configured multi-classification virtual tourism scene library, inputting the pre-processed text information into the matching push model, and obtaining a scene matching score;

[0015] S203, setting a score threshold, when only one type of scene has a scene matching score greater than the score threshold, taking the scene corresponding to the scene greater than the score threshold as the initial virtual tourism global scene, and generating an initial virtual tourism planning path through a path generation algorithm based on the text information and the initial virtual tourism global scene;

[0016] S204, when there is more than one type of scene whose scene matching score is greater than the score threshold, sort the type of scenes whose scene matching score is greater than the score threshold according to the scene matching score, and according to the sorting result, splice the corresponding scenes according to the score sorting result through the splicing algorithm to obtain an initial comprehensive virtual tourism global scene;

[0017] S205, generating an initial virtual tourism planning path through a path generation algorithm according to the initial comprehensive virtual tourism global scene, scene matching score and text information;

[0018] S206. When the scene matching scores corresponding to all scenes are less than the score threshold, a global scene of the corresponding type and geographical location is generated according to the text information and the configured scene generation model, and the generated scene is added to the multi-classification virtual tourism scene library, and the S203 process is repeated to generate the corresponding initial virtual tourism planning path.

[0019] Specifically, the steps of constructing the eye trajectory recognition model in S3 include:

[0020] S301, obtaining an eye image after initial correction, constructing an eye two-dimensional coordinate system with the pupil position as the center, and fitting an initial elliptical iris curve using the iris corresponding area in the eye image according to the eye two-dimensional coordinate system, and constructing an initial sight line according to a line connecting the initial gaze point position and the pupil position corresponding to the initial corrected eye image, and adding the initial sight line to the eye two-dimensional coordinate system to obtain the eye three-dimensional coordinate system;

[0021] S302, obtaining eye images corresponding to viewpoints at different positions and the time of change between corresponding adjacent viewpoints, fitting an elliptical iris curve and the corresponding sight line to the eye three-dimensional coordinate system after preprocessing of the iris corresponding area in the eye image, and obtaining the deviation angle between the corresponding sight line and the initial sight line and the ratio between the intersection area and the non-intersection area of ​​the corresponding elliptical iris curve and the initial elliptical iris curve in the eye three-dimensional coordinate system;

[0022] S303, according to the deviation angle between the sight lines corresponding to the adjacent time points and the initial sight line, the spatial angle between the adjacent sight lines is obtained by trigonometric function, and according to the spatial angle between the adjacent sight lines and the time of change between the adjacent viewpoints, the angular velocity of the eyeball viewpoint change and the corresponding trajectory curve are obtained;

[0023] S304, input the obtained deviation angle between the corresponding line of sight and the initial line of sight, the ratio of the intersection area and the non-intersection area between the corresponding elliptical iris curve and the initial elliptical iris curve, the viewpoint discrimination threshold, the angular velocity of the eye viewpoint change and the corresponding trajectory curve into the constructed eye trajectory recognition model for training, and output the viewpoint position predicted by the model and the trajectory between adjacent viewpoints.

[0024] Specifically, the steps of constructing the eye trajectory recognition model in S3 also include:

[0025] S305, obtaining a viewpoint error and a trajectory error according to the viewpoint position predicted by the eyeball trajectory recognition model, the trajectory between adjacent viewpoints and the corresponding true value;

[0026] S306, setting an angular velocity viewpoint output threshold, when the eyeball viewpoint change angular velocity of the corresponding viewpoint is greater than the angular velocity viewpoint output threshold, the current viewpoint is determined to be a scanning point, and the corresponding scanning point position coordinates are not output; otherwise, the current viewpoint is determined to be a fixation point, and the corresponding fixation point position coordinates are output;

[0027] S307, embedding the discrimination process set in S306 into the eye trajectory recognition model for training, and obtaining the ratio of the scan point to the fixation point in the output viewpoint, and constructing the discrimination error using the obtained ratio of the scan point to the fixation point;

[0028] S308: Set a comprehensive loss error threshold, and use the viewpoint error, trajectory error and discrimination error to construct a comprehensive loss function, and use the constructed comprehensive loss function to train the eye trajectory recognition model. When the comprehensive loss function value is less than the comprehensive loss error threshold, stop training to obtain a trained eye trajectory recognition model.

[0029] Specifically, the steps of obtaining the user's interest area level distribution map and the interest point list include:

[0030] S311, obtaining each gaze point position, the corresponding gaze time and the local scene information of the area corresponding to the gaze point position according to the eye trajectory recognition model, and obtaining the interest evaluation score of the local scene corresponding to each gaze point position area through the evaluation algorithm;

[0031] S312, according to the interest level evaluation score of the local scene corresponding to each gaze point position area, a user interest area level distribution map is obtained through a regional map rendering algorithm, and a score-color depth mapping function is established according to the interest level evaluation score of each area and the color depth corresponding to each area, and the color depth of the area corresponding to different gaze points in the current global scene is updated in real time through the established score-color depth mapping function;

[0032] S313, constructing a list of points of interest based on the corresponding local scene information and interest level evaluation scores in each area, and mapping the list of points of interest to the initial virtual tourism planning path through a linear function;

[0033] S314. According to the list of points of interest, the matching algorithm is used to match the local scene information in the multi-classification virtual tourism scene library, and the matching result is fed back to the corresponding user. The user confirms whether to add the matching local scene by blinking. When confirming to add the local scene, the matched local scene is added to the area corresponding to the current global scene in the list of points of interest through the mapping relationship, and the initial virtual tourism global scene is updated.

[0034] Specifically, the steps of constructing the global-local scene change model include:

[0035] S401, determining the viewing angle range of the global scene at the current moment according to the attributes of the virtual device, and constructing a global scene information space according to the global scene information within the viewing angle range;

[0036] S402, obtaining a local scene information subspace of a corresponding area in the global scene information space according to a user interest area level distribution map;

[0037] S403, using the interest level evaluation score corresponding to each local area scene in the global scene as the upper limit of the magnification of the current local scene, when the upper limit of the magnification of the current local scene is greater than the maximum zoom range specified by the device properties, using the maximum zoom upper limit of the device as the upper limit of the magnification of the current local scene;

[0038] S404, obtaining a local scene magnification rate according to the local scene magnification multiple and the gaze time length of the region;

[0039] S405. When the current viewpoint is determined to be the fixation point through the eye trajectory recognition model, the local scene of the area corresponding to the fixation point position is magnified within a determined viewing angle range by obtaining the upper limit of the local scene magnification multiple, the local scene magnification rate and the local scene information subspace.

[0040] Specifically, the steps of constructing the global-local scene change model also include:

[0041] S406, when the gaze point deviates from the corresponding local scene range in the user interest area level distribution map, the corresponding enlarged local scene is reduced by the enlargement rate until it is restored to the zoom factor corresponding to the global scene;

[0042] S407: When the current viewpoint is a scanning viewpoint, the corresponding local scene is not scaled;

[0043] S408. When it is determined through S405 that the gaze point area is to be enlarged and changed, a virtual question-and-answer robot configured with a voice or text algorithm is called to explain information about the enlarged local area and answer questions from the user.

[0044] An immersive digital tourism experience configuration system based on eye tracking technology includes: an initialization module, a path generation module, a recognition module, and a scene change module;

[0045] The initialization module includes an initialization unit and a correction unit; the initialization unit is used for initializing the virtual device and configuring the multi-classification virtual tourism scene library; the correction unit is used for correcting the initial viewpoint, background brightness and diopter through multi-point positioning;

[0046] The path generation module includes a scene selection unit and an initial path unit; the scene selection unit is used to select an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching list or voice input; the initial path unit is used to generate an initial virtual tourism planning path through a path generation algorithm based on matching list information or voice information, and embed the path into the selected initial virtual tourism global scene.

[0047] Specifically, the recognition module includes a trajectory recognition unit and a cluster analysis unit;

[0048] The track recognition unit is used to obtain the user's real-time eye movement track and the gaze time corresponding to each viewpoint position on the track in real time through the configured eye track recognition model; the cluster analysis unit is used to obtain the user's interest area level distribution map and interest point list according to the gaze time corresponding to each viewpoint position on the track through the evaluation algorithm and graph rendering technology;

[0049] The scene change module includes a scene change unit; the scene change unit is used to perform synchronous scene change through a global-local scene change model according to the user's interest area level distribution map and gaze time.

[0050] A computer-readable storage medium stores computer instructions, which, when executed, execute an immersive digital tourism experience configuration method based on eye tracking technology.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] In view of the deficiencies of the prior art, the present invention significantly improves the deficiencies in the existing virtual tourism experience, especially the defects in the change from global to local landscape, by introducing eye tracking technology and dynamic global-local scene transformation model. By acquiring the user's eye movement trajectory and gaze time in real time and combining evaluation algorithms and graph rendering technology, the system can accurately identify the user's area of ​​interest, and dynamically adjust the scene details accordingly to achieve a seamless transition from global to local. In addition, by constructing a user's interest area level distribution map and a list of points of interest, the system can intelligently push local scenes of interest to the user, and dynamically update the virtual tourism planning path according to the user's feedback, making the experience more personalized and rich. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of an immersive digital tourism experience configuration method based on eye tracking technology in Example 1 of the present invention;

[0054] Figure 2 This is a flowchart of obtaining the initial virtual tourism planning path according to Embodiment 1 of the present invention;

[0055] Figure 3 This is a module diagram of the immersive digital tourism experience configuration system based on eye tracking technology in Example 2 of the present invention. DETAILED DESCRIPTION

[0056] Example 1

[0057] See also Figure 1 , an embodiment provided by the present invention: an immersive digital tourism experience configuration method based on eye tracking technology, the steps comprising:

[0058] S1, initializing the virtual device, configuring a multi-classification virtual tourism scene library, and correcting the initial viewpoint, scene brightness and diopter through multi-point positioning; further, the specific steps of configuring the multi-classification virtual tourism scene library in this embodiment include:

[0059] Import the pre-prepared multi-classified virtual tourism scene data into the system data, and create a quick index for each scene; the quick index includes: type, geographical location, theme, keyword, etc.; further, in this embodiment, the tourism scenes include natural scenery scenic spots, historical and cultural scenic spots, urban sightseeing scenic spots, nature reserves and religious and cultural scenic spots, etc.;

[0060] Furthermore, the diopter correction obtains the user's diopter data through a built-in diopter measurement tool, and adjusts the focus of the virtual display according to the user's diopter data, ensuring that the user does not experience visual blur or discomfort when viewing the virtual scene.

[0061] S2, selecting an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching questionnaire or voice, and generating an initial virtual tourism planning path through a path generation algorithm according to the obtained matching list information or voice information, and embedding the path into the selected initial virtual tourism global scene;

[0062] For further information, see Figure 2 The steps of obtaining the initial virtual tourism global scene in this embodiment include:

[0063] S201, according to the configured scenario matching questionnaire or speech recognition, obtain the user demand scenario text information and pre-process the obtained text information;

[0064] Furthermore, the scenario matching questionnaire configured in this embodiment sets different questionnaire options according to the quick index in the multi-classification virtual tourism scenario library to obtain the user's corresponding initial tourism intention.

[0065] S202, constructing a matching push model and integrating it into the configured multi-classification virtual tourism scene library, inputting the pre-processed text information into the matching push model, and obtaining a scene matching score;

[0066] S203, setting a score threshold, when only one type of scene has a scene matching score greater than the score threshold, taking the scene corresponding to the scene greater than the score threshold as the initial virtual tourism global scene, and generating an initial virtual tourism planning path through a path generation algorithm based on the text information and the initial virtual tourism global scene;

[0067] S204, when there is more than one type of scene whose scene matching score is greater than the score threshold, sort the type of scenes greater than the score threshold according to the scene matching score, and according to the sorting result, splice the corresponding scenes according to the score sorting result through the splicing algorithm to obtain an initial comprehensive virtual tourism global scene; further, in this embodiment, the results are sorted in descending order according to the score, and the scenes that meet the conditions are spliced, and the larger the score, the higher the corresponding scene is;

[0068] S205, generating an initial virtual tourism planning path through a path generation algorithm according to the initial comprehensive virtual tourism global scene, scene matching score and text information;

[0069] S206. When the scene matching scores corresponding to all scenes are less than the score threshold, a global scene of the corresponding type and geographical location is generated according to the text information and the configured scene generation model, and the generated scene is added to the multi-classification virtual tourism scene library, and the S203 process is repeated to generate the corresponding initial virtual tourism planning path.

[0070] This process can accurately obtain users' virtual tourism needs and generate personalized initial virtual tourism global scenes and planned paths through the configured scene matching questionnaire or voice recognition technology, combined with the matching push model and path generation algorithm; when there is more than one scene with a matching score greater than the threshold, a comprehensive virtual tourism global scene is generated through the scene matching score sorting and splicing algorithm, ensuring the continuity and diversity of the travel experience; when no scene matching score exceeds the threshold, the system can also generate new scenes according to user needs and add them to the scene library, further enriching the content of the scene library.

[0071] S3, obtaining the user's eye movement trajectory in the initial virtual tourism global scene and the gaze time corresponding to each viewpoint position on the trajectory in real time through the configured eye trajectory recognition model, and obtaining the user's interest area level distribution map and interest point list through the evaluation algorithm and graph rendering technology according to the gaze time corresponding to each viewpoint position on the trajectory;

[0072] Furthermore, the steps of constructing the eye trajectory recognition model in this embodiment include:

[0073] S301, obtaining an eye image after initial correction, constructing an eye two-dimensional coordinate system with the pupil position as the center, and fitting an initial elliptical iris curve using the iris corresponding area in the eye image according to the eye two-dimensional coordinate system, and constructing an initial sight line according to a line connecting the initial gaze point position and the pupil position corresponding to the initial corrected eye image, and adding the initial sight line to the eye two-dimensional coordinate system to obtain the eye three-dimensional coordinate system;

[0074] S302, obtaining eye images corresponding to viewpoints at different positions and the time of change between corresponding adjacent viewpoints, fitting an elliptical iris curve and the corresponding sight line to the eye three-dimensional coordinate system after preprocessing of the iris corresponding area in the eye image, and obtaining the deviation angle between the corresponding sight line and the initial sight line and the ratio between the intersection area and the non-intersection area of ​​the corresponding elliptical iris curve and the initial elliptical iris curve in the eye three-dimensional coordinate system;

[0075] S303, according to the deviation angle between the sight lines corresponding to the adjacent time points and the initial sight line, the spatial angle between the adjacent sight lines is obtained by trigonometric function, and according to the spatial angle between the adjacent sight lines and the time of change between the adjacent viewpoints, the angular velocity of the eyeball viewpoint change and the corresponding trajectory curve are obtained;

[0076] S304, inputting the obtained deviation angle between the corresponding sight line and the initial sight line, the ratio of the intersection area and the non-intersection area between the corresponding elliptical iris curve and the initial elliptical iris curve, the viewpoint discrimination threshold, the angular velocity of the eyeball viewpoint change, and the corresponding trajectory curve into the constructed eyeball trajectory recognition model for training, and outputting the viewpoint position and the trajectory between adjacent viewpoints predicted by the model; further, the eyeball trajectory recognition model in this embodiment is constructed by a pupil cornea tracking algorithm;

[0077] S305, obtaining a viewpoint error and a trajectory error according to the viewpoint position predicted by the eyeball trajectory recognition model, the trajectory between adjacent viewpoints and the corresponding true value;

[0078] S306, setting an angular velocity viewpoint output threshold, when the eyeball viewpoint change angular velocity of the corresponding viewpoint is greater than the angular velocity viewpoint output threshold, the current viewpoint is determined to be a scanning point, and the corresponding scanning point position coordinates are not output; otherwise, the current viewpoint is determined to be a fixation point, and the corresponding fixation point position coordinates are output;

[0079] S307, embedding the discrimination process set in S306 into the eye trajectory recognition model for training, and obtaining the ratio of the scan point to the fixation point in the output viewpoint, and constructing the discrimination error using the obtained ratio of the scan point to the fixation point;

[0080] S308: Set a comprehensive loss error threshold, and use the viewpoint error, trajectory error and discrimination error to construct a comprehensive loss function, and use the constructed comprehensive loss function to train the eye trajectory recognition model. When the comprehensive loss function value is less than the comprehensive loss error threshold, stop training to obtain a trained eye trajectory recognition model.

[0081] Furthermore, the step of obtaining the user interest area level distribution map and the interest point list in this embodiment includes:

[0082] S311, obtaining each gaze point position, the corresponding gaze time and the local scene information of the area corresponding to the gaze point position according to the eye trajectory recognition model, and obtaining the interest evaluation score of the local scene corresponding to each gaze point position area through the evaluation algorithm;

[0083] Furthermore, in this embodiment, the size of the local scene corresponding to each gaze point position area is a circular area with a radius of half the base of a triangle with the gaze point as the center and the maximum viewing angle as the vertex angle under the current viewing angle state;

[0084] S312, according to the interest level evaluation score of the local scene corresponding to each gaze point position area, a user interest area level distribution map is obtained through a regional map rendering algorithm, and a score-color depth mapping function is established according to the interest level evaluation score of each area and the color depth corresponding to each area, and the color depth of the area corresponding to different gaze points in the current global scene is updated in real time through the established score-color depth mapping function;

[0085] S313, constructing a list of points of interest based on the corresponding local scene information and interest level evaluation scores in each area, and mapping the list of points of interest to the initial virtual tourism planning path through a linear function;

[0086] S314. According to the list of points of interest, the matching algorithm is used to match the local scene information in the multi-classification virtual tourism scene library, and the matching result is fed back to the corresponding user. The user confirms whether to add the matching local scene by blinking. When confirming to add the local scene, the matched local scene is added to the area corresponding to the current global scene in the list of points of interest through the mapping relationship, and the initial virtual tourism global scene is updated.

[0087] This process achieves accurate identification of the user's interest area by constructing an eye trajectory recognition model and obtaining the user's eye trajectory and the gaze time corresponding to each viewpoint position in the initial virtual tourism global scene in real time. The eye trajectory recognition model constructs an eye coordinate system and fits the iris curve to accurately calculate the line of sight deviation angle and the angular velocity of the eye viewpoint change, thereby distinguishing between scanning points and gaze points, improving the accuracy and reliability of eye trajectory recognition, and ensuring the effective capture of user interest points.

[0088] This process also generates a user interest area level distribution map and a list of points of interest through evaluation algorithms and graph rendering technology. It can not only update the color depth of the areas corresponding to different gaze points in the current global scene in real time, but also dynamically adjust the virtual tourism planning path according to the list of points of interest, thereby improving the personalization and interactivity of the user experience. Specifically, by updating the color depth of the user's interest area in real time, the user can intuitively see his or her focus, and by matching the local scene information in the multi-classification virtual tourism scene library through the matching algorithm, the user can confirm whether to add the local scene with a simple blink, making the virtual tourism experience more natural and rich, allowing users to see more different geographical locations and the same type of scenery in the same scene.

[0089] S4, according to the user's interest area level distribution map and the length of gaze time, the scene is transformed globally to locally through the configured global-local scene change model, and the explanation text information of the transformed local scene is pushed through the voice or text algorithm;

[0090] Furthermore, the steps of constructing the global-local scene change model include:

[0091] S401, determining the viewing angle range of the global scene at the current moment according to the attributes of the virtual device, and constructing a global scene information space according to the global scene information within the viewing angle range;

[0092] S402, obtaining a local scene information subspace of a corresponding area in the global scene information space according to a user interest area level distribution map;

[0093] S403, using the interest level evaluation score corresponding to each local area scene in the global scene as the upper limit of the magnification of the current local scene, when the upper limit of the magnification of the current local scene is greater than the maximum zoom range specified by the device properties, using the maximum zoom upper limit of the device as the upper limit of the magnification of the current local scene;

[0094] S404, obtaining a local scene magnification rate according to the local scene magnification multiple and the gaze time length of the region;

[0095] S405, when the current viewpoint is determined to be the fixation point by the eyeball trajectory recognition model, the local scene corresponding to the fixation point position is magnified within a determined viewing angle range by obtaining the upper limit of the local scene magnification multiple, the local scene magnification rate, and the local scene information subspace;

[0096] S406, when the gaze point deviates from the corresponding local scene range in the user interest area level distribution map, the corresponding enlarged local scene is reduced by the enlargement rate until it is restored to the zoom factor corresponding to the global scene;

[0097] S407: When the current viewpoint is a scanning viewpoint, the corresponding local scene in the area is not scaled;

[0098] S408. When it is determined through S405 that the gaze point area is to be enlarged and changed, a virtual question-and-answer robot configured with a voice or text algorithm is called to explain information about the enlarged local area and answer questions from the user.

[0099] The process first determines the viewing angle range of the global scene at the current moment according to the properties of the virtual device, and constructs a global scene information space. Secondly, based on the user's interest area level distribution map, the local scene information subspace of the corresponding area is obtained, and the upper limit of the local scene magnification factor is set according to the evaluation score. By calculating the local scene magnification rate, the local scene at the user's gaze point is magnified. When the gaze point leaves the local scene range, the local scene will automatically shrink to the global scene zoom factor. In addition, for the scanning point, no zoom change is performed to maintain the global perspective. When the gaze point area is enlarged, the system will call a virtual question-and-answer robot to explain the information and answer user questions. This process not only enhances the personalization and interactivity of the user experience, but also improves the user's immersion by dynamically adjusting the scene details.

[0100] S5. According to the list of points of interest, the corresponding local scene is pushed through a matching algorithm, and the initial virtual tourism planning path and the initial virtual tourism global scene are updated according to the pushed local scene information.

[0101] Example 2

[0102] See also Figure 3 ,Another embodiment provided by the present invention: An immersive digital tourism experience configuration system based on eye tracking technology, comprising: an initialization module, a path generation module, a recognition module, a scene change module and a push update module;

[0103] An initialization module, used for initializing the device and the scene and correcting the viewpoint; the initialization module includes an initialization unit and a correction unit;

[0104] The initialization unit is used to initialize the virtual device and configure the multi-classification virtual tourism scene library; the correction unit is used to correct the initial viewpoint, background brightness and diopter through multi-point positioning to ensure that the user's vision is clear and comfortable;

[0105] A path generation module is used to select an initial virtual tourism global scene and generate an initial virtual tourism planning path according to the selected scene; the path generation module includes a scene selection unit and an initial path unit;

[0106] A scene selection unit is used to select an initial virtual tourism global scene of corresponding type and geographical location from a multi-classified virtual tourism scene library through a configured scene matching list or voice input; an initial path unit is used to generate an initial virtual tourism planning path through a path generation algorithm according to the matching list information or voice information, and embed the path into the selected initial virtual tourism global scene;

[0107] A recognition module, used for eye trajectory recognition and the generation of a gaze time distribution map and a list of points of interest; the recognition module includes a trajectory recognition unit and a cluster analysis unit;

[0108] The track recognition unit is used to obtain the user's real-time eye movement track and the gaze time corresponding to each viewpoint position on the track in real time through the configured eye track recognition model; the cluster analysis unit is used to obtain the user's interest area level distribution map and interest point list according to the gaze time corresponding to each viewpoint position on the track through the evaluation algorithm and graph rendering technology;

[0109] A scene change module is used for switching between global and local scenes and pushing voice or text information corresponding to the local scene; the scene change module includes a scene change unit and an information push unit;

[0110] A scene change unit, used to synchronously change the scene through a global-local scene change model according to the user's interest area level distribution map and the gaze time length;

[0111] The information push unit is used to push relevant information of the current local scene to the user through voice or text algorithms to enhance the user's interactivity and immersion;

[0112] The push update module is used to push the corresponding scenes according to the user's point of interest list through the matching push algorithm, and update the initial virtual tourism planning path and the initial virtual tourism global scene according to the information of the pushed scene, so that the user's travel route is in the scene of interest to the user in real time, improving the user's travel experience.

[0113] Example 3

[0114] A computer-readable storage medium stores computer instructions, which, when executed, execute an immersive digital tourism experience configuration method based on eye tracking technology.

[0115] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements an immersive digital tourism experience configuration method based on eye tracking technology when executing the computer program.

[0116] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in the field may also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are within the protection of the present invention.

[0117] If the disclosed technical solution involves personal information, the product using the disclosed technical solution has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the disclosed technical solution involves sensitive personal information, the product using the disclosed technical solution has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. An immersive digital tourism experience configuration method based on eye tracking technology, characterized in that: include: S1. Configure a multi-classification virtual tourism scene library and correct the initial viewpoint, scene brightness and diopter through multi-point positioning; S2, selecting an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching questionnaire or voice, and generating an initial virtual tourism planning path through a path generation algorithm according to the obtained matching list information or voice information, and embedding the path into the selected initial virtual tourism global scene; S3, obtaining the user's eye movement trajectory in the initial virtual tourism global scene and the gaze time corresponding to each viewpoint position on the trajectory in real time through the configured eye trajectory recognition model, and obtaining the user's interest area level distribution map and interest point list through the evaluation algorithm and graph rendering technology according to the gaze time corresponding to each viewpoint position on the trajectory; S4, according to the user's interest area level distribution map and gaze time, the global-local scene change model is configured to perform a global-local transformation of the scene, and at the same time, a textual information push of the transformed local scene is performed through a voice or text algorithm; S5. According to the list of points of interest, the corresponding local scene is pushed through a matching algorithm, and the initial virtual tourism planning path and the initial virtual tourism global scene are updated according to the pushed local scene information.

2. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 1, characterized in that: The step of obtaining the initial virtual tourism global scene includes: S201, according to the configured scenario matching questionnaire or speech recognition, obtain the user demand scenario text information and pre-process the obtained text information; S202, constructing a matching push model and integrating it into the configured multi-classification virtual tourism scene library, inputting the pre-processed text information into the matching push model, and obtaining a scene matching score; S203, setting a score threshold, when only one type of scene has a scene matching score greater than the score threshold, taking the scene corresponding to the scene greater than the score threshold as the initial virtual tourism global scene, and generating an initial virtual tourism planning path through a path generation algorithm based on the text information and the initial virtual tourism global scene; S204, when there is more than one type of scene whose scene matching score is greater than the score threshold, sort the type of scenes whose scene matching score is greater than the score threshold according to the scene matching score, and according to the sorting result, splice the corresponding scenes according to the score sorting result through the splicing algorithm to obtain an initial comprehensive virtual tourism global scene; S205, generating an initial virtual tourism planning path through a path generation algorithm according to the initial comprehensive virtual tourism global scene, scene matching score and text information; S206. When the scene matching scores corresponding to all scenes are less than the score threshold, a global scene of the corresponding type and geographical location is generated according to the text information and the configured scene generation model, and the generated scene is added to the multi-classification virtual tourism scene library, and the S203 process is repeated to generate the corresponding initial virtual tourism planning path.

3. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 2, characterized in that: The steps of constructing the eye trajectory recognition model in S3 include: S301, obtaining an eye image after initial correction, constructing an eye two-dimensional coordinate system with the pupil position as the center, and fitting an initial elliptical iris curve using the iris corresponding area in the eye image according to the eye two-dimensional coordinate system, and constructing an initial sight line according to a line connecting the initial gaze point position and the pupil position corresponding to the initial corrected eye image, and adding the initial sight line to the eye two-dimensional coordinate system to obtain the eye three-dimensional coordinate system; S302, obtaining eye images corresponding to viewpoints at different positions and the time of change between corresponding adjacent viewpoints, fitting an elliptical iris curve and the corresponding sight line to the eye three-dimensional coordinate system after preprocessing of the iris corresponding area in the eye image, and obtaining the deviation angle between the corresponding sight line and the initial sight line and the ratio between the intersection area and the non-intersection area of ​​the corresponding elliptical iris curve and the initial elliptical iris curve in the eye three-dimensional coordinate system; S303, according to the deviation angle between the sight lines corresponding to the adjacent time points and the initial sight line, the spatial angle between the adjacent sight lines is obtained by trigonometric function, and according to the spatial angle between the adjacent sight lines and the time of change between the adjacent viewpoints, the angular velocity of the eyeball viewpoint change and the corresponding trajectory curve are obtained; S304, input the obtained deviation angle between the corresponding line of sight and the initial line of sight, the ratio of the intersection area and the non-intersection area between the corresponding elliptical iris curve and the initial elliptical iris curve, the viewpoint discrimination threshold, the angular velocity of the eye viewpoint change and the corresponding trajectory curve into the constructed eye trajectory recognition model for training, and output the viewpoint position predicted by the model and the trajectory between adjacent viewpoints.

4. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 3, characterized in that: The step of constructing the eye trajectory recognition model in S3 further includes: S305, obtaining a viewpoint error and a trajectory error according to the viewpoint position predicted by the eyeball trajectory recognition model, the trajectory between adjacent viewpoints and the corresponding true value; S306, setting an angular velocity viewpoint output threshold, when the eyeball viewpoint change angular velocity of the corresponding viewpoint is greater than the angular velocity viewpoint output threshold, the current viewpoint is determined to be a scanning point, and the corresponding scanning point position coordinates are not output; otherwise, the current viewpoint is determined to be a fixation point, and the corresponding fixation point position coordinates are output; S307, embedding the discrimination process set in S306 into the eye trajectory recognition model for training, and obtaining the ratio of the scan point to the fixation point in the output viewpoint, and constructing the discrimination error using the obtained ratio of the scan point to the fixation point; S308: Set a comprehensive loss error threshold, and use the viewpoint error, trajectory error and discrimination error to construct a comprehensive loss function, and use the constructed comprehensive loss function to train the eye trajectory recognition model. When the comprehensive loss function value is less than the comprehensive loss error threshold, stop training to obtain a trained eye trajectory recognition model.

5. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 4, characterized in that: The step of obtaining the user's interest area level distribution map and the interest point list comprises: S311, obtaining each gaze point position, the corresponding gaze time and the local scene information of the area corresponding to the gaze point position according to the eye trajectory recognition model, and obtaining the interest evaluation score of the local scene corresponding to each gaze point position area through the evaluation algorithm; S312, according to the interest level evaluation score of the local scene corresponding to each gaze point position area, a user interest area level distribution map is obtained through a regional map rendering algorithm, and a score-color depth mapping function is established according to the interest level evaluation score of each area and the color depth corresponding to each area, and the color depth of the area corresponding to different gaze points in the current global scene is updated in real time through the established score-color depth mapping function; S313, constructing a list of points of interest based on the corresponding local scene information and interest level evaluation scores in each area, and mapping the list of points of interest to the initial virtual tourism planning path through a linear function; S314. According to the list of points of interest, the matching algorithm is used to match the local scene information in the multi-classification virtual tourism scene library, and the matching result is fed back to the corresponding user. The user confirms whether to add the matching local scene by blinking. When confirming to add the local scene, the matched local scene is added to the area corresponding to the current global scene in the list of points of interest through the mapping relationship, and the initial virtual tourism global scene is updated.

6. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 5, characterized in that: The steps of constructing the global-local scene change model include: S401, determining the viewing angle range of the global scene at the current moment according to the attributes of the virtual device, and constructing a global scene information space according to the global scene information within the viewing angle range; S402, obtaining a local scene information subspace of a corresponding area in the global scene information space according to a user interest area level distribution map; S403, using the interest level evaluation score corresponding to each local area scene in the global scene as the upper limit of the magnification of the current local scene, when the upper limit of the magnification of the current local scene is greater than the maximum zoom range specified by the device properties, using the maximum zoom upper limit of the device as the upper limit of the magnification of the current local scene; S404, obtaining a local scene magnification rate according to the local scene magnification multiple and the gaze time length of the region; S405. When the current viewpoint is determined to be the fixation point through the eye trajectory recognition model, the local scene of the area corresponding to the fixation point position is magnified within a determined viewing angle range by obtaining the upper limit of the local scene magnification multiple, the local scene magnification rate and the local scene information subspace.

7. The immersive digital tourism experience configuration method based on eye tracking technology according to claim 6, characterized in that: The step of constructing the global-local scene change model also includes: S406, when the gaze point deviates from the corresponding local scene range in the user interest area level distribution map, the corresponding enlarged local scene is reduced by the enlargement rate until it is restored to the zoom factor corresponding to the global scene; S407: When the current viewpoint is a scanning viewpoint, the corresponding local scene in the area is not scaled; S408. When it is determined through S405 that the gaze point area is to be enlarged and changed, a virtual question-and-answer robot configured with a voice or text algorithm is called to explain information about the enlarged local area and answer questions from the user.

8. An immersive digital tourism experience configuration system based on eye tracking technology, which is used to implement the immersive digital tourism experience configuration method based on eye tracking technology as described in any one of claims 1 to 7, characterized in that: include: Initialization module, path generation module, recognition module and scene change module; The initialization module includes an initialization unit and a correction unit; the initialization unit is used for initializing the virtual device and configuring the multi-classification virtual tourism scene library; the correction unit is used for correcting the initial viewpoint, background brightness and diopter through multi-point positioning; The path generation module includes a scene selection unit and an initial path unit; the scene selection unit is used to select an initial virtual tourism global scene of corresponding type and geographical location from a multi-classification virtual tourism scene library through a configured scene matching list or voice input; the initial path unit is used to generate an initial virtual tourism planning path through a path generation algorithm according to the matching list information or voice information, and embed the path into the selected initial virtual tourism global scene.

9. The immersive digital tourism experience configuration system based on eye tracking technology as claimed in claim 8, characterized in that: The recognition module includes a trajectory recognition unit and a cluster analysis unit; The trajectory recognition unit is used to obtain the user's real-time eye movement trajectory and the gaze time corresponding to each viewpoint position on the trajectory in real time through the configured eye trajectory recognition model; the cluster analysis unit is used to obtain the user's interest area level distribution map and interest point list through the evaluation algorithm and graph rendering technology according to the gaze time corresponding to each viewpoint position on the trajectory; The scene change module comprises a scene change unit; the scene change unit is used to perform synchronous scene change through a global-local scene change model according to the user's interest area level distribution map and gaze time.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the immersive digital tourism experience configuration method based on eye tracking technology described in any one of claims 1 to 7 is executed.

Images